Downloading OSM data

rhumbaflappy

Moderator
Staff member
Resource contributor
#1
Hi all.

Sometimes the downloading of OSM data gets to be quite complicated. We generally want data in the shapefile format. and we want a method of getting all the data we need at a time, without trying to download or query from several different sources.

Here's a way I get water data for an area slightly larger than an LOD5 region.

First, I need to use wget. Wget allows me to actually download the data:

Visual wget website

You also need to have GDAL installed to use ogr2ogr.

I then make a query xml file to specify what data I want. I use this for getting the non-ocean water data for LOD5 region 2416:

XML:
<osm-script output="xml" timeout="400">
    <union>
        <query type="node">
            <has-kv k="waterway" v="river"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="way">
            <has-kv k="waterway" v="river"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="relation">
            <has-kv k="waterway" v="river"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>
    <print mode="body"/>
    <union>
        <query type="node">
            <has-kv k="waterway" v="riverbank"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="way">
            <has-kv k="waterway" v="riverbank"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="relation">
            <has-kv k="waterway" v="riverbank"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>
    <print mode="body"/>
    <union>
        <query type="node">
            <has-kv k="natural" v="water"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="way">
            <has-kv k="natural" v="water"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="relation">
            <has-kv k="natural" v="water"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>
    <print mode="body"/>
    <union>
        <query type="node">
            <has-kv k="waterway" v="stream"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="way">
            <has-kv k="waterway" v="stream"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="relation">
            <has-kv k="waterway" v="stream"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>
    <print mode="body"/>
    <union>
        <query type="node">
            <has-kv k="waterway" v="canal"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="way">
            <has-kv k="waterway" v="canal"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
        <query type="relation">
            <has-kv k="waterway" v="canal"/>
            <bbox-query n="45.01" w="-90.01" s="42.1775" e="-86.24"/>
        </query>
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>   
    <print mode="body"/>
</osm-script>
I then use a batch file to do the actual query, downloading and convert the data to a shapefile:

Code:
"C:\Program Files (x86)\VisualWget\wget.exe" -O ex_2416.osm --post-file=2416_query.xml "http://overpass-api.de/api/interpreter"
ogr2ogr -skipfailures -oo USE_CUSTOM_INDEXING=NO ex_2416_water ex_2416.osm
del ex_2416.osm
pause
The end result is lines of streams, rivers, riverbanks, and polygons of water.

The data will still need some culling as it includes polys of non-water data such as islands in the water-polys. And the stream, river, and riverbank data may need to be differenced out of the water-polys. But at least you now have the data in a useable format.

Another consideration here is why you need the data at all. For FSX and P3D, endusers can have the ORBX vectors which for the cost might be a better solution. I'm not sure about the availability of good lines and polys for X-Plane, but a similar solution may already exist. For FS2004, FS2002, CFS2, and CFS3, and some of the emerging sims, this could be useful.
 

Attachments

rhumbaflappy

Moderator
Staff member
Resource contributor
#2
I'm trying a slightly different approach.

My query for cvx area 1207:

Code:
<osm-script output="xml" timeout="2500">
    <union>
        <query type="node">
            <has-kv k="natural"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>
        <query type="way">
            <has-kv k="natural"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>
        <query type="relation">
            <has-kv k="natural"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>
        <query type="node">
            <has-kv k="waterway"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>
        <query type="way">
            <has-kv k="waterway"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>
        <query type="relation">
            <has-kv k="waterway"/>
            <bbox-query e="-126.0711201" n="69.6070787" s="64.9611261" w="-135.5622099"/>
        </query>                   
    </union>
    <union>
        <item/>
        <recurse type="down"/>
    </union>
    <print mode="body"/>
</osm-script>
This just gets all the OSM data for the area that has either natural or waterway attributes. I then use ogr2ogr to extract what I need for water:

Code:
wget.exe -O ex_1207.osm --post-file=1207_query.txt "http://overpass-api.de/api/interpreter"

ogr2ogr -overwrite --config OSM_CONFIG_FILE my_osmconf.ini -skipfailures -oo USE_CUSTOM_INDEXING=NO 1207_waterpolys.shp ex_1207.osm -dialect sqlite -sql "SELECT ST_MakeValid(geometry) as geometry, * FROM multipolygons WHERE natural='water' OR waterway='riverbank' OR landuse='reservoir'"
ogr2ogr -overwrite --config OSM_CONFIG_FILE my_osmconf.ini -skipfailures -oo USE_CUSTOM_INDEXING=NO 1207_lines.shp ex_1207.osm -dialect sqlite -sql "SELECT ST_MakeValid(geometry) as geometry, * FROM lines WHERE waterway='river' OR waterway='stream' OR waterway='canal'"
ogr2ogr -overwrite -skipfailures -append -update -dialect sqlite -sql "SELECT ST_MakeValid(geometry) as geometry, * FROM ex_1207" 1207_waterpolys.shp D:\OSM_Data\ex_Water_Shapefiles\ex_1207.shp

del ex_1207.osm

pause
I download the OSM data using the 1207_query.txt above.
I extract polygons for natural=water, waterway=riverbank and landuse=reservoir.
I extract lines for river, stream and canal.
I merge my polygons with an ocean water polygon.
I delete the OSM file as I no longer need it.
 
Top