GIS Data Sources

#22
Rhumbaflappy has posted some additional tools to access imagery resources via Google Maps and ESRI World Imagery:

http://www.fsdeveloper.com/forum/showpost.php?p=648232&postcount=22


The ESRI "World Imagery" tile server is particularly interesting, as it now offers access to quality aerial imagery for (some) parts of the world which had previously been available only by purchase from Digital Globe:

http://www.fsdeveloper.com/forum/showpost.php?p=648232&postcount=22



Additionally, there is a tool to access the LandSat imagery which, IIUC, has an updated URL to work around a previous slow-down by JPL to deal with "evil, repetitive requests for non-cached, small WMS tiles". :p

http://comments.gmane.org/gmane.comp.gis.openstreetmap/51607


Happy 'Tiling'; and again many thanks to Dick for contributing these to the FS Community ! :)

GaryGB
 
Last edited:
#24
Great idea for a thread. I have recently been adding to a read me for sources of aerial imagery, and have just started using the National Map viewer and USGS Earth Explorer, both already mentioned. A great site for LIDAR and raster DEM data is www.opentopography.org. They have fairly specific and small data sets (although there are some large ones too), but the effect you can get with a high resolution mesh and photo scenery is amazing.

Cheers,
 
#25
Global Self-consistent, Hierarchical, High-resolution Shoreline (GSHHS)

GSHHS database v1.1

  • Spatial Resolution: multiple resolutions can be derived
Full resolution 0.04 km (40 Meter)

High resolution 0.2 km

Intermediate resolution 1.0 km

Low resolution 5.0 km

Crude resolution 25 km


...another link refencing the same data set:

http://comlmaps.org/how-to/layers-and-resources/physical-environment/gshhs_shoreline

Be aware of the resolution differences of the above ...compared to this data set: ;)

http://www.soest.hawaii.edu/pwessel/gshhg/


NOTE: The latter World Vector Shorelines (WVS) data set links to this page:

http://shoreline.noaa.gov/data/datasheets/wvs.html

GaryGB
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#30
And another good source of vector waterbody data is the SRTM Water Body Data:

https://dds.cr.usgs.gov/srtm/version2_1/SWBD/

This is the set used to flatten the ocean and lakes for the SRTM elevational data. Contains elevations. It's based on the landsat (5?) dataset.

And there is the raster-based MODIS land-water mask:

http://glcf.umd.edu/data/watermask/

All of the above water sets do have errors, so some manual editing is usually required. Raster sets would require raster-to-vector conversion.

Dick
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#31
Coming in 2016:

Free 30m DEM from JAXA:

http://www.eorc.jaxa.jp/ALOS/en/aw3d/index_e.htm

This should be much more accurate than SRTM or ASTER, as I understand. The free 30m data is down-sampled from 5m data.

EDIT--

Some missing data, and doesn't seem to improve over SRTM 1 ARC Second. Perhaps wen the full data set is ready it will be OK...

Dick
 
Last edited:
#33
This is a very useful thread indeed... Thanks for starting it, Richard!!!

I can see lots of different software options to work with source data being shape files (SHP) in my case (eg. ArcGIS, QuantumGIS, and GlobalMapper) but can't really decide on a particular one. I though maybe the vet experts can advise me the one which is easier to use with a relatively smooth learning curve?

Thanks!
 

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#34
Hi,

Personally I prefer QGis. It's free and has a lot of features. It's also not so hard to use I think.

Combined with GDAL/OGR I have never needed any of the commercial tools.
 
#35
Hey guys has anyone got any good sources for Australia? I am having trouble finding some up to date images that I can import into Gmax

Cheers
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#36
Global Surface Water is a project by the European Commission Joint Research Centre. It has gleaned data from 32 years of Landsat imagery to derive a comprehensive set of 30 meter waterbody rasters covering the entire globe. The data is downloadable in 10 x 10 degree images. They are 8-bit indexed geotiffs.

This data is especially valuable at latitudes above 50 degrees north, which are not covered by the SRTM Waterbody data.

The Data Users Guide explains the different sets available.

I've been using batch files to process the imagery to usable shapefiles.

example.png


Code:
C:\OSGeo4W64\bin\gdaldem color-relief "D:\Global Surface Water\occurrence\occurrence_150E_0N.tif" "D:\Global Surface Water\Potrace Batches\occurrence.txt" "D:\Global Surface Water\Potrace Batches\temp.tif"

C:\OSGeo4W64\bin\gdal_translate -of PNM -ot Byte "D:\Global Surface Water\Potrace Batches\temp.tif" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm"

del "D:\Global Surface Water\Potrace Batches\temp.tif"

potrace -b geojson -o "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm" -L 150.0 -B -10.0 -W 10.0 -H 10.0

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm"

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm.aux.xml"

C:\OSGeo4W64\bin\ogr2ogr -f "ESRI Shapefile" -progress "D:\Global Surface Water\occurrence\Shapefiles\occurrence_150E_0N.shp" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json"

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json"

pause
The example uses gdaldem to make a black and white image using an index file named occurrence.txt Then I use gdal_translate to make a PGM file. Then Potrace is used to create a geojson vector file. That file is then made into a shapefile with ogr2ogr.

My occurrence.txt file makes 90% of the colors black and whites-out the lower 10% of occurrence... hopefully taking out pixels that represent extreme flooding or drought.

Potrace is much quicker than the QGIS routines for raster to vector. The options of Potrace might need some work to get the images expected, although I find they aren't too bad as is. And they have few errors, unlike QGIS' routines.
 

Attachments

Last edited:

rhumbaflappy

Moderator
Staff member
Resource contributor
#37
The Google API v3 has some interesting features. I derived a feature that uses Google imaging to make a water background as a tile server for SBuilderX_315. The same source can be used as a XYZ Tiles source in QGIS. Here's the URL:

Code:
https://maps.googleapis.com/maps/vt?pb=!1m5!1m4!1i{z}!2i{x}!3i{y}!4i256!2m3!1e0!2sm!3i431136040!3m14!2sen-US!3sUS!5e18!12m1!1e68!12m3!1e37!2m1!1ssmartmaps!12m4!1e26!2m2!1sstyles!2zcy50OjF8cC52Om9mZnxwLmw6MHxwLnM6MCxzLnQ6NXxwLnY6b2ZmLHMudDo4MnxzLmU6Z3xwLnY6b24scy50OjgyfHMuZTpnLmZ8cC52Om9ufHAuYzojZmZmZmZmZmYscy50OjgyfHMuZTpsfHAudjpvZmYscy50OjJ8cC52Om9mZixzLnQ6M3xwLnY6b2ZmLHMudDo0fHAudjpvZmYscy50OjZ8cy5lOmd8cC52Om9ufHAuYzojZmYwMDAwMDAscy50OjZ8cy5lOmx8cC52Om9mZixzLnQ6NnxzLmU6bC5pfHAudjpvZmY
(all on one line of course)

The zoom goes from 0 to 23 I believe. A layer export can yield a nice geotiff, but the result will have anti-aliasing, so some conversion to black and white would be needed if you're going to convert the raster to vector. Some views of those pesky Canadian lakes:

Google Satellite.png


BlackWater.png
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#38
Having a bit more fun with the Google API3 'BlackWater" referenced above.

Gdal can produce a Geotiff without QGIS, allowing batch processing of reprojected images. You need an XML file to describe the dataset, and then a GDAL command.

Here's the BlackWater.xml use:

XML:
<GDAL_WMS>
    <!-- BlackWater -->
    <Service name="TMS">
        <ServerUrl>https://maps.googleapis.com/maps/vt?pb=!1m5!1m4!1i${z}!2i${x}!3i${y}!4i256!2m3!1e0!2sm!3i431136040!3m14!2sen-US!3sUS!5e18!12m1!1e68!12m3!1e37!2m1!1ssmartmaps!12m4!1e26!2m2!1sstyles!2zcy50OjF8cC52Om9mZnxwLmw6MHxwLnM6MCxzLnQ6NXxwLnY6b2ZmLHMudDo4MnxzLmU6Z3xwLnY6b24scy50OjgyfHMuZTpnLmZ8cC52Om9ufHAuYzojZmZmZmZmZmYscy50OjgyfHMuZTpsfHAudjpvZmYscy50OjJ8cC52Om9mZixzLnQ6M3xwLnY6b2ZmLHMudDo0fHAudjpvZmYscy50OjZ8cy5lOmd8cC52Om9ufHAuYzojZmYwMDAwMDAscy50OjZ8cy5lOmx8cC52Om9mZixzLnQ6NnxzLmU6bC5pfHAudjpvZmY</ServerUrl>
        <SRS>EPSG:3857</SRS>
        <ImageFormat>image/png</ImageFormat>
        <Transparent>False</Transparent>
    </Service>
    <DataWindow>
        <UpperLeftX>-20037508.34</UpperLeftX>
        <UpperLeftY>20037508.34</UpperLeftY>
        <LowerRightX>20037508.34</LowerRightX>
        <LowerRightY>-20037508.34</LowerRightY>
        <TileLevel>22</TileLevel>
        <TileCountX>1</TileCountX>
        <TileCountY>1</TileCountY>
        <YOrigin>top</YOrigin>
    </DataWindow>
    <BlockSizeX>256</BlockSizeX>
    <BlockSizeY>256</BlockSizeY>
    <BandsCount>3</BandsCount>
    <UnsafeSSL>true</UnsafeSSL>
    <ZeroBlockHttpCodes>204,303,400,404,500,501</ZeroBlockHttpCodes>
    <ZeroBlockOnServerException>true</ZeroBlockOnServerException>
    <MaxConnections>4</MaxConnections>
</GDAL_WMS>
Here's the Command:
Code:
gdalwarp -s_srs EPSG:3857 -t_srs EPSG:4326 -te -89.0625 42.5390625 -88.59375 42.890625 -tr 0.00005 0.00005 -ot Byte -co "COMPRESS=LZW" -co "tfw=yes"  "C:/Users/dickl/Desktop/Blackwater.xml" "C:/Users/dickl/Desktop/LOD8_194_134.tif"
The result is a GeoTiff and a TFW file.

Turtles.png

I used to catch snapping turtles here 60 years ago. :)
 

Attachments

Top