GIS Data Sources

[rhumbaflappy]

arcgisonline background maps for QGIS

Here's some xml files that can be drag-n-dropped onto QGIS to have background maps from arcgisonline.

Dick

 

[GaryGB]

Rhumbaflappy has posted some additional tools to access imagery resources via Google Maps and ESRI World Imagery:

http://www.fsdeveloper.com/forum/showpost.php?p=648232&postcount=22


The ESRI "World Imagery" tile server is particularly interesting, as it now offers access to quality aerial imagery for (some) parts of the world which had previously been available only by purchase from Digital Globe:

http://www.fsdeveloper.com/forum/showpost.php?p=648232&postcount=22



Additionally, there is a tool to access the LandSat imagery which, IIUC, has an updated URL to work around a previous slow-down by JPL to deal with "evil, repetitive requests for non-cached, small WMS tiles".

http://comments.gmane.org/gmane.comp.gis.openstreetmap/51607


Happy 'Tiling'; and again many thanks to Dick for contributing these to the FS Community !

GaryGB

 

[Patrick Germain]

Any Shapefiles for Canada, Saskatoon?

I have fount some but none include buildings data

Geogratis from Natural Resources Canada
http://geogratis.gc.ca/geogratis/AdvancedSearch?lang=en&s=canvec

You can download shape files.

 

[scotth6]

Great idea for a thread. I have recently been adding to a read me for sources of aerial imagery, and have just started using the National Map viewer and USGS Earth Explorer, both already mentioned. A great site for LIDAR and raster DEM data is www.opentopography.org. They have fairly specific and small data sets (although there are some large ones too), but the effect you can get with a high resolution mesh and photo scenery is amazing.

Cheers,

 

[GaryGB]

Global Self-consistent, Hierarchical, High-resolution Shoreline (GSHHS)

GSHHS database v1.1

• Metadata: http://www.ngdc.noaa.gov

• Spatial Resolution: multiple resolutions can be derived

Full resolution 0.04 km (40 Meter)

High resolution 0.2 km

Intermediate resolution 1.0 km

Low resolution 5.0 km

Crude resolution 25 km

• Extent: Global
• Temporal Resolution: N/A
• Date Range: updated March 2008
• Format(s): binary, ESRI Shapefile (v 1.3)
• Access Methods: FTP
• Data Access: http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.10/
• CoML visualization guidance: TBD

...another link refencing the same data set:

http://comlmaps.org/how-to/layers-and-resources/physical-environment/gshhs_shoreline

Be aware of the resolution differences of the above ...compared to this data set:

http://www.soest.hawaii.edu/pwessel/gshhg/


NOTE: The latter World Vector Shorelines (WVS) data set links to this page:

http://shoreline.noaa.gov/data/datasheets/wvs.html

GaryGB

 

[rhumbaflappy]

Here's a neat online website for finding a latitude and longitude quickly:

http://www.findlatitudeandlongitude.com/

Dick

 

[rhumbaflappy]

There is a Global Inland Water series at 30 meters. Its a raster imagery set set (no elevation). It's from NASA. It may have value for someone...

http://www.landcover.org/data/watercover/

An online tool to find the correct WRS2 path and row for the latitude and longitude:

http://landsat.usgs.gov/tools_latlong.php

Dick

 

[rhumbaflappy]

The above GSHHS is now GSHHG:

https://www.ngdc.noaa.gov/mgg/shorelines/gshhs.html

and...

http://www.soest.hawaii.edu/pwessel/gshhg/

Dick

 

[arno]

For water polygons (sea) I have used the water data from OSM as well. There is a good source to download this:

http://openstreetmapdata.com/data/water-polygons

 

[rhumbaflappy]

And another good source of vector waterbody data is the SRTM Water Body Data:

https://dds.cr.usgs.gov/srtm/version2_1/SWBD/

This is the set used to flatten the ocean and lakes for the SRTM elevational data. Contains elevations. It's based on the landsat (5?) dataset.

And there is the raster-based MODIS land-water mask:

http://glcf.umd.edu/data/watermask/

All of the above water sets do have errors, so some manual editing is usually required. Raster sets would require raster-to-vector conversion.

Dick

 

[rhumbaflappy]

Coming in 2016:

Free 30m DEM from JAXA:

http://www.eorc.jaxa.jp/ALOS/en/aw3d/index_e.htm

This should be much more accurate than SRTM or ASTER, as I understand. The free 30m data is down-sampled from 5m data.

EDIT--

Some missing data, and doesn't seem to improve over SRTM 1 ARC Second. Perhaps wen the full data set is ready it will be OK...

Dick

 

[A34Designs]

Hey all, I recently found this website, which lists "Free Orthoimagery sources". I don't know if this has been brought up before, so I thought I would share:

http://www.worldwindcentral.com/wiki/Sources_of_free_orthoimagery

 

[bina]

This is a very useful thread indeed... Thanks for starting it, Richard!!!

I can see lots of different software options to work with source data being shape files (SHP) in my case (eg. ArcGIS, QuantumGIS, and GlobalMapper) but can't really decide on a particular one. I though maybe the vet experts can advise me the one which is easier to use with a relatively smooth learning curve?

Thanks!

 

[arno]

Hi,

Personally I prefer QGis. It's free and has a lot of features. It's also not so hard to use I think.

Combined with GDAL/OGR I have never needed any of the commercial tools.

 

[JacobR]

Hey guys has anyone got any good sources for Australia? I am having trouble finding some up to date images that I can import into Gmax

Cheers

 

[rhumbaflappy]

Global Surface Water is a project by the European Commission Joint Research Centre. It has gleaned data from 32 years of Landsat imagery to derive a comprehensive set of 30 meter waterbody rasters covering the entire globe. The data is downloadable in 10 x 10 degree images. They are 8-bit indexed geotiffs.

This data is especially valuable at latitudes above 50 degrees north, which are not covered by the SRTM Waterbody data.

The Data Users Guide explains the different sets available.

I've been using batch files to process the imagery to usable shapefiles.

C:\OSGeo4W64\bin\gdaldem color-relief "D:\Global Surface Water\occurrence\occurrence_150E_0N.tif" "D:\Global Surface Water\Potrace Batches\occurrence.txt" "D:\Global Surface Water\Potrace Batches\temp.tif"

C:\OSGeo4W64\bin\gdal_translate -of PNM -ot Byte "D:\Global Surface Water\Potrace Batches\temp.tif" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm"

del "D:\Global Surface Water\Potrace Batches\temp.tif"

potrace -b geojson -o "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm" -L 150.0 -B -10.0 -W 10.0 -H 10.0

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm"

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.pgm.aux.xml"

C:\OSGeo4W64\bin\ogr2ogr -f "ESRI Shapefile" -progress "D:\Global Surface Water\occurrence\Shapefiles\occurrence_150E_0N.shp" "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json"

del "D:\Global Surface Water\Potrace Batches\occurrence_150E_0N.json"

pause

The example uses gdaldem to make a black and white image using an index file named occurrence.txt Then I use gdal_translate to make a PGM file. Then Potrace is used to create a geojson vector file. That file is then made into a shapefile with ogr2ogr.

My occurrence.txt file makes 90% of the colors black and whites-out the lower 10% of occurrence... hopefully taking out pixels that represent extreme flooding or drought.

Potrace is much quicker than the QGIS routines for raster to vector. The options of Potrace might need some work to get the images expected, although I find they aren't too bad as is. And they have few errors, unlike QGIS' routines.

 

[rhumbaflappy]

The Google API v3 has some interesting features. I derived a feature that uses Google imaging to make a water background as a tile server for SBuilderX_315. The same source can be used as a XYZ Tiles source in QGIS. Here's the URL:

https://maps.googleapis.com/maps/vt?pb=!1m5!1m4!1i{z}!2i{x}!3i{y}!4i256!2m3!1e0!2sm!3i431136040!3m14!2sen-US!3sUS!5e18!12m1!1e68!12m3!1e37!2m1!1ssmartmaps!12m4!1e26!2m2!1sstyles!2zcy50OjF8cC52Om9mZnxwLmw6MHxwLnM6MCxzLnQ6NXxwLnY6b2ZmLHMudDo4MnxzLmU6Z3xwLnY6b24scy50OjgyfHMuZTpnLmZ8cC52Om9ufHAuYzojZmZmZmZmZmYscy50OjgyfHMuZTpsfHAudjpvZmYscy50OjJ8cC52Om9mZixzLnQ6M3xwLnY6b2ZmLHMudDo0fHAudjpvZmYscy50OjZ8cy5lOmd8cC52Om9ufHAuYzojZmYwMDAwMDAscy50OjZ8cy5lOmx8cC52Om9mZixzLnQ6NnxzLmU6bC5pfHAudjpvZmY

(all on one line of course)

The zoom goes from 0 to 23 I believe. A layer export can yield a nice geotiff, but the result will have anti-aliasing, so some conversion to black and white would be needed if you're going to convert the raster to vector. Some views of those pesky Canadian lakes:

 

[rhumbaflappy]

Having a bit more fun with the Google API3 'BlackWater" referenced above.

Gdal can produce a Geotiff without QGIS, allowing batch processing of reprojected images. You need an XML file to describe the dataset, and then a GDAL command.

Here's the BlackWater.xml use:

<GDAL_WMS>
    <!-- BlackWater -->
    <Service name="TMS">
        <ServerUrl>https://maps.googleapis.com/maps/vt?pb=!1m5!1m4!1i${z}!2i${x}!3i${y}!4i256!2m3!1e0!2sm!3i431136040!3m14!2sen-US!3sUS!5e18!12m1!1e68!12m3!1e37!2m1!1ssmartmaps!12m4!1e26!2m2!1sstyles!2zcy50OjF8cC52Om9mZnxwLmw6MHxwLnM6MCxzLnQ6NXxwLnY6b2ZmLHMudDo4MnxzLmU6Z3xwLnY6b24scy50OjgyfHMuZTpnLmZ8cC52Om9ufHAuYzojZmZmZmZmZmYscy50OjgyfHMuZTpsfHAudjpvZmYscy50OjJ8cC52Om9mZixzLnQ6M3xwLnY6b2ZmLHMudDo0fHAudjpvZmYscy50OjZ8cy5lOmd8cC52Om9ufHAuYzojZmYwMDAwMDAscy50OjZ8cy5lOmx8cC52Om9mZixzLnQ6NnxzLmU6bC5pfHAudjpvZmY</ServerUrl>
        <SRS>EPSG:3857</SRS>
        <ImageFormat>image/png</ImageFormat>
        <Transparent>False</Transparent>
    </Service>
    <DataWindow>
        <UpperLeftX>-20037508.34</UpperLeftX>
        <UpperLeftY>20037508.34</UpperLeftY>
        <LowerRightX>20037508.34</LowerRightX>
        <LowerRightY>-20037508.34</LowerRightY>
        <TileLevel>22</TileLevel>
        <TileCountX>1</TileCountX>
        <TileCountY>1</TileCountY>
        <YOrigin>top</YOrigin>
    </DataWindow>
    <BlockSizeX>256</BlockSizeX>
    <BlockSizeY>256</BlockSizeY>
    <BandsCount>3</BandsCount>
    <UnsafeSSL>true</UnsafeSSL>
    <ZeroBlockHttpCodes>204,303,400,404,500,501</ZeroBlockHttpCodes>
    <ZeroBlockOnServerException>true</ZeroBlockOnServerException>
    <MaxConnections>4</MaxConnections>
</GDAL_WMS>

Here's the Command:

gdalwarp -s_srs EPSG:3857 -t_srs EPSG:4326 -te -89.0625 42.5390625 -88.59375 42.890625 -tr 0.00005 0.00005 -ot Byte -co "COMPRESS=LZW" -co "tfw=yes"  "C:/Users/dickl/Desktop/Blackwater.xml" "C:/Users/dickl/Desktop/LOD8_194_134.tif"

The result is a GeoTiff and a TFW file.

I used to catch snapping turtles here 60 years ago.

 

[rhumbaflappy]

Hi all.

Golden Software has an online format converter:

https://convert.goldensoftware.com/Application/Conversion

It's free (but requires registration). You can easily convert from BLN to ESRI shapefiles and back.

 

[rhumbaflappy]

Hi all.

Some of us have discovered the HOT export tool website for grabbing OSM data. It's really a nice tool to use. But one irritating thing is entering the coordinates of an area manually. For those who have the Google Earth program, I have a variation of my LEC Google tool. It has a button to export a simple geojson file that can be imported to the HOT website, and avoid the tedium of entering the coordinates by hand. The tool can save data by LOD area (QMID - 2). Use the Gather button to fetch the Google Earth coords. It defaults to LOD 9 (QMID 11).