USGS Tile Servers for SBX?

[crushnik]

I see where you can add tile servers to SBuilderX for ortho imagery, but I can't figure out how/where to do it. Google tiles are great, but I would prefer public domain imagery if at all possible so I can add golf courses to my scenery. It appears that the USGS server has the required specs to be utilized in SBX, but I don't know how/where to do it.

Any ideas?

 

[GaryGB]

Hi Chris:

SBuilderX tile server download DLLs have been largely maintained and developed by Richard Ludowise (aka "rhumbaflappy").

There are threads which discussed the work-flow to implement new SBuilderX tile server downloads from alternate GIS portals online.


Some have used other utilities with imagery tile server download features such as FSEarthTiles.

However, to minimize the risk of incurring yet another steep learning curve for such utilities, QGIS can be configured to do this (...and export GeoTIFFs).

qgis download satellite imagery - Google Search

PS:

* Imagery intended as background images for 3D modeling / airport design must be non-warped EPSG:3857 GIS projection format

* Imagery intended as background Maps for 'scenery building' via SBuilder Add Map > From Disk must be non-warped EPSG:3857 *.BMP GIS projection format

* Imagery intended as source images for SDK Resample-compiled custom photo-real land class must be warped EPSG:4326 GIS projection format


GaryGB

 

[crushnik]

I've searched so many threads both here and the internet that I worry I'm using up all of FSDeveloper's server bandwidth or DuckDuckGo will tell me "no more searches." I don't know why I keep coming up empty (well maybe half of the time I find answers myself, which is highly rewarding) but you always come through, Gary!

This was all a nightmare when I started, but things are slowly starting to make sense. Thanks again!

 

[GaryGB]

Indeed, learning for FS development seems to involve a constant need for searches, and involves a vast array of knowledge ...scattered everywhere online.

Spoiler: Top Secret ;-)

Note the query strings which I tend to use at Google (that is why I post the URL of such searches, so others can see what produces 'targeted' results).

BTW: QGIS 2.x added built-in tile-server download features; there are also 3rd party plugins which may further enhance such options.

GaryGB

 

[rhumbaflappy]

Tiles are like this:
https://basemap.nationalmap.gov/arcgis/rest/services/USGSImageryOnly/MapServer/tile/16/25587/15991

 

[crushnik]

The videos would be helpful if they shared how they found the URL to enter into QGIS. Otherwise it would be very easy to do this. And I've clicked every link on the USGS site. I'll get there one way or another!

 

[crushnik]

I'm getting somewhere using your link, Dick. I've got imagery populating, but I don't have my {x} {y} and {z} in the right order maybe. I figured 'z' would be first as "16" looks to be a zoom level. But any combination I try gives me what looks like rearranging puzzle pieces when I zoom in. But I am so much closer than I was!

 

[crushnik]

I guess disabling and re-enabling fixed it. "https://basemap.nationalmap.gov/arcgis/rest/services/USGSImageryOnly/MapServer/tile/{z}/{y}/{x}" is what I have loaded and max zoom set to 23 as it shows layers for 0-23 on the website here: https://basemap.nationalmap.gov/arcgis/rest/services/USGSImageryOnly/MapServer

 

[crushnik]

That's a handy feature! Now to scour back through and read how to correctly export my imagery to SBX/ADE to have some ortho of air bases, golf courses, quarries, and other interesting features! My evil plans are all coming together (well, except the darn streams, interstate on/off ramps, and I'm not even going to THINK about shorelines)... mwahahaha [maniacal laughter]

 

[rhumbaflappy]

Here's a dll for USGS Imagery, but it only displays to zoom of 15 (15 in SBX, 16 in ARCGIS).

 

[crushnik]

Dick, you're an absolute lifesaver! I could import tiles into QGIS but I was struggling when it came to exporting them into sim scenery. Your DLL is super easy to work with. Thanks a million.

 

[crushnik]

When I save the background map from the USGS tiles in SBuilderX, I get a lovely BMP file which I copy into PaintShopPro to convert to an 8-bit greyscale blend mask to compile with the BMP. Everything seems to work just fine, but the scenery looks washed out (not enough contrast and too bright might be a better way of explaining) compared to the original BMP.

Am I doing something wrong in the design phase or could it be a wonky setting inside P3D?

 

[rhumbaflappy]

It's most likely inside P3D. If you look at the default tiles of P3D, you'll get a clue as to the needed brightness, contrast, and saturation. It was the same with FSX.

 

[crushnik]

Looks like that could be the culprit.

This is 0.8 brightness, 0.6 bloom, and 0.9 saturation, (HDR enabled, plus I have PTA) and it's still overexposed. Anything lower on the brightness scale and the other textures, sky, and autogen become too dark. I'll keep digging.

 

[GaryGB]

Here's a way to more directly access USGS data files without using The National Map Viewer web portal:

Can The National Map data be downloaded via direct links? | U.S. Geological Survey

Direct access to The National Map data is provided via browsable links through:  Amazon's Cloud USGS Server - https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/ Navigate to the appropriate theme folders to access staged products for download.  Data can also be downloaded using The...

www.usgs.gov

Example URL:

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/13/TIFF/


GaryGB

Hi Chris:

It seems all USGS-related file servers behind web portals are having 'issues' at this time, as are some older NOAA servers.


You may wish to try exploring direct downloads via 'educated guesses' to derive URLs for USGS HTTP / FTP online file repositories; these may still work even if the web portal is down.

FYI: These are where the server web portals retrieve the files we download via their 'front end' web portal GUIs:

GIS Data Sources

Here's an app that will take a TXT file from SBuilderX, and build a geojson file for import use with the HOT export tool website or for use with any GIS program. Drag'n'drop the file ending in the extension .TXT or .txt and containing lines that contain the coordinates like so...

www.fsdeveloper.com

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/13/TIFF/


Some additional example URLs:

1 meter Digital Elevation Models (DEMs) - USGS National Map 3DEP Downloadable Data Collection

This is a tiled collection of the 3D Elevation Program (3DEP) and is one meter resolution.The 3DEP data holdings serve as the elevation layer of The National Map, and provide foundational elevation information for earth science studies and mapping applications in the United States. Scientists...

www.sciencebase.gov

Hope this might help further enhance your GIS source data access options.

GaryGB

I can now find everything but ortho images now, Gary - thanks! Hunting for a back door for that too, and I'll share if I can find one.

Hi Chris:

Which imagery are you needing to access for your project at this point in time ?

GaryGB

 

[crushnik]

Which imagery are you needing to access for your project at this point in time ?

I suppose I have been using the 3.75-minute imagery tiles - I forgot what they are called on the USGS site. While the pages you linked will definitely work for the elevation I need, all I can find for orthoimagery is for Guam, the Virgin Islands, Hawaii, etc.

Progress has been slow and it sure is tedious work, downloading the file then moving the little pin marker to the adjacent grids on USGS' National Map website to download another file and so on, but it is looking pretty good so far. Especially since I am using this USGS downtime as anopportunity to learn how to feed shapefiles into ScenProc to place autogen buildings right on top of the photoreal scenery. Finding an efficient method of mass-producing watermasks has been a serious challenge. That, and coming up with a method to blend the ocean shorelines for multiple tiles to line up seamlessly where I can still see sand and whatnot under the waterline out to a certain point - and how to get mangrove areas to show through the water in the intracoastal areas, but I am sure I'll get everything sorted before long.

 

[GaryGB]

Hi Chris:

You should be able to download these tiles directly using QGIS via WMS etc.:

FSA 10:1 NAIP Imagery m_3009247_ne_15_1_20150802_20151019 3.75 x 3.75 minute JPEG2000 from The National Map - Data.gov

catalog.data.gov

Perhaps Dick has had some success getting USGS tile access via WMS configured in QGIS ?


I'll take a look at whether this can be achieved in Global Mapper when I get a chance.

GaryGB

 

[crushnik]

Your link is a great bypass for the (still non-functioning) web portal. I do have USGS data for XYZ tiles, but the resolution is around half of the jpeg2000 file. Perhaps WMS/WMTS setup is the way to go... I'll see how to make that happen.

 

[GaryGB]

Hi Chris:

Have you seen these web pages ?

Advanced Viewer Help - Overview

USGS ArcGIS REST Services Directory Folder

https://basemap.nationalmap.gov/arcgis/rest/services

Folder: /

QGIS 3.2 using a USGS Topo for a basemap

On QGIS 3.2 I add, through the XYZ Tiles tab in the browser, this URL: http://services.arcgisonline.com/ArcGIS/rest/services/USA_Topo_Maps/MapServer/tile/${z}/${y}/${x}.jpg All that is displayed ...

gis.stackexchange.com

XYZ Tiles and WMS Layers in QGIS 3

I did a lot of hiking around Sedona, Arizona a few weeks ago, and wanted to map my GPS way points and tracks in QGIS over some WMS (web mapping service) base map layers. I recently switched to QGIS…

atcoordinates.info

PS: And yes, JPEG2000 compression (aka "MrSID" format) when set at 'lossy' really sucks.

AFAIK, USDA NAIP and USGS contracts probably do retain rights to all original full-resolution imagery by contractors who fly the DOQ grid sortees, but those are only available to us by special order for a fee on hard drives; the web portal distribution of pre-sized tile versions are 'lossy'.

IMHO, with internet distribution architecture having evolved to the point of having very fast throughput bandwidth, and with use of peer-to-peer ex: Torrent-type technology being widespread, there is no longer an excuse for USGS and other federal agencies et al to restrict access to full resolution original imagery via the internet.

While we 'should' want to credit the USGS for source data, and log into our USGS accounts at least every 6 months to keep our access 'active' for certain downloads that work around the web portal logins so they are not regarded as expired access permissions and therefore 'blocked', I do not see a rationale any longer for the apparent 'job justification' activities of USGS staffers by making us order bulk imagery at full resolution on hard drives only for a fee, as the taxpayers have already paid for the imagery acquisition and distribution budget.

Thus, they are already funded for all such work acquiring / improving data sources throughout the year.

Time for another taxpayer revolt; "Power to the people !"


Hmmm... was that "covet" ...or 'Covid' ?

GaryGB

 

[crushnik]

Lo and behold! The web portal is running again. Regarding the good stuff: I wonder if a Freedom of Information Act request would open up full-resolution imagery? That would certainly be an improvement over the lossy files, but man that would eat up a lot of hard drives!