Making water vectors with multiple elevation files

[crushnik]

I have used SBuilder to make water vectors, but I would like to see if I get better results with ScenProc. I use USGS data for both water shapefiles and elevation data, so how would I write the script to draw from multiple elevation TIF files? The shapefile covers the entire state, so I would need about a dozen TIF files of elevation data.

 

[GaryGB]

Hi Chris:

There is more than 1 way to achieve your goal with this task.

AFAIK, as Arno has thus far not been specific in documenting this particular process, beyond these statements:

The main challenge that I see with making water polygons for the sim, is that they require an elevation value as well. So somehow you need to know the elevation to set for the different lakes and rivers.

In scenProc I have also added feature that it can sample raster data to get the elevation value for the water polygons. But that has not been tested a lot yet.

I hope Arno may have more details he will share with us here on how we can actually do this in ScenProc directly.


Otherwise, I suggest assigning the "Z" Altitude attribute to "X,Y" ESRI *.SHP vertices via QGIS as cited in the thread linked above (...from which Arno's reply was also quoted):

https://www.fsdeveloper.com/forum/threads/massive-water-class-creation.447758/post-845484


Or do you already have the 3D "X,Y,Z" ESRI *.SHP water body polygon data prepared, and are primarily looking for more details on the ScenProc scripting method itself ?

[EDITED]

If so, you can Append those *.SHP files to SBuilderX and assign the desired Water Class instead of 'Flatten' GUID to these vector polygons, and output a CVX vector BGL.

See: Luis Feliz Tirado's tutorial section on "Sloped Polygons" in:


Terrain Design

File Description:
Terrain Design for Flight Simulator X

This simple illustrated document explains the basic concepts in Flight Simulator X terrain, and how to change them using SBuilder for FS X. Make new water bodies, roads, vehicle traffic, airport flattens, remove autogen, add more detail - anybody can do it easily and quickly.
My most sincere apologies - I am unable, for lack of free time, to offer support of any kind. For questions or problems, please post in the Avsim scenery design forum.

Filename: terrain_design_for_flight_simulator_x.zip
License: Freeware
Added: 14th December 2009, 13:29:15
Downloads: 16648
Author: Luis Feliz-Tirado
Size: 3991kb

https://library.avsim.net/esearch.php?CatID=fsxsd&DLID=141643

[END_EDIT]

Hope this helps as an additional option.

GaryGB

 

[crushnik]

Or do you already have the 3D "X,Y,Z" ESRI *.SHP water body polygon data prepared, are are primarily looking or more details on the ScenProc scripting method itself ?

I'm sure it's quite simple if you know where to look, but I am not sure how to tell if my data has "Z" coordinates.

I have two shapefile options, so far as I can tell: one, I can use shapefiles generated from USGS NHD "statewide" waterbody data. Two, I think the watershed boundary NHD Plus data may already have elevation data attached, but their boundaries are dependant on terrain, which makes them irregular and more difficult to determine which datasets I need (plus, some areas are still under development).

Currently, I make exclude polygons for P3D's existing streamlines, shorelines, and water polys. Then I import statewide waterbody shapefiles I created in QGIS, using NHDArea and NHDWaterbody vectors, into SBuilder and import into SBuilder using the "Legacy no flatten" GUID, since any other GUID imports at what I believe is sea level.

I was thinking I had read it was possible to lay vectors on top of raster elevation data, but couldn't remember where I saw that since I have read about three million things since I began learning how to make scenery; I just didn't know it was you answering a previous question from me. Fortunately I know enough now where I can understand things that didn't make sense previously.

If there are already elevation coordinates, I wonder if ScenProc even needs to import elevation TIFs?

 

[GaryGB]

Please note my latter edits to the post above:

https://www.fsdeveloper.com/forum/t...h-multiple-elevation-files.448292/post-850076


SBuilderX will automatically retain any 3D "X,Y,Z" ESRI *.SHP water body polygon data Altitude attribute values ...when already present within the 'Appended' file.

Luis Feliz Tirado's tutorial cited above in the section on "Sloped Polygons" shows where to view / edit the Altitude parameter value for vertex 'Points'.

[EDITED]

CAVEAT: For 'sloped' water, once a 3D .SHP is Appended, do not enter a "Constant altitude" value or you may over-write all vertex Altitudes with a single flat / level value !


PS: In SBuilderX, always export a SBuilderX Exchange File (*.SBX) to save all vector and misc. project info, as not all such data is saved in a 'basic' SBuilderX Project File (*.SBP).

*.SBX files are ASCII text and can be viewed easily.


Alternatively, select the desired vectors and Export a 'surfer *.BLN' file (which is also ASCII text, and can be viewed easily) ...to verify your polygons are 3D "X,Y,Z" format.



BTW: I almost forgot to mention that I have yet to find a vector water body data set which precisely matches the visible water bodies seen in various imagery data sets.


You should be able to display your project imagery and edit a superimposed *.SHP vector layer in QGIS (both projected in EPSG:3857 before export projected in EPSG:4326):

QGIS Shape Editing

wiki.tuflow.com

https://www.google.com/search?sourc...hUKEwi8jcbbkP3qAhUDZc0KHQipDn0Q4dUDCAg&uact=5


Or, after a "X,Y,Z" ESRI *.SHP water body polygon data set is 'Appended' to SBuilderX, any / all intended project imagery sources in *.BMP format ...can also be Appended.

It is important that all imagery and vector source data being 'Appended' to SBuilderX is projected in EPSG:4326.


This is done via SBuilderX Menu > File > Add Map > From Disk ...rather than via the usual 'show Background imagery from tile server' function.


NOTE: If a *.TXT 'world' Geo-referencing file with a file name identical to the imagery file (...but with a *.TXT file extension instead of *.BMP) is present in the same folder as a imagery *.BMP , it need not be "re-calibrated" when Added as a Map from Disk.

https://www.fsdeveloper.com/forum/t...rom-disk-trouble-compiling.442290/post-794930


Then one can select and move entire polygons, or individual vertex points can be moved ...into precise alignment with intended imagery sources so everything "fits" properly.

[END_EDIT]

GaryGB

 

[crushnik]

Thanks Gary!!!

 

[GaryGB]

'Bumped' for another latter edit in my post above.

https://www.fsdeveloper.com/forum/t...h-multiple-elevation-files.448292/post-850081

GaryGB

 

[arno]

Hi,

I'm not sure what kind of scenProc script you use now, but you will see that the CreateTVecWaterPoly has arguments to select elevation raster data to sample altitudes from. So in there you would select your elevation files, that you read with the ImportGDAL step.

 

[GaryGB]

Hi Chris:

The ScenProc "stable" release dated January 3, 2020 does not cite "CreateTVecWaterPoly" in the PDF manual.

ScenProc "Development" versions (...released after 04-28-2019 ? ) do cite "CreateTVecWaterPoly" in the PDF manual.

See:

Download scenProc

scenProc is available as a stable release version 3.1 and a development release version 3.2. The development release contains all the latest features, but might be a bit unstable or undocumented sometimes. For the stable release you can download a version for 32 bit systems and for 64 bit...

www.fsdeveloper.com

http://www.scenerydesign.org/development-releases/


IMHO, since Arno rarely misses bugs in his coding, it is 'safe' to always use a development release of his FS utilities.


PS: As of today's date, there are only (2) Google query results for that ScenProc parameter "CreateTVecWaterPoly"

https://www.google.com/search?ei=rDcnX5XqO4vbtAbw9qDoDw&q=site:www.fsdeveloper.com+CreateTVecWaterPoly&oq=site:www.fsdeveloper.com+CreateTVecWaterPoly&gs_lcp=CgZwc3ktYWIQDFDwzylYt-gpYPj6KWgBcAB4AIABhQGIAb0EkgEDNi4xmAEAoAEBoAECqgEHZ3dzLXdpesABAQ&sclient=psy-ab&ved=0ahUKEwiV8KzKwv3qAhWLLc0KHXA7CP0Q4dUDCAs


Hope this helps.

GaryGB

 

[crushnik]

IMHO, since Arno rarely misses bugs in his coding, it is 'safe' to always use a development release of his FS utilities.

I have had no issues with using development releases.

 

[crushnik]

I'm not sure what kind of scenProc script you use now, but you will see that the CreateTVecWaterPoly has arguments to select elevation raster data to sample altitudes from. So in there you would select your elevation files, that you read with the ImportGDAL step.

I am wanting to import shapefiles from this state-sized dataset:

https://prd-tnm.s3.amazonaws.com/StagedProducts/Hydrography/NHD/State/HighResolution/Shape/NHD_H_Kentucky_State_Shape.zip

So is there a way to effectively import data from the numerous 1-degree 1/3 arc-second elevation rasters it would take in the ImportGDAL step?

 

[GaryGB]

One might also wonder whether it will be necessary to first segment the *.SHP files into smaller data sets.

Here's the above linked *.ZIP loaded in Global Mapper, and displayed to its full Geographic extents:

GaryGB

 

[arno]

Hi,

If you have all the elevation data in one folder, you can just use the wildcard to select them all. I'm not sure how big they are (they should fit in memory).

Another approach would be to batch run one elevation file at a time and then use the matching vector data.

 

[crushnik]

If you have all the elevation data in one folder, you can just use the wildcard to select them all. I'm not sure how big they are (they should fit in memory).

They are all in one folder, so that sounds great Arno!

One might also wonder whether it will be necessary to first segment the *.SHP files into smaller data sets.

I wanted to use statewide files, because they are complete (USGS is still working on some of the NHD Plus areas) and I have about one-third of the country already completed. But I tried making streams out of one of the NHD Plus datasets -- which are smaller -- and SBuilder got it to make streams, which I think from my prior struggles with streams that the NHD Plus datasets must include altitude data. If that's the case, I'll tell QGIS to filter out streams that aren't named (the "Flowlines" seem to be everything down to the low spots in people's yards) and go that route. Then I'll bet I can tell ScenProc to turn those streamlines into stream polys about 3 meters wide.

Then, the challenge will be to dive into shorelines... Not looking forward to that!

 

[GaryGB]

One might also wonder whether it will be necessary to first segment the *.SHP files into smaller data sets.

I wanted to use statewide files, because they are complete (USGS is still working on some of the NHD Plus areas) and I have about one-third of the country already completed. But I tried making streams out of one of the NHD Plus datasets -- which are smaller -- and SBuilder got it to make streams, which I think from my prior struggles with streams that the NHD Plus datasets must include altitude data. If that's the case, I'll tell QGIS to filter out streams that aren't named (the "Flowlines" seem to be everything down to the low spots in people's yards) and go that route. Then I'll bet I

Then, the challenge will be to dive into shorelines... Not looking forward to that!

CAVEAT: Always dive OFF of shorelines INTO Hydro polygon, and not the reverse... otherwise it 'hurts'.

FYI: In SBuilderX, one may copy a Hydro polygon, paste it back into the exact position it was copied from, select it, reverse its poly-line vertex winding direction from ex: CW to CCW, select and convert it to a poly-line; then assign it a shoreline GUID.

NOTE: It is a 'best practice' to have 'child' shoreline vertices exactly aligned with those of its 'parent' Hydro polygon.

This is easily done by making a copy of the source Hydro Polygon object, then pasting it precisely back into position, and converting it into a Poly-line object.

Perhaps you can also tell ScenProc to turn copies of those stream polys into shorelines ?

SDK SHP2VEC can compile data sets for stream polygons and shoreline poly-lines into (1) BGL.

Hi Chris:

Dick had this to say on the subject ...back in 2012:

The terrain.cfg file controls the rendering priority of vectors and polys. Streamlines are given priority over shorelines. Shorelines are given priority over waterpolys. Logically, streamlines also have priority over waterpolys.

You need to end the streamline data at the edges of the waterpolys in a GIS program, or in something like SBuilderX. Changing the terrain.cfg would lead to problems.

Dick

FYI: Thorsten Reichert (who has since gained a better use of conversational English and courtesy than seen in his many early posts here and at AVSIM over 10 to 15 years ago ) had started one of the earliest threads on this subject here:

FSXA - Render polys above vectors

This is one for the experts I guess. I created some rivers as vectors and want them to be rendered below waterpolys, because it looks funny it both overlap. It seems FSX always renders polys first and then vectors, no matter what renderpriority you assign. Can anybody confirm this? Would be...

www.fsdeveloper.com

In 2014 I revisited that thread to report updated info on yet another possible way to utilize 2 adjacent vector data types in close proximity without (yet) a true merge having been implemented in the GIS application,

According to MS-ACES' Doug Matthews following release of the FSX SDK, one technically can combine both Polygon and Line data in the same BGL via SDK SHP2VEC, although most GIS and FS utilities do not appear to attempt that task, and instead split those (2) vector object types into separate data files and BGLs.


BTW: Streams are in the *CVX*.BGL vector files within [FS install path]\Scenery\[Area Number]\Scenery sub-folder chains.


NOTE: CVX vector streams are written into a separate *ST*.SHP by Patrick Germain's CvxExtractor utility when those objects are output; they may then be Appended into SBuilderX and/or imported into a GIS application for further processing.


PS: There may be other threads on this subject wherein Arno may have posted a work-flow to merge vector polygons via ex: ScenProc, but I have not had time to further research that.

GaryGB

Hope this review of available info helps as you plan that next step in your project work-flow.


PS: August 1st, 2020 ScenProc PDF Manual shows:

"9.5.7 CreateTVecShoreline

Purpose

Create shoreline terrain vectors.

Arguments

1. The filter to select the line features that should be used to create the shorelines from.

2. The GUID of the shoreline type to use.

Example

Create airport bounds from all airport features and use the attribute elev for the elevation:

CreateTVecShoreline|type="shore"|{46bfb3bd-ce68-418e-8112-feba17428ace}

< OOPS ? >

Arno, is the example above shown for CreateTVecShoreline ...an editing error ?

AFAIK, shorelines are lines, not polygons, and they have different GUIDs.

https://www.prepar3d.com/SDKv4/sdk/world/terrain/terrain_overview.html#The Shp2Vec Tool

https://www.prepar3d.com/SDKv4/sdk/world/terrain/vector_shape_properties_guids.html


9.5.8 CreateTVecStream

Purpose

Create stream terrain vectors.

Arguments

1. The filter to select the line features that should be used to create the streams from.

2. The GUID of the stream type to use.

Example

Create streams from all river features:

CreateTVecStream|waterway="river"|{2d3fc985-a72b-473d-b23b-d78e72e63b53} "


I have not yet found a section in the ScenProc manual describing a discrete function which can be used as an intermediate step, to break Polygons into Poly-lines, then copy them.

But perhaps Arno will explain whether you actually can also tell ScenProc to turn copies of those stream polygons ...into shore-lines in a scripted sequential processing session?


GaryGB

 

[arno]

Hi Gary,

Looks like a copy-paste mistake in the manual, I'll fix that.

At the moment scenProc has no PolygonToLine step, as I never needed it. But it should not be too hard to make it.

 

[GaryGB]

Thanks, Arno.

That will save some complexity for Chris in SBuilderX making Polygon "Holes" and/or making 'reversed' vertex order (child) Poly-lines derived from source (parent) Polygons.

https://www.google.com/search?sourc...hUKEwjQyO_Eu4PrAhWPK80KHWoCAooQ4dUDCAg&uact=5


There are otherwise many options to be "explored" in the SBuilderX Help file in the section on:

Working with Points, Lines and Polygons

GaryGB

 

[crushnik]

There are otherwise many options to be "explored" the SBuilderX Help file in the section on:

Working with Points, Lines and Polygons

GaryGB

Having read, re-read, and re-re-read Gary's posts, I wanted to report that I did manage to get 4-meter streamlines by using ScenProc. I'm not the biggest fan of the consistently sized lines, but they do bring more depth into the sim. Unfortunately, I wasn't able to figure out what element of the shapefile data gives ScenProc the elevation it needs so I had to use the "Legacy No Flatten" hydro polys. Once I find that elevation data, I can re-do all my hydro polys with something that looks better than the legacy ones.

#
ImportOGR|path\to\shapefile|*|*|NOREPROJ
#
LineToPolygon|FTYPE="LINE"|4|String;polygons|stream
#
CreateTVecWaterPoly|*|{5835459A-4B8B-41F2-ADC1-DEE721573B28}|ATTR_VALUE|elev|NONE|NONE
#
ExportTVecBGL|path\to\folder

 

[GaryGB]

Hi Chris:

August 1, 2020 ScenProc documentation does not address Elevation use in "CreateTVecStream" scripting.


BTW: Do you actually want to (only) use Textured Vector Streams as Poly-lines of specified width ?

Or do you instead intend to make Polygons with surrounding Shorelines via ScenProc functions for "CreateTVecWaterPoly" and "CreateTVecShoreline" using Polygons derived from source Poly-lines, so that you will have FS inland "Living Water" with default textured Shorelines and Water Class texture colors showing under the Hydro attribute layer ...in FS at run time ?

GaryGB

 

[arno]

As far as I know the streams have no altitude and follow the elevation mesh.

 

[crushnik]

BTW: Do you actually want to (only) use Textured Vector Streams as Poly-lines of specified width ?

Or...
GaryGB

I am currently only capable of turning NHD shapefiles (NHD Area and NHD Waterbody layers) into water vectors. I was fine with the default P3D/FSX streamline, but lines sit on top of polygons so I had to find a way to turn the NHD Flowline data into a four-meter-wide poly. The water shapes look much better in the sim, but I haven't been able to get the shorelines to happen yet -- I'm sure I'll get somewhere by revisiting your always helpful posts, which I have to do often. Plus, the water texture probably isn't nearly as good as it should be, since I am using the Legacy No Flatten GUID as I can't figure out where to point ScenProc to find embedded "Z" elevation data for vectors.

Here is what I would like to do regarding water (and what I have so far is available to the FS community as freeware on my website):

1.) Turn streamlines into polys
2.) Import waterbodies and streams into sim
3.) Add shorelines to all
4.) Use non-legacy water GUID
5.) Eventually change waterclass for different types of waterbodies

Step 1 is pretty easy, I just figured out a way to do number 2 (but there may be much better ways), 3-4 I am stuck on and 5 has a lot of higher-priority stuff ahead of it. I hope I made a little more sense, and thank you!