LIDAR 1m Data and FSX P3D Default Mesh, FSGlobal2010 Mesh, GEN Mesh

Christian Bahr

Resource contributor
#1
Hello.

Although I have already gained some experience in the sceneriedesign, the handling of digital elevation models is still somewhat foreign to me. So look after me, that I ask here a beginner question.

For my current project Dortmund 2017 I am currently working on an update to adapt my scenery to FTX Germany North by ORBX. Now I would like to use my own elevation model - it has a resolution of 1m and is LIDAR and it is absolutely exact in the elevations.

The elevation model for the FSX P3D was quite simple, but there is a misrepresentation: the LIDAR elevation data have a different height than the elevation model of FTX Germany North. There is a difference in the amount of FTX germany North and the LIDAR 1m elevation data. This is particularly noticeable at the transition of both elevation models, see screenshot:


Picture: The difference in height in the simulator is visible on the left, on theright the transition of the LIDAR data in the GIS

So my question: how can I bring the LIDAR Data to the same height as the 19m elevation model of FTX Germany North?

I am grateful for every answer.

Cherrs,
Christian
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#3
I'm guessing you want to raise the LIDAR 2 meters? This could be done in Global Mapper, or some other GIS program. You might try adding BaseValue = 2 to the [Destination] section of the INF file... I think this will add 2 meters to all the points.

Otherwise you'll need to make sloped flattens to the edges of the mesh to smooth the transition of the elevations.
 

Christian Bahr

Resource contributor
#5
So, I have now tried all possible values with BaseValue. I have unfortunately no result - the data can neither be raised nor lowered. Perhaps it is because I use a grayscale GeoTiff file. What exactly do you mean that you can solve this also in the GIS software, which tool (ArcMap/Qgis) would be in question?

Here is the content of my .inf file "do_dgm_1m.inf":

[Source]
Type = GeoTIFF
Layer = Elevation
SourceDir = "."
SourceFile = "do_dgm_1m.Tif"
NullValue = 0

[Destination]
DestDir = "."
DestBaseFileName = "do_dgm_1m"
DestFileType = BGL
LOD = Auto
FractionBits = 3

I use the value "FractionBits = 3" to counteract the effect of the terraces, as documented in the SDK.

Christian
 
#6
If you have access to the source data for the 19m mesh you should blend the 1m data with the area just outside the 1m dataset (I use Global Mapper for that). I have done that with the Australian data, but I had to get the source data from Holger to do it. If there is another dataset that is closer to the 19m FTX data that you could belnd with if you don't have the FTX source then that might be a substitute, if less than optimal. You could make a 19m (ish) boundary around your 1m data sampled at that resolution, then blend the 1m data with that at 1m resolution. That would at least remove the cliff face.

Braedon
 

Christian Bahr

Resource contributor
#7
Hello Braedon, thanks for your reply.

I do not have access to the source data of FTX. But your tip with the blend of both DEM's sounds quite good. But this would not solve the problem with the different heights of both DEM's. Because there is a height difference of about 2-3 meters - the FTX data is about 2 meters higher than the LIDAR data I use. The LIDAR data I use must therefore be raised to the height level of FTX so that there is a better transition from one DEM (FTX) to the other DEM (Dortmund 2017). So the question is close to how I can raise my data with the help of a GIS by 2 meters.

Do I understand you correctly that by the blend both DEM's the problems are solved?
 
#8
Hello Christian:

Is your described source GeoTiff in a 32-bit raster gridded elevation data format ? :scratchch

https://translate.google.com/transl...ahrometrix.de/category/allgemein/&prev=search

[EDITED]

If so, you may not need to use FractionBits=3 to eliminate "terracing" in Dortmund with ground elevations ranging between approximately 200 -to- 400 Feet AMSL, when the stated gridded elevation data point interval on the ground for the area is as close together as 1 Meter, with a "Value and unit of the absolute positional accuracy" at 0.3 meter.

https://translate.googleusercontent.com/translate_c?depth=1&hl=en&prev=search&rurl=translate.google.com&sl=de&sp=nmt4&u=https://geoportal.brandenburg.de/detailansichtdienst/render?view=gdibb&url=https://geoportal.brandenburg.de/gs-json/xml?fileid=488a2b53-564f-43eb-88ec-0d87bb43ed20&usg=ALkJrhg_GnH94naFw-cVtRYQdADnvFjvYQ

https://translate.google.com/transl...bb.de/dienstleister/testdaten.htm&prev=search


NOTE
: If the "DGM-1m" 1 Meter LiDAR *.XYZ data set is processed in full-featured GIS software to make a DTM (triangulated 3D elevation grid, aka "TIN"), the resulting intermediate data set TIN can also be separately output and re-projected to Geographic (Lat-Lon) Projection / WGS84 Datum ...as a 32-bit GeoTiff compatible for use with FS SDK Resample.

You may wish to compare your own (purchased ?) 1 Meter LiDAR source data with the freely downloadable 'Test Data' ZIP file linked below, which contains a folder / file structure that may be examined to locate the example "DGM-1m" 1 Meter LiDAR *.XYZ data set in:

DGM_Testdaten\DGM_Testdaten\ASC\XYZ-ASCII\DGMLASERSCAN_[464493-5772164_dgm.xyz]


...as a preferred alternative to use of the the 8-Bit GeoTIFF file in:

DGM_Testdaten\DGM_Testdaten\Schummerung\DGMLASERSCAN_[464493-5772165_dgm.tif]

https://www.geobasis-bb.de/dienstleister/testdaten/DGM_Testdaten.zip

[END_EDIT]


FYI: This type of scenario of using both FS SDK Resample parameters "BaseValue" and "FractionBits" together ...can be rather mind-numbing. :alert:


To better understand how Resample will interpret and output elevation source data values when using both FS SDK Resample parameters "BaseValue" and "FractionBits" together, I suggest reviewing these FS Terrain SDK references: :idea:


https://msdn.microsoft.com/en-us/library/cc707102.aspx#UsingScaledElevationValues

http://www.fsdeveloper.com/wiki/index.php?title=BGL_File_Format#RCS1_Record


...and this thread:

http://www.fsdeveloper.com/forum/threads/problem-after-converting-mesh.428245/



Also, consider the additional details on the 'BaseValue' parameter from the FS2004 Terrain SDK Docs:

"You can use the FractionBits and BaseValue directives in either the [Source] or [Destination] sections, or in both sections. In the case where these directives specify different scalings in the [Source] and [Destination] sections, the resampler will attempt to form the source scale to the destination scale. If a loss of precision occurs, the resampler will print an error message."

http://www.fsdeveloper.com/forum/resources/terrain-sdk.40/



The FSX Terrain SDK docs, by comparison, regarding Resample *.INF subsection(s) where these "elevation scaling" parameters may be used, states:

"Use the FractionBits and BaseValue directives in the [Destination] section; in the [Source] section these parameters have a different, and now deprecated, purpose."


NOTE: "Deprecated" does not mean something no longer works in FSX SDK Resample (...as most legacy Parameters / Values still do). ;)


I suggest that a "bit" more experimentation may prove successful. :)


[EDITED]

PS: Just as an idea, in the event that you wish to use ESRI ArcGrid to "feather" the overlap region of 1 Meter DEM into a 19 M DEM source data set of elevation values, it is possible that one could perform a GIS re-gridding procedure.

http://desktop.arcgis.com/en/arcmap...ysis/cell-size-and-resampling-in-analysis.htm

http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Resample

https://pro.arcgis.com/en/pro-app/tool-reference/data-management/resample.htm


This could allow one to 'over-sample' the (less costly and lower resolution) 25 Meter elevation data set for the perimeter regions adjacent to the (more costly and higher resolution) central Dortmund area of interest to yield a 19 Meter effective output resolution, which is equivalent to LOD-11 (aka "QMID-13") in the FS terrain mesh grid quad tree scheme.

https://translate.google.com/transl...de/geodatenonline/seiten/dgm_info&prev=search


The resulting 'perimeter' 19 Meter elevation data might then be utilized for "feathering" with the 'central' 1 Meter data set ...as Braedon alluded to above:

http://www.fsdeveloper.com/forum/threads/lidar-1m-data-and-ftx-germany-north-dem.439525/#post-766217

[END_EDIT]


BTW
: Holger Sandmann (OrbX 'Geo-spaetzle lead' < ...OOPS, I mean "Geo-spatial lead" ... :D> would probably know what source data was used for OrbX FTX North Germany, and what type of work-flow might be feasible to achieve a compatible result that allows super-imposing your 1 Meter terrain mesh "on top of" ...the FS terrain mesh for the above cited Orbx product around Dortmund. :pushpin:


GaryGB
 
Last edited:

Christian Bahr

Resource contributor
#10
Hello Gary.

Thank you for your extensive link list - now my confusion is perfect ;-)

You can believe me that I have read the corresponding sections in the SDK up and down and have spent a lot of time to come to a conclusion. Only when I did not get any further, I have registered here in the forum to ask my question, which is: How can I bring the LIDAR data to the same level as the 19m DEM of GEN.

As I mentioned before, I have tried all possible values. It was about the combination of BaseValue and FractionBits. But nothing helped.

And yes, it's a 32-bit GeoTiff file, gridded. If I resample the GeoTiff file without FractionBits = 3, then I have a terraces effect. If I resample the GeoTiff file with FractionBits = 3, then the Terrassen effect is gone, it is also described in the SDK, see pictures:


Left side: DEM with FractionBits = 1, right side: DEM with FractionBits = 3



Left side: DEM with FractionBits = 1, right side: DEM with FractionBits = 3


Picture: THe LIDAR Data was resampelt without the Value "FractionBits". The result is the undesirable terrace effect.

But the problem is not the terraces effect. The problem is a difference in height in the respective DEM. While the LIDAR data are very accurate, the GEN DEM are about 2-3 meters higher than in reality. Therefore I have to raise my LIDAR data somewhat, preferably with a GIS (ArcMap/QGIS). But I do not know the function with which one can "raise" the LIDAR data. A winning tip would be very welcome.

In other tests it has now been shown that not only the GEN DEM has a different height, but also FSGlobal2010 and the default DEM of the P3D have a height deviating from the LIDAR data - the deviations are more than 2 meters. The assumption is therefore that the GEN DEM was adjusted to the height of existing DEM's (FSXP3D Default, FSGlobal2010) - which also makes a great sense.

Theme Feather
Applying a soft edge (feather) has not succeeded because this soft edge is misinterpreted - instead of the soft edge, the height is set to normal zero, see picture:


Picture: The soft edge causes a "normal zero".

Does the resample tool support a soft edge at all?
 
#11
The problem is a difference in height in the respective DEM. While the LIDAR data are very accurate, the GEN DEM are about 2-3 meters higher than in reality. Therefore I have to raise my LIDAR data somewhat, preferably with a GIS (ArcMap/QGIS). But I do not know the function with which one can "raise" the LIDAR data. A winning tip would be very welcome.
* For ArcMap, perhaps "Layer Offset" is what you might use ?

http://desktop.arcgis.com/en/arcmap...etting-feature-heights-z-s-for-3d-editing.htm

https://www.google.com/#q=ArcMap+elevation+offset&*


* For QGIS, perhaps "Raster Calculator" is what you might use ?

http://gis.stackexchange.com/questions/25850/elevation-correction-aster-gdem2-10m

http://spatialgalaxy.net/2012/01/25/using-the-qgis-raster-calculator/

https://www.google.com/#q=QGIS+elevation+offset&*


In other tests it has now been shown that not only the GEN DEM has a different height, but also FSGlobal2010 and the default DEM of the P3D have a height deviating from the LIDAR data - the deviations are more than 2 meters. The assumption is therefore that the GEN DEM was adjusted to the height of existing DEM's (FSXP3D Default, FSGlobal2010) - which also makes a great sense.
IIUC, "GEN DEM" refers to the terrain mesh for OrbX FTX Germany North ? :scratchch

"GEN mesh is compiled at 20-m grid spacing"

https://orbxdirect.com/product/gen


Theme Feather
Applying a soft edge (feather) has not succeeded because this soft edge is misinterpreted - instead of the soft edge, the height is set to normal zero, see picture:

http://fsfiles.org/flightsimshotsv2/images/2017/02/28/4do_dgm1_feather.jpg

Picture: The soft edge causes a "normal zero".

Does the (FS SDK) Resample tool support a soft edge at all?
AFAIK, this is not possible for FS terrain mesh BGLs that use discrete elevation data points (...see PixelIsPoint=1 in GIS Metadata and *.INF files).

Thus a "Blend" mask with such a "elevation" data type GeoTiff source file would not work in the same way as how FS SDK Resample otherwise would work for aerial imagery Raster texture data used to make terrain mesh clinging custom land class "textures".



Regarding the work-flow I suggested above to use the XYZ elvation data rather than the 8-Bit GeoTIFF you may have in your commercially sourced "DGM-1m" 1 Meter LiDAR elevation data:

Triangulation of the *.XYZ file containing ASCII numeric text for Geographic coordinates in 1-line / 3 field records: X (Lon), Y (Lat), Z (Alt), yields a vector-based "TIN", which when 'gridded' at assigned intervals on the ground, is next converted to a 32-Bit raster-based GeoTIFF of elevation data.



In theory, within a full-featured GIS application such as Global Mapper / ArcGrid / QGIS etc., one could create a "feathered edge" between a smaller central higher resolution area feature of "DGM-1m" LiDAR raster elevation data points that is 'blended' into the surrounding larger lower resolution area feature of "DGM-19m" LiDAR raster elevation data points to create a derived composite 1-piece area feature of raster elevation data points in a 32-bit GeoTIFF file format.

IIUC, the above "Feathering" process may allow the elevation data point values to be interpolated on a gradient between the 2 source area feature pixels along the edge of where those data sets meet one another ...presumably resulting in a data set without the 2 Meter elevation height mis-match.

IMHO, this presumably would not produce the same (un-desirable ?) effect seen when modifying terrain mesh display in FS at run time by super-imposing a 'sloped flatten' between the 2 mis-matched elevation sets to disguise the transition between your smaller 1 Meter custom mesh, and the surrounding larger OrbX FTX "GEN" 19 Meter custom mesh, "if" both the central terrain mesh and surrounding terrain mesh are made from data sets which use the same base elevation value, and thus do not have a 2 Meter mis-match between elevation data set base altitudes AMSL.



NOTE
: The above process involves raster processing of pixels that represent elevation data points within a GIS application, and not within FS SDK Resample, and thus does not attempt to achieve "Blending" of elevation data points within terrain mesh BGLs to form a 'ground' surface via the FS run time rendering engine.


IMHO, neither of the latter methods ("feathering" -or- sloped flattens) would be desirable, considering the amount of work that is involved; so certainly it would seem preferable to first attempt creation of your "DGM-1m" LiDAR-derived 1 Meter mesh ...with a (+)2 Meter elevation "offset" of the base altitude AMSL within the source data prior to creating your terrain mesh via FS SDK Resample.



CAVEAT: Assuming that the above "elevation base offset" process works for you, be aware that you may find the total Geographic extents of the areas covered by your "DGM-1m" LiDAR-derived 1 Meter custom FS terrain mesh BGL may need to be 'artificially' expanded to a full LOD-2 or LOD-3 quad size in order for your 'micro mesh' to successfully be rendered as FS terrain modifications at run time with a higher display priority than other underlying larger LOD extent default or custom terrain mesh files (ex: FTX GEN DEM / FSXP3D Default / FSGlobal2010 etc.) ...for user aircraft that start flights to Dortmund from significant distances outside the Dortmund region. :pushpin:

This can be achieved via inclusion of small (ex: 3) isolated Geographic coordinate feature "areas" of actual elevation data points placed at the Geographic coordinates of the outer-most NW and SE corners of FS terrain LOD-2 or LOD-3 quad size areas ...within the source data for your FS terrain mesh BGLs to be output by FS SDK Resample.

[EDITED]

This process would likely also involve use of GIS "NO_DATA" attributes for areas outside your 1 Meter elevation data coverage that extend up to the outer LOD-2 or LOD-3 quad border extents. ;)

[END_EDIT]


If you wish to learn more about work-arounds for this quirk of how FS terrain mesh is rendered with highest display priority based on largest total Geographic extent size (...and lowest LOD) of the local ground areas covered by each FS terrain mesh BGL within the FS terrain quad grid system, feel free to inquire further. :)



BTW: On a practical basis, one will only see the details of a 1 Meter custom terrain mesh at run time in FS at very low altitudes in the user aircraft (either on-ground- or very close to it-) at Dortmund airport, so one might focus primarily on fixing a "mesh mis-match" in the immediate vicinity of ICAO EDLW:

https://en.wikipedia.org/wiki/Dortmund_Airport

https://maps.google.com/maps?ll=51.518333,7.612222&q=51.518333,7.612222&hl=en&t=h&z=14

https://tools.wmflabs.org/wp-world/earth.php?long=7.612222&lat=51.518333&name=Dortmund_Airport


Also, one may wish to limit the extent of your custom EDLW coverage area with 1 Meter mesh, due to a possible added 'performance hit' imparted on top of that already reported for that OrbX FTX scenery packge: ;)

https://translate.google.com/transl...s-geht-von-simmershome-szenerien/&prev=search


PS:

Thank you for your extensive link list - now my confusion is perfect ;-)
If at any point you find that your confusion is "less than" perfect, please let me know, and I'm confident that I could provide many more links to 'GIS Geek' web sites ('purportedly' written in English) ...to ensure complete confusion ! :rotfl:


GaryGB
 
Last edited:

Christian Bahr

Resource contributor
#12
Hi Gary.

Now my confusion is already much less big :)

Your tip with the raster calculator in QGIS has brought me absolutely further. Finally!

At first a personal word: because of my lack of knowledge in English, it can sometimes happen that I have given some formulations of "hard", but I did not want to, of course, in the native language, it is much more sensitive and better to write As in a language in which one is not so much practiced. My English language skills are rather rudimentary.

I would like to thank you for your patient and detailed answers. This is not self-evident, that someone is as helpful to you as you do. Therefore in all form: have a huge thanks for your good and friendly support. Hopefully I can return some of this relief at some time.

All the best and great thanks
Christian
 
#13
I'm glad to see that you are making some further progress with this very 'under-documented' part of FS mesh-making; keep up the good work ! :cool:

GaryGB
 

Christian Bahr

Resource contributor
#14
Yes, the beginning in the topic Elevation Data is done. Now you have to look calmly how the thing further refined to come to a good result. And as I said, without your help I would probably despair :)

Cheers,
Christian
 
#15
A very interesting topic, which I will look further in to when wider awake. I am a fan of the use of lidar data in FSX and P3D, and have made several "test" meshes with lidar data from opentopography.org.

http://www.opentopography.org/

It can look amazing when used with photoreal. I have made several meshes of the Barringer Crater in Arizona, and would like to be able to blend that better to the surrounds. I have contemplated, and done some work on making a scenery of the crater and it's surrounds, but have encountered lots of hurdles. Perhaps this thread will help to overcome some of them.

I'm sure your Dortmund project will turn out awesome Christian!
 
Last edited:

Christian Bahr

Resource contributor
#16
Many Thanks!

I can only join your opinion. The LIDAR data is a very interesting topic. Certainly I could have asked for help via e-mail contacts, but I had deliberately decided for the forum. Because here many more people can read and participate. Because I am certainly not the only person who can help with this topic. So many are helped.

The Beringer Crater in Arizona is really impressive. At that time, in FS2004 there was already a elevation model of the crater. When I saw this, it was very impressive for me. I like such things very much. Was the elevation model of the Beringer crater from the FS2004 of you?

By the way: Your link to opentopography does not work, can you be so kind and correct in? Thank you :)

Greetings
Christian


Here are a few screenshots from the current update for my project Dortmund 2017:


Picture 1: Excavated earth as base in the new industrial Estate Phoenix West


Picture 2: Building materials as a hill in the port of Dortmund


Picture 3: The rehabilitated riverbed of the Emscher


Picture 4: Excavated earth as an view point at the Phoenix See


Picture 5: Wall at the Phoenix See


Picture 6: The area around the former coal power station "Kneppler" on the outskirts of Dortmund in the GIS


Picture 7: the same area in P3D ...


Picture 8: ... here is a little more overview. The motorway dam also stands out clearly.

"EDIT"

Sources
LIDAR Data: Land NRW 2017
Orthophotos: Land NRW 2017
Geobase Data: OpenStreetMap
 
Last edited:
#17
Apologies Christian. It has been a while since I visited Opentopography.org. I have downloaded a lot from there and made a lot of small meshes using their data. I have now fixed the link above in my post above, and here it is again: http://www.opentopography.org/

It is best to sign up, as you can download larger areas of data. There are different formats you can get the data in, although I prefer the raster data as it comes in Tiff format. Most data still needs to be converted, and I use QGIS for that. It is like a freeware version of Global Mapper, and has a lot of great features.

I didn't do the elevation model of the Meteor Crater in FS2004, in fact I never used FS2004. I have created a 1m mesh from 30cm(?) lidar data and have combined it with photoscenery. If you use Bluesky scenery or Megascenery then you could just use the mesh. That's what I have done quite a bit, as I have a lot of PR from Bluesky and Megascenery it looks great when you create a high res mesh for some of it. In fact the reason I started making hi res meshes was after I downloaded a hi res mesh of New Jersey at 5m which looked amazing.

These are not the best pics of it, but here is a link to pics of the first time I created the Barringer Crater (scroll down for the success pics :D):

http://aussiex.org/forum/index.php?/topic/21864-well-that-didnt-work/#comment-218914

BTW your scenery is looking amazing!


Cheers,
 
#18
Well done, Christian ! :)


BTW: It looks like you will have access to data for very accurate FS scenery of most, if not all, of Germany if you are inclined to pursue this further: :idea:

https://rapidlasso.com/category/download-portal/



PS: Some additional info related to allowing proper local display of higher LOD custom terrain mesh when flights originate from start locations outside extents of larger / lower resolution terrain mesh BGL files:

http://www.fsdeveloper.com/forum/threads/higher-lod-terrain-not-drawn.435658/


NOTE
: IMHO, end users of 1 Meter resolution terrain mesh should be advised to reduce the FS GUI Menu terrain resolution slider setting (move it to the Left) once their user aircraft are outside the extents of any such local mesh BGLs if they experience FPS performance / scenery rendering problems:

http://www.avsim.com/forums/topic/395348-fsx-default-terrain-vs-downloads/?page=2#entry2556667


https://blogs.msdn.microsoft.com/ptaylor/2007/10/28/game-engine-design-design-goals-and-choices/


GaryGB
 
Last edited:
Top