MSFS High Level Overview of MSFS2020 Tech

[Devinci]

You can compile the airport xml files using the program "C:\MSFS SDK\Tools\bin\fspackagetool"
There is an example on the sdk folder to create an airport.
You can create airports just using xml.

Regards
Alfredo Mendiola Loyola

I see,
I give it another try. Thanks for the information!

Regards

 

[spatialpro]

So MSFS uses Bing Maps spherical Mercator projection (i.e. not an ellipsoid)...

My first reactions to that are:

1) What is the vertical datum?
2) I imagine a need for a lot of vertical adjustments between 3rd party DEM data and the sim (although I appreciate it appears likely possibility to do so could be restricted to "patches" of QMID data around airfields for most mainstream devs)?

 

[theisomizer]

I have added a part 3 with more topics back near the top replacing my placeholder post. Enjoy!

 

[theisomizer]

So MSFS uses Bing Maps spherical Mercator projection (i.e. not an ellipsoid)...

My first reactions to that are:

1) What is the vertical datum?
2) I imagine a need for a lot of vertical adjustments between 3rd party DEM data and the sim (although I appreciate it appears likely possibility to do so could be restricted to "patches" of QMID data around airfields for most mainstream devs)?

Great questions, I can't answer this for sure until there is more information in the SDK on terrain.

From what I can tell, FS converts to WGS84 internally like a virtual globe. The Bing photogrammetry ships already in WGS84 coordinates, and the Bing DEM is also in relation the the WGS84 ellipsiod reference height.

Unlike FSX which just assumed the WGS84 ellipsoid for ground height, MSFS ships with a egm2008 undulations. Therefore the actual ground elevation will actually vary geographically, and which will catch you by surprise as a delta from the WGS height.

In terms of projecting QMID or raw sources to spherical mercator, that is an interesting point and I haven't wrapped my head around it yet. Maybe there is something in ESRI's tools that can help.

 

[theisomizer]

A quick update: Not an official method but users are already creating community packages to add custom liveries to the aircraft.cfg, see https://github.com/MoritzT/FS2020-Liveries for example.

 

[Timmo]

Great questions, I can't answer this for sure until there is more information in the SDK on terrain.

From what I can tell, FS converts to WGS84 internally like a virtual globe. The Bing photogrammetry ships already in WGS84 coordinates, and the Bing DEM is also in relation the the WGS84 ellipsiod reference height.

Unlike FSX which just assumed the WGS84 ellipsoid for ground height, MSFS ships with a egm2008 undulations. Therefore the actual ground elevation will actually vary geographically, and which will catch you by surprise as a delta from the WGS height.

In terms of projecting QMID or raw sources to spherical mercator, that is an interesting point and I haven't wrapped my head around it yet. Maybe there is something in ESRI's tools that can help.

Will have to wait for the Terrain part of the SDK but, yes, projecting to other vertical coordinate systems is supported in ESRI tools (Toolbox\Projections and Transformations\Raster\Project Raster) and projection to 'WGS 1984 Web Mercator (auxiliary sphere)' is easy too, ensuring you use the correct transformation. QGIS supports this too. For vertical heights, it may just be that the sim just assumes any value (in meters?) is equal, regardless of vertical projection.

What I'm trying to figure out is the relationship of biomes and if these are changeable on the client or if it's all streamed from the server. There's been lots of comments on autogen trees and, in my local area, the definition of tree types for specific areas is incorrect (i.e. plantation pine forest)

 

[Jay Bloomfield]

Has anybody figured out the format for the elevation files located at:

<drive>:\WpSystem\S-1-5-21-1205754744-1098576635-1961144884-1003\AppData\Local\Packages\Microsoft.FlightSimulator_8wekyb3d8bbwe\LocalCache\Packages\Official\OneStore\fs-base-cgl\CGL

The only common file format that uses .CGL as a qualifier is from a fairly old SDK called the Computational Geometry Library (CGL) but that could be a coincidence. It was developed by the same research team that spawned Manifold GIS, the low budget GIS alternative to ArcGIS. The MSFS files themselves may just be encrypted DEM files.

 

[Gheed]

Has anybody figured out the format for the elevation files located at:

<drive>:\WpSystem\S-1-5-21-1205754744-1098576635-1961144884-1003\AppData\Local\Packages\Microsoft.FlightSimulator_8wekyb3d8bbwe\LocalCache\Packages\Official\OneStore\fs-base-cgl\CGL

The only common file format that uses .CGL as a qualifier is from a fairly old SDK called the Computational Geometry Library (CGL) but that could be a coincidence. It was developed by the same research team that spawned Manifold GIS, the low budget GIS alternative to ArcGIS. The MSFS files themselves may just be encrypted DEM files.

I can't help with the elevation files but i was just looking at the SimpleAerial sample in the SDK and the file format for the aerial data seems to be CGL aswell.

 

[theisomizer]

Hi folks, see my Post #2 at the top of this thread.

The CGL files are a container format for raster and vector terrain data in MSFS, it is a new format by Asobo. It is simply a compressed container around the tile data, just like the bgls that were spit out by the FSX resample and shp2vec tool.

I will update my bgldec tool to support these shortly. The container format is simple - it is an LZMA compressed header with pointers to tiles within the file, and then LZMA compressed sections following.

The challenge is figuring out, in the abscence of sdk support, what the format of the underlying files is. The DEM appear to just be raw so that is an easy one, but there are many more formats that are either further compressed or otherwise quantized.

 

[Jay Bloomfield]

it is an LZW compressed header with pointers to tiles within the file, and then LZW compressed sections following

Thanks much.

 

[mendiola_loyola]

Hi folks, see my Post #2 at the top of this thread.

The CGL files are a container format for raster and vector terrain data in MSFS, it is a new format by Asobo. It is simply a compressed container around the tile data, just like the bgls that were spit out by the FSX resample and shp2vec tool.

I will update my bgldec tool to support these shortly. The container format is simple - it is an LZW compressed header with pointers to tiles within the file, and then LZW compressed sections following.

The challenge is figuring out, in the abscence of sdk support, what the format of the underlying files is. The DEM appear to just be raw so that is an easy one, but there are many more formats that are either further compressed or otherwise quantized.

Contratulations for your discoveries.
I would like to make a 10 km2 mesh terrain for a missing Island.
I have the mesh for that area each 15 arc sec.

What is the elevation data resolution in FS2020 15 arc sec?

Regards
Alfredo.

 

[Jolly]

Hey @theisomizer , I wanted to check in on the progress of the .glTF Viewer you are working on. I worked a lot with the .bin and .glTF files of the plane models in the last few days, but I have currently hit a dead end converting the .glTF files back to the structure which is readable by blender and 3dsMax.

 

[L3M0N]

16. Lighting model / rendering / world

Asobo's advanced graphics engine is multithreaded, instanced, and deferred, so this will be a structural change for most people who are coming over from FSX/P3D. The results are obviously very visually stunning. It efficiently works with the GPU (on a modern mid-high end system I am almost always GPU bound) so the performance profile will be different from what you are used to with less emphasis on the main thread. Of course, addon developers will, as always, have to be careful not to starve main thread resources, as remember that is where many of the functions you are used to using will be dispatched.

How you build meshes will most likely evolve. There is no (strong need for) drawcall batching, so you will see moves towards larger texture sheets and less materials where it makes sense. With today's GPU architectures, triangles are fairly cheap and VRAM is at more of a premium. Be efficient in how you lay out your maps and consider adding details to meshes directly.

There are numerous areas on the rendering side that saw 3rd-party modification in P3D/FSX, including base textures, weather and sky visuals, and even shader modifications. All of these will be very difficult to mod in MSFS, for many reasons. For one, other than render to texture techniques in P3D none of this was ever officially supported. Asobo has made it clear that the core of the game is to remain consistent and not be messed with, with security being a concern. The shaders, for example, ship compiled as uber-shaders already in a packaged / archived format. Textures ship in several packages including the compressed .dpc compressed containers as well as various other MSFS core-managed packages such as the bongfish texturesynth and fs base material libs.

With a deferred lighting model things such as dynamic lighting and lighting shadows can be computed much more cheaply than in forward. You see this reflected in the cockpits which come out of the box with dynamic night lighting and shadows and full PBR, which look fantastic, which are areas that have seen a lot of techniques and workarounds from 3rd-party devs in clever but hard to maintain ways. Areas such as the sky, for example, are mostly procedurally generated using precomputed atmospheric scattering and volumetric clouds for truly global illumination (see the paper from Eric Bruneton and Fabrice Neyret from INRIA for the basics), so there is little you would be able to modify there.

In terms of textures, perhaps long-term a technique for overriding these for the curious without destroying the base sim install will be devised.

I'd really like to play around with the shaders and some of the textures that are locked behind the dpc archives. The moon, stars, lens flare, ect and some effects like bloom. So far I've tried using Intel GPA to dump frames but it doesn't work. Neither does 3dmigoto, yet for some reason DXVK (directx to vulkan) works just fine.

Is this game using Oren-Nayar?

 

[theisomizer]

I'm working on a doc that I will put in the wiki of what I know about CGL.

Contratulations for your discoveries.
I would like to make a 10 km2 mesh terrain for a missing Island.
I have the mesh for that area each 15 arc sec.

What is the elevation data resolution in FS2020 15 arc sec?

See the docs on the Bing Maps Tile System here: https://docs.microsoft.com/en-us/bingmaps/articles/bing-maps-tile-system here. You can look at the chart and find the appropriate meters/pixel resolution for 15 arc second, although you may have to do some resampling yourself in a GIS package. You can try out Eugene's tool for building the tiles from this thread: https://www.fsdeveloper.com/forum/threads/aerial-images-airport-background.448431/ . The CGL compiler in the sim won't resample but it will downsample to generate mipmaps for lower LODs.

The bing system goes up to 23 levels but from what I can tell the sim (and CGL files specifically) may only use up to level 20.

Hey @theisomizer , I wanted to check in on the progress of the .glTF Viewer you are working on. I worked a lot with the .bin and .glTF files of the plane models in the last few days, but I have currently hit a dead end converting the .glTF files back to the structure which is readable by blender and 3dsMax.

It will probably be the weekend before I can get it working, day job and whatnot. I have to implement a couple of extensions to get it to load without manual editing of the gltf file.

I'd really like to play around with the shaders and some of the textures that are locked behind the dpc archives. The moon, stars, lens flare, ect and some effects like bloom. So far I've tried using Intel GPA to dump frames but it doesn't work. Neither does 3dmigoto, yet for some reason DXVK (directx to vulkan) works just fine.

Is this game using Oren-Nayar?

Great questions, I would as well. Some of the dpc data is not compressed but I'm not sure how much detail I want to go into here, since the system appears to not be designed to be modifiable and I don't want to encourage this without Asobo's support.

You can check out the preview shaders for the 3ds Max that ship in the SDK for an idea of the PBR lighting model. The diffuse appears to be Lambert and specular is Cook-Torrance.

The rendering is interesting if you manage to get a graphics debugger going, you can also google to get an analysis of the shared engine with A Plague Tale. It's a very advanced graphics engine. Keep in mind that in areas like precomputed scattering some of the logic occurs on the C++ side to generate the textures so modding would not really be feasible.

 

[earthdog]

Thanks for the info!

So WebAssembly targetting from MS/Asobo shows that they are on board the future wagon and also that more is happening under the hood that is web based ?

 

[Gabe777]

Hi all.

OK. So there's more chance that Arianna Grande will knock on my door tonight and demand I take her out for a drink... than me ever understanding a word of this post ,

But, can I ask one simple question that is really annoying me.

Is there a way of improving the size of the shadow maps in the cockpits ?

I find them very Jaggy at 2K Rez

I found in the user.cfg a setting for 2048. When I increased this to 4096 and then 8192, I noticed a small improvement but it killed Framerate as I assume it affected ALL shadows....terrain and everything !

In X plane it was possible to separate settings for cockpit and terrain...which would a good start.

Any ideas ?

Cheers clever people !

 

[L3M0N]

Does anyone know if AirlineICAOCodes.spb and PlaneTypeICAO.spb in installdrive\MicrosoftFlightSimulator\Packages\Official\Steam\fs-base-aircraft-icao actually do anything?

They could be edited without too much difficulty. The only issue is both may need to be the same number of characters for replacement names. Some script would need to be made to pull out the names and compare them to real world airports and aircraft.

 

[Jay Bloomfield]

There are a bunch of model library folders, for example:

asobo-modellib-airport-generic

which includes a file generic.bgl. Does this contain hangars, windsocks, etc. which are placeable in the scenery editor? Or does it contain other objects that show up in airports around the world but are not placeble with the scenery editor? I can't believe there aren't placeable models for VORs, radio towers, radio shacks, etc., but I have yet to find any. I checked a number of airport's and other than a beacon, hangars and windsocks, I don't see any airport specific models.

I guess that my first question is, can references to the various modellibs be added to an airport project and does that expand the available stock objects? Or is the SimObject list in the scenery editor already complete?

My second question is why does the list for tower objects include so many models that overlap with the SimObject list?

EDIT: I think I answered my first question. There are five objects named Gen_Airport_... placeable. in the scenery editor and that appears to be the extent of miscellaneous airport objects.

 

[Jay Bloomfield]

My second question is why does the list for tower objects include so many models that overlap with the SimObject list?

The answer to my second question is that any object can be/host a tower object.

I do want to add a word of caution. Adding taxiway signs can cause the Scenery Editor to CTD, because it initially adds a blank sign and this generates an error in the console. Even when you enter the correct information under properties, the damage is done. The app then becomes unstable and eventually crashes. It's much easier to port the ADE taxiway sign XML code with Notepad++ to the airport's ICAO.xml file. In fact, I'm beginning to wonder, whether the initial beta of ADE for MSFS should just be for creating the ICAO.xml file in correct format for the MSFS Scenery editor to read as part of a project.

I do have a few more points/questions. SimObjects installed by every package show up in the Scenery Editor and hence this poses questions as to what objects are legal to use (other than for personal use) and which aren't. Is there any way of limiting which libraries are are visible to the Scenery Editor (like in the ADE Library Manager)? My alternate question is, are Autogen objects also included in the SimObject list? I can't really tell, but I certainly don't see every house and building that is used in the sim. There are a lot of objects only listed by GUID, but many of these look like LEGO block airport building pieces that might even originally be from FSX.

Annnndd finally, I gave up and made my own VORDME object with Blender. Maybe I will be surprised when a VOR model shows up in the next update.

 

[scruffyduck]

Jay - it will be something like that although I plan that ADE will be able to create a new project/package. I might have some description in the next week or two