Flight Docs: Terrain System

#1
Since there aren't many terrain specific changes between FSX and Flight, I'll just go over the differences here. Terrain is the area I know the least about so some of this is an educated guess.

DEM:
Everything is the same as FSX as far as I know and you can even drop DEM from FSX straight into Flight with no modification.

Land Class:
Flight still uses worldlc.bgl for mapping what areas of the world use a specific land class, however this file is now using a 16 bit per pixel format instead of 8 so more land classes are possible. For now you can use the ContentConverter from the toolkit to convert FSX version to Flight using an undocumented option:
ContentConverter test /i:<input bgl> /o:<output bgl>

Water Class:
This is the same as FSX as far as I know.

LCLookup:
This file has changed in Flight to include the stuff that used to be in terrain.cfg (terrain.cfg is gone now). One of these files is specified for each scenery.cfg generally, and support falling back to another lclookup.bgl. I added support for compiling an xml file into an lclookup bgl for Flight:
SceneryCompiler /i:<Input xml path> /o:<output lclookup bgl path>

You can also convert the FSX lclookup and terrain.cfg to the Flight format using ContentConverter:
ContentConverter lclookup /c:<terrain.cfg path> /i:<input bgl> /o:<output bgl>

Vector:
In Flight we added a bezier spline vector type which is used for the new Vector Scenery system (bridges, elevated highways, marinas, rock walls, edges of wharfs, etc). Other than that the format should be the same. I believe Orion and Brandon have been able to just take vector data from FSX and drop it into Flight, but I'm not sure.

Aerial:
The raster layers in the aerial\photo-real bgls was changed to 32 bit per pixel instead of 16. Support for a detail lookup map was also added (see below for details).

Light Maps:
Light maps were changed from "night" maps in FSX to a more standard version of a light map, which is basically an additive light that is added to the lighting calculation in the shader. They are generally lower res and don't have any specific detail of the surface they are applied to. This allows dynamic lights to light up the terrain at night without getting the grey texture problem in FSX.

Detail textures:
Flight added support for detail textures to the terrain. There is a shared texture atlas that has 256 different slots for tiling detail textures. These are blended into the base terrain texture for the high LODs. To support this, Flight added a detail lookup texture which is an 8 bit per pixel indexed texture that specifies which of the 256 slots to use for each square meter of a terrain texture (both LC and aerial)

Cliff textures:
Flight added the notion of cliff textures which are textures that are applied to near-vertical surfaces instead of stretching the default LC or aerial over it. There are a bunch of settings in lclookup to contorl for each land class which cliff texture is applied, at what angles it is applied and at what scale.

Autogen:
The format of the agn files has changed slightly for Flight. Mostly by removing of features that are no longer supported (row houses are gone, autogen 1 buildings have been repalced with "procedural buildings"). Other than that the agn are the same. The autogen annotation repositories are still functionally the same design as before, but the format have changed quite a bit. Most of the changes are to facilitate sharing functionality across the different autogen types, and support for seasonalization across all autogen and scenery.

The autogen annotation repositories now ultimately reference a material variation instead of an MDL. This is the same for vegetation as well. All vegetation shares the same model which consists of three planes similar to FSX, but with a much improved shader. The model is 1m x 1m x 1m and is scaled to the proper size based on the instance data. The vegetation uses a texture atlas of the various trees and bushes, and the instance data also contains an index of the texture slot to use as well as some other data. The procedural buildings replace all forms of autogen 1\generic building from FSX. I think in the end I combined 3-4 different building rendering systems into a single system with the functionality of all of them (but much better performance). The building geometry is specified in .model files in a specific structure. The building is divided into 5 parts (basement, bottom floor, mid floor(s), top floor, roof). These sections are annotated via the vertex color of the geometry, and scaled to fill a 1m x 1m x 1m cube in the mode. In the shader these sections are rotated, positioned, sized, textured, and colored based on the properties in the instance data. This allows a massive amount of varition in the look of the buildings, but still allows all buildings of a similar shape to be rendered in a single draw call. I can go into more detail of the creation of new autogen assets in an autogen specific post. For most of the scenery stuff you are doing the built in content is probably good enough to start off with.

Scenery Objects \ Procedural Buildings:
This is mostly the same as FSX, although LibraryObjects reference a material variation or seasonal variation id instead of a .mdl. Some types of SceneryObjects have bene removed (effects, windsocks,m beacons). You should be able to accomplish those via regular scenery objects with animations. LIbraryObjects can also point to material variations for autogen vegetation so it is possible to place autogen as scenery, and they get rendered with the rest of the autogen (ie. very fast). There is a new type of palcement for the procedural buildings that functions in the same was as autogen palcements, so they also get rendered very fast. In Flight the placements are stored on the level 15 QMID (1km) in the obx instead of 11 (16km) in order to match the autogen, so OBX from FSX won't work in Flight as is. You can use the SceneryCompiler.exe in the toolkit to convert placement xml to obx bgl. The format of the xml is mostly the same as FSX with some differences:

LibraryObjects:
- Scale must be 1.0 (no scaling in Flight). If the scale is not 1.0 Flight will fail to load the file.
- Name references a .materialVar or .seasonVar

Effect:
- Are not supported at the root level, but may be attached to the attachpoint of a LibraryObject

Windsock:
- Gone. Replaced with animating LibraryObject

Beacon:
- Gone. Replaced with animating LibraryObject

ProceduralBuilding example (addition):
Code:
<SceneryObject instanceId="{62092C37-A390-4418-9DA4-502FFEEC2B98}" lat="35.97114167" lon="-114.85872222" alt="0.00000000F" altitudeIsAgl="TRUE" pitch="0.00000000" bank="0.00000000" heading="0.00000000" imageComplexity="VERY_DENSE" >
<ProceduralBuilding materialVariation="{97B1B8FF-76E0-43BA-91FD-ACC91F0A0E80}" colorVariation="-1" sizeX="50.00000000" sizeZ="50.00000000" sectionHeight0="3.25000000" sectionHeight1="0.00000000" sectionHeight2="3.25000000" sectionHeight3="-3.25000000" textureSlotIndex0="-1" textureSlotIndex1="-1" textureSlotIndex2="-1" textureSlotIndex3="-1" />
</SceneryObject>
 
#2
TextureLookup xml format details

TextureSet:
Texture, Mask, LightMap, Autogen - This is a list of properties that control a set of textures that represent a land class. These values are used to encode the name of the texture file. ex: 005b2su1.bmp from FSX would be: Vulcn 5 (005), region 2 (B), Summer (su), Variation 1. Lightmap is the same naming scheme, but the season tag is replaced with LM, and autogen is the same but the season is replaced with an, and the extension is .agn. I'm not sure how the masks work, but it is the same as FSX as far as I know.
MaskTextureVariation - Not sure what this does exactly, I think it remaps texture variations to other variations. Must be 16 of them.
Season - Allows remapping season to another season so you can say, have all texture map to the summer texture (needs to have a value for all 5 seasons)
CliffTexture\*AtlasSlot - The cliff texture section is a 0-15 (I think? might be 0-3) value that maps to a slot in the CliffAtlas texture. I think some slots may not have a texture in them though.
CliffTexture\MinSlopeIndex - The minimum slope that the cliff texture will show up at (0-15) these values map from 0-90 degrees I believe.
CliffTexture\TextureScaleLevel - This value (0-15) represents the vertical scale of the cliff texture when it is applied to the terrain

LandClassToTextureSet:
Maps an olson land class (I think, this is the value from worldlc.bgl) and region, to a texture set. It has different remapping types but I think most are just the default. I think the other ones are for making the land class map to something different for an airport for example.

WaterClassToTextureSet:
Same as above, but for water.

GradientMap:
This allows mapping different slopes to a different land class. This is not the same as the new CliffTexturing stuff, but is the same as the FSX tech.

VectorShapeProperties:
These map to Vector Texture entries in terrain.cfg from FSX. There can be multiple properties defined per vector type. These will all be rendered on top of each other based on the draw priority. I recommend looking up the docs for FSX on these.


Terrain texture lookup via lclookup:
This is how Flight works as far as I can tell:
Code:
- Select LC from worldlc.bgl or equivalent for the area (I believe this is called the Olson, there are special values between 252 and 254 used internally for aerial I think, but the max is 65535).
- Select region from regions.bgl or equivalent for the area.
- For a given ~1km x ~1km area on earth figure out what LCs exist (this is a combination of adjacent LCs from worldlc.bgl, gradient remapping, vector remapping, masking, etc)
- For each of these LCs:
  - It uses the current scenery layer to find the associated lclookup.bgl
  - Using the selected values above, it creates a set of keys to use to look up texture sets.  It looks them up in this order:
    - Requested Region, Requested RemapType, Class
    - Requested Region, Default RemapType, Class
    - Default region, Requested RemapType, Class
    - Default region, Default RemapType, Class
  - If it is one of the custom Olson (between 252, and 254) it does some special stuff, otherwise:
    - Look up a texture set from the keys above in the associated lclookup.bgl.  If a texture set isn't found it will then go through all the fallback lclookup.bgls as specified by the associated one to try to find it.
    - Select season from seasons.bgl or equivalent for the area.
    - Remap the season using the season remapping from the texture set
    - For the diffuse texture set the texture name to <texture vulcn><texture region>2<season><texture variation>.dds  (ex 029b2su0.dds)
    - For the detail lookup texture set the texture name to <texture vulcn><texture region>2<season><texture variation>DL.dds  (ex 029b2su0DL.dds)
    - For the mask texture set the texture name to <mask vulcn><mask region>2M<mask variation>.dds  (ex 029b2M0.dds)
    - For the lightmap texture set the texture name to <light vulcn><light region>2LM<light variation>.dds  (ex 029b2LM0.dds)
    - For the autogen annotations set the texture name to <autogen vulcn><autogen region>2an<autogen variation>.dds  (ex 029b2an0.agn)
There is a separate lookup for cliff textures, but is mostly similar to the steps above other than the keys it uses to do the lookup only use the Default RemapType.


Practical limits that I am aware of:
Number of TextureSets in an lclookup: Lots (only limited by 32 bit uint and file size)
Number of LandClassToTextureSet in an lclookup: Lots (only limited by 32 bit uint and file size)
Number of WaterClassToTextureSet in an lclookup: Lots (only limited by 32 bit uint and file size)
Number of Olsons: 65536 (index 252-255 are special, however if we are using my tool to convert from FSX, this is limited to 255 since that is the limit in FSX. If we create a new version of resample we could use the whole uint16).
Number of VULCNs: 1000 (000 - 999, due to only using 3 characters in the filename).


Also, the ESP SDK docs are a good resource. It lists the regions, Olsons, and texture name format:
http://msdn.microsoft.com/en-us/library/cc707102.aspx

There were some new Olsons created for Flight, not sure if all of them are used (especially the later ones as those are probably for the Grand Canyon that didn't get released):
148,Non Mountain Rock,LAND_CLASS_NON_MOUNTAIN_ROCK,
149,Cool Dry Conifer Forest,LAND_CLASS_COOL_DRY_CONIFER_FOREST,
150,Cool Dry Conifer Forest Meadow,LAND_CLASS_COOL_DRY_CONIFER_FOREST_MEADOW,
151,Tan Sand Beach (Wet),LAND_CLASS_TAN_SAND_BEACH_WET,
152,Black Sand Beach (Wet),LAND_CLASS_BLACK_SAND_BEACH_WET,
154,Lava Young,LAND_CLASS_LAVA_YOUNG,
155,Lava Old,LAND_CLASS_LAVA_OLD,
156,Lava Oldest,LAND_CLASS_LAVA_OLDEST,
159,Beach Gray Sand,LAND_CLASS_BEACH_GRAY_SAND,
160,Beach Gray Sand Wet,LAND_CLASS_BEACH_GRAY_SAND_WET,
163,Warehouse District,LAND_CLASS_WAREHOUSE_DISTRICT,
166,Stream Densely Wooded Banks,STREAM_DENSELY_WOODED_BANKS,
167,Stream Lightly Wooded Banks,STREAM_LIGHTLY_WOODED_BANKS,
170,Tall Grasses and Shrubs Scant,LAND_CLASS_TALL_GRASSES_AND_SHRUBS_SCANT,
171,Dry Tropical Forest Scant,LAND_CLASS_DRY_TROPICAL_FOREST_SCANT,
174,Conifer Forest with Clear Cuts,LAND_CLASS_CONIFER_FOREST_WITH_CLEAR_CUTS,
175,Conifer Forest and Meadows,LAND_CLASS_CONIFER_FOREST_AND_MEADOWS,
176,Taiga Sparse,LAND_CLASS_TAIGA_SPARSE,
178,Shoreline Rocky,LAND_CLASS_SHORELINE_ROCKY,
179,Tundra Wetland,LAND_CLASS_TUNDRA_WETLAND,
180,Glacier,LAND_CLASS_GLACIER,
181,Rural Village in Conifer Forest,LAND_CLASS_RURAL_VILLAGE_IN_CONIFER_FOREST,
182,Rural Village In Tundra,LAND_CLASS_RURAL_VILLAGE_IN_TUNDRA,
183,Large City Midrise,LAND_CLASS_LARGE_CITY_MIDRISE,
184,Mixed Forest Aiport Background,LAND_CLASS_MIXED_FOREST_AIRPORT_BACKGROUND,
185,Semi Desert Shrub Stream,STREAM_SEMI_DESERT_SHRUB,
186,Geometric Crop Stream,STREAM_GEOMETRIC_CROP,
187,Grassland Strean,STREAM_GRASSLAND,
188,Geometric Crop and Town Stream,STREAM_GEOMETRIC_CROP_AND_TOWN,
189,Grass and Shrub Stream,STREAM_GRASS_AND_SHRUB,
190,Grass Crop Shrub Stream,STREAM_GRASS_CROP_SHRUB,
191,Semi Desert Sage Stream,STREAM_SEMI_DESERT_SAGE,
192,Hot and Mild Grasses Stream,STREAM_HOT_AND_MILD_GRASSES,
193,Low Sparse Grassland Stream,STREAM_LOW_SPARSE_GRASSLAND,
194,Park Stream,STREAM_PARK,
195,Golf Course Stream,STREAM_GOLF_COURSE,
196,Shrub Deciduous Stream,STREAM_SHRUB_DECIDUOUS,
197,Gravel,LAND_CLASS_GRAVEL,
198,Ice without Cliffs,LAND_CLASS_ICE_WITHOUT_CLIFFS,
200,Great Basin Desert Scrub Medium R,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_MEDIUM_R,
202,Great Basin Desert Scrub Medium P,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_MEDIUM_P,
204,Great Basin Desert Scrub Sparse R,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_SPARSE_R,
206,Great Basin Desert Scrub Sparse P,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_SPARSE_P,
207,Great Basin Desert Scrub Sparse G,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_SPARSE_G,
208,Great Basin Desert Scrub Dense R,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_DENSE_R,
212,Great Basin Desert Scrub Rocky Outcrop R,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_ROCKY_OUTCROP_R,
214,Great Basin Desert Scrub Upland P,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_UPLAND_P,
215,Great Basin Desert Scrub Upland G,LAND_CLASS_GREAT_BASIN_DESERT_SCRUB_UPLAND_G,
222,Plains Grassland R,LAND_CLASS_PLAINS_GRASSLAND_R,
223,Plains Grassland P,LAND_CLASS_PLAINS_GRASSLAND_P,
228,Pinyon Juniper Woodland Dense R,LAND_CLASS_PINYON_JUNIPER_WOODLAND_DENSE_R,
229,Pinyon Juniper Woodland Dense P,LAND_CLASS_PINYON_JUNIPER_WOODLAND_DENSE_P,
231,Pinyon Juniper Woodland Medium R,LAND_CLASS_PINYON_JUNIPER_WOODLAND_MEDIUM_R,
232,Pinyon Juniper Woodland Medium P,LAND_CLASS_PINYON_JUNIPER_WOODLAND_MEDIUM_P,
233,Pinyon Juniper Woodland Medium G,LAND_CLASS_PINYON_JUNIPER_WOODLAND_MEDIUM_G,
234,Pinyon Juniper Woodland Sparse R,LAND_CLASS_PINYON_JUNIPER_WOODLAND_SPARSE_R,
235,Pinyon Juniper Woodland Sparse P,LAND_CLASS_PINYON_JUNIPER_WOODLAND_SPARSE_P,
237,Ponderosa Pine Forest,LAND_CLASS_PONDEROSA_PINE_FOREST,
238,Spruce and Douglas Fir Mixed Dry Forest,LAND_CLASS_SPRUCE_AND_DOUGLASS_FIR_MIXED_DRY_FOREST,
239,Pinyon Juniper Slope Dense R,LAND_CLASS_PINYON_JUNIPER_SLOPE_DENSE_R,
240,Pinyon Juniper Slope Dense P,LAND_CLASS_PINYON_JUNIPER_SLOPE_DENSE_P,
242,Pinyon Juniper Slope Medium R,LAND_CLASS_PINYON_JUNIPER_SLOPE_MEDIUM_R,
244,Pinyon Juniper Slope Medium G,LAND_CLASS_PINYON_JUNIPER_SLOPE_MEDIUM_G,
245,Pinyon Juniper Slope Sparse R,LAND_CLASS_PINYON_JUNIPER_SLOPE_SPARSE_R,
298,Sandy River Bottom,LAND_CLASS_SANDY_RIVER_BOTTOM,
299,Underwater Rocks,LAND_CLASS_UNDERWATER_ROCKS,
 
#3
Autogen Annotations:

AutogenCompiler.exe can convert SimProp xml to agn files for Flight. The schema for the xml files is defined in propannotations.xml that ships with Flight (In CorePropDefs.pak). The RowHouse type is not supported, and PolylineBuildingGroup is supported but I don't know if there are any assets for it that we shipped, so I would avoid them for now.

BuildingRoof:
ProceduralBuilding. I think this was autogen 1\generic buildings in FSX. The id references a MaterialVariation for any procedural building(generally located in SceneryObjects\Procedural\Buildings), or an autogen group that eventually filters down to a materialVar for a procedural building (see Repositories\Autogen\BuildingDescriptions.spb). All procedural building materialVars use a particular instance data format. This is a requirement for something to be placed with this system. These are used for all houses, mid-rise, high-rise, etc.

Autogen2Rect:
Modelled building\scenery. I think these are also called autogen 2 in FSX? The id references a MaterialVariation for any scenery object, a SeasonalVariation for a scenery object, or an autogen group that filters down to a scenery object (see Repositories\Autogen\Autogen2Descriptions.spb). These are generally used for one-off things, like tractors, haystacks, grain towers, gas stations, religious buildings, etc. There are perf concerns of using these, we generally recommended no more than 10-15 per square km of these.

Autogen3Rect:
Vegetation rectangle. Ids reference a MaterialVariation for the tree model (generally located in SceneryObjects\Procedural\Vegetation path), or (in most cases) an autogen group that eventually filters down to a materialVar for the vegetation. All vegetation materialVars use a particular instance data format. Authoring new materialVars requires some care to make sure it doesn't break instancing.

PolygonRegion:
Vegetation n-sideds polygon defined by a list of points. These use the same types of ids as Autogen3Rect. (See Repositories\Autogen\VegetationDescriptions.spb for the groups)

As for the agn format itself, I believe it is the exact same format as FSX, but the version was upped from 9.2 to 9.3, and certain types of data are not allowed (anything other than the 4 types above)
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#4
Regarding the WorldUI, landclasses, vectors, airports... here are some pics. This is FSX stuff converted or dropped into Flight. We still need an lclookup to cover all the FSX content (missing textures, rail lines and perhaps others not showing, etc.). DEM and vectors are not converted but are from FSX. worldlc is converted, worldwc is not converted, nor seasons, nor regions.

DelavanMap.png


Approaching Delavan, Wisconsin:

ApproacingDelavan.png


The C59 APX file is converted... some elements are missing or substituted:

C59.png


We need to fix some missing issues, and we will need to standardize the manner in which we add the FSX-based content. But fleshing in the world is possible now with Steve's tools.

Winter scene:

C59_Winter.png


Dick
 
Last edited:
#5
Great Work Dick!

As always I have a few questions to be asked about this. So to reduce the number of questions, could you please export that add-on and post it here?
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#6
I think we need to wait for Steve to coordinate the different addons. It shouldn't be long before he's got an FSX to Flight conversion (perhaps a few weeks).

The reason for this is that we need to set a standardized way to add the FSX content, or we'll end up with 50 different versions, and we will then have a mess. The goal is a single method to convert the FSX content as a standard set of addons. Then we can start adding original content via different developers.

I used the screenshots just to illustrate that Steve's toolkit is very close to adding a whole world set to Flight. I can't really distribute the addon content, and still maintain the single-set goal.

Dick
 
#7
Understood. That would be a much simpler and more organized way of going about things.
One factor that needs to be refurbished are the shorelines. Would greatly improve the scenery...

And maybe the runway sufaces...

I've looked at my attempt and as you may know, I only got the Airports working from converting the APX*.bgl's but there is no UI map. I've also dragged and dropped the CVX*.bgl's and the DEM.

So, Dick what files did you convert exactly?
 
Last edited:

rhumbaflappy

Moderator
Staff member
Resource contributor
#8
APX files are converted. worldlc.bgl is converted. The dem and cvx files are drag'n'dropped, as are the worldwc.bgl, region.bgl, and seasons.bgl. Steve had a WorldUI addon.

The worldlc places the landclasses, but we need a lclookup.bgl for the FSX texture assignments, as well as the textures that are missing from Flight. I haven't gotten that to work yet, but as Steve is dealing with it, I won't need to worry about it just yet. I think Brandon may have gotten that to work as well.

Dick
 
#9
Thanks Dick. That clears things up, but one problem I'm having is converting the worldlc.bgl file through the SceneryCompiler. I get this:

Warning: Ignoring unsupported BGL data type: 110

I haven't used FSX in over two years... I'll try again with a fresh install.

I believe that the problem is due to the Olson Land Class 110: Large City Suburban Grid Dry
 
Last edited:
#12
@stonelance thanks for this, it has been an invaluable resource. I have a question you may have an answer to.

Myself and some other developers have put together some pretty extensive documentation regarding the format for raster bgl data: https://www.fsdeveloper.com/wiki/index.php?title=BGL_File_Format#TERRAIN_SECTIONS
'
In MS Flight, the TRQ1 record has been replaced by a TRQ2 record for some of these types, notably imagery.

The format of a TRQ2 is very close to a TRQ1, with the addition of two DWORDS at the end. Past that, the rest of the data is the same. For the life of me, I cannot decode what these two fields are, and was curious if you had any information on that. My best guess is it is some sort of slope/angle or placement information. Assuming I already have the ability to read and write 32 bit compressed images in the various compression type formats, if we can decode these fields I can, using my tools, attempt to move imagery between FSX+ and Flight.

Any insight greatly appreciated!

Example TRQ2 record:
Code:
54 52 51 32 <- TRQ2 signature
30 00 00 00 <- Size (+ 2 DWORDS)
0B 00       <- Parent Section type
0A          <- Raster compression type
09          <- Mask compression type
39 3C 7A 09 <- TPACKEDQMID A
08 00 00 00 <- TPACKEDQMID B
30 00 00 00 <- Month mask
00 01 00 00 <- NROWS
00 01 00 00 <- NCOLS
EB 53 00 00 <- Raster chunk num bytes
16 0A 00 00 <- Mask chunk num bytes
BA 02 00 00 <- ?? What is this field? Often is 0, or a small number.
07 73 4E 00 <- ???? What is this field? It is often duplicated across sections, and is usually close to the value shown here, or some permutation
 
Top