Problem with an FSX library object

[EdrickV]

Hadn't thought about ModelConverterX, since the intended purpose of that is to convert macros and other old style scenery files into FSX formats. One of the problems with using that with library objects is that you don't have the individual mdl files, they are all part of airport_objects.bgl.

I managed to extract just the air_staticplanes01.mdl file using a hex editor and was able to open it in ModelConverterX. After exporting it, I hand edited the .x file to change the name/GUID and tried in in FlightSim and curiously enough the problem seems to be fixed. Curiously enough, the textures don't seem to have the night texture problem when using the default 16x16 black square textures. Only thing I can conclude is that the original MDL file was created with a buggy tool. Edit: (It's also possible that the resulting model will not use night textures. I don't know for sure.)

The problems about fixing the files this way are:
1. Far as I know there's no utilities for FSX to extract mdl files from bgl files so you have to do it manually. (Not too hard if you know how and have the right tools, but it takes a little time.)
2. ADE (my preferred scenery design tool) won't import a library using the exported MDL file unless the GUID is changed. (Changing the name may or may not be needed but is a good idea anyways.)
3. This won't fix existing sceneries since the GUID will be different, anyone wanting to use the fixed static planes would have to edit their scenery source files to change the object to the fixed one and recompile.
4. I'm not sure what people (or Microsoft) would think about someone distributing what is essentially a library made out of slightly modified stock objects. (Though exactly how they're different, other then name/GUID, I don't know.)
5. A library with these planes would either have to be distributed with the scenery or used as a seperate scenery like EZScenery libraries. (The library should be pretty small though, even with the stock textures. I would guess that, unzipped, the library with all 4 models + textures would probably under a meg in size.)

 

[GaryGB]

Hi Edrick:

Glad to see that helped the aircraft visibility issue !


I've wondered now and then if there was not an ability in the FSX SDK graphics methodology to re-use the same daytime texture and simply apply an alpha channel (ex: via those seemingly "black" 16x16 pixel bitmap "night textures") to control the 'brightness' of the image, so it could be used to provide a night texture instead of another *_lm.dds (darkened) copy of the daytime texture... to reduce distribution and installation footprint size etc.


Also, IIRC, it seems that some FSX building objects (ex: generic buildings) have mapped a wall texture onto their own 3-D model from a PORTION of another object's full-size wall texture.

I'm wondering if the use of another small (ex: 16x16 pixel) texture with an alpha channel could be (-or was actually intended by ACES to be-) used to control the brightness of that type of "mapped portion" of a texture to provide the "night texture" effect for such "borrowed" parts of another objects texture bitmaps without having to create another version of the daytime texture ?

Since IIUC the newer FSX model format seems to exert greater degrees of control over the way materials / textures are ultimately displayed at run time than was done in GMAX for the FS9 3-D model format, one might wonder just how much can be done with "shared" textures !


I'm curious how much of the "fix" for the aircraft models in your tests that apparently was imparted by ModelConverterX, may have involved resetting the way materials are mapped and displayed in FSX due to such a mechanism as I speculate about above ?


As to decompilation of object libraries and XML placement files, in my experience thus far, the recent BGL2XML GUI decompiler version 0.92x and the one in ADE9X from Scruffyduck seem to be the most up to date for use, along with Alessandro Antonini's BGLXML 1.80 (and now BGLXML 2.1 at):

http://www.fs2000.org/index.php/downloads/doc_details/9077--fsx-bglxml-21



BTW: Speaking of ADE9X, an airport BGL I created yesterday (containing an FSX autogen exclude) with Flight1's AFX updated version 1.08 from Konstantin Kukushkin could NOT be decompiled by either of the above utilities.

I had to import the airport into ADE9X, and then decompile the resulting output version created by ADE9X to get at the XML; I subsequently found that BGLXML 2.1 had also written out a MDL file for the exclude contained in the original AFX airport file!


Instant Scenery had at one time been able to use an object library which provided scalable and place-able "exclusion objects" (created as a freebie by John Dutton).

http://www.simforums.com/forums/forum_posts.asp?TID=22390


It seems this "exclusion object" approach has been implemented now in AFX; and IIUC, the previous semi-proprietary BGL created by AFX's "non-BGLComp" compiler/decompiler by author Konstantin Kukushkin now creates an even more atypical airport BGL by allowing storage of a MDL file !


GaryGB

 

[EdrickV]

Those 16x16 black textures don't have an alpha channel and do not appear to be used as an alpha channel on the modified bgl. For a lot of lighting effects trying to do the lighting by just using an alpha channel wouldn't cut it, an alpha channel alone can't do multiple colors.

As far as generic buildings, they are very configurable. ADE doesn't support creating them yet, but basically when creating a generic building you can choose what textures to use for different sections of the building, you can scale the texture in different dimensions, and much more. The BGLComp SDK has info on generic buildings, but here's an example of a generic building from LLBG:
{GenericBuilding
scale="1"
bottomTexture="126"
roofTexture="35"
topTexture="0"
windowTexture="107"}
{RectangularBuilding
roofType="FLAT"
sizeX="70"
sizeZ="64"
sizeBottomY="20"
textureIndexBottomX="256"
textureIndexBottomZ="256"
sizeWindowY="10"
textureIndexWindowX="256"
textureIndexWindowY="256"
textureIndexWindowZ="256"
sizeTopY="0"
textureIndexTopX="0"
textureIndexTopZ="0"
textureIndexRoofX="896"
textureIndexRoofZ="819"/}
{/GenericBuilding}

The {} were originally greater than/lesser than symbols, but I figured those probably wouldn't show up here so switched them for the braces. The textures for a generic building are specified using index numbers, which means you can't use custom textures without replacing an existing texture. That would affect multiple buildings. Generic buildings aren't modeled the the way library objects are so the texture mapping is basically decided by the building creator. (You could, for example, use only part of a texture by scaling the texture enough that it would be bigger then the building. But this still isn't like regular texture mapping on an MDL file. They are all created from basic geometric shapes, and can be any of the following: RectangularBuilding, PyramidalBuilding, and MultiSidedBuilding.)

FYI, if you can import an airport into ADE then you can get XML source for an airport by checking "save XML after Compile" in the settings. (I started doing that a while ago so that I could manually edit the resulting XML code to get an added airport beacon working since ADE doesn't yet support object attachment.)

Edit: Oh, and I hadn't thought of using BGLXML, my version was rather old (hadn't seen a link to a newer version before) and the last time I tried using it, it didn't really work. (I've yet to find a decompiler that can handle CVX BGL files.)

Edit2: Almost forgot, I think the "fix" actually comes from ModelConverterX not being able to fully understand and use the MDL file data. I tried a binary compare of a decompiled MDL and one imported and then exported from ModelConverterX and there were a ton of differences.

 

[GaryGB]

Hi Edrick:

Thanks again for those informative insights and caveats !


I'll certainly be welcoming the convenience of that ADE9X "save XML after Compile" option in the future.


BTW: There was also something interesting (and exciting !) about the way I initially processed the AFX airport BGL via ADE9X.

1.) I was able to open the AFX version 1.08 format airport BGL and import it including the "exclusion object" to ADE9X (...even if was initially hidden)

2.) I was able to save the entire ADE9X airport (after imported) into a generic FSX SDK BGLComp compatible BGL

3.) I was then able to decompile that ADE9X airport BGL, and have BGLXML 2.1 recover and write out that hidden "exclusion object" into a MDL file !


This bodes well for having a way to always get access to our airport code when needed, and when/if authors implement proprietary methodology.


Many thanks to Scruffyduck et. al. for their ongoing efforts to stay up to date ...and to innovate !



Regarding the ability to decompile the FSX CXV Vector BGL... < Uh-Oh: I feel a "RANT" coming on ! >

[RANT_ON]

The FS community definitely does need a decompiler, or an "Append" function (for FSX vector line and poly content at the very least !) ...such as SBuilder has in the FS9 version for LWM BGLs.


I understand the agenda that MS may have had for future content rollouts and/or "MS-Virtual Earth Sim" directions I speculate might have been considered for the FS franchise, and that may have (in their reasoning) necessitated encryption in addition to SHP2VECs very efficient and minimally-lossy compression that exceeded even the original promise of wavelet methodology.

And I understand that some FS Developers may have trepidation over what would happen to the licensing costs for accessibility to imagery and other GIS data if licensors find out that the MS encryption has been "cracked" for content packaged inside the new FS SDK CVX Vector BGL format.


Jim Keir years ago had deciphered the FS2Kx terrain mesh BGL "encoded compression", and reportedly was asked by Microsoft to not distribute decoded content; he apparently opted as well not to educate the FS community on how to do that decoding.

Christian Buchner of TileProxy reknown apparently in 2006 comprehended the "concept" of how such a FS SDK CVX Vector BGL format decryption might be done, but has not to my knowledge posted any further indications of having undertaken such a task.


It seems to me we already have an established precedence of tolerance or even acceptance by Microsoft that the FS community routinely accesses their copyrighted content in FS and re-distributes it as backups in packages that swaps it out for 3rd party replacements, substitutes/fixes textures on MDL files, fixes inaccurate LWM and VTP vectors from FS9 (by in some cases using vector data points for same as "primitives" for the corrected/modified/Appended replacement BGL).


And it is clear that online imagery server operators may have reached an agreement to tolerate or even accept their licensed image tiles being captured from software, to be used routinely as "background" maps to trace over for creating what some might argue is essentially a form of "derived" content in commercial GIS packages, and FS scenery utilities.

Online imagery server operators may not be so happy to find out that some freeware FS Developers are using those tiles "blended" (in some cases altered beyond clear recognition of the original) with default or 3rd party textures to produce distributed scenery packages.


But I do think that we as a FS Community have reached a point that we should have access to the contents of the FSX CVX Vector BGLs.


If that is resisted by Microsoft (and/or some payware FS Developers !), then I daresay we at least need to get access to the coordinates readout in TMF Viewer via some software "hook" methodology, so we can use that utility more productively in conjunction with the FSX SDK (... and a very special screen capture utility, perhaps ? )

Why do I say this ?

As the FS Developer Community endeavors to stay alive, keep moving forward productively, and fix certain FSX scenery code, this sort of thing may well become... "The way of the future."

So... "Show me all the blueprints. Show me all the blueprints. Show me all the blueprints... show me all the blueprints... show me all the blueprints... show me all the blueprints..."

-Howard Hughes: The Aviator

[RANT_OFF]

GaryGB

 

[EdrickV]

Hmm, didn't know those files were encrypted. Now I can understand why we don't have a decompiler for them.

 

[GaryGB]

I'm not yet giving up hope we'll eventually find a way to get into those CVX Vector BGLs !


BTW: I forgot to mention that the BGLXML 2.1 decompiler reportedly gives access to more objects in obscure FSX BGLs than prior decompilers supposedly can even identify, much less export *.MDLs from !


But it is apparent that the scenery objects list in FSX-KML and ADE9X seems to have a more complete inventory than I've seen depicted on Lamont Clark's website of thumbnails.


GaryGB


UPDATE:

CVX Vector BGL decompilation solved ...see Patrick Germain's CVXExtractor utility:

http://www.fsdeveloper.com/forum/threads/cvxextractor-exporting-vector-data.432918/

 

[arno]

1. Far as I know there's no utilities for FSX to extract mdl files from bgl files so you have to do it manually. (Not too hard if you know how and have the right tools, but it takes a little time.)

ModelConverterX can read in the library BGL directly and you can then browse through all the objects in there. If you want you can then export them to the format you want.

I should add that if your original MDL had very special features they might be lost, but for most scenery files it should work fine.

 

[EdrickV]

ModelConverterX can read in the library BGL directly and you can then browse through all the objects in there. If you want you can then export them to the format you want.

I should add that if your original MDL had very special features they might be lost, but for most scenery files it should work fine.

ModelConverterX does not do as good a job at exporting an MDL from a BGL as bglxml does. I've got 2 copies of the same file. One saved by bglxml, one by ModelConverterX, both from airport_objects.bgl. The one saved by ModelConverterX is very different, and in fact is not even the same size (95.8k vs 258k.) Even parts of the header are different, as I can tell by doing a binary compare. Another thing I can see that is different is the one saved by ModelConverterX is missing the nightmask texture references.

I actually tried rebuilding airport_objects.bgl using the files created by bglxml and the only difference I believe is part of a header, the original file had it all zeroes while the BGLComp compiled one has some stuff there. (I'm not familliar enough to know what that data means, but it's only 8 bytes near the very beginning of the file.)

 

[arno]

Hi,

True, I never said it would extract the MDL as it was. It reads the object into its internal representation and then exports it again. But for scenery objects the difference should not be so big.

Which object where you testing? Then I can see what is going on. Also, did you use the latest development release or the 1.0 release? I thought the night maps were already supported.

 

[EdrickV]

I used 1.0. I just tried the dev version and it did do the night texture references, though the case didn't match. (Not that it matters for Windows.) There's still a ton of differences though, the file is only 96.1kb so that's a difference of 161.9kb in file size.

The one thing I do know for sure is the 1.0 version altered the file enough to get rid of the problems I had with it. That might have been because it stripped out the night texture stuff though. The file I worked on is air_staticplanes01.mdl from airport_objects.bgl. It's a stock library object that had a problem where it became invisible at night. The imported/exported mdl file (via 1.0) did not have the same issue, but also doesn't seem to use night textures at all.

Have not tested the file created by the dev version in FSX.

 

[arno]

Hi,

I indeed see the object becomes a bit smaller, but here it is only a bit (from 97.9 kB to 96.1 kB). It seems that on export there are less texture vertices, so it might be that I have optimized the output a bit more (I need to test if that has negative influences on how the models look ).

I see the night textures assigned both in the original and the tweaked MDL file, but I did not test in FSX if there are problems with them.

 

[EdrickV]

The two 258k files I have came from 1. Extracting the file manually via hex editor and 2. Saved using bglxml 2.1. And as I mentioned earlier, the files saved by bglxml, when compiled again using the SDK, make an object library almost identical to the original. The only difference being part of the header that in the original file was zeroed out. (Probably some sort of time stamp, I just tried recompiling those same files into another copy of the BGL and that one section of code is different from the previous one.)

 

[GaryGB]

Hi:

I'm wondering if Arno's efforts to optimize output MDLs in ModelConverterX might be better illustrated and appreciated when comparing such MDLs with those output from Alessandro Antonini's BGLXML 2.1 (or via the Hex decompilation Edrick refers to) by analysis of results with Arnos' other tool "DrawCallMonitor":


http://www.fsdeveloper.com/wiki/index.php?title=DrawCallMonitor

http://www.fsdeveloper.com/forum/showthread.php?t=8967



I'm also wondering what output MDL size and complexity difference one might see when using ModelConvertX at default output settings (versus manually adjusting the Level of Detail Creator feature); if all the methods above yield a comparable number of LODs inside a complex output MDL, doesn't the optimization process become even more important to resulting FSX performance ?

And even if a MDL "performs" quantitatively better, does one lose any visual components or functionality in a MDL by either of the 3 MDL output methods described above ?


A very interesting discussion which I hope will lead to even more new insights !

GaryGB

 

[EdrickV]

Only difference is the number of texture vertices. The one saved by Model Converter X development release has fewer texture vertices. When it comes to preformance, IMHO these particular objects wouldn't have half the impact that a single AI aircraft model would have.

 

[arno]

Hi Gary,

Decompiler like BGLXML or BGLAnalyze do not change the MDL file, they just extract the information as is from the library BGL. So you see what MS put in.

But ModelConverterX reads the objects into its own internal object representation. If you then save again to MDL format you might get different results. For example animations that do not use the timer to drive them might be lost. Other parts like materials, LODs should not be changed. Although the output will not be the same as before.

Another thing to consider is that with the FSX updates (don't remember if it was SP1 or SP2) they optimized the MDL output from XtoMDL, resulting in smaller MDL files.

 

[EdrickV]

All of my experiments would be using SDK SP1a. I don't have SP2 because it causes too many issues with older planes I use that don't yet have (and in one particular case will not have) an FSX model.

 

[GaryGB]

Hi Edrick:

I'm wondering what you might think of the obscure MDL file created by Airport Facilitator X (AFX), stored inside the compiled airport BGL that program creates, which seems able to exclude objects and autogen... under discussion in this thread:

http://www.fsdeveloper.com/forum/showthread.php?t=17916


Thanks for any insights you might offer there as well !

GaryGB