Announcing some new FS Tools

Removed old application information. Sorry, these products never saw the light of day.

some unpleasant setback has kept me away for some days now, but I was just going backwards anyways with my "visualization"
Now it looks I have at least something to get lost in again, and when and if I surface I might just learn something. In the name of all forumers (even the ones that never post their trials and errors) I would like to dish out a HUGE THANK YOU mate for still holding the flashlight.

Zip

Hey Zip,

Glad I could help. If you have any questions about the code (I know it is a bit disorganized ) just ask.

Also let me know if you find any bugs. It seems pretty stable to me, but you never know what other people can do to break one's software .

Sean

Great work!

My new tool - FSREPAINT 2.0 (for FSX) is almost finished (see picture). I would like to share the information I've collected about the mdl format with you but it seems that Sean already discovered everything

Best Regards,
Jorge Santoro.
Belo Horizonte - Brasil.

The thing I don't understand is, in this "new era of openness" with the FS team, why is it that we STILL have to resort to essentially hacking and back-dooring our way into the MDL format (and BGL, for that matter)?

Jorge Santoro said:

My new tool - FSREPAINT 2.0 (for FSX) is almost finished (see picture). I would like to share the information I've collected about the mdl format with you but it seems that Sean already discovered everything

Click to expand...

Nice to see FSRepaint is getting an update for FSX. I am always happy to see more and more programs finally getting updated to be compatible with some of the radical changes in FSX. I have always just done aircraft for payware in the past, however now with FSX I am seriously considering developing payware tools as there is just so much new ground that needs to be covered.

Anyways, I encourage you to share anything you know about MDL as my knowledge is still a work in progress.

Mace said:

The thing I don't understand is, in this "new era of openness" with the FS team, why is it that we STILL have to resort to essentially hacking and back-dooring our way into the MDL format (and BGL, for that matter)?

Click to expand...

Yes, here we have reached a very sad and inconvenient realization. Although ACES promised a more open, robust relationship with the development community, we have yet to see that really come into fruition. Honestly, I'm sure the folks at ACES meant well, but they just do not understand how difficult they are making it for us to develop on the FS platform.

These days I spend more time hacking FS than I do actually developing. And it is sad, because I am a developer, not a hacker. But since they chose to make FS so closed and proprietary I have no choice but to "do things the dirty way". If they had simply published the MDL specification I wouldn't have had to waste the past few months hacking the MDL, and I could have spent my time doing something more productive, like developing a piece of payware.

My frustration levels are slowly rising. This post is not meant to be some "anti-M$ rant", and I don't want it to come across that way. However, because of some of ACES business decisions, honest developers like us are forced into doing some dishonest things just to be able to make a living. I was full of hope after hearing ACES promise to become more community-friendly, but I see them slipping back into their old habits.

I sincerely hope that ACES will eventually become more open in the future. It might just save me from premature balding .

Sean

One problem - certainly with scenery files (dunno about mdl) is that MS do not own copyright to the Bgl Format or some parts thereof. Much of what they use belongs in some way or other to third parties and I assume that makes it difficult to share some sorts of information.

I think ACES are being much more open and helpful with us now but within guidelines that are clearly set at a different level.

Mace said:

The thing I don't understand is, in this "new era of openness" with the FS team, why is it that we STILL have to resort to essentially hacking and back-dooring our way into the MDL format (and BGL, for that matter)?

Click to expand...

Despite not being "open", FS file formats are not "closed" too.
ACES did not set up any kind of protection to prevent people from discovering the secrets of their formats. In most cases, data format are just bare binary representation of the source data, in the sense that data is not extensively transformed by compilation or it does not contain black magic.
That makes the decompilation process "theorically" possible since very few information was lost. The format hacking is greatly facilited when you have a cooperative compiler (like BGLcomp or XtoMdl); which let you generate plenty of different output. That's the principle of an "automated" Rosetta Stone,
Also, one of the most important thing is to have a clear idea of what the format can do and what it can't do. For instance if you have no idea of the usage of "XZP" files in the Uires directory and you want to guess their format, it will take a lot of time just to figure out they're used to archive multiple resources.

Finally I think that's a fair deal, especially when some files contains copyrighted content from third-parties, as scruffyduck said.

The most notable exception are resampled and vector
BGLs; which are compressed. The result is surprising because an homebrew compression implementation is nearly as secure as an encryption algorithm to
protect content But the goal is different; compression is only here to optimize storage of information whereas encrypting means you really want to restrict the spreading of knowledge.

We are coming to a sad realization...Cancel or Allow?
I think marketing somin' the right way was more important to M$ than keeping developers trully happy. They must have known tho, that peepz will jump at the mdlx RIFF format dissection ASAP. As Sean mentioned it, ACES implemented a plethora of ingenious solutions(can only wonder here). Those things for sure not free or cheap. So I'd have a hard time imagining that they'd just throw the Dev.Manuals at our desk. I like breaking(changing) things after compilation. Hacking and reversing is our "home brew" learning process THEY CAN"T TAKE AWAY. Thanx for all who share the knowlege.

zip

scruffyduck said:

One problem - certainly with scenery files (dunno about mdl) is that MS do not own copyright to the Bgl Format or some parts thereof. Much of what they use belongs in some way or other to third parties and I assume that makes it difficult to share some sorts of information.

I think ACES are being much more open and helpful with us now but within guidelines that are clearly set at a different level.

Click to expand...

Certainly a very valid point. My understanding is that they do actually own the copyright/intellectual property for BGL, but (like other parts of the FS codebase) they are bound to secrecy from licensing terms set forth by Mr. Artwick/BAO when they turned development over to Microsoft. At least that is my understanding, not their official word. I have a feeling we will never know exactly why, as they choose not to share it, but I certainly won't loose any sleep over it as I have bigger problems to worry about.

And I do agree they are doing better now than they used to. Look at the SDK as an example. It is far from perfect, but it certainly gets better with every release. I only hope that they keep becoming more and more open as time goes on.

lc0277 said:

Despite not being "open", FS file formats are not "closed" too.
ACES did not set up any kind of protection to prevent people from discovering the secrets of their formats. In most cases, data format are just bare binary representation of the source data, in the sense that data is not extensively transformed by compilation or it does not contain black magic.
That makes the decompilation process "theorically" possible since very few information was lost. The format hacking is greatly facilited when you have a cooperative compiler (like BGLcomp or XtoMdl); which let you generate plenty of different output. That's the principle of an "automated" Rosetta Stone,
Also, one of the most important thing is to have a clear idea of what the format can do and what it can't do. For instance if you have no idea of the usage of "XZP" files in the Uires directory and you want to guess their format, it will take a lot of time just to figure out they're used to archive multiple resources.

Finally I think that's a fair deal, especially when some files contains copyrighted content from third-parties, as scruffyduck said.

The most notable exception are resampled and vector
BGLs; which are compressed. The result is surprising because an homebrew compression implementation is nearly as secure as an encryption algorithm to
protect content But the goal is different; compression is only here to optimize storage of information whereas encrypting means you really want to restrict the spreading of knowledge.

Click to expand...

Also very valid points. We must always remember that ACES is a business. Their job is to make money through developing simulation games. And they have made the decision that they would rather spend their time and money in marketing and new development than trying to come up with cryptic ways to protect what's already there. It yields the most profit for them. And that certainly comes out as a good deal for us.

I'd say ACES lays pretty much right in the middle of the software development continuum. On one side you have people like id Software who release the source code to practically all of their products. Then on the other you have The SCO Group who actively claims that they own UNIX. I'd like to see them go further toward the open end in the future, but for now we must be happy with what we do have and rely upon the community to figure out the rest.

zippyo said:

We are coming to a sad realization...Cancel or Allow?
I think marketing somin' the right way was more important to M$ than keeping developers trully happy. They must have known tho, that peepz will jump at the mdlx RIFF format dissection ASAP. As Sean mentioned it, ACES implemented a plethora of ingenious solutions(can only wonder here). Those things for sure not free or cheap. So I'd have a hard time imagining that they'd just throw the Dev.Manuals at our desk. I like breaking(changing) things after compilation. Hacking and reversing is our "home brew" learning process THEY CAN"T TAKE AWAY. Thanx for all who share the knowlege.

Click to expand...

Well, it is obviously more of a business decision than anything. For a product like FS, it is not the third party developers who make them the money. It is the casual end users, and that is who they market to. They have to please a large number of people, so they might as well try to appeal to the most they can. Unfortunately us developers are at a bit of a disadvantage since we are not really the 'target group' of their product. However I have to admit that they have done their best within their limitations to help us out.

Now the ball lies in our court (the FS community). It is up to us to do what we can to extend FS into an even more incredible platform than it already is. It is our job to work together to document the internals of FS to benefit the community.

And I don't think ACES will try to take that away. They have said before that they support the third party development community and they mean it. After all it is not like we are trying to steal their intellectual property and make our own rival FS. We are working to extend the MS platform and turn it into something even better.

Sean

I don't have a lot to add to this topic so I shall keep it short and sweet.

I think that with FS we have one of the most open ended platform's available. For aircraft we can just drop the required folders in and it works, we can edit sounds and change them without having to delve deep into configuration files and change bitrate parameters and a plethora of other variables.

We can drop a module into the folder and (most of the time!) it works.

In addition to this, as a developer I think that the FSX SDK is incredibly well written, ok in places it isnt as comprehensive as it COULD be, but I think it goes above the board on what is reasonable, they even included a folder full of example code, AND in 2 different languages (C++ and C#) in addition to the XML stuff.

AND ACES included a .xml gauge checker....

I'd say, as far as 3rd party developers go, we've got it pretty sweet!

Of course I'm not saying anyone above said to the contrary, just voicing my opinion.

Besides.... if we didn't want to tinker with things until they worked and solve problems that cause us to drink copious amounts of caffeine and alcohol into the early hours... then we certainly wouldn't have taken up programming!

All the best,

Alex

Besides.... if we didn't want to tinker with things until they worked and solve problems that cause us to drink copious amounts of caffeine and alcohol into the early hours... then we certainly wouldn't have taken up programming!

Click to expand...

I have often wondered what made me do it

The data in the airport facility files is licensed to Jeppesen. For that same reason they probably cannot include an AFCAD program with the SDK. These databases cost a lot of money.

If you have ever used the BGL2XML decompiler to see how the approaches are coded, you can clearly notice how they have been imported from another source (Jeppesen) and that actually not even 10% of what's available is used by the simulator. All the codes for the legs etc are not documented in the FS9 SDK, 'cause ARINC won't let them. (If you really want to know what all the codes stand for, I suggest you do an intelligent google search and something will come up.)

ACES could of course always write an AFCAD-X and publish it under a fake identity.

Joris

ACES could of course always write an AFCAD-X and publish it under a fake identity.

Click to expand...

There are currently two such utilities in development and both have forums here

Not sure the add-on community is going to be entirely happy with you publishing a tool which will allow someone to basically decomile their model (.mdl) and put it into GMax. To be honest, this type of tool IMHO hurts the community as it will decrease the likelihood that people will want to sell something for so little that can then be stolen easily We all know that the data is there, and even how to get it, but right now the only people who can are the skilled. Please don't make a tool that then turns the average joe into a "script kiddy" who with no knowledge or investment can then take others artwork and even modify it and claim it as their own.

Also, not sure why an open source xtomdl.exe is needed or even desirable, especially when you have clearly violated the MS license to try and produce it. Moreover, because you have looked at the MS code, you cannot now legally call it open source, just so you know. I'm sure you have worked very hard to do all this, but not sure why you feel the need when you can compile a model just fine with the tool you are given.

To be honest, this sort of thing is the very reason MS can't be more open about things than they are being. In essence you are causing the very thing you wish to avoid I think.

Patrick

Hi Patrick.

Almost every tool used by the scenery and aircraft designers has had it's roots in some decompiled or reverse-engineered code.

Many of the modern improvements in flight simulator are due to the expectations of simmers, that have tried aircraft or scenery derived from decompiled or reverse-engineered code, and now expect it in the sim.

Nearly every tool used by members in this forum has their roots tracable to some form of code derived from FS.

As far as the average joes, they end up here, and learn the ins and outs of design and become our friends and peers... few become pirates of other's designs or artwork.

Dick

My tool now converts from BGL/MDL2K4/MDLX format to X format. It's either that or teach people how to do decent backup.... Of those two options, writing a converter to X format proved to be the easy solution.

Hi Patrick.

Please accept my appologies for altering your post... I was attempting a reply, but instead my administrator rights interfered and I accidentally deleted your post. I'm very sorry.

Dick

Waugh said:

While I applaud your programming expertise, and skill, reverse engineering their proprietary format and releasing this program is not in the best interests of developers. There is a reason that MS did not release the format or such a tool, and I really hope you choose to respect their license and wishes in this regard.

Click to expand...

How can you be sure that these are the wishes of Microsoft ? or is it your personnal feelings as a payware editor ? If ACES really wanted to protect their secrets, they would have introduced some sorts of explicit protection in their file formats; but except one minor exception I haven't seen any ...

I think you are seeing only the evil aspects of this kind of software. There are already some decompilers for FS files, but I haven't heard of any stealing case involving decompiling of copyrighted material. Of course there were some cases, but they were rather dumb repackaging or redistribution.
We should not deprive the users community of such a great software just because of the fear of some potential uses, remember we're not talking about lethal weapons.

1) I have not stated I will release this software myself. The feature is part of a larger scenery design suite, and I am unlikely to find the time to support it for a public release. It might change, but who knows. Basically the X exporter was just 10 minutes work to do on top of other obviously needed functionality. But I see no reason as such not to release it besides my lack of time.

2) How can some peoples potential desire to infringe copyright have anything to do with my right to convert a file I hold the copyright to (or have received the permission from the copyright holder to do this conversion). Some people also use HTTP to copy copyrighted material - does this mean we should ban internet browsers?

3) The primary source of information used to develop my tool are the SDK's released by Microsoft. They really do provide quite a lot.

4) As a direct result of using such a tool the free Denmark Scenery will contain updated models where the source files had been lost - most likely including a new FS2004/FSX compatible version of EKSB. How this does not benefit the "general public" is beyond me.