• Which the release of FS2020 we see an explosition of activity on the forun and of course we are very happy to see this. But having all questions about FS2020 in one forum becomes a bit messy. So therefore we would like to ask you all to use the following guidelines when posting your questions:

    • Tag FS2020 specific questions with the MSFS2020 tag.
    • Questions about making 3D assets can be posted in the 3D asset design forum. Either post them in the subforum of the modelling tool you use or in the general forum if they are general.
    • Questions about aircraft design can be posted in the Aircraft design forum
    • Questions about airport design can be posted in the FS2020 airport design forum. Once airport development tools have been updated for FS2020 you can post tool speciifc questions in the subforums of those tools as well of course.
    • Questions about terrain design can be posted in the FS2020 terrain design forum.
    • Questions about SimConnect can be posted in the SimConnect forum.

    Any other question that is not specific to an aspect of development or tool can be posted in the General chat forum.

    By following these guidelines we make sure that the forums remain easy to read for everybody and also that the right people can find your post to answer it.

CVXExtractor - Exporting vector data.

Messages
7,450
Country
us-illinois
Thanks for the update; I look forward to seeing more of your excellent work ! :D

Hmmm.. might we see a "TMF_Extractor" some time in the future since Terrain Mesh is compiled by FS SDK Resample rather than SHP2VEC ? :scratchch

[EDITED]

In the event that your work might involve a review of the original info on TMF scenery coding and its compilation into BGLs, some references follow (if you haven't already seen these).

Hopefully your work may (eventually) also allow extraction of data from pre-FSX legacy scenery files ...as well as more recent FSX-format BGLs. :idea:


Christian Stock - TMF Manual

http://www.flightsim.com/vbfs/fslib.php?do=copyright&fid=42974


Richard Ludowise (aka "rhumbaflappy") Tutorials:

TMF-BGLC Design Tutorial V1

http://library.avsim.net/esearch.php?CatID=fs2ksd&DLID=19208


LWM Tutorial

http://library.avsim.net/esearch.php?CatID=fs2002sd&DLID=19772


Misc. links to Dick's other packages:

http://library.avsim.net/search.php?CatID=root&SearchTerm=richard ludowise&Sort=Author&ScanMode=1&Go=Change View


Additional info which might possibly prove pertinent upon review
:


Microsoft Flight Simulator 2000 Software Development Kit

Scenery Reference

http://www.fsdeveloper.com/forum/resources/scenery-sdk.25/


Microsoft® Flight Simulator 2002 Software Development Kit

Using Textured Ground Polygons and Land/Water Mask Polygons

http://www.fsdeveloper.com/forum/resources/terrain-sdk.21/


Microsoft® Flight Simulator 2004: A Century of Flight Software Development Kit

Misc. Terrain SDK Documents

http://www.fsdeveloper.com/forum/resources/terrain-sdk.40/



Jim Keir - LWMViewer v1.4

FYI: This utility can decompile LWM BGL to ASM source code as a "BGS" file extension

http://www.jimkeir.co.uk/FlightSim/LWMViewer.html


NOTE: One may wish to compare this to SCA file output from:

Winfried Orthmann - BGLAnalyze version 3.1

http://library.avsim.net/esearch.php?CatID=fs2002sd&DLID=27375

CAVEAT: Some larger files de-compiled with BGLAnayze v3.1 into SCA files may require re-formatting of lines to insert carriage returns at correct positions due to line-wrapping errors

NOTE: Some legacy BGLs may yield a better results when decompiled with Takuya Murakami's ScDis 2.3:

http://sourceforge.net/projects/freesc/files/scdis/scdis 2.3/


Jim Keir - Slartibartfast

http://www.jimkeir.co.uk/FlightSim/Autogen/index.html


NOTE: Slartibartfast also generates 'BGS' source data files:

http://www.jimkeir.co.uk/FlightSim/Autogen/FAQ.html#FAQ16


Slarti at one time at least, was also able to read MSFS Terrain Mesh BGLs directly (IIUC, now via a DLL), and write out vector flattens etc.

IIRC, rhumbaflappy once expressed concerns about errors in some such vector flatten BGLs output by earlier releases of Slarti.


Holger Sandmann and Scott Gridley have extensive experience using Jim Keir's Slartibartfast for freeware and payware scenery.


FYI: This technology for directly de-compiling terrain mesh BGLs was originally available 'briefly' from Jim Keir as a Beta for review and comment by some members of the FS development community, but at that time MS promptly asked him to remove the stand-alone utility from the internet.

It is not clear if the code for directly reading terrain mesh BGLs has been retained in recent versions of Slartibartfast:

http://www.jimkeir.co.uk/FlightSim/Autogen/FAQ.html#FAQ18


Perhaps a review of Scott Gridley's FreeFlow Scenery Tutorial in conjunction with the Slartibartfast manual may reveal the answer ? ;)

http://www.fs-freeflow.com/index.php?ind=downloads&op=entry_view&iden=87



PS: Regarding custom photo-real "land class scenery", this info might also prove interesting:

Land and Water Class Scenery - SBuilder help file excerpted from a post at PTSIM by developer Luis Sa':

http://www.fsdeveloper.com/forum/threads/ptsim-larger-maps.424686/#post-628531


And, as you may know, one must discern whether or not to retain a "Pixel is Point" attribute when preparing and processing source data otherwise submitted in the projection and datum required by FS SDK Resample.

BTW: IIRC, whether a mis-interpretation of this "Pixel is Point" attribute took place when ACES prepared and processed source data when making the default scenery in versions of FS prior to FSX ...is still 'questionable', considering that the variable land class and vector scenery placement is mis-aligned relative to the 'real world' by up to ~1/2 Kilometer in any given direction at many locations in legacy scenery.


Hope all this info helps ! :)

[END_EDIT]

GaryGB
 
Last edited:
Messages
284
Country
us-newyork
Hi,

I feel like a lot of misinformation is out there about the BGL files produced by the resampler and the terrain system in general (not necessarily from this thread, but all over the net).

As one of the above mentioned people that has developed a tool privately to extract raster data from BGL files, I can say it's not the format of the rasterized data that's difficult, nor the container structure of the BGL file itself. It's the compression schemes employed to reduce the size of the data. I've read numerous places where people claim it can't be done, or that these BGLs are "encrypted" by Microsoft to prevent people from ripping off the data. This is simply not true. (Warning: technical jargon to follow)

There are five basic compression schemes employed by the resampler: LZ1, LZ2, Delta, Bit Packing, and PTC. LZ1 and LZ2 are are rather old but very well known algorithms that are similar to zip files for all intensive purposes. FS's implementations of these are fairly standard. They also use a simple adaptive delta encoder with three pivots, which is a trivial algorithm that is well suited to things like elevation data. The bit packing scheme is a little more advanced, but is still fairly standard. Essentially they break the chunk down into further chunks and use a bitmask to throw away whatever bits from a 16-bit integer is not needed to represent any extraneous point in that chuck. So all of these are somewhat standard compression schemes and not hard to re-implement. The resampler decides which type to use based off of what it estimates would yield the best compression ratio given the current chunk's error(quality) metric and it's missing data mask. I am certainly not the only one who has implemented these four decompression schemes, Lamont Clark(lc0277) did this all the way back in 2007.

The problem is the PTC (Progressive Transform Codec) compression type. Google it, it's a Microsoft proprietary image codec not too dissimilar from JPEG, but without wavelets (how? who knows). The sim uses this very heavily as it often delivers the best compression ratio on full QMID chunks (versus edge chunks) and is the only lossy compression scheme used as well. PTC is extremely proprietary; used in a handful of Microsoft products such as Microsoft Document Imaging and in some Xbox 360 games as it shipped as part of the Xbox SDK. It looks like Aces slightly modified it further from there with different headers and such. It is an extremely complicated compression scheme and being proprietary (and patented (http://bit.ly/1cN8HbM) ) there is no real way to replicate it without completely reverse engineering it and ripping it off. And if I were a gambling man, (I'm not violating any EULA's here buddy ;) )I would say that would involve disassembling and reverse engineering about 62 pretty complicated assembly language subroutines with lots of C++ vtable and function pointer goodness. So no, it's not impossible, but it is very difficult and the end result would be a shot-for-shot remake of a proprietary and patented Microsoft technology that is used in products other than flight simulator.

Now would Microsoft do anything about this? Probably not. Very few people in our community have been asked to take something down and the great guys at MS have basically turned a blind eye to it, allowing the 3rd party community to flourish. As for Slarti, which was mentioned above, I highly doubt they were asked to take BGL reading functionality down. Their tool only works on FS9 format raster data anyways which didn't have any of this advanced compression. Regardless, I'm not sure how Microsoft would feel about somebody blindly ripping off their PTC decoder. In some jurisdictions, reverse engineering for the purpose of interoperability is permitted, but patents are another whole legal mess. But please, let's not start any EULA discussions here, we all know how those end up and we are not lawyers.

So what are the solutions to this? I have hooked into a function in the FS process (similar to how FSUIPC works) that is the entry point for decompressing PTC, no reverse engineering or copyright infringement required. The problem with this approach is that with each build, the offset for this changes, so every single permutation of FS out there needs to be accounted for. So to make this something usable by anybody and not just only on my computer would require some sort of algorithm for dynamically searching for this offset and requiring to hook into the FS process, or reverse engineering the entire PTC algorithm as prescribed above. I had never given any consideration the either until I saw the above in this thread.

And as an aside to the naysayers who say that having the ability to access this data is bad, because of data provider's licensing or whatever: It already can be done and is being done. If Microsoft didn't want us to have access to the data in our BGL files they wouldn't have given us TMFViewer. Anybody can extract raster data from a BGL by simply opening it in TMFViewer, mousing over the first data in a chunk to identify the value, opening the RAM in a hex viewer, searching for that value in the Hex, and voila! You can copy the entire decompressed chunk to a RAW file and manipulate it however you want. Very little technical knowledge and no programming knowledge required. In a few short minutes you can have RAW files for every chunk in a small BGL. Unfortunately bad people do bad things; there are multiple places that have been identified by avsim and others as selling illicit copies of other's aircraft and scenery files. If people want to take other's data and pass it off they will, it is not that hard and they will do it regardless of whether there is a tool that makes it easier. There are already tools for MDL files, airport BGL files, now CVX BGL files, and even more; so I don't see a reason why resampled BGL files shouldn't join that club as well.

Okay, enough rant. I will consider putting something together that can decompress these files for people, but it will take some work. It sounds like Patrick is on the case as well. Didn't mean to threadjack Patrick's CVX thread but I just wanted to respond to Gary's comments above.

TL;DR - Getting raster data from BGL files is easy. Building a standalone, independent piece of software to systematically extract this data is possible but could be quite the can of worms.
 
Last edited:
Messages
154
Country
ca-quebec
I totally agree with Sean. Extracting the data is possible. It’s a lot of work but it can be done. Now the real question, as far as I’m concerned, is : Should it be done?.

I will certainly spend some time on it just out of curiosity and for my personal fun (yes, this is also my idea of ‘fun’). But as to include it in a software for general use, I don’t know. Especially when some of this stuff is proprietary and patented.

When I started gathering data and information about the structure of a BGL file, it was originally to know (and share) what kind of information was in the file and where it was located. Regarding the TRQ1 record, Sean answered the question: we now know what information lies in the TRQ1 record.

Should we go further and try to interpret the data that is in that record. Once again I don’t know. Sure it can be done. But as Sean pointed out,
I would say that would involve disassembling and reverse engineering about 62 pretty complicated assembly language subroutines with lots of C++ vtable and function pointer goodness.

I will continue working on trying to interpret the Elevation data found in the BGL file but I’ve have concerns about what I should do with whatever I find out.
 

scruffyduck

Administrator
Staff member
FSDevConf team
Resource contributor
Messages
34,853
Country
unitedkingdom
Thanks Guys. It has always been a balancing act as to how far we go with reverse engineering. At the moment I think we have a reasonable balance and so far no one has asked us to stop.
 
Messages
7,450
Country
us-illinois
Hi,

I feel like a lot of misinformation is out there about the BGL files produced by the resampler and the terrain system in general (not necessarily from this thread, but all over the net).

As one of the above mentioned people that has developed a tool privately to extract raster data from BGL files, I can say it's not the format of the rasterized data that's difficult, nor the container structure of the BGL file itself. It's the compression schemes employed to reduce the size of the data. I've read numerous places where people claim it can't be done, or that these BGLs are "encrypted" by Microsoft to prevent people from ripping off the data. This is simply not true.

Hi Sean:

Thank you very much for posting the helpful, and thought-provoking information above, which expands and clarifies the topic on which I had formulated some mis-interpretations (...based on what I had 'inferred' from various sources on FS websites).


FYI: In an old AVSIM post by Christian Buchner (author of TileProxy), he had indicated, IIRC, that at least a few weeks of work might be required to reverse engineer the newer FSX format terrain scenery BGL compression methods to determine whether or not the data they contained were encrypted.

[EDITED]

One of Christian Buchner's posts on this topic is here (can't find the other one yet):

http://forum.avsim.net/topic/81263-some-ramblings-about-the-new-resampleexe-and-bgl-compression/

"The CompressionQuality=0 setting applies an extremely lossy compression on the tiles, making these very poor quality. And a setting of 100 doesn't mean "no compression", it just uses the highest quality level for the (lossy) PTC algorithm."

"PTC is described in some Microsoft Research paper as a LOSSY image compression algorithm with overlapping block transforms where the coefficients get reordered and then compressed bitplane by bitplane (progressively). Much faster than wavelets apparently."

http://research.microsoft.com/~malvar/papers/dcc00.pdf
<...See new links below -GaryGB >

"But considering this, it will not be possible to decompress a BGL file into its components and to build another identical BGL file from those."



(See work by Henrique S. Malvar aka "H.S. Malvar" and/or "Rico Malvar"):

http://research.microsoft.com/apps/pubs/default.aspx?id=101991

...also cited via:

H.S. Malvar, Fast progressive image coding without wavelets, in:
Proceedings of the Data Compression Conference, Snowbird, UT, USA, March 28–30, 2000, pp. 243–252.

http://research.microsoft.com/pubs/101991/PTC.pdf

...or:

Malvar, H.S.: Fast progressive image coding without wavelets. In:
DCC ’00: Proceedings of the Conference on Data Compression, Washington, DC, USA, IEEEComputer Society (2000) 243–252


For extra credit, consider these links as well: ;)

http://research.microsoft.com/en-us/people/malvar/malvar_publications.aspx


NOTE: Reportedly, PTC is utilized in Windows Media Photo (aka "WMP"), encoding via a "compressor based on something called 'lapped biorthogonal transform' ":

http://webcache.googleusercontent.com/search?q=cache:jK0zaur8ns4J:www.osnews.com/comments/14714?view=flat&threshold=0sort=comment_id&order=o&&perpage=5&offset=98 &cd=5&hl=en&ct=clnk&gl=us

https://msdn.microsoft.com/en-us/windows/hardware/gg463400.aspx


And, of course, one must acknowledge the utilization of PTC compression in MSFS as described by ACES' Adam Szofran in his "Global Terrain" document:

"To help fit all of our data on the distribution media, we employ a variety of compression techniques, both lossless and lossy, always choosing the method that gets the best results for a given set of inputs. One method that we've used heavily recently is the Progressive Transform Codec (PTC), a proprietary lossy technique developed by Microsoft Research [10]. PTC works especially well with both aerial imagery and elevation data because it can handle multi-channel integer and floating-point data with a variety of bit depths per channel. It also can achieve very high compression ratios without an objectionable loss of quality. Like JPEG2000 or wavelet compression, the quality is adjustable with lower quality resulting in smaller output. However, the PTC code is more compact than JPEG2000 and also runs faster. We also use the PTC entropy encoder on the back end of our predictive delta compressor for vector data."

http://download.microsoft.com/downl...obal Terrain Technology for Microsoft ESP.pdf

http://www.microsoft.com/Products/Games/FSInsider/developers/Pages/GlobalTerrain.aspx

[END_EDIT]


I also 'inferred' that encryption might be inherent to the newer FSX CVX BGL format from a post by Dean Mountford in this thread:

http://www.fsdeveloper.com/forum/threads/dissample-bgl-contains-geotiff-image.9105/


With your disclosure above, I'm quite pleased to retract my statements alluding to encryption being inherent to the newer FSX CVX BGL format (yikes... now I've got to find any such old posts of mine, and 'update' them !) :duck:


FYI: I'm glad to see efforts to make tools that can access and export data in FS' default and/or one's own BGL files, and tools which can VIEW BGLs (with additional and more precise info than TMFViewer) ...to facilitate troubleshooting, or learning pros and cons of FS development techniques. :cool:


Although I doubt MS ACES actually used a significant quantity (if any) of proprietary licensed GIS data for creation of the default terrain scenery data in versions of MSFS from FS2000 to the present FSX or ESP / P3D iterations, at the time that Jim Keir had released his "beta" stand-alone utility capable of de-compiling TMF type BGLs, he had definitely reported being contacted by MS asking him to remove the utility from distribution via his web site and IIRC, to not distribute it further in general ...according to a page I saw on his website years ago.

I cannot find that info on the 'current' version of his website, and my saved copy of that original webpage from Jim's website is presently unavailable to me, as it is on an old archived hard drive (which is offline and in storage).

However, at that time, Jim did clearly indicate that he was compelled to comply with MS' request (IIRC, in a spirit of 'co-operation' as, AFAIK, he is an 'FS-Insider' who may have periodically done some consulting work with ACES ;)).

I "inferred" that Jim may have been allowed by MS to utilize an alternate version of his de-compiler in Slartibartfast (as a DLL rather than as a "stand-alone" executable ?) given the apparent feature set it offers for creation of vector content (ex: flattens) from existing terrain mesh BGLs.

But certainly if, as you seemed to indicate, Slartibartfast is unable to read FSX format terrain mesh and/or CVX vector scenery BGLs (I haven't tested it since back when FS9 was my exclusive development version of choice), perhaps what 'decompilation' it does, may be something MS would not object to now.


AFAIK, Takuya Murakami's ScDis 2.3x, Winifried Orthmann's BGLAnalyze version 3.1, SBuilder for FS9's "Append LWM BGL", and Jim Keir's LWMViewer 1.x were all able to access and output TMF LWM vector data.

AFAIK, that type of FS 'vector' content would more likely be the data that ACES may have licensed rather than any mesh elevation data, and IMHO, if by that time MS was still interested in restricting access to that data, they would have taken additional action against other authors of FS utilities with such de-compilation features ...long ago.


BTW: I remember being surprised and dismayed after the dismissal of ACES, that MS seemed almost 'eager' to place copies of MSFS SDK's and certain documentation on specific "trust-worty" FS websites such as AVSIM and FSDeveloper, and had long been 'tolerating' a complete set of FS SDKs on the FSNordic website, apparently so they could remove that content from the FSInsider website, and distance themselves from traffic for ongoing support of the MSFS "franchise" (...and perhaps to free up space so they could instead chase the dream of becoming a paid provider of "cloud" storage" ? :p).

http://abovethelaw.com/2014/08/cloud-at-risk-as-microsoft-is-ordered-to-produce-data-in-ireland/

One might wonder whether MS' Cloud servers rely on security provided by MS' own software ? :yikes:



PS: I suspect that MS no longer has any interest in restricting access to anything inside BGLs, or in restricting the deciphering of BGL formats.

As to 'EULA's, I doubt that MS plans to send any (...more ? :rolleyes: ) of those black helicopters with the green "X" on them to hover menacingly over the house of FS Developers who:

* Completely decipher the BGL format and/or offer a tool to export BGL contents


But then, I also doubt that LM will send a drone (flown by a 'flight-simmer', BTW) to circle menacingly over the house of FS Developers who:

* Find any flying-related activity 'entertaining' or 'fun' < ...who'd fly in a sim or develop add-ons for it, unless it was enjoyable anyway ? :D>


( Hmmm... lets see now; so I can't download the "freely downloadable" LM-Prepar3d SDK until I register first ? ) :confused:


Hi,
Okay, enough rant. I will consider putting something together that can decompress these files for people, but it will take some work. It sounds like Patrick is on the case as well. Didn't mean to threadjack Patrick's CVX thread but I just wanted to respond to Gary's comments above.

I shall look forward to reading more on this... thanks again ! :)

GaryGB
 
Last edited:
Messages
173
Country
unitedstates
I will continue working on trying to interpret the Elevation data found in the BGL file but I’ve have concerns about what I should do with whatever I find out.

And here lies the most obvious question about wanting to expose the data in a Resample compiled file... "Why?"

Hopefully, someone will give an answer that makes sense, without making my head hurt...Gary

Now the disclaimer: I have in the past made mesh and photo based scenery products. I continue to develop mesh work. And hope that if the Doctors do their stinking job, I will continue to tinker with mesh and photo work.

So why is it necessary to expose my work, when IMO it's not necessary. Globally there is free mesh source available that surpasses what is in the default folders. Most already compiled and readily available to end-users. So why must we expose commercial work?

There are products available that utilize source material under license. I can't help but wonder if, for example Intergraph will appreciate that their efforts flying around in the Lear Jet have now been readily exposed by the gaming community. Will they want to continue and expand their licensing under such circumstances?

Same for photo work. Need 48-state U.S. source material? It's sitting on multiple federal and state systems waiting for people to download for free. Why does finished product need to be exposed? Same situation about licensed material applies?

I can make valid arguments about having airport information or CVX file information available. Who wants to make a time consuming redrawing of an QMID cell just because of a mis-placed coastline? I did and instead of taking a lot of time, had I access to the default data that change would have been a lot faster. Same goes for airports, the editors are a massive time saver versus the alternative of doing an airport from scratch. But how does that apply to mesh and/or photo work?

Give me an hour and I can make someone a mesh-pert(!) or a photo wizard and they don't even have to worry about excluding what's already there! Same res work? No problem, use the layering of the scenery library.

One last time I ask, "Why?" is it necessary to expose someone's work, when it's so darn easy to make their own?
 

JB3DG

Resource contributor
Messages
1,325
Country
southafrica
I think the "Why?" question would be most readily answered by gauge programmers like myself. I personally don't need the elevation data in the BGLs as I already have access to undocumented functions (I wouldn't mind getting hold of vector data though) but being able to read the FSX BGLs would open up many possibilities. The FS9GPS map system can be very hard on framerates and it is very limited in freedom for generating certain types of raster patterns (eg my Honeywell EGPWS simulation in the Milviz KingAir is very blocky low res like the real thing with some special dot patterns whereas the FS9GPS is higher res than a Proline Fusion display). Being able to access the core data however would allow for the more skilled to create their own graphics engines that are more efficient than FS9GPS with better framerates and memory consumption, as well as the greater freedom in replicating different kinds of displays.

And the reason it is better to use straight from FSX? Real World doesn't always conform to FSX and as such I think flight simmers (especially the military bunch) would appreciate getting conformity between whats outside and whats on their display.
 
Messages
173
Country
unitedstates
I took one of my mini-pain pills, so I hope I won't come across as grumpy?

First, I would have no problem with any work being read "in process" from the sim. Hack the DLL as much as you want, as far as I'm concerned.

But let's say I go to Intergraph and enter into a licensing agreement to use their product for Europe, as an example. For every product I would sell Intergraph gets paid a royalty. Now you want to come along and use the data I licensed, external of the sim, for your commercial work? Are you going to be offering to pay ME a royalty? Are you not able to license the data yourself and pay the appropriate royalties? Or is it easier to "borrow" another's work.

From a programming perspective, you are going to have to read and process data from the BGL format, on the fly. Is that the best method or could you not gather the data and find that if it was formatted in another manner that it might speed things up by 33%?

The world is out there in 30m splendor. The vast majority of the U.S. is in 10m source. You can make it be whatever you want or need. Why must you go external to the sim and piggyback on my work?

And what would I say to Intergraph when they find out my licensed data is being used external to my work and they are not being paid for the use?


And I forgot to mention, heretofore I've never had to include a "decompile" clause in my EULAs for my work. But that was then and in the future it will be present.
 
Last edited:

rhumbaflappy

Administrator
Staff member
Resource contributor
Messages
5,932
Country
us-wisconsin
Hi Patrick.

With the release of the 1 arc second SRTM elevation data, it would seem there isn't much need for a mesh decompiler, and conversely, there isn't a compelling reason not to have a decompiler. The data is already out there.

Also, there are many sources for imagery that makes an image decompiler not much of an issue (arguing either for or against). For the most part, the data is already out there.

Model Converter X has made object decompilation a moot point. CVX data is now decompiled. XML data can be decompiled. I don't see why the release of a decompiler for resampled data to be much of a point either.

It's knowledge, and is valuable as such.

For the future?

Understanding image photoreal decompiling should make it easier to understand the next generation as represented in Flight which is TRQ2, as I understand it. Flight needs an image compiler, as Microsoft never released a resample for it. Dovetail will probably use TRQ2 for holding the photoreal for it's upcoming flying games. It's unknown if they will release an SDK for that product line, but if the knowledge is already out there, they will probably acknowledge it (that did happen with FS2002-FS2009 terrain elements and Microsoft). Having a resample replacement for Flight could be a very good thing for the future.

My view is that working on decompilers is a good thing. Good for the knowledge and possibly good for our future needs. XML, model and CVX decompilers has not resulted in a loss of income for developers. I don't believe it will result in a loss of money for mesh or photoreal developers. If someone can't make mesh from easily available DEM sources, then they won't be able to make it from decompiled data either.

Dick
 
Messages
284
Country
us-newyork
Wow, lots of great discussion here! I had no idea that commenting on this would bring such a vibrant response. I think many of the most relevant points have already been brought up, but I'll throw my two cents in.

Extracting raster data is useful for:
a) Understanding how the sim works, which as a developer community is what we all want to see
b) Enabling application developers to have a greater degree of control, as mentioned by Naruto. Imagine a terrain aware GPWS with pixel-perfect accuracy, or even a tool that allowed developers to visualize how their work would look in the terrain system in realtime without even having to reload and stage the simulator.
c) Future-proofing ourselves against any future / unsupported versions. Someday the sims as we know them will no longer be supported, the SDK tools will no longer work, and new methods will be pervasive (TRQ2 is just the start ;) ). Being able to still access this data in future content pipelines keeps this entire community alive.
d) My original reason behind initially working on this: lost source data. We all do dumb things, misplacing / losing source data being one of them. You will never be able to replace your source data, but being able to manipulate your processed data sure helps.

At the same time, this degree of control makes it that much easier for people to exploit the system. I still firmly believe that this is already possible; if people want to exploit it they can. The data has already been processed and resampled from the source, it is not the same thing as releasing your raw licensed imagery out into the wild. The sim stores the raw data in memory, uncompressed for anybody to see. If somebody want your data, they already have it, with very little technical skill required. The data is compressed to reduce size on disk, not to prevent people from accessing it.

Anyways, I'm going to start a new thread on the topic of raster data as I don't want to cloud up Patrick's CVX thread with more of this discussion. He's done a fantastic job and I'll leave this thread to discuss it.
 

JB3DG

Resource contributor
Messages
1,325
Country
southafrica
I took one of my mini-pain pills, so I hope I won't come across as grumpy?

First, I would have no problem with any work being read "in process" from the sim. Hack the DLL as much as you want, as far as I'm concerned.

But let's say I go to Intergraph and enter into a licensing agreement to use their product for Europe, as an example. For every product I would sell Intergraph gets paid a royalty. Now you want to come along and use the data I licensed, external of the sim, for your commercial work? Are you going to be offering to pay ME a royalty? Are you not able to license the data yourself and pay the appropriate royalties? Or is it easier to "borrow" another's work.

From a programming perspective, you are going to have to read and process data from the BGL format, on the fly. Is that the best method or could you not gather the data and find that if it was formatted in another manner that it might speed things up by 33%?

The world is out there in 30m splendor. The vast majority of the U.S. is in 10m source. You can make it be whatever you want or need. Why must you go external to the sim and piggyback on my work?

And what would I say to Intergraph when they find out my licensed data is being used external to my work and they are not being paid for the use?


And I forgot to mention, heretofore I've never had to include a "decompile" clause in my EULAs for my work. But that was then and in the future it will be present.

Ok some misunderstandings to be cleared up. If the developer knows his stuff, he wont decompile your product into his own database and distribute it. He will simply scan the user's scenery cfg to find out what is there and then only use whats there at runtime. To decompile your scenery, and build his own database in his own format from that is daft in the extreme. Its double the work. Its a waste of hard drive space.

Second, in light of the process I have described above, MS would have to pay you a royalty so that the FS9GPS system could display your terrain data. Because that is what the developer would be doing. Merely replicating the functionality of FS9GPS in a more efficient and performance friendly way (or extending the functionality to suit the needs of the display). Once again, no distribution is being made. And its for that purpose that being able to parse the source files is needed. If the end user doesn't have your scenery installed, it will not be used.

Third, regarding efficiency, sometimes it is best not to be inside the FSX process at all. Take the case of the Flight1 GTNs. One can save considerably on VAS usage in FSX and take advantage of the user's hardware capacity by running a separate child process to FSX which handles the processing of the terrain data and rendering of the cockpit display, and then with inter-process communication links the final image can be sent to FSX for display in the cockpit with a fraction of the cost of VAS and little to no impact on framerates.

Bottom line, for a avionics dev trying to support a multitude of different scenery addons out there, being able to parse the data directly from whats installed is actually more efficient than locking himself to a specific developer's addon scenery. And since he will not be copying or distributing the data used in your scenery, neither will his creation make any use of the data in your scenery unless it is installed, he is not doing anything that would require license from your data source IMHO.
 
Messages
7,450
Country
us-illinois
"Bumped" again for another edit to my post above (found the MS Research citations and online archives of the documents on PTC compression etc. by Henrique S. Malvar (aka "H.S. Malvar" and/or "Rico Malvar") containing info pertinent to this topic):

http://www.fsdeveloper.com/forum/th...porting-vector-data.432918/page-3#post-707856


IMHO, this kinda' delivers a "EULA-gy" for otherwise well-intended concerns over risk for re-creation of identical copies of content with original quality as submitted in the source data via de-compilation / viewing / extraction ...from certain types of (raster-derived) BGLs ! :p

http://en.wikipedia.org/wiki/Eulogy

GaryGB
 
Last edited:
Messages
144
Country
italy
CvxExtrator update : New version 1.0.1.4

- Added Altitude in the BLN output.

Enjoy!

First of all thank you for this truly outstanding tool. Also, I wanted to ask, given the absence, if you has abandoned I discontinued the development of CVXExtractor?

I did a data extraction, relatively to UTX format final BLN. The altitude is extracted -9999. Importing in SBuilderX I think it is this information to give an error Globally exception - Overflow of arithmetic operation. The value for altitude UTX that accepts SBX is 0.

4,1,Utilities_4_0
-4.66328144073486,32.7747756242752,0
-4.66597080230713,32.7684670686722,0
-4.68131303787230,32.7535271644592,0
-4.6875,32.7481734752655,0

Then there is the usual annoying problem of comma / decimal point of our operating systems that adopt the system of measurement MKSA. Before you import the file into BLN SBX you must change the comma with a decimal point.
 
Messages
154
Country
ca-quebec
Also, I wanted to ask, given the absence, if you has abandoned I discontinued the development of CVXExtractor?

No, the development is NOT abandoned. There is always room for improvement and new features. I will probably create a GUI version along with the command-line version.
And maybe it would be a good idea to have the default altitude value configurable (like 0, or -9999 or anything else).
Any suggestion is welcome.
 
Top