CVXExtractor - Exporting vector data.

[See Michel]

See my reply at:

http://www.fsdeveloper.com/forum/posts/776863/

GaryGB

worked with english win7. Thanks!

 

[GaryGB]

Hi Patrick:

I tested using extraction of one of the largest FSX default CVX vector BGLs:

[FSX install path]\Scenery\1002\scenery\cvx8419.bgl

...with intended output to a ESRI SHP file for tests with ESRI SHP file processing in a 64-Bit FS scenery utility.


While processing Hydro polygons, CVXExtractor halted, but did not crash, with this error message:

This was the status of processing when the error occurred:

I suspected the "CvxExtractorGUI.exe *32" process was running without a LARGE_ADDRESS_AWARE flag set.


I have thus far had success in Windows (64-Bit) with applying the LARGE_ADDRESS_AWARE flag to 32-Bit exe's via the "4GB Patch" program by 'NTCore' (aka "Daniel Pistelli") available at:

http://www.ntcore.com/4gb_patch.php


So I backed up a copy of the original "CvxExtractorGui.exe" into a ZIP file.

Next, I Drag-N'-Dropped the original "CvxExtractorGui.exe" onto "4gb_patch.exe" to set that flag.


Then I repeated the test of extracting [FSX install path]\Scenery\1002\scenery\cvx8419.bgl

...with intended output of a ESRI SHP file.


The test with the "4 GB Patch" applied to set the LARGE_ADDRESS_AWARE flag was successful !


This was the status of processing when the task completed:

I then successfully Appended all of the SHP files output by "CvxExtractorGui.exe" to the 64-Bit FS scenery utility.



IIUC, the "4 GB Patch" applied to set the LARGE_ADDRESS_AWARE flag may fail on some 32-bit EXE's, largely due to DLL dependencies; however, I suggest this merits testing for compatibility to update "CvxExtractorGui.exe".


Many thanks again for making CvxExtractorGui available to the FS development community.

GaryGB

 

[rhumbaflappy]

i'll confirm what Gary has found. I resolved the issue by using corflags:

corflags CvxExtractorGui.exe /32bitpref-

This sets the app to not prefer 32-bits, and frees up the memory.

 

[Patrick Germain]

Thanks for the head-up guys. I'll look at that. Maybe a 64 bits version...

 

[rhumbaflappy]

Hi Patrick.

In Scenery directory 0302, cvx2817.bgl throws an error on the extraction of RDX2817. This is New York City area.

 

[Patrick Germain]

Hi Patrick.

In Scenery directory 0302, cvx2817.bgl throws an error on the extraction of RDX2817. This is New York City area.

View attachment 38113

Confirmed. I was able to repro. Will work on it.
Thanks for the head-up

 

[Patrick Germain]

Version 1.0.3.1 fixes the issue.
Note that 1.0.3.1 is a 64-bits version. From now on, there will not be any 32-bits version

Enjoy

 

[rhumbaflappy]

Hi Patrick.

I'm now getting an error: Not a bgl file for cvx2817.bgl

 

[Patrick Germain]

Hi Patrick.

I'm now getting an error: Not a bgl file for cvx2817.bgl

Weird. I downloaded the zip file in a new folder, unzipped , ran it against the file with no error.
1 - Do you have the same issue with other BGL files ?
2 - Can you send me your BGL file ?

 

[GaryGB]

Hi Patrick:

My successful results confirm your results reported above ...for CvxExtractorGui_1_0_3_1 ESRI_SHP extraction of:

[FSX install path]\Scenery\0302\scenery\cvx2817.bgl

Name.............Size (bytes)
FWX2817.shx...75,516
FWX2817.shp...1,845,084
FWX2817.dbf...386,636
FWX2817.xml...490
RRX2817.shx...41,316
RRX2817.shp...557,620
RRX2817.dbf...396,801
PKX2817.shx...25,740
RRX2817.xml...539
PKX2817.shp...2,912,548
PKX2817.dbf...246,882
FLX2817.shx...2,676
PKX2817.xml...536
FLX2817.shp...336,596
FLX2817.dbf...24,891
FLX2817.xml...544
STX2817.shx...18,476
STX2817.shp...661,548
STX2817.dbf...176,966
HGX2817.shx...5,420
STX2817.xml...538
HGX2817.shp...391,460
HGX2817.dbf...26,000
HGX2817.xml...371
HPX2817.shx...274,092
HPX2817.shp...25,605,844
HPX2817.dbf...3,527,808
HLX2817.shx...80,316
HPX2817.xml...773
HLX2817.shp...10,210,460
HLX2817.dbf...772,176
HLX2817.xml...541
UTX2817.shx...5,404
UTX2817.shp...72,604
UTX2817.dbf...51,148
UTX2817.xml...540
RDX2817.shx...574,764
RDX2817.shp...9,668,140
RDX2817.dbf...5,531,238
RDX2817.xml...536

FYI: My CVX2817.bgl is the original file from a full install of FSX Deluxe and Acceleration RTM DVDs (not 'FSX Gold').


Dick: IIRC, your FSX install is a "Steam Edition" ?


Hope this helps !

GaryGB

 

[rhumbaflappy]

Hi Gary.

Thanks for the heads up as to what I actually was decompiling.

Patrick, it was mistakenly a bgl from Dovetail's FSW! The FSX-SE 2817.bgl decompiles just fine as Gary indicates. Sorry for the scare.

 

[GaryGB]

This now begs a question as to whether we may see a version of CvxExtractor (aka "CVXE_GUI") that can decompile "FSW" CVX BGLs !

GaryGB

 

[rhumbaflappy]

New life for an old program. GaryGB has found the CVXExtractor can export a shapefile from MSFS CVX polygon bgl files! Earlier I had discovered these BGLs can be seen in the old TMF viewer. And these shapefiles can be added to a MSFS project, and after assigning the correct Properties in DevMode, they can be saved and displayed in MSFS.

 

[theisomizer]

Somewhat tangentially related -

In MSFS vector data is stored in 2 separate formats. Files processed by BGLComp (most of the airport data and "local" edits through the developer tools - rectangles, polys, airport bounds, etc) use the existing FSX entropy encoded (from PTC) cvx format projected as QMID as Dick indicated. It's great that Patrick has seamless tools to work with this format!

The global vector data (water, roads, etc, some of which is viewable if you turn on OSM debug in the dev tools menu) is taken from the new cgl files, specifically vecXXX.cgl (OSM) and vecnXXX.cgl (MS proprietary). While many of the rest of the decompressed cgl tile types are open formats or stored raw, the vector data has been very difficult to figure out, as it doesn't have a regular structure or any kinds of identifying structures within (as certain sections can be excluded simply by placing a variable length sequence of 0s). I have no idea why they've chosen to carry forward with two very different formats for different parts of the engine.

Unfortunately the cgl vectors are neither documented or contain compiler support for end users to produce through the SDK, but I was finally able to make some progress after some long debug sessions and reading between the lines of portions of the blackshark.ai patent.

With where it is now, I can decode bounding boxes and arrays of polylines/polygons for a tile, as well as types (although I don't have a mapping for each enum value yet). This represents the core but there are many more optional sections present in some tiles, particularly in higher resolution tiles deeper in the quadtree. Perhaps once I've gotten further with these unknown parts I could look at documenting the format and building a basic compiler/decompiler from tiled shapefiles.

Here's an example of a low-res tile that was decoded:

 

[theisomizer]

To follow up on the above, I have finished decoding the CGL vector format. See the other thread here to keep this one focused: https://www.fsdeveloper.com/forum/threads/bgldec-a-resample-bgl-and-cgl-decompressor.433789/

I'll add the format to the wiki shortly in case Patrick wants to add it to his tools or if anyone else is interested It's really not too complex compared to the CVX compression

 

[crushnik]

Having finally learned how to create a TIN file from USGS digital elevation file, I needed CVXExtractor to export airport bounds so I can narrow down the focus to a particular airport. Problem is, I can't get QGIS to display anything from my output file. The layer properties show there are over 200 features, but none of them appear. Could this be an issue because I am using P3D v5.2? Because I haven't had this issue before.

 

[Patrick Germain]

Having finally learned how to create a TIN file from USGS digital elevation file, I needed CVXExtractor to export airport bounds so I can narrow down the focus to a particular airport. Problem is, I can't get QGIS to display anything from my output file. The layer properties show there are over 200 features, but none of them appear. Could this be an issue because I am using P3D v5.2? Because I haven't had this issue before.

Hi Chris
CvxExtractor has been written specifically for FSX. As P3D files are (were ?) almost similar to FSX, it is possible that it works with P3D as well. But as p3D evolves (and not FSX) this will be less and less the case.

 

[crushnik]

Hi Chris
CvxExtractor has been written specifically for FSX. As P3D files are (were ?) almost similar to FSX, it is possible that it works with P3D as well. But as p3D evolves (and not FSX) this will be less and less the case.

That's about what I figured, but having reinstalled SBuilderX and appending the exported airport bounds, it worked fine so it must be a problem with importing into QGIS. Thanks for your great program Patrick!

 

[GaryGB]

Having finally learned how to create a TIN file from USGS digital elevation file, I needed CVXExtractor to export airport bounds so I can narrow down the focus to a particular airport. Problem is, I can't get QGIS to display anything from my output file. The layer properties show there are over 200 features, but none of them appear. Could this be an issue because I am using P3D v5.2? Because I haven't had this issue before.

Hi Chris:

Did you output via CvxExtractor- and use in QGIS- a *.SHP file from the 'local' CVX Vector BGL ?

Is there an ICAO you want to use as a worked example ?

GaryGB

 

[crushnik]

Gary, it's good to hear from you again!

I did, but upon further investigation I found that QGIS said the SHP file I exported from CVXExtractor had no attached coordinate reference. So I tried exporting in Open Street Map format and that worked just fine.

I have finally learned how to make a TIN file, and have an airport boundary for the airport... it's only a matter of figuring out how to limit the TIN data to just one particular airport's boundary, then what to do after that. One step at a time, and I'm getting closer to figuring it out.