BGLDEC - A resample BGL (and CGL) DECompressor

[fs1]

Updated to 0.5.2 to support P3D 3.2. Old version still attached for anybody running 3.1. I was hoping to have all of this straightened out before having to support another sim version, but no such luck.

I am working hard to remove the dependency on the sim dlls for PTC. Despite my lack of posts, I have made some major progress recently, it is now decoding all bytes in my input properly but just not dequantizing the output. Really hope to get this straightened out soon so I can get back to adding features to get this to more of a user-friendly solution that everybody can use.

Well.. I offered you the algorithm some time ago.

Offer still on if you want some help

 

[Kelvin Richardson]

Hi FS1,

Is this offer for help still open?

I am getting a JIT debugger error and it seems no further builds are available, could you possibly provide a release runtime?

Thanks in advance
Kelvin

 

[fs1]

Hi FS1,

Is this offer for help still open?

I am getting a JIT debugger error and it seems no further builds are available, could you possibly provide a release runtime?

Thanks in advance
Kelvin

I sent several PMs to assist but no reply from Sean.

Regarding the error you should address this to him directly?

 

[Kelvin Richardson]

I sent several PMs to assist but no reply from Sean.

Regarding the error you should address this to him directly?

Thanks, I guess i'll try to get hold of him, not hopeful by the sounds of it

 

[theisomizer]

Hi! I can take a look, it's been a while since I worked on this (or anything). It's been a pretty tough year on my personal life.

@fs1, email incoming.

@Kelvin Richardson, I didn't realize the last build was compiled in debug, sorry! Rather than go through and re-target the offsets for the 3.4 patches and get a release build out, I'd like to just finish the PTC work and cut a properly tested release without the sim dependencies. Let me touch base with @fs1 and see if I can get my PTC bug fixed promptly.

Thanks for the patience!
Sean

 

[theisomizer]

New build 0.7.0 is out!

This removes the dependency on the simulator runtime. This can now be run independently of the sim. Let me know if there are issues.

Thanks to @fs1 for the help!

 

[GHD]

It seems to extract the bmps from a photo bgl, but doesn't interpret the blend mask.

Extracted:



Masked:

 

[theisomizer]

Look at the corresponding Missing Data mask files that are output, although there is no graphical representation of them yet. You could open them in a hex editor or something similar. I'm betting that it implements the blend by saying that those pixels are "Missing" even though they are stored in the compressed data.

 

[arno]

Great, I'll test it again. Thanks for the new build.

 

[fs1]

Great Job Sean

 

[GHD]

Yes, some "missing" files do have hex FF bytes.

 

[arno]

Hi,

I just tried the last version and it works fine (on a machine without any FS installed even).

Do you forsee that this could be used without saving to file but by reading into another program directly? I see a usage in scenProc to do feature detection or roof colour detection directly from a photo BGL. That would be easier if it's not needed to save to disk first. In that case an option to only extract a certain level would also be useful.

Btw, I first typed the command arguments wrong (BMP in front instead of behind the BGL name) and then you directly get a crash. Would be nice to get a warning and gentle shutdown in that case.

 

[theisomizer]

New version 0.8.0 is up!

This is mostly bug fixes. The biggest addition is a new BMPMASK output type to be used with Photos. @Golf-HotelDelta , try it out and see if it accomplishes the result you are looking for.

This flag also gives alpha blending support (against black) if alpha is present in the Missing Data Mask. Using the example from the SDK:

 

[theisomizer]

@arno, interesting points. I could see how that would be useful.

I'm not sure if I would be able to turn this code into an API for some reasons, but let me think about it. If nothing else the ability to give it a QMID to filter to a subset of output, or even a single tile would be very useful. If you've run it against some of the larger default files in the sim it can take some time to decompress everything at every LOD, which would rarely be a desired use case.

Either case will require some long overdue restructuring of the data structures I am using internally now. It is very brute force and parses the input file sequentially, throwing away the results.

Also I should have fixed your crash, I hadn't given the front end much thought

 

[GHD]

Hi Sean,

The BMPMASK worked fine

Grouping them into larger areas would be great.

 

[See Michel]

Hi, thank you for that great tool. Is there any option to export elevation data in a GIS-Tool readable foramt? Like *.raw for example?
Or does any body know a tool how I can work ahead to input the extracted data into GlobalMapper or QGIS? I know have the bmp but no coordinate info or elevation info readable for GIS-Tools.

 

[theisomizer]

Hi See Michel,

Good question. I don't really have any experience with GIS systems, I come mostly from working with game engines.

The output files are named by QMID, which is the internal FS structure for translating Lat-Lon information into a quadtree(with 6 root nodes) for the scene graph. If you're not familiar with it, you can read more on it here: https://msdn.microsoft.com/en-us/library/cc707102.aspx?f=255&MSPPError=-2147217396 and here: http://www.fsdeveloper.com/wiki/index.php?title=BGL_File_Format#TRQ1_Record .

By visualizing this grid, you can imagine that a linear translation to QMID is possible. For example, QMID Level 7 ranges from U=[0,95] in Longitude and V=[0,63] in Latitude. The Lat Lon of the top left pixel could be computed from the following proportions:

double lon = ((double)U * 360 / Umax) - 180;
double lat = ((double)-V* 180 / Vmax) + 90;

The tiles all stretch a full cell with 256 pixels (elevation is a little special in that it has a 257x257 "skirt" for adjacent chunks, so ignoring first row and column), so the Lat-Lon extent of the cell could be derived pretty easily from that. If this checks out I could pretty easily automate this calculation as part of the output in the future.

If you run with a BIN output instead of BMP, I believe it would output what you would call a Band-Interleaved-Pixel raw format (BIP). It would be a signed 16 bit, single channel for Elevations. As above, for elevations they are 257x257 files, ignore the first row and column to arrive at a 256x256 tile. I could automate this as well.

Hope that helps,
Sean

 

[theisomizer]

New version 0.9.0 is up!

The major new addition in this version is the addition of the "-STITCH" option to the existing output types (so BIN-STITCH, BMP-STITCH, BMPMASK-STITCH).

The stitch flag will join all of the tiles in a given QMID level into a single image, making it easier to work with. Please take care when running this flag against large, high-resolution files. As it has to arrange all tiles in memory then write them as raw bitmaps, as I'm sure you can image, this take a lot of memory. This is a 32-bit program so for very expansive, high resolution areas it could crash if it runs out of memory.

Also there are numerous bugfixes and changes as the tool reaches maturity.

 

[GHD]

This is the result:

G:\BGLDec-9>bgldec edpro-photo.bgl BMP-STITCH .
BGLDEC v0.9.0 - A BGL DECompressor for resampled BGL data

BGL FileName: edpro-photo.bgl
Mode:  BMP
Output Directory: .\

Error Reading Input File: edpro-photo.bgl
Ensure it is a valid file that exists and you have read permissions

Directory contents:

G:\BGLDec-9>dir /b
bgldec.exe
bgldec.txt
DOIT.BAT
EDRO-Photo.bgl
vc_redist.x64.exe
vc_redist.x86.e

The input file is:

 

[See Michel]

Hi See Michel,

Good question. I don't really have any experience with GIS systems, I come mostly from working with game engines.

The output files are named by QMID, which is the internal FS structure for translating Lat-Lon information into a quadtree(with 6 root nodes) for the scene graph. If you're not familiar with it, you can read more on it here: https://msdn.microsoft.com/en-us/library/cc707102.aspx?f=255&MSPPError=-2147217396 and here: http://www.fsdeveloper.com/wiki/index.php?title=BGL_File_Format#TRQ1_Record .

By visualizing this grid, you can imagine that a linear translation to QMID is possible. For example, QMID Level 7 ranges from U=[0,95] in Longitude and V=[0,63] in Latitude. The Lat Lon of the top left pixel could be computed from the following proportions:

double lon = ((double)U * 360 / Umax) - 180;
double lat = ((double)-V* 180 / Vmax) + 90;

The tiles all stretch a full cell with 256 pixels (elevation is a little special in that it has a 257x257 "skirt" for adjacent chunks, so ignoring first row and column), so the Lat-Lon extent of the cell could be derived pretty easily from that. If this checks out I could pretty easily automate this calculation as part of the output in the future.

If you run with a BIN output instead of BMP, I believe it would output what you would call a Band-Interleaved-Pixel raw format (BIP). It would be a signed 16 bit, single channel for Elevations. As above, for elevations they are 257x257 files, ignore the first row and column to arrive at a 256x256 tile. I could automate this as well.

Hope that helps,
Sean

Tanks for your feedback. Struggling into code and Koordinate calculating does not match with my skills.
I've seen your latest version 0.9 great step ahead. For decompiling a mesh file it would be great, if the output represents only a stitched file of the highest resolution in geotiff format. There are the coordinates and elevations integrated. One file as output. Every gis-software can handle that.