FSX Help with BGL compression algorithms

[rafaeldamasceno]

Hey there,

I'm developing this tool to extract land class info from worldlc.bgl. It is available currently in https://github.com/rafaeldamasceno/BGLConverter.

However, some TRQ1 records have LZ1 compression, for which the wiki only provides some pseudocode algorithm.

I've tried implementing it in C# but I don't seem to be having great success. Currently it is trying to output to an out of bounds index, which I assume is because I'm reading the bits in a bad order. I've tried messing with the endianness of the bits and input bytes, but I don't seem to find the right one.

I also assume I'll have trouble with the double compression ones, since they all seem to require an output size and I have no idea how to calculate the intermediate value.

Any help is appreciated. Thank you!

 

[GaryGB]

Hello:

Perhaps Sean Isom might be willing to share some of his insights on this topic ?

https://www.fsdeveloper.com/forum/threads/bgldec-a-resample-bgl-decompressor.433789/

GaryGB

 

[rafaeldamasceno]

Hello:

Perhaps Sean Isom might be willing to share some of his insights on this topic ?

https://www.fsdeveloper.com/forum/threads/bgldec-a-resample-bgl-decompressor.433789/

GaryGB

It seems that would be great! In any case, just his tool alone is a great help, since I was totally unaware of it. Thank you for your reply!

 

[theisomizer]

Hi Rafael,

I'll consider releasing a reusable lib for the decompression for the non-PTC types. The bgldec lib should be able to tackle worldlc as-is. I can't release the code directly for PTC compression, however PTC is only used in DEM and Aerial imagery formats.

 

[rafaeldamasceno]

Hi Rafael,

I'll consider releasing a reusable lib for the decompression for the non-PTC types. The bgldec lib should be able to tackle worldlc as-is. I can't release the code directly for PTC compression, however PTC is only used in DEM and Aerial imagery formats.

Hello! Since bgldec works so nicely and outputs binary files as well, I think I'll have no need for my own tool anymore. It would be interesting to know what was wrong with my LZ1 implementation, though

 

[theisomizer]

My code is in raw C and works a little differently so I'm not 100% sure what's wrong with yours. The core of the algorithm looks mostly correct, there are some things like:

 if (flag == 1)
 {
     var output = BitListToByte(ReadBits(7)) + 0x80;
 }

This should be an | 0x80 (the others are + 0x40 and 0x140 correctly)

else
{
    _sequenceLength = BitListToInt(ReadBits(nbBitsToRead)) + 1 + (2 ^ nbBitsToRead);
}

Here it is doing a bitwise XOR, your probably mean to do a power, and it is base 1:

length = (1 << num_bits) + LZReadNextNBits(bit_pool, num_bits) + 1;

If that doesn't do it, may be something with the way you are handling the bit pool.

I'll clean it up and post it.

 

[rafaeldamasceno]

Thank you so much for your reply! I'll be seeing if I can get it to work with the tips you've provided.

 

[theisomizer]

I have decided to start open sourcing some of my work as a shared module, starting with the BGL decompression routines:

GitHub - seanisom/flightsimlib: Microsoft Flight Simulator C++ Common libs

Microsoft Flight Simulator C++ Common libs. Contribute to seanisom/flightsimlib development by creating an account on GitHub.

github.com

I am working towards open-sourcing all of bgldec and then internals to some of my other tools.

 

[GaryGB]

Outstanding, Sean; many thanks for your meticulous work and for helping us all better understand- and utilize- this esoteric aspect of FS Development.

GaryGB