• Which the release of FS2020 we see an explosition of activity on the forun and of course we are very happy to see this. But having all questions about FS2020 in one forum becomes a bit messy. So therefore we would like to ask you all to use the following guidelines when posting your questions:

    • Tag FS2020 specific questions with the MSFS2020 tag.
    • Questions about making 3D assets can be posted in the 3D asset design forum. Either post them in the subforum of the modelling tool you use or in the general forum if they are general.
    • Questions about aircraft design can be posted in the Aircraft design forum
    • Questions about airport design can be posted in the FS2020 airport design forum. Once airport development tools have been updated for FS2020 you can post tool speciifc questions in the subforums of those tools as well of course.
    • Questions about terrain design can be posted in the FS2020 terrain design forum.
    • Questions about SimConnect can be posted in the SimConnect forum.

    Any other question that is not specific to an aspect of development or tool can be posted in the General chat forum.

    By following these guidelines we make sure that the forums remain easy to read for everybody and also that the right people can find your post to answer it.

BGLDEC - A resample BGL (and CGL) DECompressor

Messages
284
Country
us-newyork
Hmm, looks like it is running out of memory. How big is your file? 3769 subsections is a lot.

I'll look into making how I read the file in more memory efficient, there are optimizations that can be performed.
 

rhumbaflappy

Administrator
Staff member
Resource contributor
Messages
5,932
Country
us-wisconsin
Most users of your decompiler will just want the highest QMID level of the BGL. A "-t" switch would be nice to specify the top level decompiled and stitched only, or perhaps this could be the default setting. That should save some memory.
 
Messages
284
Country
us-newyork
Thanks Dick, noted. I am planning on revamping the CLI for a 1.0 release and have added this to the list. This won't save memory the way things are structured, but would run much faster.

The memory management is getting messy due to the size of some of these files, I'm having to come up with some creative solutions as I'm sure the MSFS authors had to do as well. Examining TMFViewer, it cheats a little bit. It scans the file for the TRQ1 pointers, and just caches these QMIDs in memory along with the locations in the bgl file. When a paint call is made to GDI, it then reopens the input file, seeks, and decompresses each chunk on the fly based on the target QMID for the screen space. The "cheat" is that it can examine the screen space error for the next highest LOD, and if it is imperceptible it can render at a lower LOD. When zoomed in very far it can render at full resolution, but only needing to decompress a small subset of the tiles. If GDI forces a repaint, it will repeat the full draw, holding no data in application memory. You can observe this by panning a large distance with the mouse on a high LOD and watching the screen "blip" as it repaints. This is very similar to how the FSX terrain engine works and the programs definitely share some code. When writing the full bitmaps out to single files we don't get this luxury.

For now my algorithm reads all of the compressed data into memory, and one row at a time, decompresses, writes to an in-memory output buffer for the whole file. When it is done with each row it throws away the uncompressed. But the whole raw output remains in memory before it is saved to disk. I could pretty easily change this to where it streams one row to disk at a time which would be slower to write but should make for much better memory usage.

For example, when decompressing the default kedw_photo.bgl, the highest QMID level would result in a 2GB bmp file, requiring 4GB+ of memory the way things are currently structured, resulting in a crash. Implementing the above change would knock this to under 100MB.
 
Last edited:

GHD

Messages
12,243
Country
england
Sean,

Would it be possible for you to create an .inf file such that the output bmp could be input to Resample?
 

WebSimConnect

Resource contributor
Messages
154
if you could output one band (height in meters) in geotiff format, your tool would be great and if just could specify geo rectangle (north, west, east south) that would go to all relevant files and produce one output file, that tool would be simply perfect !!!
 

WebSimConnect

Resource contributor
Messages
154
staring point : https://trac.osgeo.org/geotiff

and I guess open source libtiff library shall be just for that task. I've never used it though, as GDAL tools and python are normally sufficient for me, anyway it does not look like rocket science:

Code:
/*
 * makegeo.c -- example client code for LIBGEO geographic
 *     TIFF tag support.
 *
 *  Author: Niles D. Ritter
 *
 * Revision History:
 *   31 October, 1995    Fixed reversed lat-long coordinates   NDR
 *
 */

#include "geotiffio.h"
#include "xtiffio.h"
#include <stdlib.h>
#include <string.h>

void SetUpTIFFDirectory(TIFF *tif);
void SetUpGeoKeys(GTIF *gtif);
void WriteImage(TIFF *tif);

#define WIDTH 20L
#define HEIGHT 20L

int main()
{
    char *fname = "newgeo.tif";
    TIFF *tif;  /* TIFF-level descriptor */
    GTIF *gtif; /* GeoKey-level descriptor */
  
    tif=XTIFFOpen(fname,"w");
    if (!tif) goto failure;
  
    gtif = GTIFNew(tif);
    if (!gtif)
    {
        printf("failed in GTIFNew\n");
        goto failure;
    }
  
    SetUpTIFFDirectory(tif);
    SetUpGeoKeys(gtif);
    WriteImage(tif);
  
    GTIFWriteKeys(gtif);
    GTIFFree(gtif);
    XTIFFClose(tif);
    return 0;
  
failure:
    printf("failure in makegeo\n");
    if (tif) TIFFClose(tif);
    if (gtif) GTIFFree(gtif);
    return -1;
}


void SetUpTIFFDirectory(TIFF *tif)
{
    double tiepoints[6]={0,0,0,130.0,32.0,0.0};
    double pixscale[3]={1,1,0};
  
    TIFFSetField(tif,TIFFTAG_IMAGEWIDTH,    WIDTH);
    TIFFSetField(tif,TIFFTAG_IMAGELENGTH,   HEIGHT);
    TIFFSetField(tif,TIFFTAG_COMPRESSION,   COMPRESSION_NONE);
    TIFFSetField(tif,TIFFTAG_PHOTOMETRIC,   PHOTOMETRIC_MINISBLACK);
    TIFFSetField(tif,TIFFTAG_PLANARCONFIG,  PLANARCONFIG_CONTIG);
    TIFFSetField(tif,TIFFTAG_BITSPERSAMPLE, 8);
    TIFFSetField(tif,TIFFTAG_ROWSPERSTRIP,  20L);
  
    TIFFSetField(tif,TIFFTAG_GEOTIEPOINTS, 6,tiepoints);
    TIFFSetField(tif,TIFFTAG_GEOPIXELSCALE, 3,pixscale);
}

void SetUpGeoKeys(GTIF *gtif)
{
    GTIFKeySet(gtif, GTModelTypeGeoKey, TYPE_SHORT, 1, ModelGeographic);
    GTIFKeySet(gtif, GTRasterTypeGeoKey, TYPE_SHORT, 1, RasterPixelIsArea);
    GTIFKeySet(gtif, GTCitationGeoKey, TYPE_ASCII, 0, "Just An Example");
    GTIFKeySet(gtif, GeographicTypeGeoKey, TYPE_SHORT,  1, KvUserDefined);
    GTIFKeySet(gtif, GeogCitationGeoKey, TYPE_ASCII, 0, "Everest Ellipsoid Used.");
    GTIFKeySet(gtif, GeogAngularUnitsGeoKey, TYPE_SHORT,  1, Angular_Degree);
    GTIFKeySet(gtif, GeogLinearUnitsGeoKey, TYPE_SHORT,  1, Linear_Meter);
    GTIFKeySet(gtif, GeogGeodeticDatumGeoKey, TYPE_SHORT,     1, KvUserDefined);
    GTIFKeySet(gtif, GeogEllipsoidGeoKey, TYPE_SHORT,     1, Ellipse_Everest_1830_1967_Definition);
    GTIFKeySet(gtif, GeogSemiMajorAxisGeoKey, TYPE_DOUBLE, 1, (double)6377298.556);
    GTIFKeySet(gtif, GeogInvFlatteningGeoKey, TYPE_DOUBLE, 1, (double)300.8017);
}

void WriteImage(TIFF *tif)
{
    int i;
    char buffer[WIDTH];
  
    memset(buffer,0,(size_t)WIDTH);
    for (i=0;i<HEIGHT;i++)
        if (!TIFFWriteScanline(tif, buffer, i, 0))
            TIFFError("WriteImage","failure in WriteScanline\n");
}
 
Last edited:

GHD

Messages
12,243
Country
england
See bgldec.txt:

Usage is: bgldec inputFile.bgl [BMP|BMPMASK|BIN[-STITCH]] ["Path\To\Output\Folder"]
 
Messages
284
Country
us-newyork
It's a command line program, like resample. Open a command prompt (Windows key + r, type cmd, press enter). In that window type cd "foldername", where foldername is the folder that holds bgldec. For me that is "C:\Users\sean\Desktop\BGL".

From there you would type the command in the post above, for example:
Code:
bgldec AA_Jordan_066.bgl BMPMASK-STITCH jordan

Hope that helps
 
Messages
284
Country
us-newyork
Version 0.9.2 is up! This is a minor release.

I rewrote the STITCH code, it now outputs the rows in place using the algorithm I previously described. This reduces memory usage by a factor of up to 20x on very large files, so it won't run out of memory. If it surpasses the 2GB limit for a BMP file it will write it as a raw .bin instead.

Also there is a sizing / overlap issue that I fixed with stitch and PixelIsPoint types (257 pixel types here: http://www.fsdeveloper.com/wiki/index.php?title=BGL_File_Format#TRQ1_Record). The easy way to explain this is to open up dem4km.bgl in TMFViewer -> View LOD 3 and set QMID Grid to QMID 13. Note how the pixels are centered on the QMID grid instead of filling it in like photoscenery, that is PixelIsPoint. For these types of tiles the 257th row and column overlaps any touching tiles in the sim. If there is a neighbor north or west it wins and overlays that row, same algorithm as in the sim.
 
Messages
52
Country
germany
Hi I have problem with bgl dec.
When I use the program, I only get a smal part (two stripes) of the image, the rest is blanked out.
See the screeshot.
 

Attachments

  • help.jpg
    help.jpg
    265.4 KB · Views: 453

rhumbaflappy

Administrator
Staff member
Resource contributor
Messages
5,932
Country
us-wisconsin
Hi Sean. New version works good for me. I see for FSW, ORBX has changed something for the worldlc.bgl. The rest of FSW seems to be just FSX files. Don't know if you are interested in FSW...
 
Messages
284
Country
us-newyork
@See Michel - could you PM me with a Dropbox/Onedrive(or equivalent) link to the BGL that is giving you trouble, so I can debug on my end? I have no known issues at the moment, would like to patch it up if I can find the issue.

@rhumbaflappy - That is great to hear Dick! I do want to keep compatibility with all FSX-derived sims, and I was hoping that would be the case. Flight seems to be the only system that has changed the format (32bpp for photoscenery, and additions to worldlc). EDIT - will investigate the differences.

Would folks be interested in adding lclookup format support to this? It is a simple format, I decoded it a long time ago. It can be a little confusing to those that haven't worked with Landclass authoring before. @HolgerSandmann has a spreadsheet out there that does a wonderful job of explaining it.
 
Last edited:
Top