Help please: decoding MSFS format airport BGLs

[Pete Dowson]

I'm posting this plea for any help folks can offer in decoding the MSFS format BGLs in order to extract information for use by other programs.

This is so that i can make an attempt at getting my freeware utility "MakeRunways" working for MSFS. There have been many requests for this -- primarily for use with ATC addons such as Pilot2ATC.

It is only the Airport records which I need. I can see a general resemblence to FSX formatted BGLs but my attempts at decoding only get so far before the FSX method crumbles. I get things like data lengths of zero (causing tight loops in scans of section lists etc). I've already been informed of the new airport ID code (0x56), which was a good start, but little afterwards makes much sense.

Any contributions gratefully accepted. Possible emailing me at petedowson AT btconnect DOT com would be better than here for anything extensive (he says, hopefully!).

In my younger days I used to be able to figure these things out for myself, but I find, at 77+, my hacking and decoding abilities have diminished almost to the point of being useless. :-(

Thanks,
Pete Dowson

 

[scruffyduck]

Pete

Check your email in five minutes. I also changed your email address in your post to make it less accessible to bots

 

[rhumbaflappy]

Hi Pete.

This github site from the maker of little navmap may help:

https://github.com/albar965/navdatareader

 

[GaryGB]

Hi Pete.

This github site from the maker of little navmap may help:

https://github.com/albar965/navdatareader

Pete may have already seen this, but other readers here also may wish to note this info at the above linked page:

"README.txt

Navdatareader is a command line tool that uses the atools fs/bgl
and fs/db modules to store a full flight simulator scenery database into
Sqlite, a relational database.

It can read flight simulator databases from FSX, Prepar3D, X-Plane 11
and the DFD format.

{ NOTE: MSFS-2020 support apparently was added in September 2020 ...GaryGB }

------------------------------------------------------------------------------

A short manual is included as PDF in directory "help" or available online here:
https://github.com/albar965/navdatareader/wiki

------------------------------------------------------------------------------

A configuration file includes various settings and filter options to
exclude files, directories, facilities or airports. If the file is not given
the simulator paths will be automatically extracted from the registry and
the conversion process will start.

The default configuration file with comments can be found here:
navdatareader/resources/config/navdatareader.cfg

--------------------
A confguration file to modify logging options can be found here:
navdatareader/resources/config/logging.cfg

--------------------
The database schema is documented in the atools project which contains
the SQL files that create all the needed tables:

* atools/resources/sql/fs/db/create_ap_schema.sql:
Airports, runways, COM, approaches, transitions and other airport related tables.

* atools/resources/sql/fs/db/create_boundary_schema.sql:
Airspace boundaries and related frequencies

* atools/resources/sql/fs/db/create_meta_schema.sql:
Metadata for BGL files and scenery areas.

* atools/resources/sql/fs/db/create_nav_schema.sql:
VOR, NDB, ILS, waypoints and airways.

* atools/resources/sql/fs/db/create_route_schema.sql:
Tables needed to route calculation.

------------------------------------------------------------------------------
This software is licensed under GPL3 or any later version.

The source code for this application is available at Github:
https://github.com/albar965/atools
https://github.com/albar965/navdatareader "


FYI: At the link cited immediately above:

https://github.com/albar965/atools


...we see this description of included functionality:

"README.txt

atools is a static library extending Qt for exception handling,
a log4j like logging framework, Flight Simulator related utilities like BGL reader
and more.

-------------------------------------------------------------------------------
Modules (by directory in "src"):

* fs
Microsoft Flight Simulator related functionality. Contains a class that automatically finds
flight simulator installations and related paths.

** fs/ap
Simple runways.xml reading tool. Needs MakeRunways by Peter Dowson.

** fs/bgl
A complete collection of classes that read airport and navigation data information from FS BGL files.

** fs/db
A collection of classes that takes the airport and navigation information read from the BGL files and
writes them into a relational database format (currently Sqlite).
See atools/resources/sql/fs/db/README.txt for schema for more information about the database schema.

** fs/scenery
Supports reading of the flight simulator scenery.cfg file.

** fs/pln
Support for reading and writing flight simulator flight plan (PLN/XML) files.

** fs/lb
Flight simulator logbook reading functionality.

* io
Simple binary file reading functionality also using exceptions to ease error handling. Also a
file/log rolling class and a reader for ini files.

* geo
Simple geometry module containing point and rectangle classes as well as various complex calculations.

* gui
GUI and dialog helper classes.

* logging
log4j like logging using the QDebug class. Supports log level filtering into multiple files and file rollover
to keep multiple log files.

* settings
Wrapper around the QSettings class to provide the settings system wide as a singleton.

* sql
Wrapper around Qt SQL classes but with added exception handling to avoid excessive boilerplate
coding for error checks.

* util
Miscellaneous utilities.

* zip
A copy of the unsupported and Qt zip class. Improved for better error handling."


BTW: We also see this information at the link Dick cited above:

GitHub - albar965/navdatareader: Navdatareader is a command line tool that uses the atools fs/bgl and fs/writer to store a full flight simulator scenery database into a relational database like Sqlite or MySql.

Navdatareader is a command line tool that uses the atools fs/bgl and fs/writer to store a full flight simulator scenery database into a relational database like Sqlite or MySql. - albar965/navdata...

github.com

Hope this helps all who may have an interest in exploring more "under the hood" in MSFS-2020 (...and FSX / P3D).

GaryGB

 

[Mikyjax]

Hello,

I'm also trying to decode the BGL without too much success, I took the APX31250.bgl as reference since it's the one used in the wiki, I don't get the same values than in the wiki, I wonder if this file hasn't changed since flight sim 2020.

Well, maybe the structure hasn't change but since the values might not be the same for everything it's really tough to find my mistake.

My goal is to simply pull an aiport Latitude, Longitude, altitude and identifier.
I layed out below a pseudo tree to represent the flow I m following in the file to retrieve values. (pasting it in notepad++ will let you select : Langage>YAML which will allow you to expand and collapse different section of the file based on the indentation which is kinda easier to read.)

//Convention :   (hex to INT)   "Value retrieved at offset"

BGL Data mapping based on APX31250.BGL FS 2020
    BGL FILE HEADER variable size due to section pointers
        FIXED PART HEADER 0x38 (56) bytes
            [DWORD : 4 bytes] Offset 0x14 (20) number of section pointers  : "0x3"
        SECTION POINTERS variable length, starts at Offset 0x38(56)
            SECTION POINTER 1 length 0x14 (20) bytes long
                [DWORD]Offset 0x38 (56) TYPE OF SECTION [DWORD] : "0x3" --> Airport
                [DWORD]Offset 0x38+0x08 = 0x40 (64) NUMBER OF SUBSECTION POINTERS IN SECTION HEADER : "0x5"
                [DWORD]Offset 0x38+0x0c = 0x44 (68) OFFSET FROM FILE START TO SECTION HEADER: "0x74"
                [DWORD]offset 0x38+0x10 = 0x48 (72) SIZE OF SECTION HEADER: "0x50" (80)
            SECTION POINTER 2 length 0x14 (20) bytes long
                [DWORD]Offset 0x4C (76) TYPE OF SECTION: "0x2C" --> Additional airport data
            SECTION POINTER 3 length 0x14 (20) bytes long
                [DWORD]Offset 0x60 (96) TYPE OF SECTION: "0x27" --> Namelist

    SECTION HEADER 1 (Airport pointer) Offset : "0x74" (116) size "0x50"(80)
        SUBSECTION POINTER 1  size = 0x10 (16) Byte
            [DWORD]Offset 0x74 + 0x08 = 0x7C (124) OFFSET FROM FILE START TO START OF OBJECT RECORDS IN THIS SUBSECTION: "0x0124" (I assume this is where the airport header starts)
        SUBSECTION POINTER 2
        SUBSECTION POINTER 3
        SUBSECTION POINTER 4
        SUBSECTION POINTER 5
        2 DWORD RECORDS 8 bytes per pointer (no idea what the use of those are if you could clarify)
        2 DWORD RECORDS
        2 DWORD RECORDS
        2 DWORD RECORDS
        2 DWORD RECORDS
    OBJECT RECORD (Assuming Airport header) Offset : "0x0124" based on the wiki I should get 0x003c for primary airport, I don't, I have a WORD 0x5600 or 0x56 (not sure how I should read it again) 
        [DWORD]longitude Offset : 0x0124 + 0x0c = 0x130 (304) value :  "0x3412920F or 0x0F921234" (not sure)
            After the math where lng = 0x0F921234 = 261231156
            lngDouble = lng * (360.0d / (3 * 0x10000000) - 180);   lngDouble = -47021607963,22057

As you can see the longitude I get makes no sense and I don't know if my position is wrong or if the math is wrong (which comes from the wiki)

Also, and this is really confusing to me, I've been told that when you pull hex from a file like this the convention is to reverse the order of the byte like so:

If I have those bytes at those offsets
0x00: AA
0x01: BB
0x02: CC
0x03: DD
And if I'm ask to read them as a DWORD I should read it as: DD CC BB AA --> int : 3,721,182,122
Does it apply all the time? because it certainly work like this to pull the magnetic declinaison from the magdec file.

I tryied to decode that file base on:

https://www.aero.sors.fr/documentation/FSX_File_structure.pdf

and

BGL file format (FSX airport) - FSDeveloper Wiki

www.fsdeveloper.com

I also tryied to open the source from littlenavmap /atool and cie but this is so well coded and professionnal that it's confusing to read when you re not that used to C++
So If you can help or provide the APX31250.BGL from FSX to compare to mine (FS2020) that would already help greatly to check if they are the same or not.
You can send to mikyjax hotmail com (not sure if it's legal to send that but I think it was from a demo)

Hope somebody can help with this
Thank you for your time.
Mike

 

[ronh]

You seem to have the correct number- After the math where lng = 0x0F921234 = 261231156

Your math is wrong. Use google. Give long of -63. -> 63 degrees west for princess juliana airport.

I used P3D bgl and got -62 for St Barthemely airport for that BGL. 26 F0 9E 0F => 0x0F9EF026 = > 62 deg 50 minutes

 

[Mikyjax]

Thank you Ronh... It works... I spent so much time verrifiyng the file logic...

Fyi in case someone has the same issue, the FSX_datastructure.pdf has a mistake on page 2:

Data types Latitude and longitude are no longer represented as before.

Each location on the earth is fixed in the LOD grid. Longitude and latitude are each represented by a 4 byte value (DWORD).

The formula for obtaining the decimal values is as follows:

(double) Lon = (DWORD) Lon * (360.0 / (3 * 0x10000000) – 180.0) (double)

The -180 needs to be outside of parenthesis as this:

lngDouble = lng * (360.0/ (3 * 0x10000000) ) - 180;

 

[Mikyjax]

Hello,
I still have an issue, I was able to pull all the data I needed so the mapping looks right.
My issue now is that I only have One airport per file (+a dupplicate by file if I count the airport summary)
In total I get 1690 airports in 1783 files. I also have 90 files too small (header size)
I read we are suppose to get hundreds of airport in a file so I'm a little concerned into where I missed something.

Could you give me a file example where I'm supposed to get more than one airport and the offset address where I should find the header of the second airport from that file?

When I count the section pointers in the header, the biggest pointer count I get is 6 (in all files), is this normal? I get that number from the offset 0x14 in the file header.
Thanks!

 

[ronh]

I am not real sure what the issue is. I only assume you have to go through all the records in the file, there may be more than one airport in a bgl. I would think it covers a whole area and you could have many airports in it. One after the other. Each section in the BGL needs to looked at to see if it is an airport???

 

[Mikyjax]

Hello Ron,
If I checked APX31250.BGL as you can see below I should only find 3 sections after the header (Offset 0x14)
Which gives me 1 airport pointer, 1 additional airport data and 1 namelist pointer
I then can retrieve those data. but never more than one airport.
I need to understand where I should look for other airports.
Thanks,


BGL Data mapping based on APX31250.BGL FS 2020
BGL FILE HEADER variable size due to section pointers
FIXED PART HEADER 0x38 bytes
[DWORD : 4 bytes] Offset 0x14 number of section pointers : "0x3"
SECTION POINTERS variable length, starts at Offset 0x38(56)
SECTION POINTER 1 length 0x14 bytes long
[DWORD]Offset 0x38 (56) TYPE OF SECTION [DWORD] : "0x3" --> Airport
[DWORD]Offset 0x38+0x08 = 0x40 NUMBER OF SUBSECTION POINTERS IN SECTION HEADER : "0x5"
[DWORD]Offset 0x38+0x0c = 0x44 OFFSET FROM FILE START TO SECTION HEADER: "0x74"
[DWORD]offset 0x38+0x10 = 0x48 SIZE OF SECTION HEADER: "0x50" (80)
SECTION POINTER 2 length 0x14 bytes long
[DWORD]Offset 0x4C (76) TYPE OF SECTION: "0x2C" --> Additional airport data
SECTION POINTER 3 length 0x14 bytes long
[DWORD]Offset 0x60 (96) TYPE OF SECTION: "0x27" --> Namelist

 

[Szpike]

The Airport record contains lots of subrecords. After 0x0056 in the Airport record you will find the length of the full airport record as a 32bit uint. If you skip ahead that many bytes from the 0x0056 you should find the next airport record with the information you're interested in.

 

[Mikyjax]

Thank you so much Matt, looks like I got all 36973 base airports now!
Sorry for the bother I haven't seen that information anywhere before you mentionned it!