Not receiving some data (e.g. engine and controls data)

Bert Laverman · 7 Feb 2017

I have a C++ client connecting to SimConnect and have inconsistent result on getting data returned.

When I try to get the light data returned, I never got data sent for "LIGHT TAXI ON", which should return the state of the taxi light, but did get data for "LIGHT TAXI", which is the taxi lights switch. This holds for all light types. I also tried to get the "GENERAL ENG COMBUSTION:1" variable, but that didn't do anything either. Same for "GENERAL ENG THROTTLE LEVER POSITION:1" and "RUDDER PEDAL POSITION". (and all other levers and yokes)

Anyone has a hint on what to do here?

PS
Tried both:

HRESULT hr = SimConnect_RequestDataOnSimObject(fsxHdl, reqId, defId, SIMCONNECT_OBJECT_ID_USER, SIMCONNECT_PERIOD_ONCE);

and

HRESULT hr = SimConnect_RequestDataOnSimObject(fsxHdl, reqId, defId, SIMCONNECT_OBJECT_ID_USER, SIMCONNECT_PERIOD_VISUAL_FRAME, SIMCONNECT_DATA_REQUEST_FLAG_CHANGED);

the excact same code for the lights and location data which does come through.

Bert Laverman

JB3DG · 8 Feb 2017

LIGHT TAXI ON appears to be another one of those broken variables that came with Acceleration.

Bert Laverman · 8 Feb 2017

Can my inability to obtain controls data come from the fact that they are connected through FSUIPC and LINDA? Shouldn't matter for the general combustion indication though...

Ronald · 8 Feb 2017

Can my inability to obtain controls data come from the fact that they are connected through FSUIPC and LINDA? Shouldn't matter for the general combustion indication though...

Hello Bert,
I do not know (yet) the exact inner architecture of your own C++ Simclient application and/or the FSUIPC.DLL and LINDA application combination.
I do know - from my early years as coder - I can image that for example - the Windows (NT, XP, 2000, XP, and so on...) Memory/ Resources manager:
- LOCKS DOWN a given (range of) memory-(a)locations when an (3rd party, external) application requests access to a given (range of ) memory-location
-> ALLOWS the requesting application to read from / write to the requested locked memory-(a)locations
-> BLOCKS OUT the rest of the applications that want to gain access at the same time
- RELEASES the memory_locations again, when the first requesting application is done, ready, finished with its job over there.

This memory-location-lockdown-mechanism guaranties that:
- only 1 application can safely access, read/write the memory-cell
- your operating system and hence application stays rock solid and stable
- protects requested "(virtual)_memory_space" by applications from one and other

On the moments that your application "does not receive data", that could be the exact moment that other applications are peeking/poking in that same memory_address_range.
So it could be that your application is ok, but it simply runs into a locked memory_cell at the time of the request. It does not have to be a FSX-SDK(A) malfunction!

I have a few links to background articles on Windows Memory management matter for you if you like:
- https://msdn.microsoft.com/en-us/library/windows/desktop/aa366779(v=vs.85).aspx
- https://technet.microsoft.com/en-us/library/cc753446(v=ws.11).aspx
- https://msdn.microsoft.com/en-us/library/windows/desktop/aa366525(v=vs.85).aspx
>>
- https://www.codeproject.com/articles/29449/windows-memory-management

TIP: If you have (memory-related) debugging tools, perhaps you can check which other application is already locking down your requested memory_addresses at the time of "empty_data_return"

Bert Laverman · 8 Feb 2017

Ronald,
Thank you for this insight. I'm however not sure it applies here, because it is not a mattor of having no data or being denied access, SimConnect is not sending any. I request a bunch of definition blocks one after the other, and only the one with the lights gets sent. CheckDispatch does not get any data in not even an exception. Maybe I should add the combustion variable to the lights data and see what that does.

Bert

JB3DG · 8 Feb 2017

What does your set of data definitions and your struct for those definitions look like?

Ronald · 8 Feb 2017

....because it is not a matter of having no data or being denied access, SimConnect is not sending any

- If the SimConnect server can not gain access to some internal FSX variables - which can be locked down by - for example FSUIPC.DLL - how then can SimConnect then send any data back to your C++ application?

I request a bunch of definition blocks one after the other, and only the one with the lights gets sent.

- Is there a capacity / time / memory bandwidth limit inside which an external SimConnect application can send those request after each other, or should the request be firmly separated in time (like 1 request / every 100 milliseconds?)
For example: If your SimConnect server-components (the gateway between your C++ application and FSX internals) is to busy handling incoming SimConnect requests, it might drop a few incomming calls, when busy answering the others

- Is this no-data-error occuring for all types of SimConnect data requests or only the ones you mention here?

Some other question:
- On which platform and Windows Version are you working (FSX / P3D)?
- Is your computer also busy doing other things?
And has for example to few CPU cycles left to power the SimConnect server and FSX?

- Does there exists a kind of "SimConnect application connection-debugging tool" (like "WireShark" https://www.wireshark.org, so one can see the content of all data-trafiic-packets?

- Could this be caused by an SimConnect SDK \ type-of-flight-simulator incompatibility

Bert Laverman · 8 Feb 2017

Naruto-kun said:
What does your set of data definitions and your struct for those definitions look like?

For example, like this one:

Code:

        // Aircraft controls data
        aircraftControlsData_.clear();
        aircraftControlsData_.add(DATAID_CONTROL_RUDDER, "RUDDER PEDAL POSITION", "Position", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_ELEVATOR, "ELEVATOR POSITION", "Position", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_AILERON, "AILERON POSITION", "Position", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_TRIM_RUDDER, "RUDDER TRIM PCT", "Percent over 100", SIMCONNECT_DATATYPE_FLOAT64);
        aircraftControlsData_.add(DATAID_CONTROL_TRIM_ELEVATOR, "ELEVATOR TRIM PCT", "Percent over 100", SIMCONNECT_DATATYPE_FLOAT64);
        aircraftControlsData_.add(DATAID_CONTROL_TRIM_AILERON, "AILERON TRIM PCT", "Percent over 100", SIMCONNECT_DATATYPE_FLOAT64);
        aircraftControlsData_.add(DATAID_CONTROL_SPOILERS, "SPOILERS HANDLE POSITION", "Position", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_FLAPS_HANDLE, "FLAPS HANDLE PERCENT", "Percent over 100", SIMCONNECT_DATATYPE_FLOAT64);
        aircraftControlsData_.add(DATAID_CONTROL_GEARS_HANDLE, "GEAR HANDLE POSITION", "Bool", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_PARKING_BRAKE, "BRAKE PARKING POSITION", "Position", SIMCONNECT_DATATYPE_INT32);
        aircraftControlsData_.add(DATAID_CONTROL_PRIMARY_DOOR, "EXIT OPEN:1", "Percent over 100", SIMCONNECT_DATATYPE_FLOAT64);
        mgr.defineData(aircraftControlsData_, [=](const void* data, const SimConnectData& dataDef) {
            switch (dataDef.id) {
            case DATAID_CONTROL_RUDDER: rudderPos_ = *static_cast<const int32_t*>(data); break;
            case DATAID_CONTROL_ELEVATOR: elevatorPos_ = *static_cast<const int32_t*>(data); break;
            case DATAID_CONTROL_AILERON: aileronPos_ = *static_cast<const int32_t*>(data); break;
            case DATAID_CONTROL_TRIM_RUDDER: rudderTrimPos_ = *static_cast<const double*>(data); break;
            case DATAID_CONTROL_TRIM_ELEVATOR: elevatorTrimPos_ = *static_cast<const double*>(data); break;
            case DATAID_CONTROL_TRIM_AILERON: aileronTrimPos_ = *static_cast<const double*>(data); break;
            case DATAID_CONTROL_SPOILERS: spoilersPos_ = *static_cast<const int32_t*>(data); break;
            case DATAID_CONTROL_FLAPS_HANDLE: flapsPos_ = *static_cast<const double*>(data); break;
            case DATAID_CONTROL_GEARS_HANDLE: gearsDown_ = *static_cast<const int32_t*>(data) != 0; break;
            case DATAID_CONTROL_PARKING_BRAKE: parkingBrakePos_ = *static_cast<const int32_t*>(data); break;
            case DATAID_CONTROL_PRIMARY_DOOR: doorPos_ = *static_cast<const double*>(data); break;
            default: LOG4CPLUS_ERROR(log_, "[aircraftControlsDataItemCallback](): Unknown dataId " << dataDef.id); break;
            }
            return true;
        }, [=](int entryNr, int outOfNr, size_t size, const SimConnectDataDefinition* dataDef) {
            LOG4CPLUS_TRACE(log_, "[aircraftControlsDataCompleteCallback](): Data block complete");
            onLocalStatusUpdate();
            return true;
        });

The first callback gets called for each data item, the second one when a block has been processed completely. The call to mgr.defineData() results first in:

Code:

2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData(): Request definition of data
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData(): Search for a free Define ID
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData(): Request definition of block with nr 21
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData(): Done

Which just preps the data and creates a task object for later processing, to ensure we're not making parallel calls to SimConnect. This task just does the corresponding addToDataDefinition calls.

Code:

2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="RUDDER PEDAL POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=36
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="ELEVATOR POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=37
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="AILERON POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=38
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="RUDDER TRIM PCT"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=39
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="ELEVATOR TRIM PCT"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=40
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="AILERON TRIM PCT"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=41
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="SPOILERS HANDLE POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=42
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="FLAPS HANDLE PERCENT"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=43
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="GEAR HANDLE POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=44
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="BRAKE PARKING POSITION"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=45
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: AddToDataDefinition: defId=21, name="EXIT OPEN:1"
2017-02-07 20:19:54 [DEBUG] nl.rakis.fsx.SimConnectManager defineData()[Requestor]: defId=21, sendId=46

For the Lights I see both this:

Code:

2017-02-07 20:20:05 [DEBUG] nl.rakis.fsx.SimConnectManager requestSimDataOnce()[Requestor]: RequestDataOnSimObject: defId=20
2017-02-07 20:20:05 [DEBUG] nl.rakis.fsx.SimConnectManager requestSimDataOnce()[Requestor]: reqId=150, defId=20, sendId=74

Then a few lines down in the logs:

Code:

2017-02-07 20:20:05 [DEBUG] nl.rakis.fsx.MultiplayManager [aircraftLightDataCompleteCallback](): reqId=150, strobes OFF, landing lights OFF, taxi lights OFF, beacon OFF, nav lights OFF, logo lights OFF, wing lights OFF, recognition lights OFF, cabin lights OFF

which is logging from the second callback. So, the dispatcher was called and passed the data through. For the block above (defId 21) No data ever comes in. Not an error, just never any data.

Bert Laverman · 8 Feb 2017

Ronald said:
- If the SimConnect server can not gain access to some internal FSX variables - which can be locked down by - for example FSUIPC.DLL - how then can SimConnect then send any data back to your C++ application?

okay, that thought hadn't occurred to me. I was assuming you were talking about my own application.

Ronald said:
- Is there a capacity / time / memory bandwidth limit inside which an external SimConnect application can send those request after each other, or should the request be firmly separated in time (like 1 request / every 100 milliseconds?)
For example: If your SimConnect server-components (the gateway between your C++ application and FSX internals) is to busy handling incoming SimConnect requests, it might drop a few incomming calls, when busy answering the others

That is an interesting question. I have no idea how SimConnect works on the FSX side, so I couldn't comment on that.

Ronald said:
- Is this no-data-error occuring for all types of SimConnect data requests or only the ones you mention here?

Ah, a small misunderstanding. I said no data comes in, but neither an error/exception message. I get no errors on the definition of the data block or the requestDataOnSimObject call, nor exceptions passed to the dispatcher.

Ronald said:
Some other question:
- On which platform and Windows Version are you working (FSX / P3D)?
- Is your computer also busy doing other things?
And has for example to few CPU cycles left to power the SimConnect server and FSX?

Well, my desktop is a Windows-10 system and I tried this with both FSX and FSX-SE. The CPU has enough cycles left. typically 50% busy during this testrun.

Ronald said:
- Does there exists a kind of "SimConnect application connection-debugging tool" (like "WireShark" https://www.wireshark.org, so one can see the content of all data-trafiic-packets?

Haven't seen one. Also, being a relative new one in the FSDeveloper arena, I'm probably not the right person to ask. I have enough C++ experience, just not enough SimConnect. Windows API's a stay far away from. I use standard C++10 with Visual Studio 2015.

Ronald said:
- Could this be caused by an SimConnect SDK \ type-of-flight-simulator incompatibility

I'm compiling and linking against the FSX SDK. Good suggestion though, I'll try the Prepar3D one and see if that makes a difference.

Cheers,
Bert Laverman

JB3DG · 8 Feb 2017

All the variables you are trying to retrieve should be of type FLOAT64. Very rare that they are any other type.

Bert Laverman · 8 Feb 2017

Oops, that was embarrassing.

Code:

LOG4CPLUS_TRACE(log_, "[aircraftControlsDataCompleteCallback](): Data block complete");

Changed it to log level DEBUG and I got results.

I guess going into detail on what is happening in the code got me to review that.

Bert Laverman · 8 Feb 2017

Naruto-kun said:
All the variables you are trying to retrieve should be of type FLOAT64. Very rare that they are any other type.

Very bad coding practice to have a Bool or Position (or any kind of integer actually) sent to you as a float or double. Equality on floats and double always turns into an epsilon comparison if you want to rule out rounding errors. I only use floats and doubles if the SimConnect variable is one. Trust me, I have enough grey hairs to warrant this insight with great conviction...

JB3DG · 8 Feb 2017

You misunderstand me....The core variables that you are trying to retrieve from the sim are FLOAT64 type by design of the simulator. Not something you can control.

Ronald · 8 Feb 2017

Bert, thanks for the answers, this helps me gain a better understanding of your situation.

This ESP-SDK article gives a clue about the back-end architecture of the SimConnect server side
"Microsoft ESP Technical Articles:Monitoring AI Objects"

https://msdn.microsoft.com/en-us/library/cc974114.aspx

These are 2 other images that show the entire data-pipeline from your C++ application into the heart of the (FSX/P3D) sim
Everything needs to function propperly otherwise, you do not get any data in or out.

Source: http://slideplayer.com/slide/7364507/

I've found a SimConnect tool, that might be able of some good use for you:
https://simconnectconfigtool.codeplex.com/

EDIT: Perhaps this tool (LINK2FS) can also assist you, but i'm not 100% sure:
http://www.jimspage.co.nz/Link2fs_Multi.htm

TIP: Never, ever assume anything, and debug and double_check everything!

Ronald · 9 Feb 2017

Hello Bert. I overlooked this line, when replying, late last night

Ah, a small misunderstanding. I said no data comes in, but neither an error/exception message.

When you look above in the Simconnect images dataflow.... So there is no reply comes back from the SimConnect serverside, I guess this can be caused by the fact that your C++ application request never have reached the other end-of-the-data-pipeline.
CONFIGURATION: there is a "software / runtime configuration error" in either:
-> A standard Windows runtime library used by Visual studio 2015
-> the SimConnect.dll (client side) and/or SimConnect (server side) component
-> Inside the flightsimulator-engine side itself

APPLICATION: Somewhere along the line, perhaps a(n internal):
-> network level firewall
-> application level firewall
-> anti-virusscanner / anti-malware scanner
blocks the sending and receiving of your requests?

I get no errors on the definition of the data block or the requestDataOnSimObject call, nor exceptions passed to the dispatcher.

Then it must be possible that some of the other components in this data-pipeline are not working properly due to one of the causes mentioned above?

Bert Laverman · 9 Feb 2017

Then it must be possible that some of the other components in this data-pipeline are not working properly due to one of the causes mentioned above?

Ronald, an impressive amount of documentation. I'll take a look.

Just two things straight away: 1) I'm not querying AI aircraft but my own aircraft. 2) while discussing the whole issue I discovered the mistake was entirely my own, because I had different logging levels on the different callbacks. Data was coming in, but the message announcing it was filtered out.

Next of course is going the other way, and trying to reproduce what I get in on another PC with AI aircraft. Given that I now get all data I wanted, that shouldn't be so big an issue, although those are probably 'famous last words'.

Cheers,
Bert

Ronald · 9 Feb 2017

I'm glad everything now works the way you want it too Bert.