FSX Replicate FSX panel on a smartphone using WebSockets (WebSimConnect?)

[ti_dabbelju]

For my ATR 72 flight simulator I want to replicate the autopilot display (AFCS) as a webpage displayed on a smartphone.

With my Flight1 software there is no other way to get the displayed text than through OCR. So my current solution is this: - use AutoHotkey to run the OCR(Capture2Text) and write a HTML page if the Text changes. - Add meta tag "refresh" for the client side update - The freeware "mini webserver" to host the html page.

The screen capture part actually works reasonably well but the 5 second refresh of the web site one the phone is awful. I have downloaded and tried WebSimConnect and looked into WebSockets where the client only refreshes when the data has changed. But I have trouble adapting the examples to my need.

Maybe with your help I can put something together.

Thanks!

 

[rpmc]

There are a few Flight Sim-http applications out there I'm aware of that can be used to help write a webpage gauge. You mentioned WebSimconnect, and I believe Marcin will elaborate once he reads this thread.

Another is HttpX.dll by Robbie McElrath. It's a token-based (FS variables) approach whose capabilities are outlined here. You need to know some javascript and html to write the gauge, and you'll have to wait a little longer until I finish documenting the HttpX API before you try it, but I'm nearly done with that.

I also see this smart phone PFD recently posted by Eric Marciano. I'm sure other web apps are available, but at least there are a few people already offering modules that you can use to make your own gauge.

Bob

 

[WebSimConnect]

and I believe Marcin will elaborate once he reads this thread.

Well, there is a alternative solution to WebSimConnect.exe using WebSimSoscket from 2.1 version. It is a bit different than using WebSimConnect http server but should suit your needs.

Look below at the architecture picture:

What I would do for your purpose is:

1) Build C++ gauge that does OCR (that you already have) which uses IWebSimClient class to communicate (using ExecucteJavaScript method) with external HTML gauge using websocket protocol.
2) Build your HTML gauge code that will connect to WebSocket server embedded in WebSimBrowser.dll. It is in-process DLL unlike WebSimConnect.exe http/websocket server. the HTML code only need to make a connection using built in JavaScript WebSocket object and listen to incoming mesgaes from your C++.

The similar solution was presented here,

the only difference to your scenario is that inside FSX you will not have any HTML based gauge as it is not needed (you have your C++ code)

In case your HTML gauge is interactive (eg mouse clicks) and want to communicate back with C++ code, you would need to send messages through WebSocket.send and using "WebSimFunction:your data" format. The C++ code would catch them by overriding OnJavaScriptFunction virtual method of IWebSimClient interface class.

The 2.1 version (required for WebSimSocket) is not released to public (I'll plan after linking to new P3D v4) , but I can give you 32bit version already if you want with some small hints how to do it. After writing this post, I see now it shall be trivial task. In that solution your external HTML page does not need to be hosted on server. It can be a local file opened by the browser-

regards

Marcin

 

[WebSimConnect]

a C++ part of the code would be:

#include "../WebSimBrowser/IWebSimClient.h"

hinstLib = GetModuleHandle("WebSimBrowser.dll");

    if (hinstLib != NULL)
    {
        IWEBSIMCLIENT IWebSimClientProc = (IWEBSIMCLIENT) GetProcAddress(hinstLib, "IWebSimClient");

        // If the function address is valid, call the function and get the pointer to IWebSimClient class

        if (NULL != IWebSimClientProc)
        {
            pIWebSimClient = (IWebSimClient*) (IWebSimClientProc) (0, "mygauge");   // get pointer for HTML page identified as "mygauge"


....

code to send data to HTML page:

pIWebSimClient->ExecuteJavaScript("data string for example in JSON format");

a JavaScript code would be :

// initialisation (prefereable onLoad event of Window/BODY or using manual "connect" button )
websocket = new WebSocket('ws://' + location.hostname + ':8080/' + "mygauge", 'websimsocket' );

// location.hostname or fixed address to the PC hosting FSX.

// please note "mygauge" name MUST be the same used in C++ to pair the HTML with C++ code.

// listener

websocket.onmessage = function(message) {
    if (message.data.substring(0,18) == 'ExecuteJavaScript:')
        eval(message.data.substring(18)); // the message text passed from C++, e.g. OCRed values in JSON format
   // here you update your HTML code with values passed from FSX
}

Link to documentation:

https://www.dropbox.com/s/y5o2ady8pasfoyl/WebSimSocket v2.1.pdf?dl=0

BTW, you could also skip the C++ code and use WebSimXML component to send messages to your HTML page using XML coding and my C: variables, e.g.

<Script>&apos;mygauge:data block - some variables&apos; (&gt;C:IWebSimClient:'ExecuteJavaScript)</Script>

it would do the same thing as C++

 

[ti_dabbelju]

After writing this post, I see now it shall be trivial task.

Well,... actually I'm quite overwhelmed now! My programming skills are not that good. I appreciate your help but right now it looks to me I should propably stay with websimconnect as I begin to understand that. But I'll take a closer look at your suggestion.

 

[WebSimConnect]

. I appreciate your help but right now it looks to me I should propably stay with websimconnect as I begin to understand that.

I misunderstood your concept. I thought you have a C++ gauge that reads via OCR the values. I see now you have standalone app (Capture2Text) which saves the output in HTML.

In that scenario neither WebSimConnect.exe nor WebSimSocket component of WebSimBrowser can help. Same for by Robbie's HttpX.dll. They are all connected to SIM while your Capture2Text is not.

As I understand your solution the Capture2Text produces files which you want to send to your smartphone. You host those files on mini http server. Honestly I have not heard of standalone websocket server that could do what you want. Mine components are for sending data from FSX/P3D and capturing events from HTML page.

What I could suggest you to improve your solution is to change output of Capture2Text and produce small files in JSON format, e.g. { "firstvar":13, "secondvar":45 ... }

and in HTML (static file) you read in a timer those small file and update HTML content whenever the values read from JSON format change. You need to read those file using JavaScript XMLHttpRequest object (you'll find lots of tutorials) . One thing you have to take of is the cache. In order not to read the file from cache you can add some counter to the querystring, e.g. http://localhost/mydata.json?counter=3287123 and with each request you increment counter value.

The advantage of this solution compared to your current one(the way I understand it) is that you do not need to refresh HTML page as the page will automatically refresh content with variables coming from JSON file.

 

[ti_dabbelju]

Ah, yes it's a little bit hard to explain, it's more of a makeshift project.

Your tips are great! I managed to get a working webpage with JSON data in under 2 hours. I have never done this before. All I need to do now is modify the JSON file with my OCR data.
Now you mentioned the cache problem, I have not addressed that yet. Actually in Chrome on my PC which hosts the html file (again using mini webserver) any change to the JSON file gets updated automatically. This also works on an Android tablet. Only my Windows Phone 8.1 won't update until I delete the browser's cache.

<!DOCTYPE html>
<html>
<body text="#FFFFFF" bgcolor="#000000">
<script>
var myVar = setInterval(myTimer, 500);

function myTimer() {
    readJson();
}

function readJson() {
        var xmlhttp = new XMLHttpRequest();
        xmlhttp.onreadystatechange = function() {
                if (this.readyState == 4 && this.status == 200) {
                        myObj = JSON.parse(this.responseText);
                        document.getElementById("row1").innerHTML = myObj.row1;
                        document.getElementById("row2").innerHTML = myObj.row2;
                        document.getElementById("row3").innerHTML = myObj.row3;
                        document.getElementById("row4").innerHTML = myObj.row4;
                }
        };
        xmlhttp.open("GET", "display.json", true);
        xmlhttp.send();
}

</script>

<font face="VERDANA" size="+1" >
         <p id="row1"></p>
         <font color="#FF8040"><p id="row2"></p></font>
         <font color="#FFFFFF"><p id="row3"></p></font>
         <font color="#11E229"><p id="row4"></p></font>
</font>

</body>
</html>

JSON file:

{
    "row1":"CAT II   ALT SEL 24000 FT",
    "row2":"NO ENGAGEMENT ON GROUND",
    "row3":"VOR             GS",
    "row4":"HDG SEL LO  VS 900 FPM"
}

 

[WebSimConnect]

glad it works for you. to resolve cache issue use the following trick:

xmlhttp.open("GET", "display.json?random=" + Math.floor((Math.random() * 1000000) ) , true);

Marcin

 

[ti_dabbelju]

This works well for me. However I tried hosting the above html page in websimconnect server and add some simulator data, too. I adapted the WebSimConnect\www\lab\tutorial1.html example like below. The JSON data is loaded and displayed upon start but is not updating when the JSON file is changed though the format is the same as before. The fsx data is updated without problems. Do you have an idea what could be wrong?

<!DOCTYPE html>
<HTML><HEAD></HEAD>
<BODY onload="body_onload()" text="#FFFFFF" bgcolor="#000000">

<SCRIPT src="../jquery-1.7.1.js"></script>
<SCRIPT >
var definition_data;
var myVar = setInterval(myTimer, 200);
function body_onload() {

    var jqXHR;

    $.ajaxSetup({ cache:false }); // nedeed for IE8
    $.support.cors = true; // required if the script is local
    $.ajaxSetup({ async: false });

    jqXHR = $.getJSON("http://" + location.hostname + "/clear_definition?definition_name=autopilot_alt", function(data) {});
    
    jqXHR = $.getJSON("http://" + location.hostname + "/add_definition?definition_name=autopilot_alt&name=my_ap_alt&PropertyName=AUTOPILOT ALTITUDE LOCK VAR&UnitName=Feet&DatumType=FLOAT32", function(data) {});
    
    $.ajaxSetup({ async: true });
    
    window.setInterval(get_values, 1000);
}
function get_values() {

    jqXHR = $.getJSON("http://" + location.hostname + "/get?definition_name=autopilot_alt", function(data)
    {
        definition_data= data;
    });

    var ap_altitude = document.getElementById("alt_element");

    if (definition_data!= undefined)
    {
        ap_altitude.innerHTML = definition_data.my_ap_alt;
    }
    else
    {
        ap_altitude.innerHTML = 'No data retrived from FSX/Prepar3d';
    }
}


function myTimer() {
    readJson();
}

function readJson() {
        var xmlhttp = new XMLHttpRequest();
        xmlhttp.onreadystatechange = function() {
                if (this.readyState == 4 && this.status == 200) {
                        myObj = JSON.parse(this.responseText);
                        document.getElementById("row1").innerHTML = myObj.row1;
                        document.getElementById("row2").innerHTML = myObj.row2;
                        document.getElementById("row3").innerHTML = myObj.row3;
                        document.getElementById("row4").innerHTML = myObj.row4;
                }
        };
        xmlhttp.open("GET", "display.json?random=" + Math.floor((Math.random() * 1000000) ) , true);
        xmlhttp.send();
}
</SCRIPT>

<H1>WebSimConnect Tutorial 1 - get definition values</H1>
<P>Current airplane ap_altitude is: <SPAN id="alt_element"></SPAN></P>
<font face="VERDANA" size="+1" >
         <p id="row1">jau</p>
         <font color="#FF8040"><p id="row2"></p></font>
         <font color="#FFFFFF"><p id="row3"></p></font>
         <font color="#11E229"><p id="row4"></p></font>
</font>
</BODY></HTML>

 

[WebSimConnect]

are you hosting the dynamic JSON file on WebSimConnect http server ? Apparently my implementation does not allow to modyfy any static content (e.g. your json) of the server. I keep the files open, so they are locked. Try it yourself, once you start the server and open json file you will not be able to replace that file with explorer copy operation. You would need to host your json on different http server that allows to replace files after server start up.

 

[ti_dabbelju]

I see, thanks.

 

[WebSimConnect]

BTW, to get data from FSX into webpage without programming, there is also alternative with 2.1 version using WebSimSocket and WebSimData, i.e. the code from my #3 post can be replaced by simple configuration file of WebSimData component:

[WebSimData]
Exceptions=1

[WebSimData00]
Frequency=1
id=mygauge
function=updateAltitude
data00=AUTOPILOT ALTITUDE LOCK VAR,Feet,FLOAT32

and your javascript would be:

// initialisation (prefereable onLoad event of Window/BODY or using manual "connect" button )
websocket = new WebSocket('ws://' + location.hostname + ':8080/' + "mygauge", 'websimsocket' );

// location.hostname or fixed address to the PC hosting FSX.

// please note "mygauge" name MUST be the same used in WebSimData ini file, i.e. id=mygauge

// listener code

websocket.onmessage = function(message) {
    if (message.data.substring(0,18) == 'ExecuteJavaScript:')
        eval(message.data.substring(18)); // the message text passed from WebSimData
}

function updateAltitude(altitude)
{
   var ap_altitude = document.getElementById("alt_element");
   ap_altitude.innerHTML = altitude;
}

WebSimData will simply get every visual frame (Frequency=1) AUTOPILOT ALTITUDE LOCK VAR variable and call JavaScript function updateAltitude. WebSimSocket will "magically" transform that call using websocket protocol by sending message "ExecuteJavaScript:" that is trapped by JavaScript code.

IMHO it is the most elegant solution in your case. IF you are interested I can give you binaries for 2.1 version to achive the task

 

[ti_dabbelju]

As a follow up here is my result.

There's a lot of moving parts here so I'll try to sum up a little.

Task: Replicate the advisory display of the ATR 72 as a webpage to show it on mobile devices.
Prerequisite: The Flight1`s ATR software for FSX has no data interface nor does it use standard autopilot (except for 2 values, see below).

#1: The numbers for selected altitude and autopilot vertical speed can in fact be read directly from FSX. These numbers should be updated quickly or else the adjustment is painful. This I achieve through Marcin's WebSimData as described further up. After following his instructions I have the values as Javascript variables in my webpage's code.

#2: The autopilot modes and messages: The only way I could find to get these is through on-screen-recognition (OCR). So I place the FSX window with the autopilot somewhere out of sight on an extra monitor. I have tried Capture2Text in combination with an AutoHotkey script and this works but produces many errors and is taking quite a bit of computing power. AutoHotkey has a nice function called "PixelSearch". This is extremely fast as it just looks for one pixel with a certain color in a given area. It is actually possible to find a unique pixel for every possible text that will show on the autopilot display.
Now how do I get this in my webpage? The solution is to write the information (just texts really) in a file and use a structure that Javascript can read: JSON format.

#3: Create a html webpage with Javascript that combines and displays all of the above.

#4: Host the html webpage on a WebServer; I am using a freeware called "Mini Webserver" - it's just an executable that you need to run.


Now, by navigating with a browser on any smartphone, tablet or my local PC I can see the replicated autopilot display. Phew...
Thanks Marcin and everyone else for assisting me with this project.

 

[ti_dabbelju]

web page:

<!DOCTYPE html>
<html>
<body onload="body_onload()" bgcolor="#000000">


<table width="100%" border="0" cellpadding="0" cellspacing="2" style="font-family: monospace; font-size: 18px;">
 <tr height="20px" style="color: #F3F3F3;">
  <td id="aduAdvisory">1</td>
  <td id="aduSelectAlt" align="right">2</td>
 </tr>
 <tr height="20px" style="color: #FB9B04;">
  <td id="aduMessage" colspan="2">3</td>
 </tr>
 <tr height="20px" style="color: #F3F3F3;">
  <td id="aduArmedLateral" width="50%">4</td>
  <td id="aduArmedVertical" align="center" >5</td>
 </tr>
 <tr height="20px" style="color: #25E611;">
  <td id="aduCapturedLateral" width="50%">6</td>
  <td id="aduCapturedVertical" align="right">7</td>
 </tr>
</table>

<script>

var aduSelectedVSpeed;                        // global vars
var aduSelectedAirspeed;
var jsonCapturedVertical;

function body_onload() {
        // initialisation (prefereable onLoad event of Window/BODY or using manual "connect" button )
        var myVar = setInterval(myTimer, 300);

        websocket = new WebSocket('ws://' + '192.168.2.109' + ':8080/' + "mygauge", 'websimsocket' );

        websocket.onmessage = function(message) {
                if (message.data.substring(0,18) == 'ExecuteJavaScript:')
                        eval(message.data.substring(18)); // the message text passed from WebSimData
        }
}

function updateAltitude(altitude,vert_speed,airspeed)        // called via WebSimData gauge, see ini file
{
   var ap_altitude = document.getElementById("aduSelectAlt");
   ap_altitude.innerHTML = "ALT SEL " + altitude + " FT";

   aduSelectedAirspeed = airspeed;        // copy to globals
   aduSelectedVSpeed = vert_speed;
   updateCapturedVertical();
}

function updateCapturedVertical(){
        var ap_cap_vertical = document.getElementById("aduCapturedVertical");

        if (jsonCapturedVertical.indexOf("|||") != -1){                // found special characters indicating IAS
                ap_cap_vertical.innerHTML = "IAS " + aduSelectedAirspeed + " KTS";
                }
        else if (jsonCapturedVertical.indexOf("---") != -1){  // found special characters indicating VS
                ap_cap_vertical.innerHTML = "VS " + (aduSelectedVSpeed < 0?'':'+ ') + aduSelectedVSpeed + " FPM";
                }
        else {
                ap_cap_vertical.innerHTML = jsonCapturedVertical;
                }
}


function myTimer() {
    readJson();
}

function readJson() {
        var xmlhttp = new XMLHttpRequest();
        xmlhttp.onreadystatechange = function() {
                if (this.readyState == 4 && this.status == 200) {
                        myObj = JSON.parse(this.responseText);
                        document.getElementById("aduAdvisory").innerHTML = myObj.aduAdvisory;
                        document.getElementById("aduMessage").innerHTML = myObj.aduMessage;
                        document.getElementById("aduArmedLateral").innerHTML = myObj.aduArmedLateral;
                        document.getElementById("aduArmedVertical").innerHTML = myObj.aduArmedVertical;
                                                document.getElementById("aduCapturedLateral").innerHTML = myObj.aduCapturedLateral;
                                                jsonCapturedVertical = myObj.aduCapturedVertical;
                                                updateCapturedVertical();
                }
        };
        xmlhttp.open("GET", "display.json?random=" + Math.floor((Math.random() * 1000000) ) , true);
        xmlhttp.send();
}
</script>


</body>
</html>

Excerpt from AutoHotkey:

getState(x,y,col,precision,options){
    ; find out if the pixel is found on the screen

    state := -1          ; -1 error | 0 not found | 1 found
    x1 := x-1        ; range start
    x2 := x+1        ; range end
    y1 := y-1
    y2 := y+1

    PixelSearch px,py,%x1%,%y1%,%x2%,%y2%,%col%,%precision%,%options%
    ;msgbox, result: %ErrorLevel% | Params: %x1%,%y1%,%x2%,%y2%,%col%,%precision%,%options%

    if ErrorLevel < 2                ; 2: Error | 1: Not found | 0: Found
        state := (! ErrorLevel)        ; convert to: 0 Not found | 1 Found

    return state
}

MakeJson(L1,L2,L3,L4,L5,L6){
    global aduFile
    
    FileDelete, %aduFile%    ;alte Datei löschen
    FileAppend, {"aduAdvisory":"%L1%"`,`n ,%aduFile%
    FileAppend, "aduMessage":"%L2%"`,`n ,%aduFile%
    FileAppend, "aduArmedLateral":"%L3%"`,`n ,%aduFile%
    FileAppend, "aduArmedVertical":"%L4%"`,`n ,%aduFile%
    FileAppend, "aduCapturedLateral":"%L5%"`,`n ,%aduFile%
    FileAppend, "aduCapturedVertical":"%L6%"`n} ,%aduFile%
}

The generated JSON file:

{"aduAdvisory":"",
"aduMessage":"NO ENGAGEMENT ON GROUND",
"aduArmedLateral":"LOC",
"aduArmedVertical":"ALT",
"aduCapturedLateral":"HDG SEL HI",
"aduCapturedVertical":"VS --- FPM"
}

There are some more files involved regarding WebSimData. This will surely be documented by Marcin once his new versin is released.

 

[WebSimConnect]

Very nice ! WebSimSocket is good alternative for FSX-HTML communication.

One remark I forgot to mention. If you set Frequency=0 in WebSimData.ini file, the socket message will be send ONLY when the FSX variable changes. IT is very handy for variables that do not change at high frequencies.

This will surely be documented by Marcin once his new versin is released.

True, I am reworking a bit SDK, documentation, examples. I will also release a setup program which will install the engine and examples without the need to do any manual configuration.

One of the feature that you might be interested in for this or any other project is that WebSimXML component (2.1 version) will allow to capture socket messages from HTML5 code and execute XML code (i.e. execute_calculator_code function will exposed for JavaScript code), for example in javaScript you would do:

websocket.send ('WebSimXMLCode:(A:Plane heading degrees gyro,degrees) (>K:HEADING_BUG_SET)'); }