Global Wildfires Open Source Project - Need Programming

[=Hollywood=]

Ever since FSX beta back in 2006 I've been longing to develop a freeware FSX addon that is very easy to do, and provides some pretty incredible experiences in the sim.

There's active global wild fire detection data (24 hr, 48 hr, and 7 day in KML, SHP and CSV formats) at https://earthdata.nasa.gov/data/near-real-time-data/firms/active-fire-data that can be easily used to place wildfire effects in FSX with flames and fire.

Jim Keir years ago grabbed some open source code and helped write a script that can convert point data to object/fx placement in an FSX/P3D bgl. Unfortunately the KML format has changed since 2006 and I don't know how to re-tool it for the new KML files.

The same tool can be used to place scenery objects from KML point files.

What I'm needing is someone who can code a small program that can pull the source KML (or CSV or SHP) file from the website every day (can be a scheduled task) and then run bglcomp.exe over the resulting placement file and save the BGL into one of the FS scenery directories.

Basically it can automate the placement of global wildfires in sim that respond to wind direction and strength. If it's automated (or even has a weather update simconnect module) then every day in sim will be a new experience. Automatic dynamic scenery.

Further to this, the project could be expanded to placement of icebergs or even NOAA buoys that have their lat/lon updated every day.

Arno, I'm thinking scenproc could be used for part of this quite easily or something similar if it could be run as a service.

Cheers,
Dean.

 

[ollyau]

Does scenProc handle this sort of thing already?

I'd be glad to try writing something (should be simple enough), but if there are already tools, I see no need to create duplicate functionality.

 

[=Hollywood=]

Scenproc could be automated through command line, but can it place FX files via bglcomp.exe? This is something I want end users to be able to either set and forget or run a shell that downloads the data and preps it for FSX/P3D every day before running FSX, basically like the weather system that pulls weather data, but for fires. Most people won't want to set up programming to make it work.

 

[ollyau]

Ah, okay. Just wanted to check before I embark on something potentially redundant.

I'm thinking it could be even better to use SimObjects for the fires, icebergs, buoys, etc., as they can be placed and updated while the simulation is running (through SimConnect). I don't have any model GUIDs or SimObject titles, so for now I'll just make a proof of concept to download the data and parse the SHP file.

 

[=Hollywood=]

That's something even more in the direction I want to head in. I have dreamed of being able to manipulate objects in sim via simconnect like we used to be able to do ala FS5-FS2002 with dynamic scenery.

If what you're saying is possible for still objects, do you think it would be possible to expand it to moving objects?

e.g. we take the animated elephant loop MDL in FSX and assign a GPS waypoints, speed and possibly even time of day variables for it to run.

Basically an AI traffic system for MDL objects.

I have a couple of dreams:

a) make moving animals possible in regular (non-mission mode)
b) make moving satellites (2 LOD's lowest LOD is basically one triangle that you can see objects moving through space at night time, the highest LOD is for Space Missions - which would need a simconnect module of itself to add spaceflight capability in FSX/ESP/P3D)

These things have been concepts for a while, I just can't believe no one else has done something like this yet.

Cheers,
Dean.

 

[=Hollywood=]

These are some of my earlier test photos. I now figured out how to do another test via scenproc without using my old tool, but I have to manually tweak the intermediate .xml file

 

[pfabian]

Hello,

there are two projects with SimObject placement in development.

Perhaps you will find that one or the other works better for you - this seems like a nice effort, so I wish you luck!

[FSX] papaPhi objectPlacer 2

and

[FSX] SimObject Display Engine

 

[arno]

Hi,

scenProc doesn't write XML effects at the moment, but I already chatted about it with Dean and I am going to add this feature (actually I am already testing it while typing this). I think it's a nice feature to have.

 

[=Hollywood=]

Arno you're the best. scenproc is the best thing that has happened to scenery development ever! Now I'm off to try and break it again with nearly a million polygons and it looks like I've compiled the OSM master and SPC files correctly, should be done by the time I wake up...

post script: looks like I need more RAM, scenProc has maxxed out at 6GB of RAM (I only have 8GB), maybe moving up to 16GB might be what I need

 

[arno]

Hi Dean,

I have added the new step to make effects now. The arguments it takes are almost the same as the step to make XML objects. You'll find it in the next development release. If you are missing certain arguments, just let me know.

In the scenProc editor there is auto completion for the effect names as well. It will scan the available effects in the FS installation.

About the RAM, for big areas scenProc can be a bit memory hungry. Are you importing OSM files? With IMPORTOSM you can only load the tags you need, that might save some RAM. I will add a similar filtering to IMPORTOGR soon as well.

 

[=Hollywood=]

Hi Arno. That sounds great. I'll push today's fire data through it.

As for the vector stuff I was pushing about 1GB of source data. All of it I'm afraid is only the data I'm processing. Was very fast until my available ram got maximized. Time for me to upgrade if I'm going to keep pushing super computer amounts of data through my system.

Scenproc worked like a gem tho!

 

[arno]

Hi Dean,

If you have ideas on how to split the processing of big chunks of data into multiple parts please let me know. Processing big quantities of data efficiently is probably a feature that can be improved

 

[=Hollywood=]

Hi Arno, about the only thing I can think of is for it to split the AGN files first, which is what it's already doing right?

Honestly at this point in history scenproc is the fastest thing out there to process this level of data, my GIS tools are taking far longer on similar tasks.

 

[BASys]

Hi Folks

Dean -
Don't think prior bgl compilation is/was required, try injection instead

Have a look at FS-Freeflow's FS-Blaze
It was written for FS9, ut not sure if it worked in FSX.

Scott G. or Ron Jeffers may be able to provide some assistance/pointers.

MS-Flight had similar native, but not activated.



EDIT -
JK wrote a module to conditionally place objects & effects in FSX.

IIRC, it, or a subset thereof,
is incorporated in at least one commercial scenery package,
for placement of icebergs/icesheets.


Satellites are already possible.
I'm just not capable of converting TLMs into live injectable data.



A belated Season's Greetings & A Happy New Year to everyone.
(Been away/offline since mid-Dec).



HTH
ATB
Paul

 

[=Hollywood=]

Thanks Paul, I'll gave to contact Jim again and see what is/isn't possible today.

Satellites could probably still be possible to inject but would need a module running that could control them, Tom Main knows a lot about that stuff.

Will keep pressing on till I get success with these concepts. I want to make the base sim come alive and anything is possible, we just have to get creative.

P.S. thanks for the heads up about FS Blaze, never heard of it before but yeah it's exactly what I'm trying to accomplish, but with global scale data

 

[ollyau]

Also this (full PrtScn, so it's large): http://imgur.com/wJUPEqp.

Seems to work.

The GUI is just there so I can test stuff. I'd make it more streamlined for release versions if anyone's interested.

 

[=Hollywood=]

That looks awesome , I was using FX_ForestFireHuge I think for my tests. Could users customize the effects call for it in case they generate their own effects?

I noticed the drop down option too, can it also pull the global dataset? The interface looks great.

Also is it pulling the 24 hour SHP file?

Oh I just had a thought too, I need to track down the source, but there's also an active volcano dataset somewhere (I think also MODIS) that gives locations of currently erupting volcanos. Those could also be good for the same thing as the active fires.

I'll send you a PM with some other thoughts

One concept if we can find data available is near real time tornado data. I remember someone made up an effect in a previous version of FS, imagine if that could be integrated as well in future if we could track down a suitable dataset.

 

[ollyau]

Thanks.

I'm using Cntrl_ForestFireSingle, which calls fx_ForestFireHuge and fx_ForestFireSmoke. I was unable to add the effect to a SimpleObject, Boat, or GroundVehicle, so I asked a friend to make a model with the effect built in using an attach point. So no, users will not be able to customize the effect (unless they replace Cntrl_ForestFireSingle, I suppose).

The drop down menu contains entries for all 24 hour shapefile datasets on the NASA page you linked to. However, the regions don't seem to be bounded properly -- the Canada data I downloaded contained fires in California. This is evident in the KML files as well.

My program could be pretty easily modified for placing other SimObjects at other shapefile points, so I'd be glad to try out additional data if you have any handy.

 

[=Hollywood=]

ollyau that all sounds awesome, I'm just wondering why you attached the effet to an object or is that because you need to call a GUID to place the effect via simconnect?

it's looking great!

 

[ollyau]

Not a GUID, but a title. SimConnect has an API call, SimConnect_AICreateSimulatedObject, which allows you to create non-aircraft AI based on reference to the SimObject's title in its sim.cfg file. SimConnect doesn't actually provide a method to add effects directly to the simulator like the SDK's special effects tool does.

My screenshot of the code is actually somewhat misleading -- the ModelGuid property is unused, and I had forgotten to remove it. I was hoping to take an approach like this if I could find a suitable scenery library object, but it didn't happen. SimpleObjects and Boats don't seem to support effects in sim.cfg files, and while GroundVehicles seem to, I crashed the simulator in testing. There's also the Liquid type SimObject (the only example of which is the WaterDrop in SimObjects\Misc), which seems to be exactly what I need -- a simple SimObject with an attached effect, but they're destroyed immediately after they're created. This left me the last option of attaching the effect to a model, which did in fact work.