Autogen from OpenStreetMap data with scenProc

This tutorial describes how you can use the data from OpenStreetMap to create autogen for FSX. We'll cover all the steps, including downloading and selecting your data, how to configure the tools to create the autogen and how it finally looks in FSX.

In this tutorial I will be using the area around Luxembourg city as an example. And just so that you can see the difference, below is a picture of the scenery without any autogen in it. In the end of this tutorial you will have populated that area with autogen.

Required tools

You will need a couple of tools to create autogen from OpenStreetMap data. This section discusses the tools you need.

GIS editor

We will be using ESRI shapefiles, which is a common format to store GIS vector data. Therefore you will need a tool to view and edit such shapefiles. In this tutorial I will be using the open source QGIS tool, but any other GIS editor should also work fine. A good way to install QGIS is to use OSGeo4W, which includes other useful GIS tools as well.

scenProc

scenProc is the tool that will create the AGN autogen files for FSX from the GIS vector files. Here you can find the manual and information on how to download scenProc.

Text editor

The configuration file for scenProc is just a plain text file. So you will need a text editor to create and edit this file. Standard notepad will be fine, but of course you can use any editor of your choice.

Downloading the data

Our first step is to get the data we will work with. We will be using OpenStreetMap data. On their own website you can view the data and you can also download it in their own OSM format (which is XML). But that format is not so easy to process with GIS tools. Therefore we will download a shapefile version of the data.

Such shapefiles can be found locations on the internet, but we will be using those made by GeoFabrik. From their website you can download shapefiles of the OpenStreetMap data organised per country. For some big countries the data is split into different packages per region. In this case I will download the Luxembourg file.

After you unzip the file you downloaded, you should see shapefiles with the following names:

• buildings
• landuse
• natural
• places
• points
• railways
• roads
• waterways

Each shapefile consists of a couple of files, so you will see files with the extensions shp, shx, dbf and prj. You should always keep these files together, as they are all needed for tools to be able to read the shapefile.

The names give a good indication of what you will find in the different shapefiles. In general this is what you can find in each of them:

• buildings: polygons with the shape of buildings
• landuse: polygons with landuse information, so where are residential or industrial areas for example
• natural: polygons with natural information, for example forests and parks
• places: points with place names
• points: all kind of other points of interest
• railways: lines for the railways
• roads: lines for roads
• waterways: lines for rivers

For the autogen we will mainly be using the shapefiles with buildings, natural and points. After loading these three shapefiles into QGIS you should see something like this:

Selecting the data you need

In most cases you would not want to process the vector data of an entire country at once. So then you need to select the data you want to use. For this tutorial I will select the area around Luxembourg city. So the first step is to zoom in on the area you want to use in QGIS.

Make sure you have the natural layer selected. Then use the "Select features by rectangle" tool to select all features on your screen. Then right click on the layer and use "Save selection as" to save the selected features to a new shapefile. The image below shows the menu.

You need to repeat this step for each layer (so natural, buildings and points). It's best to save the resulting shapefiles in a new folder. So in this case we end up with three shapefiles for the area of Luxembourg city. The image below shows how these resulting files look in QGIS.

Creating vegetation

So now it is time to start creating some autogen. To do this we need to create a configuration file for scenProc that tells the tool what we want to do. So start your text editor and create a file with the following content:

 # Luxembourgh City autogen
 IMPORTSHP|natural.shp|NOREPROJ
 #
 SPLITGRID|AGN
 #
 SETAGNBUILDINGHEIGHT|1.0;0.0;0.0;0.0
 CREATEAGNPOLYVEG|FTYPE=POLYGON;type=forest|{e8b937fd-a1f2-4bd5-8548-2c80d30102af}
 #
 WRITEAGNFILES|C:\flightsim\belgium_photo\texture

All lines starting with # are a comment. We'll walk through the other lines now and explain what they do.

IMPORTSHP tells the tool that it needs to read data from a shapefile. The first argument is the name of your shapefile, since we want to make vegetation now we will load the file natural.shp. The second argument can be used to reproject your data to WGS84, but the OpenStreetMap data we downloaded already is in that projection. So therefore we use NOOREPROJ as option.

The line SPLITGRID tells scenProc that it needs to split all features in a smaller grid. The option AGN means that the tool will automatically use the correct grid size for autogen. This size corresponds with the LOD tile size of the terrain scenery.

The next line, SETAGNBUILDINGHEIGHT, sets some building height properties for the AGN file. Since we make vegetation here the values are not important. But don't forget this line, else the autogen created will not work!

The line CREATEAGNPOLYVEG does the real work. It takes two arguments, one is the filter and the other is the vegetation GUID. How these two arguments work I will discuss below.

The final line, WRITEAGNFILES tells scenProc where it needs to write the actual AGN files. So you should provide the texture folder of your photo scenery there.

And that's all, now you can run this file through scenProc and your autogen will be created. It is best to run it from the command prompt, as you will see the feedback from the tool in that case. So go to your command prompt and type:

 scenProc.exe lux.spc

You probably need to provide the full path to where you installed scenProc and you need to use the name of your configuration file of course. Then the tool starts working and that might take quite some time if you are processing a lot of data. But once it is finished you should have the AGN files in your texture folder and when you startup FSX you should see the vegetation autogen. Just like the image below:

So that looks very nice, but you might be thinking we went over the hard work in the CREATEAGNPOLYVEG step a bit quick. So let's have a closer look at how that step works. As exampled above it takes two arguments, a filter and a GUID. These will be explained in more detail here.

The filter is used to select from which vector features you want to create the vegetation. See the [scenProc] manual for details on the syntax of the filter. What we want to do here is select all polygons that are forest. We'll use QGIS to figure out which attributes the vector data has. Actually there are multiple way to do that, what I prefer to do is render the features based on some attribute.

To do that double click on the layer, this will bring up a properties screen. In that screen you select "Categorized" and the the column "type". This means you want to render them based on the value in the type attribute. If you then press the "Classify" button you will see the different values that are in the file and each of them is given a different colour. The images below shows the results for the natural layer we are using.

So as you can see the attribute type has a value of forest that indicates the polygon is a forest. Those are the polygons we want to use for our autogen. The polygons that are park, riverbank or water we don't want to use.

So that brings us back to the filter string. We want to tell scenProc to use all polygon features, where the attribute type is equal to forest. We tell that to the tool by typing:

 FTYPE=POLYGON;type=forest

Now how to find the GUID for your vegetation? This can be a bit tricky, you'll need the Autogen SDK for this. In the folder of the Autogen SDK you will find a folder called "Autogen source xml" which contains a number of XML files. For the vegetation open the file AutogenDescriptions.xml and inside it you can find the GUIDs of the different kind of autogen you can use.

The document is full of GUIDs however, because most autogen is made up of library objects which are also identified by GUIDs. So it can be a bit confusion. But after searching through this file I found an autogen group called "Terrain Autogen Class Small Leaf Mixed Woods". And it's the GUID of that group that I put in the configuration file to tell FSX I want to use these trees for my forest. Feel free to choose another GUID if you want different kind of trees of course.

Creating library objects

So now we have the forests as autogen, but we want more. We also want to place some library objects. Similar to the way we looked at the natural shapefile attributes, you can also examine the points shapefile. You will see that it has some point features with the type water_tower. Let's put an autogen water tower at those locations!

Below is the extended configuration file. We now load a second shapefile, points.shp. And another line has been added to create autogen library objects.

 # Luxembourgh City autogen
 IMPORTSHP|natural.shp|NOREPROJ
 IMPORTSHP|points.shp|NOREPROJ
 #
 SPLITGRID|AGN
 #
 SETAGNBUILDINGHEIGHT|1.0;0.0;0.0;0.0
 CREATEAGNPOLYVEG|FTYPE=POLYGON;type=forest|{e8b937fd-a1f2-4bd5-8548-2c80d30102af}
 CREATEAGNLIBOBJ|FTYPE=POINT;type=water_tower|{450e3946-44ec-6111-95e1-56baa89b3d82}
 #
 WRITEAGNFILES|C:\flightsim\belgium_photo\texture

The line CREATEAGNLIBOBJ has two arguments again. The first is the filter to select which features the library objects should be created from. In this case we select all points, where type is equal to water_tower. The second argument is the GUID we want to use. Finding the GUID is similar to the vegetation GUID. In this case we need to look in the file default.xml, which is in the same folder of the SDK. In it you will find a autogen class called "Urban Tall Water Tanks 15x15" and it is the GUID of that class that we used here.

It is important to remember that you can't use the GUID of a specific library object directly. You need to use a GUID of a autogen class of library objects here.

After we run scenProc again with this updated configuration file you will see additional objects in FSX, as shown in the picture below. The water tower does not look that special, but at least it demonstrates how to add library objects.

Creating buidlings

TODO

Areas for improvement

TODO