Autogen from OpenStreetMap data with scenProc

From FSDeveloper Wiki
Jump to: navigation, search

This tutorial describes how you can use the data from OpenStreetMap to create autogen for FSX. We'll cover all the steps, including downloading and selecting your data, how to configure the tools to create the autogen and how it finally looks in FSX.

In this tutorial I will be using the area around Luxembourg city as an example. And just so that you can see the difference, below is a picture of the scenery without any autogen in it. In the end of this tutorial you will have populated that area with autogen.

ScenProc Lux FSX before.jpg


Required tools

You will need a couple of tools to create autogen from OpenStreetMap data. This section discusses the tools you need.

GIS editor

We will be using ESRI shapefiles, which is a common format to store GIS vector data. Therefore you will need a tool to view and edit such shapefiles. In this tutorial I will be using the open source QGIS tool, but any other GIS editor should also work fine. A good way to install QGIS is to use OSGeo4W, which includes other useful GIS tools as well.

scenProc

scenProc is the tool that will create the AGN autogen files for FSX from the GIS vector files. Here you can find the manual and information on how to download scenProc.

Downloading the data

Our first step is to get the data we will work with. We will be using OpenStreetMap data. On their own website you can view the data and you can also download it in their own OSM format (which is XML). But that format is not so easy to process with GIS tools. Therefore we will download a shapefile version of the data.

Such shapefiles can be found locations on the internet, but we will be using those made by GeoFabrik. From their website you can download shapefiles of the OpenStreetMap data organised per country. For some big countries the data is split into different packages per region. In this case I will download the Luxembourg file.

After you unzip the file you downloaded, you should see shapefiles with the following names:

  • buildings
  • landuse
  • natural
  • places
  • points
  • railways
  • roads
  • waterways

Each shapefile consists of a couple of files, so you will see files with the extensions shp, shx, dbf and prj. You should always keep these files together, as they are all needed for tools to be able to read the shapefile.

The names give a good indication of what you will find in the different shapefiles. In general this is what you can find in each of them:

  • buildings: polygons with the shape of buildings
  • landuse: polygons with landuse information, so where are residential or industrial areas for example
  • natural: polygons with natural information, for example forests and parks
  • places: points with place names
  • points: all kind of other points of interest
  • railways: lines for the railways
  • roads: lines for roads
  • waterways: lines for rivers

For the autogen we will mainly be using the shapefiles with buildings, natural and points. After loading these three shapefiles into QGIS you should see something like this:

ScenProc Lux OSM data in QGIS.jpg

Selecting the data you need

In most cases you would not want to process the vector data of an entire country at once. So then you need to select the data you want to use. For this tutorial I will select the area around Luxembourg city. So the first step is to zoom in on the area you want to use in QGIS.

Make sure you have the natural layer selected. Then use the "Select features by rectangle" tool to select all features on your screen. Then right click on the layer and use "Save selection as" to save the selected features to a new shapefile. The image below shows the menu.

ScenProc Lux OSM QGIS save selected.jpg

You need to repeat this step for each layer (so natural, buildings and points). It's best to save the resulting shapefiles in a new folder. So in this case we end up with three shapefiles for the area of Luxembourg city. The image below shows how these resulting files look in QGIS.

ScenProc Lux OSM QGIS selected only.jpg

Creating vegetation

So now it is time to start creating some autogen. To do this we need to create a configuration file for scenProc that tells the tool what we want to do. So start scenPRoc and enter the following content:

 # Luxembourg City autogen
 IMPORTSHP|natural.shp|NOREPROJ
 #
 SPLITGRID|AGN
 #
 CREATEAGNPOLYVEG|FTYPE=POLYGON;type=forest|{e8b937fd-a1f2-4bd5-8548-2c80d30102af}
 #
 WRITEAGNFILES|FSX|C:\flightsim\belgium_photo\texture

All lines starting with # are a comment. We'll walk through the other lines now and explain what they do.

IMPORTSHP tells the tool that it needs to read data from a shapefile. The first argument is the name of your shapefile, since we want to make vegetation now we will load the file natural.shp. The second argument can be used to reproject your data to WGS84, but the OpenStreetMap data we downloaded already is in that projection. So therefore we use NOREPROJ as option.

The line SPLITGRID tells scenProc that it needs to split all features in a smaller grid. The option AGN means that the tool will automatically use the correct grid size for autogen. This size corresponds with the LOD tile size of the terrain scenery.

The line CREATEAGNPOLYVEG does the real work. It takes two arguments, one is the filter and the other is the vegetation GUID. How these two arguments work I will discuss below.

The final line, WRITEAGNFILES tells scenProc where it needs to write the actual AGN files. The first argument is the FS version you want to make autogen for (FSX in this case). The second argument is the folder to save the AGN file, so you should provide the texture folder of your photo scenery there.

And that's all, now you can run this file through scenProc and your autogen will be created. Just press the run button and wait for the processing to finish. Once it is finished you should have the AGN files in your texture folder and when you startup FSX you should see the vegetation autogen. Just like the image below:

ScenProc Lux FSX vegetation.jpg

So that looks very nice, but you might be thinking we went over the hard work in the CREATEAGNPOLYVEG step a bit quick. So let's have a closer look at how that step works. As exampled above it takes two arguments, a filter and a GUID. These will be explained in more detail here.

The filter is used to select from which vector features you want to create the vegetation. See the [scenProc] manual for details on the syntax of the filter. What we want to do here is select all polygons that are forest. We'll use QGIS to figure out which attributes the vector data has. Actually there are multiple way to do that, what I prefer to do is render the features based on some attribute.

To do that double click on the layer, this will bring up a properties screen. In that screen you select "Categorized" and the the column "type". This means you want to render them based on the value in the type attribute. If you then press the "Classify" button you will see the different values that are in the file and each of them is given a different colour. The images below shows the results for the natural layer we are using.

ScenProc Lux QGIS natural layer prop.jpg

ScenProc Lux QGIS natural.jpg

So as you can see the attribute type has a value of forest that indicates the polygon is a forest. Those are the polygons we want to use for our autogen. The polygons that are park, riverbank or water we don't want to use.

So that brings us back to the filter string. We want to tell scenProc to use all polygon features, where the attribute type is equal to forest. We tell that to the tool by typing the line below. Be aware that the attributes and values are case sensitive!

 FTYPE=POLYGON;type=forest

Now how to find the GUID for your vegetation? The easy way is to just start typing the friendly name of the type of vegetation you want. The auto completion feature of scenProc will then show you the possible types of vegetation and will automatically fill in the GUID for you.

The harder way is to find the information in the Autogen SDK. In the folder of the Autogen SDK you will find a folder called "Autogen source xml" which contains a number of XML files. For the vegetation open the file AutogenDescriptions.xml and inside it you can find the GUIDs of the different kind of autogen you can use.

The document is full of GUIDs however, because most autogen is made up of library objects which are also identified by GUIDs. So it can be a bit confusion. But after searching through this file I found an autogen group called "Terrain Autogen Class Small Leaf Mixed Woods". And it's the GUID of that group that I put in the configuration file to tell FSX I want to use these trees for my forest. Feel free to choose another GUID if you want different kind of trees of course.

Creating library objects

So now we have the forests as autogen, but we want more. We also want to place some library objects. Similar to the way we looked at the natural shapefile attributes, you can also examine the points shapefile. You will see that it has some point features with the type water_tower. Let's put an autogen water tower at those locations!

Below is the extended configuration file. We now load a second shapefile, points.shp. And another line has been added to create autogen library objects.

 # Luxembourgh City autogen
 IMPORTSHP|natural.shp|NOREPROJ
 IMPORTSHP|points.shp|NOREPROJ
 #
 SPLITGRID|AGN
 #
 CREATEAGNPOLYVEG|FTYPE=POLYGON;type=forest|{e8b937fd-a1f2-4bd5-8548-2c80d30102af}
 CREATEAGNLIBOBJ|FTYPE=POINT;type=water_tower|{450e3946-44ec-6111-95e1-56baa89b3d82}
 #
 WRITEAGNFILES|C:\flightsim\belgium_photo\texture

The line CREATEAGNLIBOBJ has two arguments again. The first is the filter to select which features the library objects should be created from. In this case we select all points, where type is equal to water_tower. The second argument is the GUID we want to use. Finding the GUID is similar to the vegetation GUID. The auto completion feature will help you to find it. Or else you can look it up in the file default.xml, which is in the same folder of the SDK. In it you will find a autogen class called "Urban Tall Water Tanks 15x15" and it is the GUID of that class that we used here.

It is important to remember that you can't use the GUID of a specific library object directly. You need to use a GUID of a autogen class of library objects here.

After we run scenProc again with this updated configuration file you will see additional objects in FSX, as shown in the picture below. The water tower does not look that special, but at least it demonstrates how to add library objects.

ScenProc Lux FSX libraryobjects.jpg

Creating buidlings

Now that we got vegetation and library objects, the final part is of course to add some buildings. So for this we will load the buildings.shp file as well and add some lines to create autogen from those features. The updated configuration file is shown below.

 # Luxembourgh City autogen
 IMPORTSHP|natural.shp|NOREPROJ
 IMPORTSHP|buildings.shp|NOREPROJ
 IMPORTSHP|points.shp|NOREPROJ
 #
 SPLITGRID|AGN
 #
 SETAGNBUILDINGHEIGHT|1.0;0.0;0.0;0.0
 CREATEAGNPOLYVEG|FTYPE=POLYGON;type=forest|{e8b937fd-a1f2-4bd5-8548-2c80d30102af}
 CREATEAGNGENBUILD|FTYPE=POLYGON;FROMFILE=buildings.shp;FWIDTH<20|{5ae04eb6-934c-4f63-bb48-5e7dee601212}|MAXRATIO=2
 CREATEAGNGENBUILD|FTYPE=POLYGON;FROMFILE=buildings.shp;FWIDTH>20|{6089A0BD-CED1-4c47-9A9E-64CDD0E16983}
 CREATEAGNLIBOBJ|FTYPE=POINT;type=water_tower|{450e3946-44ec-6111-95e1-56baa89b3d82}
 #
 WRITEAGNFILES|C:\flightsim\belgium_photo\texture

The step SETAGNBUILDINGHEIGHT specifies the height distribution of the autogen buildings. In this case we use only buildings with 1 or 2 floors. Depending on the kind of area you are modelling you might want to include taller buildings as well.

The CREATEAGNGENBUILD step has the two familiar parameters again, the first being the filter and the second the GUID. In this case you can see that we filtered by selecting all polygon features that come from the file buildings.shp. An additional condition is that based on the width of the building we select a different GUID for the roof style. So if the width is bigger than 20 meters we will use a flat roof and else we will use a gabled roof. That's also why there are two lines in the configuration file for the autogen buildings.

And where did we find the GUID? The auto completion function is our best friend again. Or if you want to do it manual you might have expected to find it in that same folder in the SDK. This time I searched in the file RoofDescriptions.xml. I found a group called "Roofs Gabled _ALL_" there, which we use for the gabled buildings.

For the flat buildings I found the GUID in the same file, there is a group called "Grouping of all flat roofs". In this case I did not use the group GUID, but the GUID of one of the roofs in the group. I did that because many flat roofs have a small wall and that does not look realistic for a European city.

You might notice that the line to create the gabled roofs has a third argument, MAXRATIO=2. That is an option to tell scenProc that it should split very long and narrow buildings. So if there is a building footprint that is left than 20 meter width, but more than 40 meters in length it will be split into seperate buildings. That's because these objects are quite likely to be a row of houses. Of course you can adjust the exact parameters to fit the building style of the area you are working in.

The image below shows the resulting buidlings in FSX.


ScenProc Lux FSX buildings.jpg

Areas of improvement

This tutorial has shown the basic steps to create autogen from OpenStreetMap data. But of course there are many areas where you could improve the results. Here are just a couple of ideas:

  • Change the SETAGNBUILDINGHEIGHT properties so that you get more taller buildings in the city, the values would need to be adjusted to the area you are making autogen for.
  • Use a custom building texture for your autogen, so that they look more realistic than the default.
  • If you shapefiles would have attributes for the type of forest, you could use these to select different kind of trees for different polygons. So that you can have deciduous and pine trees at the right location.
  • If your shapefiles are rich enough, you can add many more library objects. Just think of chimneys, lightpoles and all those kind of objects.

Of course there are also some limitations at the moment when it comes to the scenProc tool. Especially for making the autogen buildings I am still working on improving the tool. Most autogen buildings have to be rectangular, so that does not always give the expected results. I am improving the tool so that you can make even better looking autogen by combining rectangular and polygonal autogen buildings.

I hope this tutorial get's you started. If you have suggestions, questions or brilliant ideas related to the scenProc tool, please post them on the [forum]. The image below shows the final results achieved with the configuration file used in this tutorial.

ScenProc Lux FSX after.jpg