P3D v5 Questions regarding the TF2 method and its steps

[Clutch Cargo]

I am finally immersing myself into the new Texture Filter Editor. Reading the manual frontwards and backwards to get a “layman’s” understanding of such a powerful tool. I am starting out just with vegetation detection. Water will come later. But I have some starting questions that are perhaps more for clarification to get stared:

1. If I recall correctly in our past conversations, you are now adding the capability of adding a 2nd image via the ADD step. So in my case I have first added a 3-band natural color image and now use the ADD step to add the infrared image?

2. In the past I would add up to 10 samples of natural color images. Is this no longer the case? I add 1 image and 1 matching IR image?

3. If that is correct, could I still add 5 natural color images and 5 infrared images to get a wider sample?

4. and if question #3 is true, Do NC and IR samples need to match exactly in size as well as the IR match their NC counterpart?

5. I used to use .bmp samples. Sounds like you suggest now using tiffs? Not a problem but if I know my bmps are 3-band does it matter? Do NC and IR samples need to all be the same file type like all tiffs or all bmps? ( I will probably try switching all samples over to tiffs).

6. Regarding the ‘Split into 3 Images’ and ‘Split into 4 Images’ steps, do the 3 and four exit side nodes represent R,G,B, IR and from top to bottom in that order? Speaking of which what is a 5-band image you mention in the manual?

7. If I have added 3-band image and an infrared image, do I first add a MERGE step and then use a SPLIT step in order to process correctly from that point forward?

Lot's of questions I know but this is pretty technical for me and really could use clarification. In the mean time I am starting to experiment!

thx

 

[arno]

Hi,

1. If I recall correctly in our past conversations, you are now adding the capability of adding a 2nd image via the ADD step. So in my case I have first added a 3-band natural color image and now use the ADD step to add the infrared image?

Instead I have added the MergeRasterFeatures step, so that way you can merge them in memory and feed the combined raster image to the texture filter. That way the texture filter does not need a second input.

2. In the past I would add up to 10 samples of natural color images. Is this no longer the case? I add 1 image and 1 matching IR image?

In the texture filter editor I would still add as many samples as you need to get the filter working. I think you would need more than 1 for sure.

Also, if you use the MergeRasterFeatures step you will have to make sure that you create some merged samples images yourself (and use ExportGDAL to write them to disk). I typically use a script that makes random samples over the entire area. When running from the texture filter editor the samples are used as input, so they would need to be similar to the data fed when running from the script in production.

3. If that is correct, could I still add 5 natural color images and 5 infrared images to get a wider sample?

See answers above, your samples would be combined already. The merging is done before the texture filter, not in the texture filter.

4. and if question #3 is true, Do NC and IR samples need to match exactly in size as well as the IR match their NC counterpart?

Answer to 3 is false. But when using the MergeRasterFeatures step you indeed need to make sure that the different images you are merging match exactly in size.

5. I used to use .bmp samples. Sounds like you suggest now using tiffs? Not a problem but if I know my bmps are 3-band does it matter? Do NC and IR samples need to all be the same file type like all tiffs or all bmps? ( I will probably try switching all samples over to tiffs).

Which format you use for the samples does not really matter. You can use GeoTIFF, PNG or BMP. If you are using vector data in your script as well you would need to use a geo-referenced format like GeoTIFF.

6. Regarding the ‘Split into 3 Images’ and ‘Split into 4 Images’ steps, do the 3 and four exit side nodes represent R,G,B, IR and from top to bottom in that order? Speaking of which what is a 5-band image you mention in the manual?

Yes, the output connectors are the 1st, 2nd, 3rd and 4th band of the image from top to bottom. What they contain depends on the raster data you use of course. For RGB images it will be R, G and B indeed. For 4 band images with a NIR band it will be R, G, B and NIR. But if you use CIR images it will be NIR, R and G.

You can make a 5 band image in the texture filter as well, if you combine color data with a blend mask and a water mask. You can feed this as a 5 band image into resample. But it is not required to use that, you can use separate images for the watermask and blendmask as well.

At the moment I am experimenting with using machine learning in the texture filter, for that feature I will most likely use raster data with even more bands than 5. But that is still work in progress, so I am not 100% sure how it will work out yet.

7. If I have added 3-band image and an infrared image, do I first add a MERGE step and then use a SPLIT step in order to process correctly from that point forward?

Like I explained before the merging would be done outside of the texture filter in your script already. As for the Split steps, it depends on your workflow if you need them. To calculate NDVI or NDWI you don't need to access the individual bands. But if you want to use extra conditions that need a specific band you might have to put a Split step in your script.

Maybe instead of focusing on if you need to use a specific step or not, can you explain what you want the detection process to do? That typically determines which steps you need.

 

[Clutch Cargo]

Thx for the detailed answers. So it is very clear now I need to provide the merging of my 3-band and IR images. I too have a script that creates samples so I will look into ExportGDAL to create a combined image that I would then use as my sample(s).

You asked "can you explain what you want the detection process to do?"

What I want the detection process to do is 1) create vegetation detection from images and 2) create waterpolys/watermasks from images. With your new release I am first concentrating on vegetation. Once I get a grasp on that I will immediately move into water detection. For my Chico, California project I used your example from Figure 6.20 literally from the manual. I believe this is what you call the NDVI process. Simple, straightforward steps in a line. For me, is was a good way to get familiar with the TFE process. Currently, I did not use 4-band samples. I was using 3-band and the results were surprising great for not utilyzing infrared.

With a new project I am starting, it appears the results are not a good (different geographic area and images). So I am going to try the 'Back Process', which looks like an infrared band will provide better results. Hence, all my questions on IR steps and did that change in the new release. Your answers allow me to now proceed. So I will use your example in Figure 6.9 to start with.

 

[arno]

Hi,

Instead of just saying I want to detect vegetation, I more meant that you try to describe which characteristics of the image will you be trying to capture to determine that something is vegetation. With the sample from the manual you are indeed using a NDVI value and then doing some erode/dilate to get rid of small noise. But often you also get some grass areas in that case. So in most of my projects I have additional steps to try to filter that grass out again. For example by looking at the standard deviation of one of the color bands, as trees are less uniform than grass. But there are other approaches possible as well.

The NDVI step really needs a NIR band to work. You can run it on CIR images or on 4-band images with a NIR band. Both will give the same result.

What the back project step does is compare the histogram of the image with the histogram of sample bits. When they are similar it will give a hit. This can be done on images without a NIR band as well.

 

[Clutch Cargo]

I figured my response was a bit too simplistic, but I am not too sure I can answer (at least not yet) or truly understand "which characteristics" you are asking about? I know the issues I am seeing, exactly what you posted above... I am get unwanted vegetation on sports fields, grassy lands, crops (which I mask out). You say you use additional steps to filter grass out again. What are those steps I could try?

I also need to get my 3-band and IR merged, that could solve a lot of my issues I presume. Is there a small app that does this or photoshop maybe? I have been so rushed on a current project I haven't even investigated. Right now, I am just "cleaning up" by hand those grassy areas due to time restraints over the next week. Then I can relax a bit and research.

marcus