P3D v5 TF2 Questions and Suggestions

[Clutch Cargo]

I am now testing with the TFE using 4-band RGBN images for about a week. I have updated to the latest version of scenProc ( v3.3.0.0 rd8da14d8 DEV 10/23/22 ), and I am following thru the steps of the scenProc manual first testing vegetation detection using Back Projection.

My learning process is as follows:

• Read the manual – trying best to understand the concepts of each step.
• Use the examples in the manual as a default, a starting place for creating TFE scripts
• Try to get any result possible at first – just to get an output for starters.
• Work at understanding then impact of what happens to end result as I change a value in a step.

Questions:
1. Is there an advantage of using the Back Projection method over the NDVI method ( or visa versa), or is is more of just a choice of works best for that project?
2. I was under the assumption that a 4-band (RGBN) image would produce better results over a 3-band (RGB) image? That it may produce better fidelity in the results? However, in your example of Fiqure 6.9 it shows the step 'Split into 3 image'. I would have thought I would use 'Split into 4 image' for Back Projection?
3. Do you have to close or save/close the TFE before running scenProc script that has that TF2 step included? In scenProc one can make changes to a script and run it without having to save it first so thinking TFE does or does not work the same?
4. For clarification, the image samples are for me to make value changes among different samples and see the impact on that image. TFE is not considering all the images in the panel in some manner for writing the TF2 file UNLESS it is used as samples in the Back Projection panel (the right-side panel)?

Unhandled Exception errors -
I get a lot of these for various reasons. I may change a value and one appears or I may switch from one step icon to another and it appears. I attached six of them as I am not sure if they all relate to the same issue or have value for you to look at. Even though the error window says I may continue, I thnk that causes more issues so I just close scenProc and start again.

Suggestions -
I have noticed I switch back and forth dozens of times between a step and checking on the output step icon to see what changes are made. Then I go back again, make a value change and check again. May I suggest you break the center panel into two observation panels. The left panel would remain what you have, the “Preview Area” panel based on whatever step I am working on, but add one to the immediate right and have it always show the ‘Output Image’, what I would call the end results. This way, one can always see what the end results would look like as I make incremental changes to various steps. Like so:

For convenience, have the default preview panel fill its panel completely with the image. Then, if someone wishes to zoom in out, they still have that option.

For convenience, let the TFE retain panel size adjustments when closing so they open with the same sizes/dimensions.

In closing, I am getting good starting results with the NDVI method but no results using the Back Projection... getting 0 feature detection in scenProc. I will make that a separate topic. Overall really enjoying the TFE and what it can do.

 

[arno]

Hi,

1. Is there an advantage of using the Back Projection method over the NDVI method ( or visa versa), or is is more of just a choice of works best for that project?

It's really a choice. The NDVI step only works when you have a NIR channel. The back project option also works in you only have RGB bands, as it does a histogram comparison. So you can try both approaches. If I have a NIR band I prefer the NDVI one in general.

2. I was under the assumption that a 4-band (RGBN) image would produce better results over a 3-band (RGB) image? That it may produce better fidelity in the results? However, in your example of Fiqure 6.9 it shows the step 'Split into 3 image'. I would have thought I would use 'Split into 4 image' for Back Projection?

That sample is made to show what you can do with just RGB images, so it does not use a NIR band. The Split step is used to get the B band, as the standard deviation of the B band is used as an additional criteria to select the forest in that example. You could still use 4 bands for the back projection in that case. It really depends on the "detection rules" you want to code in your filter.

3. Do you have to close or save/close the TFE before running scenProc script that has that TF2 step included? In scenProc one can make changes to a script and run it without having to save it first so thinking TFE does or does not work the same?

All you need to do is save the TF2 file from the editor, then you can run it in your script. The editor can remain open. But the normal workflow of course is to first try the script in the editor, you do most of the tuning there before you run it on a bigger area in your script.

4. For clarification, the image samples are for me to make value changes among different samples and see the impact on that image. TFE is not considering all the images in the panel in some manner for writing the TF2 file UNLESS it is used as samples in the Back Projection panel (the right-side panel)?

Correct, the images are used to be able to test your filter in the editor. When loading the TF2 file in your script the images are not loaded into memory at all. The exception are the back project samples indeed, those are needed for the back project step to work. So those are loaded when running the script as well.

Unhandled Exception errors -
I get a lot of these for various reasons. I may change a value and one appears or I may switch from one step icon to another and it appears. I attached six of them as I am not sure if they all relate to the same issue or have value for you to look at. Even though the error window says I may continue, I thnk that causes more issues so I just close scenProc and start again.

I see three types of errors there:

• Null exceptions when clicking on the steps or connectors in the editor. Do you have a description what you did when they happen, then I can try to reproduce?
• Exceptions because back project sample images can't be found. Did they happen when you just added the sample to the filter or after loading the TF2 file back into the editor? I both cases the files should be there. Be aware that you should not have multiple instances of the editor open, that can give issues.
• Exceptions about invalid indices in the NDVI step. Are you use you have 4 band imagery loaded in that case?

Suggestions -
I have noticed I switch back and forth dozens of times between a step and checking on the output step icon to see what changes are made. Then I go back again, make a value change and check again. May I suggest you break the center panel into two observation panels. The left panel would remain what you have, the “Preview Area” panel based on whatever step I am working on, but add one to the immediate right and have it always show the ‘Output Image’, what I would call the end results. This way, one can always see what the end results would look like as I make incremental changes to various steps.

I see what you mean, but would that not make the screen very crowded? I often feel that the preview part is too small already now. Would you not be able to see if your change has the right effect by looking at the output of that step? That is what I normally do.

For convenience, have the default preview panel fill its panel completely with the image. Then, if someone wishes to zoom in out, they still have that option.

The default behavior now is to show the image at 1 pixel per pixel scale, I think that makes more sense than zoom or stretch to fill it.

For convenience, let the TFE retain panel size adjustments when closing so they open with the same sizes/dimensions.

That's a good idea. I'll see if I can add something like that.

 

[Clutch Cargo]

Wow, quick replay... many thx, Arno. In response to the above.

Regarding the additional output screen. I am using a 27" display. What I do is resize the panels to make the preview the biggest because once to have added samples you can shrink that panel, and the Properties panel is way to large for what it needs to make value changes so I shrink that down. Even after creating my step icon path on the bottom. I tend to make that just big enough to see all the steps. So leaves me with a large area in the center to have two preview displays. Hence, why I also ask to retain panel sizes. But I feel the advantages of two preview screens out weighs any smaller preview for those that have a smaller display... and then they can always use the zoom/pan option.

Ah, so the 'Unhandled Exception' errors are different. What I will do is track and note what I did just before one appears to be more useful for you.

Response to my question 1 - I had thought the opposite as I was getting really good results using NDVI with just 3-band images in the past. I think I will switch right now back to testing NDVI as I cannot seem to produce anything with Back Projection. NDVI seems more straightforward (and frankly easier), but then there's the trial-by-error part. The more I adjust to fill in forests it also adds veg to sports fields, golf courses and grassy areas. If I adust the other way I can clear up fields and sports fields quite nicely but I lose much of dense forests and the like. You had mention in another post, you had other Steps/Values you have used to minimize these issues. Are they different than your NDVI sample? Care to elaborate so I can try? 🙏

 

[arno]

Regarding the additional output screen. I am using a 27" display. What I do is resize the panels to make the preview the biggest because once to have added samples you can shrink that panel, and the Properties panel is way to large for what it needs to make value changes so I shrink that down. Even after creating my step icon path on the bottom. I tend to make that just big enough to see all the steps. So leaves me with a large area in the center to have two preview displays. Hence, why I also ask to retain panel sizes. But I feel the advantages of two preview screens out weighs any smaller preview for those that have a smaller display... and then they can always use the zoom/pan option.

Which resolution are you running? I'm using a 24 inch HD monitor here and on that the preview is always to small. Personally I feel that zooming the preview is very inefficient and makes it hard to see the right details. That's why I tend to keep the samples around 1000x1000 pixels so that they fit in the preview section.

Ah, so the 'Unhandled Exception' errors are different. What I will do is track and note what I did just before one appears to be more useful for you.

Thanks, any information on how/when they happened will help me to reproduce them.

Response to my question 1 - I had thought the opposite as I was getting really good results using NDVI with just 3-band images in the past. I think I will switch right now back to testing NDVI as I cannot seem to produce anything with Back Projection. NDVI seems more straightforward (and frankly easier), but then there's the trial-by-error part. The more I adjust to fill in forests it also adds veg to sports fields, golf courses and grassy areas. If I adust the other way I can clear up fields and sports fields quite nicely but I lose much of dense forests and the like. You had mention in another post, you had other Steps/Values you have used to minimize these issues. Are they different than your NDVI sample? Care to elaborate so I can try? 🙏

What I typically do in my script is indeed try to use other logic to get rid of the grass again. For example by using the standard deviation as you saw in that sample as well. Grass fields are more smooth than forest in general, so they have a different standard deviation. Or sometimes certain color bands are also different and you can try these to filter them out. It takes some studying of your image characteristics and trial and error to get that working,

As I mentioned before I am currently working on updating the texture filter to also allow using machine learning concepts. I hope with such technology the tuning will get a bit easier, as the machine learning will "select" the right characteristics. I got some first results, but they are not good enough yet. Part of this change is also that you can do object classifications in the texture filter, instead of only doing pixel classifications. That might help you in the future as well.

 

[Clutch Cargo]

My center display (I have 3-across 27"s), is 1440. I was was using 1000x1000px as you suggested until I was having a white border issue when merging RGB+IR so I wrote a script to create 1200x1200px and then crop out 200px to get back to 1000x1000. Of course now my code is working where I no longer see the white borders so I am just keeping the size. Works well for me. And I agree, only zoom/pan as I need... not my preference either.

So if not too complicated, what about adding a check box on top that allows 1 preview panel or 2 preview panels? Users could decide what works best on their system?

Good tips on grassy areas. I will definitely try these out. Testing both methods currently to see which will work best for my needs. Looking forward to the next release.

 

[arno]

Hi,

I'll see if I can add the dual output as an option. It will also increase the processing time of course to calcukate two images to show.

 

[Clutch Cargo]

Great, I think it's worth a try.