P3D v5 Unhandled Exceptions with TFE machine learning

[Clutch Cargo]

Today followed the TFE video and recreated step-by-step on my scenProc as I followed along. Received many Unhandled Exceptions so I thought I would start a new thread.

Unhandled Exception Errors 2023 – scenProc with Machine Learning

UE1 - Following video, I loaded one tif sample and whenever I click on the K-Means Clustering icon I get UE. I clicked on continue however left panel will no longer update so I need to close TFE. Closed TFE, Open TFE and tried a 4-band bmp sample and it also caused UE.


UE2 - Constructed a new TF2 file from scratch thinking maybe my previous UE1 corrupted the TF2. Still got a UE whenever I click on the K-Mass Clustering. I did notice that when I went from split screen back to single preview panel it worked one time, but only once.
Also, noticed when I have a TF2 file loaded (with sample), and click on NEW the panel refreshes to default however the previous sample image is still in the preview panel yet there are no sample images loaded.
FYI – If I continue with the video where you add the Mean step, I can click on that with no issue however if I go back to the K-Means Clustering it gives UE.


UE3 – This occurred when I added the Multi Res Segment step (following the video), and clicked on it.
Here is something new I was un-aware of as I followed your video, that you can click on the samples panel and by not clicking on a sample the preview panel is cleared (turns white). When I did that I could click on all steps without a UE. So not sure if that is a work-around or a requirement when you add a step to not have a sample loaded?
I did change the tile size to 256, however whenever I tried to reload the sample I guess to show segmentation it would result in UE.

I decided to stop here until I hear back. I have attached each UE and the tiff/bmp sample I test with.

 

[arno]

I'll check the exceptions. I didn't get many exceptions myself recently, but I guess I usually test in one specific way.

 

[Clutch Cargo]

The only other thing I wish to mention is my tiff sample (1200x1200px) are not geo-referenced. But I believe you said you fixed that so they would not need to be? Regardless the bmp is not geo'd as expected.

 

[arno]

Hi,

I'm not near my computer now, so I'll have to debug later. But all three exceptions are the same. It's a problem when converting the object segment iinage to an image thst can be shown. I'll have to check with your input images what happens there.

 

[arno]

Hi,

A bug had indeed slipped in while refactoring some code and I hadn't noticed it before. I'll push a new build with a fix soon.

 

[arno]

The build with the fix is online by now.

 

[Clutch Cargo]

The fix did help but came across new issues. I got as far in the video where you are placing sample points. I was having issues with the split screen here so I went back to a single preview screen like the video. The issue is when I clicked to add a sample point, the resulting point was not aligned as to where I "clicked"... where my curser was. Look at the screenshots: If you look at the fist screenshot below the are two green sample points on the road. Where I actually clicked was on the green golf course path to the lower right.

This next shot shows a green sample point on the top of a house roof. Where I clicked was in the middle of the lake at the lower right bottom of the image?! Then When I tried to pan the image I got this UE4. Question - how does one delete a sample point? Did not see that in the manual (as well as how to add a red sample point but I discovered CNTL + Left mouse click to get that).

Regarding the sample points, since you are using green and red colors, may I suggest in the sample points list each sample point is color coded to match as well... maybe even two columns?

 

[arno]

The fix did help but came across new issues. I got as far in the video where you are placing sample points. I was having issues with the split screen here so I went back to a single preview screen like the video. The issue is when I clicked to add a sample point, the resulting point was not aligned as to where I "clicked"... where my curser was. Look at the screenshots: If you look at the fist screenshot below the are two green sample points on the road. Where I actually clicked was on the green golf course path to the lower right.

I see there is a bug in there. If you set the zoom back to 100% you will get the point where you clicked. I guess I forgot to include the zoom correctly in this function as well, I'll double check that.

This next shot shows a green sample point on the top of a house roof. Where I clicked was in the middle of the lake at the lower right bottom of the image?! Then When I tried to pan the image I got this UE4.

I will have to try to reproduce that, for some reason you tried to scroll outside of the bounds. Was this with or without the split view?

Question - how does one delete a sample point? Did not see that in the manual (as well as how to add a red sample point but I discovered CNTL + Left mouse click to get that).

You can delete them by selecting the point in the list and then press the selected button on top of the list of points. Selected removes the selected sample point, image deletes all samples point in the current images and all removes all of them.
Control click does indeed place a negative sample point.

Regarding the sample points, since you are using green and red colors, may I suggest in the sample points list each sample point is color coded to match as well... maybe even two columns?

The class (0 or 1) as shown in the point text represents that. But showing them in different color is a good idea.

 

[Clutch Cargo]

As a reply to when I got the UE, it was without split view. I was zoomed in bit. All I did was click on the vertical scrollbar and scrolled down then the UE appeared.

 

[arno]

Hi,

I have fixed all three issues now (sample points at wrong location, crash when scrolling and the colors in the sample point list). The new development release is being build and tested now, so it should be online in 1 to 2 hours from now.

 

[arno]

The new build is online now.

 

[Clutch Cargo]

Thx... we'll be testing.

 

[Clutch Cargo]

Really like the look and the dual panels makes it really productive. Point samples are dead on and non-geo tiffs are a non-issue now. Thx. Now, first attempts with the latest version leaves me a few questions and a couple of new UEs.

Question 1 - Following your video example I got a preview that looked very promising, very precise.

I then ran a very simple scenProc script to process the image and create the veg polys:

ImportGDAL|B:\USAF\Section_1\Images\USAF_B2.tif|*|AUTODETECT
SplitGrid|LOD15|*
DetectFeatures|FTYPE="RASTER"|B:\USAF\Section_1\Control Data\USAF_Section_1_TFE_Veg_Test2.tf2|String;VegType|tree|NONE|DONTPROCESSHOLES
MergeGrid
ExportOGR|VegType="tree"|ESRI Shapefile|B:\USAF\Section_1\USAF_B2\Vegetation\USAF_B2_tree.shp|tree

The result shown in QGIS is very different:

Where did all my polys go, ha!

Question 2 - You answered in a previous thread that all the steps should connect despite what I use... now they may not work correctly or produce a result but they should not create an Unhandled Exception. With that in mind, I use a simple TF2 below that produces great results of about 80-90% accuracy and only takes 29 seconds to run. The above TF2 in question 1 takes 23 minutes!

I had one more step in line, SVM step. Hoping I can use point samples to clean up the last 10%. Unfortunately, this causes a UE. Please tell me that it is just a bug and should work in this string? 🙏

UE6 log is attached regarding this.

Also received another UE (UE5) but I forgot to jot down what happen. Not sure if you can detect that without my little explanations.

Question 3 - I believe in a previous version you added to retain all the windows/panels positioning when I saved a TF2 or maybe it just saved despite saving a new Tf2 file. In any case, that is gone. I need to set up the overall size of TFE, reset the side panels, bottom panel, split screens each time I open TFE. Is it possible for you to retain all my previous sizing for when I open it the next time?

Ok, fiqured that's enuf for now. Whew...

 

[arno]

Hi,

Now, first attempts with the latest version leaves me a few questions and a couple of new UEs.

I should call you mister UE from now , you are good at finding them.

Question 1 - Following your video example I got a preview that looked very promising, very precise.

I then ran a very simple scenProc script to process the image and create the veg polys:

ImportGDAL|B:\USAF\Section_1\Images\USAF_B2.tif|*|AUTODETECT
SplitGrid|LOD15|*
DetectFeatures|FTYPE="RASTER"|B:\USAF\Section_1\Control Data\USAF_Section_1_TFE_Veg_Test2.tf2|String;VegType|tree|NONE|DONTPROCESSHOLES
MergeGrid
ExportOGR|VegType="tree"|ESRI Shapefile|B:\USAF\Section_1\USAF_B2\Vegetation\USAF_B2_tree.shp|tree

The result shown in QGIS is very different:

Where did all my polys go, ha!

That's a good question. I would expect them to be the same. I would have to double check if that works OK on my machine, as I did most of the testing in the editor myself as well. Are you sure that you are loading similar imagery in the script (e.g. same number of channels in de data as in the samples you used)?

I had one more step in line, SVM step. Hoping I can use point samples to clean up the last 10%. Unfortunately, this causes a UE. Please tell me that it is just a bug and should work in this string? 🙏

View attachment 85907

UE6 log is attached regarding this.

This configuration will not work. In this case you are feeding the SVM step with the output of the binary threshold. Which basically means it gets black and white pixels only. So the machine learning is working on that and not on your image data. You can't expect the machine learning to work on that few data. Like shown in the tutorial I would use at least the colors values, standard deviation and NDVI as inputs. And probably also add the NDWI.

The error seems to indicate that the SVM step has been trained with different input then you are feeding it now. The editor should clear the training data if you change the connector to another step, maybe there is a bug in there that that does not always happen.

Also received another UE (UE5) but I forgot to jot down what happen. Not sure if you can detect that without my little explanations.

That one seems to indicate that there is a step in your configuration that is unknown in the list of steps that is kept in the background. That should not happen, if you remember what you did I can try to reproduce it.

Question 3 - I believe in a previous version you added to retain all the windows/panels positioning when I saved a TF2 or maybe it just saved despite saving a new Tf2 file. In any case, that is gone. I need to set up the overall size of TFE, reset the side panels, bottom panel, split screens each time I open TFE. Is it possible for you to retain all my previous sizing for when I open it the next time?

Let me double check, but that feature should still be there. I have not removed it.

 

[Clutch Cargo]

"I should call you mister UE from now , you are good at finding them."

I know-I know, I'm not trying to be trust me. Regarding Question 1, The one sample I used for the test is a 4-band tiff sample taken directly from the same image that was ran thru the detection. And as you can see I zoomed into QGIS to the same location as the sample. I will resume testing on this same scenario. I will also add steps you suggested in my Question 2. Lot's of different setups to try.

Here's what I am seeing when I open TFE and when I load up a TF2 file:

Ok, back to testing....

 

[arno]

Hi,

I had a look at the difference between running the filter in the editor and from a script. I do indeed see some differences as well. They are bigger than I expected, but the difference is not as extreme as what you showed. They can be partly caused by hte object segmentation. If the image is split differently (e.g. in AGN tiles) you can end up with different objects. I will have to check this in some more detail.

 

[arno]

For the saving of the panel positions, I have checked and it still works here. One thing to remember is that the positions are only saved when scenProc and the editor close normally. So if it crashes with an error they are not saved. Maybe you can try to open the editor, position all panels as you want and then close the editor and scenProc directly. That should ensure they are saved correctly.

 

[Clutch Cargo]

Thanks for continuing digging into this as it seems it is so close to be so effective. I ran that TF2 several times and got the same results. Interestingly, I observed running it with DONTPROCESSHOLES and without that, they both took 23 minutes to process. Where when I use your simple NDVI script and use DONTPROCESSHOLES the process takes 29 seconds. When I process holes can take up to 2 hours. Maybe that has no bearing with the script used above but thought I would mention the time differences.

I also tried your SVM example found in the User's Manual. That too had very good looking preview results. I stopped the processing after 1hr. 10mins. I change the tile size to 256 (sounds like you say that adjustment can speed up processing), and see scale to 200. After 1hr processing I stopped. For me 1 hr processing will not work for me so I stopped the test.

As you can see I am just trying as many examples as I can find. Not knowledgeable enough to know what combinations will work on my own, so at this point is it sort of a hit-n-miss approach. You mentioned above my straight-forward NVDI string and inserting the SVM step will not work as is and I should "use at least the colors values". What step is that (or other color steps if there are more than one), I should try? I'm thinking if I can create a TF2 that has the speed of NVDI and adding SVM to further fine tune, that could result in a winning combination of speed an accuracy.

You are correct, the panel positions are being retained. It was the UEs causing the perceived issue.

I see as I write this you just updated scenProc. So I will update and continue testing.

 

[arno]

Thanks for continuing digging into this as it seems it is so close to be so effective. I ran that TF2 several times and got the same results. Interestingly, I observed running it with DONTPROCESSHOLES and without that, they both took 23 minutes to process. Where when I use your simple NDVI script and use DONTPROCESSHOLES the process takes 29 seconds. When I process holes can take up to 2 hours. Maybe that has no bearing with the script used above but thought I would mention the time differences.

Given that you have so few polygons being returned, I guess none of them have holes. So then it makes sense that there is hardly a performance different between DONTPROCESSHOLES or not.

I also tried your SVM example found in the User's Manual. That too had very good looking preview results. I stopped the processing after 1hr. 10mins. I change the tile size to 256 (sounds like you say that adjustment can speed up processing), and see scale to 200. After 1hr processing I stopped. For me 1 hr processing will not work for me so I stopped the test.

A texture filter with the multi resolution object segmentation step or the SVM step in it will be slower than the non-object based scripts that we used before. But one hour seems quite long indeed. I still have to run some more tests on bigger areas to see how the performance is.

As you can see I am just trying as many examples as I can find. Not knowledgeable enough to know what combinations will work on my own, so at this point is it sort of a hit-n-miss approach. You mentioned above my straight-forward NVDI string and inserting the SVM step will not work as is and I should "use at least the colors values". What step is that (or other color steps if there are more than one), I should try? I'm thinking if I can create a TF2 that has the speed of NVDI and adding SVM to further fine tune, that could result in a winning combination of speed an accuracy.

Like I mentioned above having the multi resolution segmentation and the SVM will always cost you performance, so don't expect you can have the speed of the simple NDVI scripts.

The SVM step is not a magic step that will just make things better. You will have to determine which attributes you are letting the machine learning algorithm use for the feature detection. You could only feed it the NDVI value, but that would give no better results than the simple NDVI script. So I think as a minimum you would like to use the colors and the NDVI. And probably also the standard deviation. For each of these you will have to calculate the mean value for each object (see the sample scripts). Then you merge all these attributes into one image and feed it as the input to the SVM step.

You are correct, the panel positions are being retained. It was the UEs causing the perceived issue.

Glad to hear that is solved.

I see as I write this you just updated scenProc. So I will update and continue testing.

The last release contains a fix for some of the UE that you reported.

 

[arno]

How big is the image that you are processing in your script (size in degrees lat/lon) and at which LOD level do you split the image in your script? I want to see if I can run a similar performance test here.