P3D v5 Unhandled Exceptions with TFE machine learning

[arno]

I have fixed the UE now, I will push a new development release later this morning when all the regression tests have ran on the change. That will also include the performance optimizations I made yesterday.

 

[Clutch Cargo]

Thx for the quick fix... will try that tomorrow. You sort of lost me on your explanation in the second paragraph of your post above your last one. I am processing with an Intel Core i9-12900KF which has 16 cores (8P+8E). I'll set the size to 512 but how do I change segmentation to match my cores? What do you suggest? I haven't tried running it at a splitGrid 13... I will try that too to check my times and how it looks with tree "holes".

Also will try the K-means step instead of the Segmentation step... I am willing to try all sorts of setups to squeeze out a faster process.

 

[Clutch Cargo]

I wanted to drop this in as I just finished a new test. I took my segmentation script (again, this is the same in the video example), and all I did was replace segmentation step with the K-means step. Because of the latest UE fix (thx!), I was able to retain all the sample images and sample points without having to rebuild the script from scratch. A pretty good apples-2-apples comparison test.

To my pleasant surprise, the process finished not in 23 minutes but 7.5 minutes! Is it as accurate as the segmentation? No, but looks very-very good. Of course need to review sample points and other settings but this method seems very promising to explore deeper.

 

[MyPC8MyBrain]

did a little playing around too, thinking out loud and specifically on detecting certain features in this case example ill be referring to vegetation...
right off the top a wide array of colors can be excluded instead of marking these areas on the image itself a "macro" filtering color array focusing on specific color segment could speed things up further,
for example when doing veg detection blue, white, gray, black, pixels etc., could be excluded/filtered in advance maybe?

 

[arno]

To my pleasant surprise, the process finished not in 23 minutes but 7.5 minutes! Is it as accurate as the segmentation? No, but looks very-very good. Of course need to review sample points and other settings but this method seems very promising to explore deeper.

That's good to hear. From my testing it seems the K-means clustering gives less accurate objects and therefore detection is less good. But indeed it is faster.

 

[arno]

did a little playing around too, thinking out loud and specifically on detecting certain features in this case example ill be referring to vegetation...
right off the top a wide array of colors can be excluded instead of marking these areas on the image itself a "macro" filtering color array focusing on specific color segment could speed things up further,
for example when doing veg detection blue, white, gray, black, pixels etc., could be excluded/filtered in advance maybe?

I'm not sure this would help. The performance bottleneck is in the object segmentation, not in the classification of the pixels. Masking out certain areas might always help a bit as less has to be processed.

 

[MyPC8MyBrain]

i think i get it, the tile has to be segmented first to then be masked out,
are you able to code just segmentation step to maybe execute on the gpu/dgpu?

while making silly suggestions, a zoom in/out like in the image pane area (possibly dynamic?) for the steps map area will make it bit more manageable with lots of steps,

 

[arno]

i think i get it, the tile has to be segmented first to then be masked out,
are you able to code just segmentation step to maybe execute on the gpu/dgpu?

I am not sure. the GPU is normally good at things that can be done well in parallel (like processing a lot of pixels separately). But the segmentation considers the whole image, it is checking neighbor segments of pixels to see if they need to be merged in the same segment. So I am not sure how well that would run on a GPU (I don't have a lot of experience with GPU programming).

while making silly suggestions, a zoom in/out like in the image pane area (possibly dynamic?) for the steps map area will make it bit more manageable with lots of steps,

I have been struggling with bit filters as well. But I am afraid that a zoom will make the text unreadable. So maybe only panning is enough or do you really think zooming makes sense?

 

[MyPC8MyBrain]

I don't have a lot of experience with GPU programming

gotta start somewhere
i think managing and balancing code execution between cpu and the powerful gpu's we have on the market today is key for maximum optimization,

I am afraid that a zoom will make the text unreadable

i don't see an issue if you can zoom in and out of real-estate sections (that's what I mean by dynamic zoom not linear centered),
i think it will make the real-estate more manageable, the text will be readable up close, text not so much zoomed out (step icon/symbols should still be identifiable) when needed for a more logical flow arrangement that's not limited by view,
for example i would place my steps in normal zoom but then zoom out to align and connect their logical strings, i would zoom in to an area when needed and zoom out when i want a wider view of the flow,
when adding points i would reduce steps map area so i could have more room in the image area to mark, if i could zoom out in the steps area so i could see the entire flow (text is irrelevant at that point)
i could jump from one step to the other for adjustment, properties on the top right remain the same size and viewable text there is not effected, just the workflow area itself,

 

[arno]

gotta start somewhere
i think managing and balancing code execution between cpu and the powerful gpu's we have on the market today is key for maximum optimization,

Since all these tools are just a hobby for me I would would only dive into such an area that will costs a lot of time if it either has a huge benefit or if it is a lot of fun. At the moment using gpu is not high on either of those lists. For most of the texture filter steps the performance is quite good already, it's just the object segmentation that needs some optimization.

I don't see an issue if you can zoom in and out of real-estate sections (that's what I mean by dynamic zoom not linear centered),
i think it will make the real-estate more manageable, the text will be readable up close, text not so much zoomed out (step icon/symbols should still be identifiable) when needed for a more logical flow arrangement that's not limited by view,
for example i would place my steps in normal zoom but then zoom out to align and connect their logical strings, i would zoom in to an area when needed and zoom out when i want a wider view of the flow,
when adding points i would reduce steps map area so i could have more room in the image area to mark, if i could zoom out in the steps area so i could see the entire flow (text is irrelevant at that point)
i could jump from one step to the other for adjustment, properties on the top right remain the same size and viewable text there is not effected, just the workflow area itself,

I'll give it a try. Once zoomed out I think you can't really add connections or do things like that, that will be too small. But clicking between steps to see output and properties should work.

Not sure what you mean with real estate sections and non lineair zoom BTW.

 

[MyPC8MyBrain]

none linear zoom as in zoom that is not encored in the center, zoom that zooms in where you point you mouse and scroll in,
as for the "real-estate" area i am referring to the steps work area as a whole (see area marked in red below),

(note the image below is resized to small from 4k res with text almost invisible but icons are clearly visible)

(

 

[Clutch Cargo]

"it's just the object segmentation that needs some optimization."

Curious, are you saying it could be optimized further? Like the possibility of reducing process time, say 50%? If so, would that take days-weeks-months to achieve?

Chris - Wow, looks like you detecting everything possible? What's the processing time for your script above? Are you getting results on the water detection? Haven't tried that yet.

 

[MyPC8MyBrain]

what's the processing time for your script above?

i haven't run the script yet,
if you mean how long it takes to calculate these steps its somewhere under a minute with 4096 segments,
you want to share your latest script so i can do a quick bench test?

 

[Clutch Cargo]

Sure Chris, it's already shown on page 1 of this thread. It is the example from Arno's video. The only settings I changed from the defaults was in the Multi Res Segment I changed Maximum Tile Size from 4096 to 256 and I changed the Scale from 100 to 300. THis script takes around 20-30 seconds to click on different steps or selecting different samples. Processing time is 23minutes for one, 45minutes running 5 at a time.

The Other script I tried was the one in the User's Manual - basically same processing times. And the 3rd I just tried was simply swapping out the Multi Res Segmentation step for the K-Means Clustering step. Made no changes to any settings. Selecting different steps is nearly instant and selecting different sample images is around 3 seconds. Processing time went from 23mins down to 7.5 minutes but is not as accurate.

 

[MyPC8MyBrain]

gave it a try and had to stop the script from consuming all my ram (64GB), overall it run about 21min before i killed it,
ill give that a re run, i think lod15 might be too much,

 

[arno]

none linear zoom as in zoom that is not encored in the center, zoom that zooms in where you point you mouse and scroll in,
as for the "real-estate" area i am referring to the steps work area as a whole (see area marked in red below),

Is that not just how normal zoom is supposed to work. If you zoom with the mouse, it uses the location under the mouse as anchor.

gave it a try and had to stop the script from consuming all my ram (64GB), overall it run about 21min before i killed it,
ill give that a re run, i think lod15 might be too much,

If both of you are using different images to process and have different sample points to train the machine learning with, you are comparing apples with pears.

 

[arno]

Curious, are you saying it could be optimized further? Like the possibility of reducing process time, say 50%? If so, would that take days-weeks-months to achieve?

I don't know yet how much I can optimize it further. The first implementations I made were a lot slower. So I am not sure if I can gain another 50% or not. But I am trying to think of ways to change the implementation of the algorithm that might speed it up further.

 

[MyPC8MyBrain]

If both of you are using different images

i think there's something wrong with the tif i am using (the one posted in first post), run the script again got out of memory after 30 min,
which image did you linked clutch with?

 

[arno]

i think there's something wrong with the tif i am using (the one posted in first post), run the script again got out of memory after 30 min,
which image did you linked clutch with?

The first post only contains some samples, but not the image that the actual script is ran on. Clutch Cargo provided me a LOD10 image to test with seperately. But if you want to test performance of the entire script you would need to have the same image to be able to compare results. If you are just testing in the texture filter editor having only the samples should be sufficient.

 

[MyPC8MyBrain]

i thought the tiff image Marcus posted there is the one being tested,
despite being the wrong file i noticed it is a single thread process?
only 3% of the cpu was being used at any given time by scP,
is that true for everyone or just my test case?