• Which the release of FS2020 we see an explosition of activity on the forun and of course we are very happy to see this. But having all questions about FS2020 in one forum becomes a bit messy. So therefore we would like to ask you all to use the following guidelines when posting your questions:

    • Tag FS2020 specific questions with the MSFS2020 tag.
    • Questions about making 3D assets can be posted in the 3D asset design forum. Either post them in the subforum of the modelling tool you use or in the general forum if they are general.
    • Questions about aircraft design can be posted in the Aircraft design forum
    • Questions about airport design can be posted in the FS2020 airport design forum. Once airport development tools have been updated for FS2020 you can post tool speciifc questions in the subforums of those tools as well of course.
    • Questions about terrain design can be posted in the FS2020 terrain design forum.
    • Questions about SimConnect can be posted in the SimConnect forum.

    Any other question that is not specific to an aspect of development or tool can be posted in the General chat forum.

    By following these guidelines we make sure that the forums remain easy to read for everybody and also that the right people can find your post to answer it.

GeoTIFFToINF size limits?

  • Thread starter Thread starter Deleted member 12505
  • Start date Start date
D

Deleted member 12505

Guest
I'm exporting from Global Mapper to GeoTiff file (about 2.2GB file size using "BigTiff" format). When using GeoTiffToInf after I select my .TIF file it will CTD with the following error:

Application: GeoTiffToInf.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Runtime.InteropServices.COMException

Exception Info: System.NotSupportedException
at System.Windows.Media.Imaging.BitmapDecoder.SetupDecoderFromUriOrStream(System.Uri, System.IO.Stream, System.Windows.Media.Imaging.BitmapCacheOption, System.Guid ByRef, Boolean ByRef, System.IO.Stream ByRef, System.IO.UnmanagedMemoryStream ByRef, Microsoft.Win32.SafeHandles.SafeFileHandle ByRef)
at System.Windows.Media.Imaging.BitmapDecoder..ctor(System.IO.Stream, System.Windows.Media.Imaging.BitmapCreateOptions, System.Windows.Media.Imaging.BitmapCacheOption, System.Guid)
at System.Windows.Media.Imaging.TiffBitmapDecoder..ctor(System.IO.Stream, System.Windows.Media.Imaging.BitmapCreateOptions, System.Windows.Media.Imaging.BitmapCacheOption)
at GeoTiffToInf.GeoTiff.InitGeoTiff(System.String)
at GeoTiffToInf.ViewModel.ChooseFile()
...

Since it's a 32bit app, I'm guessing there is some file size limit? So what is this limit?

Cheers, Rob.

EDIT: I've also tried "NormalTiff" format but the results are the same, CTD.
 
Answered my own question with some trial and error, appears to be about 2GB. But of course that brings up another question ... resample.exe appears to be 32bit also so I'm assuming it will have a similar limit? And, is there any performance benefit to having smaller BGLs for a given area rather than large ones?

EDIT: An is there any way to make resample go faster other than raw Ghz? I have a 32GB RAM drive I use for misc. cache ... would moving resample there and source files make any difference?

Cheers, Rob.
 
robains said:
Answered my own question with some trial and error, appears to be about 2GB. But of course that brings up another question ... resample.exe appears to be 32bit also so I'm assuming it will have a similar limit? And, is there any performance benefit to having smaller BGLs for a given area rather than large ones?

EDIT: An is there any way to make resample go faster other than raw Ghz? I have a 32GB RAM drive I use for misc. cache ... would moving resample there and source files make any difference?

Cheers, Rob.
Click to expand...


Hi Rob:

32-Bit SDK Resample has a BGL output size limit of 2GB, so in theory, it has a 'working data set' size limit.

One can use LZW loss-less compression on source GeoTIFFs to 'fit more' into BGLs, but it can result in triple the Resample processing time for the compressed data.

[EDITED]

https://www.fsdeveloper.com/forum/threads/we-neeeed-a-multithreaded-resample-arrrghhh.439700/

[END_EDIT]


I would think a fast RAM drive may help, but CPU speed is the main determinant of Resampling time.

In Windows-7 Task Manager, I assign REALTIME CPU Priority to all my (multiple) SDK Resample sessions.

IIRC, you use a SSD, which 'may' optimize initial loading of data into RAM and/or a RAM disk

But AFAIK, a 32 GB DDR4 RAM Disk is likely faster than SSD I/O may be via the peripheral main board data bus.


I use (6) 3TB 7200 RPM hard drives with 256 GB of Windows-7 built-in ReadyBoost cache memory on (16) 32 GB capacity USB-3 SanDisk 150-mbps Flash Drives, all formatted EXFAT with maximum sector sizes for optimal I/0 speed, which IIUC, is a Windows 'dynamic drive' (like a multi-drive RAID-0), that AFAIK, caches RAM, Paging File, Registry and TEMP file in conjunction with Windows PreFetch and SuperFetch distributed into a *.SFCACHE file on each of those (16) drives.

Tired of having to 'go out to lunch' to kill time after launching, closing / re-launching P3D or PhotoShop ? :laughing:

ReadyBoost 'may' still perceptibly boost app re-load speeds even though one otherwise uses a SSD. :idea:


7200 RPM internal hard drives offer more GB storage for less expense ...than 'hybrid' hard drives and SSDs.


I hope at least some of this info may prove helpful for you. :)

GaryGB
 
Last edited:
Hi Gary,

Thanks for the response.

Global Mapper does provide options to Tile output on export into small TIF files. I elected to use 2X2 tile. About 500MB set of 4 TIFs + INFs took about 20 minutes to process one TIF+INF to a BGL. I'll experiment with my RAM Disk but it looks like ReSample is NOT a threaded app when it comes to processing so I doubt a RAM Disk will help much:

uc


A bit odd that LM didn't update these SDK apps to 64bit and threaded? Or is this due to FSX ball and chain? ReSample is definitely an app that could benefit hugely from being threaded.


Cheers, Rob.
 
While you were replying, I did post regarding tiling, but you've sorted it in the meantime...
You can of course recombine the tiles into one BGL. Resample is a great tool, but yeah, you need to plan your day around it:) A two-hour resample is not uncommon for me.
 
I've posted a feature request with LM.

Thanks for the link Gary, I had thought about running a bunch of instances of ReSample but as pointed out in your link then I need set affinity for each. It looks like I can rename resample ... i.e. Res1.exe Res2.exe ... ResN.exe and fire off each process with an assigned affinity so it doesn't forget next time around. But I must admit, it's 2019, this is a bit over the top don't ya think? Guess I should add this to my feature list for my workflow app.

I can't believe you ladies/gents have been living with this handicapped for so long ... did anyone approach LM on this? Other than my post, I can't find anything in LM's forums?

I can appreciate taking a break and having some coffee, but any more coffee and I'll be working from the ceiling.

Cheers, Rob.
 
Ran a quick test with 4 separate (named differently) resample.exe under Win10 and looks like I don't need to assign affinity the OS will sort it out ...


That certainly reduced the time spent with resample.

Cheers, Rob.
 
Why would you have to rename resample for that? I often run multiple instances at once without needing to do that.

Resample can process Geotiff files bigger than 2 GB, as long as you make sure the bgl output remains under 2 GB.
 
It also runs a lot faster if the input tiff isn't compressed. But then you have to work around larger the file size.
 
I think Holger recommends a compression of 90% to maintain the look and characteristics of the water. This would be a bit less compression than the 85% we've been using as a standard.
 
robains said:
https://www.fsdeveloper.com/forum/threads/geotifftoinf-size-limits.446129/post-830220

Ran a quick test with 4 separate (named differently) resample.exe under Win10 and looks like I don't need to assign affinity the OS will sort it out ...

That certainly reduced the time spent with resample.

Cheers, Rob.
Click to expand...


Interesting that Win-10 Task Mgr. / Core Scheduler can do that without explicit core assignments.


One might wonder if batch file core assignments can be achieved by instead renaming the CMD mode window title ? :scratchch

https://www.google.com/search?ei=A2...&ved=0ahUKEwi4tI2P2_PkAhUPLKwKHUvQD5oQ4dUDCAo


Its refreshing to see how much 'semi-automation' can still be achieved with just "DOS" / CMD Mode batch coding. :)


GaryGB
 
Last edited:
arno said:
Why would you have to rename resample for that?
Click to expand...

Hi Arno, if I don't make copies of Resample the OS tends to lump most of the processing on one core rather than another available core so I see one core pegged at 100% while the other's operate much lower utilization. How can I know before hand what the output size is of Resample will be?

Cheers, Rob.
 
When I run 6 resample instances here they make each of the 6 cores I have in my machine work hard.

The output size depends on many factors, like the compression, the colors in your image, etc. Also if you add watermask, seasons, etc. matters. So what I typically do is just select a LOD size I will use for the bgl files and then try to end up with files that are not too small. It's a bit trial and error to find the best settings.
 
rhumbaflappy said:
I think Holger recommends a compression of 90% to maintain the look and characteristics of the water. This would be a bit less compression than the 85% we've been using as a standard.
Click to expand...
Yes, I use 90% compression for the output bgl.
I was talking about the input tiff. If you don’t compress it (common is LZW) then resample doesn’t have to uncompress it first so the resample process finishes a lot sooner.
 
Hi Rob:

I was intrigued by the behavioral difference between Win-10 and Win-7 with regard to load balancing on multi-core CPUs, and found this interesting info which may still infer a potential task processing performance benefit to be had when working with a non-threaded application such as SDK Resample:

https://en.m.wikipedia.org/wiki/Processor_affinity


I'd be curious as to whether that explicit core affinity assignment option may actually prove useful where many source files need to be compiled by starting multiple instances of CMD-mode-only / non-threaded FS SDK compilers.:scratchch

Perhaps methods such as these might still merit testing ? :idea:

https://superuser.com/questions/115...-of-a-windows-executable-from-the-command-lin

https://www.tenforums.com/performance-maintenance/47607-affinity-command-cmd.html

GaryGB
 
Last edited:
Hi Arno,

After more thorough testing it does appear no need to rename ... (Win10 1903 - 10.0.18362). In my particular test instance I was running on a 7900X CPU, 128GB RAM, 32GB RAMDisk ... the scheduler doesn't seem to care and the tasks are distributed across cores. Guess I'll hold off on AMD's 64 core CPU until we see what Microsoft have to offer in their SDK. ;)

Hi Gary,

Interesting articles, will test command line and see but my hunch results will be the same.

Cheers, Rob.
 
You must log in or register to reply here.
Back
Top