• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

SBuilderX Source Code

Luis_Sá

Resource contributor
#1
Dear SBuilderX users,

From time to time I see complaints about my absence in responding to questions related to SBuilderX. Actually, since 2010 I rarely use FS and most of the times I do not know (or I do not remember) the answers that users are looking for. Fortunately many users have helped me during revisions (last one was in April 2015) and I even get help from Luis Feliz Tirado to daily manage the Ptsim forum. There is a lot of spam there and new forum users need to be manually checked or the forum gets filled with rubish. Also Dick is also writing new plugins for the tile server and I am extremely grateful to him and all that respond and help SBuilderX users.

Since the tool is still in use and given that I am not able to give support or correct bugs, I decided to place the source code in github. The programme was last compiled using Visual Studio 2013 and now I am revising it using Visual Studio 2017 (the free version). It is written in Visual Basic and I never had the idea of it being placed in the public domain. So code documentation is almost nonexistent. Still to correct small bugs, I think it can be done by following the names of the routines to reach the culprit parts.

There still is a question that I need to check which is related to images obtained from tile servers. In 2008 I was extremely afraid of infringing copyrights and used the concept of external plugins that could get the tiles. But today I see many tools that get image from servers so I need to check this. I think there is no difference between the compiled or source code (one is a consequence of the other) in regard to this aspect. You can PM me if you have information on this and I will be very grateful for any feedback.

I am learning how to use git and github. I will write a notice here when I upload the files. As I plan to use parts of the code to write a tool to for my sailing hobby, I would be very interested in collaborating to improve the present code. My only limitation is with the "FS side" of the programme.

Best Regards, Luis
 
#2
This sounds like a positive step, Thank you Luis.

Availability of GIS photo and vector data has exploded in the last 3-5 years and the sheer volume of GIS coverage available compared to when you created SBuilderX is mind blowing.

:wizard: **The fact that it is still being used nearly a decade later shows that it has filled it's niche exceedingly well and is testament to it's design.**

That said, time marches on, and its limitations with data-sets and PCs that are an order of magnitude larger/more capable than when it was written, are inevitably starting to show:
ie (and these are in no way criticisms,:duck:see my glowing testimonial above :D ):
  • performance with large volumes of high res imagery data loaded,
  • Limited to import/gathering of/exporting to large areas of high res imagery in small chunks (screen sized?) ,
  • <2GB photo-scenery files export,
  • interoperability (ie. import/export) with now easily accessible GIS applications.
It will be be interesting to see where to from here, especially with GDAL libraries to handle many of the limitations without requiring much re-invention of the wheel.

I will quite happily volunteer to test and give usability/feature suggestions, and stress test it with my imagery requirements... which I have Terabytes of... :)

Thank you again Luis, and anyone who takes up the task of collaboration with this to advance the tool.

Regards

Braedon
 
Last edited:
#3
Thank you Luis,

I've always wanted to know how you coded this program. Excellent work. One suggestion is to put the code out under GPL (or GPL like) license. I say this because you mentioned "public domain". That way you retain some rights to your code. It can be updated and shared, but if someone updates it and does not share it, that is probably not what you intended.
 

rhumbaflappy

Moderator
Staff member
Resource contributor
#4
Windows bitmaps have a size limitation because the BMP header uses only 4 bytes to define the file size. That's no more than 4GB of data.

The .NET programs also have some restrictions:

https://stackoverflow.com/questions/29175585/what-is-the-maximum-resolution-of-c-sharp-net-bitmap

I don't know if resample has restrictions, but it would for bitmaps. And resample is a 32-bit program.

As far as allowing import and export of different formats, I think that would be a nice addition.
 

Luis_Sá

Resource contributor
#5
Thank you Luis,

I've always wanted to know how you coded this program. Excellent work. One suggestion is to put the code out under GPL (or GPL like) license. I say this because you mentioned "public domain". That way you retain some rights to your code. It can be updated and shared, but if someone updates it and does not share it, that is probably not what you intended.
Hello Ron,

Thank you for your valuable tip. I hope you will not be disappointed because I had never thought in sharing SBuilderX and so it is not well organized. I can push the project directly to github with one or two clicks from inside Visual Studio 2017 but I need to understand first the potentialities of github, which I am doing right now.

Regards, Luis
 
#6
Hello Ron,

Thank you for your valuable tip. I hope you will not be disappointed because I had never thought in sharing SBuilderX and so it is not well organized. I can push the project directly to github with one or two clicks from inside Visual Studio 2017 but I need to understand first the potentialities of github, which I am doing right now.
Github is a great way to share your code. It allows people to branch/fork your code and then merge later. Luis, there is some "management" of the code required here. Whom do you allow to update your official branch .... You may want some help from others and give them admin/update/merge privileges. Otherwise it's you reviewing their code and managing the updates.

Oh, and I won't be disappointed. Some of my code started out as "playing with an idea" and is not great code either. ;) Thanks again.
 
#7
@rhumbaflappy it is actually usually ~2GB, most file libraries use unsigneds by default for addressing, resample included. Our products all behave this way, even Photoshop (the large format PSB files are an extension that shares logic with the Camera Raw stuff).

I think it would be great to see support for more formats through plugins, we have some open source libs that could be integrated, more on its way pending legal review.
 

Luis_Sá

Resource contributor
#9
Hello,

Source code is available at:

https://github.com/LuisVSa/SBuilderX

Regards, Luis

Edited by Rhumbaflappy:

This version is not yet ready for distribution due to a serious error in the compilation process. Feel free to view the code and offer suggestions. Luis is currently in Brazil, and he will undoubtedly deal with the compilation errors at that time and offer the public a release version of SBuilderX315.
 

Luis_Sá

Resource contributor
#10
Hello,

I am not (yet) very familiar with GitHub. I think I will use it to hold the last "official" release of SBuilderX 3.15. Just after publishing it, Dick and GaryGB called my attention to a limitation/bug when you have a project with vectored features of mixed type (say HPX and EXX just to give an example). In such a case, 2 shape files are created and passed to shp2vec. Uiids were the same and shp2vec does not accept it. So, as from 1/2 hour ago, the source and the EXE (SbuilderX.exe) that you get from GitHub have this limitation corrected, eg, uiids are randomly (or semi randomly) generated.

So, may GitHub would be the best way to report bugs (issues) instead of the forums, say FSDeveloper or PTSim forum. This is just an idea but, again, I am not familiar with GitHub and I may be wrong.

Regards, Luis
 

HolgerSandmann

Resource contributor
#11
Hi guys,

first, thank you, Luis, for creating and maintaining SBuilderX over all these years and now making its source code available; much appreciated!!!


Dick and GaryGB called my attention to a limitation/bug when you have a project with vectored features of mixed type (say HPX and EXX just to give an example). In such a case, 2 shape files are created and passed to shp2vec. Uiids were the same and shp2vec does not accept it.
Maybe I'm not understanding the issue correctly but I have created many .SBP files with multiple types of vector data in them and they compiled just fine. I always assumed that the UUIDs are ignored by Shp2Vec given that the Terrain SDK states "The unique IDs that are marked as 'does not go into the simulation', are used to troubleshoot issues with the data creation process." Moreover, if you look at HLX and HPX .dbf files in the SDK's Vector Example1 folder you'll notice lots of UUID duplicates, indicating to me that this parameter isn't enforced by the .exe. In any case, preventing duplication of the UUIDs across vector types certainly won't hurt.


... though resample's limit seems to be 2GB
It's possible to submit TIFF/GeoTIFF files >2GB to resample.exe as long as they were saved with the (lossless) LZW compression. The downside is that the processing time increases by quite a bit, presumably because resample.exe has to de-compress the file on the fly (for example, I compiled a 2.5GB LZW-compressed GeoTIFF elevation file the other day that took about 25 minutes, and orthoimagery of comparable size would take longer still). Thus, I tend to create multi-source .inf files if I expect multiple Beta compilations. Also, I haven't experimented with files larger than ~3GB so I don't know whether there's an upper limit even with LZW compression.

Cheers, Holger
 
#12
http://www.fsdeveloper.com/forum/threads/sbuilderx-source-code.441176/#post-785351

Maybe I'm not understanding the issue correctly but I have created many .SBP files with multiple types of vector data in them and they compiled just fine. I always assumed that the UUIDs are ignored by Shp2Vec given that the Terrain SDK states "The unique IDs that are marked as 'does not go into the simulation', are used to troubleshoot issues with the data creation process." Moreover, if you look at HLX and HPX .dbf files in the SDK's Vector Example1 folder you'll notice lots of UUID duplicates, indicating to me that this parameter isn't enforced by the .exe. In any case, preventing duplication of the UUIDs across vector types certainly won't hurt.
Hi Holger:

Subsequent to Luis' latter post above, some additional factors involving sequential DBF file coding- rather than UUID randomness- was identified in a thread for the SBX315 Beta, and IIUC, Dick and Luis are working on that.

AFAIK, Luis will be traveling for the next 3 weeks, and may- or may not- have an opportunity to post further info on the status of creating intermediate 'work' SHP files within the [SBuilderx315 install path]\Shapes folder ...which are intended only for generating source data to be used for a SHP2VEC CVX BGL compilation.


FYI: SBuilderx315 'Export' of ESRI SHP files for use in ex: 3rd party GIS enabled applications is reportedly not affected by the above coding issues under investigation.

So, IIUC, the above coding issues under investigation only apply when SBuilderX315 is generating SHP files < internally > from 'selected' vector objects ...that are only used to compile a CVX vector BGL.

Thus, the current 'publicly-accessible' build of SBuilderX315 on GitHub will likely be subject to further modification ASAP, prior to having 'full' functionality restored for CVX vector BGL compilation ...when more than (1) vector object 'type' is intended to be packaged in the same BGL.

Until that time SBuilderX end users may need to utilize SBuilderX 3.14 for CVX vector BGL compilation ...when more than (1) vector object 'type' is intended to be packaged in the same BGL.

AFAIK, if only objects of (1) vector 'type' are to be packaged in a BGL, SBuilderX315 CVX vector BGL compilation works properly. :pushpin:


http://www.fsdeveloper.com/forum/threads/sbuilderx-source-code.441176/#post-785351

It's possible to submit TIFF/GeoTIFF files >2GB to resample.exe as long as they were saved with the (lossless) LZW compression. The downside is that the processing time increases by quite a bit, presumably because resample.exe has to de-compress the file on the fly (for example, I compiled a 2.5GB LZW-compressed GeoTIFF elevation file the other day that took about 25 minutes, and orthoimagery of comparable size would take longer still). Thus, I tend to create multi-source .inf files if I expect multiple Beta compilations. Also, I haven't experimented with files larger than ~3GB so I don't know whether there's an upper limit even with LZW compression.

Cheers, Holger

Considering the statements posted by Sean Isom and Dick earlier in this same thread:

http://www.fsdeveloper.com/forum/threads/sbuilderx-source-code.441176/#post-782734

http://www.fsdeveloper.com/forum/threads/sbuilderx-source-code.441176/#post-782738

...one certainly might wonder whether there may be potential to increase the input, working set, and output data sizes that MSFS / LM-P3D SDK Resample will accommodate when source files are submitted in a Resample-compatible file format other than Windows BMP.


However, since FS SDK Resample is a stand-alone application which does not rely on external libraries / DLLs etc., I am still curious as to whether we actually may be able to push the envelope of available USERVA for SDK Resample a 'bit' (or many 'bytes' !) ...further. :scratchch



If it would be possible at some point within your limited available time, I would be very grateful if you would test something that might prove useful to the FS Developer knowledge base for Resample. :idea:


After making a ZIP backup of SDK Resample.exe, set the "LARGE_ADDRESS_AWARE" flag in the executable for that original Resample.exe file, by applying the "4GB Patch" (and specifically not via MS "CorFlags") ...following the procedure described recently for patching CvxExtractor.exe in this thread:

http://www.fsdeveloper.com/forum/th...porting-vector-data.432918/page-5#post-783986


BTW: This test uses '4GB Patch', as I am not certain whether Resample is in a fully compatible Portable Executable (aka "PE") file format that may be patched via 'MS CorFlags' ...which may- or may not- be intended only for use with 'certain' Win32/64 GUI executables.

https://en.wikipedia.org/wiki/Portable_Executable


Then, if you would please, submit to that copy of Resample with the "4GB Patch" applied to it, a:

* 'Larger than 3-GB' (ex: 4-GB ?) multi-source INF source data set of GeoTIFF files (either LZW-compressed or not)

...to see if Resample will accommodate that data set, and then successfully output a 'larger than 2-GB' output BGL.


PS: I hope we 'may' be able to increase the size of both working and output data sets submitted by SBuilderX315 to SDK Resample.


Thanks in advance for considering a quick test of whether we 'may' be able to extend 64-Bit USERVA for SDK Resample task sessions. :)

GaryGB
 
Last edited:

rhumbaflappy

Moderator
Staff member
Resource contributor
#13
Hi all.

Luis is in Brazil yet he is still working on the code. I'm confident the version 315 will be ready when he returns from his trip. Many thanks to Luis' wife for putting up with all our problems. :duck:
 
#14
Interested readers following this thread may note that a discussion arising on a latter sub-topic (-of possibly being able to increase the size of both working and output data sets processed and submitted by SBuilderX315 to SDK Resample) ...which I had initially posed as a question to Holger above:

http://www.fsdeveloper.com/forum/threads/sbuilderx-source-code.441176/#post-785394


...has been split into another thread here:

http://www.fsdeveloper.com/forum/threads/image-size-problems-sbuilderx.441530/


Hopefully this re-structuring of posted content will allow discussion to proceed with a more appropriate focus on the primary topic intended to be discussed in each resulting thread.


Thanks to Luis for continuing to keep FSDeveloper forums informed the status of SBuilderX 3.15. :wizard:

GaryGB
 
Last edited: