Using Merger tool to manage 3rd party autogen (as user)

#1
Hi,

I've downloaded this tool with the intention of taking control of changes to autogen config files made by 3rd party scenery installers. I'm hoping this is reasonable use for someone who is not a developer but who has a decent working knowledge of managing an FSX or P3D install.

I've figured that, assuming I am keeping before and after copies of any affected files that I can make comparisons to find out what has changed, extract these in xml format that can be used by the merger tool and then use this to merge the autogen classes etc. as intended for developer use of the tool.

Specifically I will use spb2xml (or any tool which converts SPB files to XML) to create XML versions, compare these using a file comparison tool to extract the differences and then apply this difference file as the input to the merger tool.

Does this sound reasonable or am I missing something vital here? As I say I don't have in-depth knowledge of autogen definitions and classes but do have a basic understanding of what the various files are for.

Many thanks for any advice!

Rob W
 
#3
Thanks Arno, will give it a try to merge Flytampa and Earth Simulations autogen into an FTX modified baseline.

I've previously used symbolic links controlled by a switching tool I wrote to keep FTX and default FSX / P3D folders distinct for those sceneries that don't mix well. Of course it's a workaround but your tool is a proper solution for the autogen problem.

Thanks for taking the time to create the autogen merger tool.

Rob W
 
#4
Hi Arno,

I have ACM set up and almost working but have had a couple of problems with the XML and SPB file reading and conversion.

First off ACM failed to initially merge either of my .xml autogen definition files, failing with a .NET exception concerning unicode and byte order. I did a quick search on the error and found that the utf-16 coding was throwing the exception. If I changed it to utf-8 or removed the encoding clause things worked ok. I'm using .NET Framework 4.5.1 so don't think there's a problem here and as far as I can see the default xml files are all using utf-16. As I say though I did manage to work around the problem.

The second issue was that ACM threw another exception from the FlightToolkit library:

19:12:12 | Processing scenery area: e:\prepar3d v3 addon scenery\flytampa\grenadines
19:12:12 | >> Found config: e:\prepar3d v3 addon scenery\flytampa\grenadines\autogen\AutogenDescriptions.xml
19:12:12 | Error! Failed to process fileAutogenDescriptions. Trying to continue...
19:12:12 | Object reference not set to an instance of an object.
19:12:12 | at FlightToolkit.SimProp.XML.Read(Symbols symbols, Stream inputStream)
at FlightToolkit.SimProp.PropertySet.Load(Symbols symbols, Stream inputStream)
at FlightToolkit.SimProp.PropertySet.Load(Symbols symbols, String input)
at ASToFra.AutogenConfigurationMerger.SPBMerger.LoadPropertySet(String folder, String configFile)
at ASToFra.AutogenConfigurationMerger.SPBMerger.ProcessSceneryFolder(String folder)

19:12:12 | >> Found config: e:\prepar3d v3 addon scenery\flytampa\grenadines\autogen\default.xml
19:12:12 | Processing scenery area: e:\prepar3d v3 addon scenery\flytampa\stmaarten
19:12:12 | Processing scenery area: e:\prepar3d v3\orbx\ftx_au\ftxaa_orbxlibs
19:12:12 | Done!

This error was only with the .spb file, default.xml was processed correctly. Interestingly it appears that Orbx include the FlyTampa definitions in their default.xml as at first I didn't think ACM had done anything. But on checking I discovered that it had found the FlyTampa entries so didn't need to add them. I double checked by manually removing them and running ACM again which added them correctly. However, Orbx's AutogenDescriptions.spb does not already contain the FlyTampa entries but ACM failed as shown above.

Many thanks for any advice you can give on the above.

Rob W
 

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#5
I'll have to check where the error comes from. Would you have a file to test for me?
 
#6
Ok, posted earlier with my files but then went back and did a thorough double-double check as something about the autogen definitions I'd extracted bothered me.

It turns out that I'd duplicated the definitions in default.xml, trying to add them to AutogenDescriptions.spb - no wonder the library tools complained when they couldn't find the correct xml node!

So, going back and repeating the whole exercise properly I can report that ACM worked perfectly. I did find out something interesting though - Orbx already include the FlyTampa definitions in their autogen files. I noticed this because although ACM reported no errors it din't make any changes to the files (last modified date was unchanged). To confirm this really was the case I deliberately removed the FlyTampa entries and then ran ACM again which correctly put them back. Very time consuming converting the formats etc. with the command line tools but satisfying in that ACM worked exactly as expected and I found out something interesting about how Orbx deal with at least some compatibility issues.

Using ACM in this way does take some patience and care but works very well, provided you know how to handle the tools and perhaps have a reasonable understanding of what autogen defs to expect.

I have one very small suggestion that it might be useful to add an entry to the log file about how ACM has dealt with the definitions i.e. whether it has merged them or already found existing definitions it didn't need to merge (or a combination of both situations). Not an entry for every instance but just a general statement to aid in debugging if things don't appear to have worked as expected.

Many thanks for a very useful tool.

Rob W
 
Last edited:

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#7
Hi Rob,

I'll check if I can add that extra information to the log file.
 
#8
Hi Arno and Rob,

this thread has been really helpful as I came across the same issue Rob was having. Namely, as long as I had utf-16 in the header AMC ignored my added default.xml although it was in the right place and had the right name. As soon as I changed it to utf-8 it discovered and worked flawlessly. This info, restated, might help others, too. May I use this opportunity to say big thank you to Arno for all these splendid tools you made for scenery building. They really are intuitive, well done and most of all: reliable. the documentation is also always exemplary!

Cheers, Matt
 

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#9
Hi,

I came across that utf issue last week as well. When the file is saved in another encoding the xml reader fails to open it. Did you alter the actual file encoding yourself?
 
#10
Hi,

I just changed the header, not the encoding itself. Dummy stuff, I know, but it worked. I was positive the encoding was correct as it was derived from the file with utf-16 encoding. For what it is worth a bit of additional info on the occurence: the file was prepared on win10 computer, English US, windows display language, Slovenian regional settings, except for comma/dot delimination; it was then used and failed to be detected on win7 computer, Slovene windows display language, Slovenian regional settings. It was a default.xml file of FSX accel version, and then used and not detected in FSX:SE. As said before, the change in the header solved it.
 

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#11
Hi,

By default the file encoding should be utf-16, that's what MS used by default. So in that case the header is correct. I think some people manually or some tool are changing this encoding without updating the header. And that gives the crashes with certain XML readers. Changing the header is indeed a good work around, but I'm trying to understand where the issue comes from in the first place.
 
#12
Hi, I was copypasting to notepad (to get my classes out of default), saving and then again copypasted to notepad++ for edits. It is not a good practice I know and the encoding might got changed in the process, without me noticing and without actually changing the headers. Notepad does xmls as just another txt. I should have used notepad++ from the start.
 

arno

Administrator
Staff member
FSDevConf team
Resource contributor
#13
I'll see if that can cause this. Last week I had a similar report and I don't think that user manually edited. So it might also be a tool or installer causing this. I'll try to make ACM more robust for it.
 
Top