Utility idea

[FlyingMasterChef]

Would love to have a utility (maybe there already is one) that will show every object in all your libraries.

In Instant Scenery, you can make pics of all the items in the libraries, and then look at them. However, you can only look at one library's objects at a time. A real pain when you have 50 or 60 of them, like I do.

An even nicer tool, or maybe in the same one..would be a library cleaner. Some of the libraries I have are duplicates to other ones. Or they have like 98% of the exact same objects in them. It would be really nice to be able to look at the objects and be able to delete the ones you want gone.

While I know there are some that will do this by guid... one that will do it WHILE you are looking at the object, would really be nice.

- Greg

 

[arno]

Hi Greg,

I have been thinking to make a library viewer based on the code I made for ModelConverterX already. But that plan is not so concrete yet. With ModelConverterX you can make a report about all the objects in one specific library, but not for multiple yet.

 

[FlyingMasterChef]

Glad you are feeling better, Arno..

I can't wait for this utility!!!!! My scenery is going to be a nightmare to release...since I have sooooo many libraries and use objects from many of them. The one thing I am SURE of, is that I am using some objects from multiple libraries, when those objects are probably somewhere in the same libraries as others I use.. But, because when I need an item, I use it from the first one I find it in..... ARGH!!!!!!

A way to view and then sort, organize, delete, add, etc...would be HUGE on my list of need it yesterday utils.. LOL

- Greg

 

[arno]

I am not so sure about the idea to delete objects from libraries or move objects around between them (if that is what you mean). But making a tool to get a better overview for what is happening sounds logical to me.

 

[FlyingMasterChef]

The reason I mention moving, deleting, etc... is this:

ex: I have 5 libraries

A - 35 objects
B - 5 objects
C - 22 objects
D - 85 objects

D is the library of items I have found just on the net.

A and C share 17 of the exact same objects
B shares 2 objects with my D library.

Viewing all of them at once, would be nice. But, for me anyways, (and again, I just counted, and I have 193 libraries...some with 100+ objects in them) it would be a huge benefit to me to be able to either merge some of these, or delete the ones in C that are the same as A.

I hope you'll see why I would want this ability...but of course, if you don't make it, then I will use whatever you do make.

- Greg

 

[gsnde]

I can see where Arno's reservations are coming from, Greg. The support of and the discussions about the things we are dealing with in our area are already complicated due to the mass of elements we have to face.

If now the object libraries start to be not identical for user A and user B (and different from how the creator has released them), things start to get nasty. Most likely some people will start distributing such combined libraries and by doing so annoy the hell out of the creators.

 

[FlyingMasterChef]

I knew that would be the argument.. I guess I am just odd in that if I make something for people to use, I could give a rats a** how it is used. But apparently, that is ONLY me.

I got my libraries in ONE big huge giant large monsterous pack. Called EZ object library. I am sure some of you have this. One pack, all the rwy12 object and many many more as well. Over 3000 objects in all.

I guess, again, being the jerk that I am, if someone doesn't want their objects used and shared, they should never release them.

I never understood why someone would demand that people ONLY get the objects they create from THEM. While probably NOT a problem, if you have a popular item, why use YOUR bandwidth when others want to use theirs to share it. Oh well, I will never understand this idiocy. Are creators of objects really this whiny and childish??? If so, I won't start creating items myself.

And, if Arno wanted to make the app, he could do the standard disclaimer that people should not use it to create new libraries from other peoples work.

Then again, if this were NOT a standard practice, I would not have 43 libraries with over 1/2 the same items in each... Would I???? So apparently, this combining of objects into new distributed libraries is already common practice.. So what's the problem?

Out of 3000 items, I am guessing that there are probably less than 2000 original/different items. Would be great if the program could read the guid's and allow you to kill all duplicates. Seems to me, that it would not be violating anything if it just did that. So get rid of the move feature, which personally, I think would be a mistake, and just allow the deletion of identical guid's.

- Greg

 

[gsnde]

I am on the green side here because I can only do tools, not objects, Greg

The background of your point is clear and convincing. I often think how easy it would be to combine all objects I use for an airport in one library. Instead I am keeping notes which two dozens libraries a user would have to download and install before he or she could use my airport.

But I think we have to respect the intention of the developers. If they want to share on a specific condition only, then still they do share. And I am grateful for that.

Because without those objects my airport would look extremely sad because even seeing the interface of GMax and FSDS pushes me into deep depression. Don't ask about the state I am in after more fruitless hours trying to design a cool looking barrier

 

[scruffyduck]

LOM was designed to allow developers to manage objects located in many different libraries without the need to copy, move, duplicate etc. It was developed in response to the pigs dinner that can arise with lots of objects, textures and libraries. In a way FSX made things easier since there are already thousands of objects in the retail package.

It also has (if I recall correctly) a function that identifies the objects, libraries and textures needed for a particular scenery.

In the past many users already have the objects needed for third party scenery in their setup and therefore would not want to duplicate them.

This whole area could get technically very complex without straying into the rights and wrongs of decompiling objects from libraries and re-compiling them into another.

It would be quite easy for me to create such a tool but I am not sure I would do it. By the sounds of things LOM would not satisfy your requirement Greg but you might want to take a look and see how different it is from what you want to achieve.

 

[n4gix]

I never understood why someone would demand that people ONLY get the objects they create from THEM. While probably NOT a problem, if you have a popular item, why use YOUR bandwidth when others want to use theirs to share it. Oh well, I will never understand this idiocy. Are creators of objects really this whiny and childish??? If so, I won't start creating items myself.

It's not a case of being "whiny and childish" at all. Those are words written by one who's never had his email inbasket filled to overflowing with "support questions" and/or "complaints" from folks who've downloaded some freeware package that's been worked and reworked unto the 34th generation by others...

...who by now have made a complete dog's breakfast of the original release!

I still receive what amounts to "hate mail" from individuals about some bastardized version of the freeware Socata TB20GT that was initially released for FS2002 over a dozen years ago...

...nevermind that I subsequently rebuilt the same project for FS2004 a few years later, and even provided two additional Service Release updates, which somehow never get noticed by some folks.

My canned reply these type emails is this:

If you have not downloaded and intalled the package from either avsim.com or Flightsim.com, then I am not responsible for whatever it is that you did download.

Please obtain the freeware project package from either of the above sources and replace whatever it was you downloaded elsewhere. If you still have problems, I'll work with you to solve any problems encountered.

I want everyone to enjoy the Socata TB20GT as much as I did when building it, but I simply can not be responsible for, much less fix problems created by some other third party who's altered the original package's contents.

This is the primary reason that I have only released two additional freeware aircraft projects over the past decade. I have others which I'd like to share, but am totally unwilling to open myself to further abuse ...

 

[bob5568]

I never understood why someone would demand that people ONLY get the objects they create from THEM.

- Greg

If a person's skill at modelling is so good that people will be willing to pay for his/her scenery, then I suspect you would easily understand that person wanting to protect their product. For others to decompile the library file and present those same models in their own would diminish the brand of the payware artist.

Bob

 

[arno]

Hi Greg,

My first concern with the issue is not even the ethics/copyright part, although that also plays a role.

My biggest concern is that you are enlarging the problem you are trying to solve. Let me explain.

By putting only the objects you need in a library, you make one additional library that people will have on their system. So for the next designer the chances increase even more that he finds duplicate objects and comes in the library mess as well.

The simple rule of library objects is that they can exist in ONLY ONE library BGL. As soon as they appear in two you are in trouble. It either means you have the same object with multiple GUIDs, which makes it impossible to update an object later on which trouble. Or you have the same GUID defined in multiple libraries. This means you don't know which one FS is loading and that can cause loads of trouble as well. Try for example to find out why a texture is not loaded, when the object can come from 10 different libraries.

So an object only belongs in one library. And repacking it to another one is a big NO in my opinion. It is the task of the designer to consider which libraries he is using while designing, since that has an influence on the distribution. But when you use a certain library, you will have to ask the users to download that library as well, in the original form provided by the author.

That is the only way to keep object libraries easy to use and efficient. I still don't understand those users who took my Rwy12 tree library, decompiled it, changed all GUIDs and said it was not EZ-Scenery compatible, while it was compatible already in the first place. Due to stupid and unnecesary actions like that we get some many duplicate objects floating around.

I think the only way to solve these trouble is to educate them how libraries work and not to repack objects into other libraries when that seems convenient.

Just my

 

[scruffyduck]

I could not have said that better myself Arno

 

[FlyingMasterChef]

If a person's skill at modelling is so good that people will be willing to pay for his/her scenery, then I suspect you would easily understand that person wanting to protect their product. For others to decompile the library file and present those same models in their own would diminish the brand of the payware artist.

Bob

Bob... I do understand payware...that's why I am only thinking on the lines of freeware objects. For me, personally, and only me...I would never release something I have created as payware for FS. It would always be freeware. That said, I have paid well over $3500.00 USD for add-ons, utilities, aircraft packs, etc...for FS9 and FSX.

It's not a case of being "whiny and childish" at all. Those are words written by one who's never had his email inbasket filled to overflowing with "support questions" and/or "complaints" from folks who've downloaded some freeware package that's been worked and reworked unto the 34th generation by others...


...who by now have made a complete dog's breakfast of the original release!

No...I will say that I have not. And I see your point very clearly.

But, I guess I never thought of this for 'objects' being a problem. I guess it could be when the format is for 2000 and someone can't figure out how to get it into X. While I understand totally for aircraft, again, I am only talking about freeware scenery objects.

Hi Greg,

My first concern with the issue is not even the ethics/copyright part, although that also plays a role.

My biggest concern is that you are enlarging the problem you are trying to solve. Let me explain.

By putting only the objects you need in a library, you make one additional library that people will have on their system. So for the next designer the chances increase even more that he finds duplicate objects and comes in the library mess as well.

The simple rule of library objects is that they can exist in ONLY ONE library BGL. As soon as they appear in two you are in trouble. It either means you have the same object with multiple GUIDs, which makes it impossible to update an object later on which trouble. Or you have the same GUID defined in multiple libraries. This means you don't know which one FS is loading and that can cause loads of trouble as well. Try for example to find out why a texture is not loaded, when the object can come from 10 different libraries.

So an object only belongs in one library. And repacking it to another one is a big NO in my opinion. It is the task of the designer to consider which libraries he is using while designing, since that has an influence on the distribution. But when you use a certain library, you will have to ask the users to download that library as well, in the original form provided by the author.

That is the only way to keep object libraries easy to use and efficient. I still don't understand those users who took my Rwy12 tree library, decompiled it, changed all GUIDs and said it was not EZ-Scenery compatible, while it was compatible already in the first place. Due to stupid and unnecesary actions like that we get some many duplicate objects floating around.

I think the only way to solve these trouble is to educate them how libraries work and not to repack objects into other libraries when that seems convenient.

Just my

And I will take those 2 cents. Arno, thanks for the explanation..and you are right, now that I think about it.. moving = bad bad bad. I have come to see that. But the viewing and deleting of duplicates would be GREAT!!!! I have a TON of duplicate objects that when I look at them in FSX Instant Scenery preview (I took their pictures when I created the thumbnail/preview libraries) look great..but when I try to place them in scenery, they are just black. No textures. So I believe I am seeing exactly what you are describing. And maybe the number of objects is one reason my FSX is pathetically and disgustingly slow loading and running.?.?.?

When I finally get off my butt and put up my OBJECT site, I would be honored to have you post some info like that for people. I am now educated on this and agree 1000% with you. Apparently others have done what I wanted to do (initially) and that is now MY problem.

I could not have said that better myself Arno

Oh Jon.....you could too....and you know it. hehe

---

Ok... so back to the util.. hehe Would either of you create one that displayed the library guid's with a thumbnail of them (if available) next to the guid? Sortable by number. Should show the library that the item is in. Should have the ability/option to delete duplicate objects. Either by guid duplicates (like an auto delete feature of all duplicate numbers if that's the desired action) or by check mark boxes / Click selection....so the user could delete many at once of their choosing.

This sounds like an Excel project. LOL

This would fulfill all the requirements I would have. And would help remove the problem of the duplicate libraries.

I hope this could/would be created. This is the type of util I have no problem paying for...Even if I only use it once... For me, it would be beyond useful and needed.

- Greg

 

[scruffyduck]

Ok... so back to the util.. hehe Would either of you create one that displayed the library guid's with a thumbnail of them (if available) next to the guid? Sortable by number. Should show the library that the item is in. Should have the ability/option to delete duplicate objects. Either by guid duplicates (like an auto delete feature of all duplicate numbers if that's the desired action) or by check mark boxes / Click selection....so the user could delete many at once of their choosing.

As I recall that is what I wrote LOM to do. Except for the delete function. I still have some issues with directly affecting libraries by adding or removing objects. My belief is that each object exists only in one default library (the original one) and if there are duplicates the this will be because someone has extracted the model (mdl file) and added it to another library.

Unfortunately I know of no way to identify the original library so that the duplicates could be deleted.

 

[FlyingMasterChef]

I understand that Jon..but the fact is, of my 3000+ objects in hundreds of libraries...I have over 1000 (estimated) duplicates. I have, I know this one, 5 seperate libraries that each contain the same 30 objects. Problem is, each of the 5 also contain other objects not in the others. However, some are in OTHER libraries.

See the problem. Unless I want to spend a year looking in and cataloging every library on my system, so I can try to manually find all the duplicates and hope the libraries don't have other objects I want/like/need... the utility I would like would have to have delete function. Not knowing the original would be a problem...I see that... Maybe the util could be created to find that one. If you have 10 duplicate items..could you not remove 1/2 and see if the texture still works. If not, then it's the other half. Of course, the user would have to do this work themselves, I am guessing.

All I know is that I need to remove around 1000 duplicates and have no way to do this.

- Greg

 

[arno]

Hi Greg,

I certainly think it is possible to make a tool that gives you more inside into this issue, by showing you which libraries have objects with duplicate GUIDs, etc.

But as Jon said I am not sure about deleting, since you don't know which one is the original one. But having a tool that shows duplicates can already assist the developer to select the objects from the right library, so the minimize the amount of libraries used.

 

[GHD]

Greg,

Aren't you just going to perpetuate this problem by issuing yet another library which will have duplictates of those in users already existing libraries?

 

[GaryGB]

Some interesting and practical issues are under consideration here !


I have experienced some of what Greg mentions, and have wondered if I could solve the problems with such an additional utility that at least identifies duplicate object GUIDs.

[ DIGRESSION ]

Please forgive this "marginally-related topical digression", but I am wondering if it may not prove relevant to one of Greg's questions, and the issue of whether GUIDs cannot be somehow better controlled in FSX.


As to the "Colossally Huge EZ-Scenery Object Library From Hell" scenery area (ex: "FL390 EZ-Object Library" etc. with all BGLs in the \Scenery folder, and all textures in the \Texture folder), when activated in Scenery.Cfg on my FS system during a flight, it promptly brings FSX to a crawl.

http://library.avsim.net/esearch.php?CatID=fs2004sd&DLID=122705

http://74.125.47.132/search?q=cache...90+EZ-Object+Library&cd=3&hl=en&ct=clnk&gl=us



Some objects and their textures are then not being written for minutes or many minutes.


I don't know if this is simply a recurrent breakage of FSX indexes that has occurred on my system while attempting to inspect scenery areas under development with scenery density slider settings far beyond what my older system can handle and then crashing repeatedly, but it is a real pain.


FYI: When originally adding library BGLs and textures to this single "twin folder" for EZ-Scenery object libraries, I once accidentally placed 3,000 duplicate textures in the \Texture folder along with the originals while making sure I had the newest updated files for libraries released by various authors.

Until deleted, having those duplicate textures in the \Texture folder alongside the originals caused FSX to crash when that library was active in Scenery.Cfg.


In an attempt to try and see what is going on that brought the system to its knees, Sysinternals Process Explorer (with FSX selected) shows that all object libraries in the just-activated "EZ-Scenery Object Library From Hell" scenery area (including those that have nothing to do with objects known to be placed in the scenery for the active loaded flight)... apparently are being indexed at runtime, mid-flight.

Funny thing is, for performance I turn off the Windows Indexing Service related to the "Windows File/Folder Search" function (and BGLs are not supposed to be a "Document Type" indexed by that MSIE Index.Dat engine anyway); so I assume this is some 3rd party application function such as FSX itself doing the "indexing"... to then be written into the IDX files in the following Windows XP Pro 32-bit folder at:

C:\Documents and Settings\All Users.WINDOWS\Application Data\Microsoft\FSX\SceneryIndexes\


I have also found via Sysinternals Process Monitor that all objects called by BGLs (and textures, effects etc. associated with instructions in MDL files) apparently are "inventoried" by FS during startup.

This is true even if scenery areas are "un-checked" in the FSX Scenery Library UI (and thereby set as "Active=FALSE" in Scenery.Cfg); scenery areas must be deleted (or "Cut and Exported" using SCM2004) to halt this process which contributes to longer FS initial application loadup.

http://library.avsim.net/search.php?SearchTerm=SCM2004&CatID=root&Go=Search


In light of the profound change in FSX behavior that happens on my system in relation to having such a large number of objects all in one scenery "Area" and/or "Layer" of the Scenery.Cfg, I wondered if there is an internal limit in FSX (perhaps related to how much memory is installed/available at runtime) that determines how many GUIDs can be dealt with at a time without function slowing to a crawl or even causing a crash.

NOTE: I do only have 2 GB RAM on that FS development/testing system, but I keep LOD display radius no higher than 1/2 way up, and I use NO AI Traffic or Weather, and have seen this repeatedly even when FSX scenery density slider is set at minimum !


I am also compelled to wonder if there is not a finite limit as to just how many GUIDs that FSX can deal with globally in a configuration that has vast numbers of add-on Aircraft, Sceneries etc. ?


BTW: This same "inventorying" is done with Aircraft, and contributes to longer loading of the FSX aircraft picklist unless the infrequently used Aircraft.Cfg files are renamed to no longer have a *.CFG extension.

Aircraft not being used frequently must be "hidden" from FS to halt this process (which otherwise, IIRC, also contributes to longer FS initial application loadup).

[ END_DIGRESSION ]


I'm not very familiar with GMAX or FSDS development, but IIRC, doesn't the GUID of an object get embedded in the MDL when it is created by it's author, and isn't that "internal" GUID always the same (meaning it also cannot be changed by the end-user even when decompiled from a library and re-packaged into another library) ?


Regardless of the FS scenery library of 3D objects to be placed by XML that an object ends up inside, isn't there some unique ID and non-editable GUID in each MDL file that could be read and used by a special utility to compare with other MDL files and identify duplicates ?


That way the end user would be at least informed of which library BGLs the duplicates are in, and could perhaps in some way help mitigate the duplication problem from spreading further throughout the FS community.

Developers publicly distributing placement BGLs with their sceneries could then avoid (or at least minimize) placement of objects that call upon libraries known to have duplicate scenery object GUIDs in them.


It would be great if we could use "GUID Excludes" to "block" or filter use of duplicate GUIDs in the FSX subsystem, so FS would allow only (1) instance of an original MDL GUID to be loaded/rendered instead of duplicates at runtime depending on what FSX knows to be actively available in Scenery.CFG.


I suppose it might be a challenge to identify which object is the "proper" original, and I don't think that with these duplicate libraries already being out in the FS Community one should attempt to "disable" an entire library if somehow it were possible to determine that it contained 1 or more known duplicates, as this would likely just add more chaos to the add-on scenery world.

[EDITED]
Wouldn't it be nice though, to have a utility that identified duplicate GUID objects and then had the ability to allow the user to pick the library BGL file (copy) from which they wanted to disable a particular object or list of objects to unburden FS from having to deal with the duplicates... on just that end-user's system without altering the (original) scenery object library itself (the original being zipped up for safe-keeping). ?


Hope this helps sort out some issues and contributes some beneficial ideas.

GaryGB

 

[FlyingMasterChef]

Greg,

Aren't you just going to perpetuate this problem by issuing yet another library which will have duplictates of those in users already existing libraries?

George...NO.. I have stated already...I am over the making a new one. I want to simply delete the 1000+ duplicate objects in MY PERSONAL LIBRARIES. I just wanted the delete feature.. I understand the problem with finding out which is the 'real' one..

Once I have all the duplicated gone, I can then say, ok, to use my scenery, you need these libraries. Or maybe I'll include the libraries and piss off all the people that made them.

If you read the scenario I gave above...my problem is that I use a ladder out of scenery A. I use a forklift out of scenery B. I use a truck out of C. But, since I grab these as I need them, and search library after library until I find what I want....then I use it. Turns out, that the forklift I found in B, is ALSO in C. But the truck in C is ONLY in C. Thus, I could NOT use B, and only use library C.

Are you getting the issue here? I don't want to manually go through and attempt to figure out what libraries contain duplicates of other libraries. That would be insane. So, doing nothing, I will release my scenery and the user will have to obtain 128 libraries to use it. When in fact, with the utility that I am asking for, I can get that down to maybe 30 libraries. The extras are because of all the damned duplicates. And no, I have NO plans to even attempt to figure out where all the duplicates are manually.

And as I stated, my sim has to read the 1000's of objects every single time it loads or reloads scenery. The NOT being able to delete the duplicates just sucks. And yeah, you can say...sucks for me. doesn't affect you, so why would you care. And you would be fair in saying that. Apparently, it doesn't affect you.

As for the deletion of items.... I still think it should be allowed. If I break MY OWN object, I can fix it by putting it back and deleting another one. I can do this until I find the original, then delete all the others.

And surely, thinking logically about it...(MS made this...so why logic would play any part....) there must be a way to say, ok...library A, object B points to this texture. If library B, object F points to the same texture, then why can't I just get rid of object F in library B. It doesn't have to delete the texture. Just the call for it. Or am I a total idiot and don't understand how this all works.

- Greg