/idgames redundancies - solution?

[Tormentor667]

I noticed that the /idgames archive definitely lacks a certain feature that integrates entries into updated new entries.

Just an example: The City of the Damned 2 got two updates since its initial release, therefore we have now three entries in the /idgames archive:

http://www.doomworld.com/idgames/index.php?id=15934
http://www.doomworld.com/idgames/index.php?id=16452
http://www.doomworld.com/idgames/index.php?id=15145

...where only the latest entry's download links actually work (what makes sense, as previous versions have been buggy and removed).

I am not sure if someone is still in charge of the code (afaik, Linguica wrote that system), but it definitely needs an updated concerning this - as it is not the only mod/wad that suffers this "update" problem.

Any thoughts? Anyone being able to solve this?

[Vermil]

Certinally it is confusing, though it might be interesting to keep some elements of old versions entried around, such as reviews of older versions.

[fraggle]

The files are still in /idgames so it isn't a problem with the web frontend. To fix it you need to contact Ty Halderman to remove the duplicates.

These look like different versions of the same WAD though, so it may be worth retaining them.

[Phobus]

You can get Ty to replace or update old versions with new ones when you upload them with a specific request in the text file.

[GreyGhost]

fraggle said:

The files are still in /idgames so it isn't a problem with the web frontend.

Depends on which mirror you're looking at and their file retention policy. The German and Greek mirrors both appear to track the master server (which has only the most recent version), youfailit.net retains all versions and I'm not sure about mancubus.net.

[Graf Zahl]

The easiest way to avoid such problems would be not to change the filename each time. That way you just replace the old file without any garbage left behind by bad maintenance.

[hex11]

What about vanilla maps that somehow ended up in /levels/doom{,2}/ports, or vice-versa (advanced engine maps mixed in with vanilla stuff)? I've noticed a few instances like that, but can't remember the names.

How is the id <-> file mapping done in the web code?

[Bloodshedder]

Obviously you should stop using different filenames if you want to avoid this problem. All I do right now is merge the reviews and delete the old entries manually.

The update script relies on parsing the 'fullsort' file and adds an entry to the database if there's a new file added. There's no 'fuzzy logic' to determine if a newly-named file is an update to an older one.

If something is in the wrong directory, you need to contact Ty Halderman and bring it to his attention. If there are duplicate entries in the web database listing, you need to bring it to my attention.

[DuckReconMajor]

GreyGhost said:

youfailit.net retains all versions and I'm not sure about mancubus.net.

Heh, I did not notice these were mirrors. Any reason youfailit doesn't want to host porn wads?

[Csonicgo]

DuckReconMajor said:

Heh, I did not notice these were mirrors. Any reason youfailit doesn't want to host porn wads?

Some hosting doesn't like it very well, and I know lots of "bank" sites (like paypal) will throw a fit if you host anything with boobies.

[Tormentor667]

Bloodshedder said:

Obviously you should stop using different filenames if you want to avoid this problem. All I do right now is merge the reviews and delete the old entries manually.

The problem with similar filenames is actually playing maps and wads online. If it always has the same filename it's quite confusing for players and also for Skulltag (to mention a prominent example), that's why I started doing this.

[esselfortium]

Tormentor667 said:

The problem with similar filenames is actually playing maps and wads online. If it always has the same filename it's quite confusing for players and also for Skulltag (to mention a prominent example), that's why I started doing this.

Would it be okay to keep the zip and txt filename the same, but change the name of the wad inside the zip?

[Sigvatr]

The entire idgames system is horribly out of date and clunky. It needs to be updated to a modern system.

[GreyGhost]

Agreed, it should be more like FilePlanet. There's nothing like being bombarded with adverts for stuff you don't want/need while waiting in a non-existent queue.

[Maes]

Sigvatr said:

The entire idgames system is horribly out of date and clunky.

It only makes more sense that an oldschool game should also be served by an oldschool (or rather, the most basic and reliable) type of service, IMO.

If by "out of date" you mean "OH NOES, IT GIVES YOU DIRECT HTTP AND FTP LINK!!! THAT IS LIKE SO OOOOOOL MAN!111 NO XML, NO JAVASCRIPT, NO WEB 2.0, NO POPUPS, NO "DOWNLOAD MANAGERS", IT'S SHIT!!! FUCK IT AND DIE!" then I'd rather do just that, than having a fucked up filehost that hides downloads behind complex ad-delivering systems that won't work unless you have THE latest in Web 3.0 compliant browsers.

BTW, rather than idgames/, I'd complain about all those old fileplanet and atomicgamer links scattered everywhere in the "user friendly" download portions of DW: they are all broken. Now THAT is something out of date and clunky, by any definition of the word. Idgames mirrors work without a hitch, OTOH.

[kmxexii]

Yeah I'm pretty happy with idgames the way it currently is.

[hex11]

It's not the fault of /idgames if someone uploads dups or worse, uploads crap on purpose as a joke (who's this "Terry" guy anyway?)

It would be nice though if some of the other sections of the repo had "Ports" subdirs, like in the /idgames/levels tree. That way you can tell at a glance what'll work in vanilla Doom (hence any port). The /idgames/combos dir is a mess...

[Jodwin]

hex11 said:

It would be nice though if some of the other sections of the repo had "Ports" subdirs, like in the /idgames/levels tree. That way you can tell at a glance what'll work in vanilla Doom (hence any port). The /idgames/combos dir is a mess...

It doesn't help if the mappers mislabel their maps' compatibility with ports.

[hex11]

True, but not much can be done about that, short of testing every single file before it's accepted, in order to verify it works as claimed. The next best thing is for the archive maintainer to correct errors when they're pointed out, by moving the file to the appropriate directory, and maybe also prepending an explanation line to the TXT file.

[hervoheebo]

Mostly I like /idgames the way it is, but a few things are jarring. Mainly the uploading. IMO, it's archaic and could be replaced with just submitting the file in an HTTP interface. Some computers and connections have trouble uploading files with the FTP, probably related to the low cap on concurrent users. Lastly, the default search parameter could be switched to "title". I don't think many people search a file by the 8-character filename, I've had to redo the search many times because I forgot it searches by filenames and not titles. Just a nitpick.

[Blastfrog]

hervoheebo said:

Mostly I like /idgames the way it is, but a few things are jarring. Mainly the uploading. IMO, it's archaic and could be replaced with just submitting the file in an HTTP interface. Some computers and connections have trouble uploading files with the FTP, probably related to the low cap on concurrent users. Lastly, the default search parameter could be switched to "title". I don't think many people search a file by the 8-character filename, I've had to redo the search many times because I forgot it searches by filenames and not titles. Just a nitpick.

I agree with both points. Downloading should be kept the way it is though.

Though, would it be doable for it to have it's own web-based uploader, or is it fundamentally incompatible?

[Bloodshedder]

We can't set up an uploader because we don't run the archives. We provide a web-based interface to the archives. Gamers.org runs the archives itself. Complain to them if you think FTP is archaic. I think it works just fine.

[Memfis]

Bloodshedder said:

I think it works just fine.

I can't access it for several days...

[kmxexii]

hervoheebo said:

Mostly I like /idgames the way it is, but a few things are jarring. Mainly the uploading. IMO, it's archaic and could be replaced with just submitting the file in an HTTP interface. Some computers and connections have trouble uploading files with the FTP, probably related to the low cap on concurrent users. Lastly, the default search parameter could be switched to "title". I don't think many people search a file by the 8-character filename, I've had to redo the search many times because I forgot it searches by filenames and not titles. Just a nitpick.

Filenames are the only dependable parameter for searching. A significant number of files either have no titles or have titles that are messed up (perhaps due to user error?). Others have no listed author, etc.. However, every file on the archive has a filename. They may not always be intuitive, but they're there.

Similarly, Prior to unifying my indices, I noticed that my index by filename had far more visits than by title or by author.

[hervoheebo]

Yeah, well, that was just my experiences, don't have any hard data on large-scale preferences. When casually searching for something I tend to recall titles faster than filenames. By filename is definitely better for specialized searching.

I think it was mentioned earlier but there are TONS of dead links around the main site. Namely, everything 3D downloads.

[Tormentor667]

esselfortium said:

Would it be okay to keep the zip and txt filename the same, but change the name of the wad inside the zip?

I guess the problem is still that Skulltag's file crawler are looking for the ZIPs, not the content of a zip.

[printz]

Graf Zahl said:

The easiest way to avoid such problems would be not to change the filename each time.

Dunno if I agree with that. I prefer to have some information what version I'm going to use, even if it means there will only be one file available at any time.

[Tormentor667]

printz said:

Dunno if I agree with that. I prefer to have some information what version I'm going to use, even if it means there will only be one file available at any time.

Still then the /idgames script needs an update. It doesn't help having more than one entry in the database if the creator doesn't want them to have. Ty is responsible for the files but is he also responsible for the entries in the script db?

[Gez]

Really, there's no way around it.

If you want a file to be overwritten by another, you give the new file the same name as the old file. It's not even a question of script, it's a question of filesystem. This is how it has always worked ever since the first filesystem was programmed.

Asking for an update of the script isn't reasonable -- after all, it's an interface for an archive and there are many obsolete versions that are being deliberately kept around for historical purposes. It makes just as much sense to ask the Skulltag/Doomseeker/getwad/whatever fancy online thingamajig you care to cite to start using something more robust than just the name. E.g., Skulltag could use a MD5 sum of the entire file in addition to the name to check that it's indeed the correct version, and display a "version mismatch" error message if it isn't. (After all, it already does such checks on a lump-by-lump basis to make sure people don't try to cheat with hacked colormaps, huge sprites or the like.)

[Maes]

Gez said:

If you want a file to be overwritten by another, you give the new file the same name as the old file. It's not even a question of script, it's a question of filesystem. This is how it has always worked ever since the first filesystem was programmed.

Well now there are some "meta" file systems or even resource management systems that allow non-unique filenames/names. Ofc they are backed by something really unique (ids, hashes etc.) but they can end up being confusing if there's no versioning or namespace mechanism.