Suggestion for standard IWAD location specification

It's 2012 and we still don't have a standard method for people to indicate the location of their IWADs to source ports. I think this is a bad situation, and one that could be easily fixed.

Here is my suggestion, open for discussion by anyone:

IWADSTD_ can be used as a prefix for an arbitrary number of system environment variables (the most widely supported global configuration method available, not dependent on any non-standard APIs and available on more or less all operating systems) which are set to fully qualified file paths in the system-dependent path format.

Standard suffixes could include the following:

DOOMSW     - Doom Shareware Version
DOOM       - Doom
DOOMU      - The Ultimate Doom
DOOM2      - DOOM II
DOOM2F     - DOOM II, French Edition
DOOM2G     - DOOM II, German Edition
TNT        - Final DOOM: TNT
PLUTONIA   - Final DOOM: Plutonia
HTICSW     - Heretic Shareware Version
HTIC       - Heretic
HTICSOSR   - Heretic: Shadow of the Serpent Riders
HEXENDEMO  - Hexen Demo Version
HEXEN      - Hexen
STRIFEDEMO - Strife Demo Version
STRIFE     - Strife
HACX       - HACX: Twitch 'n Kill 1.2 or later
FDOOM      - FreeDoom
FDOOMU     - Ultimate FreeDoom
FDOOM2     - FreeDoom 2
FREEDM     - FreeDM
CUSTOMxx   - Support for up to 99 custom IWADs (xx == digits 0-9)

Whether a given port wants to support any of these is up to that port obviously, and support for CUSTOM IWAD specifications would be especially optional.

I am definitely in favor of this. The current directory-based solutions, which depend on a strict and port-defined naming convention, strike me as dodgy. And while Eternity's solution gets around that, it's not very convenient having to muck in its config. And of course that only helps the one port.

As an aside, I also like the wide coverage, but I notice you did not list an entry for Strife's voices.wad.

This is a good idea. A nice graphical utility to configure IWAD paths would take a lot of the hassle out of setting up a source port, especially for new users that aren't expecting to have to dig through cfg files or use the command line. It might be nice to include something like this in ports' download packages, actually.

I am against this.

There already exists DOOMWADDIR and DOOMWADPATH, which can contain IWADs (and depending on the port, it could be recursive directory wise).

Since you can already do:

DOOMWADPATH=/home/steven/wads/iwad:/home/steven/wads/pwad:
/home/steven/wads/:/usr/share/games/doom:/usr/local/share/games/doom

The only thing this would gain is specifying conflicting named IWADs outside of those paths (or people who don't name their IWADs using standard names). Most usually Doom and The Ultimate Doom.

If a source port can handle custom IWADs, it can usually be passed via -iwad <name of WAD>. However, with custom IWADs, which game is being played? Is the IWAD Doom? Doom2? Heretic? Hexen? Sonic Robo Blast 2? Strife?

At least for ReMooD (don't know the way other ports do it), You can specify say -iwad dwango5.wad and it will search DOOMWADPATH and a few other places for the file. But since dwango5.wad is not a preexisting IWAD and not in the index it will go though autodetection means to at least try to see what it is, unless you force a mode.

Also, you forgot Blasphemer.

And any graphical utilities could be made to support the existing DOOMWADDIR and DOOMWADPATH (DOOMWADPATH being the recommended choice). And for new users, this would make specifying WAD locations that would affect every port already supporting this method.

EDIT: Since you say it is 2012, then in reality it should be this:

export DOOMXMLINFO=/home/steven/doom.xml

or via -xml <file>, -xmlinfo <file>, -doomxml <file>, -doomxmlinfo <file>, -doominfo <file>

And on top of that search for the file in DOOMWADDIR/DOOMWADPATH

Which then contains

<?xml version="1.0"?>
<doom>
  <iwad>
    <filename>doom2.wad</filename>
    <title>Doom II: Hell on Earth</title>
    <location>/dev/i/put/my/iwads/in/dev</location>
  </iwad>
  <iwad>
    <filename>flufbuny.wad</filename>
    <title>ATTACK OF THE FLUFFY BUNNIES!!!11</title>
    <location>/</location>
    <comment>After the innocent murder of the Doomguy's pet daisy,
the bunnies had it. The bunnies launch an all out offensive on the
forces of hell. Play with up to 64 bunnies per level and take bac
nature!</comment>
  </iwad>
</doom>

etc.

EDIT: Also the XML could stack too and you could specify multiple XML files by separating them with a PATH separator.

Look for XML files in every single specified DOOMWADPATH and some standard locations:
* $DOOMWADPATH
* $DOOMWADDIR
* $DOOMXMLINFO
* .
* <Your Ports Config Dir>
* /usr/share/games/doom
* /usr/local/share/games/doom
* C:\DOOMDATA

I have avoided using environment variables for program specific uses.
They pollute the environment name space for all programs.
They need to be setup every boot in a place far removed from the actual program that uses them.
They are available to all programs and advertises the Doom programs existence on the system and where its files are.
You must be using Windows ??

I prefer a config file of a known name in a standard place.
This is much easier to setup, all you need to know is how to edit.
It is no more difficult to indicate to a new user to edit such a config file as it is to tell them to setup some environment variables.
A user setup program can create such a config file easily enough too.
Some content can be standardized along the lines you have suggested.

I would not use names with numbers, where the numbers must be registered with some central authority. Just use the IWAD name with a
suitable prefix. In a config file, the line syntax would make this much easier.

Being able to specify a default IWAD location is of first importance.

IWAD: DEFAULT IN "/usr/local/games/doom"
IWAD: HACX IN "/usr/local/games/hacx"
IWAD: "some_strange_IWAD" IN "/usr/local/some_strange_directory"

Being able to set a default saves most duplicate work.

Both of these solutions are far more complicated and cause far more problems, such as needing to account for user-variant writable locations on operating systems that do not give full write access to the drive (a limited user on Linux can't write much of anywhere but their user directory, for example).

Also, neither DOOMWADPATH nor DOOMWADDIR are capable of identifying which game a file is for; they only list directories that can potentially contain any kind of WAD file for any game.

Virtually no operating system requires you to manually setup environment variables every time you boot. This ain't DOS. Windows for example saves them and they are persistent.

Quasar said:

Both of these solutions are far more complicated and cause far more problems, such as needing to account for user-variant writable locations on operating systems that do not give full write access to the drive (a limited user on Linux can't write much of anywhere but their user directory, for example).

In these days you SHOULD be using the user data directories, on Windows this being

C:\Users\SomeUser\AppData\<Local,Roaming,etc.>\<Your Port>

Linux would be say:

$XDG_CONFIG_DIR/<Your Port>/
$HOME/.config/<Your Port>/
$HOME/.<Your Port>

If you aren't accounting for this and you are still using the current directory or the forbidding Program Files, then you aren't up to standards. Of course for legacy reasons you can search for a config file in the current directory (or where you used to search) but if it does not exist there, don't save it there (this is what ReMooD did).

Quasar said:

Also, neither DOOMWADPATH nor DOOMWADDIR are capable of identifying which game a file is for; they only list directories that can potentially contain any kind of WAD file for any game.

Virtually no operating system requires you to manually setup environment variables every time you boot. This ain't DOS. Windows for example saves them and they are persistent.

See the XML solution in my edit.

GhostlyDeath said:

In these days you SHOULD be using the user data directories, on Windows this being

C:\Users\SomeUser\AppData\<Local,Roaming,etc.>\<Your Port>

Which can't be properly obtained without adding shell32.dll as a dependency and calling SHGetFolderPath. This is not a standard function, unlike getenv. Assuming AppData\Local or even Users assumes Vista or Windows 7, and even worse, assumes the installation is English. Getting "SomeUser" otherwise (ie. not through SHGetFolderPath) also requires a Win32 API call, and assumes the files might not be stored in Shared Documents (XP) or Public (Vista/Win7).

Assuming Linux user filepath setup may also exclude other Linux-alikes (MacOS X for instance?).

I'm not even going to explain why having to parse XML just to find someone's IWADs is significantly more complicated. That should be self evident.

Quasar said:

Which can't be properly obtained without adding shell32.dll as a dependency and calling SHGetFolderPath. This is not a standard function, unlike getenv. Assuming AppData\Local or even Users assumes Vista or Windows 7, and even worse, assumes the installation is English. Getting "SomeUser" otherwise (ie. not through SHGetFolderPath) also requires a Win32 API call, and assumes the files might not be stored in Shared Documents (XP) or Public (Vista/Win7).

Assuming Linux user filepath setup may also exclude other Linux-alikes (MacOS X for instance?).

On the contrary.

Windows has http://en.wikipedia.org/wiki/Environment_variable#Default_Values_on_Microsoft_Windows which means: %APPDATA% (and a few friends).

You can still use shell32.dll functions though by #if defined(_WIN32) and exporting the call rather than referencing it statically (if you care about Win95/Win98 support, which uses an older function for the same thing).

Linux and Mac OS X have $HOME.

Linux could define $XDG_CONFIG_DIR (afaik)

EDIT: Mac OS X uses some funky layout for settings, but it still stands with local user configuration.

Quasar said:

I'm not even going to explain why having to parse XML just to find someone's IWADs is significantly more complicated. That should be self evident.

The XML format (or another extendable format) would be for describing custom IWADs, although it could also include data on IWAD locations.

Quasar said:

Also, neither DOOMWADPATH nor DOOMWADDIR are capable of identifying which game a file is for; they only list directories that can potentially contain any kind of WAD file for any game.

Is this such a big deal? We already have a naming scheme: doom.wad for the Doom 1 IWAD, doom2.wad for Doom 2, etc.

I believe wesleyjohnson's objection was that defining lots of environment variables pollutes the environment by adding lots of extra variables that are exposed to every program. It probably doesn't do any real damage but it's a valid objection. Arguably DOOMWADDIR/DOOMWADPATH have the same problem, albeit on a smaller scale.

I guess I just don't really see what problem this is supposed to be solving. I'm with GhostlyDeath here: DOOMWADPATH already solves this.

Whatever you do, please don't use XML. There's no way I'm going to add an XML parser library as a Chocolate Doom dependency.

If you really want to be doing this "properly", you shouldn't be using environment variables at all. The "proper" way to do it would be to use the appropriate configuration store system for the operating system: the Registry on Windows, probably a dotfile on Unix (or dconf if you want to be fancy), plist files on OS X. Pretending that you can solve this kind of problem with a single cross-platform solution that is equally bad tasting everywhere is the lazy way out.

Here are my issues with this IWADSTD_ approach:

* "IWAD" suggests path directs to a .wad (what about other formats/games?).
* "Custom" makes no sense to me. If I want to support a new IWAD/game - I'd declare a new env variable. If I want to allow users to add new IWADs - let them define the suffix string in whatever GAMEINFO file/lump defines the rest of that stuff.
* What about comma/semicolon delimited paths to support games requiring multiple files?

Doomsday (like most ports these days) already supports both DOOMWADDIR and DOOMWADPATH, so I've no principal objections to supporting another schema. Although, we are in the process of migrating to runtime configuration for this sort of thing (resource location specification).

I would be against having a separate variable for each game. I think most people would have them all in the same directory (for me /usr/games/Doom-Data/iwads) anyways so it would just be a waste. I guess there's some merit in that you can have ultimate doom and registered doom installed with both having the name doom.wad, but I think this would be better solved by supporting doomr.wad or something along those lines as a standard alternative. There's no real reason to have the identification in the variable since it can be done through inspection of the file (which ZDoom does), or you can always use the lazy route of comparing the file name.

I would rather use just use $DOOMWADDIR, add a $DOOMIWADDIR, or have a defined standard path for each OS.

The only problem I can see with environment variables is that a user may be forbidden to modify them in any way. this would likely be a WinNT system on an extremely limited account.

If I find 5 different doom.wad files on someone's DOOMWADPATH, which one is the one they want to use? It is ambiguous.

EE already supports different config variables for loading shareware, registered, retail, etc. - every version of all the supported games can be independently loaded and played. But this requires each to be manually configured before the game can be played.

Without a standard location to find these paths, this cannot be automated, and I'm not about to guess just by scanning the DOOMWADPATH - that's lazy and could be extremely wrong.

I'm also not about to ask people to rename or move around their files just for the sake of my own port. That is a non-solution.

Why is it suddenly acceptable to say "Well, it works for most people, so that's enough for everybody."? I'm aware that some edge cases are too remote to bother supporting, but we're talking about forcing people to set up their filesystems in a particular way and to name their files just so.

It's not like we're proposing the abolishing of all existing mechanisms for locating IWADs. We just want a way for the user to unambiguously and explicitly declare the location of each game's IWAD.

Quasar said:

If I find 5 different doom.wad files on someone's DOOMWADPATH, which one is the one they want to use? It is ambiguous.

Shouldn't they all be the same? If you have every version of doom.wad released in your DOOMWADPATH/DOOMWADDIR you're probably doing it wrong. I think ZDoom only recognizes one of each iwad name which can be in any path specified in the ini.

DavidPH said:

Why is it suddenly acceptable to say "Well, it works for most people, so that's enough for everybody."? I'm aware that some edge cases are too remote to bother supporting, but we're talking about forcing people to set up their filesystems in a particular way and to name their files just so.

Not sure what you're referring to here. If you want to configure your system in a odd way you should always be free to do so, but most people are not going to bother to touch configuration files (or even use the in game configuration). Defaults matter a lot.

If this is a suggestion for power-users (which would have multiple historical iwads etc) I would instead suggest just doing what ZDoom does right now and have the paths in the port-specific configuration file. Even without a set standard my unified iwads directory works well with every source port that supports full paths in -iwad (which I believe is all of them).

esselfortium said:

This is a good idea. A nice graphical utility to configure IWAD paths would take a lot of the hassle out of setting up a source port, especially for new users that aren't expecting to have to dig through cfg files or use the command line. It might be nice to include something like this in ports' download packages, actually.

So you're talking an application that allows the user to manually specify the IWAD paths for any Doom source port to use so we no longer have to go through a per-port IWAD selection process? sounds good to me, but unless most source port developers jump in, it's pretty much useless.

Thread tl;dr.

Original post: I find the idea complicated: too many env variables to set up. At the end of the day, I'll just settle with what the default ZDoom or PrBoom+ or Eternity or anything else chooses as a designated folder for IWADs, even if it means duplicating them. I'm okay with other users complying to this environment scheme, but I hope I'm not urged to do so too.

Or you can use a launcher such as ZDL or CDL which have already solved the problem of providing a GUI to set up IWAD locations, and allow using any IWAD/Source port/PWAD combo. Problem solved.

Now, if you want said launchers to be autoconfigurable or adhere to some standard.... DOOMWADDIR should be enough as a tentative default path, but by no means promise miracles or enforce a 50-variable mess like in the OP. If people want to play Doom, they will, one way or the other, even though that means a couple more of "HOW DO I SELECT DOOM.WAD?" threads over here.

I'm not going to be setting a couple dozen DOOMWADPATH-clone variables which would end up all being the same.

At most I'd accept one DOOMIWADLIST variable that'd point to a single file. Which certainly wouldn't be XML. Something simpler, like IDENTIFIER=PATH. If a port does not recognize an identifier, it can gleefully ignore it (so PrBoom+ wouldn't have to bother with HERETIC, HERETICSW, or HERETICSOTSR).

Looking at Ghostly's XML proposal, I don't see the point of separating filename from location, and the comment field seems useless.

A bunch of variables for different games imo is over the top.

Expansion on Gez's idea: How about a single variable with multiple definitions inside it for each game? you could specify key value pairs for different games, hell it could work alongside DOOMWADDIR/PATH if it didn't contain absolute paths to the wad files

RTC_Marine said:

A bunch of variables for different games imo is over the top.

Expansion on Gez's idea: How about a single variable with multiple definitions inside it for each game? you could specify key value pairs for different games, hell it could work alongside DOOMWADDIR/PATH if it didn't contain absolute paths to the wad files

I am liking this idea. But how would it be formatted? I'm not sure if = is allowed inside the value of an environment variable. Also, if there are platform-specific limitations on the length of environment variable values, you might run into that eventually.

Quasar said:

If I find 5 different doom.wad files on someone's DOOMWADPATH, which one is the one they want to use? It is ambiguous.

Well, a couple of points here: first, if you configure DOOMWADPATH with an ambiguous configuration, you get what you deserve. Garbage in, garbage out so to speak. Secondly, this is exactly the same problem that already exists with the PATH variable: "if someone has 5 xyzzy.exe files on their PATH, which one is the one they want to use?" The traditional one is to use the first found in the list.

esselfortium said:

This is a good idea. A nice graphical utility to configure IWAD paths would take a lot of the hassle out of setting up a source port, especially for new users that aren't expecting to have to dig through cfg files or use the command line. It might be nice to include something like this in ports' download packages, actually.

Esselfortium actually brings up a really good idea here: a graphical tool to configure IWAD paths would be very useful to users. But what I'd suggest is that this be implemented as an editor for DOOMWADPATH. You could quite easily write a program that parses DOOMWADPATH, finds all IWADs and allows the user to add new paths. On Windows I believe the default environment variables are saved in the registry somewhere, so it should just be a matter of updating that registry key.

fraggle said:

Well, a couple of points here: first, if you configure DOOMWADPATH with an ambiguous configuration, you get what you deserve. Garbage in, garbage out so to speak. Secondly, this is exactly the same problem that already exists with the PATH variable: "if someone has 5 xyzzy.exe files on their PATH, which one is the one they want to use?" The traditional one is to use the first found in the list.

Esselfortium actually brings up a really good idea here: a graphical tool to configure IWAD paths would be very useful to users. But what I'd suggest is that this be implemented as an editor for DOOMWADPATH. You could quite easily write a program that parses DOOMWADPATH, finds all IWADs and allows the user to add new paths. On Windows I believe the default environment variables are saved in the registry somewhere, so it should just be a matter of updating that registry key.

You're still not allowing people to use multiple files that are named the same. As a port author I regularly test my code against all versions of DOOM, including shareware, registered, and retail. I don't believe I should be required to change the DOOMWADPATH to do that, or to rename the files from their original names either.

EE already supports DOOMWADPATH and *will* "guess" which doom.wad to use if you, for example, use "-iwad doom.wad" and it cannot first be found in the configuration file settings or in a higher priority IWAD location, such as in the current working directory, EXE directory, etc.

I simply do not consider a list of paths a suitable solution in place of an unambiguous pointer to specific files. They are two different things that have two different purposes.

Quasar said:

I simply do not consider a list of paths a suitable solution in place of an unambiguous pointer to specific files. They are two different things that have two different purposes.

For Chocolate Doom at least, you can use DOOMWADPATH to specify paths to IWAD files directly, not just the directories they're found in. So you can configure it in such a way that it is completely unambiguous, even if there are multiple IWADs in the same directory. Perhaps you should just adopt the same feature.

You're still not allowing people to use multiple files that are named the same. As a port author I regularly test my code against all versions of DOOM, including shareware, registered, and retail. I don't believe I should be required to change the DOOMWADPATH to do that, or to rename the files from their original names either.

If the only reason this is needed is to deal with the fact that the registered/retail versions of Doom 1 are both named doom.wad, this seems like massive overkill.

fraggle said:

For Chocolate Doom at least, you can use DOOMWADPATH to specify paths to IWAD files directly, not just the directories they're found in. So you can configure it in such a way that it is completely unambiguous, even if there are multiple IWADs in the same directory. Perhaps you should just adopt the same feature.

If the only reason this is needed is to deal with the fact that the registered/retail versions of Doom 1 are both named doom.wad, this seems like massive overkill.

The registered and SoSR versions of Heretic are also named indistinguishably. Different versions of the same game have the same name. DEUSF-merged IWADs sitting around from the DOS days would more than likely still have the original file names.

If there were no demand for the ability to set an exact pointer to your IWADs, EE wouldn't have those settings in its system.cfg file. Appreciation for this ability has been expressed to me by a majority of the users. They just dislike needing to manually configure it for every new installation of the program. I don't like having to do it myself. It takes forever to write in all of those file paths.

I can go write a utility that just edits system.cfg, but that's not of benefit to other ports.

I wasn't aware that DOOMWADPATH was allowed to contain absolute file names, BTW. I thought it was strictly a list of paths.

fraggle said:

For Chocolate Doom at least, you can use DOOMWADPATH to specify paths to IWAD files directly, not just the directories they're found in.

Indeed, Doomsday handles it this way also.

If the only reason this is needed is to deal with the fact that the registered/retail versions of Doom 1 are both named doom.wad, this seems like massive overkill.

Not to mention that you already have to put them in different directories in any case, or rename them thus modifying them (bad idea).

graphical tool to configure IWAD paths would be very useful to users

Indeed, thats what the Doomsday Frontend presently does and which will be integrated into the engine itself in an upcoming version.

I foresee a lot of duplication of effort here: many third-party frontends already do have IWAD path configurations, and ports with their own launchers also do have them. The simple solution would be for everyone to:

• Use a frontend
  
• Use the same frontend
  
• Not using ports that can't work with said frontend
  

This solution also has the advantage that there's no need to define a new standard, nor to retrofit existing ports to support it, nor writing anything new. The only ones left out are those refusing to use a frontend for whatever reason. Those can keep drag & dropping WADs or make 1000 .bat files, AFAIC.

Also, any program complex enough to set the paths to a zillion different files, would soon degenerate into, guess what, yet another frontend, because sooner or later someone would suggest "Wouldn't it be cool if I could set source ports path with it too? And launch them?". Good night...

Quasar said:

The registered and SoSR versions of Heretic are also named indistinguishably. Different versions of the same game have the same name. DEUSF-merged IWADs sitting around from the DOS days would more than likely still have the original file names.

These seem like distinctions that are perhaps relevant to source port authors like you or me, but that probably aren't relevant to the average user, and indeed would quite likely be needlessly confusing.

Think of the typical user who is installing a source port for the first time to go on a nostalgia trip back to the '90s. They copy doom.wad off their old install CD that they found in the attic, fire up the (as yet unwritten) graphical IWAD configuration editor and see two boxes, "Doom 1 registered" and "Doom 1 retail". Which do they want? Now you're forcing the user to make a choice, when they don't even know that different "registered" and "ultimate" versions of Doom 1 exist. Nor should they need to know, for that matter.

Sure, that's probably a contrived example (total newbie), but what about other users? Among the many users on these forums - level authors, etc. - how many of them actually change between the two versions of doom.wad on a regular basis? No doubt there are even fewer people for whom the distinction between the two versions of Heretic holds any relevance.

If there were no demand for the ability to set an exact pointer to your IWADs, EE wouldn't have those settings in its system.cfg file. Appreciation for this ability has been expressed to me by a majority of the users. They just dislike needing to manually configure it for every new installation of the program. I don't like having to do it myself. It takes forever to write in all of those file paths.

I can go write a utility that just edits system.cfg, but that's not of benefit to other ports.

But if we standardize on DOOMWADPATH, we can make a utility that edits that instead. It has the advantage that a good number of ports already support it anyway. We perhaps just need to ensure that all ports support specifying absolute IWAD paths as well as directory paths. I'd much rather we just work to properly standardize that than to go inventing some complicated new system to replace it.

I wasn't aware that DOOMWADPATH was allowed to contain absolute file names, BTW. I thought it was strictly a list of paths.

To be fair, it's an extension that I added later. It seemed like a sensible thing to add.

Is a utility for editing DOOMWADDIR/DOOMWADPATH even necessary? On Windows I can use the operating system's own GUI to accomplish this. The five mouse clicks it requires to get me to that UI and what to do once there could simply be documented on the DOOM wiki.