What do .pk3 files do that .wads don't?

Some things I'm going to say:

1. Quake 3 Arena was the first id game to use renamed ZIPs as resource files. ZDoom's naming its resources PK3s is a homage to this.

2. ZDoom zip files can have the WAD extension too. Why don't people do this? It would be great for obfuscation both against wad-managers and users complaining it's redundant to include PK3s in ZIPs.

3. I believe Ty Halderman only accepts ZIP files, not PK3s. Also, the Doomworld software only loads ZIP entries as pages. For example, there's this B2KILL.ARJ located in /idgames/levels/doom2/a-c/ (download it here if you want). There's NO Doomworld/idgames entry for this one, because of the non-ZIP extension.

4. I'd really love to see PK4s instead, like with Doom 3. Even more, mods that work with both games.

Gez said:

Heh, I always put it in the mod, if there's not a copy already there, and delete the external file afterwards.

This is where I expect to see it too.

I don't know if many Doomsday users do this already but I can simply download pretty much any (compatible) mod from /idgames directly into my addon folder and be playing it within four clicks (once the front end has started). This is why supporting loading of nested archives is pretty important imo ;)

Enjay said:

I disagree. However, the usefulness is not from the program POV but primarily for the user.

You can't agree because I don't disagree. I never said I didn't see worth in it from that POV.

Take, as an example, the DMD model format.

Not a good example there Enjay. DMD is not merely a name changed MD2 file (see DMD format features) so it should have another name. MD2 is not a container (multiple files, different formats). That came later in the evolution of the MD# model format series.

Gez said:

I think it has to do something with memory consumption. From what I understand, an embedded archive is still a lump, so to access any of the embedded archive's data, the entire archive must be loaded in memory. So embedded wads take up a lot more memory than if they're repackaged as PK3. One especially infamous mod in particular, renowned for the sheer size of its distribution (several hundred megabytes), has been forced to be repackaged as a PK3 for its fifth major version because of that.

Pretty much that. By forcing people to use the actual directory structure the risk of badly designed mods is greatly reduced. After seeing how people abuse the WAD embedding feature to create an impenetrable mess of data I'm a bit hesistant to allow it for Zips/7z's etc. as well.

printz said:

ZDoom's naming its resources PK3s is a homage to this.

Citation Needed. ;)

DaniJ said:

You can't agree because I don't disagree. I never said I didn't see worth in it from that POV.

My apologies. I stand corrected.

DaniJ said:

Not a good example there Enjay. DMD is not merely a name changed MD2 file so it should have another name. MD2 is not a container (multiple files, different formats). That came later in the evolution of the MD# model format series.

Fair point. It's not the best example, but I think it does illustrate the point that I was trying to make - that an extension tells an end user (rather than a program) what to expect from a file. However, I now see that we didn't really have a disagreement on that point.

However, I would still contend that an idtech3 PK3 is still a different beast to a Doom PK3. Yes, of course they are both just containers and, more than that, they are exactly the same kind of containers. However, the directory structure to allow the engine to read the data in the files appropriately is quite different between the two games. This may not make the container type different, but it does mean that the data structures within the container are different. This means (to my mind) that an extension that distinguishes the two would have been beneficial. Also, their purpose (ie provide data for Quake3 versus Zdoom) and the kind of data they contain is also different which also, to my mind, makes a different extension a desirable thing.


If, for a hypothetical example of end user usefulness, I want to search my hard drive for Quake3 PK3s, I'm going to find a bunch of useless (in this case) Doom PK3s instead. Doom PK3s would not be what I want because their end purpose and suitability is not that same as what I am looking for even though their basic container format is the same. (Again, that's just an example of a minor inconvenience so I don't need anyone posting to tell me to organise my files better. :P )

Graf Zahl said:

Pretty much that. By forcing people to use the actual directory structure the risk of badly designed mods is greatly reduced.

I'm all for that. Any measure that prevents badly constructed resource files is a good thing IMO.

It protects the mod maker from inadvertently making a mistake in their file (which may work at the time but break later - I know I've done that) and it also means that the developers are less likely have their changes break mods (and then have the users complain that (G)Zdoom sucks because the mod broke).

Also it should reduce/eliminate the possibility of 1001 exceptions having to get coded into the engine because some mods are badly put together but still work with older engines - as is the case with the many, many poorly constructed yet (somehow) working WAD files that exist.

Enjay said:

However, I would still contend that an idtech3 PK3 is still a different beast to a Doom PK3. Yes, of course they are both just containers and, more than that, they are exactly the same kind of containers. However, the directory structure to allow the engine to read the data in the files appropriately is quite different between the two games. This may not make the container type different, but it does mean that the data structures within the container are different. This means (to my mind) that an extension that distinguishes the two would have been beneficial. Also, their purpose (ie provide data for Quake3 versus Zdoom) and the kind of data they contain is also different which also, to my mind, makes a different extension a desirable thing.

Agreed.

This is why dengteam built the manifest system into the Doomsday front end. The idea being that if you can't play a mod as-is from the ZIP/WAD archive (which you just got from say /idgames) you can supply additional loading instructions (e.g., virtual directory mappings) in a human-writeable text file. I'd like to see a master server for these manifests at some point but that's not happening for a while.

I notice that ZDoom has support of 7zips in the same way it has support for regular zips. Shouldn't there be greater usage of 7zips, especially considering not many programs can edit .pk3 anyway? (mostly it's slumped and slade, as doombuilder requires you to have your directory uncompressed) I certainly intend to use 7zip instead of a .pk3 from now on. I mean, it may not be much more as far as compression goes but I just like supporting the superior format.

Decompressing 7z files is a bit slower, so your mod will take longer to run.

But mostly, DB2, SLumpEd and SLADE3 will not be able to read it. (DB2 can read zip files perfectly well, it just cannot write them, so your maps must be uncompressed but your resources can be in a PK3.)

I hope that SLADE3 will eventually use its own zipstream classes rather than rely on the crappy ones that come from wxWidgets, but for the moment this is how it is.

Enjay said:

However, I would still contend that an idtech3 PK3 is still a different beast to a Doom PK3.

PK3 seems to have become synonymous with "game package" (in the same way MP3 became synonymous with music files).

Quake engines were probably the first to adopt them, since you don't need any special tool to work them (unlike PAK), and then it has spread to many open source engines (helped along by Ryan Gordon's very nice LibPHYSFS library no doubt). DOOM engines were a bit late to the party I think.

Although I have nothing to base this on other than my gut, I don't think PK3 has become synonymous with "game package". They aren't referred to on other more modern-game forums that I have visited. Perhaps talking about PK3s is an indication of a community that follows older games. ;)

If anything is universal "PAK" still seems pretty commonly associated with "game package" and there are a number of tools that call themselves "PAK something or other" even if they support more than just PAK files. Also, there are a few games that use PAK as an extension and not all of them are id based games. However, I'm not familiar with any games using PK3 other than idtech3 ones.

However, all that aside, when frequenting forums for other games, I don't think I have ever seen anyone referring to PK3s when discussing a game that uses a package with a different name (and some of those packages are just renamed zip files like PK3s are).

A logical name could have been PK1, since it's for (enhanced) "id Tech 1 technology" and id Software seems determined to call their ZIP game data packs "PK + engine number".

I've just remembered where I *think* I have seen PK2 files. I'm pretty sure that it has become a modding convention for at least one Quake2 source port that now supports zip files as an alternative to the original PAK files. However, I can't currently find evidence of that, so it could be a brain fart.

Enjay said:

Citation Needed. ;)

It doesn't have to be exact, but I can easily consider that the ZDoom devs thought of Quake 3 Arena when deciding to implement compressed pack support.

printz said:

It doesn't have to be exact, but I can easily consider that the ZDoom devs thought of Quake 3 Arena when deciding to implement compressed pack support.

I can tell you that this is definitely not the case.

At the time I wrote the code PK3 was already in use by Doomsday and it somehow caught on for ZDoom quicker than I expected myself. Naming ZDoom.pk3 like that was done long after the community decided on this being the proper extension. There is absolutely nothing on the developer side that pushed this extension to be the 'norm'.

Graf Zahl said:

At the time I wrote the code PK3 was already in use by Doomsday and it somehow caught on for ZDoom quicker than I expected myself. Naming ZDoom.pk3 like that was done long after the community decided on this being the proper extension. There is absolutely nothing on the developer side that pushed this extension to be the 'norm'.

Then this source is to be asked why they preferred pk3 :)

Just to toss another couple of coins in, I've never seen any problem with .pk3 simply because lots of people immediately know what it is.

Exactly, a Quake 3 resource file. Ohwait... ;)