Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
  • 0
meapineapple

Trying to determine the purpose of apparently unused bytes in IWADs

Question

Posted (edited)

I've been working a bit on code for reading and writing WADs. I noticed that in DOOM.WAD and DOOM2.WAD there are very frequently 1 to 3 bytes in between the data for each lump. (For example, in between the DEMO1 and DEMO2 lumps in DOOM.WAD are two apparently unused bytes, specifically 0x6D 0x6D.) These bytes are ignored by SLADE 3; loading one of these IWADs and then re-saving it removes the in-between bytes. This makes me believe that the bytes probably do not affect how the Doom engine or its ports behave when loading the file, but I would still like to understand how they got there. Maybe they're some kind of metadata used and outputted by the old WAD editing tools? I was hoping someone could shed some light.

 

edit: On further investigation it looks like the bytes at least function as padding, such that every lump begins on a 4-byte word boundary. But that doesn't explain why it's not all just a bunch of null bytes?

Edited by meapineapple

Share this post


Link to post

3 answers to this question

Recommended Posts

  • 0

Aha! I've got it - thanks for your post @fraggle, it set me down the right road.

 

It turns out that they are in fact padding bytes and, instead of the (I would think) obvious choice of padding with null bytes, the end of each lump is padded with the first byte in the lump. The DEMO1 example in the OP: The DEMO1 lump data is padded with 0x6D bytes because the first byte of the DEMO1 lump is 0x6D.

 

Share this post


Link to post
  • 1

We have the source code to some of the Doom utilities but I don't believe we have the source to wadlink, the tool id used to build WADs. So we have to kind of infer.

 

The 4 byte boundary thing makes sense - I know some x86 CPUs process data less efficiently when it's not aligned. I didn't realise that included the 486 but I'm not too surprised. Some CPU architectures straight up do not support reading unaligned data and will fault.

 

However it's a bit weird. What I've just said relates to memory alignment, not alignment of data on disk. One answer might be that it was added in a console port (maybe the Jaguar port Carmack did himself?) - in that case the WAD is just part of the ROM. One thing to check would be if the padding was missing in the early IWADs and appeared in later versions. Maybe they were working on the Jaguar Port and just happened to be using the same tool to build both versions.

 

As for the data itself, it's probably random data from wadlink's memory at the time it was run. Maybe the same buffer used to write the lumps themselves - see if you can find the same byte sequence in the preceding lump.

 

EDIT: I guess I was confused because we do have the source code to wadlink: check here for the code that ensures the alignment. Since it writes a byte at a time the padding will always be the same byte, but it does reuse the buffer. But no comments to hint as to why it's aligned in the first place.

 

EDIT 2: Looks like versions of Doom before v1.2 didn't do the 4 byte alignment thing. So it was probably added early in 1994 (v1.2 was released February 17th).

Share this post


Link to post
  • 0

Note that's around when both 32X and Jag Doom were being worked on, and that in both, the entire IWAD is mapped to memory as part of the ROM, and therefore is subject to those systems' rather stringent data alignment requirements. I get the distinct feeling that it is in fact related.

Share this post


Link to post

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
×