Interesting Strife Bits

[Quasar]

I've been working on Strife with a certain tool and have discovered some interesting tidbits. I will post more of them here as I discover them, but to start off, here are two facts that are more than likely unknown in general:

1. Strife has the highest visplane limit of any commercially released DOOM executable: 200. Oddly, few of the levels are detailed enough to actually use this many visplanes at once. Were they being overly cautious, or had they initially planned to push the engine much further?

2. Strife is the only commercially released DOOM executable to include a fix for the "all ghosts" bug, also known as an intercepts overflow. A check added at the end of PIT_AddLineIntercepts makes sure that the intercept being added is within range of the static array. If it isn't, the function returns a value to P_BlockLinesIterator which makes the iteration stop prematurely, rather than trashing memory.

This latter one means that the Rogue team must have hit intercept overflows during development, since the problem was not very widely known back then, nor was its cause diagnosed by the community until relatively recent times.

[Lich]

That's some good detective work there, dude.

What was the original visplane limit for Doom? And did it change for Heretic and Hexen at all?

[esselfortium]

In Doom 1.9, the visplane limit is 128. It was lower in early versions. I'm not sure about Heretic, but if I remember correctly it's 160 in Hexen.

[Quasar]

Heretic was still limited to 128. But also, keep in mind that Heretic was based on DOOM 1.2, so I call into question your assertion that early versions of DOOM had lower visplane limits. Unless you mean version 0.99 or 1.1 ;)

[kristus]

I thought versions before 1.666 or something had lower limit.

[Vermil]

I thought Doom's visplane limit was raised from 128 to 256 for the v1.4 beta version.

It seems everything between v1.2 and v1.666 was "beta", so officially it was 1.666 which raised the visplane limit.

[Quasar]

Haha no. The visplane limit was 128 in v1.9 - check it in the source code if you need, in the r_plane.c file.

[myk]

kristus said:
I thought versions before 1.666 or something had lower limit.

I think it was just the two-sided lines limit that was lower in earlier versions, thus you'd get some HOMs in E1M8 outside.

Quasar said:
1. Strife has the highest visplane limit of any commercially released DOOM executable: 200.

Built by id or Raven for DOS, but what is the VPO limit in Doom95?

[Grazza]

Quasar said:

the "all ghosts" bug, also known as an intercepts overflow

The two are not 100% equivalent, as you can get an intercepts overflow that causes no bad effects, and sometimes the effect in the vanilla engine is a crash.

But certainly it is interesting that they tackled this. I have suggested on a few occasions in the past that some aspects of Doom2 level design (e.g. no hitscanners in map30) were to reduce the chance of intercepts overflows, and this seems to reinforce the idea that the issue was something on the developers' minds.

BTW, I am a bit puzzled why this is in the "Source Ports" forum.

[exp(x)]

Grazza said:

BTW, I am a bit puzzled why this is in the "Source Ports" forum.

Because the Strife engine is a port of the Doom engine?

[myk]

It could well be in General because it deals with an out-of-the-box component of the games and not a community made port, but I think it's also okay to use "source ports" as an "game executable" forum in general. A previous example of this is the Doom-plus thread.

[Quasar]

It's mainly here because this info will be useful to those looking to recreate Strife as closely as possible :) When my efforts are finished, I intend to publish the results in a format that will be useful to such people :)

[kb1]

Quasar, how did you get Idapro to disassemble both the 16-bit DOS extender, and the 32-bit embedded program? More specifically, did you find the file offset as well as segment load addresses, and the entry-point for the 32-bit code? I've been trying to use the latest free version of Idapro to check out some of the old Doom DOS exes, which could be really cool for things like Doom 1.2 demo compatibility, etc. Idapro reads the DOS4GW code ok, but seems to stumble (understandably) on the game's 32-bit code.

By the way, nice finds!

[entryway]

kb1 said:

Quasar, how did you get Idapro to disassemble both the 16-bit DOS extender, and the 32-bit embedded program? More specifically, did you find the file offset as well as segment load addresses, and the entry-point for the 32-bit code? I've been trying to use the latest free version of Idapro to check out some of the old Doom DOS exes, which could be really cool for things like Doom 1.2 demo compatibility, etc. Idapro reads the DOS4GW code ok, but seems to stumble (understandably) on the game's 32-bit code.

You should remove DOS4GW stuff from EXE with HEX editor before giving it to IDA. For example for DOOM2.EXE v1.9 (709 905 bytes) you should remove first 152 084 bytes. The real EXE starts from 4D 5A B2 sequence (the 4-th occurrence of MZ string from the start of EXE)

btw, I have almost fully disassembled DOOM2.EXE v1.9 and DOOM95.EXE (it was an easy way for fixing some desynch in prboom+, because I can debug win32 doom95.exe from IDA). Approximately 95% of functions names and variables are defined in my DOOM2.idb DOOM95.idb. Also I have IDBs for others EXE like dosdoom, tasdoom, etc, but they are only for special cases like emulation of spechit overrun for corresponding complevels in prboom+, heretic+ and doom+ hacks, etc

[Quasar]

Yes, entryway is correct. Immediately following the DOS4GW stub is another MZ header for an LE, aka "linear executable." After locating this header, I literally just copy-pasted the file from that point onward into a new file I named strife.le - feeding this to IDA Pro as the appropriate type of EXE file resulted in a disassembly of the proper code. I didn't have to set any segment addresses or anything; this info is either in the header already, or IDA was smart enough to figure it out on its own. Either way, this was fortunate, because I've heard it's rarely the case.

Also, the free version of IDA Pro is almost useless compared to the commercial version. It's light-years behind in terms of what it can do, especially when you have the Hex-Rays plug-in installed. I'm not very good at ASM so being able to look at C pseudocode and instantly recognize much of it is a major boon.

[Gez]

Is there a "disassembling for dummies" resource somewhere? I have a (long term and pipe-dreamish) plan of updating the engine used by Shadowcaster but I have no idea where to begin. (And I figure my chances of getting Raven to release the source code, if they even still have it, are somewhere between slim and none, closer to none.)

[kb1]

Thanks guys for the quick replies. Good to know that I can just chop the stub off (that didn't sound right...) . I was afraid I would have to relocate segments and fixup jump/call offsets. I've been doing 16 (and 8) bit code for far too long (anyone remember 16-bit segment registers? 64K segments? 64K total??? heh)

@Quasar: 515 USD is not chump change by any means, but maybe I'll have to bite the bullet. C pseudocode generation would help facilitate my dream: "Remastering" the source code for all versions of Doom/Heretic/Hexen/Strife, including the alphas/betas. Why? To have it, of course...

More practically, though, getting old version demos to sync would be nice.

@Entryway: Your skills with disassembling the source are known, respected, and appreciated. The overflow emulation stuff is pretty hard-core, and pure genius. I was going to ask for your .idb files, but I think doing the work myself might be fun (tedious, but fun)

@Gez: If I were you, I might consider modifying a Doom source port and recreating the Shadowcaster functionality. Who knows what shape the Shadowcaster code is in - on the other hand, the Doom source is stable, works on modern platforms, and can almost use the maps already!

[Quasar]

Tonight I found an interesting bit of dead code for loading a headered form of linear graphic. It would probably have looked like this in the original source:

typedef struct
{
   short w, h, x, y;
   byte data[1];
} blocklump_t;

void V_DrawBlockLump(const char *name)
{
   blocklump_t *lump;

   lump = (blocklump_t *)W_CacheLumpName(name, PU_CACHE);

   V_DrawBlock(lump->x, 
               lump->y, 
               0, 
               lump->w,
               lump->h,
               lump->data);
}

This function has no xrefs though, so I am assuming that Rogue backed out of actually using any resources in this format. Please correct me if you know otherwise :)

I have no idea why Watcom didn't strip unused functions, but that fact has left a virtual treasure trove of unknown stuff laying about in the executable :)

[andrewj]

Quasar said:

I have no idea why Watcom didn't strip unused functions

Most likely the developers didn't give it the option to do that. Less likely the compiler didn't have that capability.

but that fact has left a virtual treasure trove of unknown stuff laying about in the executable :)

Well one man's treasure is another man's junk :-)

[Graf Zahl]

Ajapted said:

Most likely the developers didn't give it the option to do that. Less likely the compiler didn't have that capability.

15 years ago? The compilers I was using back then did not have a feature like MSVC's function level linking.

Well one man's treasure is another man's junk :-)

Agreed. :) And in this case it's not even that spectacular. I somehow doubt that Strife's code contains any hidden treasures. My impression of this game is that its developers took the Doom engine and only made the most superficial changes to it so that they could (barely) make their game. In many ways their code is hack upon hack upon hack to make the engine do what they wanted instead of a clean implementation of the new features. Seriously, the conversation system which makes the heart of the game is an implementation joke. This could have been done 10 times better if only a little more thought would have been invested into it.

[Quasar]

Well sorry I'm the only one that finds this kind of stuff interesting :P

[Quasar]

Figured out what the undocumented -flip command line parameter is supposed to do: make your character left-handed by flipping the gun sprites.

Unfortunately the feature is unfinished. Since it doesn't reverse the sprite offsets as well, the positioning of many of the weapon frames is incorrect. For example, your punch dagger starts in the middle of the screen and nearly goes off the right side when you use it (I tested it out personally).

[fraggle]

Quasar said:

I have no idea why Watcom didn't strip unused functions, but that fact has left a virtual treasure trove of unknown stuff laying about in the executable :)

I seem to remember that there are various obscure reasons why it isn't practical to strip out unused functions. I seem to remember it having something to do with interrupt handlers but a more precise explanation eludes my memory.

[Graf Zahl]

From my experience the reason is much more banal:

Creating .obj files that are suitable for function level linking is a lot more complex than ones that just get linked as a whole - and linking obviously takes longer and requires more RAM with more complex data structues to maintain.

[Quasar]

Found an external control driver interface, enabled by -control <p> on the command-line. The pointer passed as an argument appears to be to a structure which is laid out like this in memory:

{ DWORD interrupt; ticcmd_t ticcmd; }

The first member of the structure is passed as the first argument to the low-level function DPMIInt, which apparently does a protected-mode interrupt via int386x. Calling this interrupt is apparently expected to fill in the ticcmd portion of the structure, which is then used by I_BaseTiccmd.

This would have enabled external device drivers to be written to get input from any sort of device. You can use your imagination as to what it was meant to work with.

The function I_BaseTiccmd is a no-op in the released DOOM source code, and is missing entirely from Heretic and Hexen. I can't rule out that this code wasn't in DOOM v1.666, but I've never seen or heard of the -control command-line parameter personally, so I think that this is new code.

[myk]

Quasar said:
I can't rule out that this code wasn't in DOOM v1.666, but I've never seen or heard of the -control command-line parameter personally, so I think that this is new code.

If you mean new for Strife, see here. A quick search over Doom executables of various versions shows the parameter was introduced with v1.4.

[Quasar]

Thanks for the clarification. So this is something that was added in v1.4 (after Raven's branch) and then removed in the Linux DOOM source. At least we now know what it does. I'll update the DOOM wiki with the information later :)

[Quasar]

I've found lots of little interesting bits in the past week or two, but nothing I felt too much like posting. But here are some odds and ends facts:

• Strife does not raise the openings limit.
  
• Strife does not raise the drawsegs limit.
  
• Strife *does* allow more scrolling lines - up to 96 in a single map IIRC (I'm at work so I cannot double check ATM).
  
• The terrain system does not appear to be derived in any fashion from Raven's, putting down (in my opinion at least) rumors that they used Raven code in the project - I haven't found any other indicators of this yet either.
  
• P_LoadThings stores objects 9001-9010 in an array of 10 mapthing_t structures, but this array is never referenced anywhere else in the program (or at least nowhere that IDA has found so far). What were these objects supposed to be? Their description in editors has the sound of nonsense and doesn't match up with what I am seeing in the disassembly.
  

[Graf Zahl]

Quasar said:

P_LoadThings stores objects 9001-9010 in an array of 10 mapthing_t structures, but this array is never referenced anywhere else in the program (or at least nowhere that IDA has found so far). What were these objects supposed to be? Their description in editors has the sound of nonsense and doesn't match up with what I am seeing in the disassembly.
[/list]

Looks like development garbage. Some of these are only used in one of the teaser maps of the commercial IWAD.

From the placement it wouldn't surprise me if they were some kind of abandoned sector action triggers. No idea what for though.

[Quasar]

I found a totally unexpected oddity tonight while doing another pass over the menu code with hopes of deciphering some more of the save game logic, which has been one of the more stubborn parts of the code to give up its dark secrets.

It is a function in the menu module which calls into the finale code, and specifically sets it up to perform a castcall, using the code from DOOM II, which I previously had thought was completely unreferenced. This function's only caller has no xrefs, meaning that it is unused code, apparently from some type of unfinished feature.

Modified output from Hex-Rays for the two functions:

void __cdecl M_StartCast()
{
  int v0; // edx@1
  int v1; // eax@1

  usergame = 0;
  gameaction = 0;
  viewactive = 0;
  automapactive = 0;
  castnum = 0;
  v1 = 20 * mobjinfo[stru_86298[0]].seestate;
  gamestate = 2;
  caststate = (state_t *)((char *)states + v1);
  v0 = *(signed int *)((char *)&states[2] + v1);
  wipegamestate = -1;
  if ( v0 > 50 )
    v0 = 50;
  castdeath = 0;
  finalestage = 2;
  dword_9F260 = 0;
  dword_9F250 = 0;
  dword_9F254 = 0;
  casttics = v0;
}

void __cdecl M_CheckStartCast()
{
  if ( usergame )
  {
    messageLastMenuActive = menuactive;
    messageToPrint = 1;
    messageString = "You have to end your game first.";
    messageRoutine = 0;
    messageNeedsInput = 0;
    menuactive = 1;
  }
  else
  {
    M_StartCast();
    menuactive = 0;
    menupause = 0;
  }
}

I totally was not expecting to see code to start a castcall from the menu system, and it's a damn shame there's no way to call this to see what would happen, aside from getting somebody to hack up one of the relocation entries for the menu callback structures to point it to the M_CheckStartCast function (I myself have no clue how to do this).