Spechits, Reject and Intercepts Overflow Lists

myk said:

Yeah. I could keep trying later if it could help. Any hints on how to somehow get different spechit numbers that might work? Is that even possible on a single machine?

This number is only the address of the lines array in P_LoadLineDefs:
lines = Z_Calloc (numlines, sizeof (line_t), PU_LEVEL, 0);
It will differ always, but its values will be approximately equal for the same systems. Memory with value of tmbbox variable (bounding box for line intersection checks) is rewritten after overflow by value depending from the address of a lines array (by address of a concrete line)

I even tried going to full DOS, but Doom+ starting the demo crashed; not sure why, but it said something weird, that there was an unknown thing 32007 (or so) then DOS got monged up (all the dirs and were messed up) and I had to restart.

It crashes in DOS always? Weird. With DOOM2.EXE also?

EDIT: heh. This level (map32@hr2.wad) does not work with doom95.exe

---------------------------
Application Error: D:\games\Doom2\doom95.EXE
---------------------------
The instruction at 00427595 referenced memory at 0000004e
The memory could not be written

Click on OK to terminate the application
---------------------------
ОК
---------------------------

I'll look into it. It's interesting for me.

Share this post


Link to post

I think I have understood the problem with h2322437.lmp@map32@hr2.wad

LineDef 1233 has no first sidedef and PrBoom (and DeePsea immediately after opening this map) informs us on that. Demos on maps with such wad errors most probably will desync with prboom and crash with doom95. I don't understand why this level has no failure with doom2.exe. It should.

EDIT: Comment from prboom sources

// cph 2002/07/20 - these errors are fatal if not fixed,
// so apply them in compatibility mode - a desync is better than a crash!

Share this post


Link to post

a long time ago, I recorded probably 8 finished demos on that map.
I think 6 of them desynced when replayed because of the open sector(or whatever) behind the blue armor.
impressive how resilient doom.exe is to bad bsp trees, nodes, and what-not.

Share this post


Link to post
entryway said:

EDIT: Comment from prboom sources

// cph 2002/07/20 - these errors are fatal if not fixed,
// so apply them in compatibility mode - a desync is better than a crash!

I wonder if that is related to the desync of my XM22-621.lmp@map22@h2h-xmas.wad (recorded with Prboom) when played back with Doom2.exe.

Share this post


Link to post

entryway said:
It crashes in DOS always? Weird. With DOOM2.EXE also?

Yeah, the map crashes always. This is the error, always the same, but varying according to the skill level:

  0: P_spawnMapThing: Unknown type 6144 at (-9, 32767)
1-3: P_spawnMapThing: Unknown type 3468 at (-11, 3467)
4-5: P_spawnMapThing: Unknown type 32703 at (-4095, -1)
And the directories and files in DOS are turned to random garbage.

There are no ill effects launching it from Windows 98 in Desktop mode, except for the glitchy demo synching behavior, apparently. Maybe because the memory handling or the memory settings are different?

Share this post


Link to post

Updated with the results of testing all compet-n pwad demos.

(No, I wasn't testing primarily for overflows, but for desyncs and the new wad autoselection feature in -plus. But while I was at it, I reckoned I might as well note any overflows.)

Share this post


Link to post

I didn't see this in the Intercepts table, so I figure I'll mention it here: h2323105.lmp contains an intercepts overflow, but it was recorded with an early version of GLBoom that didn't have emulation, so make sure it is off.

Share this post


Link to post
ultdoomer said:

so make sure it is off.

...and remember to turn it back on after watching it.

Share this post


Link to post

I've updated the tables a little bit, notably with some info concerning Ultimate Doom's e4m1, where some demos desync if emulation is turned off, and one demo needs the alternate magic number.

I have also added some tips for wad-makers (at the start of the first post in this thread) on how to avoid overflows occurring.

Share this post


Link to post

What is a "reject overflow" ? I don't see any arrays in p_sight.c that could overflow.

Share this post


Link to post

What are reject, intercepts, and playeringame overflows exactly? I love learning about this sort of pointless minutiae.

Share this post


Link to post

playeringame overflow refers to the situation described here. That is the only example I know of.

For information on intercepts overflow, see this page, from "The All-Ghosts Bug" onwards.

A reject overflow affects in-game behaviour because the vanilla engine would use junk data in place of the missing part of the reject table.

Share this post


Link to post

As I mentioned in the misc. demos thread, ACADEMY.WAD by Tony Bruno suffers from a REJECT overflow (1326<1327), although both of my demos on that map are played back correctly in PrBoom-plus with or without emulation. Should it be added to the list?

Share this post


Link to post

Yes, it could well be in the list (I'll add it next time I overhaul this), though I am not really aiming to keep the list utterly complete - there are many maps with short reject lumps, and the emulation behaviour seems to be well worked out. You were lucky with the academy demos, as emulation cannot be guaranteed in this case.

In fact, it was this precise wad that led to the realization that emulation couldn't be guaranteed when the lump size was not a multiple of 4. Link

If you wish to record further demos on this wad, I suggest using the fixed version of the wad that Andrey supplies there as an attachment - this way you'll avoid the risk of desynching demos. (I'm not sure if this is just a risk of desynching without Doom.exe's precise memory management, or if it is a risk of the demo even desynching with the exe that was used to record it, as is the case with larger reject overflows - maybe Andrey can answer that.)

Share this post


Link to post

Thanks for the info, Grazza. I'll use that file the next time I record on the map, and if I encounter another level with a REJECT overflow, I'll generate a fixed REJECT and include the fixed version with any demos I record on it. Just for the record, ACAD-055 desyncs in Chocolate Doom, but ACAD-209 doesn't. Hmm...

Share this post


Link to post

I've imported the PrBoom-plus intercepts overrun emulation code into Chocolate Doom (thanks to entryway for his tireless research, as always :-).

I've noticed that there are a few potential problems with the overrun emulation code:

  • Assumes little-endian processor: the playerstarts[] array is an array of mapthing_t structures, which contain 16-bit integers. On a big endian system, values could be swapped between some structure members.
  • Doesn't translate memory addresses of intercept_t::d.{thing,line} pointers; ideally these should be translated to the memory addresses that they would be in Vanilla Doom, or approximations. This is also a 64-bit issue.
  • Makes assumptions about the memory alignment of intercept_t structure members (fixed in my version).
Most of these are non-issues at the moment, but they could bite us in the ass in the future for 64-bit or big endian versions :-)

One thing I have noticed is that blackbug.lmp doesn't play back exactly the same as it does in doom2.exe. I don't know if this is just a bug in the Chocolate Doom overrun emulation, though (haven't tried it in PrBoom-plus yet). In doom2.exe, the sergeant walks through the player, while in Chocolate Doom he walks around him. Also, in doom2.exe, the par counter counts all the way up to 2:30, while in Chocolate Doom the demo ends before the par counter finishes.

Share this post


Link to post

I can confirm that prBoom+ doesn't play blackbug.lmp correctly either. Where in DOOM2 v1.9 the player walks through the Sarge, in prBoom+ the Sarge goes to the left of the player. Also the demo duration is reported as :22 rather than the :21 in v1.9.

Share this post


Link to post

Yes, blackbug.lmp is a known issue (mentioned in the table). I reported this to Andrey at the time he introduced the code - he had also noticed the problem, but couldn't solve it. Still, it works better this way than without emulation. :)

Share this post


Link to post

One thing I have noticed is that blackbug.lmp doesn't play back exactly the same as it does in doom2.exe

As you saw, some fields which should be overwritten during overrun are not overwritten with my code, because of possibility of access violation on current systems

Makes assumptions about the memory alignment of intercept_t structure members (fixed in my version)

What do you mean? Which line?

Share this post


Link to post
entryway said:

As you saw, some fields which should be overwritten during overrun are not overwritten with my code, because of possibility of access violation on current systems
What do you mean? Which line?

You make the assumption that an intercept_t is the same as an array of three 32-bit integers. That isn't necessarily true. It's a bit of a nitpick really. The only situation where I can imagine this being an actual problem is on 64-bit machines, where the last field in the structure would be a different size.

I'm kind of paranoid about this stuff because structure member alignment is one of those subtle things that has bitten me in the ass in the past.

Share this post


Link to post

To reduce the chance of intercepts overflows: Make sure that linedef 0 is very short, and perhaps out of the main area of play.

Is it true? I have a level with very short linedef0 (6) which is out of the main area of play, but intercepts overflow occurs permanently there

Share this post


Link to post

entryway said:
Is it true?

Didn't Grazza just meant that so that the bug found here would certainly not occur? But then an intercepts overflow could happen for other reasons even if line 0 were harmless or out of the way.

Share this post


Link to post
myk said:

Didn't Grazza just meant that so that the bug found here would certainly not occur?

That is due to the same issue, yes. The presence of a long linedef 0 tends to increase the number of intercepts, and thus make an intercepts overflow more likely. But it doesn't make one inevitable, and one can happen without this.

Before Ajapted made a change related to this, Oblige maps would quite often feature intercepts overflows. http://forums.newdoom.com/showpost.php?p=496183&postcount=43

Share this post


Link to post

Two intercept overflows in TNT: Evilution I ran in during my prBoom+ v2.4.8.2 -complevel 4 sessions.
1) on map09 "Stronghold", as you open the door leading to the nuke pool with Imps in it and two chaingunners in the back. Didn't save this, unfortunately; not even sure if it wasn't a spechits overflow.
2) on map04 "Wormhole" in the first half of the level, in the room where there are three rows of computer panels you can lower, with Imps and Hell Knights between them. It occurs at 10:58 of this 4-level run - use -skipsec 720 to get to it. The demo plays fine to the ignominious end (I didn't have much enthusiasm to go on recording) on prBoom+; but the vanilla Final DOOM exec aborts at that point with P_RemoveActivePlat: can't find plat!.

Share this post


Link to post

entryway said:
I have a level with very short linedef0 (6) which is out of the main area of play, but intercepts overflow occurs permanently there

Is that a test map meant to abuse the overflow? I'm interested in getting more technical information on avoiding intercept overflows during mapping because they seem the most unpredictable type of overflow and don't really have full emulation in PrBoom for stability reasons. The main problem is, of course, that they get even more likely with larger levels that Doom+ can take.

Share this post


Link to post

I see that as of 10-Jan-2008, fraggle has switched the default spechits magic number in Chocolate-Doom to match that in Prboom-plus. So ignore stuff I wrote above about differences in playback between the two, with respect to the current SVN version of Chocolate-Doom.

This means that Chocolate-Doom should now play back (and record) demos with spechits overflows in the same way that Doom2.exe does when running under Windows XP (previously it had played them back as if Doom2.exe were running under DOS/Win98). But remember that for the majority of these demos it doesn't matter in any case - there are just a small number of demos in total affected.

Share this post


Link to post

Simon "fraggle" Howard did: "Refactor the intercepts overrun code so that it should work properly on big endian machines as well as little endian machines". I liked it and have copied back to prboom+. Tested only with blackbug.lmp. Also, I do not have PowerPC machine to test big endian part of work, but I trust fraggle.

http://prboom-plus.sourceforge.net/history.html

Share this post


Link to post

BLOODSEA.WAD ("Master Levels")
Intercepts overflow, all ghosts, emulates ok.

blds-iof.zip

Sorry, but the demo itself is completely uninteresting :(

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now