Demos desync with doom2.wad from the BFG Edition

[fabian]

Dear DW,

could someone please confirm that 30uv2325.LMP desyncs in the middle of map02 with doom2.wad from the BFG Edition?

WTF is going on there?

- Fabian

[Gez]

BFG edition MAP02 is slightly different (at the very least, there's the easter egg exit to Betray, which means a linedef has been split, which means the nodes were rebuilt and the blockmap is probably different too) so that might be why.

Edit:

CRC-32          STD MAP02:	BFG MAP02:
THINGS:  	ebab0c73   ==   ebab0c73
LINEDEFS:	abe6319c   !=   96d3f5ef
SIDEDEFS:	992b8c40   ==   992b8c40
VERTEXES:	97fc7cd8   !=   b37e0540
SEGS:    	26783bc1   !=   3642a27f
SSECTORS:	 1b8a438   !=   b736d090
NODES:   	99783d5f   !=   dece9367
SECTORS: 	16e84977   ==   16e84977
REJECT:  	bdd1a118   !=   5e4b140e
BLOCKMAP:	da242216   !=   54e6100c

[40oz]

Thanks a lot idsoftware!

[Maes]

Well, what's the surprise about? The BFG engine itself is different and so far, its actual compatibility with v1.9 demos (or the ability to record and play ANY demos) is unknown.

[kb1]

Maes said:

Well, what's the surprise about? The BFG engine itself is different and so far, its actual compatibility with v1.9 demos (or the ability to record and play ANY demos) is unknown.

No, no surprise, but definitely a shame. With the wealth of demos out there, it's kinda irresponsible to change the map, this late in the game, if you ask me. But, I guess it's like trying to play a 1.9 demo with the 1.1 WAD file.

[Gez]

kb1 said:

With the wealth of demos out there, it's kinda irresponsible to change the map, this late in the game, if you ask me.

Remember that the map was originally changed for the Xbox port, to hide an easter egg secret level, as a way for the port's author to say "look how far I've come, I used to make crappy maps for this game when I was a kid and now I'm officially working on it!"

The change itself was not advertised, and it was not a design decision from higher up. The port was quite unlikely to get to run the wealth of demos out there.

For BFG edition, they started with the Xbox port, so that stuff was kept; more out of inertia than deliberate decision. Honestly, the weird changes they made to MAP31 and 32 are much worse.

As for giving a damn about backward demo compatibility, this has never been a worry in id software, and the guys who made the BFG edition port probably didn't even know demos were a thing that existed.

[kb1]

Didn't know the lineage since 1.9. Starting from the XBox port explains how the maps remained changed.

As for backward demo compatibility, that really wasn't feasible at the beginning, while major behavioral, structural changes were occurring. And, they were basically letting the public find the maps bugs (look at E1M3, yikes!) Understandable at that stage. It would have made no sense for them to be adding (If version < 1.4 Do_A Else Do_B) at first - hell, they were still trying to support 386 20's :)

They did care enough to add a version number in the demo header, but that probably helped in debugging more than anything. Then, of course, they ruined it with Final Doom...

Ok, I've gone full circle. Gez, you're right - they never gave a damn! But, 20 years later? If those hacking on the source and the WADs did not know about the massive Doom demo movement..., yes, shame on them, absolutely. Had they hired any of the various source port experts that frequent DW and friends, such problems would simply not be.

(hmmm, or not :)

[Quasar]

I don't think id is actually even aware that they're still using the Vicarious Visions version of Doom 2. The version of Ultimate Doom included on Xbox 360 and in BFG Edition after all is based on the original IWAD, *not* VV's version containing E1M10.

The fact they've left the non-functioning secret exit in MAP02 in two versions of the product to me says they're not aware of it being there and don't really know why there's a seemingly useless MAP33 in the IWAD but must figure it was always there and aren't worried about it.

[Dr. Sean]

Quasar said:

I don't think id is actually even aware that they're still using the Vicarious Visions version of Doom 2. The version of Ultimate Doom included on Xbox 360 and in BFG Edition after all is based on the original IWAD, *not* VV's version containing E1M10.

id is definitely aware of the differences. I (and others) have spoken to Sponge regarding the Steam version of Doom 2 and the topic of BFG Edition came up as an example. Whether or not id chooses to introduce new IWAD changes in future releases is up to them but at least the topic of maintaining compatibility is on their radar as well as a few options for making changes without breaking IWAD compatibility (hard-coding the executable to load PWADs containing changed maps instead of changing the IWAD).

[Quasar]

Dr. Sean said:

id is definitely aware of the differences. I (and others) have spoken to Sponge regarding the Steam version of Doom 2 and the topic of BFG Edition came up as an example. Whether or not id chooses to introduce new IWAD changes in future releases is up to them but at least the topic of maintaining compatibility is on their radar as well as a few options for making changes without breaking IWAD compatibility (hard-coding the executable to load PWADs containing changed maps instead of changing the IWAD).

Perhaps I should have said "were" then - I assume you had these conversations after those releases, at least or else they couldn't have been part of the topic.

[fabian]

Maes said:

Well, what's the surprise about? The BFG engine itself is different and so far, its actual compatibility with v1.9 demos (or the ability to record and play ANY demos) is unknown.

I am asking myself, if they relase an official product which contains both a doom.wad and a doom2.wad file and they even consider continuous versioning of the executable, why couldnt they care about backwards compatibility as well?

Dr. Sean said:

[...] but at least the topic of maintaining compatibility is on their radar as well as a few options for making changes without breaking IWAD compatibility (hard-coding the executable to load PWADs containing changed maps instead of changing the IWAD).

If they had just released NRFTL as a new IWAD - instead of a PWAD that requires additional hard-coded changes to the engine - they would have caused less chaos, IMHO.

[Quasar]

fabian said:

If they had just released NRFTL as a new IWAD - instead of a PWAD that requires additional hard-coded changes to the engine - they would have caused less chaos, IMHO.

That would actually be relatively more work for my engine, at least. When NR4TL was officially released as a PWAD on PC, it was trivial for me to fold it into EE's existing "managed mission pack" system, which was previously only being used for the Master Levels.

It is an extension of the SMMU runtime wad loading capability, together with a class that inherits from WadDirectory called ManagedDirectory. It adds the capability of being hashed by its file name, and the capability to track a list of the levels that have been detected inside the wad.

[Blzut3]

Likewise the NRFTL PWAD works better for ZDoom as well. It could have been done a little better, but run time lump renaming allows us to support it easy enough and the user doesn't need to use the BFG edition IWAD.

If they included it in the IWAD, we'd have to actually put up with the compatibility breaking map changes they did just to play the new mission. (Seriously, they couldn't just change the graphics?) Now we can just tell people wanting to play online to "update" their IWAD to the normal version.

[fabian]

You guys are talking about enhanced ports which are already rather prepared for that kind of modification. It was quite a hazzle, however, to get this special-casing implemented in Chocolate Doom, ne Crispy Doom, though (and I am still not sure I got everything right).

[fabian]

Sorry for bringing this up again, but i am not sure I have understood this correctly. As far as i understand, demos save what happens in every tic. So how does a changed blockmap affect "what happens in every tic"? Isn't only player movement recorded and everything else is deterministic by the engine?

[Edward850]

You seem to already know that only player commands are recorded due to the deterministic environment, but don't seem to understand why changing where things collide would have any affect on a deterministic outcome.

I'm not sure how you have put yourself in such a mindset.

[Gez]

fabian said:

As far as i understand, demos save what happens in every tic.

They don't. Demos do not record what happens; they record player input.

So they record "player moved forward this much, turned to the left this much, changed weapon to shotgun, fired". They don't record "imp saw player, hissed, moved a bit in a zigzag way, got shot, and died".

Watch a deliberately desynched demo (e.g. one for a completely different level) and see the player carefully taking aim at the void between monsters, running headfirst against the wall for a while, and doing all sorts of similar nonsense.

[fabian]

Edward850 said:

I'm not sure how you have put yourself in such a mindset.

I am not sure, either, but maybe you can help me.

In the case of MAP02 from the regular IWAD vs. the one from the BFG Edition, the level geometry (i.e. its architecture) is unchanged. One wall has been assigned a new special meaning (i.e. the secret exit switch to MAP33), but all the other walls and things are still exactly where they used to be. Or not?

[Graf Zahl]

That may be, but the blockmap is different, meaning that the order of collision checks is no longer the same.

That's enough for a demo to desync.

[fabian]

Graf Zahl said:

the blockmap is different, meaning that the order of collision checks is no longer the same

THIS is the answer, thank you!

[BlueFeena]

While I don't have the programming level understanding of the Doom code that everyone else here does, I should point out that a different blockmap could also affect its position. That means, in vanilla Doom at least, shots that should hit or miss do not act accordingly.

[Vermil]

What alternative demo formats have source ports created?

I know many ports on here are listed as having their 'own'; http://doomwiki.org/wiki/Comparison_of_Doom_Source_Ports

But how many of those are actually all new format's and not various incompatible extensions of the Vanilla Doom demo format?

[Quasar]

Vermil said:

What alternative demo formats have source ports created?

I know many ports on here are listed as having their 'own'; http://doomwiki.org/wiki/Comparison_of_Doom_Source_Ports

But how many of those are actually all new format's and not various incompatible extensions of the Vanilla Doom demo format?

As far as I know, they're all "incompatible extensions".

[DaniJ]

Doomsday demos are based around recording a stream of networked-game deltas. The network protocol and high level architecture are also very different. Absolutely nothing at all like vanilla.

[RTC_Marine]

Vermil said:

What alternative demo formats have source ports created?

I know many ports on here are listed as having their 'own'; http://doomwiki.org/wiki/Comparison_of_Doom_Source_Ports

But how many of those are actually all new format's and not various incompatible extensions of the Vanilla Doom demo format?

I'll mention that Odamex supports recording and playing (including fast forward and rewind) of network demos. It also supports recording and playback of vanilla and longtic demos.

[Vermil]

RTC_Marine said:

I'll mention that Odamex supports recording and playing (including fast forward and rewind) of network demos. It also supports recording and playback of vanilla and longtic demos.

Couldn't Vanilla Doom also record and play multiplayer demos.

I was trying to avoid port promoting in my question.