Gothic DM 2's brokenness in BOOM-based ports

[Quasar]

I guess it has to be common knowledge by now that the stock gothic2.wad file won't load in any BOOM-derived source port that uses the unmodified W_CoalesceMarkedResource algorithm. My question is why nobody's ever thought about addressing it during all this time.

It appears to me that the cause of the issue is an unmatched FF_START demarcator which appears as the last lump in the wad. This rubs out any namespace that gets coalesced after it, leading to, in Eternity, the following error:

WadDirectory::GetNumForName: S_START not found!

I would suggest that the simplest fix is to ignore demarcator lumps that occur at the very end of a wad directory.

EDIT: On closer examination I see it's missing its proper F(F)_START lump at the beginning of its flats entirely. What a mess >_>

[DaniJ]

We can't use a heuristic analysis of the whole lump directory because all we have is the lump size (4096 bytes) to base it on. However I suppose you could reverse back up the lump directory from the F_END marker and stop when the size differs. This could be a fallback algorithm that is ran after the normal approach if it fails to touch all lumps named F(F)_START/_END

Certainly not fool-proof but its something to consider.

EDIT: Ultimately the fact of the matter is that vanilla DOOM doesn't treat the flat markers as a "namespace". Therefore it could be argued that we are trying to apply logic that doesn't work within the "rules".

We can't really describe the mod as being "broken" when its the new coalesce algorithm which stops it from "working".

[Gez]

DaniJ said:

EDIT: Ultimately the fact of the matter is that vanilla DOOM doesn't treat the flat markers as a "namespace".

Aren't you confusing the flat markers and the patch markers? There's this code in vanilla Doom:

//
// R_InitFlats
//
void R_InitFlats (void)
{
    int        i;
    
    firstflat = W_GetNumForName ("F_START") + 1;
    lastflat = W_GetNumForName ("F_END") - 1;
    numflats = lastflat - firstflat + 1;
    
    // Create translation table for global animation.
    flattranslation = Z_Malloc ((numflats+1)*4, PU_STATIC, 0);
    
    for (i=0 ; i<numflats ; i++)
    flattranslation[i] = i;
}

[Graf Zahl]

Boom's idea of merging namespaces was a design flaw from the very start. Like a lot of things in Boom the underlying idea was correct but the implementation was shit. This should have been done differently and the hacks ZDoom had to work around some of the problems were hideous.

Fortunately, with a proper texture manager, none of this is necessary anymore so since the last resource manager rewrite namespace merging is no longer performed which makes handling of such 'broken' WADs a lot easier.

[DaniJ]

Gez said:

Aren't you confusing the flat markers and the patch markers?

No, I'm not confusing it at all. W_GetNumForName returns the index of the last lump found in the entire directory using a reverse name search.

This means that if the second group of flats in the PWAD is missing it's F_START marker all lumps between the F_START in the IWAD and the F_END in the PWAD will be interpreted as a flat.

Worse still is the situation where the last F_START and F_END is inversed (which very nearly happened in gothic2 bar for a single 'F' character).

So in short, the implementation in both DOOM and BOOM is pretty crap to say the least.

EDIT: Whoops, made a silly mistake in my calculation and removed it. I've been awake for nearly 36 hrs...

[Gez]

I know about the flat hack. I don't see how that makes it not a namespace, though.

[DaniJ]

Its not a valid space because the lumps that are included aren't all of the same type. Using the configuration seen in gothic2 you can actually coerce vanilla DOOM into loading any lump at all as flat data.

EDIT: Yes, I'm aware that this doesn't actually "hurt" anything. I just don't think it can be thought of as a "namespace" when lumps outside of the supposed boundaries of that space are included in it.

[Gez]

That just makes it a poorly-implemented namespace rather than not a namespace at all. (Contrarily to patches where the engine does not actually make use of the P_ markers from the IWADs.)

[Quasar]

I don't really see eliminating the namespace coalescence as an option for EE, at least not at this point in time. I may have to put in a specific hack for this wad file for now.

[Graf Zahl]

Quasar said:

I may have to put in a specific hack for this wad file for now.

I have to say that removing the hack was actually more fun.

No idea if it may help you but you can check older ZDoom sources for the ScanForFlatHack function and see if it's good for your purposes.

[Quasar]

Graf Zahl said:

I have to say that removing the hack was actually more fun.

No idea if it may help you but you can check older ZDoom sources for the ScanForFlatHack function and see if it's good for your purposes.

At this point I'm looking at doing something with SHA-1 hashes based on the combined header and directory data of the wad file, and then I can do some highly specific treatments for any given wad that needs it via a hashed set of pointers to "fix-up" routines specific to each nasty situation that might be identified, like this one.

[Graf Zahl]

Sounds like a mess to me. When I rewrote the ZDoom resource manager one of the main goals was to keep it hack free and capable of dealing with broken WADs in the most generic manner possible.

[Gez]

Doesn't seem too different from the automatic compatibility settings in ZDoom, except on the archive level instead of the map level.

[Graf Zahl]

The same applies there. It's a necessary evil but by no means a good solution.