Compatibility lump - killer source port feature?

[BrainMuncher]

So there are a lot of compatibility options in modern source ports. When designing a wad, I feel like you're pretty much stuck with targeting default port behaviours, because expecting people to read text files and change individual compatibility settings to reach the desired behaviour of your wad is a bit much.
Even where ports are seemingly capable of compatibility, they can do things differently by default. For instance prboom+ and zdoom do boom scrolling floors differently, and there are other things you have to be aware of when making a map like "things" blocking projectiles, monsters falling off ledges, infinite monster height etc.

But imagine if you could put a compatibility lump in your wad that told the source port to enforce certain compatibility options. You could make it so monsters don't fall off ledges, and have your wad work identically across multiple ports without the player having to even think about compatibility! Demos would be always work as long as you played them back in the port they were recorded in, because the wad is needed to play the demo and the wad has the compatibility options in it.

Instead of releasing a "boom compatible" map and having to work around slightly different behaviours in different ports, you could release a map with "compatibility: prboom+, zdoom, zandronum" (or whatever) with a compatibility lump for each port. Then the player could choose their preferred port and still have the game behave the way the author intended.

Is this a stroke of genius or am I a moron? Please tell me why this is a good/terrible idea.

P.S. This would still be useful even when targeting something like vanilla compatibility where setting individual options is out of the question. You could put the equivalent of -complevel 2 in the prboom lump, whatever combination of options best approximates vanilla in the zdoom lump, and so on. So the player wouldn't even have to worry about remembering to use the correct complevel.

And it should be fairly easy to implement, since ports in general already have all these settings in ini files. The lumps could basically be incomplete ini files with only the compatibility options that need to be set, and let the user choose the ones that aren't critical for the functioning of your wad. So port authors wouldn't even have to define or document any new settings since they are the same as the ini file settings and in the same format!

[Quasar]

The problem isn't it not being possible, but how much more complicated it really is than you're thinking. PrBoom+, OK. What version? At what complevel? Within that complevel, IF the complevel supports it, what MBF comp bits are on or off?

[Gez]

As far as ZDoom is concerned, you can set or unset most compatibility settings in MAPINFO.

[Linguica]

It's not a bad idea, but as with lots of not-bad ideas involving source ports, the problem is that it's impossible to get done without endless amounts of bikeshedding, and even if it got done, it's impossible to make people use it.

[BrainMuncher]

Gez said:

As far as ZDoom is concerned, you can set or unset most compatibility settings in MAPINFO.

I didn't know that, that's pretty much exactly what I had in mind but better, as it allows you to have different compatibility for each map in the same wad. I will start using it immediately.

Quasar said:

The problem isn't it not being possible, but how much more complicated it really is than you're thinking. PrBoom+, OK. What version? At what complevel? Within that complevel, IF the complevel supports it, what MBF comp bits are on or off?

Well, that's the point of having all of that info in the wad itself, so you never have to worry about any of that. Everything that's not explicitly specified would be exactly as it is now, whatever the user has configured.

Linguica said:

It's not a bad idea, but as with lots of not-bad ideas involving source ports, the problem is that it's impossible to get done without endless amounts of bikeshedding, and even if it got done, it's impossible to make people use it.

While I appreciate your cynicism, being quite cynical myself, as gez pointed out it looks like zdoom already has this capability, by including a ZMAPINFO lump with the compatibility options in it. I'd hope bikeshedding (I learnt a new word) wouldn't be an issue because it's not making something new, only automating something that everyone already does (such as playing a vanilla map with the appropriate complevel/compatmode). And making it more granular.

[Linguica]

Well sure, making a ZDoom WAD bypasses the whole issue because then you can just do what ZDoom supports. But experience has shown that trying to promote some generalized cross-port thing is doomed to failure.

[BrainMuncher]

Quasar said:

The problem isn't it not being possible, but how much more complicated it really is than you're thinking. PrBoom+, OK. What version? At what complevel? Within that complevel, IF the complevel supports it, what MBF comp bits are on or off?

I think I misinterpreted what you were getting at here. Perhaps this will make it more clear:

The point isn't for one port to behave exactly as another port. I'm not suggesting a "zdoom 2.81 with monsters see invisible players" mode to be built into eternity, for instance. The point is to allow a wad author to specify which compatibility options should be set while playing their wad, since all ports have different defaults. Wad authors already do this, but they do it in a txt file saying "you should play this with options x and y", and the player has to manually configure those options. All I'm suggesting is to have that same information in the wad itself in a form readable by the port.

Linguica said:

Well sure, making a ZDoom WAD bypasses the whole issue because then you can just do what ZDoom supports. But experience has shown that trying to promote some generalized cross-port thing is doomed to failure.

That's not what I meant. If I put a ZMAPINFO lump in a vanilla wad it only affects zdoom. I can play the wad in chocolate doom without it having any effect whatsoever. But if I play in zdoom, it could automatically know which compatibility options to use, because they are set in the wad.

This wouldn't be a generalized cross-port thing. This is each port implementing (or not), a configuration lump in whatever manner they feel is appropriate. Then it's up to the wad author to include those settings (or not) in their wad.

[Graf Zahl]

BrainMuncher said:

since all ports have different defaults.

And here is the problem:

Each port has different options, and different ports may need different options to make a map work.

A good mapper who wants to target multiple ports normally makes sure that their map does NOT need any compatibility option to be set at all! Most are not to enable certain features but to re-enable old bugs. Clean mapping does not need to rely on bugs.

To make a long story short: Either a mapper dilligently tests their map on all ports they want to support and make sure that it doesn't run into problems or they don't really care in which case any feature to help them out is probably going to be ignored as well.

From an end user perspective this may be a nice idea but the sad fact is that precisely those people who should use such a feature are the least likely ones to use it in the first place, leaving the mess to the end user. Which is precisely the reason why I added the compatibility options to MAPINFO - so that I can set them myself for any maps that needs them.

[Maes]

First there would need to be a standard for compatibility. De-facto there are a few well-known ones, e.g. vanilla Doom, original Boom extensions, original MBF extensions etc. as well as game mode (Doom, Ultimate Doom, Doom II, v1.9, Final Doom, maybe specific older versions etc.) so that's a starting point. In theory, at least Vanilla and Boom can function as minimum common denominators for most ports, since they must at least have vanilla v1.9 compatibility and/or a Boom base (itself, vanilla-compatible with a lot of asterisks). In any case, Vanilla and Boom should be considered a sort of "ground truth" for everybody.

For stuff that's specific to ports other than that "ground truth", e.g. ZDoom, Eternity or Edge-specific behavior, there would need to be comitees which decide which subsets of those features are worth the effort to have in all ports, and already have functional equivalents across different ports (e.g. the same feature might have a different name or a different parameterization) and if necessary, interface or complement them as needed. Of course, there will be countless debates if feature X from ZDoom should be a "must have" for all ports, or if everybody should do feature Y the way Eternity does.

Since we know nobody is gonna do the latter, I'd say stick to "ground truth", and maybe add a sort of manifest information in the lump if your map only runs with specific source ports (e.g. a specific version of ZDoom). In such cases, the lump will only serve as a warning that the map will not work properly in your port if you try loading it. In essence, it's like marking your map "vanilla", "boom", ZDoom"-compatible etc.

[Gez]

Linguica said:

Well sure, making a ZDoom WAD bypasses the whole issue because then you can just do what ZDoom supports. But experience has shown that trying to promote some generalized cross-port thing is doomed to failure.

A cross(-port compatibility option lump is never going to be feasible, because each port that could theoretically join such an endeavor has completely different compatibility settings.

What makes sense is to have each port has its own mechanism for specifying compatibility settings as required; anything else wouldn't make sense.

[BrainMuncher]

Graf Zahl said:

And here is the problem:

Each port has different options, and different ports may need different options to make a map work.

That's exactly the reason this is/would be useful, the mapper can set different options for different ports where appropriate.

Graf Zahl said:

A good mapper who wants to target multiple ports normally makes sure that their map does NOT need any compatibility option to be set at all! Most are not to enable certain features but to re-enable old bugs. Clean mapping does not need to rely on bugs.

To make a long story short: Either a mapper dilligently tests their map on all ports they want to support and make sure that it doesn't run into problems or they don't really care in which case any feature to help them out is probably going to be ignored as well.

This ends up severely limiting your options though as you target more ports. For instance in most ports missiles can go through trees, but not in zdoom.
For a diligent mapper just trying to target default behaviours, without that ZMAPINFO lump they might have to conclude that they simply can't use trees if they want a consistent experience. There are tons of little things like this and you end up with almost no options if only targeting defaults. Which is why most ports have a few baseline compatibility presets like vanilla and boom, so mappers have a reasonably consistent behaviour to target. But this comes with its own limitations.

In other words, everyone already does it, except the config info is put on the command line instead of in the wad to be applied automatically. And we're basically limited to a handful of presets because anything else is too inconvenient for the end users.

From an end user perspective this may be a nice idea but the sad fact is that precisely those people who should use such a feature are the least likely ones to use it in the first place, leaving the mess to the end user. Which is precisely the reason why I added the compatibility options to MAPINFO - so that I can set them myself for any maps that needs them.

Well FWIW now that I know it's there I'm going to use the ZMAPINFO feature in levels I make, where appropriate. Thanks for putting it in there.

Maes said:

First there would need to be a standard for compatibility. De-facto there are a few well-known ones, e.g. vanilla Doom, original Boom extensions, original MBF extensions etc. as well as game mode (Doom, Ultimate Doom, Doom II, v1.9, Final Doom, maybe specific older versions etc.) so that's a starting point. In theory, at least Vanilla and Boom can function as minimum common denominators for most ports, since they must at least have vanilla v1.9 compatibility and/or a Boom base (itself, vanilla-compatible with a lot of asterisks). In any case, Vanilla and Boom should be considered a sort of "ground truth" for everybody.

For stuff that's specific to ports other than that "ground truth", e.g. ZDoom, Eternity or Edge-specific behavior, there would need to be comitees which decide which subsets of those features are worth the effort to have in all ports, and already have functional equivalents across different ports (e.g. the same feature might have a different name or a different parameterization) and if necessary, interface or complement them as needed. Of course, there will be countless debates if feature X from ZDoom should be a "must have" for all ports, or if everybody should do feature Y the way Eternity does.

Since we know nobody is gonna do the latter, I'd say stick to "ground truth", and maybe add a sort of manifest information in the lump if your map only runs with specific source ports (e.g. a specific version of ZDoom). In such cases, the lump will only serve as a warning that the map will not work properly in your port if you try loading it. In essence, it's like marking your map "vanilla", "boom", ZDoom"-compatible etc.

I think you've missed my point a little bit. The point is to expand the options available to mappers by allowing ports to auto-configure themselves based on a lump in the wad. There doesn't need to be any standards decided upon, no presets, no committees, hell there doesn't need to be any communication between port authors whatsoever. Just each port makes available whatever options are already there, to be applied automatically from a wad lump. zdoom already has ZMAPINFO. Maybe cripsy doom could take CRISPINI, prboom PRBCFG, etc.
They don't even have to use the same format, one port might decide to make the lump a one-byte bitfield, with only two options available. Another might use xml syntax. It doesn't matter. Although the simplest way would be to use the ini/cfg file format that they already use to read their settings startup, because less work.

Gez said:

What makes sense is to have each port has its own mechanism for specifying compatibility settings as required; anything else wouldn't make sense.

Exactly this.

[Graf Zahl]

Well, ZDoom already has all the means to deal with it. It had this stuff for over 10 years.

And still, you can probably count the number of maps that actually did care enough on the fingers of your two hands.

90% of all vanilla compatible maps don't even have a MAPINFO lump.

Like I said before, you do not have to convince the port developers but the mappers.

[Maes]

BrainMuncher said:

I think you've missed my point a little bit. The point is to expand the options available to mappers by allowing ports to auto-configure themselves based on a lump in the wad. There doesn't need to be any standards decided upon, no presets, no committees, hell there doesn't need to be any communication between port authors whatsoever.

The way such proposals are implemented in the real world often call for some sort of comittee or at least a "gentlemen's agreement" between all interested parts. Otherwise, they remain just pie-in-the-sky wishful thinking.

BrainMuncher said:

Just each port makes available whatever options are already there

And how do you propose making this happen without some "Grand Source Port Unification" of some sort? I mean beyond the level of vanilla/boom compatibility.

BrainMuncher said:

zdoom already has ZMAPINFO. Maybe cripsy doom could take CRISPINI, prboom PRBCFG, etc.
They don't even have to use the same format, one port might decide to make the lump a one-byte bitfield, with only two options available.

Wait, so this lump is going to be port-specific after all? With every port only being able/supposed to read its own? So what if tomorrow I make a new port, do I kindly request that whoever makes "compatibility lump" editors also add one for my own port? Also, tool authors are supposed to know how to generate lumps for each port specifically?

BrainMuncher said:

Another might use xml syntax. It doesn't matter. Although the simplest way would be to use the ini/cfg file format that they already use to read their settings startup, because less work.

OK, so this makes things a bit clearer: essentially you propose that for each port, there's simply a kind of "patch" to their ini/cfg systems, with the required compatibility options already preset, and of course specific only to those ports, not as a way to interpret settings in a more general way. That would have more chances to work, but then again, why simply not give an external CFG file (or rather, a way to append settings to a loaded configuration).

IMO, even with this simplified scenario, it would be a redundant feature: ZDoom-only maps only work in ZDoom, and in 99.999% of cases you don't have to fudge with settings (and, if you do, as you said there's already ZMAPINFO). For Boom/prBoom+ maps, usually you only need a single complevel setting, written in the .txt. For vanilla you need nothing, obviously...that's really a solution in search of a problem.

[Gez]

Maes said:

Wait, so this lump is going to be port-specific after all? With every port only being able/supposed to read its own?

Yes, because anything else would be dumb.

Maes said:

So what if tomorrow I make a new port, do I kindly request that whoever makes "compatibility lump" editors also add one for my own port? Also, tool authors are supposed to know how to generate lumps for each port specifically?

Make it a text lump and you won't need any sort of tool that doesn't already exist.

[Maes]

Gez said:

Yes, because anything else would be dumb.

This entire feature needs to be dumbed down so much that it barely makes sense. Can be summed up as: "put a patch/custom cfg file for each port into a specialized lump".

[Graf Zahl]

and then we get mods that put in a blanket list of options that are not needed but may compromise the entire experience of playing in a modern port.

Think of any stupid application of compatibility options, and you'll find someone who tried it.

[BrainMuncher]

Maes said:

OK, so this makes things a bit clearer: essentially you propose that for each port, there's simply a kind of "patch" to their ini/cfg systems, with the required compatibility options already preset, and of course specific only to those ports, not as a way to interpret settings in a more general way.

Right, that's what I was going for all along, I guess I didn't explain it clearly enough.

Maes said:

That would have more chances to work, but then again, why simply not give an external CFG file (or rather, a way to append settings to a loaded configuration).

That's essentially what I'm proposing, a cfg file embedded in the wad. But only for compatibility settings, because I'm pretty sure users wouldn't want wad authors messing with their mouse sensitivity or screen resolution.
I can't say I'm familiar with all of the myriad of tools people use, but something like slade can already put an arbitrary file or lump of text into a wad. There would be no need for "compatibility lump editors" to be made.

Maes said:

IMO, even with this simplified scenario, it would be a redundant feature: ZDoom-only maps only work in ZDoom, and in 99.999% of cases you don't have to fudge with settings (and, if you do, as you said there's already ZMAPINFO). For Boom/prBoom+ maps, usually you only need a single complevel setting, written in the .txt.

Usually you only need a single complevel setting because mappers have made sure that's all you need. And many people don't even do that much, maybe they just drag the wad onto the exe with no settings at all.
I'm guilty of this myself, I always try to use the right setting but sometimes I forget or mess up. I played someone's level and forgot to change from doom 2 mode to boom mode. It was only afterwards that I realised what was going on in those mysterious empty rooms with monster sounds emanating from the walls. I didn't feel like replaying the entire level just to see the few things that didn't work right. Wouldn't it have been nice if the correct settings were applied automatically for me?

Maes said:

For vanilla you need nothing, obviously...that's really a solution in search of a problem.

It's not obvious at all. Some vanilla maps might work fine with default behaviour in all ports. Some might only need one specific vanilla thing to work correctly. Some might need the full suite of strict vanilla behaviour.

The more granular settings are already there, why not let mappers use them? If the only setting that's actually important for playing your map is that monsters don't fall off cliffs, and all the other settings don't matter, you can just put that one setting in, it gets applied automatically, and the player can decide the rest. Instead of making them read a txt file, and manually apply a preset group of unnecessary settings that don't actually matter.

Graf Zahl said:

and then we get mods that put in a blanket list of options that are not needed but may compromise the entire experience of playing in a modern port.

That's exactly what people do now. Apply -complevel or -compatmode, with a blanket list of options that probably aren't needed, just because one of them *might* be.

Let's say someone uses zdoom because they like to be able to run under cacodemons and whatnot. They download a community megawad project that targets boom compatibility. It might happen that all of the maps work just fine in default zdoom, but there's no way for them to know that. So this dilligent player might see the "boom compatibility" and enforce that preset, causing them to be unable to run under cacodemons against their preference, despite the fact that it doesn't actually matter.

So yes, mappers could put in a blanket list of options that don't matter. But the reverse is also true and already happening, players applying blanket lists of options when they might not need to.

It is possible that irresponsible mappers will put in unnecessary options. So what? Bad mappers can already make bad maps. The only thing that could be compromised is the quality of their map and their reputation.

[Graf Zahl]

And do you really expect that Vanilla mappers care about such things?
You can argue as much as you want, the end result will always be that they either do nothing or too much. It's not the lack of a feature that makes them act like this but a lack of awareness of a potential problem with a port they rarely or never use.

The problem is simply not solvable in such a simplistic manner.

That said, have a look at the list of maps ZDoom sets an option for. It's really not that long. The vasr majority of maps out there plays fine, it's a tiny subset that needs special treatment.

[BrainMuncher]

Graf Zahl said:

And do you really expect that Vanilla mappers care about such things?
You can argue as much as you want, the end result will always be that they either do nothing or too much. It's not the lack of a feature that makes them act like this but a lack of awareness of a potential problem with a port they rarely or never use.

The problem is simply not solvable in such a simplistic manner.

That said, have a look at the list of maps ZDoom sets an option for. It's really not that long. The vasr majority of maps out there plays fine, it's a tiny subset that needs special treatment.

The problem is solvable, you've already solved it for zdoom.

Many mappers are meticulous and will spend months agonising over texture choice and alignment, monster placement, etc. Then testing in multiple ports to makes sure everything works right, and making changes to work around inconsistencies. Most things they can rely on to always work the same, like if they put line action 29 on a switch to open a door, they know it will work in all ports. A soulsphere always gives 100 health, but the mapper can change that if they want with a dehacked lump. The cyberdemon always shoots rockets. A lift always moves at the same speed. But some things, are not reliable. A tree may or may not block projectiles. A monster may or may not fall off a ledge, etc.

I don't see how letting the mapper decide whether the tree blocks the projectile or not is any different to letting the mapper decide whether the switch opens the door or not, how much health a soulsphere gives, or how the textures are aligned.

[Graf Zahl]

If mappers cared about this, maps would already come with a ZDoom compatibility lump. But guess what: They don't!

If mappers won't even do this for one port, what do you think are the chances that they do it for multiple ones?

The best you can expect is a note in the readme saying "Best played with PrBoom on complevel 2." Note that they are not FORCING complevel 2 but RECOMMENDING it. Which is a major difference.

[snapshot]

There's a ZDoom ACS Script command that can check current value of a command before it is executed, this can be used to warn players of incorrect gameplay settings or totally disallow players from viewing game (fades or hudmessages ... etc) .

[Gez]

DMGUYDZ64 said:

There's a ZDoom ACS Script command that can check current value of a command before it is executed, this can be used to warn players of incorrect gameplay settings or totally disallow players from viewing game (fades or hudmessages ... etc) .

What is simpler: writing a mapinfo lump that turns on (or off!) a compatibility option, or writing a script (for a non-ZDoom map format!) that looks at compatibility settings and warns if one isn't set right?

[Graf Zahl]

Hey, did you really expect anything intelligent from the resident Ignorant Member??? :>

[snapshot]

Graf Zahl said:

:>

Heh, i mentioned that ACS Thing because i was aware people may turn that option on/off right after map is loaded (MAPINFO is only useful when map is loaded), but when you actually literally prevent them from changing it while playing, that would be much better .

Oh by the way ..... :>

[Graf Zahl]

You can't turn a compatibility option off that is being set or unset through MAPINFO. Those are deemed important for playing and override the user settings.

[snapshot]

Really ? didn't know that .

[kb1]

I think the cleanest way to gain momentum on this is in a specialized launcher that knows the options available in most ports, and can set them before launching the port.

If you got that far, the community could build configuration files for each wad that needs these specific settings. Those config files could be stored on a server and downloaded.

For this to really work, the community would have to be generating these config files. Alternately, you could build your own each time you go to play a wad.

The power of this approach is that the source ports need not comply - it's up to the launcher to have the knowledge about which options were available. This launcher would need to be regularly updated, as ports gained extra features.

I would love to see this happen. It's quite frustrating to be 1 hour into a wad, only to discover that you can't possibly complete the map. I try not to cheat, so I end up humping all the walls in the level, before resorting to noclip, idkfa, or iddt.

To me, this is the only way to gain momentum towards the goal. Any time you require multiple port authors to support something, it typically goes south. But, if this approach DID become popular, port authors may make efforts to make better support for the launcher.

Yep, last time we got hung up on which flags should be in the config files - it got quite ridiculous. It's a shame really - it seems that there's a hidden force preventing ports from working together at times.

[BrainMuncher]

But if each port implements its own port-specific lump, they don't need to co-operate. The mapper is already deciding which ports his map will work on when they make the map, and testing in those ports. They already have intimate knowledge of what options are and are not required. At that point, it's not much of a leap for them include a config lump for each of those ports.

For historical wads that aren't going to get updated, you could still do the community-generated config thing if you wanted. Just distribute a PWAD containing nothing but the various config lumps, to be loaded alongside the main PWAD. No need for a specialised launcher. Community wad projects could include the lumps in their resource file, so that all contributors do their testing with the same options while building the maps, and then eventually players play with those same options.

[Jon]

BrainMuncher said:

But if each port implements its own port-specific lump, they don't need to co-operate. The mapper is already deciding which ports his map will work on when they make the map, and testing in those ports. They already have intimate knowledge of what options are and are not required. At that point, it's not much of a leap for them include a config lump for each of those ports.

On the other hand, if you are a port author and interested in implementing this feature, surely you'd be tempted to be compatible with the existing implementation that's out there?

[kb1]

These ideas are good, and they usually always fail because they require all the port authors to comply before there's even any data available.

With confidence, I can state that the launcher approach is the best bet. It's a place to start. No, it's not the best approach, but it does a couple of things:

1. It forces the enumeration of all settings that all of the ports offer.
2. It forces the creation of a standardized compatibility file format.
3. It provides a standard interface to enter the data.
4. It facilitates the creation of the database, for anyone that plays Doom, not just port and map authors.
5. It gets the feature working immediately, without requiring the port authors to do stuff.
6. It gets a database created.

If the data is there, and the mechanism for using the data is there (the launcher), and it can be demonstrated that the process works, then you have the port author's attention. At that stage, the developer has:
1. A standard file format, with predefined parsing rules.
2. Validation that the data works.

At that stage, it becomes a feature work implementing. But, without data? It's chicken and egg. The launcher approach lets you "go back in time", so to speak. Really, it doesn't have to be a launcher - it could be called a multi-port config file manager.

You could even go simpler, and just have a spreadsheet that lists the appropriate settings to set for each wad, for each port. A simple app could do an MD5 hash of the selected wad(s), and lookup the settings in the data file, and, in Phase 1, just list the appropriate settings on the screen. For Phase 2, the app could actually edit the desired port's config file.

But, for port authors to add support for this data, the data must already exist, be in a standard format, be easy to parse and read, and be proven to be effective.

Now, for the data to exist, people will have to do data entry, wad by wad, port by port.

And, finally, for people to be motivated to enter that data, it must be easy to enter, and immediately beneficial, which is why I suggested a launcher. That's my best guess.

Good luck - I'd really love to see some progress on this!