Doomworld Forums - Wad compatibility option database

Doomworld Forums : Powered by vBulletin version 2.2.5

Doomworld Forums > Classic Doom > Source Ports > Wad compatibility option database - ideas

Pages (6): « 1 2 3 [4] 5 6 »

Author

All times are GMT. The time now is 04:52.

kb1
Junior Member

Posts: 167
Registered: 11-06

DaniJ said:
Edit2: Huh? The mask belongs to the database? Since when? The whole point is that the "absolute" hash (i.e., same for all supporting sourceports) is in the database, while the mask(s) exist in the implementer's domain (sourceport, editor, whatever...).

YES!!! That's what I'm saying!! The mask exists in the hash!! Or as another field the same size as the hash, but in the database.

From that point, yes, the implementer's may then choose to mask further, if desired, to his heart's content, but the database entry would tell the implementor that the community believes that, say, only these 3 fields are really important.

That's what all that interleave stuff was that I was discussing way back when. I was trying to choose a format that would easily fit into a database field, to try to convince you that it really didn't add complexity.

09-13-11 02:25 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

Ugh! Triple post!!

pseudocode:

code:
#define BIT_USE_DOOM_STAIR_METHOD = 0x08;

use_doom_stair_method = userdefaults.use_doom_stair_method;

optionhash = get_compatibility_db_hash();
optionmask = get_compatibility_db_mask();

if (optionmask & BIT_USE_DOOM_STAIR_METHOD)
{
    // only override user preference if
    // db mask enables the option, otherwise
    // leave it as the user wants 
    use_doom_stair_method = !!(optionhash & BIT_USE_DOOM_STAIR_METHOD);
}

09-13-11 02:35 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

I understand the point. I don't understand how it applies.

The database cannot know whether a given option is even applicable to some port. Let alone what the user's preference is. Only the port has that knowledge.

The options set in the database are absolute because they define a change to vanilla logic in some port (it doesn't matter which). Vanilla logic itself does not change, it is set in stone.

Edit: Clearly you are wanting to have a set of hashes for each port in the database. What I'm saying is that you don't need to if you inverse the problem.

Last edited by DaniJ on 09-13-11 at 02:46

09-13-11 02:40 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Moriarti
Warming Up

Posts: 19
Registered: 02-04

It's not about whether a given option is applicable to some port. It's about whether a given option is applicable to some map.

09-13-11 02:45 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

DaniJ said:
The database cannot know whether a given option is even applicable to some port. Let alone what the user's preference is. Only the port has that knowledge.

The database assumes nothing, and believes that it may be thrown into any old port.

DaniJ said:
Edit: Clearly you are wanting to have a set of hashes for each port in the database.

Absolutely false. It the port's job to interpret the database's info, but it's the map's job to know which options are relevant or irrelevant.

By your logic, why not just have a single bit per map?: 1=user preference, 0=Vanilla Doom

EDIT: The "WCDB" should only describe exceptions. And, I don't mean exceptions to Vanilla behavior. I mean exceptions to user preferences. What I describe would leave 90% of the database with completely blank entries, which is a good thing. It's just the rare strange map that requires a strange setting to play properly - that's what I'm trying to target.

Yes, of course, ports will vary in their interpretation, but hopefully not so much as to make the mechanism not work properly.

Please think along the lines of "required exceptions to user preferences".

Last edited by kb1 on 09-13-11 at 03:08

09-13-11 02:57 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

You have the design completely back to front. The port isn't asking the database "hey which of my options are vital for this to work in my port" its asking which of the vanilla behaviours are vital for the map to work in vanilla (otherwise the port can do whatever it pleases with the rest).

09-13-11 03:06 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

DaniJ said:
You have the design completely back to front. The port isn't asking the database "hey which of my options are vital for this to work in my port" its asking which of the vanilla behaviours are vital for the map to work in vanilla (otherwise the port can do whatever it pleases with the rest).

No, the database is asking the port "Port, can you try your best to support these things, because the community has noticed that these things are really important." See previous post about Exceptions.

09-13-11 03:11 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

Ok, I fold. I'll caulk it up to being being incapable of communicating the design. I'm looking at the problem from an entirely different perspective.

09-13-11 03:24 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

I've been thinking about the problem further since my last post. I'm going to take one more stab at explaining why I think kb1's solution is unworkable because I would actually like to see a solution of some description. The core issue is "port variance".

kb1's solution persists which logical user preferences a port must override for a given map into the database. However in this scheme you have no knowledge of what the override should be (emulate vanilla or not?) hence the desire to include additional metadata into the persistent hashes. Furthermore, this solution fails to consider that the need to override a user preference is port-variant, therefore in this scheme you invite contention at central database level among the users of different ports.

My solution inverses the problem by persisting which vanilla behaviours are vital (and which the port must respect) for a given map into the database. This is port-invariant by definition. Therefore this is the logical opposite of kb1's solution because rather than exempting a user preference, it defines an absolute exemption to whatever implementation port X has used to replace a vanilla logic; i.e., don't use your normal algorithm - replicate the logic of vanilla.

Apologies for the double post.

09-13-11 19:47 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Graf Zahl
Why don't I have a custom title by now?!

Posts: 7130
Registered: 01-03

Agreed.

Even ZDoom's compatibility defaults work that way.

In general people having compatibility options enabled as their default need to be aware that this is not a good idea when playing port specific mods.

@kb1:
Sorry, but I am thinking you are making a mountain out of a molehill. If you complicate a simple thing to the point where nobody can follow you it's inevitable that you won't get any support.

09-14-11 00:41 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

wesleyjohnson
Member

Posts: 486
Registered: 04-09

There are two almost independent projects here, doom database server on the web, and port compatibility option input to source ports.

I see no way that this can go anywhere if it requires source ports to host any web database server. And hash communication violates all the local user editing requirements.

It can be broken into two projects. A common file format that can be parsed by source ports and edited by ordinary dumb users. Separately, a database that can store this same data and produce such common files on demand, over the web or not.

The common file format needs to come first. When you start collecting data it needs to be recorded in this file format. No one wants to have to go over and hand-convert 100 files to a new format.
Until this data is actually collected, it is only a rough guess as to what all the keywords and value ranges are going to be. Whatever limited format you have when this is started, it is going to get expanded in unwelcome places, so might as well plan on an easy to expand format.

Expansion of the format must be independent for each source port.

There are many source ports and they have different compatibility options. They are certainly not limited to BOOLEAN.

To keep ports from clashing it will be necessary to be able to distinguish an entry meant for only one particular source port.
This becomes necessary because it is inevitable that two source ports will have the same keyword and use it differently. Also, some port may need a switch, but other ports can keep user preference settings.
Example:
YELLOWKEY=Y
(ZDOOM) SPRITELIMFIX=Y
(VAVOOM) SPRITELIMFIX=2
# other ports can use user preferences
A leading () would list the port names.

Some ports might have 1/0 settings, but others will need menu choice
answers. Also most answers should not even be in the form of true/false, as that is too one port, one concept limiting.

I see the value as being:
- : default (don't care)
0 or N : clear the compatibility option
1 or Y : set the compatibility option
<name> : by name selection from a port specific selection list
(which is more tolerant of changes between port versions)
VAN : set it to whatever is closest to buggy vanilla
BOOM : set it to whatever is closest to Boom defined behavior
MBF : set it to whatever is closest to MBF defined behavior
HER : set it to whatever is closest to Heretic buggy behavior
HEX : set it to whatever is closest to Hexen behavior
REQ : required to play (a required 1)
SUG : suggested for best play, but not necessary (an optional 1)

Tried to make it to one character to parse (not two bits), but names are inevitable (for multi-choice options), and might as well plan for them now. Such choices should not be encoded into bit meanings. That is too much to remember and too limiting. It should say exactly what is meant, and the value keywords should be easily expandable when new PWAD problems are discovered.

How a port handles something like SUG is not important now. It is important to record the discovered information accurately. The ports will evolve, and may or may not become able to use the information later.

You do not want to be going back and looking at 1000 of these files to see if the latest release of xxx-port has changed anything.
That is the biggest reason that 1/0 encoding is self-defeating.
After too many new version releases this whole system could become a new huge compatibility liability.

Versions will inevitably become important.
Let (ZDOOM<1.3) mean, apply this line to ZDOOM of version less than 1.3 (and any other port that understands and can handle that).

Example:
YELLOWKEY=Y
(ZDOOM<1.3) SPRITELIMFIX=REQ
(VAVOOM) SPRITELIMFIX=ALWAYS

Any language (JSON, XML, etc.) violates the editable by the dumb user requirement. Finding the assignments in the examples would require a course in the language. DEH lump, or BEX, or EMI, or something that looks close, with simple name=value assignments is going to satisfy that. Anyone who can figure out BEX should be able to edit it.

I also think that the kinds of entries can include just about anything. I do not care if 90% of the lines get ignored by every source port, as long as each gets somethings that helps.
Why force the user to download and keep a second file to get this additional information. We should be able to tolerate any information that can be encoded into a line
keyword=value
keyword=value, value, value
keyword="string"

Last edited by wesleyjohnson on 09-14-11 at 01:48

09-14-11 01:21 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

* This functionality does NOT require each source port to host their own database.
* This functionality does NOT require a complex user-editable data collection format.
* This system is NOT intended as a generic mechanism for source ports to solve their self-induced, inter-versional compatibility hystericism's.

09-14-11 03:54 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

Graf Zahl said:
Agreed.

Even ZDoom's compatibility defaults work that way.

In general people having compatibility options enabled as their default need to be aware that this is not a good idea when playing port specific mods.

@kb1:
Sorry, but I am thinking you are making a mountain out of a molehill. If you complicate a simple thing to the point where nobody can follow you it's inevitable that you won't get any support.

I have been known to do that, and I appreciate the comment, but nah, it in fact reduces complexity, provides the most basic flexibility, and is the most extensible. But, it could be viewed as more complex, once it's forced to be inside bit strings, all on one line, all in one file, blah blah. The original idea - the .ini, would make almost everyone completely happy. It supports ease of edit, optional options, custom options, custom non-boolean values, and, honestly, I think it has been made more complex unjustifiably. My system would inherently support standard options (preferrable), special options for picky ports, custom values required for ports (Act like MBF, Act like Vanilla, like Boom, like ZDoom, whatever). Knowing what is required for each map for each port can be decided case-by-case, if necessary - simple, extensible format.

However, realistically, I don't have the facilities to host and support a solution at this point in time, and I don't want to discourage *any* effort towards the goal.

Here's some final thoughts: (I'm blowing it out my ass with these figures, but probably somewhere near reality anyway.)

Let's say your port does MBF/Boom support with a few extras. First take the subset of available wads that your port's philosophy allows it to support. Maybe that gives you 100,000 maps to work with. Let's also say that you could actually test each one in a reasonable amount of time.

A totally ballpark figure might let you play 75,000 of them with totally random user settings. Most maps are ok with any possible setting. That gives you a 75% success rate anytime.

The goal (in my eyes) of the option database is to allow the user to play those 75,000 wads with any settings they desire, and for the other 25,000 wads, set the (arguably) one or two settings required to bring your success rate up to, maybe 90%. This would be a huge improvement. At that point, some port-specific stuff could bring that number higher to 95% or better, for those ports. Sure, a Batman Doom wad will need purely Vanilla settings, hands-down. But, I want to play Final Doom with sliding corpses falling off ledges, damnit, and I want to play that wad without the stupid stairs changes that break it. *And*, I want to be able to do all that, without having to review each setting each time a map opens. I want my preferences to be used always, unless it absolutely screws up the map, in which case I want the setting to be changed automatically for me.

Sure that might be asking a lot, but, gosh darnit, the database structure to make that happen is purely simple. Surely if you can step out of the programming aspects, and place yourself into the mind of a person simply wanting to play a map their own way without complications, these desires make sense. You must consider the actual usage of the finished product, as well as the ease of getting the data in the first place.

09-15-11 22:48 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

@kb1
Although I believe that I've already adequately explained why your solution is unworkable, this isn't just about whether it can be done or not.

Yes, we could implement support in the database for preference-like functionality but by doing so, you further complicate the whole solution. As an experienced programmer I'm quite sure you will be aware of the Separation of Concerns principle. By incorporating preference-like functionality into the central database, you lose the encapsulation which excluding it provides and you introduce another vector of change at a level it doesn't need to touch.

User preferences can (and I believe should) be handled by the source ports which consume the metadata stored in the central database.

09-16-11 01:15 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

Ok, let's try your solution. All I ask is to be open to the possibilities that may present themselves.

09-16-11 03:52 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

wesleyjohnson
Member

Posts: 486
Registered: 04-09

I think that kb1 and I are very close on our needs and requirements.
We want to know what busts playing a PWAD, but not have it interfere with other settings.

When I work on FreeDoom I do it by their rules, when I fix DoomLegacy I keep compatibility with Boom rules, but when I play Doom by myself it is by my rules. I am not going to let any database interfere with that, even if I want to jump around and crouch on a PWAD meant for vanilla doom. I do not need a PWAD police system, nor PWAD training wheels, (or pick your own metaphor).

Any database has no need to make decisions on optional features.
The data that the database provides should not be absolutist true/false.
I am not sure what need DaniJ's proposed solution is to solve, but it does not appear to solve our needs. It really sounds like we are supposed to change our needs to fit that proposal.
I am not sure we are going to understand that proposal until it is actually built. The biggest problem is that it sounds closed and obfuscated; not open to user interpretation.

Even an unformatted text file with this information would be of help. If a database could supply that instead of highly encoded bits, it would be much more useful (because I could read it, and only use what works for me).

Of course the source ports are central to this. Unless you are going to play Doom on your thumbs, it is which source port you are using, and the differences from what the PWAD was developed on, that causes all the problems.
It makes no difference if the information is read by the source port, by the user (and put in by hand), or by any other method.

It seems to me that this is going to fork.

We could initially enter data in DaniJ's database, and then extract it and enhance it later. But it really sounds like a pain to have to visit all these PWAD twice.

Last edited by wesleyjohnson on 09-17-11 at 20:37

09-17-11 20:09 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

I think you are missing the point somewhat. What we have here is a compound "problem". The solution I proposed is not meant to solve the whole problem.

There is a subsection of that compound problem (i.e., the persistence and delineation of PWAD compatibility options) which is common to all source ports. This is the part of the problem my solution addresses.

The remaining part of the problem can be solved separately using other solutions which build implementer domain logic and systems on top of the solution to the common problem. This part of the compound problem cannot really be addressed using a common component because each implementer has widely varying requirements (both technical and domain).

Do you not see how the proposed solution to the common sub-problem can be used to solve the whole?

09-18-11 11:55 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

DaniJ said:
Do you not see how the proposed solution to the common sub-problem can be used to solve the whole?

DaniJ, you didn't see how my solution could solve the whole problem, either (although it got really close). You talk about port-specific domains, subsystems, etc. All of that is solved by a custom set of key/value pairs that can be present, missing, ignored, etc. at will. It's extremely simple to visualize, conceive, implement, modify, and enhance. Data entry/modification is a breeze. Enahncements to the spec are inherently and naturally supported. Old entries are *always* valid, and would be valid 10 years from now. What I've proposed is simple, limitless, and straightforward. A port has a special need? Add a line with a keyname that's specific to that port. Boom/MBF options can pretty much be considered universal - if your port handles them differently, it's probably incorrect. But, if for some reason your port stubbornly must do something different with a Boom setting, that can be handled by a PortX_Stair_Method key. That's the strength of the proposed system. Why limit it's abilities? Why limit the user's abilities? Why make the data entry guy's job more difficult?

Please describe a scenario that would fail under this scheme - make me understand what the issue is, please.

09-19-11 21:42 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Gez
Why don't I have a custom title by now?!

Posts: 7046
Registered: 07-07

I don't see why we'd need to have on and off values for compatibility options; or worse, on, off and "don't care". The values for compatibility options should be "needed" and "unneeded", defaulting to unneeded. If something is unneeded, then it can stay at 0 to be governed with user setting.

The problem comes from having to turn off compat options. But then, it's just as simple to create another option. For example, "Build stairs like Doom" would be one, "Build stairs like Boom" would be another, and "Build stairs like Boom attempting but failing to build stairs like Doom" could be a third if needed.

Sure, this mean you can possibly have stupid stuff like setting all "build stairs like whatever" options to "needed", but who cares? The engine is free to prioritize them in any fashion, ignore the conflicting options or reject the entire database and abort. Creating contradictory and inconsistent data would be just as possible with any other scheme.

09-20-11 13:08 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Graf Zahl
Why don't I have a custom title by now?!

Posts: 7130
Registered: 01-03

... and that brings us back to the start, I think.

Should the database define all the options available or would it be better to have an editable set of names, each one describing a compatibility setting?

If it's just a set of names there wouldn't have to be any intelligence behind the whole thing, just some means to maintain such a list.

And ports not understanding a certain option can just ignore it - but if the need arises new settings could be added without any hassle at all.

But if the database has to have complete knowledge of what it can store I don't think this will ever work out.

A simple to use frontend can be designed for both storage schemes so better go for the more flexible one.

This does not mean, however, that I support any of the overcomplicated stuff that has been discussed and suggested by kb1 and wesleyjohnson. That will definitely kill the whole thing.

Simplicity is the key here. But simplicity should also mean that one does not have to jump through hoops to define something because there is no flexibility and extensibility here.

09-20-11 15:16 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

Graf Zahl said:
...editable set of names, each one describing a compatibility setting?

If it's just a set of names there wouldn't have to be any intelligence behind the whole thing, just some means to maintain such a list.

And ports not understanding a certain option can just ignore it - but if the need arises new settings could be added without any hassle at all.

This is exactly what I was envisioning.

Graf Zahl said:
This does not mean, however, that I support any of the overcomplicated stuff that has been discussed and suggested by kb1 and wesleyjohnson. That will definitely kill the whole thing.

I never wanted to overcomplicate it, I was simply trying to adapt to a few of the desires presented here - the desire to switch from files to bit strings, CSV, etc, which, IMHO, is over-engineering the idea.

Gez said:
I don't see why we'd need to have on and off values for compatibility options; or worse, on, off and "don't care"...

Typically, that's true. Well, with a keyname/value pair, "Don't care" is the condition you get automatically when the keyname/value is not present, so it's supported in a manner inherent to the storage scheme. I'm talking .ini file here. And, yes, 90% of the time, you're turning on compatibility options, or, more accurately, turning off modifications to original behavior. But, it's been a few years since source ports have been around, hence the need for both "on" and "off", and, if needed, "1", "2", "3", or whatever. It's up to the port programmers to decide what settings their port needs, and what values are to be supported/required. That should satisfy wesleyjohnson's needs, which are not unreasonable. But, we want to strive for having as few settings as possible = we don't want too many port-specific settings if we can avoid it. Once again, MBF/Boom settings will cover most of it, and, yes, most of those are "On" and "Off" type settings.

That being said, yes, I see merit in the bit string approach in general as an efficient, easy-to-store mechanism, but it doesn't really fit the application. You must choose the format that most naturally fits the situation. The need to include only the required settings while avoiding others led me immediately to the .ini approach. I'm not against another approach, but you'd be hard-pressed to find something more flexible and easy to understand.

If it's simplistic approach covers 95% of the needs, we've made a huge improvement.

09-20-11 19:25 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Vermil
Senior Member

Posts: 1224
Registered: 03-04

I am assuming each wad in the database will have a list of compat options that are required to be on to complete it and a list of compat options that must be off to complete it.

I think a compromise to the fact that some ports want to present the user with "always don't accept/ always accept" or "always ask" like options, would be a simple "priority" like field on each entry in the database itself.

I think such would provide a port that wants to ask users about setting some/all compat options for a given wad with all the information needed, with the minimum of alteration to the database.

On a scale of 1-10, a 10 will tell the user that the option has to be on or off, to be able to complete the wad, while a 1 could indicate to the user that the compat option can allow the user to break the map, but won�t automatically render it broken from the beginning.

I imagine this will lead to let's say, something like:

HR's map07 666 tag getting a 10 indicating that the map will be broken if this compat option isn�t set the right way.

Icuras' Map26 ghost monsters would get an 5, indicating that disabling ghost monsters won�t break the map, but will alter the game play.

While a nojump option would get a 1. A warning to the user that jumping can break the map.

In practice, I imagine that the numbers 2-9 will probably near entirely be used for things like the Icuras example.

Last edited by Vermil on 09-20-11 at 19:43

09-20-11 19:36 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

Vermil said:
...I think a compromise to the fact that some ports want to present the user with "always don't accept/ always accept" or "always ask" like options, would be a simple "priority" like field on each entry in the database itself...

Well, it does complicate the implementation a bit, not necessarily the storage, but that there would be some user interface requirements to do it justice. Then again, if a port didn't want to provide such an interface, it could automatically set the 10s and ignore the rest.
Personally I would think 1 thru 10 was a bit much - I'd like the slots to be more rigidly defined - I'm all for automation. I would be inclined to lean towards "set and forget" as much as possible, but the "ghost monsters" setting is a perfect example of a "5".

Maybe a compromise:
* Compatibility Setting:
* Always use vanilla settings
* Apply Required Settings Only (10s)
* Apply Required and Recommended Settings (10s and 5s)
* Leave Settings Alone

Based on the above, I might drop the "1 thru 10" in favor of the following modifier:
+RECOMMENDED

I don't think any more granularity can be expected to be honored equally by anyone entering the data. Basically, any option that's present would be considered "required", unless "+RECOMMENDED" was also present.

Even better, the value could be in parenthesis or something to indicate that it's simply a recommendation:

code:
archvile_ghosts = (true)

code:
(archvile_ghosts = true)

code:
?archvile_ghosts = true

Of course that's not the only way to do it, just an idea.
Does this sound reasonable?

09-20-11 23:24 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

wesleyjohnson
Member

Posts: 486
Registered: 04-09

It seems to me that one way of doing it is to test a PWAD on a port and record all the compatibility settings. This will be independent for each port. It hardly requires any cooperation as most other ports would not be able to handle that ports compatibility settings anyway.
Perhaps there could be a large group of compatibility settings that are common to many ports, but that again requires some values that are more informative than 1/0, or T/F, such as I proposed earlier.

This also begs for a port specific setting capability like that I proposed earlier, such as.
(ZDoom) compat54=Y
Without general situation descriptive values, there is little that can be done other than record the 1/0 settings needed for each port, and maybe a REQUIRED/RECOMMENDED attribute attached.
(ZDoom) comp3=1 RECOMMENDED

The other way is to record the assumptions that the PWAD was developed under. In such a system most PWAD will have one entry, such as
"Vanilla"
or "Port=Vanilla".
For most PWAD, nothing more is needed. This would save so much effort that it would be worth it for this reason alone.

Every port should be able to understand the nine or so major PWAD types as values (with port specific names also accepted (which a port only has to recognize their own name and a few others it can handle)).
enum("Buggy", "Vanilla", "Doom2", "Plutonia", "TNT", "Heretic", "Hexen", "MBF", "Boom", "ZDoom", "Eternity", "Legacy",
"Edge", etc...
REQUIRED, MAJOR, RECOMMENDED, SUGGESTED, MINOR,
T, F ).

Unexpected problems may occur for a few PWAD, like
"Stairs= Buggy Vanilla".
But in this system, the entries will general to all ports.
The modifier "Buggy" meaning that a known bug fix must be disabled.

It is also possible to combine the two methods.
I do not find this complicated at all. It is actually easier for the person writing the description.

I would prefer some easily readable encoding like above, rather than 1..10 with special meanings for some numbers.
I disagree that all settings will be disabling bug fixes. There are
other kinds of differences between ports and even between id games code. Some PWAD require that some enhancements be on, or that some capabilities be present.

Many PWAD require DEH, or BEX. That info should also be recorded.
DEH=REQUIRED # not effective without DEH patches
DEH=RECOMMENDED # DEH changes some non-critical text
BEX=REQUIRED
SKINS=RECOMMENDED
MULTIPLAYER=SUGGESTED
SOLO=Y

Last edited by wesleyjohnson on 09-23-11 at 05:04

09-21-11 00:34 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

DaniJ
Senior Member

Posts: 1740
Registered: 08-03

I'm going to respectfully bow out of this discussion because to be frank, we are getting absolutely nowhere. All I'm seeing are a list of "possible" shortcomings with no substantiation behind their reasoning.

As far as I'm concerned, including preference-like functionality into the central database is not just bad design, it is fundamentally unworkable. I never expected my back-of-napkin solution to be accepted as-is, however I have yet to see any suggestion for improvement on its design that I even consider beneficial.

Those bringing up the multiple option state issue ask yourself one question - why do we need separate values for BOOM, Legacy, MBF, whatever functionality, or'ed in to every single property value when it is logically impossible that this will vary on a per-option basis. If we absolutely must include support for every possible configuration then it makes far more sense to include a textual, port-name-identifier in the hash that identifies the logic which the hash is exempting to and which an implementer should attempt to mimic.

The problem we are trying to solve is a significant one, which affects player usability on a daily basis. This is something I would really like to see a solution to. However if the discussion to date is any indication, we are never likely to achieve a consensus and as such, it is a waste of all our time even participating in the debate.

Fact is we are nearly three weeks into this discussion and we are no further forward than we were on day zero.

Last edited by DaniJ on 09-21-11 at 02:10

09-21-11 02:01 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

wesleyjohnson said:
...Perhaps there could be a large group of compatibility settings that are common to many ports, but that again requires some values that are more informative than 1/0, or T/F, such as I proposed earlier.

This also begs for a port specific setting capability like that I proposed earlier...

I was hoping for port-specific stuff to be the exception, at least at first. My initial thoughts were to achieve the most bang-for-the-buck, by identifying the common (Boom/MBF) boolean options, which would cover 90% of the issues we face when loading wads. Of course there are some *very* port-specific wads that simply won't ever run properly without the intended port. And, of course, there's some grey area here (the other 10%) of issues.
I wouldn't intentionally cripple the spec by limiting what can be done, but I think that last 10% would come naturally as needs arise. In fact, I would suggest that, one day, this could be extended to make slight in-memory wad repairs, like setting the flags on the yellow key in that broken Final Doom map. There are lots of possibilities.

wesleyjohnson said:
Every port should be able to understand the nine or so major PWAD types as values (with port specific names also accepted (which a port only has to recognize their own name and a few others it can handle)).

Again, I think this might be overkill, at least at first. I know there are port differences, but, are those intentional? Especially for Boom/MBF options?

wesleyjohnson said:
Many PWAD require DEH, or BEX. That info should also be recorded.

Agreed, if the DEH/BEX is a separate file.

DaniJ said:
I'm going to respectfully bow out of this discussion because to be frank, we are getting absolutely nowhere.

Maybe we all need some time to kick it around a little bit. This thread was just for ideas anyway at this point. Personally I haven't discounted anyone's ideas as of yet. I provide counterpoints to provoke discussion, and I do appreciate everyone's time and effort. This is obviously an important and necessary function, and I fully intend to see it through.

My biggest drawback currently is time. I have downloaded many, many wads, and have written some very complex programs to automate their organization. My unpolished but functional tools extract the zips into separate folders, remove intermediate blank folders, and perform byte-by-byte comparison/duplicate removal. Untidy files are removed (thumbs.db, etc.). I've also built a small, fast database engine for the express purpose of being used in a custom Doom launcher that I've been also building.

Does anyone know of a way to extract the Doomworld archive database? That would be massively helpful.

So now I basically have 1 folder-per-wad, containing the unzipped wad file, and typically a text file. Thousands and thousands of them. So, for me, organizing the wad files is extremely important - I will eventually be creating my own database of exactly what we've been discussing, as the next logical step. However, I'm only one man, and this will take a long time to even get ready to begin, not to mention playing all those wads!

DaniJ said:
...if the discussion to date is any indication, we are never likely to achieve a consensus and as such, it is a waste of all our time even participating in the debate.

Fact is we are nearly three weeks into this discussion and we are no further forward than we were on day zero.

The work I will eventually be doing will be able to be included in whatever final outcome we decide, at least in part. We *are* further forward - we now know what's important to each of us that has contributed. No, we haven't "written any code" yet, or entered any data, but, we'll get there if we keep our eyes open and work toward a solution. It will take some time for me to have anything to show - maybe we can resume at that point. Once again, I appreciate the time and energy.

09-21-11 17:30 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

wesleyjohnson
Member

Posts: 486
Registered: 04-09

This is just like most projects I have worked upon. Determining the user requirements is absolutely necessary for a project to be successful according to those users.

I work on DoomLegacy, and DoomLegacy can only use two bits of all the compatibility settings from Boom/PrBoom. I have visited this code recently while trying to handle Boom demos, (which FreeDoom likes to use). Encoding things along those lines will only be frustrating for DoomLegacy and DoomLegacy players.

Need flag for DEH/BEX lumps present. They are hidden and the player would not know it is there. The player must use a port that has DEH/BEX capability. DoomLegacy only got BEX reading capability this month. Cannot assume that all ports have the capability.

Need string for DEH/BEX file needed. This string should record the file name. Then the user could not forget it so easily.

I think we need to work up sample info files for the worst PWAD we know of. There is no escaping this. The worst PWAD will force the issues. Might as well know what they are. About 30 samples would be minimum.

I hoped the port line would have got a better reception. It simplifies everything. It does need that large enum that I proposed.
Once having that, the other flag lines can use the same values to simplify the whole per port issue, by making the value independent of any port. There are many PWAD that are so port specific that it is inescapable that there will be some line to record that. Might as well use it for the first line and make all following lines be more specific details or exceptions. It would reduce the amount to information recorded for most PWAD to a trivial one line.

PORT=BOOM

Otherwise, we could end up with the per PWAD solution. Every PWAD with a problem generates a special keyword that flags its particular problem.
At least it would be port independent, but will create a new keyword for each PWAD bug.
TNT_MAP13_BUG = Y

And lastly, we could end up with a per port solution. Every port has all its compatibility settings recorded. Much work here, as every port has to be tested on the PWAD. This solution cannot deal with ports that fix their code to deal with a PWAD problem. It has no way to identify the actual problem. There will be old versions of the port, new versions of the port, old versions of the info file, and updated info file present simultaneously. And who wants to update 20 to 300 PWAD info files each time a port fixes some bug.

Maybe, it will some combination of the above possibilities. Really need to deal with this and not try to slip by, trying to ignore and look away from the issue. It may be that some PWAD will force the issue later, but at least it should be decided what kind of format we each want. Otherwise, I expect that the every batch of sample info files will be totally along different lines, and mutually incompatible.

By the way, I argue all sides of any issue. People mistakingly think what I argue is what I want (not true). Cannot properly decide any formatting issue without knowing the PRO and CON of each considered format line. This drives insane any co-worker who just wants to jump to the first solution they considered.

Syntax clues like () to indicate RECOMMENDED will likely get broken soon. There are just too many other possible special values that will be wanted. Better to just append modifiers, like RECOMMENDED. Those that are not understood can be ignored. It should be modifiable to adapt to various special cases, (just add new special attribute names to an ENUM of the attributes, and not every port has to understand them all).
special_flag = 1, RECOMMENDED, PLAYER2, FLAGS4

I know that many think this is overly complicated. Unfortunately the issue is complicated by a dozen independent uncoordinated port projects, that constantly update their code and change their behavior.
And there are 100's of PWAD, made over 14 years, to dozens of platform assumptions, and some dependent upon platform bugs that have now been fixed. How can you actually think that some simple compatibility flag recording is going to get control of this, or even record it. That would only work if it is restricted to Boom compatibility flag ports only (DoomLegacy is not one of those).

Last edited by wesleyjohnson on 09-23-11 at 05:34

09-23-11 05:01 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

Graf Zahl
Why don't I have a custom title by now?!

Posts: 7130
Registered: 01-03

If you really think you need such a monster there's something wrong.
Trying to address every single difference in behavior is utterly counterproductive because it won't get anywhere due to lack of support.

The goal is to make a WAD just work by enabling some flags. ZDoom's approach to compatibility settings is to only set those which are absolutely necessary to make a map work right. Anything more would be overkill and in fact would probably cause more user irritation than anything else.

For example, trying to play a ZDoom map with compatibility settings set to 'Doom.exe' is guaranteed not to work in many cases but there's no safeguard for it. User's stupidity should not be part of the equation.

09-23-11 07:11 #

Profile || Blog || PM || Email || Search || Add Buddy

IP || Edit/Delete || Quote

kb1
Junior Member

Posts: 167
Registered: 11-06

wesleyjohnson said:
I hoped the port line would have got a better reception. It simplifies everything. It does need that large enum that I proposed.
Once having that, the other flag lines can use the same values to simplify the whole per port issue, by making the value independent of any port. There are many PWAD that are so port specific that it is inescapable that there will be some line to record that. Might as well use it for the first line and make all following lines be more specific details or exceptions. It would reduce the amount to information recorded for most PWAD to a trivial one line.

The "port line" is problematic too - ports are not set in stone - they evolve capabilities. It's up to the the port authors to respect their own settings and keep them operating consistently throughout the lifetime of their port, if they are to gain an entry into the compatibility database for that setting. And, no, there's nothing wrong with having a field that identifies which port(s) the map was tested with, but I'm uncomfortable with having that setting, once again, changing multiple options, and, really having any effect other than to inform the player that the map being played has not been tested in this port.

DaniJ had it right when he talked about starting with the Boom options - they are either ignored by a non-compliant port, or they are treated exactly the same in every port. They can be expected to work typically the same on all ports. That's about all we can expect. And, if your port has Custom Port-Specific XYZ Setting, and, I as a developer want to incorporate XYZ Setting in my port, I'd be unwise to make my implementation work differently, in which case I'd be unwise to call it "XYZ Setting". It's possible that, having this option database will encourage developers to make their options a bit more compatible.

It's unrealistic to expect 100% accuracy out of this thing, for many reasons, and I think that throws off some of the commenters here, especially the developers. We like things to be well defined. But, what I propose is not an exact science - the database will be incomplete for a while and will need to be fine tuned over time. I am looking for a "loose coupling" to the most common options that screw up my playing experience. Not a one-stop-shop for automatically getting every standard and custom setting exactly right. Just improving the odds. You have to remember that, building up this dataset is a monumental task in itself and it will be bug-ridden at first. You absolutely don't want to create a system that encourages some ports to work and forgets about others, for example. The idea of specifying the port name as part of the spec fills me with fear, I must admit.

Graf Zahl said:
If you really think you need such a monster there's something wrong.
Trying to address every single difference in behavior is utterly counterproductive because it won't get anywhere due to lack of support.

The goal is to make a WAD just work by enabling some flags. ZDoom's approach to compatibility settings is to only set those which are absolutely necessary to make a map work right. Anything more would be overkill and in fact would probably cause more user irritation than anything else.

Agreed fully. This is what I would like to see out of this thing. ZDoom specifically prides itself on "doing it's own thing". That's fine, and it's why we have source ports. But, it also has a mini compatibility db built-in that adjusts options to get certain problem-child maps to operate sufficiently well. Is it an exact replica of the gameplay one might experience playing that same map on doom.exe? Absolutely not. But, the maps *are* playable, completable, etc. That's the best we can hope for - it's the reason I started this thread.
The analogy is what you would do manually: "Oh shit, to play this map, I need to set this option, otherwise X doesn't work right." The options we choose to include in the spec must be agreed upon, to be options that we all areee to support in a similar manner. And, if a port adds map-breaking options, that can be considered too, port-per-port, strictly as a rare exception, but if it becomes commonplace to need per-port options, we are all doing a disservice to the community by making optional behavior that acts radically different port-to-port. Sure, a port may have maps made with features specifically for that port - that's fine. That's where the "tested-in-port=" stuff could be useful. But, what if I add this special feature to *my* port? Remember, whatever option-keys we use should survive the test of time - Doom ports evolve everyday.

Maybe eventually I'll take this project and run with it, and create some type of web presence to allow the entering of the data. But, it's going to take me some considerable time for my priorities to allow it.

Last edited by kb1 on 09-23-11 at 17:49

09-23-11 17:34 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

wesleyjohnson
Member

Posts: 486
Registered: 04-09

I guess there are no two people with the same plans now.

The PORT line (maybe call it DEVPORT or maybe BASEPORT instead) was to record the port the PWAD was designed upon (is dependent upon, is based upon, was designed for), which would never change. It was not the port to play it on, or what it was tested with.

Could get the DEVPORT from that formatted text that accompanies many wads. Search for ZDOOM, and if found then assume it is a ZDOOM wad (and hope they didn't put in the comments "You may have problems with ZDOOM because of ..."). Most of the PWAD will be Doom or Doom2.

It matters little the reason, but sooner or later some port is going to need a setting that no other port should use (Rare thing or not).
Then someone will want to rename a setting or make a new switch name so it can be ignored by others on that basis.

We cannot get three people to agree on anything, how are we going to get them to agree on exactly what a setting name is going to do.
It would be far better to let the source ports deal with their own difference issues on their own and only describe what issue the PWAD has. How each source port deals with that can evolve over time, but the recorded switches in the info database should never have to change to deal with that.

Just recording Boom compatibility settings is not going to get far.
It does little to help me from deal ZDoom PWAD, or MAP31 yellow key bugs, or stairs that depend upon Vanilla buggy code.

Developing port code that can detect a ZDoom PWAD from lumps present, or code to detect stairs that depend upon buggy Vanilla stairbuilder would serve me better.

The biggest shame would to go through the trouble of downloading these PWAD and testing them, and then not recording everything that you found. The format is immaterial, if anybody uses half the info in the next 3 years is immaterial, it is to avoid having to go back and test 300 PWAD a second time.

Last edited by wesleyjohnson on 09-25-11 at 23:23

09-25-11 22:40 #

Profile || Blog || PM || Search || Add Buddy

IP || Edit/Delete || Quote

All times are GMT. The time now is 04:52.

Pages (6): « 1 2 3 [4] 5 6 »

Doomworld Forums > Classic Doom > Source Ports > Wad compatibility option database - ideas

Show Printable Version |

Email this Page |

Subscribe to this Thread

Forum Rules:

You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

HTML code is OFF
vB code is ON
Smilies are OFF
[IMG] code is ON

< Contact Us - Doomworld >