Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Sign in to follow this  
wesleyjohnson

DEH implementation consensus

Recommended Posts

In another attempt to have cross-platform compatibility, I wish to discuss DEH code modifications (Dehacked lumps), with perhaps no longer having original source strings in DEH lumps.

I think that it is about time that some consensus opinion was forming.

1)There is an issue with copyright. FreeDoom wants to remove possible copyrighted strings from their wad.
2)There also is the issue of the IWAD (FreeDoom) modifying a string using a DEH lump and then have a PWAD (pick one) also try to modify the string using DEH.

3)There is little problem in keeping the const string around and always comparing to it rather than the current replacement string.

4)Can even have DEH code compare to the original const string, and the latest replacement string, and replace the string upon either detection.
Would comparing both really be necessary ??
Would any other ports implement that way ??

5)Would there be any consensus on using a hash value of the original string, such as the 32bit hex sum of all its characters, (i.e. "0x0003A58F" ) as an alternative to the DEH first string, an upon detection of a recognized numeric value, replace the matching string.
(I think this would only affect old ports playing new wads using such a new system)
Detect the new system by detecting the lead "0x" in the string.

Share this post


Link to post

4)Can even have DEH code compare to the original const string, and the latest replacement string, and replace the string upon either detection.
Would comparing both really be necessary ??
Would any other ports implement that way ??


Are there any deh files in existence that use anything other than the original exe strings as their baselines to replace?

5)Would there be any consensus on using a hash value of the original string, such as the 32bit hex sum of all its characters, (i.e. "0x0003A58F" ) as an alternative to the DEH first string, an upon detection of a recognized numeric value, replace the matching string.
(I think this would only affect old ports playing new wads using such a new system)
Detect the new system by detecting the lead "0x" in the string.


I would think that using the bex string replacement system is fine; are there seriously any copyright concerns from using GOTSTIM, DETAILLO, HUSTR_E1M1, CC_HELL, etc in a bex file?

Share this post


Link to post
wesleyjohnson said:

2)There also is the issue of the IWAD (FreeDoom) modifying a string using a DEH lump and then have a PWAD (pick one) also try to modify the string using DEH.

This ought to work anyway, and when it doesn't work, it ought to be considered a bug in the source port. DOS dehacked can handle multiple patches applied sequentially in this way.

Essentially there are two possible behaviors: 1) string replacement matches against string in the original game (how DOS dehacked and Chocolate Doom work) and 2) replacement matches against "current value" of the string (how Boom-based ports tend to work).

I think that (1) is a much more useful behavior than (2) because it allows you to load multiple patches in sequence without dependencies between them. Furthermore, I don't see why (2) would ever even be desirable.

I've already written a patch to PrBoom+ that changes the dehacked code to behavior (1). It's not a big change, I suspect it wouldn't be a lot of work to make a similar change in other ports either.

3)There is little problem in keeping the const string around and always comparing to it rather than the current replacement string.

Correct. It's part of the code that is loaded in memory already.

4)Can even have DEH code compare to the original const string, and the latest replacement string, and replace the string upon either detection.
Would comparing both really be necessary ??
Would any other ports implement that way ??

I don't think that's useful or desirable, and it would potentially lead to unpredictable behavior - for example: first patch changes a string to be the same value as another, different, internal string, second patch is loaded, and replaces the wrong string.

We ought to conceptually think of a patch as always being against the original Doom executable. When you do that, you naturally get the ability to layer patches as you wish.

5)Would there be any consensus on using a hash value of the original string, such as the 32bit hex sum of all its characters, (i.e. "0x0003A58F" ) as an alternative to the DEH first string, an upon detection of a recognized numeric value, replace the matching string.

You mean as part of the file format itself? No, I don't think that's a good idea. We already have the BEX extensions which provide a symbolic way of referencing strings to replace, which is a much better mechanism. It would be much better to simply add that mechanism to ports that don't already support it.

One of the nice things about the dehacked format is that it's editable simply with a text editor. I'd really hate to see a hack like this added which would damage that.


Having now said all that, I do want to highlight what I do think is an equally important thing that needs to be brought up: the order of loading patches. There are different ways that a dehacked patch might be loaded:

  • From the command line, with -deh (or -bex)
  • From a DEHACKED lump within an IWAD
  • From a DEHACKED lump within a PWAD
Getting these in the right order is important and most (if not all?) source ports don't do it in a sensible order at the moment. What I think the order ought to be is:
  • DEHACKED lump from IWAD is loaded
  • Command line patches (-deh parameter) are loaded
  • DEHACKED lump from PWAD is loaded
It's important to get them in the right order because otherwise the different use cases aren't all covered. For example:
  • User plays Batman Doom with Freedoom. The -deh batman.deh parameter needs to be applied after the Freedoom patch, otherwise the Batman Doom strings get overwritten with Freedoom ones...
  • User plays a PWAD with embedded DEHACKED lump, using Freedoom IWAD. Currently some source ports (eg. PrBoom+) only load a single DEHACKED lump, so all the Freedoom strings get lost, even though the PWAD maybe just had a small patch to change level names...
I think to get sensible behavior, the most important thing is to load the IWAD's DEHACKED lump (if present) before anything else. The order of the remaining two is less important. Conceptually, providing the -deh parameter is like changing the executable's default behavior, so I think it should go first. You might however argue that the PWAD DEHACKED lumps should be loaded first, so that the behavior in PWADs can be overridden.

Share this post


Link to post

What if you're using a port that doesn't handle DEH/BEX lumps or patches? Like say, linuxdoom itself, SDL Doom (linuxdoom in SDL), maybe some others... Javascript Doom? :D

Share this post


Link to post

Thanks Fraggle, I was about to propose another alternative using enum values, but it sounds like BEX already has string replacement without using a copy of the original string, which was the point.

(Mine would have had it all be text (no binary editing), such as
STR 0x00043652 = new string
)

I have yet to find a BEX source doc. It must go by another name.
Instead, I have 2 or 3 source code examples to steal BEX from, including prboom.

I agree with a consensus that all ports should test against only the original string, and not any replacement values.

So much of the DOS source overwrote the const strings, that I think it became a standard practice. Got that behavior in DOS based level editors, prboom, and old DoomLegacy code.

I think I agree on the order of DEH application.
1) The command line DEH entries must be considered mods to the IWAD, equal to DEH lumps in PWAD, so they must be applied to the IWAD after any IWAD internal DEH.
2) Any PWAD DEH lumps must mod the IWAD.
3) The command line DEH probably should come last, as a deliberate DEH lump should have highest precedence.
4) For some ports it should be considered in order of command line items. Apply DEH in the order of appearance on the command line, same as PWAD are applied in the order of appearance, with the last one winning.


[ If you are using a port without DEH/BEX, then for FreeDoom you will not see their changes and will get the originals built into the program. I think as far as FreeDoom should be concerned, it is then the source port's copyright problem.
It also means some PWAD will not work very well (like twilight.wad)]

If the string names that BEX uses would be a problem, then there are hundreds more names in the source that would also be a problem.
Having the source released to GPL has to allow using some names.
It was using the actual strings, which could conceptually be considered part of the game, that was being considered defensively.

Share this post


Link to post
hex11 said:

What if you're using a port that doesn't handle DEH/BEX lumps or patches

Then too bad for you, you'll miss out on some features.


There's a point at which people ought not to worry about non-compliant ports that much. It's like worrying about people trying to play a Doom II map with the vanilla heretic.exe.

Share this post


Link to post
wesleyjohnson said:

1)There is an issue with copyright. FreeDoom wants to remove possible copyrighted strings from their wad.

Why? If the strings are in doom.exe then they are also in the GPLed source.

Share this post


Link to post

That's a thorny issue that has already been debated. Since they are in the GPL code, they are free to use in source ports. However; they're still part of the Doom games as a whole and re-using them in other games such as Freedoom, even if indirectly, constitutes copyright infringement. Hence the whole point of replacing them.

Share this post


Link to post

Freedoom doesn't work in all ports anyway. First off some maps (like E1M1) are Boom format. Second, there's some problems with animated textures when playing with SDL Doom, and probably anything also directly derived from linuxdoom. I doubt anyone has really tested it on many ports (by warping to E1M2 or loading plain vanilla PWAD).

Share this post


Link to post
Gez said:

That's a thorny issue that has already been debated. Since they are in the GPL code, they are free to use in source ports. However; they're still part of the Doom games as a whole and re-using them in other games such as Freedoom, even if indirectly, constitutes copyright infringement. Hence the whole point of replacing them.

Someone has misunderstood the GPL then. Since they are in the GPL code, id (who still own the copyright on the strings) have authorized them to be used freely in any GPL-covered work. There is no "for source ports only" condition attached.

Or is it FreeDoom being BSD (rather than GPL) licensed that is the problem?

Share this post


Link to post

The problem isn't that they're in the program somewhere. The problem is that they're shown while playing the game. A game that isn't Doom.
http://fraggle.despayre.org/carmack_reply.txt

You've got to keep in mind that Freedoom is an IWAD and nothing more. It's not the exe. It's not a source port. It doesn't use code. The GPL is entirely irrelevant here. It's a data file that doesn't do anything on its own; it requires a source port to work. The aim was that you could package it with a source port and get a free game that can be included in stuff like Linux distros and such. There's also the secondary goal of making it compatible with Doom addons.

The big problem is that the latter goal makes it necessarily a plagiarism of Doom. Not using the strings from the original game helps make it less plagiarized; so it's a good step to make.

I'll note by the way that the compatibility of Freedoom with Doom mods is already not legally sound. A lot of mods use modified resources from the Doom IWADs and by the terms of the licensing agreement this is only allowed if the mod is meant to be used with that IWAD. By allowing to circumvent this requirement, Freedoom makes these modders retroactively guilty of breaking the terms of the license. Fortunately, nobody cares, as evidenced by all the Hexen/Quake/Duke 3D/whatever resources used in a lot of Doom mods; but when you're a stickler for legal legitimacy as the Freedoom project aims to be, it is an issue.

Share this post


Link to post

Maybe this is better illustrated with a simpler example.

This program isn't Doom either. It isn't even a game. It's copyrighted by id Software (being a derivative work, since I copied a piece of text straight from the linuxdoom source.)
But it's still completely legal to distribute it under the GPL:

#include <stdio.h>

int main()
{
printf("YOU HAVE ENTERED DEEPLY INTO THE INFESTED STARPORT.");
return 0;
}


And if John Carmack doesn't like me using that string in a non-Doom program? Too bad, he authorized it back in 1999 and whether he likes it or not he cannot revoke that permission. Similarly I can put that same string in any other GPL-covered work, whether it's a dehacked clone or a WAD or anything else.

Share this post


Link to post
finnw said:

Why? If the strings are in doom.exe then they are also in the GPLed source.

The issue isn't copyright, it's trademarks. The distinction is important, and the fact that the Doom source is GPLed is essentially irrelevant. For example, "You got the BFG-9000!" contains a reference to the BFG-9000, an Id trademark. The E2 end text references the "Cyberdemon lord" that ruled Deimos, and Cyberdemon is an Id trademark.

The point of having a dehacked patch is to deliberately distance Freedoom from Doom. It is a (minor) tweak that helps to make clear that it is a separate game, and Freedoom is a game based on the Doom engine, and not seeking to infringe on trademarks. My concern was that, to a casual observer playing Freedoom, it contains "BFG-9000s" and "Cyberdemons", etc. We were explicitly told not to do that, that what we could do was to make a new game based on the Doom source.

Using a dehacked patch there are two options available to implement this: one is to use text replacements, the other is to use a BEX [strings] section. To some extent I don't think it really matters which is used; however, the BEX option is preferable because it's cleaner - bearing in mind that the whole point is to avoid Id trademarks, using this method, Freedoom avoids any references to them, even internally.

If you want a similar interesting example of the relationship between trademarks and open source, Mozilla Firefox is one. Firefox is open source, you can download the source code, modify it and distribute your own modified versions, just like any other open source program. However, you can't call your modified version "Firefox", because that's a trademark. The fact that it's open source and GPLed is irrelevant - the GPL is a copyright license, and this is a trademark issue.

Share this post


Link to post

fraggle said:
For example, "You got the BFG-9000!" contains a reference to the BFG-9000, an Id trademark. The E2 end text references the "Cyberdemon lord" that ruled Deimos, and Cyberdemon is an Id trademark.

Are they? Is there any notice indicating that anywhere? They don't seem to be registered in the US, unlike DOOM and DOOM II, for example. (I just searched the TESS system.) John Carmack once mentioned not to use their characters but that has to do with their identifiable visuals and copyright. A trademark name has to be protected by a registration or at least claimed explicitly.

In any case, using a DeHackEd patch is a great idea because it allows some freedom in giving the game its own background and feel and helps any contributors not feel inclined to mimic the original stuff in potentially risky ways (avoiding the "id characters" problem).

Share this post


Link to post

They are not registered, but they are claimed explicitly. "DOOM likenesses are trademarks of id Software, inc.,(C)1993."

Share this post


Link to post

I don't think it works that way. I think you have to explicitely register every trademark you claim to own, and when its term is up you have to keep it registered, or else you no longer own it. I seriously doubt that id has trademarks for everything inside the game.

For sure the name "DOOM" and its logo/likeness are trademarks of id, but every single graphic or text string insde the game isn't. Some of them are copyrighted though (pretty much everything in the IWAD).

Share this post


Link to post

Copyright (©) is automatic. Just the act of creating something means it's copyrighted to you. So it's not just "some of them", it's "all of them unless otherwise specified".

Trademark is a completely different thing. First, trademarks do not have to be registered. There is a reason why there are different symbols for plain trademarks (™) and registered trademarks (®).

Secondly, trademarks are perpetual. Contrarily to copyright, which theoretically expires after a while (but in practice doesn't for anything newer than Mickey Mouse), trademarks stay in effect as long as the person or company owning them keeps using or defending them. It is possible for something to be both into the public domain on the copyright front and yet still an active trademark.

Trademarks, on the other hand, expire if they are not actively defended. Which is why companies insist that you use "LEGO® blocks" instead of "legos", or "enhanced with Adobe® Photoshop®" instead of "photoshoped". Even though most people don't care, their repeated pleas serve to show an active defense of their trademark and prevent competitors from reusing the same names in their own products. For example, "aspirin" used to be a trademark of Bayer AG, whereas it is now a common noun and Bayer can't sue you if you start selling Hex11 Aspirin.

Carmack's email about not using cyberdemons and BFGs counts as trademark protection. The continued sale and development of the Doom series games counts as trademark use.

Share this post


Link to post
hex11 said:

I don't think it works that way.

You're wrong. ™ is used for an unregistered trademark, ® is used for a registered trademark.

hex11 said:

For sure the name "DOOM" and its logo/likeness are trademarks of id, but every single graphic or text string insde the game isn't.

I never suggested that was the case. Indeed, it wouldn't be possible to do so, because that's not what trademarks are for. To use a broad definition, trademarks are used to protect distinctive designs/concepts, not individual works (which is what copyright is for). The Doom logo is a trademark, and so is the Doom name. The Doom monsters - at least the distinctive ones - would count as trademarks. That includes the Cacodemon, Cyberdemon, Mancubus, etc. So would the BFG-9000, and possibly some of the other weapons. These are all distinctive and memorable elements of Doom that make the game what it is.

What can't be trademarked are generic things - you can't trademark a game that has a shotgun for example, or the concept of a zombie. We can use those elements without any problem. Freedoom's replacement for the Mancubus looks reminiscent of Doom's Mancubus, and that's okay - but to have the game refer to it as the "Mancubus" isn't. That's why I put together the dehacked patch for the Freedoom IWAD - it's supposed to be a distinct, separate and different game, and having Id trademarks appear while playing it (in the form of the original text strings) might be considered reason enough to issue a legal threat against the project.

Basically, when you think of trademarks, think of Coca-Cola (the name and the logo), or Mickey Mouse. Nobody's stopping you from making your own cola, but you can't call it "Coca-Cola" (or even "Moca-Cola"). Similarly, nobody's stopping you from creating a cartoon with a talking mouse, but you can't call it Mickey Mouse and it mustn't look identical to it.

Share this post


Link to post

OK, but is there any consensus on how to cope with the IWADs that want to change strings (DEH, and BEX).

And Trademarks are weaker than copyright, they only require that no consumer be confused by the product being sold, see Rollerblades vrs inline-skates. A new product that is clearly labeled can use terms that an existing company uses, if it can show that the consumer is not confused by the usage. There have been many court battles over Trademark where the product was not even close, and the court is not predisposed to enforce the Trademark in such cases. Another way is to modify the term sufficiently to uniquely identify the company/product.
The offended company must show that they are being hurt in more than a minor way. There are many products where the term is modified by only one letter.

RC Cola (Royal Crown Cola)

Apple stores in China are a clear violation of trademark as they aim to get customers by deception.

Mini-Apple (the Minneapolis Apple Users Group) had discussions with Apple over the name and kept it, as it was not selling computers or computer equipment, and was sufficiently distinct. Apple could not claim trademark over every use of the term Apple. If you ask me for more details, please realize it was a long long time ago.

To be a trademark, I believe it has to be visible to the customer before the sale, and be a basis for selecting the product (thus subject to trade law). I do not think that terms buried in source code that the consumer never saw, could ever be considered trademark on that basis alone.

Terms like "Mancubus", which do not appear in advertising, nor on the product cover, and are not visible before the sale to a potential buyer, are in a gray area as a Trademark, even though they would be Copyright (but then again GPL). It would be very difficult to argue that sales were lost because the offending product used such a term, when it is only visible long after purchase and use.

Share this post


Link to post

Yes, but as I've pointed out before, the Freedoom project has no realistic way to defend itself from any legal threat. A claim of trademark infringement could sink the project. The stakes are higher than they might otherwise be in other situations; my concern is that we should seek to avoid there being anything for Id (or Zenimax, who seem to be even more sensitive to this sort of thing) to complain about in the first place.

wesleyjohnson said:

OK, but is there any consensus on how to cope with the IWADs that want to change strings (DEH, and BEX).

I think I've presented a good argument for how it ought to be done in my previous post, and it seems like you don't have any fundamental objections to my reasoning. If nobody else has any objections I suggest we start developing patches for common source ports - or petitioning source port authors to adopt that behavior?

Share this post


Link to post
wesleyjohnson said:

I have yet to find a BEX source doc. It must go by another name.

BOOMDEH.TXT? It's in the Boom source distribution. Nearest thing to bex documentation I can think of off the top of my head.

wesleyjohnson said:

I was hoping for more consensus than just the two of us.

I'm not committing to anything until I figure out how much work it's going to be :-P

Share this post


Link to post

Gez said:
They are not registered, but they are claimed explicitly. "DOOM likenesses are trademarks of id Software, inc.,(C)1993."

There's nothing explicit there about cyberdemons and BFGs. From what I understand, that refers to the logo image and variants and not to other things in DOOM. Later notices exclude it, either because they were doing something redundant or improper or because trademark registration made the clarification unnecessary. Under your interpretation, that line gives no way to define what is or isn't trademarked in the artwork, and a trademark has to be very clear. The fact that imps and cyberdemons are distinctive is just a quality of creative copyright and by no means makes them trademarks.

Carmack's email about not using cyberdemons and BFGs counts as trademark protection. The continued sale and development of the Doom series games counts as trademark use.

Carmack's email was not an official document or statement of trademarks. The continued use of easily identifiable and repeated designs doesn't make them trademarks, it simply reinforces the ease with which their copyrights can be protected.

fraggle said:
The Doom monsters - at least the distinctive ones - would count as trademarks. That includes the Cacodemon, Cyberdemon, Mancubus, etc. So would the BFG-9000, and possibly some of the other weapons. These are all distinctive and memorable elements of Doom that make the game what it is.

No. If that were the case, they would be declared trademarks. That's why I asked for an instance saying "cyberdemon is a trademark of id software." They need that claim at least or (more strongly) to register it. They are copyrighted artwork, but their names themselves aren't protected in any way.

Basically, when you think of trademarks, think of Coca-Cola (the name and the logo), or Mickey Mouse. Nobody's stopping you from making your own cola, but you can't call it "Coca-Cola" (or even "Moca-Cola"). Similarly, nobody's stopping you from creating a cartoon with a talking mouse, but you can't call it Mickey Mouse and it mustn't look identical to it.

With Mickey Mouse, you have two aspects. For one, I bet the name is trademarked, secondly, the animated character in all its incarnations is copyrighted and very recognizable as a character. This makes any art that, if the context makes it clear, appears derived from that character, copyright infringement. For Mickey's image to be trademarked it has to be specific, such as a certain design of his face or a pose in some style. They would be like "Mickey logos." But in general, you don't need that to protect the copyright of an identifiable character versus any derivate works by others. The thing is that a character is easy to identify from repetitious use and thus easier to protect its copyright than with other more incidental artwork.

That said, the Wikipedia says "Mickey Mouse is also trademarked" although it doesn't provide citations, at least not there. This could be correct, bad research (especially if it just means the name, which is certain, but not any and all images), or based on some claim Disney made which may or may not be legally valid. In any case, Mickey Mouse is registered multiple times as a trademark for different purposes. I did see some mentions of "trademarked characters" in notices, as well.

Trademarks are badges for companies, products lines and the like that specifically serve as identifiers. The cyberdemon is just artwork in the game. Key artwork, but that does not make it a trademark.

Share this post


Link to post
fraggle said:

What I think the order ought to be is:

  • DEHACKED lump from IWAD is loaded
  • Command line patches (-deh parameter) are loaded
  • DEHACKED lump from PWAD is loaded

How about incompatibility with Boom? Internal dehacked lump is Boom thing. Most wads with dehacked lump are Boom wads. Next time freedoom team (or any other team who will want to make an IWAD) will replace behaviour of monsters in internal dehacked lump and any other wads with this iwad will play differently with prboom and Boom. So we should change behaviour only for "strings" sections, right?

Share this post


Link to post
hex11 said:

Freedoom doesn't work in all ports anyway. First off some maps (like E1M1) are Boom format. Second, there's some problems with animated textures when playing with SDL Doom, and probably anything also directly derived from linuxdoom. I doubt anyone has really tested it on many ports (by warping to E1M2 or loading plain vanilla PWAD).

MAP07 has slopes :)

Share this post


Link to post
entryway said:

How about incompatibility with Boom? Internal dehacked lump is Boom thing. Most wads with dehacked lump are Boom wads. Next time freedoom team (or any other team who will want to make an IWAD) will replace behaviour of monsters in internal dehacked lump and any other wads with this iwad will play differently with prboom and Boom. So we should change behaviour only for "strings" sections, right?

The DEHACKED lump in the Freedoom IWAD is only to change strings, there aren't any plans to change other behavior (monster behavior etc). Essentially it's a "cosmetic patch" - it gives preferred strings but the game doesn't depend on it being loaded. That way, ports like Boom that don't support DEHACKED lumps can still use the IWAD.

As you mention, DEHACKED lump support appeared in MBF, not Boom. But MBF supported multiple DEHACKED lumps, while Prboom/Prboom+ only support one?

Share this post


Link to post

Can I have it explained to me how the observed behavior out of Eternity currently differs from the expected behavior when loading DEHACKED lumps?

Eternity enqueues DEHACKED lumps for processing in the order that wads were added to the wad directory, via use of a FIFO queue. This means that a DEHACKED lump in the IWAD ought to be applied before any in a PWAD.

If both patches use BEX-style string replacement, then there is no possible problem. BEX replaces strings via use of constant mnemonics that cannot be altered by a patch.

I also believe that Eternity, when doing DeHackEd-style string replacement, which works by value, compares against the original string value when doing subsequent replacements, and not with the value as established by the previously applied patch.

Or at least this is my intention. If for some reason this is observed to not be working now, then I have something to fix somewhere.

I must stress that it would be much better if this Freedoom patch were to use BEX mnemonics and not DeHackEd-style string replacement. Eternity considers the latter to be deprecated for use in new modifications, as it's really quite awful from the fundamentals up.

Share this post


Link to post

fraggle:
As you mention, DEHACKED lump support appeared in MBF, not Boom. But MBF supported multiple DEHACKED lumps, while Prboom/Prboom+ only support one?

Yeah, there were a few bits missed when MBF was merged into LxDoom/PrBoom and this seems to be another one.

Quasar:
I must stress that it would be much better if this Freedoom patch were to use BEX mnemonics and not DeHackEd-style string replacement.

Don't worry - this is freedoom's dehacked lump

Share this post


Link to post

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
Sign in to follow this  
×