Proposing a new State table standard for DeHacked

[VGA]

Well, added code pointers and extra frames are cool, but some of the biggest shortcomings of DEHACKED imho is the inability to change intermission placement, episode names and number of maps, map song names, map placement of secret exits ... things that are remedied with ZMAPINFO.

Can someone propose something to address those issues with this new DEHACKED?

[scifista42]

That's a damn great idea, easy to forget about (which I did), good that you brought it up. It would be cool if non-standard map slot names, map order, episodic formats and similar things could be freely messed around with.

[Da Werecat]

VGA said:

map song names

Possible in vanilla DEHACKED, but not in BEX patches for some reason.

[Gez]

VGA said:

Can someone propose something to address those issues with this new DEHACKED?

Why? I'd rather see plain old MAPINFO get broader acceptance. The old ZDoom syntax (which is just the Hexen syntax with map number replaced by map lump name, so that it can work with ExMy maps and not only MAPzz maps) is already accepted by ZDaemon, Vavoom, and IIRC even Doomsday and Eternity. It does all these things and has been widely used in a lot of mods already.

The main problem of the old syntax was that it was difficult to extend it, because there's no real way for the parser to know whether a given unknown token is a keyword it doesn't recognize or a value for the previous keyword; that's why ZDoom went with a new syntax with curly braces and equal signs eventually.

But for a standard, you could go for a minimum feature set, forbid new features entirely, and therefore avoid this problem.

No need to plug DEHACKED into this.

[Quasar]

This has run off the rails completely at this point. Lemme know when we're back to sanity.

[kb1]

Quasar said:

This has run off the rails completely at this point. Lemme know when we're back to sanity.

You know, we could find an open-source C compiler, and rename it "Doom.exe", and modify it to read source files from WADs. Then, we could place the source into a WAD file, one lump per source file. When you run Doom.exe, it opens the WAD, and compiles the source to a file named "Game.exe", and then launches it. FULL EDIT CAPABILITIES!!!

...mandatory heh...

Yeah, I was sure that what I was proposing was so simple and obvious that it would not require a lot of discussion. It kind of explains why there are no standards - every suggestion just gets beat to the ground. It's almost as if the port devs don't ever actually play WADs, and don't experience the frustration of not being able to play various enthusiastic WADs, due to silly incompatibilities.

Some suggest "Just use the intended port to play the WAD." Well, that's good, except some ports just feel different. Some ports can't handle my favorite control scheme, or just handle the mouse a bit differently. Some ports don't have my favorite little game tweaks that I like to use. Some ports desync during LAN coop sessions (I play a lot of coop). And, in my home port, I've added some multiplayer features unique to my port, that I've come to expect.

Now, if the WAD in question has, say, portals, I need to use Eternity Engine. If a WAD has tons of DECORATE, ZDoom/GZDoom are a must. But, for the large majority of WADs, the custom port-specific features are minor, and are already built into my port, but are specified in a port-specific way, so they tend to be ignored.

So I end up with 5 choices:
1. Not play WADs that specify custom features.
2. Just use the intended port to play the WAD, and miss out on my favorite features and favorite control scheme.
3. Build parsers for each and every port's proprietary formats.
4. Propose standard formats that encapsulate every port's specifics.
5. Meet halfway: Extend formats that already exist in most ports, in a way that's easy to code and easy to adopt. Try to get the most out of what's already there.

To me, #1 and #2 are not acceptable. #3 is do-able, and may be necessary, but it's painful, in a way that seems irresponsible, and unnecessary.

#4 is the most ideal (or it was 15 years ago), but it can require lots of custom code to get it working.
#5 seems to me to be a nice middle ground, and requires little consideration (or so I thought).

[Graf Zahl]

kb1 said:

So I end up with 5 choices:
1. Not play WADs that specify custom features.
2. Just use the intended port to play the WAD, and miss out on my favorite features and favorite control scheme.

To me, #1 and #2 are not acceptable. #3 is do-able, and may be necessary, but it's painful, in a way that seems irresponsible, and unnecessary.

Ok, so in other words your standpoint is that in order to satisfy your limited options everyone should cater to the lowest common denominator, and port specific features are best forgotten?

Sorry, it's not going to work like this. Every dev has different interests. Some want demo compatibility at all costs, others go for preserving most of the original glitches while extending the feature set to their own liking and yet others are more interested in serious engine improvements, including fixing the aforementioned glitches. And all that will inevitably cause the ports to diverge and lead to the creation of maps that actually USE the features a port provides.

If 'use the intended port' is unacceptable to you, I really do not care about the consequences this has for you. Your specific problem is fundamentally unsolvable.

[kb1]

Graf Zahl said:

Ok, so in other words your standpoint is that in order to satisfy your limited options everyone should cater to the lowest common denominator, and port specific features are best forgotten?

My limited options? Everyone should do stuff? Forget port-specific features? Could you misrepresent my rather clear, concise language any more thoroughly? How do you possibly reach those conclusions from what I typed?

Graf Zahl said:

Sorry, it's not going to work like this. Every dev has different interests. Some want demo compatibility at all costs, others go for preserving most of the original glitches while extending the feature set to their own liking and yet others are more interested in serious engine improvements, including fixing the aforementioned glitches.

Yes. And some devs would like a port that does ALL of those things.

Graf Zahl said:

...And all that will inevitably cause the ports to diverge and lead to the creation of maps that actually USE the features a port provides.

No, it doesn't. The devs choose to deviate, by avoiding the collaboration necessary to create solutions that could be cross-port compatible. Instead, they plow forward, sometimes stomping on lump names, doomednums, linedef types, etc. Some even go so far as to change the meaning of original concepts.

It's not some mysterious inevitable force that magically causes diversion. It's a completely foreseeable consequence of not promoting the concept of compatibility amongst ports that all claim to render a game called Doom.

Graf Zahl said:

If 'use the intended port' is unacceptable to you, I really do not care about the consequences this has for you.

So, you don't care...yet you comment. What was your ultimate goal when you typed that? Did it have it's intended effect?

My proposal is designed to benefit everyone, now, and in the future, and somehow, you miss the point, over and over.

The "intended port" could become the "intended ports", or "all ports that support X", if compatible solutions are used.

Graf Zahl said:

Your specific problem is fundamentally unsolvable.

...by you. At one time, my port could not play Boom maps, but, now, they play great.

To be somehow more clear: There are a lot of WADs that utilize features that are exclusive to a subset of ports (and, this is ok). "Custom monster support" does not have to be a port-specific feature.

[Graf Zahl]

kb1 said:

"Custom monster support" does not have to be a port-specific feature.

Depends on what the monsters are supposed to be able to do.
You not only need a content definition format but an entire library of action functions and internal capabilities to implement it.

So in order to define new monsters with all the capabilities of ZDoom you pretty much need to implement all of Heretic, Hexen and Strife, plus ACS, ZDoom's entire thingdef_codeptr.cpp and countless new engine features which the monsters may use - like all the flags that have been added over time (Note: ZDoom is at its 8th flags word right now, getting close to 256 flags!) and all the custom properties, custom state labels, full virtual inheritance between classes (i.e. you can create a base class doing state jumps with A_Jump and then override the target states in child classes), expression evaluation in action function parameters, that will NEVER EVER be implemented by a port like PrBoom.

Sure, it is certainly possible to define a more common format that's supported by a wide variety of ports - but you still won't get what you need because most of ZDoom's custom monsters use features your port of choice most likely does not have, making the entire discussion pointless.

Which brings us back to 'play with the intended port'...

[VGA]

I've re-read the OP and believe that what is needed for custom monsters is:

1) Ability to define new REHACKED actors by inheriting from the main game's actors.
2) New state table standard to use namespaces to compartmentalize actors to avoid conflicts between them. So the user can load different .REH files

For example something like this modified Whacked output:

Spoiler

# I gave it a new random doomed id so it doesn't replace the original in the maps and it inherits everything from the standard Imp
# From now on its frame use its name as namespace + the frame numbers
Thing 15001 BuffedImp:Imp
Speed = 12
Hit points = 120
Bits = SOLID+SHOOTABLE+COUNTKILL+TRANSLUCENT

# I reduce the durations of its melee frame, original Imp stays unchanged
Frame BuffedImp.452
Duration = 4

Frame BuffedImp.453
Duration = 4

Frame BuffedImp.454
Duration = 3

And to use new frames, you can use BuffedImp.20001, BuffedImp.20002, BuffedImp.20003, thousands of new frames per actor.

[Graf Zahl]

Why not do a CLEAN implementation for a definition format then.
If you go so far to define inheritance, a simple parser for something SANE is only a minor addition.

No, sorry. Dehacked is not the way to go. You may squeeze a bit more life out of it by extending the actor and state tables but that's basically it.

Once you want real new features, a dedicated definition language is a requirement.

[kb1]

Graf Zahl said:

Depends on what the monsters are supposed to be able to do.
You not only need a content definition format but an entire library of action functions and internal capabilities to implement it.

So in order to define new monsters with all the capabilities of ZDoom you pretty much need to implement all of Heretic, Hexen and Strife, plus ACS, ZDoom's entire thingdef_codeptr.cpp and countless new engine features which the monsters may use - like all the flags that have been added over time (Note: ZDoom is at its 8th flags word right now, getting close to 256 flags!) and all the custom properties, custom state labels, full virtual inheritance between classes (i.e. you can create a base class doing state jumps with A_Jump and then override the target states in child classes), expression evaluation in action function parameters, that will NEVER EVER be implemented by a port like PrBoom.

Sure, it is certainly possible to define a more common format that's supported by a wide variety of ports - but you still won't get what you need because most of ZDoom's custom monsters use features your port of choice most likely does not have, making the entire discussion pointless.

Which brings us back to 'play with the intended port'...

Please read this very carefully:
The goal is NOT to implement ZDoom. The goal is to provide a simple viable way for port devs to spend < 1 hour, to provide lots of missing capability to DEHACKED editors, without having to make major modification or code additions. It is for port devs that are not interested in adding an actor definition language, above and beyond DEHACKED support. Graf, your discussions have been pointless, because you refuse to acknowledge, or see value in these simple goals, despite a lot of rather famous map and wad authors showing their enthusiasm.

[Blastfrog]

I think the most sensible solution is to allow the definition of new states and things, nothing more. Something of a BEX+.

New action pointers, while convenient, stray too far into the territory of more advanced content definition, at which point you may as well use the target port's native format instead of some universal one.

[kb1]

Sodaholic said:

I think the most sensible solution is to allow the definition of new states and things, nothing more. Something of a BEX+.

New action pointers, while convenient, stray too far into the territory of more advanced content definition, at which point you may as well use the target port's native format instead of some universal one.

I do not disagree. It would be nice to do, but it slows the whole thing down, and, I still have not heard a lot of enthusiasm from port devs, which surprises me. However, if we could cobble together a half dozen or so, really useful functions, before port support happens, then yes, they might be able to be included. Otherwise, having the new states and things alone would be a huge help to DeHacked authors.

Speak up, port authors!

Unfortunately, what always happens happened: Someone doesn't "get it" for 5 pages of bickering nonsense, and everyone gets tired of hearing it. Some numb-nut always stirs the pot, and ruins any chance of a new idea getting adopted. And, I think they do it on purpose, for self-serving, biased reasons, or, sometimes, it's just a mean-streak.

Funny thing is, I practically begged everyone not to do that to this thread, which, I suppose, was like an invitation :(

[fraggle]

kb1 said:

Funny thing is, I practically begged everyone not to do that to this thread, which, I suppose, was like an invitation :(

I'm sorry. FWIW, I liked your idea.

[Blastfrog]

kb1 said:

And, I think they do it on purpose, for self-serving, biased reasons, or, sometimes, it's just a mean-streak.

That's not a very diplomatic attitude. The discussion has generally consisted of people proposing more and more involved definition formats, and the port authors became understandably wary of that. They've clearly expressed interest, they just want a solid focus before even considering the proposals.

I think everyone can agree on at least extra states and actors. Anything more is prone to bikeshedding.

[Quasar]

Personally I'd be alright with some kind of DeHackEd tweaks so long as it's possible somehow to tell the intent of the file to adhere to the "standard;" otherwise, it will conflict with Eternity's existing capabilities to create and give DeHackEd numbers to EDF entities.

For example I cannot just "give" away the next few frames in the sequence immediately following the ones defined by MBF - Eternity already defined states with those DEH numbers years ago. Changing them could break things for the 2 or 3 mods we have :P

EE has supported a large number of generalized action functions through DeHackEd since 2003 or so. It does this by adding Args0 through Args5 fields to the frame definition (these have since just become aliases for the same arguments as they now exist in EDF, though, in EDF the limit is 16 arguments because "nobody would ever need more").

In my eyes the action function issue is totally separate because the same action functions can be exposed through DeHackEd or through the port's own CDL. I'd have no issue sitting down to talk about introducing more overlap with a set of basic parameterized action functions. Let's not pretend it's dependent on how the rest of it works.

On the other hand I'm also alright with ideas for a simplified shared CDL but I believe the probability of any of it ever making it into prboom-plus is near 0%, and therefore I am afraid the entire thing becomes a waste of time - if only EE and ZDoom implement it, let's look at what you achieve:

• Your mod has the vast ZDoom community audience of hundreds of DECORATE fanatics
  
• Your mod can also be played by the 32 people in #noteternityenginerelated.
  

Well I guess that's not so much worth the effort from anybody :P

[Graf Zahl]

Quasar said:

For example I cannot just "give" away the next few frames in the sequence immediately following the ones defined by MBF - Eternity already defined states with those DEH numbers years ago. Changing them could break things for the 2 or 3 mods we have :P

Oh, shit!
That truly means we are doomed...

Seriously now. I think you should review what you expose to Dehacked and reduce it to what's absolutely necessary. For example, I really don't understand why you have to expose Dehacked indices through EDF.

With the handful of mods that need to be reviewed - I don't even think any of them actually uses Dehacked in any way - that shouldn't be too hard.

I wouldn't ever expect that those who actually use EDF will ever resort to Dehacked for altering that content so the need for such a feature blocking potential changes of the 'standard' seems a bit dangerous to me.

Sodaholic said:

That's not a very diplomatic attitude. The discussion has generally consisted of people proposing more and more involved definition formats, and the port authors became understandably wary of that. They've clearly expressed interest, they just want a solid focus before even considering the proposals.

That's part of the reason. But it wasn't what my reaction was about. kb1 clearly expressed the hope that a simple Dehacked extension would solve the 'problem' that mods defining custom actors are restricted to certain source ports he doesn't want to use.
But that clearly overlooks the issue that it's not merely the ability to define new actors which may drive a modder to a certain port but also the extent of the ability to define new actors. If all the new extension can do is shuffling around the existing code pointers the capabilities of such an extension and therefore the appeal will inevitably be limited.
To use a simple example, if you want to create another monster with an attack similar to the Mancubus, with Dehacked it only can shoot the same kind of projectile. So in order to make it worthwile all these attack functions need to be expanded to allow shooting other projectiles as well - and now we are getting into a territory where some port authors who might entertain the idea of expanding the Dehacked tables will quit so let's better right away forget about such an extension.
Ergo: All those who want to let loose their creativity to create new content will stay where they are right now - with a port that offers more features. This proposal will simply not solve the issue kb1 hopes it will.
That has nothing to do with me being against it. Expanding ZDoom's Dehacked tables would be a matter of less than an hour of work as it's completely abstracted away from the underlying engine. All I'd need to do is adding some actor definitions to the internal DECORATE and expose them to the Dehacked parser. I wouldn't even have to touch the source for it.

[Da Werecat]

Sodaholic said:

I think everyone can agree on at least extra states and actors. Anything more is prone to bikeshedding.

Even with the existing amount of states and actors my main problem is the lack of sounds to replace. So I'd propose adding a bunch of dummy sounds in addition to dummy actors and dummy states.

Extra sprite sets also won't hurt, but it's not quite as important.

[Quasar]

Graf Zahl said:

Oh, shit!
That truly means we are doomed...

Seriously now. I think you should review what you expose to Dehacked and reduce it to what's absolutely necessary. For example, I really don't understand why you have to expose Dehacked indices through EDF.

With the handful of mods that need to be reviewed - I don't even think any of them actually uses Dehacked in any way - that shouldn't be too hard.

I wouldn't ever expect that those who actually use EDF will ever resort to Dehacked for altering that content so the need for such a feature blocking potential changes of the 'standard' seems a bit dangerous to me.

That's part of the reason. But it wasn't what my reaction was about. kb1 clearly expressed the hope that a simple Dehacked extension would solve the 'problem' that mods defining custom actors are restricted to certain source ports he doesn't want to use.
But that clearly overlooks the issue that it's not merely the ability to define new actors which may drive a modder to a certain port but also the extent of the ability to define new actors. If all the new extension can do is shuffling around the existing code pointers the capabilities of such an extension and therefore the appeal will inevitably be limited.
To use a simple example, if you want to create another monster with an attack similar to the Mancubus, with Dehacked it only can shoot the same kind of projectile. So in order to make it worthwile all these attack functions need to be expanded to allow shooting other projectiles as well - and now we are getting into a territory where some port authors who might entertain the idea of expanding the Dehacked tables will quit so let's better right away forget about such an extension.
Ergo: All those who want to let loose their creativity to create new content will stay where they are right now - with a port that offers more features. This proposal will simply not solve the issue kb1 hopes it will.
That has nothing to do with me being against it. Expanding ZDoom's Dehacked tables would be a matter of less than an hour of work as it's completely abstracted away from the underlying engine. All I'd need to do is adding some actor definitions to the internal DECORATE and expose them to the Dehacked parser. I wouldn't even have to touch the source for it.

Not if the file states, for example in the header, that it's for the new standard. Then I can special case the treatment of such DeHackEd files.

The ability to control DEH through EDF is a fundamental architectural element in Eternity, it is too late to backtrack on that. EDF, unlike DECORATE, defines *everything*. There are no such thing as "native" classes or definitions with more weight or authority than what is in EDF. This means that EDF also defines how DeHackEd works. I believe it was the right decision and stand by it.

However if I can tell a DeHackEd file is "special" then I can use any deh #'s it uses outside the normal range in a special way, to command the EDF subsystems to create new entities for those frames or things as necessary, and nothing conflicts.

[VGA]

Awww noone liked my namespace states idea? Seems to me that's the solution to have infinite actor replacements and infinite states. Since each new actor can exist in its own namespace you can have infinite Mancubus replacements which also fire their own unique projectile replacements while the original things stay unchanged.

You just have to make sure the DoomEd numbers are unique. Example where two new Mancubus variants are created while the original is left unchanged.

http://pastebin.com/cUuPGVQe

Is that hard to implement, port authors?

[Graf Zahl]

VGA said:

Awww noone liked my namespace states idea? Seems to me that's the solution...

That's not a solution. The reason:

VGA said:

Is that hard to implement, port authors?

Yes, indeed, this would require a vast undertaking to get working and will certainly kill all momentum.

[kb1]

Sodaholic said:

That's not a very diplomatic attitude. The discussion has generally consisted of people proposing more and more involved definition formats, and the port authors became understandably wary of that. They've clearly expressed interest, they just want a solid focus before even considering the proposals...

Yes, you're right, of course. I am frustrated with the difficulties involved in trying to get momentum, and I am frustrated in general, about the fact that the ball was dropped a long time ago, making discussions like this all the more difficult. And, when the discussion turns to custom formats, I have to request that we start a new thread.

Clearly my proposal is not the most versatile approach, but is has the one benefit: It can be added to ports very easily, which kinda makes it a no-brainer.

Quasar said:

Personally I'd be alright with some kind of DeHackEd tweaks so long as it's possible somehow to tell the intent of the file to adhere to the "standard;" otherwise, it will conflict with Eternity's existing capabilities to create and give DeHackEd numbers to EDF entities...

Good to know! And, I think it is quite reasonable for these files to be marked somehow, as being built to expect the extra state support. Would it suffice to use the DeHacked version number, or a new identifier, or as a per state thing? I'm thinking a special identifier would do the job. Then again, BEX got away with changing the parser somewhat (I'm thinking about your extra Args - other people did mention wanting more than the 2 ints, Misc1+2).

Quasar said:

...EE has supported a large number of generalized action functions through DeHackEd since 2003 or so. It does this by adding Args0 through Args5 fields to the frame definition (these have since just become aliases for the same arguments as they now exist in EDF, though, in EDF the limit is 16 arguments because "nobody would ever need more").

Again, there's the balance between wanting to support new functionality, and limiting the changes needed for ports. If we support additional Args, ports need to add them to the mobj struct, save/load them, etc, which widens the scope a bit. I like the idea, but worry about it.

Quasar said:

...I'd have no issue sitting down to talk about introducing more overlap with a set of basic parameterized action functions. Let's not pretend it's dependent on how the rest of it works.

Very encouraging! I think we have a few good candidate action funcs, already mentioned in this thread, and I'd love to consider some more. Hopefully, we would end up with a small number of powerful funcs.

Graf Zahl said:

...But it wasn't what my reaction was about. kb1 clearly expressed the hope that a simple Dehacked extension would solve the 'problem' that mods defining custom actors are restricted to certain source ports he doesn't want to use...
...This proposal will simply not solve the issue kb1 hopes it will.

Fair enough. I stand corrected - Graf, you [do] understand my goal...somewhat. But, I never said that I didn't want to use certain source ports. What I said was that something as simple as having a few new monsters should not always require a source port with a custom actor definition language.

It's not about driving a modder to go to a specific source port. It's quite the opposite: This proposal should allow many modders to use ANY source port. Of course, what they will be able to do is limited. We will provide a small number of new functions to expand the capabilities a little bit: A "SpawnCustomMissile" func, for example. People already accomplish some great things with DeHacked (think Batman Doom).

Da Werecat said:

...I'd propose adding a bunch of dummy sounds...

Agreed - it should be included.

VGA said:

Awww noone liked my namespace states idea? Seems to me that's the solution to have infinite actor replacements and infinite states. Since each new actor can exist in its own namespace you can have infinite Mancubus replacements which also fire their own unique projectile replacements while the original things stay unchanged...
Is that hard to implement, port authors?

Sorry VGA, somehow I missed your idea. It could almost be implemented inside the DeHacked editor, without any special support in the game. In the editor, you'd select the object to clone, and the editor would copy all of that object's states into new frames, and a new actor definition. It would not allow "infinite" replacements, it would just make cloning a bit easier.

Quasar said:

...I'm also alright with ideas for a simplified shared CDL but I believe the probability of any of it ever making it into prboom-plus is near 0%, and therefore I am afraid the entire thing becomes a waste of time...

Ok, this is a common concern in this thread, so I left it for last. Here's how I see it, anyway:

I do understand the chicken-and-egg issue. As far as PrBoom+ goes, I do not know what entryway's intentions are. But please note that, upon the original source release, there were no community source ports. Nowadays, I can take prboom-plus, implement this proposal, compile it, host it, and call it "pr-shrooms". Then, I could make a monster resource WAD, and challenge the community to make me 3 colossal, beautiful maps with those new monsters, and release them.

I wonder how long it would take for the code changes to appear in a bunch of ports... If the maps were really fun, and the monsters were interesting, it just might happen. I'd add the changes in my home port (which will get released, one day), just so I can play the maps in my port.


EDIT: I just had a change of heart on this whole issue. I just got done typing up a huge new thread about reviving the old Doom Standards Committee, with all of the benefits, and a straight-forward method to handle management of this idea across different ports, bla bla. Then, I deleted it.

Once I get my port up to speed, and published, I will be in a position to suggest ways to bring compatibility and compliance to the various other ports. Until then, I want to concentrate on my own efforts. I can't keep fighting for everyone else, especially when I have nothing to demonstrate.
Thanks for your enthusiasm, support, and understanding. Until then...

[Worst]

bump

I have suggestion regarding extra states/things:

Since different sourceports may have extended the dehacked indices differently (like Eternity), what if the extra states had their own range that would be referred to with for example a $-sign infront of the number.

Frame $1
Sprite number = 29
Sprite subnumber = 0
Next frame = $2
Duration = 4

Frame $2
Sprite number = 29
Sprite subnumber = 0
Next frame = 174
Duration = 5

[CODEPTR]
Frame $2 = Explode

Thing 3 (ShotgunGuy)
Initial frame = $1
First moving frame = 209

The sourceport could just map these numbers to whatever range fits best for it internally.

[Quasar]

Worst said:

bump

I have suggestion regarding extra states/things:

Since different sourceports may have extended the dehacked indices differently (like Eternity), what if the extra states had their own range that would be referred to with for example a $-sign infront of the number.

While it would work, it's more hacky than is necessary IMO. Then again, parsing comments as metadata is a bit of a hack also, and that's what I had in mind earlier (BOOM itself ignores the DeHackEd version header at the top of the file and doesn't require that it exists, so, you could technically put any kind of identifier you wanted there).