DSDhacked [unlimited everything]

[GoneAway]

2 hours ago, Graf Zahl said:

Maybe we can just do away with them entirely. If the parser could be changed to optionally parse these as strings in addition to numbers we could just write
"sprite = POS2" or "Attack Sound = poshot" and remove the [SPRITES] and [SOUNDS] blocks entirely and just dynamically add content to the arrays when a new name is found.

I considered this - actually I considered removing indices completely - but I discarded it as basically being too much of a departure and inconsistent in the format.

[Graf Zahl]

Sadly in return it makes the entire thing extremely more complicated for some ports because there is no good way to handle this indirection without the linear table, not to mention being less editing friendly and error prone.

 

BTW, I'd be interested to hear how Team Eternity thinks about all this, they have been quite absent from this discussion so far.

 

[Gibbon]

1 hour ago, Graf Zahl said:

Sadly in return it makes the entire thing extremely more complicated for some ports because there is no good way to handle this indirection without the linear table, not to mention being less editing friendly and error prone.

 

BTW, I'd be interested to hear how Team Eternity thinks about all this, they have been quite absent from this discussion so far.

 

What about EDGE?

[GoneAway]

1 hour ago, Graf Zahl said:

Sadly in return it makes the entire thing extremely more complicated for some ports because there is no good way to handle this indirection without the linear table, not to mention being less editing friendly and error prone.

 

BTW, I'd be interested to hear how Team Eternity thinks about all this, they have been quite absent from this discussion so far.

 

Altaz and I discussed this with MTrop. The main issue was just needing to know that the dehacked file is targeting dsdhacked, hence the new version at the top. With decohack, I think editing is not an issue these days. At least, it abstracts away what actually happens in the dehacked itself.

 

Since dsdhacked is only particularly relevant for mbf21 and the advanced ports implementing that standard (gzdoom and eternity) have also either implemented dsdhacked or plan to, it doesn't feel like there's a problem that really needs to be redeliberated to me 😄

 

On the non-advanced side dsda-doom and woof both have it already as well.

 

These 4 basically represent the frontrunners in standard adoption. And these ports also cover the popular different groups that would participate in mbf21 (in doomworld anyway - obviously it's not a perfect measurement https://www.doomworld.com/forum/post/2366714). Crispy / Choco are out of scope. This isn't to say that other ports aren't important and relevant to the discussion, it's just to put things in perspective.

[Graf Zahl]

14 minutes ago, kraflab said:

Since dsdhacked is only particularly relevant for mbf21 and the advanced ports implementing that standard (gzdoom and eternity) have also either implemented dsdhacked or plan to, it doesn't feel like there's a problem that really needs to be redeliberated to me 😄

 

No, you are just leaving us with the mess this creates - and of course give Ketmar more reason not to implement the spec at all...

Granted, the sound and sprite issue is minor compared to the states, but it's still a totally needless and useless complication of things compared to dynamic allocation of sounds and sprites at the places where they get referenced.

 

1 hour ago, Gibbon said:

What about EDGE?

 

No idea. I haven't heard anything from them at all, but then, there has not been much activity on their repo in recent months.

 

 

 

[GoneAway]

1 hour ago, Graf Zahl said:

but it's still a totally needless and useless complication of things compared to dynamic allocation of sounds and sprites at the places where they get referenced.

Nothing stops you from dynamically allocating wherever you want to. There are plenty of ways to approach it. If you made the identifier the lump name then it would make it impossible to define multiple sounds with different properties but the same lump name, which would mean removing features from dehacked.

 

There's a huge level of defeatism about source ports. A new standard like mbf21 is impossible, unlimiting dehacked is impossible, implementing heretic and hexen in prboom+ is impossible, adding new map format support is impossible, we'll never have decorate in classic ports. It's sad that so much time was wasted because of this obstinate and pessimistic outlook.

[Graf Zahl]

I already do dynamic allocation for everything. Remember, I do not have static arrays for these things.

I just do not like the way this was done because it's needlessly too low level and exposes implementation details that could have easily been hidden away.

 

 

[HavoX]

10 hours ago, kraflab said:

Tbh I just don't know enough to answer something like this definitively right now 😅

Well, I'd love to see a semi-universal DECORATE solution for most compatible ports (not just ZDoom) but I wouldn't be in a rush about it.

There would be a lot of obstacles in the way that have to be dealt with.

 

...and what Graf said, obviously. ;-)

[Gibbon]

28 minutes ago, kraflab said:

Nothing stops you from dynamically allocating wherever you want to. There are plenty of ways to approach it. If you made the identifier the lump name then it would make it impossible to define multiple sounds with different properties but the same lump name, which would mean removing features from dehacked.

 

There's a huge level of defeatism about source ports. A new standard like mbf21 is impossible, unlimiting dehacked is impossible, implementing heretic and hexen in prboom+ is impossible, adding new map format support is impossible, we'll never have decorate in classic ports. It's sad that so much time was wasted because of this obstinate and pessimistic outlook.

Decorate in classic source ports could be possible, though the reference implementation should be in C.  It's easier to add C into C++ than visa versa.  Ports that chose to convert to C++ have to realise that this meant a lack of portability of code between them.  None is more 'correct' but a decent middle ground has to be agreed on.

 

There is no way I would wrap externs around stuff.  Very messy.

[Graf Zahl]

Please don't come with that C argument again. The language sucks when serving as a 'reference' . it often means that one can take the code and toss it into the dumpster right away. Reusability of C code is no better than C++ unless you restrict yourself to the language core which lacks virtually everything needed to do a good parser. So the result will be to bring in some third party code to make the job easier and we've just lost the 'portability' argument.

 

Besides, the UMAPINFO parser already is C++, because I had no working C parser capable of doing the job and saw no point writing one, as C++ variants were readily available, and it hasn't harmed anyone.

 

[ketmar]

i don't think that the language of the first implementation matters at all — because the implementation shouldn't serve as specs anyway. and with proper specs people will write the proper implementation in any language they want. DECORATE already suffers from the absence of the specs, so if we will ever start seriously considering DECOLITE, we shouldn't repeat the same mistake. specs first, implementation next, not vice versa.

 

p.s.: and this time, PLEASE, make newline a proper terminator! and don't allow numbers in quotes, or unquoted strings, or quoted identifiers.

[ChopBlock223]

What causes problems with numbers in quotes?

[ketmar]

10 minutes ago, ChopBlock223 said:

What causes problems with numbers in quotes?

because it is impossible to tell them from strings. is `A_Jump(256, "2")` a jump with offset, or a jump to a label? and yes, i've seen DECORATE with such labels. and the label can be defined later in the code, so you cannot tell if it is a label name or not at the parse time, you have to make an additional pass over the parsed code to resolve such things.

 

also, it makes syntax/type checking harder. if something expects a number, and then a string, it is easier to tell that something like

A_Smth("42")

is a bug, because somebody missed the numeric argument.

 

it is all about consistency, and make catching bugs easier.

 

ZDoom never bothered to properly check argument types, and oh, you prolly can't even imagine how many mods around have bugs nobody noticed for years! and many of such bugs seems to be simply copy-pasted over and over again. i had to add the whole "analyze and bugfix" pass to k8vavoom DECORATE parser to fix at least some well-known DECORATE misuse, which could be catched by a simple type check like "the number should not be quoted, ever, but the string cannot be without quotes".

Edited September 16, 2021 by ketmar

[Graf Zahl]

That "numbers in quote" thing is owed to Hexen's sc_man parser which was used for that first DECORATE version. ZDoom's version of that only got a proper tokenizer years later when it was too late and some idiots had actually released stuff that used numbers in quotes... (no kidding here... :?)

 

 

[Kinsie]

Having thought about and discussed it a little, my current feeling is that DSDhacked should probably wait and see how MBF21 pans out first.

 

MBF21 is extremely early in its lifespan - I don't think it even have a demo wad publicly released yet! - and as such we don't really have a good idea of the ifs, whens and hows of people using an extended, expanded version of DEHACKED or the various pitfalls and issues that may crop up while using it in projects both big and small. Give it some breathing room and take care of the growing pains as that new standard establishes itself, and then we can use what we've learned from that experience for an across-the-board implementation of a limitless version of DEHACKED that'll be all the better as a result.

[MTrop]

I don't know why this discussion even exists when I've taken up the mantle in making DECOHack (a DECORATE-flavor-to-DeHackEd converter). It was the easiest task for me to take because:

 

A) I wanted to make it since forever.

B) The burden of converting a better definition language to a common one is taken off the table for port authors who would puff their chests out and potentially complain about needing to backport a superior, human readable-and-writable definition language.

 

So I'm gonna keep plugging away at it, if you don't mind, while this slapfight continues.

 

You're welcome.

[Gibbon]

1 hour ago, MTrop said:

I don't know why this discussion even exists when I've taken up the mantle in making DECOHack (a DECORATE-flavor-to-DeHackEd converter). It was the easiest task for me to take because:

 

A) I wanted to make it since forever.

B) The burden of converting a better definition language to a common one is taken off the table for port authors who would puff their chests out and potentially complain about needing to backport a superior, human readable-and-writable definition language.

 

So I'm gonna keep plugging away at it, if you don't mind, while this slapfight continues.

 

You're welcome.

Now this is interesting!  Is there a repo anywhere?  I'd love to see what you're doing.

 

And I agree, the arrogance level here is getting a bit much.

[Astronomical]

55 minutes ago, MTrop said:

The burden of converting a better definition language to a common one is taken off the table for port authors who would puff their chests out and potentially complain about needing to backport a superior, human readable-and-writable definition language.

That's the whole issue boiled down in a sentence.  When mapping, the issue is picking a standard to use when making it.  I have never had the desire to understand decorate because it isn't a cross platform standard.  Now seeing this thread it calls into question the whole goal of MBF21, from my understanding the plan was to make a standard that was more advanced than mbf, that would be cross platform.  Boom is still one of the most popular formats because every port supports it, while being able to do so much more than the original exe.

[MTrop]

6 minutes ago, Gibbon said:

Now this is interesting!  Is there a repo anywhere?  I'd love to see what you're doing.

Here you are, happy to oblige: https://github.com/MTrop/DoomTools

Main website here: https://mtrop.github.io/DoomTools/

 

It's included in a bunch of other tools (and the parser needs a tiny code revamp), but it's completely functional.

Original thread, as well: 

EDIT: I wish I had a better (public) project to show off, but here's an example one: https://github.com/MTrop/doommake-example

 

The project format is not up-to-date, but it still runs (most of the project code is in scripting logic).

[ketmar]

1 hour ago, Graf Zahl said:

(no kidding here... :?)

nope, no kidding. i actually had to write some code to deal with such crap. i won't call names (i simply don't remember now), but that's definitely not something i was going to write just because i had no better things to do. ;-)

[Gibbon]

6 minutes ago, MTrop said:

Here you are, happy to oblige: https://github.com/MTrop/DoomTools

Main website here: https://mtrop.github.io/DoomTools/

 

It's included in a bunch of other tools (and the parser needs a tiny code revamp), but it's completely functional.

Original thread, as well: 

 

Thanks!  Luckily my Java is excellent :) nice work

[ChopBlock223]

2 hours ago, MTrop said:

I don't know why this discussion even exists when I've taken up the mantle in making DECOHack (a DECORATE-flavor-to-DeHackEd converter).

I like being able to read and write the code directly, without having to convert/deconvert, or compile/decompile it, and there's also some things which DeHacked just will not do that I think could be very useful. If it's also easier to implement and maintain such a functionality and language for other devs, that does sound better to me.

 

 

[dew]

6 minutes ago, ChopBlock223 said:

I like being able to read and write the code directly, without having to convert/deconvert, or compile/decompile it, and there's also some things which DeHacked just will not do that I think could be very useful. If it's also easier to implement and maintain such a functionality and language for other devs, that does sound better to me.

Well, what's stopping you from mapping in Zdoom then?

[Redneckerz]

2 hours ago, MTrop said:

I don't know why this discussion even exists

Well this was originally to determine whether or not DSDHacked should be treated as a separate spec or being intertwined with MBF21.

 

Somehow we are now talking DECOLITE which as it turns out is a far older idea being discussed now, coming back from 2017.

 

I just want clarity in that regard.*

 

Spoiler

* And if someone with actual knowledge could either PM me or edit in the specifics of the MBF21 implementation and DEHEXTRA on the wiki, i'll be delighted.

 

[ChopBlock223]

1 minute ago, dew said:

Well, what's stopping you from mapping in Zdoom then?

People don't speedrun ZDoom maps, I want my maps to be demo recordable, something which ZDoom ports aren't particularly conducive to. I also find that Boom/MBF21, along with UMAPINFO fills 95% of my needs for map features.

 

Finally, people really don't seem to want to play ZDoom maps much, I do, there's some sets I'm fond of, but many won't. I know that this is a circular problem, but if I don't feel much of a need for many advanced map features in UDMF, and people are significantly less likely to speedrun, record, or even play them at all, those are pretty significant detriments to me.

[dew]

I don't understand how the boom environment/pr+ with its backward compatability remains the same if dehacked is jettisoned in favour of an expansive decorate implementation. How many pieces of zdoom can you fix the Ship of Prboom with until it's just zdoom again?

[Graf Zahl]

If you get slopes working, a lot, actually. The places where the real problem starts is custom inventory items. Everything before that can be ported to a more vanilla-like port if someone took the needed effort to make it work.

 

That'd mean, up to 2.0.98 should be possible.

 

[LexiMax]

 

2 hours ago, dew said:

How many pieces of zdoom can you fix the Ship of Prboom with until it's just zdoom again? 

 

I'd like to think of it more as a re-implementation of certain selected desirable features, with the benefit of hindsight and with an eye towards portability among ports with codebases that resemble the original source release.

 

A lot of ZDoom's excesses (at least when I was modding Zandronum) were enabled by the tons of extra codepointer functions and the fact that you could compose expressions.  And one of those functions just happened to be "Run ACS Script", which opened up the entirety of the ACS language and its myriad of available functions.  Graf mentioned custom inventory items and I recall seeing some hot nonsense with those too.

 

A hypothetical DECORATE for frametables would just mean I don't have to open up WhackEd to do a thing the engine was already capable of doing.  Seems like a convenience and accessibility win.

 

2 hours ago, Graf Zahl said:

If you get slopes working, a lot, actually.

 

Odamex was the first GPL port to have slopes, and our port is compatible enough to play the demo reel.  I haven't looked the physics of it in a hot minute, but the code is there for the taking if someone is interested.

[MTrop]

For what it's worth, a lot of points brought up about DeHackEd being this ancient "standard" format are pretty salient, but even if it weren't referencing a contiguous set of resources, it's still a low-level mapping of "stuff to other stuff". The moment when things get mixed with the more modern implementations of that stuff is when things get hairy.

 

GZDoom's got pretty dang good DEH tolerance/support plus how it handles DECORATE/ZScript overriding, and Eternity's gonna have some terrible (yet very workable) growing pains, so I'm not too worried.

 

God help you if you try to implement DECOHack's flavor of DECORATE into your port, which I don't recommend at all, especially when I still have that "0" in front of the version number. Everything's documented, but I really gotta write up its grammar one day...

[Graf Zahl]

If you want something grammar based you may just take ZScript minus the actual code part which is essentially DECORATE with a stricter syntax.

Of course an ad-hoc parser will be significantly less work to implement than a grammar-based one.

 

40 minutes ago, AlexMax said:

A lot of ZDoom's excesses (at least when I was modding Zandronum) were enabled by the tons of extra codepointer functions and the fact that you could compose expressions. 

 

None of that is in a form that can't be ported to other ports. Even the entire expression evaluator contains nothing show stopping.