Defining a new enhanced source port standard (Boom+/MBF+)

You know, NIH isn't that big a deal. And functionally there's no difference between telling people to learn this syntax to use in INI file strings and telling people to learn this syntax to use in a format where the only differences with INI is that sections are delimited with curly braces and strings are within quote marks.

 

I'm going to make a small digression and speak about one of the largest mod hosting website out there, Nexus Mods. It started as TES Nexus (the www.tesnexus.com URL still works) and it was all about hosting Morrowind mods, then also Oblivion, then also Fallout 3, and so it started branching out more and more to cover Dragon Age, the Witcher, what-have-you. So the thing is that this website's ascension was borne from the momentum it had acquired hosting mods for Bethesda games.

 

So I'm now going to go about Bethesda modding. Have you ever looked at Morrowind scripting? It's an absolutely horrible language with a terrible, terrible parser. It causes no end of pain to modders, and it has some really weird quirks. Did it stop people from learning about it and using it to make amazing things? Did it dissuade them to invest into the game and cause them to instead go back to Minecraft? Nope. Oblivion scripting was largely the same, except just different enough that people who had learned Morrowind scripting had to relearn it. Skyrim did the same thing again, this time giving the scripting language a name (Papyrus) but when you look at it you see it's still a new incarnation of that same scripting language. Anyway, this idiosyncratic scripting language didn't deter the Elder Scrolls and nuFallout games from having large and persisting modding communities. Likewise, titles such as Dragon Age have their own scripting language instead of using Lua or AngelScript. So first point is this: custom formats aren't as much of a problem as you claim.

 

On the other hand, you can have games that use tried-and-true external standards, and yet people still need to learn a lot of game-specific stuff. Take Civilization V, it uses Lua and XML (yay) but it's not exactly standard anyway. Which brings to the next point of this digression: even if you use a standard format, you'll have non-standard elements, because the scripting language needs to interface with the engine. Out of the box, they rarely do. This is often a problem. And it'd be the same thing with Doom: the Doom engine defines what's in a mobj, how they interact, the kind of things they can do (such as respawning). All these particularities would necessarily result in added constraints and some special subformats for newbies to learn. So the second point is this: standard formats will not stop people from having to learn.

 

To put it in another way: if the most important thing is to have a standard defined entirely so that people don't have to learn about anything internal to the game engine, then the best we can do is toss the ports to the trash and just use Unity instead. Everyone uses Unity.

 

Who needs PrBoom when you can have that instead? :p

 

On mardi 2 mai 2017 at 6:32 AM, kb1 said:

I called it GAMEINFO. I'll have to change that name, but that's beside the point.

May I suggest "KBCONTENT"? It should be safe from collision. It shouldn't even be confused with any sort of keyboard-related stuff.

 

Let's pick this apart...

 

1 hour ago, Ladna said:

Hey!  I work on D2K, a PrBoom+ derivative.  In the past I've worked on EE (a lot) and Odamex (a little).

 

I will never sign on to a standard that bakes in Doom-specific formats.  I'm not doing this because I like it, in fact I've already blown a lot of coding time on this thread and I kind of resent it.  But I genuinely think an advanced port standard that uses proprietary,

 

That's complete and utter bullshit. What do you want?

INI? JSON? XML???

Sorry, but you are completely missing the intended target audience here, which is not programmers but regular people who enjoy making maps. You cannot come to them with trying to shoehorn something really complex into a generic container format. That may work for MAPINFO, even TERRAIN, SNDINFO or SNDSEQ, just to name a few of the lumps ZDoom inherited mostly from Hexen, but it doesn't work for a complex thing like an actor definition, which isnt just some 'key = value' thing. You cannot express the states like that and end up with something totally and utterly user-unfriendly.

 

You also do not want to saddle end users with formats that require following some very strict formular approach of defining the data that may make sense to a programmer but not to a modder. If you go that route, behold Doomsday's definitions, they are strictly formulaic to the extreme, but also horrendously bloated. In terms of programming it's the assembly language where DECORATE is C. You wouldn't write your program code in assembly, would you? You'd prefer a language that abstracts away the gory details from the modder. Why not here? DECORATE was specifically developed with the goal to reduce the boilerplate stuff to the maximum extent possible, and it's only that which even allowed the stuff some ZDoom modders came up with. With your way we'd still be at redoing the simpler stuff of 2004's monster resource WAD.

 

 

1 hour ago, Ladna said:

 

I sort of considered not writing a big post.  Of course I did anyway.

 

Doom is already mired in its own vocabulary.  No one knows what DECORATE is, so if you, say, make mods for other games or want to get started on a mod for Doom, you get to wade through ZDoom's wiki.  Sure that's the case whenever you don't know something, but the difference between something DECORATE and INI is that there's 832,902,382,340,929 examples of INI files out there, not so much when it comes to DECORATE.  Besides the (obvious) fact that you've probably come across an INI file or two in your life.

 

That's a very stupid comparison. Show me the INI that defines a Doom actor. The format cannot even handle the SEMANTICS of what makes up a Doom actor! It's also completely pointless to show someone who wants to create a Doom actor in a hypothetic INI-based format if all you can show them is an INI that ZDoom had written out as its config file, for example. The basic syntax is completely irrelevant, it has absolutely no meaning whatsoever, what fills this stuff with life is the semantics behind the keywords and their values and none of your 832,902,382,340,929 examples would contain that.

 

 

 

1 hour ago, Ladna said:

 

No one knows what ACS is, so if you, say, make mods for other games or want to get started on a mod for Doom, you get to wade through ZDoom's wiki.  Sure that's the case whenever you don't know something, but the difference between something like ACS and Lua is that there's 832,902,382,340,929 examples of Lua code out there, not so much when it comes to ACS.  Besides the (obvious) fact that if you're a game modder, you've probably come across a game scripted in Lua or two in your life.

 

Yes, and again: Lua may be popular by some people but it's also very divisive in how people react to it. Regardless, it's totally irrelevant again, because in the end it doesn't matter what the syntax of the language is. What matters is the functionality of the available keywords. No matter whether ACS, ZScript, FraggleScript, C++, C#, Java or Lua is being used, Floor_LowerToLowest will still be the same in all of them. And if you take a look at all the languages I listed, behold: Lua is the sole outlier when it comes to general syntax!

 

 

1 hour ago, Ladna said:

 

See the difference is in our perspectives.  Most of the posters here are focused on this tiny (though very cool) DW community, but I'm commenting from the perspective of a professional developer and general human being.

 

I am also a professional developer but I see absolutely nothing good in your approach. Sorry to burst your bubble. Standards have a place, but sometimes standards just do not work to completely express a task in an efficient manner. What then?

 

 

1 hour ago, Ladna said:

 

 

 It's true (maybe?) that most DW mappers are familiar with DECORATE, but I want more and more people working on and experiencing Doom.  

 

And the best way is to define things in a way that is first and foremost expressive, i.e. it focusses on what is important for the job at hand. Strict adherence to some simplistic container format is not going to cut it.

So, this is a Zombie in ZScript, which just happens to be a tad different from DECORATE but it will show my point:

 

//===========================================================================
//
// Zombie man
//
//===========================================================================
class ZombieMan : Actor
{
    Default
    {
        Health 20;
        Radius 20;
        Height 56;
        Speed 8;
        PainChance 200;
        Monster;
        +FLOORCLIP
        SeeSound "grunt/sight";
        AttackSound "grunt/attack";
        PainSound "grunt/pain";
        DeathSound "grunt/death";
        ActiveSound "grunt/active";
        Obituary "$OB_ZOMBIE";
        DropItem "Clip";
    }
    States
    {
    Spawn:
        POSS AB 10 A_Look;
        Loop;
    See:
        POSS AABBCCDD 4 A_Chase;
        Loop;
    Missile:
        POSS E 10 A_FaceTarget;
        POSS F 8 A_PosAttack;
        POSS E 8;
        Goto See;
    Pain:
        POSS G 3;
        POSS G 3 A_Pain;
        Goto See;
    Death:
        POSS H 5;
        POSS I 5 A_Scream;
        POSS J 5 A_NoBlocking;
        POSS K 5;
        POSS L -1;
        Stop;
    XDeath:
        POSS M 5;
        POSS N 5 A_XScream;
        POSS O 5 A_NoBlocking;
        POSS PQRST 5;
        POSS U -1;
        Stop;
    Raise:
        POSS K 5;
        POSS JIH 5;
        Goto See;
    }
}

 

You can run a newbie down this definition from top to bottom and have them understand this stuff rather quickly. There's the labels on the states which are special entry points for predefined actions. There's some basic properties like monster health, the sounds being played and a few other things. Most importantly, the state sequences are IMPLICIT! There's no need to explain that all states are physically linked together by some internal pointer. Nobody needs to know that.

Now try the same with an INI based definition. You first will have problems expressing those state sequences adequately, so due to the formats limited options the whole thing will end up mired in some messed up syntax that needs to be invented to work around the container format's limitations. The same problem, albeit a bit less pronounced will happen with JSON and XML. That's because INI, JSON and XML are all designed as low level container formats, a job for which they are very well suited. But aside from the Default block this isn't a simple set of values, it is a very complex construct that requires a lot of intelligence to construct from the above data, none of which the user needs to really know.

 

If you were talking about some simple definition format I'd agree on all accounts, but that's not the case here.

 

 

1 hour ago, Ladna said:

 

 

These NIH formats are a huge barrier to that, which is why I'm so vociferously against them.  Most people don't know C/C++; if they know anything it's a scripting language.  Scripting languages don't have support for DECORATE, or libConfuse, or whatever Doom-y format you decide on.  So if they want to write a cool Doom thing, there's a huge hurdle.  I get why mappers are down with DECORATE and EDF: they've invested the time to become experts.  But you shouldn't have to invest that much time, and the skills you get should be applicable to other areas.  Like when you learn C# for Unity, hey you learned C#!  When you learn ACS for ZDoom... rofl.

 

Again you are missing the point by a huge margin. ACS is not that different from other programming languages. If you cannot program you cannot possibly comprehend what's being done here. So what if they know a scripting language? Which one? Lua, AngelScript, or Javascript or what? Each one is different so someone being used to Lua will have the same problems with AngelScript as with ACS. On the other hand, someone familiar with Angelscript will tear their hairs out when subjected to Lua but mostly feel right at home with ACS or ZScript. The bottom line is, that the only people who would have problems with the syntax are Lua users, but that's something that cannot be helped. Lua chose to stand apart, so it there will be some barrier between it and C-like languages. As for the specifics of ACS, we now again get into the matter of semantics. It's not the language's syntax that's the barrier, but how it needs to be used to do something useful. And that requires learning and understanding, that won't magically go away if everything was done with preexisting standard solutions.

 

 

1 hour ago, Ladna said:

 

Feel free to make a standard without me!  Honestly I'm already planning to support a lot of the creepy toy store that is ZDoom features, so whatever.

 

Pretty much all of those are jokes, but OK I'll stick to substantive criticism from now on.

 

Assurances of "there will be a spec and you will like it" don't do anything for me.  You can write a perfect spec, but if it's a Doom/NIH format, I won't support it.  You can write a C library for it, but if it's a Doom/NIH format, I won't support it.  I think I've been pretty clear why, let me know if there's any confusion or rebuttals about it, but quit asking me why.  I pretty much type the same things in every post, and the response is, "get over it".  No, I guess.

 

I don't want to standardize on a library or a reference implementation.  I want a spec'd format.  Essentially if there's no maintained Python library for it, I won't support it.

 

In that case: Goodbye! Either you are out or I am out because our goals are obviously on the opposite end of the spectrum. You failed to bring your point across until here so any further talk won't make it any better.

 

 

 

1 hour ago, Ladna said:

 

- INI is either as short or shorter than any DECORATE-lite example posted, but the difference either way is like 2-3 lines.

- INI is used by goddamn everything.

- Millions of people use INI files and have for decades.

- INI totally does, and the sprites string is an obvious win

 

And it won't help you one bit because the superficial parsing here is not even half the job of compiling DECORATE. If you want a solution that is user friendly, more is needed, if you want something that can get parsed directly into the data structures it will be worthless.

 

And that doesn't even address the fact that INI as a format is garbage, except for the most simplistic one-dimensional stuff out there. Half of ZDoom's definition lumps need at least subblocks which rules out INI as a format.

JSON and XML may work but they are just too rigid for hand written data.

 

I decline to comment on the rest because it obviously serves no point.

 

 

 

The whole thing about state sequences has already been hashed out already. See here.

Gez: Those are reasonable points.  I disagree though, those Bethesda games were genre-defining sandbox games.  The same thing happened there that happened with Doom 20 years ago: people put up with a pretty bad mod system to create something for an amazing new thing.  Doom now is not TES, so I think we have to change our approach.

 

I agree functionally there is no real difference between simple key-value formats when writing them.  But when working with them, if you can't just say 'require ini' or whatever, that's a big speed bump.  Cool, now I get to write a parser.

 

I agree people will always have to learn.  The question is if they might already know some things, and if they'll know a useful, general skill after learning.  With INI, they'll probably already know.  With Lua or something else, they'll learn a general purpose programming language, and it's at least possible they already have experience with it.  These are all clear wins over DECORATE/ACS.

 

I think Godot's technical reasons for GDScript are fine, but having pretty well integrated Lua into D2K, I can say most of them don't apply.  Inheritance is easy, no threads, gc pauses are negligible, etc.  Can't really speak to code completion but, whatever.

 

I don't think we need to throw away things that are integral to Doom, like, I guess, things have heights, a radius, etc.  But we should be careful about what we assume the base is.  Voxels are a good example of this; the state stuff doesn't really handle that, so effectively if you standardize on the frame stuff you're saying any method of displaying things has to somehow map to frames, which voxels don't necessarily.  Or you can say "we only really mean sprite actors here", or have some kind of extension I guess?  Maybe it's an argument for separating the definitions.

 

Graf:

 

INI can easily deal with everything you listed in your example, and most of the time it's more concise.  I posted a full INI example of an imp from EE's EDF.  If you have specific opposition to it, I'd love to hear about it.

 

I didn't mean to specifically make this about Lua, it's just a common game scripting language.  I think JavaScript is maybe a better example, but whatever.  The point remains that ACS is a niche language, and after you learn it you can't really do anything other than mod Doom.  Even inside Doom you need a lot of special support.  Writing to the framebuffer, saving stats to a database, etc.  You need custom bindings for all of it.  That's not the case with established languages.

 

For example, D2K's console and HUD elements are all scripted.  They're full color, UTF-8 ready, accepts font files like TTF and OTF, and it uses Pango for layout so it will even render things like Arabic and Chinese properly.  I used Lua's existing bindings for all of this.  If I'd gone with ACS, I'd have had to write those bindings all myself.  Remember those speed bumps I've been talking about?

 

19 minutes ago, Ladna said:

INI can easily deal with everything you listed in your example, and most of the time it's more concise.  I posted a full INI example of an imp from EE's EDF.  If you have specific opposition to it, I'd love to hear about it.

 

Not helpful because you still need to parse the states block. It's the same problem as with EDF and will require a secondary parser to completely process it.

It'd also be a wholehearted "fuck you" to all tool developers who have worked with the existing formats. Both DoomBuilder and Slade can work with DECORATE, I'm not sure about EDF, but this would cause more work to them again. What for? So that you can get an INI based format?

I'm sorry but it's a given that this is not going to work out so why should I even think about it?

Seeing the whole picture, from editors to existing implementation to what the prospective users have done in the past it'd only serve to hold up an empty technocratic ideal.

 

27 minutes ago, Ladna said:

I didn't mean to specifically make this about Lua, it's just a common game scripting language.  I think JavaScript is maybe a better example, but whatever.  The point remains that ACS is a niche language, and after you learn it you can't really do anything other than mod Doom.  Even inside Doom you need a lot of special support.  Writing to the framebuffer, saving stats to a database, etc.  You need custom bindings for all of it.  That's not the case with established languages.

Guess what: ACS was made to script simple level actions, nothing more. That's still where it stands and where it will be 10 years down the road.

29 minutes ago, Ladna said:

 Even inside Doom you need a lot of special support.  Writing to the framebuffer, saving stats to a database, etc.  You need custom bindings for all of it.  That's not the case with established languages.

But these are outside of ACS's scope. They are even mostly outside the scope of what should be scriptable. I have been extremely careful what to expose to ZScript in ZDoom, and direct access to the file system or the video hardware is out, there needs to be some abstraction to allow refactoring of the bottom level subsystems (e.g. a Vulkan based renderer is definitely planned for the future.)

 

And in case you ask, why developing our own language? The simple answer is that none of the stock languages was able to do the most important thing of how content definition works in ZDoom: subclassing native classes. I would have loved to use Angelscript but as it stands it is simply not possible to create a scripted subclass of a native class, that works on the native side. You have to jump through hoops to get a connection, but then it's only be an indirect one, which made the whole thing worthless, refactoring ZDoom to support such a thing would have made the code a mess.´-not to mention forcing a rewrite of epic proportions well beyond writing the new compiler.

Concerning TrueType support, why would that even need direct scripting access? That should be buried deep in the text outputting system with the user not having to bother what format a font is in. If that ever gets added to ZDoom it should have no impact at all on the scripting language - once done, calling SetFont with a TrueType font should be all the difference on the user side.

 

I can't contribute much to this thread as I'm not a developer, but from the point of view of a lowly mapper, I will just point out: the different examples of coding on this page all look to me precisely as complicated as each other.  At least with DECORATE I know there are examples and tutorials I can copy and paste to get working.  

 

Edited May 3, 2017 by Bauul

States: all these tools already have a DECORATE state parser.  Why is it OK to ask me to write one, and not OK to ask Gez and CodeImp to use an INI library (which I'm pretty sure C# includes in the stdlib anyway)?  Again I don't care about the work, but your argument is that it's work for them.  Well the other way is work for me, and every other port that doesn't currently support DECORATE, so it feels like you're playing favorites here.

 

I'm not holding up an empty technocratic ideal.  I've listed many reasons why I support INI.  Stuff like this isn't constructive.

 

Re: extending classes.  I read a lot that generating C++ bindings is hard for scripting languages.  I don't know why exactly, but it's really easy in C.  It's maybe a little tedious but you want that level of control anyway and most of it you could probably preprocessor away (but, probably, you shouldn't).  I've had absolutely no problem making Lua bindings, and I think the process is even easier in JS (but the good JS engines with the good bindings to libraries are all C++ though, otherwise I would've picked it instead).  I don't know what state SpiderMonkey was in when ZDoom was language shopping, but whatever.

 

Re: fonts, that's how it works.  You just give a typeface to an element and it renders with it.  All the font support is in native code (fontconfig, FreeType, etc.).  But without Lua bindings for that stuff (rendering happens via Pango and Cairo, so bindings for that specifically) I'd have to make them myself.  That's no fun.

As a non-programmer, I found @Graf Zahl's zscript example the easiest to follow. It clearly lays out exactly what everything is and what it does. As far as actor and thing definitions, that format (or a very similar one) would be my preference as a mapper/modder.

17 minutes ago, Ladna said:

States: all these tools already have a DECORATE state parser.  Why is it OK to ask me to write one, and not OK to ask Gez and CodeImp to use an INI library (which I'm pretty sure C# includes in the stdlib anyway)?  Again I don't care about the work, but your argument is that it's work for them.  Well the other way is work for me, and every other port that doesn't currently support DECORATE, so it feels like you're playing favorites here.

 

Like I said: I will not even consider a completely new format just for the sake of being able to use a specific standard parser. It won't improve anything, except that where there are now 5 formats, in the future there will be 6. It has absolutely no advantage. In addition, I expect more from a parser than being able to parse entire lines. A non-tokenizing parser is a dead end in my opinion, but that's precisely what INI is, you have to do all the format conversion yourself and the more complex the possible data gets the more work needs to be on the side that drives the actual parser.

 

However, with a proper tokenizing parser that knows the difference between identifiers, string literals, integers and floating point numbers your own code will look a LOT cleaner than if you were to back your code with a simplistic INI parser and then have to break down the parsed values again with another parser.

 

 

17 minutes ago, Ladna said:

Re: extending classes.  I read a lot that generating C++ bindings is hard for scripting languages.  I don't know why exactly, but it's really easy in C.  It's maybe a little tedious but you want that level of control anyway and most of it you could probably preprocessor away (but, probably, you shouldn't).  I've had absolutely no problem making Lua bindings, and I think the process is even easier in JS (but the good JS engines with the good bindings to libraries are all C++ though, otherwise I would've picked it instead).  I don't know what state SpiderMonkey was in when ZDoom was language shopping, but whatever.

How can this be easier in C if C doesn't even have class hierarchies? Of course it is possible to do a certain kind of extension with most scripting languages, but not on the level that ZDoom needed.

 

A quick example about AngelScript which shows where the actual problem lies:

 

Want to inherit from Actor: Write an interface and equip Actor with some scripting glue.

Create native class Inventory that inherits from Actor.

Want to inherit from Inventory: Write an interface and equip Inventory with some scripting glue.

Create native class Weapon that inherits from Inventory.

Want to inherit from Weapon: Write an interface and equip Weapon with some scripting glue.

 

So far, so bad. But the final insult is that on the scripting side these classes would be unrelated and incompatible. Crap, can't use that. How am I supposed to assign a weapon to an inventory variable and call the proper virtual functions if the classes are considered unrelated?

What ZDoom needs is to inherit the entire native hierarchy of its backing class, for a weapon that'd mean, the script class actually *is* an Inventory and *is* and Actor and *is* a Thinker in any context it gets used in. No C can help you here.

 

 

17 minutes ago, Ladna said:

 

Re: fonts, that's how it works.  You just give a typeface to an element and it renders with it.  All the font support is in native code (fontconfig, FreeType, etc.).  But without Lua bindings for that stuff (rendering happens via Pango and Cairo, so bindings for that specifically) I'd have to make them myself.  That's no fun.

 

Render to where? Don't forget that in ZDoom there's currently 4 backends to render to (OpenGL full 3D, OpenGL 2D for SWRender, D3D for SWRender and software canvas with Vulkan a prospective fifth render backend) Any use of such libraries too close to the surface would cause so many problems that it cannot be done without blocking future changes, they will have to be heavily abstracted that all the backends can do what is necessary for their specific needs.

 

13 hours ago, Edward850 said:

@kb1, before you start talking about how things are correctly done in your mega-cool-and-whatnot source port, make sure you have something to show first (screenshots, source code, release downloads, etc). Keep in mind we've already seen thousands of people claiming to know how to do everything correctly in source code that never-ever have anything to show for it.

 

@Edward850, before you start trying to act mega-cool-and-whatnot by using the newproject tag, make sure you check both that the person doesn't have something to show first (posted code on DW, previous work, etc) and that the situation calls for it. Keep in mind we've already seen thousands of uses of the newproject tag that only succeeded in making the person using it look like a complete moron, because of incorrect use.

 

'A new standard' isn't going to just pop up all of a sudden with screenshots, source code and release downloads; people are going to have to talk about it.  If any of us wants this to be a thing we should at least not present ourselves publicly as "rolling our eyes, uncommittedly making cheeky comments in our non-support." Not saying we should all jump in and blindly follow a paper Jesus, but we should be supportive in at least a small, inconsequential (to our own time), and public way to encourage something like this to eventually come to fruition, even if this particular attempt is thrown to the wayside by snarky comments and a seeming lack of support (not saying that's how it appears to be heading, just saying that's how projects like these wind up without some form of backing).

 

It'd be cool if something like this was made, right? Treat it as such while keeping your skepticism in mind: "Hey that'd be cool, but I just dont have time to help. Here are my (free) thoughts tho..." That's what's being asked for now, not any real work or time. There is no reason for the newproject tag here, as nothing real is being asked of anyone yet. Don't shut down discussion; the transfer of ideas is a priceless thing.

I was just reading through all this lengthy stuff and I have to wonder: Why this heated discussion? I'm sorry, but I do not get what Ladna is up here.

To me his counterexample for an INI-based DECORATE replacement looks like a perfect example of redundancy for the sake of doing it differently.

 

It's not even proper INI because with INI I would assume that the value is just that - a value being stored somewhere, but here it ends up as some block of code that needs to be parsed again. Which is ironic because he's been rambling on about standard parsers endlessly but to crunch this stuff into engine-digestable data it needs to be processed by - guess what - a non-standard parser geared at parsing this particular chunk of data again. So where's the gain here?

I have seen great parsing solutions that can be given a text and they return a stream of tokens - here's an identifier, next is a number, a string literal, a comma or whatever else needs to be isolated. These parsers are ridiculously easy to use, they help write short and concise code that gets directly to the point of the matter and allow to create data format that help present the data in the most adequate form possible. INI is the total antithesis of such an approach, here the formality of the defintion takes center stage above everything, even at the risk of misrepresenting the data or not being able to express it adequately.

And what's so much better about first parsing the key/value pair and then cracking the values into several subvalues with such a tokenizing parser anyway? Why not define a format that uses the tokenizing parser from top to bottom? But wait - isn't that what DECORATE is, assuming people do not abuse the shenanigans that are allowed by Hexen's script parser?

 

Here's another example, this time using Graf's ZScript stuff:

[thing.ZombieMan]
Health = 20
Radius = 20
Height = 20
Speed = 8
PainChance = 200
Type = Monster
Flags = FloorClip
Sounds.See = grunt/sight
Sounds.Attack = grunt/attack
Sounds.Pain = grunt/pain
Sounds.Death = grunt/death
Sounds.Active = grunt/active
Obituary.Normal = $OB_ZOMBIE
DropItem = Clip
States.Spawn = POSS AB 10 A_Look
               Loop
States.See = POSS AABBCCDD 4 A_Chase
States.Missile = POSS E 10 A_FaceTarget
                 POSS F 8 A_PosAttack
                 POSS E 8
                 Goto See
States.Pain = POSS G 3
              POSS G 3 A_Pain
              Goto See
States.Death = POSS H 5
               POSS I 5 A_Scream
               POSS J 5 A_NoBlocking
               POSS K 5
               POSS L -1
               Stop
States.XDeath = POSS M 5
                POSS N 5 A_XScream
                POSS O 5 A_NoBlocking
                POSS PQRST 5
                POSS U -1
                Stop
States.Raise = POSS K 5
               POSS JIH 5
               Goto See

So clear!  So obvious!  So simple!

 

Here's the Python code necessary to parse the entire thing:

#!/usr/bin/env python

import sys
import configparser

def parse_state_line(line):
    tokens = line.strip().split()
    if len(tokens) == 1 and tokens[0] == 'Loop':
        return {'keyword': 'loop'}
    if len(tokens) == 1 and tokens[0] == 'Stop':
        return {'keyword': 'loop'}
    if len(tokens) == 2 and tokens[0] == 'Goto':
        return {'goto': tokens[1]}
    if len(tokens) == 3:
        return {
            'sprite': tokens[0],
            'frames': tokens[1],
            'rate': int(tokens[2]),
            'bright': False,
            'code_pointer': None
        }
    if len(tokens) == 4 and tokens[3] == 'bright':
        return {
            'sprite': tokens[0],
            'frames': tokens[1],
            'rate': int(tokens[2]),
            'bright': True,
            'code_pointer': None
        }
    if len(tokens) == 4:
        return {
            'sprite': tokens[0],
            'frames': tokens[1],
            'rate': int(tokens[2]),
            'bright': False,
            'code_pointer': tokens[3]
        }
    if len(tokens) == 5 and tokens[3] == 'bright':
        return {
            'sprite': tokens[0],
            'frames': tokens[1],
            'rate': int(tokens[2]),
            'bright': True,
            'code_pointer': tokens[3]
        }
    raise Exception('Invalid state line')

def main():
    file_name = sys.argv[1]
    ini = configparser.ConfigParser()
    ini.optionxform = lambda opt: opt
    with open(file_name, 'r', encoding='utf-8') as fobj:
        ini.read_string(fobj.read(), source=file_name)
    for section in ini.sections():
        if section.startswith('thing.'):
            print(':::: %s ::::' % (section[6:]))
            for name, value in ini.items(section):
                if name.startswith('States.'):
                    for n, state_line in enumerate(value.splitlines()):
                        print('    %sState%d = %s' % (
                            name[7:],
                            n + 1,
                            parse_state_line(state_line)
                        ))
                else:
                    print('    %s = %s' % (name, value))

main()

 

Note there's about 40 lines for the DECORATE state parser, and like 4 lines for the INI parser stuff because Python has built-in support.

 

BUT STILL, 40 lines is nothing to complain about.  I simply don't accept "LADNA!!! YOU'LL HAVE TO PARSE THE DECORATE STATES IN A SECOND PAAAAASSSSSSSS BAWWWWWWW" as an excuse.

2 hours ago, Fonze said:

@Edward850, before you start trying to act mega-cool-and-whatnot by using the newproject tag, make sure you check both that the person doesn't have something to show first (posted code on DW, previous work, etc) and that the situation calls for it. Keep in mind we've already seen thousands of uses of the newproject tag that only succeeded in making the person using it look like a complete moron, because of incorrect use.

We already know he has nothing to show for it. He posted one set of code for the long wall error that didn't even get used, and has since talked about his mega-awesome source he has so far had nothing to show for it, for about 5 years now? Kb1 has actually produced less to the source port scene than wesleyjohnson.

22 minutes ago, Ladna said:

Here's another example, this time using Graf's ZScript stuff:

[...]

So clear!  So obvious!  So simple!

Err, no. What you have there reminds me of the headaches I get when I'm forced to look at a Windows driver .inf file.

8 hours ago, Ladna said:

Sure that's the case whenever you don't know something, but the difference between something DECORATE and INI is that there's 832,902,382,340,929 examples of INI files out there, not so much when it comes to DECORATE.

One small counterpoint to this statement I thought it's worth making.

 

I googled "How to create new Doom monster in Decorate" and got a literal whole page of examples and tutorials.

 

I googled "How to create new Doom monster in INI" and got nothing of any use.  

 

While there might be "832,902,382,340,929" examples of INI out there, if they aren't written specifically for Doom editing, they are pretty much totally useless to the average Doom mapper.

There's not really anything to learn.  There are a few attributes like health and radius, and the states, which you look at examples for anyway.  That's the cool thing about such a simple format, there's nothing tricky about it, so you don't need ZDoom's giant (cool, though) wiki just to tweak a monster.

9 minutes ago, Ladna said:

Here's another example, this time using Graf's ZScript stuff:

So clear!  So obvious!  So simple!

 

 

So shortsighted. The problem that's very apparent here is that this minimalistic parser is just that: Minimalistic. It contains no semantic analysis and is horrendously limited by what it can do. The DECORATE state parser, stripped of all non-basic functions is 120 lines of code, but it's complete (that is, without the function parameter parsing, which you did not even consider an option!)

It also has no code size advantage over using a real tokenizer and a freely defined format that is tailored to the job at hand. And please don't tell me that such tokenizers do not exist for Python. Even if not, how much code does such a tokenizer require? The simplest one I found in Doom related tools is 11kb of C++. Not really a deal breaker when it allows to be far more liberal in defining how a definition lump should look.

2 minutes ago, Ladna said:

There's not really anything to learn.  There are a few attributes like health and radius, and the states, which you look at examples for anyway.  That's the cool thing about such a simple format, there's nothing tricky about it, so you don't need ZDoom's giant (cool, though) wiki just to tweak a monster.

You do not need ZDoom's "giant Wiki" to learn how to write DECORATE syntax but to understand how the stuff being defined in DECORATE interacts with the engine. Regardless of whether you use EDF, DECORATE, DDF, LadnaINI or whatever, people first need to know what properties an actor can have, what they can so, what values they can take, or for states, what stock labels exist, when they get called, what action function can be used where and what it does and so on and so on.

 

Getting the actor parsed is only the very first tiny step in understanding what this is about, it doesn't need a large Wiki, that stuff is explained on one short page that summarizes the basic building blocks of an actor. You need all the same for an INI based format as well (i.e. here's a property, it's defined "name = value", here's a flag, it's defined "whatever", here's a chain of states, it needs a sprite name, an animation frame, a duration in tics, and optionally the 'bright' keyword and a code pointer that may or may not have some parameters.)

 

In other words: The documentation for your format would be exactly the same in terms of complexity, it's just a different syntax, that ultimately needs the same amount of documentation to get right. The 4 added braces and the 'States' keyword are hardly complex enough to complicate the documentation, and these are ultimately the only elements your idea does not have.

Basically the whole thing about reskinning (or, reskINIng ahahah) DECORATE syntax to turn it into a big wall of text sounds like a lot of work reinventing the wheel for honestly not much of a gain. Are we supposed to see a sudden explosion in tools and mods because the lack of INI syntax was holding at bay hordes of eager contributors?

 

For the modder: I don't see how INI presentation is simpler. It's just a basic key-value pair. Really the biggest difference is about whether the states are enclosed in a curly brace block or not. I suppose also the INI format means that this below would become legit. Which I'm not sure whether it's a good idea, really.

[thing.ZombieMan]
Health = 20
States.Spawn = POSS AB 10 A_Look
               Loop
States.Death = POSS H 5
               POSS I 5 A_Scream
               POSS J 5 A_NoBlocking
               POSS K 5
               POSS L -1
               Stop
Radius = 20
Height = 20
Speed = 8
PainChance = 200
States.Pain = POSS G 3
              POSS G 3 A_Pain
              Goto See
Type = Monster
Flags = FloorClip
States.Raise = POSS K 5
               POSS JIH 5
               Goto See
Sounds.See = grunt/sight
States.See = POSS AABBCCDD 4 A_Chase
Sounds.Attack = grunt/attack
States.Missile = POSS E 10 A_FaceTarget
                 POSS F 8 A_PosAttack
                 POSS E 8
                 Goto See
Sounds.Pain = grunt/pain
Sounds.Death = grunt/death
States.XDeath = POSS M 5
                POSS N 5 A_XScream
                POSS O 5 A_NoBlocking
                POSS PQRST 5
                POSS U -1
                Stop
Sounds.Active = grunt/active
Obituary.Normal = $OB_ZOMBIE
DropItem = Clip

Personally, I find it clearer when the state table and the property blocks are clearly separate; which they aren't in your proposal. No, prefixing "states." isn't enough to make things clearly separate. It does make things more boilerplatey.

 

Now as long as we're having to go through an additional parser anyway, I could be amiable to this:

States =
    {
    Spawn:
        POSS AB 10 A_Look;
        Loop;
    See:
        POSS AABBCCDD 4 A_Chase;
        Loop;
    Missile:
        POSS E 10 A_FaceTarget;
        POSS F 8 A_PosAttack;
        POSS E 8;
        Goto See;
    Pain:
        POSS G 3;
        POSS G 3 A_Pain;
        Goto See;
    Death:
        POSS H 5;
        POSS I 5 A_Scream;
        POSS J 5 A_NoBlocking;
        POSS K 5;
        POSS L -1;
        Stop;
    XDeath:
        POSS M 5;
        POSS N 5 A_XScream;
        POSS O 5 A_NoBlocking;
        POSS PQRST 5;
        POSS U -1;
        Stop;
    Raise:
        POSS K 5;
        POSS JIH 5;
        Goto See;
    }

Hey it's still a valid multiline key = value pair, isn't it? (semicolons are optional)

 

 

For the programmer, it's also quite simple. If you want to support UDMF, you'll have to support parsing of curly brace blocks. And in any case, you'll have to have parsing for DECORATE state syntax. Mix those two requirements together and you have no obstacle to parsing a simplified, streamlined form of DECORATE.

 

Personally if I don't need to write a new parser for SLADE 3 I'll be happy. And speaking of which, I'd like to invite you to look at the content of the "INI" (well, .cfg) files of Doom Builder or SLADE... Yep! Curly blocks! Nested curly blocks, even -- because that allows hierarchical groupings, something which the INI syntax cannot; it's a bit besides the point but it's just to show that this approach would have been taken even without UDMF.

2 hours ago, dpJudas said:

Err, no. What you have there reminds me of the headaches I get when I'm forced to look at a Windows driver .inf file.

See, I couldn't have asked for a better example of "everyone's worked with an INI file".  Thank you, dpJudas.

 

1 hour ago, Graf Zahl said:

It contains no semantic analysis and is horrendously limited by what it can do. The DECORATE state parser, stripped of all non-basic functions is 120 lines of code, but it's complete (that is, without the function parameter parsing, which you did not even consider an option!)

Ehhhh, the function to parse state defs is like 300 lines, and it leaves a lot of its "semantic analysis" to some other module somewhere.  This argument is pretty disingenuous.

 

Plus you were arguing that parsing the state string in an INI is super difficult, even if you already have a state parser.  I just wrote the small parser to prove that wasn't true, and even your own code in thingdef_states.cpp shows you can just hook it into whatever "semantic analysis" you want to run, because that's what your code does.

 

I deliberately left out function parameter parsing for 3 reasons:

• they've been in exactly zero examples, and I'm tired of moving goalposts
• Quasar even left them out of his grammar
• I think they're a bad idea

1 hour ago, Graf Zahl said:

It also has no code size advantage over using a real tokenizer and a freely defined format that is tailored to the job at hand.

It definitely does have a code size advantage.  I parsed an INI file, iterated over its sections and name/value pairs in like 4-5 lines of code.  You could maybe write a tokenizer that amount of space, but it'd be in like APL or something hellacious.  The smallest INI library I could find was 305 lines, and that's a ridiculously easy format.  But anyway, such is the advantage of libraries.

 

1 hour ago, Graf Zahl said:

Even if not, how much code does such a tokenizer require? The simplest one I found in Doom related tools is 11kb of C++. Not really a deal breaker when it allows to be far more liberal in defining how a definition lump should look.

 

1 hour ago, Graf Zahl said:

The 4 added braces and the 'States' keyword are hardly complex enough to complicate the documentation, and these are ultimately the only elements your idea does not have.

You gotta decide: either the format is specifically defined for the task at hand (thing defs) and worth 11kb of additional source code, or the only difference between INI and your "highly specialized" format is 4 added braces and the 'States' keyword.

 

EDIT: Ooooooh Gez, I like it.  +1

This discussion is ridiculous. If the options being presented are really "either use decorate or something else with decorate states bolted onto it", which this seems to be, just use decorate and save everyone the trouble.

Also there's been some mumblings about separating states from thing defs, so you could do:

 

[Thing.ZombieMan]
Health = 20
Radius = 20
Height = 20
Speed = 8
PainChance = 200
Type = Monster
Flags = FloorClip
Sounds.See = grunt/sight
Sounds.Attack = grunt/attack
Sounds.Pain = grunt/pain
Sounds.Death = grunt/death
Sounds.Active = grunt/active
Obituary.Normal = $OB_ZOMBIE
DropItem = Clip

[Thing.ZombieMan.States]
Spawn = POSS AB 10 A_Look;
        Loop;
See = POSS AABBCCDD 4 A_Chase;
      Loop;
Missile = POSS E 10 A_FaceTarget;
    	  POSS F 8 A_PosAttack;
    	  POSS E 8;
    	  Goto See;
Pain = POSS G 3;
       POSS G 3 A_Pain;
       Goto See;
Death = POSS H 5;
        POSS I 5 A_Scream;
        POSS J 5 A_NoBlocking;
        POSS K 5;
        POSS L -1;
        Stop;
XDeath = POSS M 5;
         POSS N 5 A_XScream;
         POSS O 5 A_NoBlocking;
         POSS PQRST 5;
         POSS U -1;
         Stop;
Raise = POSS K 5;
        POSS JIH 5;
        Goto See;

And I mean, if Sounds. or Obituary. is getting to you, you can do the same thing with them.

 

Essel:  The difference between DECORATE or not DECORATE is external library support.  I agree this borders on the insane, but I guess this is the way the process is.

9 minutes ago, Ladna said:

Ehhhh, the function to parse state defs is like 300 lines, and it leaves a lot of its "semantic analysis" to some other module somewhere.  This argument is pretty disingenuous.

 

Yes, but if you look closer you'll find out that the majority of this is features your code cannot handle at all. Strip that out to the feature set we need for a base feature and we'll end up at 120 lines, strip out the error checking and move the opening braces to the previous line like you did and we're almost at your 40 lines.

 

 

9 minutes ago, Ladna said:

 

Plus you were arguing that parsing the state string in an INI is super difficult, even if you already have a state parser.  I just wrote the small parser to prove that wasn't true, and even your own code in thingdef_states.cpp shows you can just hook it into whatever "semantic analysis" you want to run, because that's what your code does.

 

I didn't say 'super difficult', I said, 'requires a secondary parser'. The contents of the state blocks are not simply space separated words, so such a simplistic approach is not going to work, you need a real tokenizer to parse function calls with actual parameters. Which brings us back to the point where I question the whole setup. Since you need that tokenizer anyway, you can also use it to parse the outer definition and turn it into something nicer to look at.

 

 

9 minutes ago, Ladna said:

 

I deliberately left out function parameter parsing for 3 reasons:

• they've been in exactly zero examples, and I'm tired of moving goalposts
• Quasar even left them out of his grammar
• I think they're a bad idea

 

So you think that what made DECORATE such a success (i.e. complex functions that can be configured by the user) is a bad idea? Well, that actually figures.

 

 

 

9 minutes ago, Ladna said:

It definitely does have a code size advantage.  I parsed an INI file, iterated over its sections and name/value pairs in like 4-5 lines of code.  You could maybe write a tokenizer that amount of space, but it'd be in like APL or something hellacious.  The smallest INI library I could find was 305 lines, and that's a ridiculously easy format.  But anyway, such is the advantage of libraries.

 

The size of the parsing library should not be the concern here. What should be a concern is how flexible it is. Most awful formats are created because the actual parsing tools were shit. Same here. INI is a crappy format for anything but the simplest stuff, putting that on a pedestal as the ideal solution for generic parsing is just dubious in my eyes. Not much good can come out of it, unless the stuff you want to express is as simple as the format.

 

 

9 minutes ago, Ladna said:

 

You gotta decide: either the format is specifically defined for the task at hand (thing defs) and worth 11kb of additional source code, or the only difference between INI and your "highly specialized" format is 4 added braces and the 'States' keyword.

 

EDIT: Ooooooh Gez, I like it.  +1

 

You need the 11kb anyway to not let the format degenerate into a syntactic clusterfuck so whatever. Use INI for something that's too complex for the format's simplicity and you end up with both the 11 kb and the INI parser. Also, the 11kb for the parser can be repurposed to make other formats look nicer as well.

So I guess the conversation now devolved into:

 

Someone: Let me provide an example of how INI isn't really helping with anything.

 

Ladna: Ohhh, thank you for providing a perfect example of how awesome my INI idea is, you did my job for me!!

 

Edited May 4, 2017 by Da Werecat

Hey hey! Can you New Standard makers make it so that I can tell from the status bar whether I have the chainsaw/berserk/ssg? That would be really helpful, thanks.

 

Second topic, time permitting: this is project is probably doomed.

Hey, you paste a DECORATE example on a non-DW gamedev forum, people will be all "WTF is that".  You paste an INI example, everyone will know.  That's all I've been saying :)

2 minutes ago, Ladna said:

Hey, you paste a DECORATE example on a non-DW gamedev forum, people will be all "WTF is that".  You paste an INI example, everyone will know.  That's all I've been saying :)

 

Post a DECORATE example on a non-Doom forum, people who do not know Doom will ask "What is this?"

Post a LadnaINI version of the example on a non-Doom forum, people who do not know Doom may ask "Ok, this is an INI but what does it mean?"

 

You are presuming that people are idiots and fail to understand a file's content just because the syntax may be unfamiliar.

But it's not the syntax that makes it difficult for them to understand but the content.

And that's the same, regardless of format. Without a context in which to understand the data, it's both all Greek to them and the INI syntax won't make one bit of a difference, because the entire difference between both is mere boilerplate.

 

To be honest, if they can't understand Decorate, I don't think your slaughtered-Decorate-in-INI would fare much better, aside from them being to tell it's an INI. Maybe.

 

Frankly the way you're doing states makes me question the last bit.

13 minutes ago, Graf Zahl said:

Yes, but if you look closer you'll find out that the majority of this is features your code cannot handle at all.

What are those features?  I seriously don't see anything that isn't handled by an external module somewhere else.  Even lookups for sprites and what-not are external.

 

15 minutes ago, Graf Zahl said:

you need a real tokenizer to parse function calls with actual parameters

You don't really, just say everything after the 4th token or 'bright' is a function call

 

SeeState = TROO AB 8 bright A_EatDirt "happily" 666

 

16 minutes ago, Graf Zahl said:

So you think that what made DECORATE such a success (i.e. complex functions that can be configured by the user) is a bad idea? Well, that actually figures.

I super hate complexity, and putting something that borders on expression evaluation in a declarative format is complex.  If you want a scripting language for declaring things, I know of a few ;)  But if you want a data declaration format, parameterized functions are... probably out of scope.

 

Plus, what about standards?  Are we saying all ports have to have a certain subset of code pointers all with the same signature and behavior?  That's pretty wide-ranging.  There's, I'm sure, a good deal of overlap already, but I feel a linedef-collision-type problem already brewing.

19 minutes ago, Graf Zahl said:

putting that on a pedestal as the ideal solution for generic parsing

I'm so tired of this straw man.  Thing definitions are pretty much just key/value pairs.  INI is good for key/value pairs.  Bladow.  Where it isn't good, I've put forward multiple solutions that honestly compromise a lot of what I want, despite numerous accusations to the contrary from you and others.  I've been super flexible.  I haven't seen you compromise on this even once.

 

21 minutes ago, Graf Zahl said:

You need the 11kb anyway to not let the format degenerate into a syntactic clusterfuck so whatever. Use INI for something that's too complex for the format's simplicity and you end up with both the 11 kb and the INI parser. Also, the 11kb for the parser can be repurposed to make other formats look nicer as well.

I built one in literally 15 minutes and this didn't happen.  So don't worry!

4 minutes ago, Graf Zahl said:

 

Post a DECORATE example on a non-Doom forum, people who do not know Doom will ask "What is this?"

Post a LadnaINI version of the example on a non-Doom forum, people who do not know Doom may ask "Ok, this is an INI but what does it mean?"

 

You are presuming that people are idiots and fail to understand a file's content just because the syntax may be unfamiliar.

But it's not the syntax that makes it difficult for them to understand but the content.

And that's the same, regardless of format. Without a context in which to understand the data, it's both all Greek to them and the INI syntax won't make one bit of a difference, because the entire difference between both is mere boilerplate.

 

Yeah sure but that's true of DECORATE too.  The difference is I can parse the format with an INI parser.  So ceteris paribus INI wins.