Defining a new enhanced source port standard (Boom+/MBF+)

[Graf Zahl]

Just now, Ladna said:

What are those features?  I seriously don't see anything that isn't handled by an external module somewhere else.  Even lookups for sprites and what-not are external.

 

Several more flags that can be put on a state, the offset, light and random keywords that take parameters, and of course code for each one of those to actually handle the feature. Just parsing the flag is not the issue, you also need code to actually do something with the knowledge that the flag exists, and all that is part of those 300+ lines. Your code was the mere parser, so all that would have to be added on top of that as well.

 

Just now, Ladna said:

 

You don't really, just say everything after the 4th token or 'bright' is a function call

 

SeeState = TROO AB 8 bright A_EatDirt "happily" 666

 

Yeah. How about

 

SeeState = TROO AB 8 bright A_EatShit "nonsense" random 4 6 -2

 

The lack of structural elements makes this one ambiguous. Does this mean 'random(4,6)-2', 'random(4,6-2)' or 'random(4,6,-2)'. Or even 'random(4), 6-2' or  'random(4), 6, -2'. Or '..., random, 4, 6, -2'

Hard to tell. And impossible to parse.

And a textbook example why I have a strong dislike for such unstructured formats where everything is valid. Funny that you argued against DECORATE because it is not clearly defined and yet here you commit the same crime yourself. Space separated formats are garbage and I'd eliminate every one that's still present if it wouldnt break all mods out there.

 

 

Just now, Ladna said:

I super hate complexity, and putting something that borders on expression evaluation in a declarative format is complex.  If you want a scripting language for declaring things, I know of a few ;)  But if you want a data declaration format, parameterized functions are... probably out of scope.

 

Yes, it is complex, but it is what made DECORATE such a powerful tool. It's not merely a declarative format meant to handle simple static data, it quickly turned into a scripting tool that went far beyond the simplistic origins.

 

 

Just now, Ladna said:

 

Plus, what about standards?  Are we saying all ports have to have a certain subset of code pointers all with the same signature and behavior?  That's pretty wide-ranging.  There's, I'm sure, a good deal of overlap already, but I feel a linedef-collision-type problem already brewing.

I'm so tired of this straw man.  Thing definitions are pretty much just key/value pairs.  INI is good for key/value pairs.  Bladow.  Where it isn't good, I've put forward multiple solutions that honestly compromise a lot of what I want, despite numerous accusations to the contrary from you and others.  I've been super flexible.  I haven't seen you compromise on this even once.

 

A minimal set of code pointers to do commonly requested stuff needs to be part of the base, otherwise it's useless. That includes a custom missile firing function, a custom hitscan function and a parameterized A_Explode as the barest minimum.

 

 

 

[Graf Zahl]

13 minutes ago, Ladna said:

Yeah sure but that's true of DECORATE too.  The difference is I can parse the format with an INI parser.  So ceteris paribus INI wins.

 

Yes, you can PARSE it with your INI parser - but only half way. And what then? Instead of a stream of tokens you have a stream of key/value pairs, so the remaining work is roughly the same. Or not. Because your values are not simple values, they are LISTS of values if a property needs more than one, requiring a second pass over the value. Ugh, should have thought of that one first and invested time in a better outer parser.

 

 

[Ladna]

8 minutes ago, Graf Zahl said:

Several more flags that can be put on a state, the offset, light and random keywords that take parameters, and of course code for each one of those to actually handle the feature.

"RANDOM" is a good case actually.  The code for RANDOM doesn't actually implement the feature, it just sets parameters on a state (and swaps params if the user goofs... which is questionable but I get it).

 

10 minutes ago, Graf Zahl said:

Space separated formats are garbage and I'd eliminate every one that's still present if it wouldnt break all mods out there.

This is probably our disagreement.  I disagree I guess, but I don't really have strong feelings.  I still think it's good for this use case?  You can do:

SeeState = TROO AB 8 bright A_TurnThatFrownUpsideDown "OhYeah" random 6 -2

And because you know the arity of `random` is 2 (and the arguments are integers) you only parse the next 2 integers as arguments.  In fact that's exactly what your code does.  If you eliminated the checks for "(", "," and ")" you're just parsing a space-delimited format.  You're not parsing an expression because there's no operators and no precedence (like no shunting yard or top-down operator precedence parsing).

 

And since you're not using operators or precedence or evaluating expressions, you don't have to worry about things like "did they mean 6, -2 or 6 - 2?".  They couldn't have meant 6 - 2, but that should be a parse error anyway because "-" isn't an integer.

 

All of which is to say you seriously just don't need a recursive-descent parser to parse this stuff, and I know that because you yourself don't have one.  Space-delimiting is fine.  We don't need a tokenizer.

 

19 minutes ago, Graf Zahl said:

Yes, it is complex, but it is what made DECORATE such a powerful tool. It's not merely a declarative format meant to handle simple static data, it quickly turned into a scripting tool that went far beyond the simplistic origins.

This DEEPLY worries me.  Putting some kind of evaluation engine into the standard is way, way out of scope.  We absolutely shouldn't be considering this.

 

And I'll say why!

 

If you think there's no agreement on thing definitions, wait until you start discussions on scripting languages.  I can imagine it now, "just use DECORATE/ACS; it's what everyone already knows, it's tailored specifically for the use-case; too bad if you don't already have an implementation; you can use our implementation; we'll make a reference implementation; we'll write one for you; why do you have to be so obstinate; we don't in any way care about what everyone else does; don't write your own you'll just make it so your implementation's quirks will cause incompatible mods".

 

22 minutes ago, Graf Zahl said:

Yes, you can PARSE it with your INI parser - but only half way. And what then? Instead of a stream of tokens you have a stream of key/value pairs, so the remaining work is roughly the same. Or not. Because your values are not simple values, they are LISTS of values if a property needs more than one, requiring a second pass over the value.

If you look at what the parser outputs, you'll see everything is parsed out:

 

:::: ZombieMan ::::
    Health = 20
    Radius = 20
    Height = 20
    Speed = 8
    PainChance = 200
    Type = Monster
    Flags = FloorClip
    Sounds.See = grunt/sight
    Sounds.Attack = grunt/attack
    Sounds.Pain = grunt/pain
    Sounds.Death = grunt/death
    Sounds.Active = grunt/active
    Obituary.Normal = $OB_ZOMBIE
    DropItem = Clip
    SpawnState1 = {'sprite': 'POSS', 'frames': 'AB', 'rate': 10, 'bright': False, 'code_pointer': 'A_Look'}
    SpawnState2 = {'keyword': 'loop'}
    SeeState1 = {'sprite': 'POSS', 'frames': 'AABBCCDD', 'rate': 4, 'bright': False, 'code_pointer': 'A_Chase'}
    MissileState1 = {'sprite': 'POSS', 'frames': 'E', 'rate': 10, 'bright': False, 'code_pointer': 'A_FaceTarget'}
    MissileState2 = {'sprite': 'POSS', 'frames': 'F', 'rate': 8, 'bright': False, 'code_pointer': 'A_PosAttack'}
    MissileState3 = {'sprite': 'POSS', 'frames': 'E', 'rate': 8, 'bright': False, 'code_pointer': None}
    MissileState4 = {'goto': 'See'}
    PainState1 = {'sprite': 'POSS', 'frames': 'G', 'rate': 3, 'bright': False, 'code_pointer': None}
    PainState2 = {'sprite': 'POSS', 'frames': 'G', 'rate': 3, 'bright': False, 'code_pointer': 'A_Pain'}
    PainState3 = {'goto': 'See'}
    DeathState1 = {'sprite': 'POSS', 'frames': 'H', 'rate': 5, 'bright': False, 'code_pointer': None}
    DeathState2 = {'sprite': 'POSS', 'frames': 'I', 'rate': 5, 'bright': False, 'code_pointer': 'A_Scream'}
    DeathState3 = {'sprite': 'POSS', 'frames': 'J', 'rate': 5, 'bright': False, 'code_pointer': 'A_NoBlocking'}
    DeathState4 = {'sprite': 'POSS', 'frames': 'K', 'rate': 5, 'bright': False, 'code_pointer': None}
    DeathState5 = {'sprite': 'POSS', 'frames': 'L', 'rate': -1, 'bright': False, 'code_pointer': None}
    DeathState6 = {'keyword': 'loop'}
    XDeathState1 = {'sprite': 'POSS', 'frames': 'M', 'rate': 5, 'bright': False, 'code_pointer': None}
    XDeathState2 = {'sprite': 'POSS', 'frames': 'N', 'rate': 5, 'bright': False, 'code_pointer': 'A_XScream'}
    XDeathState3 = {'sprite': 'POSS', 'frames': 'O', 'rate': 5, 'bright': False, 'code_pointer': 'A_NoBlocking'}
    XDeathState4 = {'sprite': 'POSS', 'frames': 'PQRST', 'rate': 5, 'bright': False, 'code_pointer': None}
    XDeathState5 = {'sprite': 'POSS', 'frames': 'U', 'rate': -1, 'bright': False, 'code_pointer': None}
    XDeathState6 = {'keyword': 'loop'}
    RaiseState1 = {'sprite': 'POSS', 'frames': 'K', 'rate': 5, 'bright': False, 'code_pointer': None}
    RaiseState2 = {'sprite': 'POSS', 'frames': 'JIH', 'rate': 5, 'bright': False, 'code_pointer': None}
    RaiseState3 = {'goto': 'See'}

 

(this is an internal format, don't wig out)

 

See?  Everything's parsed tidily.  No lists anywhere.

[Arctangent]

For the sake of the argument ...

Spoiler

 

 

[Da Werecat]

8 minutes ago, Ladna said:

See?  Everything's parsed tidily.  No lists anywhere.

You still need to expand that into a set of states somehow. "PQRST", for example, is not one state, but five, described in a concise manner.

[Ladna]

This is a Python semantic thing.  To Python, strings are sequences of strings, so "PQRST" is "P" + "Q" + "R", and if you want the first character it's just `frames[0]` (gives you "P").  So technically, they are already split up.  You can also call `list("PQRST")` to get `["P", "Q", "R", "S", "T"]` if you so wish.  But if you're not going to mutate the thing (strings are immutable in Python) then you should just leave it as a string.

 

Now people say you shouldn't index into strings because it's based on the old ASCII concept of strings, i.e. every element of a string is a character and every character is a byte.  Unicode literals are not that way, so if you do `frames[3]` you might get a junk value -- depending upon your platform/language.  But we're not doing Unicode literals for the frames so it's fine.

 

ATAN:

 

I thought about your survey a little more :)

 

I think that's cool and pretty much what I would expect.  Some of the stuff is explanatory, "TROO AA 8 A_Bleck" is not, but I think the compromise there is that if you're doing this by hand you want a shorthand for it right?  Purist that I am, I originally thought JSON would be good for this, so you'd sort of have an array of each state, so like

 

"states": {
  "missile": [
    {"sprite": "POSS", "frames": ["E"], "rate": 4, "code_pointer": "A_FaceTarget"},
    {"sprite": "POSS", "frames": ["F"], "rate": 8, "code_pointer": "A_PosAttack"}
  ]
}

But like, even typing out that small example enraged me, and I don't honestly know what it really buys you.  "code_pointer" and "rate" are still pretty opaque., you know what sprites and frames are but like, why "E" and shit?

Edited May 4, 2017 by Ladna

[dpJudas]

4 hours ago, Ladna said:

I'm so tired of this straw man.  Thing definitions are pretty much just key/value pairs.  INI is good for key/value pairs.  Bladow.  Where it isn't good, I've put forward multiple solutions that honestly compromise a lot of what I want, despite numerous accusations to the contrary from you and others.  I've been super flexible.  I haven't seen you compromise on this even once.

 

I built one in literally 15 minutes and this didn't happen.  So don't worry!

I think you're really missing the point. The key problem with INI isn't whether you can shoehorn something into an INI file, but rather if the file doesn't turn to shit if you do.

 

My reference to driver .inf files was exactly to point out what happens when you realize your data isn't simple basic key/value pairs and start inventing your own semantics to the key and the value parts. Your tiny example did BOTH. For the key part you invented that styles are keys prefixed with a kind of namespace. For the value part you cheated by placing a complex state sequence as if it was a single string.

 

In both cases the familiarity of the original format becomes mostly useless because now everyone is scratching their heads wondering what the heck those two things means, with the advantage of the original INI format long gone. To illustrate what I mean, here's a CSS file converted to JSON:

[
  {
    "h1 > .foobar.foo .moo": [
      { "background": "pink" },
      { "font": "12px solid dodgerblue Comic Sans, sans-serif" }
    ]
  }
]

As you can see, the JSON syntax here is not really helping anyone. This format has all the disadvantages of JSON without any of the advantages. The program still has to parse the strings outputted by the JSON, and the user needs to type useless structure without dodging the needs to understand the hard parts: what a CSS selector is, and what can be typed into the background and font properties.

[kb1]

23 hours ago, Edward850 said:

@kb1, before you start talking about how things are correctly done in your mega-cool-and-whatnot source port, make sure you have something to show first (screenshots, source code, release downloads, etc). Keep in mind we've already seen thousands of people claiming to know how to do everything correctly in source code that never-ever have anything to show for it.

You read my post! You like me, you really like me!!

16 hours ago, Quasar said:

...Not that I'm necessarily in favor of this over any other particular solution, though. I am not even convinced that a reference parser is the most important part of this - a properly written spec should allow me to write my own if I feel so inclined. And people w/o the skillset to write a basic parser *probably* should re-evaluate their learning priorities before they go much further into advanced source port development, IMHO.

This is a good point. You're right - the spec should contain everything required to make a parser. A reference parser is useful, though, for devs that want to just drop it in, or, especially, to test your own parser. It's a convenience that I'd like to provide...along with specs good enough to build a new parser from. But the specs come first - well noted.

 

15 hours ago, Ladna said:

Hey!  I work on D2K, a PrBoom+ derivative.  In the past I've worked on EE (a lot) and Odamex (a little).

Cool - That helps me remember.

15 hours ago, Ladna said:

 

I will never sign on to a standard that bakes in Doom-specific formats.  I'm not doing this because I like it, in fact I've already blown a lot of coding time on this thread and I kind of resent it.  But I genuinely think an advanced port standard that uses proprietary, NIH formats harms Doom, maybe irrevocably, and I would never be a part of it.

Doom is chocked-full of specialized formats. And, you are a part of it - that's what all the replies are about. But, right now, you are dead set on one specific format that, honestly, doesn't have much going for it. I've been cheerleading DECORATE-lite as a possible solution. Maybe it's not the best idea. That's ok. But I am not claiming that I'm going to take my ball and go home, unless it's done in my favorite format.

15 hours ago, Ladna said:

 

I sort of considered not writing a big post.  Of course I did anyway.

 

Doom is already mired in its own vocabulary.  No one knows what DECORATE is, so if you, say, make mods for other games or want to get started on a mod for Doom, you get to wade through ZDoom's wiki.  Sure that's the case whenever you don't know something, but the difference between something DECORATE and INI is that there's 832,902,382,340,929 examples of INI files out there, not so much when it comes to DECORATE.  Besides the (obvious) fact that you've probably come across an INI file or two in your life.

 

No one knows what ACS is, so if you, say, make mods for other games or want to get started on a mod for Doom, you get to wade through ZDoom's wiki.  Sure that's the case whenever you don't know something, but the difference between something like ACS and Lua is that there's 832,902,382,340,929 examples of Lua code out there, not so much when it comes to ACS.  Besides the (obvious) fact that if you're a game modder, you've probably come across a game scripted in Lua or two in your life.

 

See the difference is in our perspectives.  Most of the posters here are focused on this tiny (though very cool) DW community, but I'm commenting from the perspective of a professional developer and general human being.  It's true (maybe?) that most DW mappers are familiar with DECORATE, but I want more and more people working on and experiencing Doom.  These NIH formats are a huge barrier to that, which is why I'm so vociferously against them.

I am also a professional developer, and, arguably, human-like. And, I've seen, edited, generated, and used my share of .INI files. You claim to want more people to use Doom, but you plan on denying them the vast library of existing DECORATE libraries, the expansive user-maintained Wiki, the combined knowledge of the community. And, you claim that there's a huge barrier. Why? What barrier? This supposed barrier hasn't stopped the literally thousands of projects that use this technology.

15 hours ago, Ladna said:

Feel free to make a standard without me!  Honestly I'm already planning to support a lot of the creepy toy store that is ZDoom features, so whatever.

Pretty much all of those are jokes, but OK I'll stick to substantive criticism from now on.

They're not funny. Do you honestly believe that, when DECORATE was being developed, that the goal was to make it difficult to use? It is built specifically to provide streamlined access to the available features.

15 hours ago, Ladna said:

Assurances of "there will be a spec and you will like it" don't do anything for me.  You can write a perfect spec, but if it's a Doom/NIH format, I won't support it.

No one ever said that, or anything resembling it, and you know it. I claimed that a spec would be created, and presented, in first draft fashion, complete with all the details. I then stated that the spec will be presented for all to see, discuss, modify, and tweak until we can come to an agreement, or vote upon it, and that that may call for the need to compromise. Don't talk shit.

 

Then settle for less than perfection. You, and you alone are adamantly denying your users from having access to some cool features. Do you think they will thank you for that?

 

15 hours ago, Ladna said:

 You can write a C library for it, but if it's a Doom/NIH format, I won't support it.  I think I've been pretty clear why, let me know if there's any confusion or rebuttals about it, but quit asking me why.  I pretty much type the same things in every post, and the response is, "get over it".  No, I guess.

No one is talking to you - you don't have to respond. You have not been clear about why, just that you refuse. I was assuming we were discussing reasons based in logic. My bad.

 

15 hours ago, Ladna said:

 

I don't want to standardize on a library or a reference implementation.  I want a spec'd format.  Essentially if there's no maintained Python library for it, I won't support it.

 

- INI is either as short or shorter than any DECORATE-lite example posted, but the difference either way is like 2-3 lines.

- INI is used by goddamn everything.

- Millions of people use INI files and have for decades.

- INI totally does, and the sprites string is an obvious win

 

Again we've run into the DW vs. Global perspective thing.  I don't want Doom to be just for us.

 

I have never, even once, acted in bad faith here on DW or anywhere else in the Doom community.  You can leave this shit at home.  I've made my opposition to DECORATE so clear over so many posts with so many details that I really have no idea what you're talking about.  You think I'm full of shit because I haven't said anything nice about DECORATE?  I like the name.  Cool?

Guess what? INI files suck. INI files are A LOT larger than the DECORATE version, in almost all cases. MS deprecated INI file APIs long ago. Millions of people are, yes, forced to use .INI. So what?

 

No, I don't think you're full of shit. But, your statements have not been technically logical. The requirement, of a Doom programmer, that file reads must be done by a built-in library excludes the entire WAD file, most all of its contents, and provides no security. There's no encryption, no check-summing, no type-checking, no test for valid keys.

 

It boils down to this: I have accepted the job of trying to come up with techniques and formats that can be utilized by a wide range of source ports, without causing problems. For me to be able to do the absolute best job possible, I would need to get us to discuss and collaborate what could work well, for players, for modders, and for developers of all these ports. The qualities I should look for is interoperability, completeness, efficiency, simplicity, etc.

 

But you've got me in a bind. We cannot discuss any of this. The only format we can use is a, honestly, not-so-special format you are dictating. The only thing it has going for it is that they are abundant, and that libraries exist to handle the easy task of managing them. No talk about fit, efficiency, etc.

 

So, yes, I am inclined to questions motives. A professional programmer knows that all of these topics are important. A professional programmer looks at all available options to find the best set of properties that match the given task. A professional does not piss on another professional programmer's hard work, and call it a joke.

 

I am going to continue to try to do the absolute best job I can on this spec, because I believe that if something is worth doing, it's worth doing right. And, my users come first.

 

15 hours ago, Arctangent said:

The immediate issue I see with that stance is that I feel like any developer turned off by a non-standard format is going to be turned off by many things Doom, accentuated by the fact that this subject is about a standard meant to be supported by a lot of ports that keep a lot of Doom's idiosyncrasies.

 

Which ... leaves me very confused about your port, because it seems kinda like it's walking two opposite directions in that regard to me. You're in the exact position to make Doom more newbie friendly in this regard, so what's with the insistence on compatibility with its non-newbie friendly quirks, such as the blockmap bug?

I agree. And, I don't understand. I want to, but I just don't get it.

 

14 hours ago, Graf Zahl said:

 

Yes, and then we end up with the same dilemma as always: Different ports use different implementations with different quirks to implement a feature. Which leads to stuff like the zero-tagging issue with ZDoom handling it as 'activate backside' and all other ports not handling such maps properly. The same would happen here. That's why I am very much in favor of having a working reference implementation and the ports only having to connect the dots from the already parsed data to its native data structures.

Quasar's got a good point, though. Even if most people use the reference, the docs should be good enough to allow a completely new parser to be built, with identical results. Sure, bugs do happen, but yeah, this is the level of quality I'm striving for.

 

 

9 hours ago, Hell Theatre said:

I was just reading through all this lengthy stuff and I have to wonder: Why this heated discussion? I'm sorry, but I do not get what Ladna is up here.

To me his counterexample for an INI-based DECORATE replacement looks like a perfect example of redundancy for the sake of doing it differently.

 

I don't know. I was just hoping for some good ideas, and light discussion about a chance to achieve a new level of cooperation. I think it's my personal curse. I really do try to keep things open and light-spirited. Text communication is hard.

 

However, there's been quite a bunch of nice, good ideas and information in this thread. It's really going to be helpful, and it already has. It's all good.

[Graf Zahl]

2 hours ago, dpJudas said:

 To illustrate what I mean, here's a CSS file converted to JSON:

And here I thought that it cannot get any worse than CSS, but congratulations to convert something essentially unparseable into something even more unparseable... :D

 

[axdoomer]

I find this thread entertaining. 

[Gez]

8 hours ago, Ladna said:

Hey, you paste a DECORATE example on a non-DW gamedev forum, people will be all "WTF is that".  You paste an INI example, everyone will know.  That's all I've been saying :)

There's nothing magical about that.

 

Post this on a gamedev forum:

[Fruits]
Apple = 10
Banana = yes, we have none
Cranberry = 4.53
Datte = "12"

People won't say "oh, it's an INI! I love it! I can plug that in my game!" they'll say "well it's a list of variable and their values".

 

Now post this:

Fruits
{
	Apple 10
	Banana "yes, we have none"
	Cranberry 4.53
	Datte "\"12\""
}

They won't say "what is this strange format, I don't understand how its syntax works at all!" they'll say "well it's a list of variables and their values".

 

Human brains are good at pattern recognition, so from an ease-of-use point, there's really no intrinsic  advantage to INI syntax over any other trivial key value association. Heck you could even use wiki template syntax, people would still understand it:

{{Fruits
	|Apple=10
	|Banana=yes, we have none
	|Cranberry=4.53
	|Datte="12"
}}

 

8 hours ago, Ladna said:

Yeah sure but that's true of DECORATE too.  The difference is I can parse the format with an INI parser.  So ceteris paribus INI wins.

But here, ceteris non paribus. The existing format has the advantage of being already supported by map editors -- videlicet the Doom Builder family and SLADE, but also miscellaneous other utilities, e.g. this one. A small subset that'd be backward compatible would be the thing that requires the least amount of work to support.

 

I get that you don't want a new standard to reinvent the wheel, but stubbornly refusing to innovate in wheel technology would result in cars like this:

Trying to shoehorn the square peg of DECORATE into the round hole of INI because you don't want to reinvent the wheel is one heck of a mangled metaphor, but seriously, all that work for just being able to use an INI parsing library seems like a very minimal return on investment. The idea that we'd be paid of for this investment by a sudden influx of new utilities made by people outside the community that are just waiting for us to see the light before welcoming us in their glorious INI commune seems frankly chimeric.

[Graf Zahl]

8 minutes ago, Gez said:

I get that you don't want a new standard to reinvent the wheel, but stubbornly refusing to innovate in wheel technology would result in cars like this:

The irony being that he wants to reinvent the car to avoid reinventing the wheel.

So what's more efficient? Putting sub-par wheels under the car or make better wheels? I'd go for choice #2.

 

Does that analogy make sense?

 

 

[Jaxxoon R]

I'd... prefer to keep my interactions with ini reserved to keybinds, honestly.

[Ladna]

9 hours ago, dpJudas said:

My reference to driver .inf files was exactly to point out what happens when you realize your data isn't simple basic key/value pairs and start inventing your own semantics to the key and the value parts. Your tiny example did BOTH.

You don't need to do both.  Just make Sound.See SeeSound, like DECORATE.  There are no collisions, I just thought it was consistent with the namespaced states.

 

9 hours ago, kb1 said:

Doom is chocked-full of specialized formats.

I really mean NIH formats that can't be parsed by external libraries.

 

9 hours ago, kb1 said:

And, you are a part of it - that's what all the replies are about. But, right now, you are dead set on one specific format that, honestly, doesn't have much going for it. I've been cheerleading DECORATE-lite as a possible solution. Maybe it's not the best idea. That's ok. But I am not claiming that I'm going to take my ball and go home, unless it's done in my favorite format.

Actually I'm the one who's compromised the most.  I'm into Gez's curly braces, although I think it would be better to make a separate INI section for states, but either way.

 

In fact, if people want to volunteer to create and maintain DECORATE libraries for the major languages I'm down with DECORATE.  I'm verrrrry skeptical of it, but that solves my issue completely.

 

Also Graf's issuing ultimatums too, so maybe be a little more fair in your criticism here.

 

9 hours ago, kb1 said:

You claim to want more people to use Doom, but you plan on denying them the vast library of existing DECORATE libraries, the expansive user-maintained Wiki, the combined knowledge of the community. And, you claim that there's a huge barrier. Why? What barrier? This supposed barrier hasn't stopped the literally thousands of projects that use this technology.

This is, and maybe you don't know it, an argument for "let's just make ZDoom the standard".  I'm against that, for basically all the reasons I've laid out before: no diversity of implementations, no actual specs, no external library support, etc.  Plus it's sort of a middle finger to other ports who have their own way of doing things, or might want to later.

 

If every proposal for an advanced port standard is really "which ZDoom feature should every port have", then I'm not into it.

 

And, for what it's worth, your argument here is an argument against UDMF too.

 

9 hours ago, kb1 said:

INI files suck. INI files are A LOT larger than the DECORATE version, in almost all cases.

The syntax difference between DECORATE lite and INI, as Graf said, is miniscule to humans.  I mean look at it, they're pretty much the same.

 

The big difference is no external library support.

 

9 hours ago, kb1 said:

The requirement, of a Doom programmer, that file reads must be done by a built-in library excludes the entire WAD file

That's why I like PK3 and why I wish UDMF were JSON.  But making non-optimal decisions once shouldn't lock you into non-optimal decisions forever.

 

9 hours ago, kb1 said:

But you've got me in a bind. We cannot discuss any of this. The only format we can use is a, honestly, not-so-special format you are dictating. The only thing it has going for it is that they are abundant, and that libraries exist to handle the easy task of managing them. No talk about fit, efficiency, etc.

I'm happy to discuss other formats.  toml is nice, for example, or yaml, there are (I think?) lots of alternatives.

 

9 hours ago, kb1 said:

I agree. And, I don't understand. I want to, but I just don't get it.

External.

Library.

Support.

 

Gez: that's a fair point; I think it's right to weigh the benefits of external library support against the work to add support for a format like INI to existing tools.  I don't want to speak for you or other tool devs, but for D2K I don't think I'd have a problem.

Edited May 4, 2017 by Ladna : Can't delete quote boxes on mobile... weird

[Graf Zahl]

25 minutes ago, Ladna said:

External.

Library.

Support.

 

Since you regurgitate these three words ad nauseum, I still believe you are missing the forest for the trees. Definition formats are not made to make it convenient for the programmer to implement them, they are made to make it easy for the content creator to use them. And for that something clear, concise, boiler-plate-free and most importantly robust is preferrable.

 

INI or even JSON will never solve the problem of an expressive language whose first concern is to be Human readable and comprehensible, they are only boilerplate standing in the way of creating something made for Humans, locking oneself into such formats is the epitome of Made For Machines, which is the worst software around.

 

Also, your INI library in UMAPINFO seems to be incapable of tracking line numbers, and it uses a small static buffer for parsing, so it'll break with lines that are too long and probably got a few other gotchas that are not relevant when reading back machine generated INIs but is just a hostile piece of software when being asked to deal with user-created data. It effectively lost a lot of user-friendliness from my ad-hoc parser I wrote in maybe 15 minutes.

 

[Gez]

Consider this:

[CONVERSATION]
Text = Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua? Click
[YES] or [NO].

Yes it's a perverse example, but this is the kind of things that wouldn't risk happening if we were talking about a format where strings are quoted. So just because something is simple and readily available doesn't mean it's good.

 

I thought of this because [ and ] just so happen to be valid frames in Doom, thanks to the arch-vile (which also uses \ by the way).

 

(Also lol: this forum's code widget has syntax highlighting for a lot of stuff, but not for INI.)

[Ladna]

INI is actually exceptionally good for humans.  Its format is clear, simple, and straightforward.  It's also, by your own admission, not that different than DECORATE.  Furthermore, there are lots of non-human-readable formats for content creators, PDF for example.  It's all about usage, and my argument is that we're not considering all usage cases here.

 

INI is more concise than DECORATE (all my conversions of your examples are shorter), as boilerplate-free as DECORATE, and its robustness has been proven over decades of use in a variety of environments - unlike DECORATE.

 

UMAPINFO stuff: wrong thread, but:

 

There are defines to increase line length or use the heap .

 

Error line number handling is like a 3 line change, but that's fair.

 

I'll update the PR, thanks for the feedback :) .

[Bauul]

55 minutes ago, Ladna said:

External.

Library.

Support.

Just for us non-coders trying to follow this thread, could you (or anyone else for that matter) give us an example of how external library support (which I admit I only vaguely understand what they are) would help an average mapper such as me create (for example) a new weapon in Doom?

[Graf Zahl]

20 minutes ago, Bauul said:

Just for us non-coders trying to follow this thread, could you (or anyone else for that matter) give us an example of how external library support (which I admit I only vaguely understand what they are) would help an average mapper such as me create (for example) a new weapon in Doom?

Not at all. It only would help some programmers save some work on coding support for a format. (And I consider even that part dubious at best.)

Edited May 4, 2017 by Graf Zahl

[Graf Zahl]

21 minutes ago, Ladna said:

INI is actually exceptionally good for humans.

[citation needed]

 

 

21 minutes ago, Ladna said:

 Its format is clear, simple, and straightforward.

Depends on the use case. This isn't a good one. Even UMAPINFO is borderline because the values need to be reparsed.

 

21 minutes ago, Ladna said:

 It's also, by your own admission, not that different than DECORATE.  Furthermore, there are lots of non-human-readable formats for content creators, PDF for example.  It's all about usage, and my argument is that we're not considering all usage cases here.

 

It is very different from DECORATE. DECORATE is something complete, FuglyINI is some added semantics inside INI syntax. So any simplicity of INI does not apply. Parsing the inner stuff is just as complex as parsing it in DECORATE.

 

 

 

 

[Ladna]

2 hours ago, Bauul said:

Just for us non-coders trying to follow this thread, could you (or anyone else for that matter) give us an example of how external library support (which I admit I only vaguely understand what they are) would help an average mapper such as me create (for example) a new weapon in Doom?

It's exactly as powerful as DECORATE, so you're not giving up anything.  Like Graf said it's important for developers of external tools, and even stuff like SLADE or Doom builder because they can then just use libraries instead of implementing parsers themselves.  Ultimately it should lead to even better tools and thus better mods, although there are a lot of legacy things like this to hammer out before you'd really start to see benefits.

 

Gez: my posted Python parser handles all that.  It thinks that sections are lines that start with '[', so that will fail, but that's an actual format error not a weird corner case.  The archvile frames work just like you'd expect.

[Xaser]

I've got some thoughts on this whole shebang, but frankly, I don't know if there's anything else to be said at this point. I'll at least make the observation that only 1 out of n developers has any serious issues with the way things are going. If the rift is irreparable, I'd rather see the other n-1 devs carry this thing forward rather than let the one drag the idea to a permanent grave.

 

Now, this is way late, but there was a call much earlier for mappers in particular to weigh in on stuff they'd like to see, but frankly, my list (sans scrollers) hinges pretty heavily on UDMF, which is already in the "do later" bucket. For the Doom map format, I can't really imagine there's much we could shove in without making the metaphorical container explode.

 

I can idea-vomit my post-UDMF wishlist if folks think it would be valuable, but only if it's deemed relevant. No sense in starting a fire in another frying pan, for bastardized analogy's sake.

[Hell Theatre]

5 minutes ago, Xaser said:

 I'd rather see the other n-1 devs carry this thing forward rather than let the one drag the idea to a permanent grave.

Me, too. But sadly, if you check the discussion, that one person who disagrees is shitting over the entire thread, essentially telling everybody "I am right and you are too blind to see my wisdom." This will more likely result in the other n-1 devs just giving up. Maybe that's the sinister master plan here because it should be clear by now that Ladna can not win this discussion. So much blabbering and not one single 'like' received for it... :P

 

[Xaser]

@Hell Theatre To be perfectly blunt, pessimism of the "they're just going to give up" sort is extremely discouraging and doesn't help anyone in the slightest either -- it's a self-fulfilling prophecy if that mindset catches on.

 

In contrast, I suppose I should toss in the fact that I'm uber-psyched to see this sort of stuff worked on (UMAPINFO yesssssss) and seriously hope it comes to fruition. So there you go. :P

[Combinebobnt]

just glue together prboom++++++ and gqzdoom together with some super computer glue and thats good enough for a superport

[Graf Zahl]

That's ... just ... not possible...

 

[Xaser]

"Super Computer Glue" is now the name of my new band. Thanks.

[Liberation]

Perhaps we could just start with all ports supporting UMAPINFO? That alone would be a good start and would raise the standard up one notch, and I'm thinking everyone wants UMAPINFO as a minimum.

[Gez]

1 hour ago, Ladna said:

even stuff like SLADE or Doom builder because they can then just use libraries instead of implementing parsers themselves

As I have already pointed out already, both SLADE and Doom Builder have configuration files that are based on a nested block syntax because this allows hierarchical groupings, something which INI doesn't do. Stuff like this:

parentblock
{
	childblock
	{
		grandchildblock
		{
			value1 = 1;
			value2 = 2;
		}

		grandchildblock
		{
			value3 = 4;
		}
	}
	
	childblock
	{
		value4 = 8;
	}
}

So: they will keep their own parsers, thank you very much. Besides, keep in mind that SLADE's integrated tokenizer is currently used for parsing:

• SLADE's own configuration files (including the "ini")
• Various formats: DECORATE, GLDEFS, SBARINFO, ZMAPINFO, EMAPINFO, EDF. Those formats are parsed to identify included lumps as belonging to the same language (see WadArchive::detectIncludes()); DECORATE is also parsed for custom actors.
• Translations which you can save and load from the Colour Remap window
• Console commands because SLADE has a console and some commands have parameters, so we need to tokenize the user input
• TEXTURES
• PaintShopPro and GIMP palettes
• SWANTBLS lumps to convert them into SWITCHES and ANIMATED lumps
• UDMF (duh)
• ACS source - to detect information about the map by reading OPEN scripts, see MapSpecials::processACSScripts()

So for many internal and external reasons, SLADE will retain its own parser forever and never ever ever ditch it to replace it with an INI parsing library. Not gonna happen. Furthermore, if there's a need to parse INI files? It's gonna use the already integrated parser instead of plugging in another dependency for something so trivial. The INI library would just add more bloat. Just gonna be clear about that: the external parser library argument is entirely irrelevant to SLADE; it's about as appealing as a suggestion of burning down your house so as to get to go to a homeless shelter. No, for SLADE's purposes, the most convenient format would be something that can just be processed by Parser::parseText(). That's the minimum-effort solution as far as I am concerned.

[Graf Zahl]

and to be blunt: The same applies to most ports that read definition lumps. If their tokenizer is not complete garbage it will do all the heavy lifting needed for any format thrown at it. The only reason I started UMAPINFO with its own parser was to make things easier, but let's be clear: If some parser dependency needs to be added to handle it, I'd rather oick something that doesn't dictate syntax but is flexible to parse anything that isn't too mired in omitting as much structure as possible. That crappy INI parser is 5kb, the tokenizer I extracted from somewhere else is 11 kb, after adding a few convenience functions it's 14kb. Well, big deal - it can parse DECORATE, it also can parse a better MAPINFO format and it can even parse UDMF. So why use INI when I can use the same solution to parse everything that may be planned for the future? UDMF is a format that cannot be changed to be parsed by an INI parser so we'd need an alternative anyway, and since that is the case, it's the INI parser which is the bloat.