New (cross-port?) scripting languages for Doom

[Gez]

Meh, if you're gonna use ACS anyway, it's simpler to spawn these actors from your scripts in the first place, this way you can give them a TID and set their specials.

[Mordeth]

6 hours ago, Gez said:

You can use ACS without the Hexen map format, but you'll be limited in that there are some features you won't be able to use easily, notably anything that uses a thing ID, since you won't be able to give TIDs to things in the map editor.

 

True. That's why I don't use the map editor but an ACS script to place 99% of the things that require a TID in the level at map start. Usually these are (interactive) custom mapspots. Only when I absolutely have to (eg. if I want to check if a group of monsters is dead before I open a door) do I need to resort to CLED / ExtraData.

 

And it's not ExtraData that is a drag to use... it really is not. It's the requirement to use CLED to assign a number to a Thing's options flag that represents its tag. CLED works by thing sequence number, so if you delete a Thing in your WIP those sequence numbers change and your CLED script assigns stuff to the wrong Things.

[printz]

Yeah seriously. Editors really should let you put arbitrary thing flag values.

[Graf Zahl]

I really do not want to see such a crutch become part of an editing standard. It'd be better to define an UDMF set for the needed features.

 

[Mordeth]

I think it's more or less a given that any cross port stuff ought to be based on the UDMF map format.

[VGA]

4 hours ago, Mordeth said:

I think it's more or less a given that any cross port stuff ought to be based on the UDMF map format.

Why not Hexen format? It adds some basic map scripting and you can assign Tids to things etc? Right?

[Albertoni]

1 hour ago, VGA said:

Why not Hexen format? It adds some basic map scripting and you can assign Tids to things etc? Right?

Because if you're going to implement support for a new format, might as well implement the arguable best.

UDMF allows translucency (from 0 to 100% transparent), scaling, and lighting directly on a linedef. To align, rotate and control light level of floors and ceilings independently of sector brightness. Upper, middle, and lower textures on the same linedef can have different offsets.

And I'm not 100% sure, but I believe it allows much more fine control over linedef triggers, like repeatable gunshot-activated that does something else than opening a door.

[Blastfrog]

2 hours ago, VGA said:

Why not Hexen format? It adds some basic map scripting and you can assign Tids to things etc? Right?

I suppose it could serve as a stepping stone for some of the things that UDMF supports? But then, if you're going to eventually go with UDMF anyway, is binary Hexen format support really all that necessary or worth it?

[Arctangent]

4 hours ago, VGA said:

Why not Hexen format? It adds some basic map scripting and you can assign Tids to things etc? Right?

ZDoom's UDMF basically is Hexen format, only a helluva lot more expandable and with far fewer numeric limitations.

 

One of the main differences I can think of is the fact that instead of picking a single trigger for line specials, you can pick as many as you want. Also, you can provide a line a tag without forcing it to have a certain special, requiring you to use a script to change it afterwards if it's meant to be a door or switch.

[Jon]

Here's a question. What kind of things can you not achieve with existing scripting solutions like ACS?

[Ladna]

Going off what I've been able to find on ZDoom's wiki.  You have a single data structure, the array.  String manipulation is... non-existent?  Functions aren't first class.  There's no way to declare a new type.  There's also no way to declare a composite type, but in fairness ACS doesn't have those so consistency I guess.  There are no external libraries, which means no support for things like JSON, databases, filesystem access, HTTP, etc.  There are no tools for ACS, no debuggers, no analysis, no IDEs.  You have very little control over allocation, and the lifetimes of objects isn't clear.

 

I'm not trying to be a jerk but ACS is an extremely limited language.  Whether you think that's a virtue or a fault is up to you, but you should also consider that there are other languages that can be similarly sandboxed but don't necessarily have these fundamental limitations.

[Albertoni]

38 minutes ago, Ladna said:

Whether you think that's a virtue or a fault is up to you

Used to be both. Now that there's ZScript, it's mostly an advantage. If you just want some ifs, or calling a bunch of commands from a single button, use ACS, it's easy to learn.

Anything really complex? ZScript.

 

I'll agree, ACS is incredibly limited, but Doom is also incredibly limited as an engine. Thus, ACS covers 95% or so of the mapper/modder needs.

 

Also, filesystem access is kind of implemented. You can set variables on the config files.

[Arctangent]

51 minutes ago, Ladna said:

There's no way to declare a new type. ... There are no external libraries, which means no support for things like JSON, databases, filesystem access, HTTP, etc.

the frig are you trying to make your skull-shaped wall buttons do exactly

[Ladna]

Stats, new game modes, new weapons, new items, HUD mods and displays, console stuff, menu mods.  Honestly stuff you use scripting for in nearly every other gaming engine from the last 10 years.

[Albertoni]

> Stats

Player classes. Not able to do everything, but almost anything sane is possible. See: Kyle873's Doom RPG Mod

 

> New game modes

Difficulties 6-8, and some usage of ACS. Again, not able to do every fucking thing, but 99%.

 

> new weapons and items

DECORATE. ACS is a logic language, while DECORATE is the thing state language.

 

> HUD mods

SBARINFO lumps. Again, not able to do every single thing, but anything I can imagine that the average mod wants to do is possible.

 

> HUD displays

You can fake 99% of the cases with SBARINFO. The other 1%, yeah. I agree.

 

> Menu mods

Define those more clearly, please. You can edit the difficulties, but actively removing stuff from the options menu seems like a player-hating thing. I can see you making a mod with some specific option on or off. Use the default .cfg file to specify that, and mention it ASAP in the readme. Or hell, if you're feeling evil, check for the variable in ACS and tell the player to change it.

[Ladna]

Your answers all essentially reduce to "no ACS doesn't do this, so use something else".  Hobbes asked "what doesn't ACS do?"  I think if you pick a scripting language to do these things and it doesn't do them, you didn't do it right.

 

But besides all of the functionality ACS simply lacks, it's legitimately a very limited language.  It doesn't even have namespaces.

 

Scripting should be the way modders extend the engine, because there's really no other way they can.  It's supposed to be the thing you resort to when DECORATE or action functions don't cut it.  If even that last resort is "if you can't do it in DECORATE you can't do it" then I really don't see the point.  Plus I think it's generally counterproductive to pick a super limited language, and then extend it with an entirely different technology.  It would have been better to just add that functionality in the first place.

 

And finally, and I include this specifically because I think it's very important, just because you can do something doesn't mean you should.  I can absolutely create a thing type in DECORATE, spawn an instance in ACS, and use its attributes to hold game state for my mod.  But if I use this as an argument for why you don't need composite data types, I've totally lost it.  There's a reason programmers like things like namespaces, custom types, or composite data structures, because even though in the end it's all ones and zeros, these tools allow us to write more featureful and sophisticated programs while avoiding errors and crashes.

 

Your rebuttals boil down to "it's all just ones and zeros", which is true, but there's a reason we don't program in ones and zeros any more.

[Graf Zahl]

Let's not confuse two entirely different types of 'scripting' here.

 

For once, there's scripted map actions. For those no complexity is needed, all it needs is some basic arithmetics and the ability to trigger the desired actions. ACS is perfectly fine for this type of scripting.

 

Second, there's extending the game, e.g. new menus, different status bars, new actors or weapons. ACS is clearly not suited for such a task, you need a language that can handle more complex data and expose it to the modder.

 

The one important thing is that the second type of scripting will require broad exposure of implementation details, so it will inevitably be engine dependent and not a good candidate for a cross-port standard.

 

[Arctangent]

What's basically happening is that we're discussing a more programming-centric option to be used instead of the Rube Goldberg machines that are voodoo doll conveyer belts, and you're dismissing a candidate because it can't be used to make an RPG.

 

Which you don't seem remotely aware of, considering you apparently want one of these

 

 

to do any of this

 

2 hours ago, Ladna said:

Stats, new game modes, new weapons, new items, HUD mods and displays, console stuff, menu mods.  Honestly stuff you use scripting for in nearly every other gaming engine from the last 10 years.

 

[Jon]

6 hours ago, Ladna said:

Going off what I've been able to find on ZDoom's wiki.  You have a single data structure, the array.  String manipulation is... non-existent?  Functions aren't first class.  There's no way to declare a new type.  There's also no way to declare a composite type, but in fairness ACS doesn't have those so consistency I guess.  There are no external libraries, which means no support for things like JSON, databases, filesystem access, HTTP, etc.  There are no tools for ACS, no debuggers, no analysis, no IDEs.  You have very little control over allocation, and the lifetimes of objects isn't clear.

 

I'm not trying to be a jerk but ACS is an extremely limited language.  Whether you think that's a virtue or a fault is up to you, but you should also consider that there are other languages that can be similarly sandboxed but don't necessarily have these fundamental limitations.

 

Thanks. I should explain why I asked: not because I'm a fan of ACS (I hardly know it) but because I'm interested in the design of a hypothetical successor (or, to put it another way, some interesting problems to solve with another solution).

 

But your answer is very much in terms of tooling rather than results; things like string manipulation, data types, first-class functions etc., that's the tools of the craft, of course I have opinions on those things but what I was angling for was what kind of cool doom effect cannot be reasonably achieved with ACS (or for that matter ZScript)? 

 

 

3 hours ago, Ladna said:

Stats, new game modes, new weapons, new items, HUD mods and displays, console stuff, menu mods.  Honestly stuff you use scripting for in nearly every other gaming engine from the last 10 years.

 

Excellent examples, thanks. Let's assume I've not played any other gaming engine in the last 10 years (not far from the truth actually)

Edited July 4, 2017 by Jon

[Gez]

7 hours ago, Ladna said:

Going off what I've been able to find on ZDoom's wiki.  You have a single data structure, the array.  String manipulation is... non-existent?  Functions aren't first class.  There's no way to declare a new type.  There's also no way to declare a composite type, but in fairness ACS doesn't have those so consistency I guess.  There are no external libraries, which means no support for things like JSON, databases, filesystem access, HTTP, etc.  There are no tools for ACS, no debuggers, no analysis, no IDEs.  You have very little control over allocation, and the lifetimes of objects isn't clear.

Do you believe it'd make sense for PrBoom+ to get a scripting language that does all these things?

[Jayextee]

5 hours ago, Albertoni said:

Define those more clearly, please. You can edit the difficulties, but actively removing stuff from the options menu seems like a player-hating thing.

 

Can I leap in and, as someone who wants to use source ports to make original games, offer a counter POV here?

You could probably use G/ZDoom to make a Mario Kart game or something (to wit, I've seen Donkey Kong Country and Resident Evil 2 remade with varying degrees of success in the engine); you're not going to want to offer players who may not be from the DOOM modding scene here the option to enable/disable MLook or set DMFlags or define a key for 'crouch' in this case. Giving more control to the developer isn't necessarily a "player hating thing" if their focus is likely streamlining the experience by removing now-redundant elements

 

I've seen the argument pop up time and time again that the removal of menu items and features would disrespect a player's choices, and yet I can't bind camera controls in the new Crash Bandicoot N-Sane Trilogy or save my game mid-match in Tekken 7 (just to bring up two admittedly-ridiculous examples of games that shouldn't have features mandated by most source ports). Sure, I can see the Terry modders abusing such things; but aren't we already in the habit of keeping such things at arms length?

[Altazimuth]

I'll be frank, I think doing much more than "execute a bunch of specials with delays and keep track of variables" is too much for something that's supposed to be easily-agreeable and added; maybe add basic printing to tell the player they opened a door somewhere else or something. As I see it, the baseline should be doing everything voodoo scripting can do, but with an actual scripting language. Having 4 switches to open a single thing (and printing that they SOLVED 1 4TH OF THE PUZZLE), or just lowering a bunch of tagged sectors is plenty for an initial feature-set of this scripting language.

 

Sure, we should have something that can be extended eventually, but we have to go for lowest reasonable denominator IMO.

 

I'm sticking by ACS as suitable for the tasks I listed, but I don't think we'll ever agree on what to use if we can't agree on what functionality any language we use needs.

 

EDIT: Is string manip really that important? Anyways if you really want something powerful with ACS you can use DavidPH's GDCC, which is extremely powerful. You can literally compile C into ACS bytecode.

 

EDIT 2: I want to make it clear that I support anything that would work well and isn't too complicated, if it can be agreed on. I'm not saying ACS or die.

Edited July 4, 2017 by Altazimuth

[Jon]

7 hours ago, Gez said:

Do you believe it'd make sense for PrBoom+ to get a scripting language that does all these things?

 

A (new/common) scripting language was ruled out-of-scope in the other thread. This thread is meant to be a more broad discussion, and there's nothing to say PrBoom+ acceptance is a necessary criterion here.

[VGA]

Yes, we already have GZDoom for extensive scripting, can people agree on a simple language that doesn't need a new format (except Hexen map format maybe) and is easy to implement on prboom+ and enhanced-oldschool ports? To replace the Boom baseline in the triad of:

 

Vanilla/Limit removing

Boom

ZDoom

[Jon]

I am strongly considering picking up where AlexMax left off (with Lua on top of Crispy Doom) which is why I was looking for useful use-cases of interesting things that are difficult or impossible to do with the tools we already have today. They'd be the first things I'd work on.

[Graf Zahl]

2 hours ago, VGA said:

Yes, we already have GZDoom for extensive scripting, can people agree on a simple language that doesn't need a new format (except Hexen map format maybe) and is easy to implement on prboom+ and enhanced-oldschool ports? To replace the Boom baseline in the triad of:

 

ACS or not, the main issue with Doom map format is always the same: It lacks a few crucial bits of information that are basically a requirement for robust scripting. From a pure point of implementation no other language has any advantage. The minimum requirement in any case would be robust definition of IDs for sectors, lines and things. Doom can only robustly tag sectors, it can tag lines, but it's not robust because the same field duplicates as the line action's parameter and thing IDs don't exist at all.

 

Once all these features exist, the next step will be generalized (i.e. more flexible) sector actions so that a script can actually do what is required. So something will have to be implemented, be it Hexen-style specials or something else doing similar features.

And once we get there ACS is almost the default choice - because it already exists and already got broad support from various ports that need to be targeted. Why even go further and require adding yet another language runtime?

 

[Ladna]

8 hours ago, Graf Zahl said:

Let's not confuse two entirely different types of 'scripting' here.

That's a valid point, but I desperately want to avoid having more than one scripting language in an engine.  This is sort of the same design perspective that leads to having dozens of different text lump formats, with the corresponding code bloat necessary to support them.  Building a source port is already maybe an impossible amount of work, and I believe that adding to that workload stifles source port innovation.

 

8 hours ago, Graf Zahl said:

The one important thing is that the second type of scripting will require broad exposure of implementation details, so it will inevitably be engine dependent and not a good candidate for a cross-port standard.

There's nothing wrong with an API spec.  There's plenty of examples, probably the most famous are the C standard library, the Java standard library, and POSIX.  I'm not saying I want something on the level of detail of a "Doom POSIX", but it's absolutely doable at a high level.  `actor.set_position(4.0, 2.0, 8.0)` isn't hard to define.

 

6 hours ago, Gez said:

Do you believe it'd make sense for PrBoom+ to get a scripting language that does all these things?

I'm actively working on that, so yes.

 

7 hours ago, Jon said:

But your answer is very much in terms of tooling rather than results; things like string manipulation, data types, first-class functions etc., that's the tools of the craft, of course I have opinions on those things but what I was angling for was what kind of cool doom effect cannot be reasonably achieved with ACS (or for that matter ZScript)?

Ahhhh, well I kind of have two responses.

 

First, I think the best example is enemy/friend/bot behavior.  Building a sophisticated AI in ACS isn't the easiest thing because reusing logic isn't very easy.  As the Doom playerbase dwindles, I think it's probably an important feature to have pretty smart bots to fill in spaces.  Like if you want to have a huge 100-player FFA on a pretty big map, that equals bots.  But if you don't want it to be an idiot carnival, you need good AI, and doing that in ACS is harder than needs to be.

 

5 hours ago, Jayextee said:

You could probably use G/ZDoom to make a Mario Kart game or something (to wit, I've seen Donkey Kong Country and Resident Evil 2 remade with varying degrees of success in the engine); you're not going to want to offer players who may not be from the DOOM modding scene here the option to enable/disable MLook or set DMFlags or define a key for 'crouch' in this case. Giving more control to the developer isn't necessarily a "player hating thing" if their focus is likely streamlining the experience by removing now-redundant elements

This is a great lead-in to my 2nd, and more hand-wavy point.  I think "doom effect" is at this point limited by ACS (and other stuff, but w/e).  Vehicles, for example, aren't really "doom-y", but that's because we don't have the infrastructure to really support them.  If we had a powerful scripting language 5 years ago we might feel differently about that now.  Like, to a lot of people CTF is Doom because CTF was added a decade ago, but that's because C++ programmers added it to ZDoom.  If they'd added vehicle racing instead there might be a lot of people who think DoomKart is Doom.  If it was possible to do this kind of thing in scripting and not C++ (probably the hardest mainstream language), I think our idea of "doom effect" would be a lot wider than it is today.

 

2 hours ago, VGA said:

Yes, we already have GZDoom for extensive scripting, can people agree on a simple language that doesn't need a new format (except Hexen map format maybe) and is easy to implement on prboom+ and enhanced-oldschool ports?

 

3 hours ago, Blastfrog said:

Don't start again, Ladna, it was bad enough the last time you derailed one of these threads with insane ideas originating from a total lack of perspective.

I get the frustration with me, and I'm not at all offended.  But I think it's a very real possibility that the community comes to some sort of tacit understanding of "Boom Plus" as "GZDoom Minus", and while that's obviously a step up in terms of functionality, it's a step backwards in future development.  I stand by my other position in favor of external library support, and my position here in favor of a powerful, mainstream scripting language that suits all purposes instead of multiple, limited, custom scripting languages that don't, because I think external library support and something like Lua or JavaScript would be a real boon for Doom's featureset and playerbase.  I really view things like custom text formats and custom scripting languages as technical debt that port authors have to pay later, and that's why there are far fewer ports these days than there used to be.  That's a real shame, and I think that general lack of innovation has a direct effect on our playerbase.

Edited July 4, 2017 by Ladna

[Graf Zahl]

28 minutes ago, Ladna said:

I get the frustration with me, and I'm not at all offended.  But I think it's a very real possibility that the community comes to some sort of tacit understanding of "Boom Plus" as "GZDoom Minus", and while that's obviously a step up in terms of functionality, it's a step backwards in future development.

 

It's an even bigger step back if what you come up with remains yet another insular implementation - and that will happen if you try to do map scripting and game scripting with the same language.

 

Whatever you want to name it, this is not about making PrBoom better but about finding a consensus everybody can agree with.

Some well intentioned pie-in-the-sky solution isn't worth the investment if it clashes with the ports it needs for broad support.

 

The nice thing about ACS is that it's an established standard for map scripting - and by far the most likely candidate for getting accepted.

If you try to shoehorn in a general purpose scripting language you will most likely kill any momentum here because the ports are just too different to handle that. I also believe it's way out of scope for a baseline port to implement such a feature because it imposes far stricter limitations on implementation freedom.

 

 

 

[Ladna]

20 minutes ago, Graf Zahl said:

It's an even bigger step back if what you come up with remains yet another insular implementation - and that will happen if you try to do map scripting and game scripting with the same language.

This is simply untrue, as shown by the hundreds and hundreds of games that have a single, powerful, general purpose scripting language (google "games using Lua" and "games made with Unity").  The standard in industry is a single scripting language for everything.  Many games are even written entirely in a single scripting language.  The major barrier is piling languages (and text formats, for that matter) on top of each other because it requires so much work on the development side.

 

8 minutes ago, Graf Zahl said:

If you try to shoehorn in a general purpose scripting language you will most likely kill any momentum here

That's entirely fair, and while I'm not trying to be inflammatory, my position is that I'd rather there be no standard than a bad one because then you don't need to continue to support it, and you don't have a constant, "just use [the standard]" argument to everything you might try and do.

 

A good example is Zandronum's stats DB support.  They essentially had to build database bindings for ACS.  It's absolutely the case that even if a scripting language is only intended to be for very limited, simple map scripting, there will be significant pressure to use it for everything.  There's real-world examples right now.  There's a number of reasons, most of which have been brought up in all these threads ("ACS is a standard", "everyone already knows ACS", "ACS support already exists in [port x], don't add an entirely separate scripting engine for this 1 feature").

 

25 minutes ago, Graf Zahl said:

the ports are just too different to handle that.

You can't apply this argument to general purpose scripting language and not ACS.  It's just `actor.set_position(...)` everywhere.  Besides, the whole point of adding a scripting language is to smooth out those differences so you don't need to have multiple C/C++ implementations of an engine feature.

 

28 minutes ago, Graf Zahl said:

I also believe it's way out of scope for a baseline port to implement such a feature because it imposes far stricter limitations on implementation freedom.

This is also false.  If anything, "use ACS" is far more restrictive than "implement these APIs in [language x]".

 

===

 

Variously, people say, "Ugh Ladna, you're just trying to derail these standards threads", or here it's "you will most likely kill any momentum", and I guess I am?  I mean I guess if the thread ends up being "hey let's make ACS the standard" and I disagree with it, I'll keep arguing until someone convinces me.

 

But I've said pretty much everything I want to say, so I'll try and avoid that here and bow out.  Please don't make ACS the standard Doom scripting language.

[Linguica]

9 minutes ago, Ladna said:

Please don't make ACS the standard Doom scripting language.

But ACS has been the de facto "standard" Doom scripting language since, like, 1999.