New (cross-port?) scripting languages for Doom

[Jon]

Split from here

 

11 hours ago, kb1 said:

And, I wish the script stuff was in a different thread...

 

Me too, so here it is!

[Graf Zahl]

I'll be blunt here. It's ACS/FraggleScript or die, I really see no other option here.

 

Either you restrict yourself to the kind of superficial map scripting that Hexen invented or you have to deal with so many differing approaches to extend the engine that you'll never be able to mash it all together into one working implementation.

 

[kb1]

Thanks, Jon.

 

@Graf Zahl

I can't devote the right amount of mental energy to such a question at this point in time. I would think you'd have a much better chance standardizing a byte-code engine, than a language. And, yeah, that's what ACS is. ZDoom has thoroughly extended it as-is. The nice thing about byte-code is that, as long as you've left escape bytes in, you can always extend it. We could take a chapter from Microsoft's .NET approach (as much as I hate to say it), and have a common VM, programmed by any number of source languages. This requires insane planning though. It'd be a neat project. But my plate is overflowing, full. Your's too huh?

[chungy]

Has anyone tried to integrate Lua into a port?

[Albertoni]

27 minutes ago, chungy said:

Has anyone tried to integrate Lua into a port?

https://eev.ee/blog/2016/11/26/embedding-lua-in-zdoom/

 

Graf won't accept it though, for his own reasons.  I can imagine a few.

[Quasar]

EE already has ACS for mapping, will be adding AngelScript for advanced custom game logic, and may eventually support Doom 64 macros as well. I really don't see the need for something else when ACS already acts as a lingua franca for map scripting.

[zZaRDoZz]

2 hours ago, chungy said:

Has anyone tried to integrate Lua into a port?

Before EDGE 1.34, edge used Lua to control the heads up display. Eduke32 was switching to lua for scripting a while ago.

[Graf Zahl]

The only 'problem' with ACS is that it requires Hexen features to work as intended. For ports like ZDoom or EE that's not a problem, obviously, but for PrBoom it would, because a lot of code needs to be added to gain support.

And of course one should not discount the resistance of a certain type of mappers which grew up with the Doom map format and show no incentive to move beyond it. How else can one explain that these weird feature requests about 'scripting but no real scripting' come in?

 

But ultimately, all what got suggested is more work than just adding sufficient support for Hexen specials. The code for supporting those can be taken from several places - obviously I'd not alter the existing Doom special code at all but use the Hexen code in parallel, even if it duplicates some stuff.

 

(Sigh, I really need more time to work on PrBoom... :( )

 

 

[Lollie]

The call for introducing a new cross-port scripting language comes down to wanting something that is high-level. It only needs to be able to perform common Doom-related tasks in a way that is moderately approachable to a non-programmer. Importantly, a scripting language shouldn't require non-programmers to wrap their heads around C/C++ style programming syntax. It should benefit level designers first and foremost.

 

Hypothetically, if ports like PrBoom or Doom Retro (which currently lack support for map scripting) implemented support for high-level Doom scripting, along with ports like GZDoom and Eternity (which already support scripting languages like ACS), this could ultimately give map-makers more freedom to create maps, and give players more freedom to play maps, without anyone having to worry about whether said maps are compatible with their port of choice. The entry bar for creators and players still has some room left to be lowered.

[Graf Zahl]

Define High level!

 

Normally when digging deeper, what people mean by that is stupidly simplified syntax because it is presumed that average people are incapable of learning some basic programming concepts.

 

So here's the problem:

 

True high level coding comes with a lot of internal baggage. The more you want to abstract the mapper from the gory details, the more work needs to go into the VM/interpreter or whatever else is supposed to run the code and the more internal supporting code you need.

Have you ever asked yourself why most true high level languages come with some excessively bulky runtimes? The reason for this is precisely because that abstraction comes at a price. But for Doom mapping that's not going to work - you need something simple and small - and that also by necessity implies that you need to go to a lower level.

 

Lua may work, but it'd ultimately amount to reinventing the wheel again. If ACS is out due to Hexen map format requirements I'd rather take FraggleScript and remove some of the more problematic parts (e.g. string support which has massive memory leaking problems and only has some use for printing notification messages in a simpler port.) and actually fix some of the design problems in there.

 

 

 

[Gez]

1 hour ago, Lollie said:

The call for introducing a new cross-port scripting language comes down to wanting something that is high-level. It only needs to be able to perform common Doom-related tasks in a way that is moderately approachable to a non-programmer. Importantly, a scripting language shouldn't require non-programmers to wrap their heads around C/C++ style programming syntax. It should benefit level designers first and foremost.

That makes no sense to me. I'm sorry, but it really doesn't.

 

 

First, C/C++ style programming syntax is at its core extremely simple:

int function doSomething(int a, int b)
{
	return a + 2 * b;
}

script 1 (void)
{
	doSomething(4, 7);
}

I'm sorry but if you can't understand that syntax, you can't understand any scripting or programming syntax at all, ever. We're not talking about C/C++ programming here, just the syntax. C/C++ syntax is basically defined as "semi-colon after instructions, curly braces around blocks, parentheses around parameters, commas between parameters". There's nothing arcane or strange about that.

 

Secondly it's important to keep in mind that simplicity and complexity are both sides of the same coin. If you want to make something that is very simple to grasp, it will also be very limited, meaning that making advanced things with it will be very complex. Inversely, a language designed to make it simple to do advanced stuff will be very complex. Certainly if you start talking about macros, templates, lambdas, factories, etc. non-programmers will quickly be lost, but nevermind that stuff, none of that is relevant to something like ACS.

 

Thirdly, the whole "level programming should be easy for non-programmers" sounds to me like saying that game art should be easy for non-artists.

[Altazimuth]

Is the language of ACS inherently tied to Hexen? Would it not be feasible to use ACS but just have a different "[foo]common.acs" for this new standard, that could maybe have Boom specials instead of Hexen ones?

[Graf Zahl]

10 minutes ago, Altazimuth said:

Is the language of ACS inherently tied to Hexen?

Not to Hexen, but to a few of its features which Doom does not have:

 

- parameterized action specials

- thing IDs

- thing specials

 

Of course you can try to use Boom specials but  that'd ulitmately be self-defeating because they are too limited.

I think in such a case duplicating the major thinker structures (or at least the initializer functions) with Hexen-format compatible versions (i.e. doors, floors, ceilings ans platforms) to allow all specials to work is a better approach.

But the main problem is that the Doom map format is missing some fields so you'll also need a new map format, be it Hexen or UDMF.

Nothing of this is a huge show stopper but it requires some work.

 

35 minutes ago, Gez said:

Secondly it's important to keep in mind that simplicity and complexity are both sides of the same coin. If you want to make something that is very simple to grasp, it will also be very limited, meaning that making advanced things with it will be very complex.

 

I can only second that. A former boss of mine was a proponent of such approaches. It was always the same with his designs: It was exceedingly simple to get a quick mock-up working, but then doing the real work afterward always devolved into a catastrophic mess with bug-ridden code and an endless string of problems because everything was designed to hide the actual complexity, leading to situations where you had to work against the engine design to do the needed work.

 

 

[Lollie]

On 7/1/2017 at 8:06 PM, Graf Zahl said:

Define High level!

 

Normally when digging deeper, what people mean by that is stupidly simplified syntax because it is presumed that average people are incapable of learning some basic programming concepts.

That's what I mean. Simple. You might hate it as a programmer, but artists love that layer of abstraction — it's why tools like Game Maker Studio or "PlayMaker" for Unity have ended up being so popular. (No, I'm not suggesting visual scripting for Doom maps. ...It sounds nice though, shit.)

 

In Doom's case, a scripting language wouldn't need to do half of what ACS is capable of doing, it just needs some basic "Doom Level"-specific conditions and actions. eg: Perform Action [X] when Sector [Tag ID] has completed an action. Tell Linedef [Tag ID] to change [Property] to [X] when Thing [TID] is dead. Spawn explosions/bullet puffs/blood/etc along Linedef. End the level after a duration of [Y] ticks.

 

On 7/1/2017 at 9:46 PM, Gez said:

If you want to make something that is very simple to grasp, it will also be very limited, meaning that making advanced things with it will be very complex.

That's okay! It's perfectly fine to have a scripting language that can only do a limited set of actions. You could take FraggleScript, strip out all the stuff about handling cameras, HUDs, etc, limit it entirely to functions that are specific to levels and in-game objects, and it'd still be able to handle a lot of basic tasks. It'd be nice to be able to just take a linedef and plug in a couple basic lines when needed.

 

It doesn't need to be simple and powerful. Simple is enough. ACS isn't disappearing any time soon — If someone needs more advanced capabilities, then that is what ACS should exist for.

 

Edit: Most of my misgivings came down to the tools available, plus a misunderstanding about the scope of use that ACS is intended for. Sorry! Glad to see discussion continued despite my misinformed pot-stirring.

Edited July 10, 2017 by Lollie : womp womp

[Ladna]

On 6/28/2017 at 10:36 AM, Quasar said:

Kaiser proved in his Kex 2 project that the VM would totally break down once too many objects were created, just because it started taking too long to search the GC chains for memory to free.

The main thing here is to do as much as you can in the engine and provide only an interface in the scripting API.  You're right the more shit you store in the VM the more it will slog, so don't do that.  Boxing is the enemy.  Or:

 

class Actor:
  
  def __init__(self, god, help, me, etc):
    self.god = god
    self.help = help
    self.me = me
    
def main():
  ac = Actor('a', 'b', 'c')

Don't do that.  Do this instead:

 

from doom import Actor

def main():
  ac = Actor('a', 'b', 'c')

 

Or you can use a language/runtime with a different GC model.  Python uses refcounting, which is more deterministic and controllable, for example.

 

On 6/28/2017 at 10:36 AM, Quasar said:

While newer versions of SpiderMonkey are still available, they have multiple issues, such as only being able to be compiled with the enormous and unruly Mozilla build system, and no longer offering a stable API for embedding. To build Chrome's JS engine instead you need Python of all things so, at that point you may as well use Python instead.

Yeah but you don't necessarily need to ship Python, you just need it installed for the build system... which I agree is aggravating but I guess it takes hours to compile Chrome on a single machine so I can see why they would try insane things.

 

There are a couple other engines besides v8 and SpiderMonkey; the "in-vogue" ones are Duktape, MuJS, and v7 (har har).  Duktape and MuJS use the stack-based API that I remember you don't like from Lua, but v7 looks pretty dynamic in its API.  Honestly the stack-based thing has been no problem at all; I basically have a single file that wraps these things and I never think about it.  You rarely go further than 3-4 deep in the stack anyway so it's not like a cognitive burden besides.

 

The main thing that pulled me off JS (I re-evaluated recently) is the lack of libraries.  It's just hard to find them because there's not really a standard embedded JS engine.  SpiderMonkey is close, but even then there aren't, say, database libraries.

 

My $0.02 is "look at industry and see what they do".  Some games have embedded Python, some games have invented their own language or their own toolchain (Sucker Punch Studios, QuakeC), but I think the standards now are basically C# (Unity) or Lua.  I'm kind of OK with either; I think C# is a little boring and offputting to modders but meh.

 

I don't personally think this is an interesting discussion, either in terms of tech or philosophy.  Like, if this thread ends up being about punctuation vs. words, semicolons vs. line breaks, types vs. no types, then it's a huge waste of time.  Let's focus on some basic stuff:

• Is a given solution technically feasible (fast enough, powerful enough, sandboxable)
• Is it a pain in the ass
• Have people used it before (sure, ACS and Fragglescript count)
• Is it extensible, i.e. are there libraries for it
• Can ports use it (PrBoom+ is still C, so no giant C++ JS engines or C++ binding generators)

I'm not saying these are requirements; I'm saying they're the things we should weigh.

[Graf Zahl]

16 minutes ago, Lollie said:

In Doom's case, a scripting language wouldn't need to do half of what ACS is capable of doing, it just needs some basic "Doom Level"-specific conditions and actions. eg: Perform Action [X] when Sector [Tag ID] has completed an action. Tell Linedef [Tag ID] to change [Property] to [X] when Thing [TID] is dead. Spawn explosions/bullet puffs/blood/etc along Linedef. End the level after a duration of [Y] ticks.

 

If that's what you want, go back to Gez's post and try to understand what he was saying about simplicity and complexity being two sides of the same coin.

You can do that by defining complex actions you need to invoke or by splitting those complex actions into their constituents (i.e. one command for 'check condition X' and another for 'perform condition Y'.) That's what ACS does. The first command is called 'if' and the second is a long list of possible actions.

 

Or you can do it your way. This involves a lot of native coding to descibe complex actions and in the end it's still not guaranteed to provide what you need.

In terms of existing features, you propose SBARINFO while ACS proposes ZScript to code status bars. And now try to find any SBARINFO script that wants to do more complex and more involved stuff. You'll find across the board that these things are gargantuan clusterfucks of code because  the more complex commands it provides make it very hard to do fine grained checks for specific conditions.

 

 

16 minutes ago, Lollie said:

That's okay! It's perfectly fine to have a scripting language that can only do a limited set of actions.

 

No, it's not. That may be fine for YOU but the next person will ask "Why can't I do XYZ? Please implement it!"

Take Doom64 macros. It's simple, it's straightforward but it's also very limited. It's good for people who know nothing about programming, but as soon as your needs become more complex, it will fail. Even doing a two switch setup to trigger an action will be hard if you do not have status variables, for example.

 

 

16 minutes ago, Lollie said:

You could take FraggleScript, strip out all the stuff about handling cameras, HUDs, etc, limit it entirely to functions that are specific to levels and in-game objects, and it'd still be able to handle a lot of basic tasks. It'd be nice to be able to just take a linedef and plug in a couple basic lines when needed.

 

Yes, but most of what you want to take out is not the language but external functions. All the language defines is "Give me a list of functions the mapper should be able to use."

 

 

16 minutes ago, Lollie said:

 

It doesn't need to be simple and powerful. Simple is enough.

The typical misconception of a non-user. Trust me, it's not enough.

 

[Lollie]

5 minutes ago, Graf Zahl said:

No, it's not. That may be fine for YOU but the next person will ask "Why can't I do XYZ? Please implement it!"

32 minutes ago, Lollie said:

ACS isn't disappearing any time soon — If someone needs more advanced capabilities, then that is what ACS should exist for.

 

[Gez]

But the thing here is that ACS is simple.

[printz]

1. ACS the language is fine. But we should get rid of the need of compilation. It just feels like something that shouldn't be in 2017. Just run the wad with the script in text format. Let the port do the compilation!

 

2. Of course the Doom format may be handled in ACS! You just have to use something like this:

StartSpecial(special, tag)

Doesn't FraggleScript already have something like this? It's obviously less expressive than named Hexen specials, but it can still be managed if you have named constants for the "special" parameter. Of course the UDMF format is superior and should be used instead, but I've simply illustrated this for completeness' sake.

 

3. We already have non-programmer "visual" scripting, in the form of Boom conveyors and fake players. So that problem is already solved.

 

4. The idea of using general purpose languages like Python, third-party VMs and depending on overly verbose class definitions makes me sick. Python classes are ugly.

 

5. I expect advanced inventions like ZScript to also be usable in levels, as possible replacements for ACS.

 

6. If you're a mod author and you've been feeling limited by the features agreed by the port author: isn't ZScript there to the rescue to let you modify the behavior of Doom as you want? That problem ought to be solved as well.

[Graf Zahl]

27 minutes ago, printz said:

1. ACS the language is fine. But we should get rid of the need of compilation. It just feels like something that shouldn't be in 2017. Just run the wad with the script in text format. Let the port do the compilation!

 

I generally concur, it's just too bad that ACC is a) a really shitty piece of code (and compilation depends on its shitty design) and b) comes with a useless license.

 

 

27 minutes ago, printz said:

 

2. Of course the Doom format may be handled in ACS! You just have to use something like this:

StartSpecial(special, tag)

Doesn't FraggleScript already have something like this? It's obviously less expressive than named Hexen specials, but it can still be managed if you have named constants for the "special" parameter. Of course the UDMF format is superior and should be used instead, but I've simply illustrated this for completeness' sake.

 

That surely can be done. What also can be done is porting FraggleScript's sector movement functions to have something more meaningful. But still... I'd prefer to do it right.

 

 

27 minutes ago, printz said:

 

3. We already have non-programmer "visual" scripting, in the form of Boom conveyors and fake players. So that problem is already solved.

 

Heh. I never thought about looking at it from that angle. ;)

 

27 minutes ago, printz said:

 

4. The idea of using general purpose languages like Python, third-party VMs and depending on overly verbose class definitions makes me sick. Python classes are ugly.

 

For map scripting this clearly isn't needed. Of course things are different if you want to extend internal classes with newly coded features.

 

27 minutes ago, printz said:

 

5. I expect advanced inventions like ZScript to also be usable in levels, as possible replacements for ACS.

 

Not really. ACS can delay a script. ZScript (or any other external VM for that matter) just execute code but cannot suspend execution in the middle of a function. That makes it a bit awkward to do the proper coding. It surely is possible but far less straightforward than using ACS.

 

27 minutes ago, printz said:

 

6. If you're a mod author and you've been feeling limited by the features agreed by the port author: isn't ZScript there to the rescue to let you modify the behavior of Doom as you want? That problem ought to be solved as well.

 

You cannot override everything. The actual movement code would simply be too slow if run in a VM.

 

[Albertoni]

36 minutes ago, printz said:

ACS the language is fine. But we should get rid of the need of compilation. It just feels like something that shouldn't be in 2017. Just run the wad with the script in text format. Let the port do the compilation!

To my eyes, pre-compilation only gives advantages. It at the very least forces you to provide code that will compile before running your map.

[Graf Zahl]

Precompilation only gives disadvantages with one exception, namely the syntax check is being performed in the editor instead of in the engine when the map starts. But its biggest drawback is that a modder may hide their code. Just have a look at RTC-3057 as a good example of a mod that exhibits some problems but cannot be fixed because its creators thought it was a good idea to withhold the source.

 

[Jon]

18 hours ago, chungy said:

Has anyone tried to integrate Lua into a port?

 

Asides from eevee¹, AlexMax did an early attempt https://github.com/AlexMax/luadoom and I hope can be persuaded to return to the idea some day.

 

¹ who, as a side note, I find a really interesting but also perplexing character. They clearly care about Doom, or perhaps just ZDoom, but I've had absolutely zero interaction with them ever. I don't think they've ever posted on DWF and the few times I've tried to interact on twitter I've just been ignored.

[VGA]

UMAPINFO and Hexen map support in prboom+ and other ports would allow mappers much more freedom than vanilla or Boom.

 

Is it hard to add support for the Hexen map format?

[Albertoni]

18 minutes ago, VGA said:

Is it hard to add support for the Hexen map format?

It's not hard depending on the internal workings of the port, but the amount of work is pretty huge.

[Coraline]

11 hours ago, printz said:

1. ACS the language is fine. But we should get rid of the need of compilation. It just feels like something that shouldn't be in 2017. Just run the wad with the script in text format. Let the port do the compilation!

 

 

 

;-)

[Blastfrog]

On 6/30/2017 at 9:17 AM, Graf Zahl said:

I'll be blunt here. It's ACS/FraggleScript or die, I really see no other option here.

/thread

 

13 hours ago, Albertoni said:

To my eyes, pre-compilation only gives advantages.

I don't see how. The only ones I can think of is smaller storage size and faster level bootup. Everything else about it is unfortunate, that source can be lost is one.

 

13 hours ago, Albertoni said:

It at the very least forces you to provide code that will compile before running your map.

The way I see it, it's not the engine's responsibility if one feeds it broken content. Besides, if the engine can't compile the script on runtime due to an error in it, it would probably inform you, either with a note in the console log or outright crashing.

 

15 hours ago, Graf Zahl said:

No, it's not. That may be fine for YOU but the next person will ask "Why can't I do XYZ? Please implement it!"

My thoughts exactly when reading Lollie's posts in this thread. If I want to use scripting in my map, I want real and direct control.

 

16 hours ago, Lollie said:

You might hate it as a programmer, but artists love that layer of abstraction — it's why tools like Game Maker Studio or "PlayMaker" for Unity have ended up being so popular.

ACS is simpler to work in than those programs. Not sure what it is that makes you think that ACS is somehow a barrier to entry, because it's not.

 

[Jon]

8 hours ago, VGA said:

UMAPINFO and Hexen map support in prboom+ and other ports would allow mappers much more freedom than vanilla or Boom.

 

Is it hard to add support for the Hexen map format?

 

Wrong thread dude!

[Gez]

You can use ACS without the Hexen map format, but you'll be limited in that there are some features you won't be able to use easily, notably anything that uses a thing ID, since you won't be able to give TIDs to things in the map editor.

[printz]

Doesn't ExtraData provide thing ID and args? Even in GZDoom.