ACC++ Project Announcment

[Blzut3]

So I've been tossing the idea of a new ACS compiler around for a year now and just recently I've actually started work on it. (http://skulltag.net/ViewVC/accpp/) I've decided it's time to make a formal announcement because I'm almost constantly being asked what it's all about and I figured I should get input from the Doom community outside of selective IRC channels.

So first off what is ACC++? As the name suggests it is a new from scratch ACS compiler that will support some basic OOP. The idea came around when Apothem was messing with global arrays in ZDoom and noted that their ever expanding nature can be exploited to create a database or “plugin system.” With this I formulated that it might just be possible to create implementations of malloc() and free() from within ACS. From there all that would be needed is an advanced compiler.

Basically ACC++ will be a drop in replacement for ACC, hopefully having a Hexen compatibility mode (of course not all features will be available). Along side ACC++ will be an ACS library which will provide an extra functionality needed (such as malloc() and free()) as such no changes will be necessary to any port which supports ZDoom's ACS extensions. In addition ACC++ will have a more advanced preprocessor with features similar to that of the C preprocessor. The fact that it is merely a replacement for ACC means that ACC++ will not make anything new technically possible, just hopefully make certain things easier to do.

I haven't completely decided on all the features I want to implement into the compiler. I do know for sure basic structs will be possible and that the compiler will do stronger type checking than ACC. One idea I'm currently tossing around is implementing an assembly like language so that any “built-in” functions can be added to the common.acs header files and be used without compiling new versions.

Why do I want do this? First off it will solve any licensing concerns over ACC. Currently I'm using the BSD license on the source I have written thus far. The only library I have plans to use right now is the standard C++ library so it may be possible for source ports to pull in ACC++ for run time compilation. Another reason I'm doing this is simply for the experience in (and the “fun” of) writing a compiler.

Some questions for people who have worked on ACC and/or source ports: is there anything done wrong in ACC that I should know about so that I may avoid the same mistake? For source port authors, would this project affect your project in any way (run time compilation or support for ZDoom ACS scripts (if you don't support them already) for example)? Obviously some strong ACS scripters are excited about what I have announced for the project, but I feel it's worth asking: do you think ACC++ would be a waste of time?

[Graf Zahl]

Blzut3 said:

Basically ACC++ will be a drop in replacement for ACC, hopefully having a Hexen compatibility mode (of course not all features will be available).

What do you mean by that? Do you mean it may not compile all scripts made for Hexen or that it just can't create 100% identical bytecode?

Along side ACC++ will be an ACS library which will provide an extra functionality needed (such as malloc() and free()) as such no changes will be necessary to any port which supports ZDoom's ACS extensions. In addition ACC++ will have a more advanced preprocessor with features similar to that of the C preprocessor. The fact that it is merely a replacement for ACC means that ACC++ will not make anything new technically possible, just hopefully make certain things easier to do.

Ok, here's a suggestion. Don't limit yourself this way. Create an ACC++ branch of ZDoom and if you think you absolutely need some new stuff just add it. Free and malloc, for example sound like they should be implemented on the engine side, not in scripts.

Some questions for people who have worked on ACC and/or source ports: is there anything done wrong in ACC that I should know about so that I may avoid the same mistake?

It depends how you want to create your parser. ACC's biggest shortcoming is its limited one-pass compiler. It's just too restrictive and prone to problems.

For source port authors, would this project affect your project in any way (run time compilation or support for ZDoom ACS scripts (if you don't support them already) for example)?

Run time compilation support has been on my wishlist since forever but the original ACC source is just too bad for that - so I wholeheartedly appreciate your project.

Obviously some strong ACS scripters are excited about what I have announced for the project, but I feel it's worth asking: do you think ACC++ would be a waste of time?

Most definitely not.


Also: Why not host this on the ZDoom SVN? It think it'd be a better place for it.

[Blzut3]

Graf Zahl said:

What do you mean by that? Do you mean it may not compile all scripts made for Hexen or that it just can't create 100% identical bytecode?

What I mean is I hope to have a Hexen bytecode compatibility mode, much like ZDoom's ACC. Of course this means that OOP won't be possible. So in essence the compiler will hopefully be able to switch between ACS and ACS++ (depends on how well I do on designing it).

That said I don't guarantee all regular ACS scripts will compile as-is, much like not all C code compiles in C++.

Graf Zahl said:

Ok, here's a suggestion. Don't limit yourself this way. Create an ACC++ branch of ZDoom and if you think you absolutely need some new stuff just add it. Free and malloc, for example sound like they should be implemented on the engine side, not in scripts.

Actually I don't see it as much of a limitation. I'm about 90% certain malloc() and free() can be implemented in ACS (Apothem claims to have something working), and besides those two functions the only other things I could see in LIBACSPP is generic classes for things like dynamic strings.

Of course I'm thinking LIBACSPP should serve as a reference implementation and could be optionally "accelerated" through native functions. Kind of like how you could write a software implementation of the OpenGL library. In any case I'll keep this in mind in case I come to a point where an extension to ACS is needed, but right now the only thing I can think of is possibly removing hard limits. (I seem to recall seeing a limit on the number of functions in the ZDoom source which would be the only thing stopping full OOP.)

Graf Zahl said:

It depends how you want to create your parser. ACC's biggest shortcoming is its limited one-pass compiler. It's just too restrictive and prone to problems.

This is something I'm going to have to do a little bit of research on. I'm not all too sure what advantages doing multiple passes will serve to ACS. (I'm assuming the preprocessing stage is not considered a pass.) A glance at wikipedia tells me that one advantage to multi pass is to be able to use things defined later in the file at earlier point, but I certainly plan on at the very least having forward declarations as in C(++) for this.

Graf Zahl said:

Also: Why not host this on the ZDoom SVN? It think it'd be a better place for it.

The reason I chose to host it on skulltag.net is actually because I wanted to host it on my own website (which are currently hosted by drdteam.org as well). The svn services just happen to be on skulltag.net so it was just convenient to put it there. I also have ssh access to the server so I'm able to perform maintenance on the repository myself (such as adding post-commit scripts to update sites such as cia.vc). Is there any specific reason why you feel it would be better in the ZDoom repository?

[boris]

Graf Zahl said:

Ok, here's a suggestion. Don't limit yourself this way. Create an ACC++ branch of ZDoom and if you think you absolutely need some new stuff just add it. Free and malloc, for example sound like they should be implemented on the engine side, not in scripts.

If changing the engine code is an option, wouldn't it be better to ditch ACS and put in something like LUA?

[Gez]

boris said:

If changing the engine code is an option, wouldn't it be better to ditch ACS and put in something like LUA?

Heh. Ask Enjay about football, and Graf about Lua.

[Graf Zahl]

boris said:

If changing the engine code is an option, wouldn't it be better to ditch ACS and put in something like LUA?

Changing the engine and implementing yet another script interpreter are two fundamentally different things.

[Quasar]

Personally I think if it can't compile normal ACS byte code for the normal standard interpreter it loses a bit of its usefulness.

Eternity does not currently support ZDoom's custom bytecode format and I don't know when or if it ever will. I have no problem with those things being usable, but I think also supporting the normal format would be a good idea, because it's the base standard that everything, including Hexen itself, supports.

[Graf Zahl]

I think the problem with that is that many of the advanced features planned here need the ZDoom extensions to work.

[Blzut3]

Quasar said:

Personally I think if it can't compile normal ACS byte code for the normal standard interpreter it loses a bit of its usefulness.

As do I. The ACS mode is a goal, but the deciding factor will be how well I do at designing the compiler (for example I need to make sure I can disable the OOP stuff).

Quasar said:

Eternity does not currently support ZDoom's custom bytecode format and I don't know when or if it ever will. I have no problem with those things being usable, but I think also supporting the normal format would be a good idea, because it's the base standard that everything, including Hexen itself, supports.

As a side effect of doing this project I'll be able to provide documentation (out side of the source code) of ZDoom's custom byte code. So one of my questions was would a project like this be able to convince you to add ZDoom ACS support? As Graf said some of ZDoom's extensions will be needed in order to provide the features of ACS++ (basically the only thing I can see ACC++ providing at this point for ACS is the advanced preprocessor).

[Quasar]

Blzut3 said:

As a side effect of doing this project I'll be able to provide documentation (out side of the source code) of ZDoom's custom byte code. So one of my questions was would a project like this be able to convince you to add ZDoom ACS support? As Graf said some of ZDoom's extensions will be needed in order to provide the features of ACS++ (basically the only thing I can see ACC++ providing at this point for ACS is the advanced preprocessor).

Possibly so. I found the ZDoom-native documentation of the format to be pretty dense, and the format has a lot of things that will not ever be useful to Eternity, such as localization support with Unicode-capable strings (EE is 7-bit ASCII only and uses chars > 127 as control characters).

If there are simple docs or even example code that can help me skip over such things I don't need without knowing every nitty gritty detail of them, I shouldn't have any problem.

[Graf Zahl]

Quasar said:

Possibly so. I found the ZDoom-native documentation of the format to be pretty dense, and the format has a lot of things that will not ever be useful to Eternity, such as localization support with Unicode-capable strings (EE is 7-bit ASCII only and uses chars > 127 as control characters).

There is no unicode support in ZDoom. It just uses all 256 values of the 8 bit character set.

Why are Americans always this blind to the needs of international users? The amount of crappily written software that just can't think out of the box is just driving me up walls. I have lost count of the stuff that just doesn't work right on my system because it assumes that:
a) all keyboards are American QWERTY, no questions asked
b) no language uses characters outside 7bit ASCII. (Ever thought about why it's called 'ASCII'?)

Sadly all Doom ports except ZDoom miserably fail the first of these (thanks in no small part to the abomination called SDL.) The second is more understandable when it comes to game graphics - but in the console?

And sorry to say but designing any software to be '7 bit ASCII' today is as bad as developing for DOS 'because it's all my application needs'.

[/rant]

[RestlessRodent]

Graf Zahl said:

There is no unicode support in ZDoom. It just uses all 256 values of the 8 bit character set.

Why are Americans always this blind to the needs of international users? The amount of crappily written software that just can't think out of the box is just driving me up walls. I have lost count of the stuff that just doesn't work right on my system because it assumes that:
a) all keyboards are American QWERTY, no questions asked
b) no language uses characters outside 7bit ASCII. (Ever thought about why it's called 'ASCII'?)

Sadly all Doom ports except ZDoom miserably fail the first of these (thanks in no small part to the abomination called SDL.) The second is more understandable when it comes to game graphics - but in the console?

And sorry to say but designing any software to be '7 bit ASCII' today is as bad as developing for DOS 'because it's all my application needs'.

[/rant]

Hey, I am not blind (even though I am blind), ReMooD supports internationalization to display all those glyphs and all (Not using ASCII letters, you can use any letter you want but if there's no picture for it a placeholder is placed in, and no don't say "You'd have to create every glyph for every accent attached to a character" because I got that handled too, pretty cool actually). You can kinda type in Unicode but it gets snipped (& 0xFF) because the console doesn't fully support it yet. But I am working on it! I just need translators and nobody really steps up to translate anything, but a Russian buddy did translate some things to Russian so I played ReMooD in Russian for a bit heh. However, I couldn't help a few people coming into my channel not speaking English being stuck behind a bad Google Translator.

But the topic is derailing.

If you are making an OOP implementation of ACS and if you are OOP crazy then you'd probably prefer a new and delete over something like malloc() and free() (since malloc() and free() are evil things supposidly).

Also it seems that you are writing a compiler that removes the limits of a system for a system that limits what you are trying to break (naturally). Unless you wrap up bytecode into other pieces then you could do such a thing but it would increase overhead and compiled code complexity and the complexity of the compiler.

Also ACS isn't just a ZDoom thing, it's from Hexen. Even though Hexen is super old and all, you can't forget about that. Your compiler would be more useful if it could compile existing ACS code and your new ACS++ code. People suggested that I use ACS instead of what I'm currently planning to do for ReMooD since it's already done but ACS is pretty limited and was really only intended for Hexen so it's expansionability is pretty limited.

[Blzut3]

GhostlyDeath said:

If you are making an OOP implementation of ACS and if you are OOP crazy then you'd probably prefer a new and delete over something like malloc() and free() (since malloc() and free() are evil things supposidly).

Why do you think I'm writing the compiler? Basically new and delete will just be a simple way to call malloc() and free(). Take this tentative ACS++ code:

class MapSpot
{
    fixed x;
    fixed y;
    fixed z;

    function void moveTid(int tid)
    {
        setactorposition(tid, x, y, z, 0);
    }
}

script 1 (void)
{
    MapSpot *obj = new MapSpot();
    obj->x = 1234;
    obj->y = 5678;
    obj->z = 0;
    obj->moveTid(0);
    delete obj;
}

Would simply translate to the following ACS code (of course the ACC++ compiler will go straight from the previous to byte code, but for the sake of discussion lets look at the equivalent ACS):

// LIBACSPP will automatically be imported in ACS++ projects
#import "LIBACSPP"

function void MapSpot_moveTid(int ptr, int tid)
{
    setactorposition(tid, g_Heap[ptr], g_Heap[ptr+1], g_Heap[ptr+2], 0);
}

script 1 (void)
{
    int obj = malloc(3); // Everything in ACS is an int
    g_Heap[obj] = 1234.0;
    g_Heap[obj+1] = 5678.0;
    g_Heap[obj+2] = 0.0;
    MapSpot_moveTid(obj, 0);
    free(obj);
}

GhostlyDeath said:

Also it seems that you are writing a compiler that removes the limits of a system for a system that limits what you are trying to break (naturally). Unless you wrap up bytecode into other pieces then you could do such a thing but it would increase overhead and compiled code complexity and the complexity of the compiler.

I can't make much sense out of what you just said. All I'm doing is writing an ACS library and then providing a compiler that allows you to work with the library in a easy to read format. Hence why it shouldn't require any changes to ZDoom. The only thing I need are some of ZDoom's already existing extensions (I don't think Hexen had support for arrays, functions, or libraries for example).

GhostlyDeath said:

Also ACS isn't just a ZDoom thing, it's from Hexen. Even though Hexen is super old and all, you can't forget about that. Your compiler would be more useful if it could compile existing ACS code and your new ACS++ code. People suggested that I use ACS instead of what I'm currently planning to do for ReMooD since it's already done but ACS is pretty limited and was really only intended for Hexen so it's expansionability is pretty limited.

As I said before I plan on having the equivalent to ZDoom's ACC -h parameter. The only thing that would stop it is if I suck at designing the compiler.

[DaniJ]

Just wanted to point out our plans with Doomsday regarding scripting for anyone interested:

Support for ACS is currently being completely redesigned. In Doomsday 2.0 there will be no runtime interpreter for ACS bytecode. Instead we have an ACS bytecode to Doomsday script source interpreter and a JIT Doomsday script compiler. So for example, running Hexen will result in interpretation of BEHAVIOR lumps to Doomsday script source files which are written to the map cache along with the rest of the map data. During map load, Doomsday script sources are read from the cache and compiled.

As far as we are concerned ACS as it stands today is a design dead end. Yes other ports (primarily) ZDoom have extended the functionality of ACS extensively but its still not a good general purpose language. For Doomsday we wanted one uber scripting language capable for use everywhere, from mobj AI to the HUD/UI and interface layer.

Although this ACC++ is an interesting project I will point out that at this time deng team has little interest in supporting it in Doomsday.

[Graf Zahl]

DaniJ said:

As far as we are concerned ACS as it stands today is a design dead end. Yes other ports (primarily) ZDoom have extended the functionality of ACS extensively but its still not a good general purpose language. For Doomsday we wanted one uber scripting language capable for use everywhere, from mobj AI to the HUD/UI and interface layer.

I'd say we are in full agreement here. The language and the compiler flat out suck and once ZDoom's scripting is working I had plans to do the same thing there (also for Fragglescript in GZDoom.) Of course it's still a long way to go...

Although this ACC++ is an interesting project I will point out that at this time deng team has little interest in supporting it in Doomsday.

If you make a cross-compiler for Hexen bytecode as a plugin another one for ZDoom bytecode shouldn't be that big a task, should it? The only thing that'd need to be added would be array support. The rest is more or less trivial.

[Quasar]

Let me know when you add all the requisite accents, umlauts, ogoneks, ligatures, etc. required to support extended ASCII in a format that can be used by any DOOM port and looks good alongside all those wads with custom font replacments and maybe I'll see the purpose to having extended ASCII in a DOOM port.

Until then, I see it as a complete and utter waste of time.

EE should support international keyboard mapping so long as it is set to use SDL's "unicode" system, which is the default setting. This is totally unrelated to allowing extended characters in game strings.

[Graf Zahl]

Quasar said:

Let me know when you add all the requisite accents, umlauts, ogoneks, ligatures, etc. required to support extended ASCII in a format that can be used by any DOOM port and looks good alongside all those wads with custom font replacments and maybe I'll see the purpose to having extended ASCII in a DOOM port.

What about the console? ZDoom's console font is the full Western European code page, for example. Since I can type the characters in the console I would like to see them on screen.

Quasar said:

EE should support international keyboard mapping so long as it is set to use SDL's "unicode" system, which is the default setting. This is totally unrelated to allowing extended characters in game strings.

Well, it's apparently not the default. How can I set that?

[Gez]

The Eternity SVN builds I compile do support international keyboards. I've just tested the latest.

[DaniJ]

Graf Zahl said:

If you make a cross-compiler for Hexen bytecode as a plugin another one for ZDoom bytecode shouldn't be that big a task, should it? The only thing that'd need to be added would be array support. The rest is more or less trivial.

Yes implementing another cross-compiler for ZDoom bytecode shouldn't be much more work. However that will happen over time along with their associated features as its a can of worms I'd rather ignore for now at least.

[killingblair]

From a user standpoint, this is heaven-sent. I come from a rather OOP standpoint, as when I started out, I went straight for C++, and to this day my languages of choice are C# and Java. I would often get frustrated at ACS for its current limitations, lack of expandability, and general clunkyness of the code.

I can't wait to adopt ACS++.

[Xaser]

Heh, just found out about this. Spot on, I say -- being able to actually do some slight OOP-ish stuff would really help clean things up. It's amazing how quickly scripts can become jumbled up thanks to some of the copypasta required to repeat basic functions.

I feel that I must ask, though: Will this allow for real support of strings or is that a limitation within the byte code format that can't be changed on the compiler-side of life? Probably super-wishful thinking, but it's worth an ask.

[Blzut3]

There's nothing in the byte code preventing dynamic strings. In fact ACS does already have them. They problem is they can't be used as the argument for functions (dynamic strings and static strings will never be completely compatible), and until ACS++ is standardized there is no practical way for the engine to "return" a string. I'm of course referring to character arrays. A CString class will probably be a standard feature in LIBACSPP. (Good discussion question right here, what classes/functions should I include in the standard library?)

Once ACS++ is "standardized" it wouldn't be impossible for the engine to return a pointer to CString (or write to previously set up buffer), however at this point I don't think it is likely that such extensions will happen.

While I'm at it, project update! I've started what I call the "ACC++ Manual" which currently contains full documentation of the format of the BEHAVIOR lump, including a brief description of all of the pcodes. Here's a link to a snapshot of the working copy in PDF form: http://maniacsvault.net/manual.pdf (Note: this is not a permanent link.)

[Xaser]

Ah, foot. Not being able to use dynamic strings in functions shoots down a lot of potential uses for them (well, a few I had in mind at least). Still, it's one of those things that, intuitively, seems like it should be possible -- ACC's handling of strings is one of its biggest kludges, and not even simple things like concatenation are doable. Still, not a super-high priority for this project, but maybe something to think about later on once the implementation grows solid.

[solarsnowfall]

Agreed. Simple string concatenation has been on the collective wish list of a great many end-users since time and memorial. It would be a pity to come out on the other side of this and still have to employ shitty hackery to get around this shortcoming. Not that saying that makes implementation any easier...

Oops, bumped the thread. Eh, I don't make it around that much anymore...

[boris]

Is this project still alive?

[Blzut3]

Yes, I'm just busy with other stuff at the moment. In regards to LIBACSPP, Apothem said that malloc/free was the next thing on his to do list.