Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
axdoomer

Mocha Doom is not dead

Recommended Posts

Hi everyone, I just want to say that Mocha Doom is not dead. The work on Mocha Doom has been continued here.

 

This is an announcement so that people are aware of what the project as become, although I really don't know if anyone is still using this port.

 

I've been working on it in my free time and it has progressed, albeit slowly. Other people have contributed now that's it's easier to make changes using git, GitHub makes the whole task easier for people to ask for pull request of their changes. People are welcome if they want to fix bugs and add functionalities that are missing.

 

It started back in March 2015 when the project had been dead for a while. Since then, I have fixed a crash that prevented from completing Doom II, a bug that prevented from using Freedoom as the IWAD and more. There have been major contributions to the renderer made by another user who is known as Good Sign. I have also been cumulating changes made by other people on GitHub in their own independent repositories, because I wanted mine to be the most up-to-date.

 

What I would like to do is complete the project by making it stable (removing bugs and other issues as much as possible), adding the multiplayer which is the most important part which is missing and probably adding support for Boom formated levels (which would make a wider range of levels playable).

 

Contributions from anyone are welcome as long as it remains in line with the philosophy of the project, which is being compatible with vanilla Doom. As a reminder, it's written in Java and runs fine on Windows and Linux.

Share this post


Link to post

Say something about that other feature, man... :) I'm pulling my hair out...

Share this post


Link to post

C/S would be the best, I'd check on how Chocolate-Doom is doing it because I must keep vanilla compatibility. I see it as a challenge that I would like to do, I have never written such network code before and it would be great for my knowledge. There are a lot of things to take in consideration when writing network code and I will learn "on the job".

 

@kb1 I'd like you to be more specific about "that other feature" so I can know exactly what you are referring to. If you're referring to Boom formated levels, then I don't know what I could add. Do you have any specific questions?

Edited by axdoomer

Share this post


Link to post

@kb1 maybe you mean demo compatibility? That's where we had left it in 2013 (?), more or less. It got to the point where major IWAD demos did all play correctly, as well as most UV speedruns I tried it with, but never quite got NM to work correctly. In any case, it's great to see that there are people with the energy and will to work on it. Needless to say it, when I did most of the development I had LOTS of time to spare and virtually no other challenging code tasks at hand, professionally or otherwise.

Share this post


Link to post

I didn't especially plan to work on demo compatibility. I don't know how much work is remaining to be done on that part, but if you have any advice, that would be welcome. I will have to fix such demo compatibility bugs that I will find. I don't know which techniques are used when comparing demo playback on a source port versus vanilla Doom and I'm curious. A demo can desynch way before it becomes apparent, but you can't debug vanilla and watch what's going on with the variables because it's compiled code. I wonder how you did it.

Share this post


Link to post

We had developed a complex text output-based system that logged Information at certain critical points, such as all calls to P_Random, and compared per-tic outputs. This system requires a "donor" port to be used as a reference, and the logging must be implemented on both sides. It also required quite a bit of back and forth, so I wouldn't really recommend it for "debugging by proxy" as we did. I think remnants of this system exist in the "demosync" branch in the old repo.

 

Surprisingly, the system worked well enough to allow us to narrow down desyncs to specific function calls and fix at discrepancies on my side. I even emulated an overflow at some point, which didn't work on kb1's port ;-)

 

It would be awesome if all ports had such a logger built-in, or at least conditionally buildable.

Share this post


Link to post
41 minutes ago, Maes said:

I even emulated an overflow at some point, which didn't work on kb1's port ;-)

Wait, you've seen the mythical KBDoom?

 

Also,  since this is active again,  should it be added to this list?

Share this post


Link to post
9 minutes ago, Death Egg said:

Wait, you've seen the mythical KBDoom?

Only by proxy, apparently. One of the reasons that debugging in this manner reached its limits is because the reference port itself (KBDoom, though we didn't really call it that) was not 100% accurate at the time.

Share this post


Link to post
1 hour ago, Maes said:

(KBDoom, though we didn't really call it that) was not 100% accurate at the time.

Another vanilla compatible source port? You've successfully picked up my attention. Now I don't want to work on Mocha Doom until I know more about it.  ;-p

 

I'll be working on Mocha Doom during the autumn. I'll be moving to a city known as the most boring city in Canada for an internship, so I shouldn't be busy. It will be one of my many projects, one of which is learning Rust and writing a 3D engine using this language.

Share this post


Link to post

 

21 hours ago, Maes said:

@kb1 maybe you mean demo compatibility? That's where we had left it in 2013 (?), more or less. It got to the point where major IWAD demos did all play correctly, as well as most UV speedruns I tried it with, but never quite got NM to work correctly. In any case, it's great to see that there are people with the energy and will to work on it. Needless to say it, when I did most of the development I had LOTS of time to spare and virtually no other challenging code tasks at hand, professionally or otherwise.

That's it, of course. I was and am extremely proud of what we (mostly you) accomplished. For everyone's benefit: We got Mocha Doom, Maes' complete, hand-written conversion of Doom from C to Java, to properly play back most original IWAD demos! (please read that again and understand the gravity of what was done!!)

 

Maes was advertising this complete conversion of his, and I argued, in a thread on DW, that, if it truly was a proper complete conversion, it should be able to play back IWAD demos (which would prove that it was correct, mathmatically, so to speak). Of course, this was a ridiculous statement of mine. I knew that one simple difference in the way C vs. Java handles the 15th digit of a division, one reordering of the thing list, or a thousand other as-of-yet-unknown slight differences would utterly destroy demo playback.

 

But, we got it basically working, over e-mail :) I think there's a TNT demo, that still desyncs, and there may be an issue with NM demos (some PWAD demos work too...). After getting as far as we did, RL kicked in and we were never able to get that remaining few, but I think it's still possible.

 

11 hours ago, axdoomer said:

I didn't especially plan to work on demo compatibility. I don't know how much work is remaining to be done on that part, but if you have any advice, that would be welcome. I will have to fix such demo compatibility bugs that I will find. I don't know which techniques are used when comparing demo playback on a source port versus vanilla Doom and I'm curious. A demo can desynch way before it becomes apparent, but you can't debug vanilla and watch what's going on with the variables because it's compiled code. I wonder how you did it.

As it stands, it's very close. I'd say 1 or 2 desyncs left max, that may or may not be able to be fixed. I relied heavily on Maes' ability to convert C code into Java. My time is extremely limited these days, but, let me know when you're ready with a PM, and I'll try to give you a hand.

 

10 hours ago, Maes said:

We had developed a complex text output-based system that logged Information at certain critical points, such as all calls to P_Random, and compared per-tic outputs. This system requires a "donor" port to be used as a reference, and the logging must be implemented on both sides. It also required quite a bit of back and forth, so I wouldn't really recommend it for "debugging by proxy" as we did. I think remnants of this system exist in the "demosync" branch in the old repo.

 

Surprisingly, the system worked well enough to allow us to narrow down desyncs to specific function calls and fix at discrepancies on my side. I even emulated an overflow at some point, which didn't work on kb1's port ;-)

 

It would be awesome if all ports had such a logger built-in, or at least conditionally buildable.

 

9 hours ago, Death Egg said:

Wait, you've seen the mythical KBDoom?

 

Also,  since this is active again,  should it be added to this list?

Yep - the donor port was KBDoom, running SyncDbg v1.0 - a system I developed for this very purpose. I have since rewritten SyncDbg to version 2.0 - it's 100x more efficient, with more info stored. It does live within the KBDoom source, compiled conditionally, as it can slow the port down a bit in hectic fighting scenarios. It can almost always tell you which function is non-compliant, and often where in the function the non-compliance is. In its current state, it basically requires the tested port to be at a Boom level of compliance.

 

Maes says that he doesn't recommend "debugging by proxy", but it really wasn't too bad 3-4 dozen replies or so. Most of that should go away with the new SyncDbg system, which is much more readable.

 

I would much rather have you update Mocha Doom's SyncDbg code to v2.0, instead of try with v1.0. This will put your C-to-Java capabilities to the test, as it contains some tricky allocations! SyncDbg 2.0 is built as stand-alone modules, ready to be dropped into a C port fitted with Boom additions, with a handful of "glue" code (typically one-liners that call into SyncDbg at various places) that must be added in the main engine source, to tie in to the SyncDbg system. There is a great emphasis on self-containment, so the code that has to be added to existing engine modules is as minimal as possible. This would be quite easy to use in a C port, but, for Mocha, it must first be converted to Java, of course, as the rest of Mocha was.

 

You can see a very small portion of SyncDbg v2.0 output while in Verbose mode, in the spoiler in my reply on this page:

How RNG works for Super Shotgun in Doom

 

7 hours ago, axdoomer said:

Another vanilla compatible source port? You've successfully picked up my attention. Now I don't want to work on Mocha Doom until I know more about it.  ;-p

 

I'll be working on Mocha Doom during the autumn. I'll be moving to a city known as the most boring city in Canada for an internship, so I shouldn't be busy. It will be one of my many projects, one of which is learning Rust and writing a 3D engine using this language.

I haven't released KBDoom yet, which has generated a bit of unnecessary, silly controversy (hence my title...). Release is planned, when I feel that it is ready. (it's no big mysterious super secret code, or anything - I just like to release once it matches my standards, you know? It still has some unique features, but no one is missing anything huge).

 

Please let me know when you have time. I am currently preparing instructions for the integration of SyncDbg 2.0 code into a port, which someone else is also waiting for, so I'll need a little time too.

 

Edit: Just re-read that you want to work on net code. Net code is very similar to demo code - instead of playing previously-recorded tics, you play tics brought in from other players. So, if you get the infrastructure down, the demo sync code will help there as well.

 

Good luck with Mocha Doom. Make Maes proud! This is his baby, and it deserves a loving touch!

Edited by kb1

Share this post


Link to post
3 hours ago, kb1 said:

You can see a very small portion of SyncDbg v2.0 output while in Verbose mode, in the spoiler in my reply on this page:

How RNG works for Super Shotgun in Doom

Interesting, so now you output much more stuff (though we did get down to some pretty gory details too), in a much more structured manner (one of the main problems we had was that comparison relied on text output being exactly identical down to the last whitespace character, so that diff-like tools could be used, but a structured approach that can be parsed even without strict line-by-line equality would be way more flexible).

 

In Java, this sort of "marking" variables and functions to be output could be done quite cleanly with annotations, instead of the traditional way of just "stuffing" the code with output directives. Of course, if one's feeling too lazy to port/engineer the system from the ground up, using JNA to a precompiled DLL of SyncDBG is always a possibility ;-)

Share this post


Link to post

The name of the mocha doom port and its goals alone make it top notch in my book! It'd be really awesome if this could be fitted to a useful niche, kinda like doom in web browser might have looked to us 10 years ago, heh.

Share this post


Link to post

I'm gearing up to this year's WadC update, and I should really take a look at Mocha Doom. We could even share some code.

Share this post


Link to post
20 hours ago, Maes said:

Interesting, so now you output much more stuff (though we did get down to some pretty gory details too), in a much more structured manner (one of the main problems we had was that comparison relied on text output being exactly identical down to the last whitespace character, so that diff-like tools could be used, but a structured approach that can be parsed even without strict line-by-line equality would be way more flexible).

 

In Java, this sort of "marking" variables and functions to be output could be done quite cleanly with annotations, instead of the traditional way of just "stuffing" the code with output directives. Of course, if one's feeling too lazy to port/engineer the system from the ground up, using JNA to a precompiled DLL of SyncDBG is always a possibility ;-)

It can actually be set to just spit out a single hash value per level, then a hash value per tic, virtually guaranteed to show you exactly which tic differs, with a tiny output file. You then run a second or third time, requesting a small range of tics to be output in verbose mode. Again, a tiny file. Regardless, it now only outputs deltas, even in verbose mode - a third space-saving feature. It hones in to the problem very quickly - I'm pretty proud of it.

 

The formatted output is done manually. I did not know how else to fake the structuring I'm getting out of it. It generates a block from, say, a state, which encapsulates P_Random calls, which make "recurse" into deeper attack actions, even deeper into damage states, etc. I am very surprised how clean it can make a seemingly random set of events look, all nested exactly at the level I wanted to see. A small set of output directives are included which support the decision to either inline, or nest. I will eventually release it, probably separately, with full instructions. It will also be in the KBDoom release, whenever that happens :) But, because there's immediate interest, I'll try to expedite it. Maybe then you can tell me if JNA could help streamline it. It's really cool to see what Doom is doing in list form, vs. trying to follow a debug session!

Share this post


Link to post
20 hours ago, Jon said:

I'm gearing up to this year's WadC update, and I should really take a look at Mocha Doom. We could even share some code.

Just as an aside, I'd love to see someone do a WadC community project, heh.

Share this post


Link to post

If you can release it separately so that it's possible to build it into a DLL or a .so (I don't have Windows anymore, I only use Linux), then using it with JNA should be a piece of cake. Porting it to Java and having to maintain two code bases would likely be a nightmare. The principle of least effort is a win.

 

I only have a vague idea of SyncDBG, because I have never seen it. I'll probably have a better idea when I'll be able to read the instructions when they'll be available, but I would like to know how it works, technically. Does it work like I logger that I could put in a singleton? I'm scared of tentacles that touch everything. I'm already trying to imagine how it works, although it may be too complicated for me to guess right.

Share this post


Link to post
1 hour ago, axdoomer said:

I only have a vague idea of SyncDBG, because I have never seen it. I'll probably have a better idea when I'll be able to read the instructions when they'll be available, but I would like to know how it works, technically. Does it work like I logger that I could put in a singleton? I'm scared of tentacles that touch everything. I'm already trying to imagine how it works, although it may be too complicated for me to guess right.

 

The implementation from my side at least was kind of hacking: there was an interface ISyncLogger:

 


public interface ISyncLogger {

    public void debugStart() throws IOException;
    public void debugEnd();
    public void sync(String format, Object ... args);
}

 

which was however implemented by the "God-like" DoomStatus object:

 


 ///// SPECIAL DEMO TOOLING STUFF, INSPIRED BY kb1 /////
    
    protected PrintWriter syncdebugfile;

    public final boolean DEBUG=true;
    
    public void debugStart() throws IOException {
        // [kb] open the sync debug file
        // [Maes] does it really have to be in append mode?
        syncdebugfile = new PrintWriter(new BufferedWriter(new FileWriter(
                "syncdbg.txt", false)));
        syncdebugfile.printf("*** Debugging started ***\n");
    }

    public void debugEnd() {
        // [kb] be sure to close the file proper on program shutdown
        syncdebugfile.printf("*** Debugging stopped ***\n");
        syncdebugfile.close();
    }

    // [Maes] call this to dispatch arbitrary reports
    public void sync(String format, Object ... args){
        if (sync_debug) syncdebugfile.printf(format, args);
    }
    
    private final boolean sync_debug=true;

 

 

, and was passed as an ISyncLogger object to specific objects, e.g. the RNG. Thus, modified calls to P_Random were used that looked like this (notice the SLY object, which is actually now a field of the P_Random class):

 


/** [Maes] I'd rather dispatch the call here, than making IRandom aware of
 * DoomStatus. Replace RND.P_Random calls with DM.P_Random(callerid) etc.
 *
 * Fixme: this could be made into a proper enum
 *
 * @param caller
 */

public int P_Random(int caller) {
    int value = P_Random();
    SLY.sync("PR #%d [%d]=%d\n",caller,prndindex,value);
    return value;
}

public int P_Random(String message) {
    int value = P_Random();
    SLY.sync("PR %s [%d]=%d\n", message,
            prndindex, value);
    return value;
}

public int P_Random(think_t caller, int sequence) {
    int value = P_Random();
    SLY.sync("PR #%d %s_%d [%d]=%d\n", caller.ordinal(),caller,sequence,
            prndindex, value);
    return value;
}

public int P_Random(think_t caller, mobjtype_t type,int sequence) {
    int value = P_Random();
    SLY.sync("PR #%d %s_%d %s [%d]=%d\n", caller.ordinal(),caller,sequence,
        type, prndindex, value);
    return value;
}

 

The various "flavours" of P_Random logged different amounts of information. Needless to say, the formatting and amount of info was more or less synced with what kb1 did on the other side.

 

Needless to say, lots of room for improvement here...you can see all this for yourself at the "DemoTooling" branch on the old MochaDoom CVS.

Share this post


Link to post
8 hours ago, Marcaek said:

Just as an aside, I'd love to see someone do a WadC community project, heh.


Yeah I'd love that. I tried to start one once but didn't get any interest. I could try again but I think I would benefit from some help from someone with more experience running something like that, marketing it, motivating mappers, etc. I'd love to see what someone other than me could achieve with WadC.

Share this post


Link to post
15 hours ago, axdoomer said:

If you can release it separately so that it's possible to build it into a DLL or a .so (I don't have Windows anymore, I only use Linux), then using it with JNA should be a piece of cake. Porting it to Java and having to maintain two code bases would likely be a nightmare. The principle of least effort is a win.

 

I only have a vague idea of SyncDBG, because I have never seen it. I'll probably have a better idea when I'll be able to read the instructions when they'll be available, but I would like to know how it works, technically. Does it work like I logger that I could put in a singleton? I'm scared of tentacles that touch everything. I'm already trying to imagine how it works, although it may be too complicated for me to guess right.

As Maes stated, the first implementation we used was a very hackish, "just get it done" mess that wrote enormous files that Maes would zip and email, and I would run lengthy comparisions against. Avoid!

 

The latest code was designed with 3 goals in mind: Self-containment with minimal dependencies, high performance/minimal output size, and easy human-readable output.

 

I can't speak for the Java implementation requirements, but I am writing up the documentation to be released soon. Yes, it could be made into a DLL, though this would have nominal benefit. Here I will provide a very brief preview description:

 

This version is 4 .c modules, each with its own header file:

sdb_crc, sdb_data, sdb_eng, and sdb_main. The bulk of the code is in data and string arrays which tend to duplicate the same data within the Doom engine. For example, there's mobj names, state names, action_function names, thing flags names, etc. This was done to totally avoid depending on the Doom engine for these names, and this ensures that your port's strings exactly match strings generated by comparison ports.

 

There is also a crc-32 engine with some data type primitives allowing a constant hash stream being updated per tic. Also included are some variable and tabbed-block output primitives to, again, ensure byte-accurate, humanly-readable textual output in parallel with identical hash stream update.

 

Finally, the code maintains a dynamically-maintained shadow copy of certain key mobj properties. These are compared against the game's real mobj properties, to provide delta-only output, making the output at least 100x smaller than previous implementations.

 

The "tentacles" you describe all consist of one-liners calling into sdb, surrounded by conditional compilation directives, in a couple-dozen key places within your code. Here's an example:

 

void P_LineAttack(mobj_t *t1, angle_t angle, fixed_t distance, fixed_t slope, int damage, int damagetype, int SpawnSplats)
{
	fixed_t 	x2;
	fixed_t 	y2;

#ifdef SYNC_DEBUG
	syncdbg_BeginLineAttack(t1, angle, distance, slope, damage);
#endif

/*

   ... the bulk of the P_LineAttack function

*/

#ifdef SYNC_DEBUG
	syncdbg_EndLineAttack();
#endif

}

I literally leave this code in my "release" versions. Of course, a single #define SYNC_DEBUG controls its existence. Also, P_Random is hooked which, believe it or not, provides the best desync information of all! Finally, there's some additional command-line arguments that need to be checked to fine-tune the sync debug process.

 

Quick sample run example:

 

Let's assume that you know that the first demo desyncs. So, you start up your game in per-tic hash mode. This causes sync_debug to write a single 32-bit hex value for each tic of the demo. This is, of course, a very small file. When the demo is done, you compare this file against a known-correct file. This tells you in exactly which tic the deynce occurs. Let's say that it starts at tic 750. That's a dozen or so seconds in. Now, you run the game again, but tell it to output verbose info from tic 700 to tic 1000.  This produces a much bigger file, to be again compared against a known working copy. You can do a Find in your text editor to find tic 750, then back up a few tics to get an idea what action is taking place.

 

This tells you that you have just fired a rocket, and an imp has just killed a shotgunner. Also, a pinky has just turned towards you to bite you. When you read down to 750, in your port's file, your missile continues to fly. But, in the known working file, that rocket explodes against a barrel.

 

You then remember that, at one point in time, you lowered the height of barrels as a test 2 years ago :) (This actually was tur for one of my desyncs - for a torch.)

 

It used to be a nightmare to compare outputs - Maes can tell you. It is an absolute joy to use now. It also has a super-compact "regression test" mode which will output one single 32-bit hash for a whole demo. This can be used against a batch of demos to ensure your port didn't break between releases. The way it's designed, hash numbers should be identical, regardless of verbosity level, regardless of which port is used.

 

If it were me, I'd suggest just converting it to Java - it's not that big, and I can more-easily help you debug it. Let me get my documentation together.

 

@LinguicaAt one time you described the idea of allowing members a small hosted space on Doomworld, for screenshots, demo files, and the like. Can I use that to post a few source files and a couple of HTML pages?

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×