Only on DW: Mochadoom Techdemo [now with some fixes]

OK, so I was "Mocha Doom this" and "Mocha Doom that" all the time, posted progress, CVS repo, benchmarks, some screenshots and statements... all fine and dandy but up to this day you just had to take my word for it.

Well, not anymore. For the first time, I'm releasing a TECH DEMO of MochaDoom. Unlike the older non-playable map viewer, this one has come a long way and is "explorable", though not really playable.

Get it here (techdemo v1.2).
Get it here (techdemo v1.1).

plus:

In case you have an older Athlon XII or another CPU that hangs at the wiper screen, try either of these experimental hotfixes:

1. Changing a timeout check to ticks-->0 from ticks--!=0 in the wiper code alone, with no other changes. (Variation 12a)
   
2. As a), plus using System.currentTimeMillis() instead of System.nanoTime(). This has the side effect of limiting maximum timer resolution down to 15-16 ms on Windows, and thus maximum frame rate under any circumstance can only be about 67 FPS, but should not require particular processor affinity. Linux systems should be less affected, as that function has true millisecond resolution. (Variation 12b)
   
3. As a), plus using a "sanitized" getTime method: if a discrepancy is detected such as attempting to return a number of elapsed tics smaller than the previously returned time, then that previous time is returned instead. (Variation 12c)
   

Also I updated the status on the official project page

And here's some historical background (kinda like a wannabe dev blog, more ranting than anything :-p)

Edit: I have updated the demo to address a few common complaints. The older wall of text with features/instructions/known limitations is in the included readme file.

Fixes since last release:

* Weapon bobbing implemented.
* All-cheats bug (Windows key related) now fixed.
* It will now try to force US-EN layout on startup to allow functioning over international keyboard layout. However, this is only a half-fix since there's no way for Java to read raw keyboard scancodes, only cooked reads.
* Now CTRL, ALT and SHIFT work properly, and automap also toggles properly.
* Screen sizes now scale up properly (screen doesn't start up super-small). However, the -/= issue hasn't been fixed yet, and people with non US keyboards will actually have to press their 'minus' and 'equals' keys (Doom actually maps to minus and equals, not minus and plus). Believe it or not, it's not easy to read raw keyboard scancodes in Java without using native code -_-
* Palette effects implemented. Now getting hurt on the nukae will flash the screen.
* IDBEHOLDx cheats now work, and you'll see proper flashing.
* Fixed a bug that caused the Player sprites to be redrawn multiple times when many masked segs were visible, thus causing noticeable slowdowns when looking in certain directions.
*Fixed the bug encountered by Gez with his modified Doom 2 WAD, which was related to the killough hashtable code. Ditched that and switched to the built-in Hashtable function, cleaner AND faster.

OK, so if you just want to try it, just uncompress the zip into a folder and run either:

• mochadoom.bat for the "normal game"
  
• benchmark_parallel.bat for the parallelized spinner benchmark.
  
• benchmark_serial.bat for the normal spinner benchmark.
  

Under Linux, same thing with the .sh scripts.

You can use most standard doom command line arguments with mochadoom.bat or mochadoom.sh

I would like to hear some feedback, FPS counts, etc. and in particular how it behaves under different OSes and with different graphics cards (some combinations of video drivers and JREs just won't give a fast enough rendering with Java, others will be almost as fast as a native port). Shared video memory is also a bane to performance.

NOTE: after starting a game, the screen will be small. Use the +/- keys on the keyboard (not the num keypad) to scale to full view. Unfortunately the status bar is still broken, so don't mind it yet. Also scaling toggles only between full screen and vanilla/subvanilla resolutions.

PS: doom1.wad (shareware) is included but you can try it with your own full versions, if you wish. Usual vanilla priority applies.

Catoptromancy said:

Posting in this thread.

Same here.

This is great. It runs super fast and smooth, which I did not expect, because my netbook sometimes struggles even with simple Java applets.

I was also quite surprised to see doom2.wad loaded instead of doom1.wad, it looks like Mochadoom recognizes the DOOMWADDIR environment variable. :)

First of all: Unix scripts should use Unix line endings, otherwise they won't run. Also having a newline at the end of the script is a good practice, although not having it doesn't prevent it from running.

Benchmark results:
serial: 130-160 FPS
parallel: 130-190 FPS

CPU:

vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Core(TM)2 CPU         T5500  @ 1.66GHz
stepping	: 2
cpu MHz		: 1667.000
cache size	: 2048 KB

Relevant PCI devices:

00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller (rev 03)

OS: Arch Linux (x86_64)

Relevant software packages:

kernel26 2.6.35.8-1
xorg-server 1.9.2-2
xf86-video-intel 2.13.0-4
intel-dri 7.9-1
jre 6u23-1

Thanks for the feedback!

I was actually surprised to hear people with notebooks using Intel GMA and netbooks getting relatively good performance, because I have also tested it on an Athlon X2 640 Quad Core which is really struggling with it, due to the mobo's integrated ATI 4250 graphics (and shared video ram), which is also what prompted me to develop the multithreaded renderer.

On a system with a bad video driver/java synergy, up to 50% of CPU time may end up being spent just updating the display (after actual rendering has completed), and the only way to speed it up is to speed up the rendering itself. Seems however that Intel actually has decent DirectDraw and even Linux drivers, at least they seem superior to ATI's under Linux (performance under 64-bit Ubuntu was half that under Windows).

I also fixed the bash scripts, only tried them under Ubuntu and they worked, no idea they'd break other distros.

Here are my Linux benchmarks:

Benchmark results:
serial: 230-280 FPS
parallel: 250-280 FPS (higher average)

CPU:
processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz
stepping	: 6
cpu MHz		: 800.000
cache size	: 3072 KB

Relevant PCI devices:

00:00.0 Host bridge: Intel Corporation Mobile PM965/GM965/GL960 Memory Controller Hub (rev 0c)
01:00.0 VGA compatible controller: nVidia Corporation G84 [GeForce 8600M GT] (rev a1)

OS: Ubuntu Linux 64-bit

Relevant software packages:

kernel26 2.6.32-26-generic
xorg           1:7.5+5ubuntu1
nvidia-185-mod 195.36.24-0ubu Transitional package for nvidia-185-modalias
jre 6.22-0ubuntu1~10.04  

It works very nicely! The parallel benchmark does near 300 FPS on a 3.2 Ghz i7. Excellent.

If there's interest, I can create versions hardcoded for lower/higher resolutions and different threading configurations (current is 1 wall + 1 floor thread for the renderer, a quad core may benefit from a 2 + 1 or 2+2 configuration, depending on what's being displayed).

A Pentium III can get 100 fps at vanilla resolution, with an old APG2 s3 videocard. Low-end single core celerons/Athlons XP/Pentium 4 under 2 GHz should probably work well with 640*400/480 or maybe a little more, if the video drivers are half decent.

Again, the actual "playable" demo uses the parallel renderer (some parts are in common with the serial one anyway).

Also, you can test mochadoom with the included sprites.wad, a NUTS-like level testing sprite display. Of course, you'll need to idclip but you can see how the monsters are rendered, including the spectres' shadow effect ;-)

Maes said:

What works:

• Menus are functional enough to start new games, choosing episode and difficulty. They don't fully support ultimate doom though.
  

Couldn't get menus to display. Nor a titlepic, either. Just a black screen.

However, the benchmarks worked; if what they're supposed to do is rotate the camera continually.

Java:

java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)

Computer specs: here, plus Vista32 HPSP2.

Benchmarks results: http://pastebin.com/guYECfGw

Can you post the output from the console during the normal startup? What IWAD were you using?
Also, were you using -warp? This will boot straight into a black screen (the automap) and you'll need to press TAB to switch to toggle to the normal view.

Similarly, when loading PWADs you'll get the modified game nag screen in the console, you'll have to press Enter there to continue (yup, it's a bit lame to have it in when testing).

Also try giving the window mouse focus before attempting any keyboard input (provided that you didn't actually crash).

I am unable to increase the screen size by pressing "+" (it works if I increase it from the menu). It also freezes the game and prints an exception to the console:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 521
        at m.cheatseq_t.CheckCheat(cheatseq_t.java:169)
        at st.StatusBar.Responder(StatusBar.java:747)
        at doom.DoomMain.Responder(DoomMain.java:1687)
        at doom.DoomMain.ProcessEvents(DoomMain.java:233)
        at doom.DoomMain.DoomLoop(DoomMain.java:450)
        at doom.DoomMain.Start(DoomMain.java:1361)
        at i.Main.main(Main.java:69)

I also have some benchmark results (observed values in a 5 minute test | all values are rounded to nearest integer):

Serial:

• Lowest: 56 FPS
  
• Highest: 71 FPS
  
• Average: 64 FPS

Parallel:

• Lowest: 67 FPS
  
• Highest: 83 FPS
  
• Average: 77 FPS

System specs:
Lenovo S10e Netbook

• OS: MS Windows XP Home 32-bit SP3
  
• CPU: Intel Atom N270 @ 1.6 GHz
  
• Graphics: Mobile Intel 945 Express Chipset Family
  
• RAM: 1.0 GB Single-Channel DDR2 @ 265MHz (4-4-4-12)
  
• HDD: WD Scorpio 156GB @ 5400 RPM (SATA, 8 MB cache)
  
• Java:

  java version "1.6.0_21"
  Java(TM) SE Runtime Environment (build 1.6.0_21-b07)
  Java HotSpot(TM) Client VM (build 17.0-b17, mixed mode, sharing)

Also, here are some screenshots I made while playing with the -file parameter:

Maes said:

Can you post the output from the console during the normal startup? What IWAD were you using?

Here you go: http://pastebin.com/UxhQZ47u

outlog first, followed by errlog.

Using doom2.wad and no warping, for what it's worth.

@Xtroose: great feedback, thanks! The +/- keys should be those on the keyboard (not on the numerical keypad). At least in US layouts they should be above the P,[ and ] keys (however they actually are '-' and '=' unshifted. The keypad '+' doesn't register as a resize commands, and probably breaks the cheat checking array (will look into it, this release is also to catch such bugs).

@Gez: Interesting, I'm using doom2.wad dated 1/2/1995 (v1.9 version) with an MD5 sum of 25e1459ca71d321525f84628f45ca8cd and it seems to work fine, while you get an exception as soon as it tries loading the textures. Are you using an older/modified version?

Apparently modified, the checksum isn't the same. So I just grabbed a fresh copy from Steam, and tried again. This time, it's the same checksum, and now it works. Well, works as much as it can, anyway. I've got about the same problem as Xtroose, too: the screen is ridiculously small. Hitting + once, it's still ridiculously small, though a bit less. Hitting + a second time, it's suddenly fullscreen, and frozen.

Maes said:

benchmark_parallel.bat for the parallelized spinner benchmark.

benchmark_serial.bat for the normal spinner benchmark.

Win7 x64 + Core 2 Duo E6750:
benchmark_serial.bat - 198 fps
benchmark_parallel.bat - 198 fps

I would like to take a look at the modified doom2.wad, or at least have a diff patch so I can see exactly what caused it to break (although stuff like lower-case lump names or malformed lump directories are the most probable culprit).

As for the scaling, the small screen issue happens because only vanilla scaling levels are implemented (1 to 11, where 11 jumps straight to fullscreen regardless of resolution). The other sizes are actually sub-vanilla only, not scaling up to the maximum smoothly.

It takes a couple of taps on +/= to bring it to full screen, or as Xtroose said, through the menus. Abour freezing, it will apparently "freeze" if the mouse moves beyond the window's boundaries (I have no mouse trapping yet) and you'll have to move it back in and click to give the window focus again. However the game will keep on running (I even have the console spit out a message each time the window accepts/does not accept keyboard input), along with other debugging stuff.

@entryway: yup, the parallel version helps most when the video output itself is a major bottleneck. On systems with relatively slow video it will make a noticeable difference, on others that have fast video and can push the renders out as fast as possible, not as much.

There still might be some benefit at higher resolutions where there's more screen area to render, while multithreading is probably redundant at lower resolutions.

The floor renderer actually gives a separate visplane to each thread (if more than one is used) until all have been rendered, while the wall rendering works by pipelining "RenderWallInstructions", aka instructions to draw a single column between available threads. Plus, the wall and floor rendering work in parallel with each other, on top of being internally parallel.

The next step will be a way of making sprite drawing parallelized, which could give a speed boost in the case of NUTS.WAD numbers.

Since there seems to be some confusion about the controls, I'll illustrate: the +/- keys I'm referring to when I'm talking about screen resizing and automap zooming are those ones, cycled in red, on a standard US layout keyboard. Dunno if other layouts have them in more inconvenient places:



Cycled in blue are the strafe keys, since alt + shift are not handled yet, and mouse turning is not very sensitive. Once again, movement controls and automap follow mode are with arrows, not WASD (no configuration implemented yet, just using defaults).

Function keys and ESC work for some things. E.g. F1 will bring up help, F2-F3 and F6-F9 will try to load and save (but it's unimplentented, you will only see the dialogs). F7 will end a game after confirmation, F8 won't do anything (in general, HUD messaging doesn't work yet), F5 will switch detail but the low detail functions are broken and it will look like ass.

F10 will ask you to quit, while gamma levels are not implemented yet, nor is taking screenshots.

Also, cheats work but you may need to persist a few times. Beware of nukage floors, they may kill you ;-)

About input focus: input from either mouse or keyboard will be accepted only when the mouse is inside the view area AND the window has focus. The console will show when the mouse is in/out pretty clearly with a message. In any case, press LMB once to regain full focus (or click on the window's bar).

I will try to clarify things a little bit.

This is my keyboard layout (Slovene and modified to fit on a netbook in case you are interested):
Note the [A], [B] and [C] markers

(the key to the left of "1" is actually between "Esc" and "F1", but this is not relevant here)

Here is how it works:
When writing, the key marked with [A] produces an apostrophe ('), key marked with [B] produces a plus sign (+) and key marked with [C] produces a minus sign (-).
In vanilla and every source port I tried the key marked with [A] decreases the screen size and key marked with [B] increases it. (This is probably because most of them treat the keyboard as if it has the US layout)
In Mochadoom pressing the key marked with [A] does not do anything, pressing the key marked with [B] does not increase the screen size, but it freezes the game and prints an exception to the console, and the key marked with [C] decreases the screen size.
I also tried the numerical keypad equivalent of the plus sign ("Fn" and the key marked with [C]) and it did not do anything.

Interestingly Mochadoom recognizes "Y" as the letter Y. The majority of source ports (and other programs that do not recognize non US keyboards) recognize "Y" as the letter Z, exceptions include ZDoom (and compatibles) and the Eternity Engine.

I guess the main issue behind this is keyboard layout recognition, because if I change the input language from Slovenian to English (US) the key marked with [B] increases the screen size and Java never complains to the console. I hope this will help you solve the problem.

Seems it's a problem of internationalization. Since Mochadoom is based on the Linuxdoom 1.10 code, I kinda of copied the original X Server application and adapted it to Java (the function that maps keystrokes to Doom's keyboard codes is just like the one found in linuxdoom, only adapted to use Java's VK's as an input).

By your description of it, seems that most source ports actually read raw keyboard codes and use fixed key positions, rather than fixed key characters while Java VK codes have already gone through internationalization.

In your case, you must probably press shift +0 to increase the screen size (by typing "="), since, if you look at the US keyboard, the plus sign is actually on the "=" key, so in all those years we've been actually using 'minus' and 'equals', not 'minus' and 'plus' to change size ;-)

Mochadoom parses the actual character being typed rather than the keyboard code. I can probably work around that by intercepting the keycode events directly rather than the charachters (however it's considered deprecated style).

Again, thanks for the input, and keep telling me more :-)

Edit: I think the crash when you pressed that special character was because Java reads its unicode value, which is greater than 255 (521 would be unicode 0x0209), and thus overflows doom's cheat decryption array, which only has 256 entries. That can be sanitized easily by clipping unicode chars with a 0xFF mask, though.

Maes said:

The next step will be a way of making sprite drawing parallelized, which could give a speed boost in the case of NUTS.WAD numbers.

I tried to parallelize some work with sprites (checking for visibility and precalculation of GL data in separate thread) some time ago and got only 10% boost on nuts.

entryway said:

I tried to parallelize some work with sprites (checking for visibility and precalculation of GL data in separate thread) some time ago and got only 10% boost on nuts.

I don't know if the actual bottleneck in nuts-like wads is drawing the sprites or running the thinkers, probably it can be either depending on the port (some seem to run just fine until the monsters are awakened, e.g. I recall prboom+ had the "help dying friend" flag on by default at some point, and this resulted in an O(n^2) hell. Older versions of ZDoom also behaved like that, now I think it's possible to run nuts even on those.

In any case, if drawing the sprites amount so, say, 15% of the total time spent in a tick, even getting that time down to zero will be at most 15% faster in total. Still, if you are looking for every possible way to squeze performance gains and use those extra cores for something, the floor and walls are much better candidates: they take up the bulk of the computations (normalls) and can even be run in parallel with each other (if you precompute visplane boundaries without actually drawing the walls).

Sprites suck in that they must be run after everything else is completed :-/

Parallel:

ACCEPTING keyboard input
200 frames in 0.025599658 = 7812.604371511526 fps
200 frames in 1.783993173 = 112.1080523327765 fps
200 frames in 1.808242031 = 110.60466274495087 fps
200 frames in 1.780974782 = 112.29805274132175 fps
200 frames in 1.788680861 = 111.81424498956497 fps
IGNORING keyboard input
200 frames in 1.720708684 = 116.23117954811228 fps

Serial:

ACCEPTING keyboard input
200 frames in 0.02546277 = 7854.604978170089 fps
200 frames in 1.89957653 = 105.2866240666808 fps
200 frames in 1.944043354 = 102.87836410051584 fps
200 frames in 1.852667549 = 107.95244948720155 fps
200 frames in 1.917332537 = 104.31158713497595 fps
200 frames in 1.866440793 = 107.1558233993228 fps
IGNORING keyboard input
200 frames in 1.856087732 = 107.75352724544597 fps

Core 2 Duo E8400 with 4 gigs of RAM and an ATI Radeon HD 5770

Fantastic work Maes.

Maes said:

I recall prboom+ had the "help dying friend" flag on by default at some point, and this resulted in an O(n^2) hell

Just because prboom had a bug and now it is fixed for mbf complevel (and will fixed for "-1" complevel if new level of compatibility will be introduced ever)

Maes said:

Edit: I think the crash when you pressed that special character was because Java reads its unicode value, which is greater than 255 (521 would be unicode 0x0209), and thus overflows doom's cheat decryption array, which only has 256 entries. That can be sanitized easily by clipping unicode chars with a 0xFF mask, though.

Wouldn't do that - that will cause Unicode characters to arbitrarily map to the first 256 codes in a modulated fashion. Rather, I'd clip the input. If the character code is not within range, just ignore it entirely. This is what Eternity does.

Benchmark results:
serial: 120-140 FPS
parallel: 190-230 FPS

CPU:
processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 7
model name	: Intel Mobile Core 2 Due P8400
stepping	: 6
cpu MHz		: 800.000
cache size	: 3072 KB

NVIDIA Geforce GT 229M (9600m). 512mb.

after starting a new game I get crazy shit and a black screen. :( I don't know how to fix this. I have a GeForce 6800 which should support what you're doing, obviously.

http://dl.dropbox.com/u/14696120/mochauhoh.png


EDIT: updating my drivers fixed it.

EDIT: and now it won't run again. something makes java hate this after running it a few times. it talks about this:

cache miss on lump 9
cache miss on lump 8
cache miss on lump 12
cache miss on lump 13
cache miss on lump 11
cache miss on lump 7
Trying to raise weapon
wp_pistol height: 8388608
Not on top yet, exit and repeat.
Trying to raise weapon
wp_pistol height: 7995392
Not on top yet, exit and repeat.
Called!

@Csonicgo: the "Called!" refers to the screen wipe (and it seems you're getting a partial screen wipe for some reason), just after starting a new game.

What OS and what version of the Java Runtime Environment are you using?

Arey you getting any text files with a hs_err_pidXXXX name pattern? Those would indicate a JVM crash.

Maes said:

@Csonicgo: the "Called!" refers to the screen wipe (and it seems you're getting a partial screen wipe for some reason), just after starting a new game.

What OS and what version of the Java Runtime Environment are you using?

Arey you getting any text files with a hs_err_pidXXXX name pattern? Those would indicate a JVM crash.

WinXP, 1.6 update 22, the latest java I think. and no I didn't get any text files....

Probably it's still the video drivers acting up, however I'd try doing a clean reinstall of the JRE, just in case.

Can you post more details about your rig? E.g. service pack level, graphics adapter, resolution and color depth.

Also, try running the main game with this command:

java -Xint -cp mochadoom.jar i/Main

the -Xint parameter will force Java to run in pure interpreted mode without dynamic compiling, which will make everything a LOT slower but may reveal if certain bugs are compilation-dependent (they shouldn't, though, this sounds more like a case of bad synergy with your video drivers). Since under Windows DirectX is the Java-Video middleware, you might want to check its version too.

Also, you get this sort of video corruption and freeze only with the main game, or also with the spinners? Are you using a regular IWAD or a modified one? (In the latter case, download the latest version of the demo, it handles IWAD stuff a bit better).

I suspect that you might have a modification that's enough to throw something into an infinite loop, but not actually crash with an error. Again, like I asked Gez, I'd like you to send me a bsdiff patch of your IWAD vs a standard Doom Shareware 1.9 IWAD, if there are any modifications at all (e.g. errors after download). Also, check out that the java.exe that you invoke at the command line (with java -version) is indeed pointing to the Sun/oracle 1.6 r22 Java, and not on another JVM.

I don't get teh freeze with the spinners at all. resolution is 1680x1050x32, SP3, NVidia GeForce 6800.... I will attempt a reinstall of the JRE, but why would it call the wipe twice?????

java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)

EDIT: running it via command line worked. the batch file always crashes! What the fuck?
EDIT2: ok, -Xint makes the program run. it's slow as fuck but it runs. taking it out makes it break.
Edit3: now both don't work. I don't understand this at all.
Edit4: just running the program over and over with -Xint eventually makes it work. I don't understand how that's possible but there you go.