Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Maes

A Mocha Doom demo (non playable)

Recommended Posts

I originally wanted this to be a stand-alone applet, but it
turns out I can't read files from URLs or internal .jar resources as random access files (only as data streams, so I can't put a WAD on a server or on an URL and have it read it from there without extensive modifications to how files are handled).

So, this is a "quasi-applet" that can only run locally, just for showcasing purposes. The final project will be intended as a standalone Java application, with a Web-based wrapper to be added in the end.

Get it here.

It showcases some of the functional -or almost- parts stringed together so far:

  • Intro screen with screen wipe (take that, DoomCott!)
  • Menu with simulated key input (you can see that in the output console), frame rate is artificially capped to 60-something.
  • Automap uncapped framerate scrolling and panning, also showcasing palette shifts.
  • Endlevel with simulated interruption and some pretty funky figures ;-)
Also, things you don't see but which are there:
  • WAD handling and lump caching.
  • Marshaling of Doom's data structs to Java objects.
  • Fixed point arithmetic (used in the automap)
  • Ticker/Responder systems for the implemented modules.
Just unpack in a directory and open index.html with a Java capable browser.

I'd appreciate some feedback on how/if it runs on other systems and browsers, including Linux and Mac OS , as so far it only seems to works inside Mozilla Firefox (IE 8 runs it but it doesn't display correctly after the initial wipe).

A big novelty -as far as Java 2D imaging is concerned- is the hack I used in order to have a "directly modifiable" indexed display with several palettes: I "hardwire" a BufferedImage to screen[0] of the software renderer, which then is modified directly without using setPixel/getPixel methods, and without using drawing primitives such as polygons etc. on a Java AWT Canvas.

Furthermore, I constructed 14 BufferedImages, one for each palette in the game, which however all share screen[0]'s data array. To switch between palettes, I just switch to another BufferedImage when drawing it on the canvas, and voila, draw once, display EVERYWHERE, with no speed penalty!

This is as fast as it possibly can get. How fast? A Pentium IV 3.0 GHz obtains 1100 fps on the automap, and a T8300 Dual Core more than 2300, when displaying E1M1's map. This means that there should be more than enough computational leeway to actually make this into a playable port.

Unintuitively, I FIRST create the buffered image, and THEN assign its backing data array to be screen[0], rather than the opposite, which is not possible due to object immutability.
/** This implementation will "tie" a bufferedimage to the underlying byte raster.
 * 
 * NOTE: thie relies on the ability to "tap" into a BufferedImage's backing array,
 * in order to have fast writes without setpixel/getpixel. If that is not possible,
 * then we'll need to use a special renderer.
 * 
 */
@Override
public final void setScreen(int index, int width, int height){

    // We must FIRST initialize the image, so that the (immutable) color model will be set.
    
    if (this.icm==null)
    screenbuffer[index]=new BufferedImage(width,height,BufferedImage.TYPE_BYTE_INDEXED);
    else
        screenbuffer[index]=new BufferedImage(width,height,BufferedImage.TYPE_BYTE_INDEXED,this.icm);    
    
    // Hack: hotwire the screenbuffers directly to the image. T3h h4x, d00d.
    // Now, HERE is where the magic happens. Once a BufferedImage is created, the internal raster is
    // immutable, but its backing data array is accessible for read/write.
    // Ergo, we can now "hardwire" the vlb to the raster's backing array. Whatever we write in there,
    // will also appear in the image.
    
    screens[index]=((DataBufferByte)screenbuffer[index].getRaster().getDataBuffer()).getData();

}

Share this post


Link to post

Your HTML file uses backslashes for path separators, automatically making that file only work on Windows (and may possibly be even the failure for IE8; nobody in the real world uses nor should depend on backslashes in HTML files as a path separator). On Debian/amd64 with OpenJDK the applet still doesn't run.

Also, it's gonna sound crazy but: Source? It would be nice to see what you've done, and possibly for other(s) to even see errors (for example, why isn't it running on Debian? I dunno.)

Share this post


Link to post

You mean this?

<applet archive="AutoMapApplet.jar" code="doom/DoomApplet.class" codebase="./" height="240" width="320">
The failure mode I see in IE8 has to do with the rendering, not with being unable to read the applet at all. It loads, displays the titlepic but then it displays everything as black after the initial screen wipe (it keeps running however, and you even see the black becoming colorized as the palette is switched during the automap part). Its java container probably doesn't like the aggressive display method I'm using.

The most reliable way to run it, it seems, is to use AppletViewer from the command line (AppletViewer index.html) or as a plain command line java stand-alone application.

The source code has been public since day one at sourceforge (checkout the CVS), although the specific applet code that pieces it all together is not (yet). Since I intend this to be a standalone application, I won't bother adapting it to the quirks of this or that container (yet).

Also, run it only with Sun's JRE, as I sincerely don't know how some of the more advanced operations (such as mapping rasters to BufferedImages) will work on other JREs.

Share this post


Link to post

With the Sun JRE the applet still fails in the browser. with "appletviewer" it works on both Sun JRE and OpenJDK (not too surprising; OpenJDK is basically just the GPL release of Sun's code).

It's pretty cool. I'm gonna have to checkout the source later :-)

Share this post


Link to post
chungy said:

With the Sun JRE the applet still fails in the browser. with "appletviewer" it works on both Sun JRE and OpenJDK (not too surprising; OpenJDK is basically just the GPL release of Sun's code). However, is it supposed to run super fast?


Yes, the automap is deliberately uncapped to stress test the renderer (it also displays a benchmark in the console).

The more I test the applet form the more disappointed I am though: most browsers seem to fail silently upon loading (probably a memory error?)

chungy said:

It's pretty cool. I'm gonna have to checkout the source later :-)


Please do :-) I'm eager to get somebody else interested in this endeavour too, as I've been practically re-designing Doom's innards while still trying to make sure they will work as before. No wonder nobody ever seriously tried to port Doom in a non-C language (with the exception of Delphi Doom, which is more of the same): it requires to redesign EVERYTHING, get rid of pointers, functors, etc.

Share this post


Link to post

Sorry I realized that you already said that but I was stupid and forgot while making the first version of that post =p

I'm not sure what's wrong with the applet, but also from running it straight on the command line.... well, see if you can find the error:
$ java -jar AutoMapApplet.jar
Exception in thread "main" java.lang.NoClassDefFoundError: doom\DoomApplet
Caused by: java.lang.ClassNotFoundException: doom\DoomApplet
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: doom\DoomApplet. Program will exit.


(protip: forward slashes will work on Windows. and everything else)

Share this post


Link to post

You can run almost every tester in the testers\ directory as main-method entry point, and see what happens ;-)

Some write only text to the screen, some do benchmarks, some write PNGs to disk (careful...) and some (the latest) have super-sweet video output.

Share this post


Link to post

In chrome the applet loads but nothing happens. In firefox 3.6 there seems to be some lagging in the automap stress test. Latest Java update installed.

Edit: 528 FPS on 900MHz Celeron. Chrome actually closes the applet. I am not sure why.

Share this post


Link to post
Csonicgo said:

Edit: 528 FPS on 900MHz Celeron. Chrome actually closes the applet. I am not sure why.


The FPS is adequate for the hardware at hand, considering a Pentium IV would get "only" double that.

As far as to why it behaves differently under different browser...my educated guess is that the plugins have different access privileges under different OSes, and some of the "trickier" operations are not guaranteed to work. E.g. Explorer apparently doesn't like my BufferedImage hack, and doesn't allow me to assign the same raster to multiple images (even though I should be able to, there weren't any deprecation/unsupported operation warnings).

For Chrome, without at least some way to log what happens, I can't really tell, but I suspect it has its own plugin limitations/policies.

Now, SOME of these may be workable if e.g. I switch to a less "wild" renderer at the expense of speed (I have a "nice and slow" full color renderer that's about 80% as fast as the indexed one) but by and large, this is intended to be a standalone app just like Jake2 (I'm sure they stumbled upon similar inconsistencies when/if they ever tried making an Applet version of their port).

Share this post


Link to post

I've investigated further on what happens with applets, and even had a chance to see how it runs under Mac OSX: runs just fine, as long as appletviewer or the stand-alone (non-applet) version is used. However Safari and Firefox crashed upon initializing the applet

Also, I managed to produce "stable" versions of the applet that ran under Ubuntu, with both Firefox and Chromium: I had NOT to override the default start() and paint() methods, which caused the plugins to behave unpredictably (under eclpise sometimes AWT threw null pointers, in browsers this results in a "grey box of death" or a total crash), probably due to race/timing conditions. Also, some libraries (in particular the ImageBuffer library) behave differently when run in applet plugins (e.g. IE doesn't support the raster sharing trick)

My "fast writes" method worked on Mac OS X too, which means that the stand-alone version will definitively use that one (got 3700+ fps on an iMac!).

So, so far I've deduced:

  • A stand-alone version (a-la Jake 2) will be able to use clever acceleration tricks, easier to code.
  • An applet version will have to use a tamer screen rendering method, introducing a bottleneck.
  • A stand-alone version can have direct and very efficient random disk I/O
  • An applet version can only use random file access when used locally, for files stored on a server directory or a remote URL I'll have to provide buffering, thus "disk" I/O will be slower (there's a library, uio, that can sort of wrap data streams into random access files, so maybe I'll use that)
So, if an applet version is produced, it will need extra sandboxing and more restraining that a full-fledged standalone version, and probably reduced display performance. Then again, the stand-alone version seems perfectly capable of executing on Windows, Linux and Mac OS, which is no little thing...

Update: turns out that IE actually has a problem with using java.nio.* routines, not with rendering: it just loads every patch as a black mess and makes everything "invisible", while the applet "runs", so to speak.

In any case... applets are so full of shit, that making one will definitively not be a priority, even if it would be kinda cool having a browser-based Doom that's tons faster than the "Flash port" and can actually load user-pointed IWADs/PWADs...

Share this post


Link to post

This is fantastic work Maes. I've said it to you before, but thank you for braving this tricky port.

Hopefully this will open up a new age in doom source ports that aren't tied to C/C++.

Share this post


Link to post

I am now closer -well, closer than anyone else has got, at least- to produce a functional derivative. And, the very least, I put my code on the public domain so this won't die the deaths of Stark and Doomcott -both long forgotten.

I had some doubts about how to handle function pointers -in some cases, I used "invokable classes", aka inner classes that have a single method, implement a specific interface to enforce their calling convention and can be passed as parameters. This works OK for code that doesn't need to be called a lot and when there aren't too many such functions (a good example are the Menu draw & handling routines, and the wipe screen effects).

Others, like the infamous think_t struct and the PIT_ and PTR_ functions/iterators in the game mechanics, are too critical to performance and used within already cluttered up code, so I decided to use a "dispatcher method" for those (instead of functors, thinkers and other objects will just store an enum entry, and will call a dispatcher method with their particular enum value and parameters). This is very clean, and even slightly faster than having "invokable functoins".

In case you wondered why I'm not using Java's reflection that allows calling arbitrary methods from within classes...

  • The syntax to set "Method" objects up is uglier than the ugliest hack I had to come up with.
  • They are still tied to a particular class instance
  • They need to be checked for exceptions when called
  • When microbenchmarked, they are about an order of magnitude slower than both hand-coded invocable functors and dispatcher methods.

Share this post


Link to post

When I try to load this on Internet Explorer, it freezes on the Java loading screen, and when on firefox it just gives a black screen.

Share this post


Link to post
Super Flip said:

When I try to load this on Internet Explorer, it freezes on the Java loading screen, and when on firefox it just gives a black screen.


I know, explorer has actually a broken plugin that just ceases reading stuff after a while (showing all 0s/black images). It doesn't actually freeze (because you can see the palette cycling), but doesn't work as intended either. In you really want to try the demo, run it from the commandline with appletviewer index.html.

The Java official appletviewer application is the only one that runs it correctly on all tested platforms.

In any case, it doesn't matter, as I probably won't be releasing an Applet version (too many limitations, too many browser-specific quirks, too many problems regarding file I/O even when ran with permissions, and nearly impossible to debug, especially on instacrashes/grey/black screens of death).

I'm currently experimenting with an AWT interface for the main branch, which has buffered video output and event handling (menus work OK, keyboard and mouse inputs work too, I even set up the main loop but I can't stop it from trying to go into the (unimplemented) demo code...).

You can find that in the testers directory of the current build (testers/AWTMenuTester.java)

Share this post


Link to post

Crashes the current release versions of Safari and FF on my Mac, the Java "loading" screen get stuck on Chrome, but doesn't crash.

Share this post


Link to post

*shrug* as I said, I found a way to make it stable on Firefox and Chromium but didn't update the download, and probably I won't since too much has changed since I made that -admittedly crude- tester.

If you really want to try it in its current form, use appletviewer.

In the CVS there's my current -almost- full AWT-based implementation that loads the menu, intercepts mouse and keyboard input, loads a dummy sound driver (sound interface calls are active but have no effect) and allows going through all menu options (including starting new games) but currently has some trouble when setting up a loaded level with thinkers (goes through loading, calls rendering functions but screen doesn't update unless the menu is called with ESC). I closely based it on linuxdoom's xserver "video driver", like most of the code, for the rest.

Maybe I will make a "disk based applet" based on the AWT code when it's stable enough to load at least the level while allowing playing with the automap alone, but since debugging from within an applet (especially with on-load crashes that generate no logs whatsoever) is almost impossible that's really not a priority.

Status bar, Automap and HU responders are fully functional but I need to implement the net code as well, as even single-player input is actually closely tied to that. The way user input is handled is fucked up though: the Automap handles most of the non-game input including cheats, while actual movement is handled quite "dirty" outside of the Responders code.

Share this post


Link to post

Maes, based on what you have written so far, how big of a leap would it be to go from what you're expecting to end up with at the end of the day over to a Quake/HL-style architecture? By that, I mean putting all the game-specific code into its own module and making the code client/server top to bottom. And does the transition to Java make this any more or less feasible?

Share this post


Link to post
AlexMax said:

Maes, based on what you have written so far, how big of a leap would it be to go from what you're expecting to end up with at the end of the day over to a Quake/HL-style architecture? By that, I mean putting all the game-specific code into its own module and making the code client/server top to bottom. And does the transition to Java make this any more or less feasible?


I think Jake2's code already provides an answer to that: it's surely possible, but they already had such an architecture to step on. In the case of Doom everything would have to be rewritten from scratch. Using my particular Java implementation for that wouldn't really help, as it's deliberately kept close to the C original regarding its inner workings. It would surely provide a first complete object model to base a possible derivative on, but not an all-new client/server architecture, other than what's already present in Doom itself.

Doom's on the other hand is a mishmash of traditional C procedural programming ranging from "crude hack" to "total hack", with some delicate OO flavor thrown in.

That being said, the various modules (sound, video, wad handling, map loading, etc.) are interconnected via interfaces that expose specific methods (or, formerly, C functions) to one another, which would be quite close to a client-server architecture, if it wasn't for the fact that often you need access to some member variable or array, thus forcing you to either go for direct access or riddle the code with getters/setters. That is TOTALLY against any sane client-server model, unless you rewrite these parts (ha).

The p_ modules are the worst, as they need to be aware of EVERYTHING, including the renderer, and in turn the renderer needs to be aware of the objects, and splitting them would require a major redesign.

In any case, I'm almost there (I hope)...the current status looks like some mostly OO-code with some "bad practices" such as abuse of static imports (then again Jake2 used "const interfaces" which is considered a worse practice) thrown in.

To keep the modules communicating, references to the most important, isolated objects (e.g. Menu, Status Bar, Automap etc. ) are passed around in the constructors or in "status update" methods that are called when all objects are instatiated and before they are first initialized (the usual Init(), Start(), Stop(), Ticker() etc. methods are preserved). The task to build the modules in their correct order dependency falls on the "main" method.

Share this post


Link to post
Maes said:

stuff


This is fascinating stuff, thanks. I was aware that what I asked was probably far outside the scope of your project, but your reply was really insightful.

Just out of curiosity, are you aware of how most client/server doom ports manage to actually work given these constraints?

Share this post


Link to post
AlexMax said:

This is fascinating stuff, thanks. I was aware that what I asked was probably far outside the scope of your project, but your reply was really insightful.

Just out of curiosity, are you aware of how most client/server doom ports manage to actually work given these constraints?


Maybe I don't understand the question, but to my best understanding of existing source ports, they are almost entirely "beefed up" versions of the linuxdoom C source code, even those that have been "ported" to C++ Delphi, so to speak, so no major architectural changes to be found regarding the game engine: it's always the same mess of fixed-point arithmetic, unsigned 32-bit BAM "angles", globally visible variables and functions etc. with maybe some refinements for ports such as ZDoom that have their own scripting and extension subsystems.

OpenGL ports have something quite close to a client-server architecture for their renderer, which is essentially rewritten from scratch with the contraint that it must be -the very least- bootstrapped from the same old sprite and BSP data. Then again the implementation details are down to individual ports...

I understand your curiosity though: when I discussed this project of mine with colleagues and other 1337 h4xx0r kidz, I was almost always redirected to Java-OpenGL libraries or even Java-C wrappers and the such, hoping that they'd help somehow.

Unless someone has become "dirty" with the SC, he can't possibly know that the main challenges are at much more basic, elementary levels (e.g. I only recently managed to correctly implement Doom's 32-bit "unsigned" angles with Java!)

Share this post


Link to post
Maes said:

more stuff


Ah, yeah, you didn't quite understand my question (though this is more cool stuff to read about). Ports like ZDaemon (1.06 has open source), Skulltag (97c2 has open source) and Odamex (It's all open source), all have internet play using a client/server model. I was wondering if you know how such a thing was even possible given how unsuited for C/S Doom is.

Share this post


Link to post

Uhh... ah you meant the network play...I thought you meant an internal programming client-server model such as those used e.g. in Unix applications or the GNU/Hurd kernel, or even Quake 2/3.

Well, strictly speaking the original Doom had a "super-peer" network model, where one player initiates the game and the others joined in. This is different than true client-server because the "server" is not really different than the other peers, it also "plays" but is generally responsible for final arbitration on state incosistencies. Works OK in a LAN, but may give the hosting player unfair advantages, since its his machine that can arbitrate inconsistent states due to lag etc.

This is a whole other can of worms, but I understand there are games that support "dedicated servers" e.g. Battlefield or CoD Servers, where the server just keeps state and arbitrates commands/consistency. It's not perfect, but generally fairer when there's too much lag discrepancy between players, and more robust for large number of players.

I guess a "dedicated server" Doom port made, if that's what you ask, but the feasibility of that has nothing to do with it being implemented in Java, C or whatever: it's all about designing a solid message-passing system between the server and the nodes and a way to keep status consistent accross nodes, compensate motion, trading off gameplay fluidity with accuracy/fairness (pretty much like what happens in all real-time online games). But IMO we went far OT :-p

Share this post


Link to post
Maes said:

But IMO we went far OT :-p


Oh I agree, it's just that you were saying some interesting things and I decided to keep picking your brain. :P

Share this post


Link to post

Now that I think about it, setting up e.g. a ZDaemon game and the host player not actually playing would actually be quite close to a client-server architecture. That's a common trick done in games that lack dedicated servers, but if I recall correctly it's possible to start a "server only" instance of ZDaemon on your machine and then joining it with a "client" one. Again, local players will get an advantage over open internet ones, but c'est la vie.

Share this post


Link to post

I've been making a library in Java that can read Doom/Hexen/Strife/ZDoom structures here.

I haven't written the sound information reader yet, but if you were attempting a complete rewrite of the Doom Engine, it could be useful, but I'd understand if you didn't want to rely on another's code.

Share this post


Link to post
Mista_T said:

I've been making a library in Java that can read Doom/Hexen/Strife/ZDoom structures here.


Thanks, I could probably use some reference on the formats I've yet not touched (namely MUS and ENDDOOM lumps). The rest I already handled pretty much by myself, by using the original C source as a reference and sometimes sources such as Jake2 and Ruin.

I ended up adopting a system similar to Jake2, which essentially wrote its own "Serializable" interface (I call it CacheableDoomObject) for objects to "autoread" themselves out of an input stream, recursively calling any of their child-objects deserializers on the same input stream. This allows even for the Doom's caching system to work, kind of, as any object requested from the WadLoader is a "CacheableDoomObject", and can be uniquely referenced in memory even after deserialization.

Some of the most raw stuff like palettes and colormaps are read out of byte buffers straight into raw structures such as byte arrays, while I introduced utility methods to read whole bunches of similar objects from a stream, or to initialize objects in bulk quantities.

The most tricky structure for me so far was the patch_t format, with its implicit "column_t" and "post_t" structs: in order NOT to rewrite the renderer, I kept the format with some syntactic sugar: I read the raw column data into an array, but I arrange each column_t in an array stored under a patch_t (so I don't need the columnofs table anymor), and within each column I precompute the post offsets and lengths at load time, so "skipping" to a particular post is now trivial.

However, your library and what I'm trying to do have slightly different goals: I'm trying to read stuff with as little overhead as possible and use it in a total adaptation of the original Doom, while keeping close to the original code wherever possible (though I "conceded" myself the luxury of using Boom's hashtable system for lump searches).

Share this post


Link to post

I figured that was your goal, anyway. If you need any additional insight for some of Doom's hairier structures, though, I'd be happy to oblige. :)

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×