RealDOOM: DOOM Ported to 16-bit Real Mode

[sqpat]

14 hours ago, viti95 said:

Another idea to reduce memory usage, compress better the IWADs. @fraggle has updated wadptr and fixed old issues, this reduces the amount of memory used for certain graphics, and specially the size of the sidedefs.

 

Hmm, so my original intent was to make it be compatible with the original commercial WADs. However, there's no reason I can't add some initialization code (in an overlay, which wont take up ingame runtime memory) to run the wadptr code on the input commercial WAD at startup if it's detected as uncompressed then  to compress the IWAD ... so for now, I will probably just use the compressed WADs and make a TODO  to port wadptr into some 16 bit code that can optionally run at startup.

 

Thanks for the suggestion. Some doom 2 sidedef lumps were over 64k and I didn't really want to write a kludge to deal with that.

 

 

Edited November 22 by sqpat

[sqpat]

Well, I spent a week cleaning up bugs but also trying out some optimizations.

 

There are some notes in the comments of the codebase about precalculating lineopenings and i gave it a try. You can precalulcate it in level setup but have to treat it as a cache and mark things dirty whenever platforms move and sector floor/ceils change, basically. It wastes 6-7 KB and unfortunately its hard to measure the improvement. I tried average ten runs worth of timedemos and it's a tiny bit faster, maybe 0.25% on average or so?  (is there a better way?) . 

 

I tried 8 bit validcounts for linedefs to save some space but it seems that's not enough uniqueness to avoid overlaps/collisions of values and bugs happen like a thousand tics into demos.

 

Removing the back references in mobj_t (for sector lists and block lists) is not hard to do  since they aren't used all that much. A few KB can be saved by rewriting a bit of code and removing sPrev and bPrev fields. I think they are ultimately a tiny bit measurably slower, so I may revert this later.

 

I'm realizing that if i can get the EMS 4.0 improvements I mentioned earlier to work, then I will have way more memory than i need during the physics portion of the code, so all these tricks to reduce sizes of certain fields wont really matter - what will matter is only speed during that phase, and memory usage becomes more important during rendering due to the amount of memory used by textures.

 

I'm going to clean a couple remaining broken features and known bugs, make a simple release build that can hopefully run all the shareware levels, then after that focus on allocating UMB space and EMS 4.0.

[sqpat]

I've managed to fix a half dozen or so known bugs (corrupted palettes, timedemo desyncs, readthis rendering, level intermission crashes) and all the demos are running correctly again with all the improvements from the past few weeks. There is a weird random bug happening where the level begins to render in a mess - A lot of the walls suddenly rendering at the wrong angles among other garbage. I've managed to dump all the level data (sectors, lines, segs, etc etc etc) from memory to file after this has happened but all the data checked out as good. Automap also renders fine when this happens. I don't think this seems like a visplane bug either. I will probably check a few other things like if the trigonometry lookup tables are getting corrupted or something.

 

I think once this last bug is fixed, I will cut the 0.1 release based off of that. Basically, all the shareware content plays fine except for E1M6 which is so much bigger than all the other shareware levels - I would need to clear up another 30kb or so to make it fit. That'll be easy once I have UMB allocation working, which should be another easy 64k (at least)- I will probably update with another minor release after that, then begin on EMS 4.0 work, which feels like it will free up at least another 100k. Stability is as good as it's ever been, outside of the random render bug popping up every couple minutes and other known issues (savegames, sound, nightmare respawns) I don't see too much wrong anymore.

 

[sqpat]

0.1 Release

 

OK, the game has been working pretty cleanly in DOSBOX again, which for it's faults with compatibility at least makes development cycles so much faster since I don't have to mount images and stuff. Thanks to that I got tons of work done in the past few days - I cleaned up a dozen or so bugs and also converted lots and lots of physics and render code to 16 bit logic (then fixed the bugs that caused). Tomorrow I will figure out how github releases work and update the project page there, but attached here is the exe I assume will go out as RealDOOM 0.1.

 

Basically this should work with all maps in shareware except E1M6. I don't *think* it will crash anymore - I haven't seen funny memory bugs recently, I think they are all fixed... There's some features not in there yet like sound of course, savegames, and screen wiping. There are some render bugs like overdrawing in the intermission screen, and a minor fuzz draw bug in timedemo3 and a also some sprite masking bugs. 

 

You need about 605-610k for this one, a little less than the previous one but if you use a standard nearly blank MS-DOS 6.22 config with EMM386 loaded (sample config.sys in the zip file) it should work. The zip also has the dstrings.txt file which is necessary. Don't forget to include DOOM1.WAD

 

The 286-25 bench (screenblocks 5 hi quality Demo3) ran at 31821 realtics, compared to 36074 a month ago. That corresponds to about 12% faster, and the fps is around 2.37 now. This has pretty much just been from rewriting algorithms and refactoring code - there's still no ASM or anything. The pentium times are around 10% faster from the same period... it's hard to say for sure but I think some of the improvements aimed at the 286 specifically (like reducing shifts) have been beneficial.

 

EDIT: 

I've run some extra benches, and it seems like on real hardware (vs 86box) i get 15% faster times on my 286 and 4-5% faster times on my 386 DX-40. My mmx meanwhile ran about 30% faster. Not really sure what's the reason, maybe memory access times or something, but it's a good sign anyway.  I may try to get it to run on the turbo xt tomorrow for funsies... i wonder if it will beat 100,000 realtics...

 

realdoom_0.1.7z

Edited Monday at 08:08 AM by sqpat

[realjohnmadden]

Cool stuff. I'm surprised it runs, slow as it may be. I tried running it normally as well as running it on the highest cycle speed in DOSBox for funsies, and you can very well brute-force your way to a stable-ish 20fps through emulation, although that defeats the point.

 

Not sure how helpful this will be, but I'll post some of my findings after a lot of messing around:

 

- Doom 2 does not run, no matter what. I slimmed the IWAD down massively - only the textures used in MAP01 (as well as only MAP01), removing all of the sprites for the Doom 2 enemies and any enemies that aren't in MAP01 (e.g Pinkies, Chaingunners), and removing some sounds. It's now at just 4,637 KB, and it runs fine in the original executable, but not here, giving a "ran out of refs" error, presumably something to do with memory.

 

- After beating E1M8 with the shareware IWAD, the text scroll doesn't show up. The game just hangs before it even appears, perhaps it's because of the screenwipe? I'm not super knowledgable about Doom's inner workings.

 

- Compressing the truncated Doom 2 IWAD using wadptr didn't cause an "out of refs" error, but instead hanged on Init Playloop State. The same happened with the shareware Doom IWAD. Both IWADs work just fine in the original executables, so it seems it has trouble with the compressed wadptr output.

[sqpat]

11 minutes ago, realjohnmadden said:

Cool stuff. I'm surprised it runs, slow as it may be. I tried running it normally as well as running it on the highest cycle speed in DOSBox for funsies, and you can very well brute-force your way to a stable-ish 20fps through emulation, although that defeats the point.

 

You can definitely get faster fps on 86box by a factor of 2 or 3 so, but yeah dosbox should at least be playable and is generally more convenient.

 

11 minutes ago, realjohnmadden said:

- Doom 2 does not run, no matter what. I slimmed the IWAD down massively - only the textures used in MAP01 (as well as only MAP01), removing all of the sprites for the Doom 2 enemies and any enemies that aren't in MAP01 (e.g Pinkies, Chaingunners), and removing some sounds. It's now at just 4,637 KB, and it runs fine in the original executable, but not here, giving a "ran out of refs" error, presumably something to do with memory.

 

Oh, yeah - sorry, I thought that was clear - only shareware doom is supported right now. Doom2 generally has larger levels and different content and will need more work. I haven't tested doom2 content at all and it may be buggy - especially viles and the last boss and stuff. Once I've freed up a lot more memory via some upcoming improvements, it'll make more sense to get things like doom 2 or sound working,.

 

11 minutes ago, realjohnmadden said:

- After beating E1M8 with the shareware IWAD, the text scroll doesn't show up. The game just hangs before it even appears, perhaps it's because of the screenwipe? I'm not super knowledgable about Doom's inner workings.

 

 

Oh yeah, the finale... I've not tested at all! I'm sure it's probably broken, haha. Maybe it is because of the wipe. Wipes were removed because they basically need 128KB of free space to run (two VGA screen buffers) which is a huge chunk of your 640 KB. Once EMS multitasking is in there I can probably make it happen.

That reminds me - level restarts after game over also don't work - perhaps because of the screen wipe. You have to go to the menu and select a new game in that case
 

 

11 minutes ago, realjohnmadden said:

- Compressing the truncated Doom 2 IWAD using wadptr didn't cause an "out of refs" error, but instead hanged on Init Playloop State. The same happened with the shareware Doom IWAD. Both IWADs work just fine in the original executables, so it seems it has trouble with the compressed wadptr output.

 

Wadptr will crash without extra work on the codebase due to some of it's optimizations. In order to save space on indexing the wad, I calculate and store the sizes in a certain way and when wad entries overlap it creates 'negative size' entries which leads to trouble. I think what's likely to happen down the road is that rather than using wadptr, I will basically generate the wadptr-style compressed wad at runtime by finding duplicate entries and filtering out duplicates.

 

Thanks for your testing and input!

 

[sqpat]

Some updates - I got UMBs working, and now pull around 70k from there. I then went ahead and allocated a bunch more memory to level data allowing e1m6 to be playable. The conventional memory usage for the build went from 615818 -> 543738 -> 569802, which is still pretty comfortably low now.

 

I might fix a couple bugs and make a quick 0.11 release before going heavy on EMS 4.0 multitasking prep work. EMS 4.0 hardware is somewhat more difficult to come by, especially for a board that will be compatible with XT class systems. The lo-tech EMS boards won't work, and a lot of other ones are 16 bit only I think. Later, faster 286es have chipset support and aren't a problem. I'm not sure how emm386 will work out yet. 

 

Meanwhile, I benched the 0.1 release on a few pieces of hardware over the past day. Below is a full quality timedemo 3 playback on a 4.77 mhz 8088.  It's a little under a million realtics, about a 7 hour video and 0.0888 fps :) 

 

 

 

A V20 at 9.5 mhz finished the typical 5 screenblocks hi detail run bench I tend to run at 162157 realtics, which is about 1/5th of the speed of a fast 286. I have a 16 mhz turbo v20 board I can try it on too at some point.