port to device with 256k Ram

Hello there!

I'm thinking about trying to port doom. The target device only has 256k static ram. Flash is not a problem, the microcontroller is also pretty juicy. Is there any way I could make this work? The WADs are already a lot bigger... 

Any answer would be appreciated:)

No, not unless you massively strip down the game.

Even back in 1994 it ran into serious performance issues with 4MB of RAM.

You need the global texture tables, the sprite animation tables and some other things before you can even think about starting to load a level. And the in-memory representation of a level requires quite a bit more RAM than the naked file size.

First of all, thanks for your answer : )

I don't know so much about memory and stuff (that's also why I want to start this project) so correct me if I'm wrong but wouldn't it be possible to load the WAD in chunks? And only keep the stuff in RAM which is really needed at the moment?

You'll have to completely rewrite the memory management code, then.

Keep in mind though that the entire level is in memory because actors anywhere on it are active. If you load in chunks based on proximity to player position, how do you handle teleporters? The teleport destination can be far away enough from the player so as not to be in the current chunk, so the teleport doesn't have a destination until it loads the chunk with the destination in it, and it can't load this chunk until the player is teleported there.

Yes and no. You can do that for textures, but not for the map. To do anything with the map you need all the linedefs, all the sidedefs, all the sectors all the ... (whatever else is in there) with the sole exception of the REJECT table which isn't significantly large for IWAD maps.

You must also consider that the maps spawns an internal actor object off the things in the map - and since these keep track of these actors' state they cannot be deleted because you cannot recreate them.

Make no mistake here - Doom has been ported to memory constrained systems but these were like the Playstation which had 2MB of RAM plus one MB of texture RAM. You are at 1/10th of that here.

3 minutes ago, Gez said:

 

Keep in mind though that the entire level is in memory because actors anywhere on it are active. If you load in chunks based on proximity to player position, how do you handle teleporters?

Not only that - the level data is also needed for other processing tasks - the BSP would not work if parts of the level were flushed out.

Author DoomHack (and later KippyKip) ported PrBoom over to the GameBoy Advance, which has similar RAM limitations. The maps have to be finetuned and stripped off from details in order to make it work, so whatever you are trying to do with a microcontroller, you likely need to write a converter to strip the details down whilst still looking like Doom.

Thanks for all your answer guys:)) I was already typing "okay i'll  do wolfenstein then" but this is amazing @Redneckerz and a very good starting point! 

1 hour ago, xda46 said:

The target device only has 256k static ram. Flash is not a problem, the microcontroller is also pretty juicy.

I'm more optimistic than others here about your prospects since you have underlying flash memory. Doom's WAD cache and zone memory system was implemented with the assumption that the underlying storage system was slow (ie. hard disk) but that isn't the case nowadays with flash memory and SSDs. You can be as aggressive as you want about moving stuff in and out of RAM from flash and it shouldn't really matter much.

One thing to look into is whether the flash is memory-mapped. If you can access the data in flash directly just like it's RAM then you can bypass the caching entirely. Making a broad assumption about your choice of target device, I'm guessing it probably is memory mapped (otherwise you end up wasting RAM just to execute code, which is wasteful on embedded devices). Chocolate Doom supports this for example through the POSIX mmap API (w_wad.c; w_file_posix.c); I just checked and it seemed to run fine with a 1MB zone heap. I wouldn't be surprised if it can scale down smaller.

If you can make the caching part a non-issue the limiting factor then becomes stuff like the data structures that represent the level and its contents. I'd probably recommend instrumenting the zone memory code so that you can figure out what takes up the most space. At the very least dump some details about all the zone memory blocks whenever you run out of memory.

EDIT: Managed to scale down Choco's heap to 650KB before MAP01 wouldn't load any more. But this is with a 64-bit compile; things would be marginally more efficient on a 32-bit machine where all the pointers are half the size.

The SNES had 256 kb of RAM, split between 128 general + 64 video + 64 audio; but SNES Doom "cheated" by using a SuperFX cartridge with its own extra 512 kb of RAM.

Also, it did not use the Doom engine at all.

@fraggle I have around 7 MB free memory-mapped flash. I could strip some stuff down a bit and get maybe 200K more but that's it (and should be enough, I think, I can remove all the sound stuff... no problem)

As I said, I don't know a lot about anything really low-level memory-related (thus my idea to start this project) so I probably have to learn a lot before actually being able to implement it. Do you have any resources for me where I could start? I don't even know what you mean by a zone heap...

Edit: I have reached the maximum number of posts I can make per day? Okay... whatever...

 If I have other questions I'll just edit this post ;)

There's a posting limit for new users to prevent abuse - I think it goes away after the first day.

7MB of flash is tighter than I expected. I assume this is built into the chip? However, that's more than enough space to fit the Doom executable and the shareware WAD (which is ~4MB).

The zone memory system is essentially Doom's implementation of a virtual memory system that caches data on disk and brings things in and out of memory as they're needed. There's a page about it on the wiki and I can also recommend reading the Doom Engine Black Book which is a great resource on the Doom engine: check out chapters 5.7 and 5.8.

To try to clarify: normally when Doom uses something from a WAD file, it allocates some memory, reads the lump into that memory and then releases the memory when it's finished. In your case that's pointless since the WAD file will be in memory (flash memory) from the start anyway. That should be relatively simple to do and will save you a lot of memory, since usually, the majority of all Doom's memory allocations are just caching stuff read from disk.

14 minutes ago, xda46 said:

Edit: I have reached the maximum number of posts I can make per day? Okay... whatever...

New accounts have a limit just so as to reduce the work for the admins when a spambot goes through the CAPTCHAs and stuff and starts flooding the forums with ads for casinos in Macao or something. If your account survives its first day, you get promoted to full "probably not a spambot" member and no longer have a daily limit.

UPD: It's not impossible to make the Doom game with 256k RAM, but the result would be not so good as PC original does.

xda46 said to me in a PM:

Hello there:)

 

I hope it is okay, that I write you a PM. I still can't post new comments. 

I really don't know where to start. mmap() is a POSIX system call and after some reading, I understand (more or less) how that works and how to use it. Problem is, the target device doesn't have a file system. And it's obviously also not POSIX-compliant. Everything is one big blob. When I started to think about how I could implement this, it thought that I could just use something like xdd -i and include the binary data via a header file (or link it). I wouldn't have to write any memory management, only the video/event stuff and I'm pretty sure even if I'm not the best programmer that that wouldn't be a problem. But that obviously doesn't work because of the memory restrictions.

  

I googled a lot and I just can't find anything about what you called "accessing the flash like memory". Could you give me more info regarding that topic?

 

 Thanks a lot in advance!

Basically there are two ways that flash memory can be accessed by a processor. If you think about how a USB flash drive is connected for example, there's an overhead to read anything from it because you have to send a USB request to the device to fetch the data. It's a similar situation with things like SSDs.

The second way is that the flash memory may be connected directly to the CPU's bus. In that case there's no difference between reading something from flash memory and reading from RAM - it's fast and always available. It sounds like that's the setup you have, which helps.

Sometimes when you have an embedded operating system, it provides a filesystem interface that's backed by the flash memory. That's why I mentioned mmap(), since it means you can avoid wasting RAM just to access the contents of the flash memory (Doom's WAD & zone memory system usually does this). In your case you don't have a filesystem though so it's a moot point.

  

The easiest way for you to proceed is probably to embed the shareware WAD file inside your program. Boom's UTILSRC for example has a program called BIN2C that will convert a file into a C source file containing a huge byte array. Then you might want to take a look at Chocolate Doom's memio.c which contains drop-in replacements for the standard C file functions. That should allow you get the WAD code working without an underlying filesystem - just reading stuff out of the big byte array.

See if you can get the initialization code up to the point where it's running R_Init() successfully - if you can, that should be a big milestone. 

  

 I'm obviously biased but you might find things easier if you start with Chocolate Doom as the codebase since I've already made all the changes needed to support memory-mapped I/O, which you'll probably find helpful later. There are also portability issues you can run into with the original linuxdoom source that id released.

Another suggestion is that you might want to try using miniwad.wad as your IWAD initially since it will use much less RAM and Flash space. 

Yay! I can finally write again! Although already privately said: Thank you!

I will definitely update this thread during my process.  Even though I am not new to programming, I have never done anything like this.  I don't know much about the low level stuff.  So this project is meant to be a learning project and I will (try to) implement everything myself (although a look at chocolate dooms source code won't hurt...)

I'm already having problems getting the original source code to work with a C header file containing the data, so don't expect my updates too soon:)

Managed to get doom loading from a c header file. Seems like not such a big achievement, but I literally had to google every function like fseek() to replicate it because I don't know anything about that stuff. 

Okay, so that's the current state:

V_Init: allocate screens.
M_LoadDefaults: Load system defaults.
Z_Init: Init zone memory allocation daemon. 
W_Init: Init WADfiles.
===========================================================================
                                Shareware!
===========================================================================
M_Init: Init miscellaneous info.
R_Init: Init DOOM refresh daemon - [..                 ]
InitTextures
InitFlats........
InitSprites
InitColormaps
R_InitData
R_InitPointToAngle
R_InitTables
R_InitPlanes
R_InitLightTables
R_InitSkyMap
R_InitTranslationsTables
P_Init: Init Playloop state.
terminated by signal SIGSEGV (Address boundary error)

It is failing in R_InitSprites which calls R_InitSpriteDefs which fails after some loops (probably the memory is full...). But I don't really know how I could do it so it does not use memory but directly the big byte array... If I comment it out, everything goes flawlessly to 

HU_Init: Setting up heads up display.
ST_Init: Init status bar.
Error: xdoom currently only supports 256-color PseudoColor screens

I obviously still have to implement the display stuff...

Nevermind, I'm trying chocolate doom now. I already implemented the video stuff (it should theoretically work). Now the rest.

More or less working now.

What does it look like fam?

Well, console ports back in the day had to make do with far less static RAM than even an entry level PC at the time of Doom had (4 MB). The catch was that at least those which were cartridge-based could use the ROM space as RAM (e.g. 32X and later GBA and others), while disk-based ones had it much tougher. So, if your target architecture/compiler can use Flash/ROM space for storing "const" data and access it just like RAM, and you pre-process the WAD data somehow so that there is no difference between the on-disk and in-memory representation of certain structures (no space for such luxuries here), it should be doable. That might require changing certain loaders as well (in particular those for level data and composite textures).

Hello!

I am not used to be a forum thread necromancer, but I stumbled across this page many months ago when I wanted to assess if the MCU I was considering had enough RAM to run Doom.

Thanks to your contributions I found the awesome doomhack’s GBA Doom port, which I took as the starting point.

That helped me porting Doom to a device featuring only 108 kB RAM (yes, 96+12 kB!), 1 MB internal flash, and 8 MB external SPI flash, which I added to store the Shareware WAD file (this slows down everything, but that MCU has no external memory bus).

With respect to the GBA port, mine lacks music (SFX work well instead), and it has a resolution of  160x128 pixels (however rendered in high detail i.e. no pixel doubling). Furthermore, Z-Depth lighting effect has been restored (it was removed by doomhack to gain some RAM). Moreover, the GBA port used mip-mapping in composite textures, but in my port rendering has been restored to full detail.

All the other features are kept as in the doomhack's GBA port.

Full source code and link to my work will be released very soon! (probably next Sunday)

In the meantime, I have attached a close-up of what I achieved on the development board.

The final device where I ported Doom is, let's say, quite unusual (no, it is not a pregnancy tester!), more info on Sunday!

In the picture, the ammo shows the current FPS multiplied by 10, for testing purposes. On the github code, it will correctly display the ammo.

Finally, the colors are not caught very well by the camera.

Cheers!

That looks like one of Silicon Labs's Gecko Starter Kits at the bottom there. :-)

You are totally right! :)

Ok, I promised you to give more information on Sunday. It is Saturday, but whatever...

I have created a new topic about this here: 

As the topic says, the final device is an IKEA lamp :)

In that topic you will find all the links to the github repository, and the how-to articles.

Cheers!