Chocolate Doom

I hope you don't mind me changing the conversation, but I have a few things to ask about "-setmem". What decision led to dos71 being the default? For Vanilla Doom, is there a difference in the "NULL pointer dereference" between running in DOS 7.1 (Windows 98 SE in DOS mode) and running a VDM in Windows 98 SE's GUI interface? If something that triggers a "NULL pointer dereference" occurs in a multiplayer game with one person running DOS 7.1 and another running DOS 6.22, will the game desync? What exactly is a "NULL pointer dereference" and how doo PrBoom-Plus's demo playback capabilities handle this kind of issue?

On 8/12/2018 at 2:31 PM, PVS said:

kb1
We have been sitting for almost 20 years now, drinking coffee and quietly analyzing each variable and line of code and sometimes we say - here it's bad and there was something strange, right? Of course, for iD it was a job that needs to be done as quickly as possible and make money. Similar hard-coded limits, as here, if sum it all together and wish the stable work of the whole engine - this may well be needed for the hardware of the 90s, not?

Don't get me wrong: I didn't mean to sound superior, and I meant no insult. I just find code like this unusual, especially considering that they wanted to get the job done quickly. Just in this particular case, the code was a lot more long-winded than it needed to be, as if it was maybe written at a frantic pace, which, it may well have been.

22 hours ago, Danfun64 said:

I hope you don't mind me changing the conversation, but I have a few things to ask about "-setmem". What decision led to dos71 being the default? For Vanilla Doom, is there a difference in the "NULL pointer dereference" between running in DOS 7.1 (Windows 98 SE in DOS mode) and running a VDM in Windows 98 SE's GUI interface? If something that triggers a "NULL pointer dereference" occurs in a multiplayer game with one person running DOS 7.1 and another running DOS 6.22, will the game desync? What exactly is a "NULL pointer dereference" and how doo PrBoom-Plus's demo playback capabilities handle this kind of issue?

These questions require detailed answers. I'll try to answer as briefly as possible:

Many programs, like vanilla Doom, have hard-coded limits on how much memory they can use for storing various types of information. Some vanilla examples: visplanes ("visplane overflow"), intercepts ("intercepts overflow"), number of plats ("too many plats!"), save game size limit, etc. When the game was built, the developers had to strike a balance between memory usage, and functionality. So for each of these sets of information, they decided the maximum number of each entity that they be expected to be needed. It turns out that they were very frugal and conservative with memory.

For plats, they decided that no one should need more than 20 plats for a level. So, in the code, they asked the operating system to give them 20 slots to store plats. There are literally dozens of places in vanilla Doom where this was done. Unfortunately, they did not check to make sure that these limits were enforced everywhere. For plats, they actually do check how many plats are in use, and if it's more than 20, the program is stopped with an error message "Too many plats!". But in many other cases, there was no check built into the code. So, sometimes Doom tries to store or read more memory than was reserved for a specific type of information. Sometimes this info "spills into" places in memory that were devoted to other parts of Doom, or even memory not reserved for Doom, but for DOS itself, or the mouse driver, sound driver, etc.

This is called a "buffer overrun", or "underrun", and it results in corrupting the running game code, other Doom data (for example, this causes the 'all ghosts' bug), or it can actually overwrite other programs' code (which usually causes a crash, but not always), or data, or even the OS itself. In extremely rare cases, this type of bug can actually send bad commands and data to hardware devices directly.

When you run Doom.exe, the operating system has to provide memory to load the program in, and memory to store these sets of information. For a given OS that always loads up the same way (like DOS 6.22), you can expect that the mouse driver, sound driver, etc., as well as Doom, is probably placed in memory the same way each time you boot. And, again, for DOS 6.22, the memory layout is probably similar for many people across many computers. Because of this, a buffer overrun may cause a similar effect on a lot of computers, especially if they're using the same OS. In this case, demos just might stay in sync, especially with the same OS, or in a virtual box that virtually loads a similar OS.

Depending on the severity of the corruption, demos may stay in sync even with different OS's. Modern CPUs work with newer OS's to detect that this type of corruption has occurred, in which case the program is usually halted, with a big red X. But DOS 6.22 did very little to detect these overruns.

NULL pointers

In memory, you can store programs, data, or locations of (or 'pointers' to) memory. In the latter case, the numeric position of a data table in memory is stored in another location in memory. If this location becomes corrupted, and the program tries to find data using this location, the program can read the wrong part of memory. And, if the location happens to be 0 (or 'NULL'), your program will most likely crash, because location 0 is always considered invalid, by convention.

PrBoom-Plus Fixes

1. In many cases, once memory has been corrupted, you cannot reliably recover from that. However, in a few select cases, PrBoom-Plus developers spend a lot of time studying the possible effect that certain types of corruption would have. If the corruption is not too severe, PrBoom-Plus simulates this corruption by altering adjacent memory tables the same way that vanilla would corrupt memory, but in a controlled, safe way. This allows PrBoom-Plus to keep such demos in sync with vanilla. It's actually quite amazing!
2. Bad thing usually happen when buffer overruns occur while writing outside of the proper memory range. However, it can be just as bad to read the wrong memory location. This happens in vanilla Doom, though I don't exactly remember where (I think it had to do with moving plats, or doors). Anyway, this bug would cause vanilla to read from a specific memory location owned by the OS. The theory is that this memory location would hold a specific different value for each OS. PrBoom-Plus has a command-line option that let you provide this special value, known to programmers as a "magic number". If you can provide the proper number for the OS that the demo was recorded with, you can help the demo play in sync. This is, of course, a very crude thing to have to do, but it makes the difference between a bad and a good demo. Again, it's pretty amazing that this was found, and that there's a way to watch such demos!

I ended up writing a book, anyway :) Hope that made it a bit clearer - it's not easy to describe in words.

Well you helped answered my last question and maybe my second to last question, but the rest of my questions have been left unanswered.

Quote

What decision led to dos71 being the default? For Vanilla Doom, is there a difference in the "NULL pointer dereference" between running in DOS 7.1 (Windows 98 SE in DOS mode) and running a VDM in Windows 98 SE's GUI interface? If something that triggers a "NULL pointer dereference" occurs in a multiplayer game with one person running DOS 7.1 and another running DOS 6.22, will the game desync?

Now there are other things concerning this topic which I am curious about. For example, alongside -setmem are two "magic number" settings that can be chosen. Is there a way to guarantee that a demo that can play in Vanilla and Chocolate can also run in PrBoom-Plus, or does PrBoom-Plus lack these kind of settings? Also, how are the limit increasing/removing ports like Doom-Plus, Boom, and MBF handled?

On 8/12/2018 at 9:20 PM, Danfun64 said:

I hope you don't mind me changing the conversation, but I have a few things to ask about "-setmem". What decision led to dos71 being the default?

I think we just copied the default from PrBoom+. I suspect the reason probably boils down to "there are a small number of problem demos and most of them were recorded under Win98". But most people who inquire about that parameter pay far too much attention to it. It's only relevant when playing back a very small handful of obscure demos.

This post contains detailed information. As you can see, there are only a handful of demos where the magic number is relevant, out of the tens of thousands of recorded demos. Note that it is only a small subset of the demos that feature a spechits overflow (itself a small portion of all demos) where it is relevant.

IOW: ignore it unless you get a mysteriously desynching demo. Even then, there are many more likely explanations to consider (e.g. wrong wad version) before assuming it is to do with this.

On 8/13/2018 at 7:26 PM, Danfun64 said:

Is there a way to guarantee that a demo that can play in Vanilla and Chocolate can also run in PrBoom-Plus, or does PrBoom-Plus lack these kind of settings? Also, how are the limit increasing/removing ports like Doom-Plus, Boom, and MBF handled?

Well, that's kind of the wrong way round, as Prb+ was the project that investigated this problem and found a workaround. Vanilla (and Doom-plus) can't [edit: that should read "can't be guaranteed to"] play these demos back unless it is running under the same OS.

It isn't revelant to Boom or MBF at all.

Hello and excuse me for my "newbie" question.

I'd like to enjoy Chocolate-DOOM on Raspberry Pi 3 with Raspbian Stretch LITE (so, no X environment and no SDL2 acceleration out of the box).

I can't precisely understand if the SDL2 branch is "the way to go" now on RPi (eventually I'll need to manually compile SDL 2.0.8 with hardware acceleration in framebuffer) or if I can install the SDL1 version and still have all the code updates in terms of gameplay.

Another question is: what are dependancies for chocpkg and where can I find a command example to install Choco-DOOM on RPi?

Thanks!

29 minutes ago, Grazza said:

IOW: ignore it unless you get a mysteriously desynching demo. Even then, there are many more likely explanations to consider (e.g. wrong wad version) before assuming it is to do with this.

To elaborate on my previous comment: I've had people literally email me and ask if using -setmem dos622 would somehow make their Doom experience subtly more "authentic". Almost like a bizarre kind of mysticism around that parameter; if you don't really understand what something does you can imagine all kinds of invisible effects. 99.999% of the time it has no effect whatsoever so I try to tell most people to ignore it.

21 minutes ago, antiriad said:

I'd like to enjoy Chocolate-DOOM on Raspberry Pi 3 with Raspbian Stretch LITE (so, no X environment and no SDL2 acceleration out of the box).

Since you're using Raspbian (=Debian) my advice would be to just run apt-get install chocolate-doom and use the package that comes with your distro. I believe there should be a package in the Raspbian repository.

Quote

Another question is: what are dependancies for chocpkg and where can I find a command example to install Choco-DOOM on RPi? 

Yes, chocpkg is the way to go if you really want to compile the latest and greatest, but it's still poorly documented, so apologies for that.

On a Debian-based system I'd suggest something like:

• apt-get install libsdl2-dev libsdl2-net-dev libsdl2-mixer-dev git automake autoconf libtool
• git clone https://github.com/chocolate-doom/chocpkg.git
• ./chocpkg/chocpkg/chocpkg build chocolate-doom/latest
  (Will give you the latest bleeding-edge version from Git HEAD; or remove '/latest' if you just want the most recent 3.0 stable release)

Thank you very much!!!

I'll try to compile latest (stable) version. ;)

On 8/14/2018 at 2:26 AM, Danfun64 said:

Also, how are the limit increasing/removing ports like Doom-Plus, Boom, and MBF handled?

These ports remove the limits, but do not emulate the overflow of these limits in Vanilla Doom.

On 8/13/2018 at 8:26 PM, Danfun64 said:

Well you helped answered my last question and maybe my second to last question, but the rest of my questions have been left unanswered.

Sorry about that - I gave you what I knew.

On 8/14/2018 at 2:56 PM, fraggle said:

To elaborate on my previous comment: I've had people literally email me and ask if using -setmem dos622 would somehow make their Doom experience subtly more "authentic". Almost like a bizarre kind of mysticism around that parameter; if you don't really understand what something does you can imagine all kinds of invisible effects. 99.999% of the time it has no effect whatsoever so I try to tell most people to ignore it.

I didn't know about -setmem until I read this comment. I don't know why, but I find this very amusing! I totally understand the temptation to use a "-setmem dos622" to "get a more authentic experience" :) Bless their hearts, I can't blame them one bit. This is what Chocolate is all about, after all.

2 hours ago, fabian said:

These ports remove the limits, but do not emulate the overflow of these limits in Vanilla Doom.

Right - actually they just increase the limits a whole lot, by redirecting the memory used for each limit, to a place where more memory is reserved. However, ports like PrBoom-Plus actually grow these memory areas, by asking the OS for as much memory as is needed at the time. This actually removes those limits. Because of this, PrBoom-Plus has to "fake" buffer overruns, because in PrBoom-Plus, the buffers never actually get over-run.

On 8/15/2018 at 6:56 AM, fraggle said:

99.999% of the time it has no effect whatsoever

There's not nearly enough nines after that decimal place.

On 8/14/2018 at 8:56 PM, fraggle said:

Since you're using Raspbian (=Debian) my advice would be to just run apt-get install chocolate-doom and use the package that comes with your distro. I believe there should be a package in the Raspbian repository.

Chocolate-doom package in Raspbian Stretch Repository is 2.3.0 (SDL1 branch). I get a black screen (even with -geometry 640x480 -bpp 24) if in raspi-config I set the LEGACY (Videocore) driver. It is normal? Everything's OK if I enable one of the experimental OpenGL desktop driver, but since I don't use X11 (I'm on Raspbian Stretch LITE) I'd like to understand why I can't use LEGACY driver anymore.

I've manually compiled 2.3.0 and I get the same results.

It seems that things are becoming unnecessarily complicated on RPi for Raspbian Stretch Lite users (this is not a criticism to chocolate-doom, of course).

Thanks

23 hours ago, antiriad said:

It seems that things are becoming unnecessarily complicated on RPi for Raspbian Stretch Lite users (this is not a criticism to chocolate-doom, of course).

I just wish it wasn’t such a moving target. A year or so ago, a git build of choco v3 and sdl2 from the raspbian repos worked great.

I’ve got proper debian on my pi3 at the moment (running 64 bit) but I might have a go of a recent raspbian this week.

7 hours ago, Jon said:

 

I just wish it wasn’t such a moving target. A year or so ago, a git build of choco v3 and sdl2 from the raspbian repos worked great.

I built choco v3 on Raspbian LITE getting a "no video device" error.

At the end I gave up and I switched to "standard" Raspbian (with desktop manager). Removed all that educational junk, cleaned up a bit and installed SDL 2.0.5 hardware-accelerated from the official repository. I was able to build choco v3 and DoomRetro and they both run at full speed. I even installed Quakespasm and it's very fluid (~50fps without optimizations) in 1080p (OpenGL driver).

I'm impressed.

I'm having a small dehacked issue. 

I have changed the chaingun ammo type to cell. When I run out of ammo using the chaingun it drops down and then reselects it, even though it has no ammo. It only happens in choc and crispy doom and not in any other port (prboom, zdoom etc).

I have discussed the issue with someone of great dehacked expertise, so its not something stupid I have done.

The project I'm doing is for UD.

Doom version = 21
Patch format = 6

Weapon 3 (Chaingun)
Ammo type = 2

Any thoughts? Possible bug? 

I guess it's because you still have ammo of type `am_clip` available:

https://github.com/chocolate-doom/chocolate-doom/blob/master/src/doom/p_pspr.c#L188

If you first empty your clips by shooting with the pistol, it will change to a different weapon if you run out of ammo with the chaingun.

@fabian Just checked that out and your right. Anyway around it? Could I change the pick up in dehacked perhaps?

5 hours ago, fabian said:

I guess it's because you still have ammo of type `am_clip` available:

https://github.com/chocolate-doom/chocolate-doom/blob/master/src/doom/p_pspr.c#L188

 

If you first empty your clips by shooting with the pistol, it will change to a different weapon if you run out of ammo with the chaingun.

Why would it do that? It's not like the SSG, which requires two shells. As far as I knew, the chaingun would keep firing until you had completely exhausted your ammo and not until you got to some low number of bullets.

I'm not doubting you, I'm just confused by it.

16 hours ago, Pegleg said:

Why would it do that?

Well, because it works like that in the code.

If you run out of ammo, the CheckAmmo() function checks (among others) if you have the chaingun and at least one clip available. If yes, it selects the chaingun for you. It does not check, though, if you have the chaingun available and the ammo that you have assigned to the chaingun by means of a DEH patch.

1 hour ago, fabian said:

 

Well, because it works like that in the code.

 

If you run out of ammo, the CheckAmmo() function checks (among others) if you have the chaingun and at least one clip available. If yes, it selects the chaingun for you. It does not check, though, if you have the chaingun available and the ammo that you have assigned to the chaingun by means of a DEH patch.

OK. So that's why I was confused. I missed the Dehacked portion of the discussion. I was just thoroughly confused because I did not remember that happen on Chocolate Doom, but I had not tried to replicate it in Chocolate Doom on a map that was patched in such a way.

By the way, thank you for the fuller explanation, instead of just leaving it at your first sentence.

The strange thing is I don't remember this problem from I used to mess around with dehacked years ago. But looking at the code, it clearly would of been an issue back then.

Quick question, are the current instructions for building Chocolate Doom on Windows up to date?

There was some discussion last year on GitHub about incorporating CMake (link to that discussion), but I haven't been able to find more info about this.

I believe they should work, yes. Page was updated earlier this year.

Happy to confirm too, just did a test build and got a working executable. ✌ I did get a few warnings/errors (under the spoiler), but they seem completely inconsequential and didn't appear to affect the build in any way.

./convert-font small small.png small.h
WARNING: Could not update small.h. Please install the Python Imaging library orPillow.
./convert-font normal normal.png normal.h
WARNING: Could not update normal.h. Please install the Python Imaging library or Pillow.
./convert-font large large.png large.h
WARNING: Could not update large.h. Please install the Python Imaging library orPillow.

for f in README.md README.Music.md COPYING.md NEWS.md; do                             \
        pandoc  -f gfm -o staging-doom/$(basename $f .md).html  \
                -s ../../$f;                     \
done
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
make: [GNUmakefile:44: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/CMDLINE.html ../../man/CMDLINE.doom.md
make: pandoc: Command not found
make: [GNUmakefile:45: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/INSTALL.html ../../man/INSTALL.doom
make: pandoc: Command not found
make: [GNUmakefile:46: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/NOT-BUGS.html ../../NOT-BUGS.md
make: pandoc: Command not found
make: [GNUmakefile:29: hook-doom] Error 127 (ignored)

19 minutes ago, Lollie said:

Happy to confirm too, just did a test build and got a working executable. ✌ I did get a few warnings/errors (under the spoiler), but they seem completely inconsequential and didn't appear to affect the build in any way.

 

  Hide contents

These three warnings appeared after using "make" in the "chocolate-doom" directory.
(included the commands as reference for when the warnings appear)

./convert-font small small.png small.h
WARNING: Could not update small.h. Please install the Python Imaging library orPillow.
./convert-font normal normal.png normal.h
WARNING: Could not update normal.h. Please install the Python Imaging library or Pillow.
./convert-font large large.png large.h
WARNING: Could not update large.h. Please install the Python Imaging library orPillow.

And this set of errors appeared for each game (doom, hexen, etc) after building release zips in the "pkg/win32" directory.

for f in README.md README.Music.md COPYING.md NEWS.md; do                             \
        pandoc  -f gfm -o staging-doom/$(basename $f .md).html  \
                -s ../../$f;                     \
done
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
/bin/sh: line 1: pandoc: command not found
make: [GNUmakefile:44: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/CMDLINE.html ../../man/CMDLINE.doom.md
make: pandoc: Command not found
make: [GNUmakefile:45: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/INSTALL.html ../../man/INSTALL.doom
make: pandoc: Command not found
make: [GNUmakefile:46: staging-doom] Error 127 (ignored)
pandoc -f gfm -s -o staging-doom/NOT-BUGS.html ../../NOT-BUGS.md
make: pandoc: Command not found
make: [GNUmakefile:29: hook-doom] Error 127 (ignored)

I'm assuming these can be safely ignored, but I figure it's worth mentioning these warnings in any case.

 

I know this might sound like a dumb question, but "Have you installed the Python Imaging library or Pillow"?  I know if I were compiling from source, I'd want to see as few errors (preferably none) as possible, no matter how small they were. 

I know, lol. I just installed the build dependencies exactly as they were listed in the Choco Doom wiki article. Copy and pasted the install command right into the terminal, leaving nothing to chance. All I can assume is that the Python Imaging library / Pillow are not included in that command, and weren't installed when updating MSYS2 with "pacman -Syu", either. Same with "pandoc".

For what it's worth, I'm not seeking help for the warnings. I'm just stating I encountered them, in case anyone else attempting to build Chocolate Doom on Windows happens to encounter the same warnings.

pillow is probably completely optional, I don't think there are any drawbacks to not having it installed.

pandoc is used for converting the Markdown files to HTML, you might check "pacman -Ss pandoc" to see if it is available.

This weekend I implemented some improvements to the digital music pack support. If there are some volunteers who could help me test these out I'd really appreciate it.

Instructions:

1. Download the daily build of Chocolate Doom. If you've been using music packs already then please nuke your existing config.
2. Download some of the music packs from @MusicallyInspired's website.
3. See if you can configure choco to use the music packs using the setup tool and without referring to any documentation.

Feedback requested, please let me know if you succeed (or don't), any problems you encounter etc.

EDIT: Realised one unintuitive thing already - you need to change the music type to "Native MIDI" inside the setup tool. I'll see about refactoring to fix this.

I just remembered Symbian port of Chocolate Doom. I found it in some russian site for smartphones in 2012 or 2013. I used it to play Doom on my Nokia C5.00 5MP. Author of port named himself "someOne" if i remember correctly. He also made many ports of other PC games and programs for Symbian.

ChocolateDoom_S60v3.7z