Things about Doom you just found out

[Da Werecat]

People are rarely surrounded with black fog only they can see. That's why light diminishing is not entirely realistic, not because it fades wrong or something.

But light diminishing is still a good effect, because it creates volume where Doom's simple sector lighting can't.

[Linguica]

Never_Again said:

Simply amazing. That means you could disconnect all streetlights in Manhattan and hoist just one to the top of the Empire State building, The streets at night would be just as bright as with thousands of the current streetlights, but think of all the electricity this would save.

There just might be a Nobel Prize in this for you, Linguica.

I don't know if you're being willfully obtuse or what. The apparent brightness of an object does not diminish as it gets further away from you. If you're standing 100 feet away from a wall, the wall does not look dimmer than if you are right next to it.



This is a gray card, which you've probably seen being used in photography / film to provide a reference for proper exposure. It reflects a particular amount of ambient light. Notably, it does not matter how far away from the camera the card is, because given the same amount of light hitting it, it will always appear to have the same brightness, no matter its distance or apparent size, because that's how the math works. It does not fade to black as it gets further away. Camera distance does not affect exposure.

[Linguica]

On a hopefully less contentious subject, I was thinking more about the partial invisibility effect and I think I can reasonably claim that the existing implementation is bugged. An subtle algorithmic bug, and one with an outcome that id decided to keep, but still bugged.

The normal code uses as the engine steps down each column of the invisible sprite is:

*dest = colormaps[6*256+dest[fuzzoffset[fuzzpos]]]; 

Where fuzzoffset[fuzzpos] is either a certain positive value or a certain negative value. The effect of this is that when it's positive, the engine copies the pixel value below the current position, and when it's negative, it copies the pixel value above the current position. The value is then darkened a little by the "6*256" part and applied to the current position. The effect of this is that when the code is going down a column, and it copies the pixel above the current position, it's copying a pixel that was just itself copied and darkened, so the new value gets darkened a second time. If the next pixel down gets copied from the pixel above it, then the same value gets darkened a third time, etc.

Like I said, this is a fairly subtle bug, but it does seem like it's due to an ill-thought-out algorithm, rather than an intentional effect. Anyways, you can "fix" it by only darkening a pixel if you're copying from below it, since any pixel copied from above is being copied from an already-darkened one. (Except for the very first pixel in a column, so we special-case that as well.)

if(fuzzoffset[fuzzpos] == -FUZZOFF && count != (dc_yh - dc_yl))
	*dest = colormaps[dest[fuzzoffset[fuzzpos]]]; 
else
	*dest = colormaps[6*256+dest[fuzzoffset[fuzzpos]]]; 

Anyways, this is the result:



The spectres are definitely a little harder to see, but the pixel-warping is more obvious as well. Look at the UAC logo on the door - in the normal version, it mostly appears as if it is rendered normally but with a sort of stipple effect applied on top of it, whereas in the "fixed" version it's clearly being warped.

[Gez]

Never_Again said:

Simply amazing. That means you could disconnect all streetlights in Manhattan and hoist just one to the top of the Empire State building, The streets at night would be just as bright as with thousands of the current streetlights, but think of all the electricity this would save.

There just might be a Nobel Prize in this for you, Linguica.

Light diminishing in Doom isn't about the player being a lamp atop the Empire States Building, it's about areas appearing darker when you're far away from them.

It's very realistic: if you look out at the sun, you'll notice you don't see the star, because it's so far from Earth that its brightness becomes 0. By night time, you have the same effect: turn on the lights, then go outside and look at the windows. If you start walking away from your home, you'll notice that suddenly the lights go out and the windows are dark again.

So as you can see, it's a very very realistic effect that is used in the Doom engine.

[sheridan]

How dumb do you have to be to believe light actually dims to nothing after travelling just a few meters, eesh.

But yeah, doom definitely looks better with it anyway, if only because it forces some shading on what are otherwise very flat and pixellated surfaces.

[Blastfrog]

Wow, Linguica, you've been kicking ass and taking names as of late. It really is interesting to see the remnants of ideas they started to put in there but didn't go forward with.

Could you do an analysis on the Jaguar version's lighting? I don't know what it's doing but it seems different, somehow. It almost seems like it subtracts luminance as low as half-brighness or something, but I can't tell.

It looks quite a lot like how the walls were shaded in Wolfenstein 3D, which I'm pretty sure was subtracted from and not multiplied by a fraction, there's too much contrast/"highlighting" to appear to be what's going on.

[sheridan]

Wolfenstein's renderer doesn't actually shade the walls at all. Adrian Carmack instead had to manually produce "bright" and "dark" variations for each wall texture, which were then applied in alternating fashion to each side of the wall cubes (so the north and south sides of a wall would use the "bright" textures, and the west and east sides would use the "dark" textures). In a few cases id took advantage of this to produce walls which looked drastically different on each side.

source: I hacked wolfenstein a lot ten years ago and distinctly remember that this was a thing.

[dg93]

I just found out that when you rev up the chainsaw to kill your enemy, and you die at the same time, the chainsaw makes this sound: http://soundfxcenter.com/video-games/doom/088c25_Doom_Chainsaw_Sound_Effect.mp3

[Platinum Shell]

the_miano said:

I just found out that when you rev up the chainsaw to kill your enemy, and you die at the same time, the chainsaw makes this sound: http://soundfxcenter.com/video-games/doom/088c25_Doom_Chainsaw_Sound_Effect.mp3

Same thing happens if you draw the chainsaw and then put it away right after; its idle sound cuts off the equip sound.

[Blastfrog]

sheridan said:

Wolfenstein's renderer doesn't actually shade the walls at all.

Thanks, but I already knew. I was just saying the way the walls were shaded look more like Jaguar Doom's darkening method than PC Doom's. I'm more concerned with the algorithms than I am with whether or not it's engine-side.

[printz]

Platinum Shell said:

Same thing happens if you draw the chainsaw and then put it away right after; its idle sound cuts off the equip sound.

That's only because you use ZDOOM.

[Never_Again]

Linguica said:

I don't know if you're being willfully obtuse or what. The apparent brightness of an object does not diminish as it gets further away from you. If you're standing 100 feet away from a wall, the wall does not look dimmer than if you are right next to it.

It does not when the light source is static, like the sun, for example.
It does if the light source is you.

Linguica said:

<...> Camera distance does not affect exposure.

From the linked page:

Light appears dimmer at greater distances from the light source

[Linguica]

Never_Again said:

It does if the light source is you.

Oh, the "the doomguy has a lighted visor" theory. OK, we can turn that light off then too.





Hmm, very Doom 3 esque...

[Never_Again]

You got me stumped now, Ling. What are these GIFs meant to illustrate? Perhaps the frames are too brief for me to catch it. Or maybe it's the rare case when two dozen words are worth two pictures. :)
BTW, I appreciate your explanations and the extra effort with the pictures to elucidate them further.

[Marnetmar]

Linguica said:

I was bored and thinking about part of the original press release Tom Hall wrote for Doom:

And I was thinking, well that's all well and good, but "light diminishing" is hardly a realistic phenomenon. Lighting doesn't actually get dimmer the further away you are from it, that's silly. There's no "black fog" in real life. So I wondered what it would look like if I disabled it:

Turns out when things don't fade into darkness in the distance the game looks a lot worse, who knew??

I think it'd be cool if a source port made the lightamp visor work like this.

[mouldy]

Linguica said:

The apparent brightness of an object does not diminish as it gets further away from you. If you're standing 100 feet away from a wall, the wall does not look dimmer than if you are right next to it.

This is interesting, I've never considered this in relation to doom's lighting. I can only imagine the reason it looks more "real" in doom to have the light fading with distance is because it gives the illusion of more natural graduated lighting in rooms, which are usually centrally lit and have dark corners.

[40oz]

I think dooms lighting can be realistic enough in that in dark lighting in real life can make the details of an object unclear until you get closer to them. A person can be a shadowy in the distance but up close you can see enough to determine if they are a man or woman, what kind of clothes theyre wearing, etc. So in the setting of doom, it does well to simulate the feeling of seeing something, not being sure what it is, and having to get close enough for it to get you to get a good look at what it is.

resolution isn't entirely to blame for something like that.

[Marnetmar]

Sodaholic said:

Note how the alpha versions deliberately y-sheared your view downward, you did not look straight-forward like the final version. That would make the no-ceiling mode less of a loss since you see less ceiling in this perspective anyhow. It also kinda looks "aim-down-sights-ish" and would allow more of the cut-off weapon sprite to be used (probably why those cut-off areas exist to begin with, aside from how the weapon raised when firing like in Wolf before that was cut).

[Linguica]

While thinking about Doom lighting, I wondered how exactly the Doom engine calculates which light level to show.

Remember that even though the Doom engine shows sector light levels as 0-255, the engine internally only really cares about 16 light levels. (So basically 0-15, 16-31, etc.)

The engine calculates how to color a pixel based on the light level of the sector it's in (duh) and the distance from the player, using an arcane equation. Floors / ceilings are somewhat more interesting than walls, because the lighting table for them has 128 entries per lighting level.

Anyways, here's the COLORMAP entry for a random medium-gray in the palette. This shows the 32 possible light levels for this color. (Well, 21 colors in this particular case, because the palette doesn't have enough colors for it to be perfectly smooth.)



And here's the table for this color as applied to floors and ceilings. Sector light level is Y-axis, distance from player is X-axis:



A couple interesting things to note:

* at 0 distance from the player, a color is fullbright no matter what. This never occurs in the game because it's impossible to see a floor/ceiling that is 0 units away from your eyeball.

* there is no sector light level in the lighting table that results in the game using anything CLOSE to the full 32 entries of the colormap available. The most colors mapped to any one sector light level is 16 out of those 32. That means that when drawing flats, the game is only using half the lighting precision it could potentially use. That's kinda lame. Maybe that helps explain the light banding visible on floors / ceilings?

* In any event, what a waste that table is. The entire right half of the table is basically pointless. Storing light levels in linear increments is not the best way to go about this; in a perfect world, you would want your table to store values as the square root of the distance or something similar, so you got more fine-grained banding closer to the player, and less entries for values way off in the distance, where it doesn't really matter. Obviously square roots are not fun if you're trying to avoid floating point math, but Carmack could have used some tricks like a LUT, and even if the result is off by a couple percent, it's just for lighting and it wouldn't have been very visible if at all.

Aaaaaaanyways, a partial "fix" for the banding is incredibly simple. You take the code defining some stuff about the lighting table:

#define MAXLIGHTZ	       128
#define LIGHTZSHIFT		20

And increase MAXLIGHTZ while decreasing LIGHTZSHIFT like so:

#define MAXLIGHTZ	       256
#define LIGHTZSHIFT		19

And get double the number of potential bands of color. Hell, you can keep increasing the table to make it even more precise:



Even doubling the table from 128 to 256 entries makes a TON of difference, and you get further diminishing returns from there. In a sense, it's kind of crazy that Carmack left the table at 128 entries instead of at least 256; it's a very minor increase in memory, no further processing overhead I can think of (except for generating the table, which is only ever done once, so who cares), and produces noticeable better looking results.

I'm not sure this has ever really been changed in a software port. I do know that Maes messed with these values for his true-color Java port, but that's a little different.

[Szymanski]

mouldy said:

This is interesting, I've never considered this in relation to doom's lighting. I can only imagine the reason it looks more "real" in doom to have the light fading with distance is because it gives the illusion of more natural graduated lighting in rooms, which are usually centrally lit and have dark corners.

It also avoids distant enemies like a trooper or sergeant appearing as a bright red pixel. Doom doesn't have mip maps, and I don't believe it bothers to actively choose the best colour. Making the image darker makes the image less distracting.

[_bruce_]

The fuzz effect was something Dave Taylor came up with if i remember correctly.
There's is one caveat - line (199) isn't drawn not when a monster or a weapon would fill this line. This is probably a hasty effort to prevent out of bounds drawing.
The light falloff is pretty limited in 8 bit drawing since it can only tap into the same 256 colors available which explains why things are browning or greying out.
I added two modes in ChocolorDoom - one which uses the old light table solution to its "full" potential and one which calculates the falloff. The array based solution looks better and is more dynamic, meaning the move from dark to light and vice versa is more dramatic/aggressive.
The massaging of the numbers for the lookup version was a pain in the butt.
Carmacks idea for the light falloff is very simple but pretty effective... as usual.

[Linguica]

Here's a chart of flat lighting. I reversed the colormap axis because it's more logical to think of 0 as darkest and 31 as brightest.



You can really see how generally useless the lower half of the sector lighting range is; especially below 112 or 96, flats fade to black almost instantly.

If you're mathematically inclined, you might say, "hey, this looks kind of like a hyperbola." Well, it's not "like" a hyperbola, the lines ARE hyperbolas.



Hyperbolas aren't exactly known for their use in lighting functions, which is undoubtedly one reason for the, uh, "unique" look of Doom's lighting. In this case, however, it does allow the lighting to be nicely pixelated:

[kb1]

Linguica,

Question #1:
So, what happens to your "partial fix" when you change resolutions? One big issue with vanilla Doom lighting is that it uses the texture scaling values directly to calculate light level, which means that it all goes to shit when you increase resolution. Most ports try to compensate for this by bit shifting.

So, my questions is, is your method of increasing the LUT robust, in that the resolution-independent scaling hacks still work well enough? Stated differently, using your method, does 320x200 and 1920x1080 look similar in light levels?

Question #2:
Does your method track linearly with the original code, so a room that's say 500 units away look the same brightness using your code as it would not using it?

Basically, I'm trying to figure out the pros and cons. If both questions are "yes", I would consider adding this new lighting method as a rendering option to my port. I always wondered why the bands were as defined as they are. I tried to go from 16 to 32 levels, and it did not seem to make much difference, and now I know why :)

Thanks, again, for the informative lecture.

[Linguica]

I believe so, yes.

Basically, at runtime, the engine fills out an array, zlight[LIGHTLEVELS][MAXLIGHTZ]. LIGHTLEVELS is the number of different internal sector light levels, so 16. MAXLIGHTZ is 128, which is basically a "magic number". For each of the 16 light levels, the engine takes each cell from 0 to 127, does its weird distance calculation, which includes a distance >> LIGHTZSHIFT, and fills in that cell with the final light level for that combination of sector light level and distance.

Think of it like the engine laying out a 2048-unit-long ladder on the ground in front of you, where the ladder has 127 equally spaced rungs. Then the flat in the space between each rung is drawn at a particular light level according to which space it's in.

So when you double MAXLIGHTZ, you have the same 2048-unit-long ladder lying on the ground, but now there are 255 rungs, and again, you determine the light level of each portion of flat between each rung.

This means the calculations have double the granularity, so you decrease LIGHTZSHIFT by one so that doing distance >> LIGHTZSHIFT is double the size and thus double the granularity.

There's nothing particularly special about the numbers or why they are powers of two, except for being able to do the obvious bitshifting -- you could have a 1337-cell array and then divide the distance by some x*1337 value, and it would still work the same.

As far as I know, it should "just work" regardless of resolution. Everything will look exactly the same, except there will be twice as many potential bands on the floor/ceiling light gradient.

What it *won't* do is provide an easy path to eliminating banding altogether. at 320x200, increasing the array to 2048 cells (I think?) was enough to basically make it so each row of pixels ended up with their own row in the array, but at higher resolutions you would need to make the array ridiculously large.

[june gloom]

Ling I want to echo the sentiment that you're doing the Lord's Work here. I'm not a programmer so I can only barely understand the basics of how Doom works (and most of that through osmosis) and you're doing a good job of helping right-brainers like me understand.

[kb1]

Linguica said:

I believe so, yes...

Very nice. You know, I looked at that code many many times, and it never really dawned on me - never really clicked.

Linguica said:

As far as I know, it should "just work" regardless of resolution. Everything will look exactly the same, except there will be twice as many potential bands on the floor/ceiling light gradient.

Well, the reason I mention it is that, for most ports, the light does get kinda funky when you change resolutions to a value somewhere inbetween powers-of-2, in relation to 320x200. Ports try to compensate, and, for speed, bit shifting is involved. But, that doesn't quite cut it.

Try it in a few ports. Take a screenshot of 320x200, then 640x400. These should look about the same. Then, go to something not a power of 2, like 1024x768. I think you'll see the whole scene get a bit brighter or darker, cause, at 1024, that bit shift is set to (1024/320) = 3.2, so the bit shift = 3, missing the 0.2 portion.

Or, maybe my implementation sucks, and I got it wrong, I don't know. Personally, I looked at how some other ports attempt to fix the issue, and I fiddled with it until I got it close.

I would love to have an exact, mathmatically correct way to ensure that the scene is rendered with the exact correct lighting, regardless of resolution. Maybe a combination of the bit shifting, and your LUT-resizing technology would provide exactly what I'm looking for, without slowing things down.

The only issue I see with larger LUTs is memory cache issues. When custom colored lighting is used, the lighting array becomes 3-dimensional, so it could get large, I think.

[Linguica]

kb1 said:

Try it in a few ports. Take a screenshot of 320x200, then 640x400. These should look about the same. Then, go to something not a power of 2, like 1024x768. I think you'll see the whole scene get a bit brighter or darker, cause, at 1024, that bit shift is set to (1024/320) = 3.2, so the bit shift = 3, missing the 0.2 portion.

OK, I see what you're saying. The code *does* use SCREENWIDTH in setting up the table, BUT it assumes the value is 320 in order to get the proper levels. So you should go in and hardcode it to 320 in this instance. See, e.g., here

kb1 said:

I would love to have an exact, mathmatically correct way to ensure that the scene is rendered with the exact correct lighting, regardless of resolution.

Well... if you wanted to keep the same setup with 8-bit palettized graphics, 16 sector light levels, 32 colormap light levels etc, there's no point in having a full LUT like this at all. If you look at the graph I made, you'll see that the distance where each colormap light level drops to the next is the same regardless of sector light level. All you would need to do is store the distances of each discontinuity, and then do a simple comparison with the unique z-distance of each line being drawn, which is something you have anyway.

Later tonight I might try this and just replace the LUT with a big switch-case structure and see if it works.

[kb1]

Linguica said:

OK, I see what you're saying. The code *does* use SCREENWIDTH in setting up the table, BUT it assumes the value is 320 in order to get the proper levels. So you should go in and hardcode it to 320 in this instance. See, e.g., here

Yes, I've looked at that code many times. But my lighting still seems to be a bit off at different resolutions - sometimes a bit brighter in places, sometimes darker - it depends on what resolution I use. I thought it was that way across all ports, but, maybe not.

Linguica said:

Well... if you wanted to keep the same setup with 8-bit palettized graphics, 16 light levels, 32 colormap levels etc, there's no point in having a filled-out LUT like this at all. If you look at the graph I made, you'll see that the distance where each colormap light level drops to the next is the same regardless of sector light level. All you would need to do is store those discontinuities at their full fixed-point value, and then do a simple comparison with the unique z-distance of each line being drawn, which is something you have anyway.

Naw, I see a lot of potential in what you've shown here. What I would actually like to see, is Doom, lit like vanilla Doom is lit, but using the full possible range of colors, so banding is softer. Maybe that's not what you've shown, and I'm mis-interpretting it.

My personal eventual goal is to go with full 32-bit color, exactly interpolated with original Doom, so it looks just like Doom, without the bands. But, as an interim goal, I'd like to get more color out of the 8-bit mode (and, therefore, smoother banding), but retain overall original brightness everywhere. Is that what your larger LUTs offer?

EDIT: I need to read what you wrote a few more times, to make sure I comprehend. Pretty deep stuff.

EDIT 2: Looking at the graph, are you considering adding more steps closer together, at the short distance side of the graph, making the right side even more "useless", or were you thinking that those steps should be stretched out non-linearly from left to right, to add more dynamic at the right-side (far distances) of the graph? Or, am I just clueless? :) "What's your point, man??!!"

[kb1]

A followup to the lighting vs. resolution issue:

Let me see if I can articulate this properly:

It is my understanding that Doom doesn't use distance to calculate the brightness of a wall strip - it uses that wall strip's rendering scale instead, which, should be proportional to the distance...for a given resolution. But, change the resolution, and you've changed the wall scale for a given wall strip at a given distance. Thus, the light calculation will yield a different result. Ports scale this result by applying a shift, but resolutions do not always come in powers-of-two, so lighting is not consistent across all resolutions.

Does that make sense, and do you agree?

[damerell]

Linguica said:

While thinking about Doom lighting, I wondered how exactly the Doom engine calculates which light level to show.

Might we trouble you for before-and-after pictures for this change?