Is a more powerful system of static surface lighting feasible in the Doom engine?

[Tango]

I am not very familiar with the technical side of the Doom engine, so this could all very well be a massively dumb or unrealistic idea. But seeing what good lighting looks like in games like Quake has made me wonder if there is a system 'better' than sector lighting or dynamic lights that would work well in the Doom engine, without the usability issues of sector lighting (which becomes tedious, especially with complex level geometry), or the (I assume?) performance cost of dynamic lighting. Would it be feasible to have some sort of static 'lightmap' that a mapper could easily create?

 

I'm envisioning (or rather fantasizing) a system that would make it pretty trivial to do complicated lighting across large surfaces, even with complicated level geometry. As a mapper, I want to be able to just throw down some shadow shapes on my map without having to think too much about it. Imagine if I could just open a map up in 3d mode and arbitrarily 'paint' shadows or light around my map. No need to deal with sectors (and all the bullshit that comes with sector lighting when you have moving floors or any other tagged sectors), and no need to drop dynamic lights around in all the right places.

 

Consider some examples with ugly mockups:

 

Say I start with a surface I want to 'paint':

 

 

I then activate lightmap mode, which reveals a grid (whose size I could configure, just like in 2d views):

 

 

Then I throw some points down to make my polygon:

 

 

Then I tell the editor that I want my shape to be light level +64, or something like that:

 

Then maybe, as a bonus, I can tell my editor that I want to make this a gradient, with a single band of light interpolation, and it automatically constructs the following:

 

 

 

 

Alternatively, say I wanted to do something similar in the 2d view. I start with the following layout:

 

 

Then I go into lightmap mode, paint a bunch of stuff, and end up with something like this:

 

Now my map is statically lit without me ever having to touch sector lighting!

 

Again, I have no idea if this is feasible, and I haven't very well thought through all the specifics. Some of my remaining questions/points for discussion:

 

1. How does this interact with sector or dynamic lighting?
2. How is the lighting of walls vs flats handled (ie how do my 2 demonstrations here work together)?
3. Is this even doable in the Doom engine?
4. Should I just suck it up and use dynamic lights?
5. Is there another superior system that would accomplish the same goals?

[zzzornbringer]

that's a neat idea. i'm no programmer, so i have no clue how hard or even possible that would be.

 

the way it's handled now is ok i guess. it adds a little charme to doom editing for how you create your shadow and lightning. it's by no means necessary at all. but adding it in your last polish pass separates a good map from a fantastic map. i've recently played dimensions of the boomed and way 2 many dead guys and i have never seen a map execute shadow and lightning that good. also looks fantastic in the editor.

 

i personally would rather have something a lot more simple. like the way doom 3 did it's shadows. just put a light in there and the engine renders hard shadows. done. aesthetically those hard shadows would fit well into doom. it also has no issues with dynamic elements. i would imagine that we don't need per pixel lightning for doom though. just a per sector basis which would probably make this a whole lot easier to implement. but that's just dreams. i have no clue if such a thing would be possible.

[boris]

There's this:

Not sure if it works on ceiling/floor and UDMF, though.

 

If you're targeting GZDoom you could just use huge textures (especialls for floor/ceiling) and draw on those... but then I don't really see how that's easier than to use dynamic lights. Especially since GZDoom has had shadowmaps for some time now (although they are a bit limiting, like not working on 2-sided lines).

 

If you just draw on huge textures you'd still have the problem that things are not affected by it.

 

[ketmar]

it is certainly doable. k8vavoom has quake-style lightmap lighting mode, for examle. while it calculates lightmaps on level loading, there is thechnically no problems in loading pre-backed lightmaps from wad. you won't get doom-style "hard" edges, though, because lightmaps are way smaller than the actual geometry, so engine have to filter 'em. of course, it is possible to increase lightmap resolution to get more details. and it is possible to implement in-game "lightmap brush" to paint over lightmaps too (almost all the code is already there).

 

maybe someday i'll add this to the engine. just4fun. ;-)

[Kappes Buur]

GZDBBF has something similar to this: Spotlights

 

Spoiler

 

 

 

 

 

 

Edited January 13, 2019 by Kappes Buur

[SaladBadger]

dpJudas is working on lightmap support for GZDoom too, so this kinda thing is theoretically possible, but I'm honestly not aware of any sort of workflow that involves manually painting lightmaps. It's possible, but I just don't know if anyone's done it or if it's that worth.

[ketmar]

if GZDoom will get lightmaps, i guess that it will be possible to implement "lightmap painting" with some special weapon, for example. just4fun, if nothing else counts. ;-)

 

p.s.: i am somewhat jealous, and some small part of me hopes that dpJudas won't finish it. very small part, but... ;-)

[GooberMan]

...I might be doing something for GZDoom already with static lightmaps.

 

[Vermil]

Doomsday has bias lighting that is somewhat similar: https://talk.dengine.net/discussion/1394/bias-lighting-for-heretic.

[boris]

Played around with brightmaps a bit and came up with this:

 

 

It's a bit sloppy (off by 1 pixel on each axis) because I crafted it by hand.

 

Editor screenshot:

 

 

Texture for the floor (it's just FLOOR0_1 tiled to 512x512):

 

Brightmap for the texture:

Unfortunately brightmaps do not influence the brightness of things standing on them, otherwise we'd have static lighting right there.

 

Example is attached.

brighttest.zip

[Bauul]

17 hours ago, InsanityBringer said:

dpJudas is working on lightmap support for GZDoom too, so this kinda thing is theoretically possible, but I'm honestly not aware of any sort of workflow that involves manually painting lightmaps. It's possible, but I just don't know if anyone's done it or if it's that worth.

 

Is he still working on it? Last I saw he'd hit a road block and had moved on to other things

[GooberMan]

17 minutes ago, boris said:

Played around with brightmaps a bit

Yep, you're exactly on the right track.

 

GZDoom already has lightmaps, but as usual they're implemented in a sloppy way for a specific purpose. Brightmaps are exactly a lightmap, but with a bunch of checks in there to ensure they only ever give you a greyscale map.

 

My screenshot above is composed of lightmaps like the following. And a small shader modification means it actually uses that colour information.

 

[ketmar]

4 hours ago, GooberMan said:

...I might be doing something for GZDoom already with static lightmaps.

'cmon, switch to my engine! the whole new world of fantastic bugs you have never heard about, and VaVoom C (it is almost like D, but VaVoom C ;-). without people making projects for k8vavoom i have little interest in fixing rendering and such: i like to work on low-level systems, VC compiler, and so on. so come to our side, we have cookies with bugs!

 

p.s.: i implemented support for textures with pathes in UDMF recently, but as usual, i don't have a good test map at hand, so just commited it and declared as "done". ;-)

[GooberMan]

Do you read UDMF format and support compatability with all of ZDoom's features?

 

Then it already works in k8vavoom.

 

Besides, I'm writing my tool in D. Actually, might as well chat on IRC about what I'm doing.

[boris]

17 minutes ago, GooberMan said:

Yep, you're exactly on the right track.

 

GZDoom already has lightmaps, but as usual they're implemented in a sloppy way for a specific purpose. Brightmaps are exactly a lightmap, but with a bunch of checks in there to ensure they only ever give you a greyscale map.

 

My screenshot above is composed of lightmaps like the following. And a small shader modification means it actually uses that colour information.

 

So that's without any modifications to GZDoom itself? How do you handle lighting things?

[GooberMan]

6 minutes ago, boris said:

So that's without any modifications to GZDoom itself? How do you handle lighting things?

By making some assumptions.

 

GZDoom is very destructive with its data. Getting anything in to the shader is a difficult task. So I've hijacked a little-used render path and modified it to suit my purposes.

 

I'll make a thread soon about what I'm doing. It's not quite ready to be shown to the public. But y'all are gonna love it.

[dpJudas]

5 hours ago, GooberMan said:

GZDoom already has lightmaps, but as usual they're implemented in a sloppy way for a specific purpose. Brightmaps are exactly a lightmap, but with a bunch of checks in there to ensure they only ever give you a greyscale map.

No, brightmaps are not lightmaps. If they were they'd be named lightmaps.

 

Yes, they both add light to a scene but that doesn't make them the same. Removing the illumination checks in itself isn't really enough to convert them as lightmaps need their own texture coordinates or you'll end up with an extremely inefficient set of lightmap textures.

 

If you want to be really pedantic you can of course argue that it meets the definition of a lightmap texture without the texture unwrapping part. However, you don't really have a working solution if it doesn't scale. The asset pipeline and the performance of the renderer are the hard parts.

[GooberMan]

Pedantics. Here we go. You can bring up that word all you want and write out implementation details until the cows come home, but they're still lightmaps. That's fact.

 

But you're right about one thing. Texture coordinates. If you only have one set, either you split your surfaces up by texture limits so that lightmaps are still unique and texture data doesn't require repeating in memory; or you repeat that texture data. It's not the lightmaps that will be inefficient since they're the part that actually needs to be unique.

 

Like I said. They were implemented in a sloppy way for a specific purpose. It's far from the only thing in GZDoom that is guilty of that.

[dpJudas]

On 1/13/2019 at 10:06 PM, ketmar said:

if GZDoom will get lightmaps, i guess that it will be possible to implement "lightmap painting" with some special weapon, for example. just4fun, if nothing else counts. ;-)

 

p.s.: i am somewhat jealous, and some small part of me hopes that dpJudas won't finish it. very small part, but... ;-)

You can beat me to it by adding something that allows k8Vavoom to load its lightmap data directly from a wav lump. Then update ZDRay to output a lump in that format (or use the lump format it already outputs). :)

[dpJudas]

5 minutes ago, GooberMan said:

Pedantics. Here we go. You can bring up that word all you want and write out implementation details until the cows come home, but they're still lightmaps. That's fact.

Yes lets be pedantic and agree that it is NOT a fact. If they were lightmaps you wouldn't have had to change the shader math.. and no, it doesn't just gray color it - the whole purpose there is to create minimum light at the art pixel level, which is quite different from what a lightmap is trying to do.

[GooberMan]

Sector light zero, eliminating Doom lighting entirely from the equation.

 

Remove that unnecessary desaturate from every brightmap lookup.

 

Oh look now they're exactly lightmaps.

 

Fact.

 

EDIT: The only thing stopping them being lightmaps under your definition is that Graf neglected to give a second set of texture coordinates back in the day. A second set that maps to the brightmap texture exactly will both match the specifications of brightmaps; and allow unique surface lightmaps.

[dpJudas]

And yet now they can't be used as brightmaps. Fact.

[GooberMan]

Rubbish. Associating a lightmap value explicitly with a texel versus a surface coordinate does not discount them from being used as brightmaps.

 

Try it.

 

Go on. I'll wait.

 

EDIT: Don't forget to sample the brightmap correctly during lightmap generation.

Edited January 14, 2019 by GooberMan

[dpJudas]

Well aside from the fact that you can't apply a brightmap to a level with your "lightmaps". And aside a U shaped sector floor/ceiling will be horribly inefficient (both for texture data AND lightmap data). You also pretty much forced the mapper to set a zero light level for all sectors hit by a lightmap light source.

 

Let's just say that whatever you are describing here would never survive a pull request review because brightmaps aren't lightmaps.

 

Try it. Go on. See if it will pass. ;)

[GooberMan]

Alright, that edit I did above happened far too quick.

 

Lightmaps are baked data. If you change a texture on your map, that data is stale. If that texture is a brightmap, then you need to sample the brightmap texels in your lightmap generation.

 

There's of course nothing stopping a user setting sector light. Sector light is exactly ambient light in this case. ie a minimum light level. If a user decides that their minimum light level of 192 is fine, then that's correct. If you're really worried, then just like the brightmap above you can sample that in your lightmap rendering.

 

You keep using the word "inefficient". If this were a pull request where I'd be fixing brightmaps to how they should have been implemented in the first place, your U shaped sector would be an irrelevant argument since I'd be rendering lightmaps for convex subsectors.

 

And I'd be doing something that it looks like GZDoom still isn't doing - creating texture atlases to reduce draw calls. And ditching those atlases entirely for a DX12/Vulkan renderer since it's unnecessary.

 

And I'd remove all those if branches from the shader. It's like the renderer wants to be inefficient by default.

 

But hey, if you think adding yet another pipeline to GZDoom to handle something that's basically already there is the way to go, who am I to argue? It's not like I have relevant experience in this department. Not me. I'm just some guy on a forum.

[ketmar]

39 minutes ago, dpJudas said:

You can beat me to it by adding something that allows k8Vavoom to load its lightmap data directly from a wav lump.

but why? k8vavoom is perfectly capable to generate lightmaps by itself, on map loading. sure, it doesn't do proper light bouncing, but meh, most of the time results are looking good anyway, and i certainly don't want to roll full-featured radiocity raytracer into engine codebase. ;-)

[dpJudas]

25 minutes ago, GooberMan said:

There's of course nothing stopping a user setting sector light. Sector light is exactly ambient light in this case. ie a minimum light level. If a user decides that their minimum light level of 192 is fine, then that's correct. If you're really worried, then just like the brightmap above you can sample that in your lightmap rendering.

 

The difference between a brightmap and a lightmap is exactly the part of the equation you removed by lowering the light level to zero. But that doesn't make them the same because it only applies at light level zero. Once you bring up the light level their behavior is different and that's why they have different names. It is essentially the same as saying imaginary numbers are the same as normal ones because they behave the same in the range that they are the same.

 

As for the inefficiency, even if brightmaps used lightmap compatible math, you can't control the shape of the sector. You can't specify different brightmap textures in GZDoom on the subsector level. That's actually what the most of the code on the lightmaps branch is doing - generating second texture coordinates and picking the lightmap texture for the specific subsector. The extra code in the shader itself is actually just one line: if (vLightmap.z >= 0.0) color.rgb += texture(LightMap, vLightmap).rgb;

 

Note that I'm only talking in the context of GZDoom here. If its for your own engine none of this applies, naturally. :)

 

17 minutes ago, ketmar said:

but why? k8vavoom is perfectly capable to generate lightmaps by itself, on map loading. sure, it doesn't do proper light bouncing, but meh, most of the time results are looking good anyway, and i certainly don't want to roll full-featured radiocity raytracer into engine codebase. ;-) 

 

I'm not sure which port I was looking at, but I don't think the lightmap generation code I saw did any light source visibility tracing at all. It was only a very quick glance though so maybe I'm wrong. In any case, if you're happy with the lightmaps there's of course no reason to do anything. :)

[ketmar]

11 minutes ago, dpJudas said:

but I don't think the lightmap generation code I saw did any light source visibility tracing at all

sure it does, both in original Vavoom, and it k8vavoom too. original didn't trace dynamic lights, tho (CPUs were too slow back then), but all the code was there, of course, so i just added several code lines, and now dynlights are casting proper shadows too. the engine is even capable to create lightmaps as it has to render unlit map areas (althrough this is a quick hack, and it is done in the same thread as everything other, so it may be little sluggish; but it helps with Skulldash EE hub level alot).

 

yet i must say that lighting code may be somewhat hard to follow (not that i fully understand it myself ;-), so no wonder that you missed some details. the reason i am jealous it that i wanna be The Only Guy With Proper Traced Lightmaps In My Engine. ;-)

 

and i am designing new lightmap caching system now, so i can get rid of "static lights" and "dynamic lights" separation -- it is not hard to detect moving lights, after all, and cache tracing info. that is, i have alot of unimplemented tricks in my bag... but i prefer to procrastinate, of course. ;-)

[GooberMan]

4 minutes ago, dpJudas said:

The difference between a brightmap and a lightmap is exactly the part of the equation you removed by lowering the light level to zero. But that doesn't make them the same because it only applies at light level zero. Once you bring up the light level their behavior is different and that's why they have different names.

Sampling sector ambient + brightmap texel in lightmap generation would deal with that.

 

Let's clarify for those playing at home: brightmaps as defined in ZDoom's material system are additive lightmaps, not blending lightmaps. But they're only additive on sector lighting. Any other kind of lighting operates after brightmaps have been evaluated.

 

7 minutes ago, dpJudas said:

The extra code in the shader itself is actually just one line: if (vLightmap.z >= 0.0) color.rgb += texture(LightMap, vLightmap).rgb;

Eesh. Just one line with a branch. Modern GPUs are a bit better with branches than the simple predication of old. The modern "threaded" approach that NVidia GPUs take still results in pausing execution units and switching new ones in though, which really isn't desirable.

 

The BRIGHTMAP define is certainly the better way to go about it. No branch because the CPU has already done that once for you, now you already know there's a brightmap and the correct code has been uploaded to be executed thousands of times.

[ketmar]

as most (if not all) doom engines still doing front-to-back rendering (not strict, but still), trying to get rid of branches in shaders won't pay with any noticeable speedups, i believe, 'cause depth buffer will reject everything even before executing any fragment shader. yet the code will be full of ifdefs -- Hell Revealed! ;-)