GL port developers; Drawing surfaces with masked textures

[DaniJ]

The purpose of this thread is to canvas opinion on methods for drawing surfaces with masked textures (in DOOM, an example of which are the masked textures used on twosided linedefs in the "middle" position) when using a graphics library like OpenGL.

Current versions of Doomsday do not handle this situation awfully well in my opinion, a good recent example are the artefacts seen on the vines in the very first map of Plutonia2 <PL2.wad>; as a result of bilinear texture filtering. No special case manipulation of the textures themselves (other than patch compositing, naturally) is done before they are uploading to OGL.

I intend to rectify the issue and have a couple of my own ideas but I would like to hear how others approach this. Do you do anything special to the source textures? Do you perhaps have a funky draw method involving tweaked fragment depth test rejection?

[Graf Zahl]

I have opted for the brute force approach:

render them with blend mode (GL_ONE, GL_ZERO) and use a GL AlphaFunc setting of 0.5 so that no transparent borders appear and they can be treated like solid geometry.

I also run every texture through a very simple filter that alters the color of translucent pixels bordering on solid ones with the color of one neighboring pixel (no complex logic here, I just use the first one I find) which eliminates those awful black borders around them. I do this second step for sprites, too.


For translucent geometry I actually sort them very thoroughly by depth including splitting polygons if it is necessary to ensure proper ordering

[entryway]

glboom-plus uses the same method

if (gltexture->flags & GLTEXTURE_HASHOLES)
{
  SmoothEdges(buffer, gltexture->buffer_width, gltexture->buffer_height);
}

SmoothEdges is from GZDoom (tnx Graf for the permissions)

By unknown reasons I used it only for patches (sprites). Now it's for walls too.

http://prboom-plus.sf.net/smooth.png

[Kaiser]

My approach is rather half-ass but it meets up to my expectations.

I use GLSL shaders to adjust the 'outline' of a masked texture. Here's a example of my fragment shader

uniform sampler2D ufTexture1;
uniform float ufOutlineAlpha;

....

void main(void)
{
vec4 texObj;

....

texObj =  texture2D(ufTexture1, gl_TexCoord[0].st);
if(!(texObj.a <= ufOutlineAlpha)) texObj.a = 1;

....

}

Problem with this is that it will leave an outline of the transparent color (Doom64's is black so you'll see a very thin black outline so its somewhat acceptable) but ufOutlineAlpha could be tweaked by the game to adjust how aggressive the alpha check should be in the shader.

This sorta functions the same as GLAlphaFunc.

[Graf Zahl]

The SmoothEdges filter function I mentioned above would take care of the black border. And doing this with a shader is a waste of resources as it can be done without the shader's processing overhead by the Alpha function. That's after the pixel processing and will not be skipped by the shader so it's available even if you are using a shader elsewhere.

[DaniJ]

Graf Zahl said:

render them with blend mode (GL_ONE, GL_ZERO) and use a GL AlphaFunc setting of 0.5 so that no transparent borders appear and they can be treated like solid geometry.

Whilst I personally wouldn't use this method, I actually like the look of the edges when drawn using an additive blend mode, I suppose the vast majority do not. Currently, we always sort anyway as 99% of the time there are textured particles, models with translucent skins (etc, etc...) flying about.

I also run every texture through a very simple filter that alters the color of translucent pixels bordering on solid ones with the color of one neighboring pixel (no complex logic here, I just use the first one I find) which eliminates those awful black borders around them. I do this second step for sprites, too.

We currently do a similar thing with sprites so yes it would make sense to employ it here too.

For translucent geometry I actually sort them very thoroughly by depth including splitting polygons if it is necessary to ensure proper ordering.

I should really look at doing this at the same time. Right now we simply "chicken out" and sort the whole lot alongside the sprites but not bothering with splitting (just fudge the positioning of the primitives a little).

I'm not keen on a GLSL approach just for this for the exact reasons Graf mentioned. Are there any other benefits to doing so Kaiser?

Has any port tried anything wild like custom compositing of masked geometry using the frame buffer? Other?

[Graf Zahl]

DaniJ said:

Has any port tried anything wild like custom compositing of masked geometry using the frame buffer? Other?

Why? Until now my sorting algorithm has worked fine. The only place where it doesn't work would be translucent 3D floors so these are disabled at the moment.

[DaniJ]

No real reason, just looking for inspiration really.

[lucius]

DarkXL isn't a Doom port but I'll chime in anyway. :)

Transparent textures in Dark Forces can be placed at adjoins (edges that connect one sector to another) and each adjoin is essentially a portal. The renderer goes through a 2 stage process, on the first stage portals are traversed from front to back (starting from the sector the camera is in) and the visible geometry is stored in order with the frustums that the geometry is seen through. Then the geometry is rendered back to front, with the proper portal frustum clipping. This ensures that all the transparent world geometry is in the correct order (pixel accurate) and that inter-sector geometry (sprites) are roughly ordered within the sector but accurately ordered with respect to sector geo. and sprites in other sectors.

When generating the textures, I use a dilation filter to remove the dark outlines, similar to others - except that it does several passes in order to spread it far enough for reasonable mipmapping. When rendering it uses alpha blending to smooth out the edges (since order is correct). In addition alpha test is used for sprites so that inter-sector ordering errors with only result in errors on the edges of the sprites (which are small in screenspace and rare to boot).


For those that don't know, DarkXL is a Dark Forces port/remake using the original data files. See here for more info http://www.df-21.net/phpbb/viewtopic.php?t=1387 Also Dark Forces, like Duke Nukem 3D, is a portal engine with dynamic sector geometry which is why the renderer does portal traversal and clipping at run time.

[rpeter]

Well, I render the scene in my pet Boom port (not GL, but D3D, no difference though) this way:

1. traverse the bsp and collect stuff to render into separate render lists, handle sw hacks.
2. render the different render lists in the following order:

-skydomes
-skymasks
-walls (masked and unmasked together)
-flats
-anti-hacks (missing flats, walls)
-sprites
-spectres
-coronas (halos)
-translucent walls
-dynlights
-player sprites (the weapon in hand)
-HUD colormap (picking up bonuses, taking damage).

Before rendering I sort the walls (just to prevent Z-fighting) without splitting,
the translucent walls (a must, see boomedit.wad)
the sprites, spectres and coronas.
So far I haven't run into wads where I needed to split polygons to sort properly.

I only detect masked textures to disable dynlights on them, I'm quite happy with the edges,
by I will definietly try the above mentioned SmoothEdges algorithm.

Had a lot of trouble with sprite edges, had to disable 3D filtering on them completely.
I've borrowed the scaling functions from Doomsday, they filter the sprites just well, at least they don't look too blurry,
still retaining a little of the blockiness of the original renderer.
I see if I can improve them by smoothing out the edges as well.

[Csonicgo]

I'm not a programmer but since this is a GL discussion thread I do have a question for you guys:

I'm sure you're all aware of the problem with the exit door texture in doom. it seems.. a bit squished in just about every GL port. some switches also exhibit this behaviour, with some blurriness on the red/green "lights". What exactly causes this? It has always bothered me for some reason and it seems to happen most on textures with lots of layers of patches.

[DaniJ]

Csonicgo said:

I'm sure you're all aware of the problem with the exit door texture in doom. it seems.. a bit squished in just about every GL port. some switches also exhibit this behaviour...

I can't say that I've noticed anything remarkable about that texture in particular. Got any screenshots showing the problem?

[Csonicgo]

DaniJ said:

I can't say that I've noticed anything remarkable about that texture in particular. Got any screenshots showing the problem?

No, but I can pinpoint which cards have the problem. ATI Radeon X850 shows the problem, I'm not sure about the newer cards. Applying Filtering usually solves the issue. GeForce 6800 shows this problem, but the 7 series and up don't. Generally the older the card, the more likely this problem shows up. I wouldn't consider it a big deal, however.

[esselfortium]

If I had to guess, it's because the textures in question are 72 units tall. AFAIK, OpenGL ports need textures to be at an even power-of-two size like 64, 128, 256, etc., so the texture gets squeezed or stretched to a nearby usable size, and then drawn scaled to its original size ingame.

[Graf Zahl]

esselfortium said:

If I had to guess, it's because the textures in question are 72 units tall. AFAIK, OpenGL ports need textures to be at an even power-of-two size like 64, 128, 256, etc., so the texture gets squeezed or stretched to a nearby usable size, and then drawn scaled to its original size ingame.

...unless you use a graphics card which doesn't have this limitation (at least any semi-modern NVidia card) and a port which checks for the feature.

[DaniJ]

I only recently (late Oct?) reworked things in Doomsday to support non-power-of-two textures and as yet, there is no release build with this in. It'll be in the upcoming 1.9.0-beta6 release.