Doom in true colour [ testers for exe welcome/src snapshot/links updated ]

[Update]
+GL backend should work now on older and onboard hardware.
+idlsd is now back with a vengeance
+automap fade/color strobing enabled
+source is available for download


[Attention]
This is an unofficial test(no release) since so many things still need to be done.
This version of Doomtest is the one with the GL-/ and Software- scaling backend.
(Just run doomtest.exe or doomtest_gl.exe)
The additional directory 'wad' can be ignored though you can rename the unfinished .lit files from _"mapname".lit to "mapname".lit to make them work - beware that lighting your level this archaic way is a LOT of work and a MAP01.lit will be used for any map01 entry -> a custom and an original map in Doom2 currently will use the same .lit file.

[Limits]
All render limits have been increased partially due to increase of visplane usage in extended mode.


[Keys]
Keypad '+' switches between classic/classic smooth/extended render mode


[.EXE Link/Pass]
http://speedy.sh/D4h5u/ChocoTest-10.10.2013.rar

[Source Link/Pass]
http://speedy.sh/Bhfu5/ChocoTest-MSVC-2005-10.10.2013.7z


[Source Info]
-a snapshot
-the name DoomNX is just a dev name(Dreamcast who?)
-the project is in VisualStudio2005 format
-it requires MASM(included in full edition)
-many things are not in their proper form yet
-all input/ideas welcome(PM)

[Known issues]
-composite textures that have zero length patches or out of bound patches crash(e.g. trinity college or TNT maps featuring crate textures containing oversized vine/blood patches)
-render mode selection won't get saved as of now
-in multiplayer all players are green



[Megawad Preview]
As a goodie you get the first 3 levels of my megawad as an added bonus.




[Peek]


[NOTE]
Input/thoughts/etc. for both Doomtest and the Megawad preview can be posted in this thread.

[Thanx to]
id - for removing the bore from games,
Fraggle "The Man" - for providing such a wonderful base,
Maes - for sparing me from a heinous bug,
all Doomers for keeping the faith


!Enjoy your demos and play time!






[Old]
I played a little bit with Chocloate Doom's source and added a few goodies.
This is still a work in progress, so things may change.

The "only" thing that is still left from the 8bit voodoo are the 14 PLAYPAL's that are needed to draw the artwork and the functions to retrive it - for loading and drawing.
The colormaps are gone for good.

Supported as of now: 24 bit blending so one can draw inverted, tinted, transparent or whatever.

Adding 24bit texture support will take a while.

It will be released in combination with my Doom2 megawad.

Special Thx goes to Fraggle for the wonderful base and Maes for knowing what goes on behind the scenes in the engine.





before


after

Looks like a happy medium between software modes and special effects-ridden GL modes. :)

Adding proper high-resolution would be good too (I don't know exactly what's involved with it though :P)

The demon before and after really showcases the new colors. Looks really awesome

Neat stuff for sure -- are you using GL rendering for this? It's odd but cool to see a "Chocolate GL" sort of approach here.

Looks neat! You should consider releasing it separately. :)

Xaser said:

Neat stuff for sure -- are you using GL rendering for this? It's odd but cool to see a "Chocolate GL" sort of approach here.

I'm sure that he isn't.
BTW, PrBoom's renderer has a 32-bit color mode and bilinear filtering. Still uses colormaps, though.

Megamur said:

Looks like a happy medium between software modes and special effects-ridden GL modes. :)

You can disable most, if not all of the special effects that come with the typical OpenGL port, including most of the filtering.

This looks very nice! Have you considered interpolating the depth in the lighting, so it isn't in steps?

Also, are you considering releasing this before your megawad?

Oh wow, very nice job here indeed! Strange, I was only just asking a dev last week whether this very thing could be achieved in the software renderer. Following with joy.

I have actually done this before in Chocolate Doom --- except the lighting was completely smooth and the fuzz effect still worked. I'm guessing it's not completely finished?

One thing I found interesting is that when I did the same thing to Heretic, it looked like complete garbage when it came to water and lava, because everything looked "grayed out".

Xaser - no, everythings old Doom though different canvas and the way colours are treated.

Chungy -
if the stuff I want to work, works I'll give it a try for sure. Don't hold your breath because enhancing Doom's colour while keeping the MAME look is numero uno. Additionally this would be very slow because everything's done pixel per pixel on the cpu... in 320x200 a joke - in 2560x1440 quite painful.

Sodaholic - it is thought of as a goodie in addition to the wad, which I want to deliver in standard and enhanced version... suited to the modification.

Wagi - Fuzz effect will be kept as a feature. Very nice work with the port/enhancement. Did you use Dooms way to calculate light falloff or used your own algorithm?


Spectre has a new suit... it's hard to spot and surpises are for sure.



Invulnerable - simple invert



Smooth


Standard or smoother version can be set in the options menu in 4 levels.

Nice and quick work! I like the new effect you've added in that last screenshot.

And yes, it's essentially Doom's lighting algorithm, but the way I do lighting is to "unroll" doom's light/distance calculation so there isn't so much lossy bitshifting going on. Eventually a few values are passed down to some functions that look like this:

//====================================================================
//Plane lighting is calculated here because it is too large to be stored in a LUT anymore.
//This function is, more or less, an accurate replication of Doom's original zlight LUT.
//====================================================================
byte R_PlaneLight(int lightlevel, int scale)
{
	int level = (lightlevel+8)*2 + FixedDiv(5, (scale << 5) + 1) - 255;

	if(level < 0) level = 0;
	if(level > 255)	level = 255;
	return level;
}

//=========================================================================
//Pretty similar, except this one is to replace the scalelight LUTs.
//=========================================================================
byte R_ColumnLight(int lightlevel, int scale)
{
	int level = (lightlevel+8)*2 + scale*WLSCALE/4 - 255;

	if(level < 0)  level = 0;
	if(level > 255) level = 255;

	return level;
}

I also did a smoothened damage/pickup effect, where the effects can also happen on top of eachother (if you get hurt with the Radiation Suit on, the green won't go away). There's also some code in there to make sure that it doubles "red" and "gold" values below a certain range in order to emulate how Vanilla skipped the first red and gold palettes. If you want, I could contribute some old code from my version wherever you would find it helpful.

Awesome work!

Now the obvious next step is for the community to agree on a common extended patch format for truecolour/hicolour modes, e.g. a 16-bit extension to the classic patch_t format such as a patch16_t format, which would allow to use the same lighting algorithm and even use intermediate bit depths (between 8 and 16).

I was thinking of extending Mocha Doom to support a 16-bit indexed mode (with existing colormaps and palettes duly interpolated) which could mix 8~16-bit resources in drawing functions, and with loaders that could handle properly extended palettes and colormaps.

What would be the point of a 16-bit palette? Sure, you get 65536 colors. Cool. But do you have any tool to generate such palettes? How about graphic software to use them? Intermediate image formats (such as PNG) to make them go from the graphics program to the conversion utility? The image world is split in two, 256-paletted and 24/32-bit RVB, with other forms of representation merely obscure crumbs left to superspecialized niches.

So why not go with 24-bit directly and get truecolor patches?



As for a new patch_t format... Sorry, but I don't really see this happening anyway. It'd be redundant with things such as PNG, which are now adopted by several of the most influential ports out there (Eternity recently got PNG support, for instance).

I agree. A 16 bit patch_t would be pointless.
Today there's no real need to have the external representation of a graphic be the same as the internal one.

So just load a PNG, JPG or whatever and do the conversion internally. Much less trouble for anyone involved...

_bruce_ said:

Smooth

Very, very nice work.

I'm reminded of the work done recently by perkristian with the improved weapon animations and also the remastered sound effects. It's nice to see this kind of subtle improvement to the game that stays true to the original look and gameplay.

It would be nice if you could record a Youtube video of your mod.

Gez said:

What would be the point of a 16-bit palette?

Gez said:

So why not go with 24-bit directly and get truecolor patches?

Uhm... to use the same indexed lighting trickery of Doom with more colours? I think that was the whole point.

Gez said:

Sure, you get 65536 colors. Cool. But do you have any tool to generate such palettes? How about graphic software to use them?

Didn't look into it, but I assume that since an indexed 16-bit format is actually supported/defined, there must be tools that support them, too. Even if there weren't, that would 99.999% not be a problem, seeing how prolific developers can be.

TBQH though, the closest I've seen in practice was a RTS (Warlords Battlecry) that used a 16-bit indexed display but 8-bit sprites (with 16-bit palettes), and a great deal of the visual effects were achieved by using per-sprite and per-tile individually applied palettes.

Gez said:

As for a new patch_t format... Sorry, but I don't really see this happening anyway. It'd be redundant with things such as PNG, which are now adopted by several of the most influential ports out there (Eternity recently got PNG support, for instance).

You seem to be forgetting the needs of column-based software renderers, other than the on-disk storage. Not even OpenGL renderers utilize PNG directly (they would have to be converted to textures), and no software renderer so far is able to use anything other than raw 8-bit patch_t graphics. A 16-bit renderer capable of actually using 16-bit indexed column data, would also need a 16-bit patch16_t format, no matter if it was loaded from disk, forcibly blown from existing patch(8)_t or generated from PNG files by color downsampling.

Graf Zahl said:

I agree. A 16 bit patch_t would be pointless.
Today there's no real need to have the external representation of a graphic be the same as the internal one.

*sigh* You too, Brutus? Of course a patch16_t format would be primarily a renderer hack, which could or could not necessarily have on-disk equivalents. I didn't suggest anywhere that it should be a FORCED and OBLIGATORY on-disk format (although it COULD be used for that too).

As for going straight to 32-bit, at that point you are losing the advantages of indexed manipulation. A "patch32_t" format would be pointless, unless you accepted the overhead of doing straight RGB and/or Luma/Chroma manipulation.

Maes said:

Uhm... to use the same indexed lighting trickery of Doom with more colours? I think that was the whole point.

You can replicate it algorithmically. A 65536x34 colormap is unwieldy, as is a range of fourteen 16-bit palettes. For the colormap (which is what's used for lighting in Doom), then it's not so much a concern; indeed _bruce_'s first post says that "the colormap is gone for good". For palettes, I've written code in SLADE 3 that can recreate them for any RGB values you feed it.

Maes said:

Not even OpenGL renderers utilize PNG directly (they would have to be converted to textures), and no software renderer so far is able to use anything other than raw 8-bit patch_t graphics.

ZDoom and Eternity seems to be doing just fine without them. In fact, any software-rendered port that wants to allow to put flat textures on sidedefs will need to get around that patch_t thing, and guess what, they exist.

Maes said:

Of course a patch16_t format would be primarily a renderer hack, which could or could not necessarily have on-disk equivalents. I didn't suggest anywhere that it should be a FORCED and OBLIGATORY on-disk format (although it COULD be used for that too).

Sorry for inferring this from you talking about the necessity of it being a community standard. If it's only a renderer hack, then each port author can implement it in their own way without bothering to care about some other people did it.

Gez said:

"the colormap is gone for good"

Hmm I admit I didn't take not of that. Therefore it's a clear CPU vs memory tradeoff here. Still, even if full-fledged 16-bit would be unwieldy, there is the possibility of using only 9-bit or 10-bit colormaps, unless someone definitively proves that 8-bit is the "sweet spot" between the convenience of pure indexed manipulation, and the memory/cache overhead caused byLUTs.

Gez said:

ZDoom and Eternity seems to be doing just fine without them. In fact, any software-rendered port that wants to allow to put flat textures on sidedefs will need to get around that patch_t thing, and guess what, they exist.

I didn't really nitpick their code, but you imply that they are using something other than patch_t for rendering columns, or that they have forgone column rendering (in SOFTWARE mode) entirely? I could understand doing a flat -> patch_t or rows-first bitmap -> column patch_t conversion programmatically. Again, are we discussing in-memory or on-disk structures here?

Edit: I remembered that at least ZDoom is able to sort of "underlay" a truecolour, NON-patch_t format sky beneath the "regular" graphics, but that's a quite special case. I never heard that you can get a regular truecolour PNG and simply use it as a sprite and it will show & shade just fine.

ZDoom does use columns, yeah. Each possible image format (including some non-conventional ones such as composite textures or warped textures) has its own GetColumn(), GetPixels() and CreateSpans() methods that are called as appropriate.

But the column is a subpart of the patch_t struct.

Maes said:

I didn't really nitpick their code, but you imply that they are using something other than patch_t for rendering columns, or that they have forgone column rendering (in SOFTWARE mode) entirely? I could understand doing a flat -> patch_t or rows-first bitmap -> column patch_t conversion programmatically. Again, are we discussing in-memory or on-disk structures here?

patch_t as such is gone in ZDoom. It's just one more input format for graphics data and not used for internal storage. The texture manager creates the columns/posts on the fly and does that for any kind of texture it can load - patch_t, PNG, JPG, TGA, PCX and DDS.

Maes said:

Edit: I remembered that at least ZDoom is able to sort of "underlay" a truecolour, NON-patch_t format sky beneath the "regular" graphics, but that's a quite special case. I never heard that you can get a regular truecolour PNG and simply use it as a sprite and it will show & shade just fine.

Of course you can't do that with an 8-bit display buffer.
However, once you got the 32- or 16-bit shading tables in place the actual format the graphic comes in is completely irrelevant. Any internal tables needed to speed up rendering can be created upon texture load. There's no good reason anymore to force this into the stored data format.

Gez said:

But the column is a subpart of the patch_t struct.

Heh that was one of the hardest part to "get" about the source code: the column_t struct itself only clearly defines the topdelta and length fields, while the actual data payload (and the fact that it's actually daisy-chained to a whole bunch of other column_t structs, and that they are terminated with an 0xFF) is only hinted at by the (vague) comments, and by the way they are used in the drawing functions.

They also had a "post_t" struct defined somewhere, which is structurally exactly the same as a column_t, but semantically different: only the first "column_t" of a series of "column_t"s would be a "true column", while the rest would simply be "post_t"s. Pretty moot, since they typedeffed post_t to column_t anyway....

Is there a patch for this anywhere? I would love to see this implemented into Odamex some time.

Sorry Graf, for some reason I saw your post appear only now :-/ Here goes anyway:

Graf Zahl said:

patch_t as such is gone in ZDoom. It's just one more input format for graphics data and not used for internal storage.

Still, there must still be a column-based, 8-bit indexed equivalent in use somewhere, right? The point isn't about it being exactly classic patch_t or a simple blow-a-byte-to-a-short type extension, it's about still working in the classical column-renderer-friendly manner. 16-bits is more of a convenience limit: most OSes support a 16-bit indexed format natively, anything larger than that is direct color. None forbids doing your own emulation of course...

That being said, the patch_t as defined by "mama id" is, admittedly, an awkward format to work with: extremely cryptic at the source code level, full of limitations and vestiges like the padding bytes. I've thought of using a "cleaner" equivalent myself.

Graf Zahl said:

Of course you can't do that with an 8-bit display buffer.
However, once you got the 32- or 16-bit shading tables in place the actual format the graphic comes in is completely irrelevant.

The way I see it: up to 16 bits you can still have a fully indexed-based manipulation and use the old colormap code (with due extensions) and most importantly, you can have variable bit depths (and palette/colormap extensions) from 8 to 16 bits. E.g. who says that an indexed 64K or 32K or even 1K color display wouldn't look as good as a truecolour one, while keeping the same rendering code and colormap system?

With 24/32 bits , you are essentially going full-blown RGB, which is a quite different beast. It will be interesting to see if a -properly implemented- "super indexed" display can beat a full pixel-level RGB one in terms of performance while being extremely competitive in terms of looks.

With full RGB manipulation, the good side is that since you decided to forego the simplicity of indexed-based manipulation completely in terms of CPU overhead, you will have no more qualms about using expensive effects like proper RGB blends etc.

Then again, if I remember correctly from DelphiDoom32, which also sported a 32-bit display, that didn't work out too well in terms of performance or at least it wasn't as impressive as bruce's work.

Am I the only one that thinks that generating a true-color colormap from the original 8-bit one is a good idea? Just interpolate the 1st and 32nd indexes, and you can have it still fade to any color you want instead of just black. (for example, if there was a PWAD loaded that had white or red fog in the colormap instead)

Am I the only one that thinks that generating a true-color colormap from the original 8-bit one is a good idea? Just interpolate the 1st and 32nd indexes, and you can have it still fade to any color you want instead of just black. (for example, if there was a PWAD loaded that had white or red fog in the colormap instead)

IIRC Doom's colormap isn't fully black for certain colors in the 32nd colormap, so this would end up looking weird when you get into the darker colors as not everything would be fading directly to black.

The advantage of true color is that you can blend them in different amounts using only math operations (you don't have to use a LUT anymore), so generating a new colormap would take up too much memory (255 colors * 255 light levels * 3 channels = 192k per colormap), wouldn't work on any colors that aren't already in a palette, and would provide only an extremely small speed boost.

Functions to do fast color operations like colored lighting (putting a yellow object under a magenta light and getting red), or additive translucency can be easily written by a highschooler.

Wagi - your code is always welcome.
Fraggle - will try to record a vid.
Maes - patch_t is evil. Coder's from the vatican are already on a mission.

)The smooth version of the standard "light-dimmer" still gets it's values from the colormap offsets - not what is referenced by them.
The third option will just calculate the value.

)Added OpenGL option for screen up scaling - this is the quasi economy mode variant to Fraggle's upscaler. Choco eats now around 22% cpu time of an A64 at 2.2 Ghz no matter how big your monitor resolution.
You can smear or crisp it.
In software at 1600x1200 a Core2Duo at 3.2 Ghz is at nearly 100% when scaling up in software - 1920x1200 lags. Still software is truest to the original and hence the measuring stick.
A setup switch - not on the fly.

) Added a baby root of a quasi "shader" function with which one can recolor artwork given s/he has the patience to create a mask.
If you don't have a job - masking some monsters' frames may be your calling.

) The zombie man is fullbright now when shooting... added this due to an email from the league of neglected Doom sprites.

) There is a contrast wad(revision 3) out there which looks pretty interesting. This can be done way easier by using the HSL or HSV color space when manipulating pixels... maybe add a Value/Lightness thermo in the options...


Shot from Map02 of my megawad




In original Doom the metal room in E3M3 suffered the most from the palette... it can rest in piece now.

I would like to see a GL backend get trunked in Chocolate Doom if that is possible. Please discuss the matter with fraggle and see if he will adopt your GL code back into the port itself ;)

We shouldn't be depending on that deprecated DirectDraw backend in SDL any longer, and without it, you can't currently record Choco ports in programs like fraps.

Wow, you just fixed the biggest problem I've had with Doom since I first played it (and singlehandedly saved E3M3 in the process). Smooth looks especially great.

How about some screenshots from an area that uses the red brick SP_HOT1 walls in a dark area? I always thought that was one of the ugliest light-to-dark transitions in the entire game, and it would be nice to see it with the smooth lighting.