Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
legolas558

Forking GZDoom, anyone?

Recommended Posts

After being really pissed off on another forum, I am going to start this discussion here where there is no medieval censorship at least.

Can somebody please say what is the status of true OpenGL ports of Doom?

I see it disapponting that we now have sci-fi graphics hardware (all devices of past 10 years can be considered so) and still need to relay on software rendering (ZDoom) without any good, scalable and modular OpenGL engine. I know that Doom needs virtual textures to run well on modern hardware, but still looks like a solvable problem to me (although I don't have alone the skill to implement such thing).

This is the premise to talk about forking or continuing development on GZDoom; there's to say that Vortex is trying to fill such gap, although I have no direct experience of his work.

What are you (players/modders/developers) thinking about this?

Thanks

Share this post


Link to post
legolas558 said:

After being really pissed off on another forum, I am going to start this discussion here where there is no medieval censorship at least.

lulz.

If you don't see why passive-aggressive bitch sentences such as "I would really like to offend you because you do not seem very smart" get a thread locked before it turns uglier than it already is, then I'm afraid I have to tell you that you do not seem very smart.

legolas558 said:

Can somebody please say what is the status of true OpenGL ports of Doom?

Doomsday, EDGE, GLBoom+, GZDoom, Skulltag, Vavoom.

legolas558 said:

I see it disapponting that we now have sci-fi graphics hardware (all devices of past 10 years can be considered so)

Do you have a ten-year-old, second-hand holodeck for sale?

legolas558 said:

I know that Doom needs virtual textures to run well on modern hardware, but still looks like a solvable problem to me (although I don't have alone the skill to implement such thing).

Right. In other words, you're yet another of those guys who have The Answer but don't actually know if it truly is the answer.

legolas558 said:

This is the premise to talk about forking or continuing development on GZDoom; there's to say that Vortex is trying to fill such gap, although I have no direct experience of his work.

To be frank, I consider VC's super-duper OpenGL renderer to be a vaporware prospect. Just like your virtual texture thing, it's just half-formed thoughts about a possible way to do things, with no actual groundwork behind it.

Maybe it would be a hundred thousand times more efficient than what already exists, as claimed. Maybe. But I'm not indulging in such fanciful speculation until a working implementation of those grand theoretical constructs exists and demonstrates how it actually perform. And I'm not holding my breath waiting for such an implementation to appear.

Share this post


Link to post

Gez said:
lulz.

If you don't see why passive-aggressive bitch sentences such as "I would really like to offend you because you do not seem very smart" get a thread locked before it turns uglier than it already is, then I'm afraid I have to tell you that you do not seem very smart.

I would have apologized, if I had the chance to do so! I was indeed not behaving good at formulating "bitch sentences", as you said, but I don't think that it was moderated only for that.

Doomsday, EDGE, GLBoom+, GZDoom, Skulltag, Vavoom.

Do you have a ten-year-old, second-hand holodeck for sale?

Right. In other words, you're yet another of those guys who have The Answer but don't actually know if it truly is the answer.

I am asking in this topic the current status, it seems a pretty clear question. If I had any answer I would not be asking anything.

To be frank, I consider VC's super-duper OpenGL renderer to be a vaporware prospect. Just like your virtual texture thing, it's just half-formed thoughts about a possible way to do things, with no actual groundwork behind it.

Maybe it would be a hundred thousand times more efficient than what already exists, as claimed. Maybe. But I'm not indulging in such fanciful speculation until a working implementation of those grand theoretical constructs exists and demonstrates how it actually perform. And I'm not holding my breath waiting for such an implementation to appear.

This chunk of text is already an answer to what I was asking, so thanks :)

Technically speaking, what are exactly the blocking issues which prevent such renderer to be incrementally developed?
Hardware support of OpenGL? (I don't understand this one, if it stands as one)
Legacy support of Doom rendering?

I'd really like to know where it becomes so hard...

Share this post


Link to post

  • Doomsday: currently undergoing complete rewrite of just about everything.
  • EDGE: I'm not sure if development continues now that Andrew has quit. SVN log shows no activity for the last five months.
  • GLBoom+: Still maintained and developed actively. I'm not sure where to see its SVN log, though, the repository won't let me and there does not seem to be a ViewVC/WebSVN thing over there, but the revision number keeps increasing.
  • GZDoom: Rendering code feature-frozen.
  • Skulltag: Same as GZDoom.
  • Vavoom: Still maintained and developed actively. The SVN log shows recent development in dynamic lights, shadow volumes, and hardware shaders.

As for what's so hard: it simply requires someone who has the know-how, the time, and the motivation to code all that. It's not more difficult than any similar project; it's just that the amount of people wanting to do that job is extremely small.

Share this post


Link to post
legolas558 said:

Technically speaking, what are exactly the blocking issues which prevent such renderer to be incrementally developed?



The only blocking issue is the lack of technically knowledgeable people that want to work at it. This is not really doable as a one-man project. You need others to discuss ideas with or just for inspiration if you are out of ideas.

Share this post


Link to post

virtual textures


Do you mean texture atlases? If so, I tried to do a map viewer that used one big texture, but it didn't work so well. The Doom engine requires that textures repeat, something that really can't be done with an atlas. It is possible to fix the problem with a pixel shader. I tried that as well, but the performance was worse then just batching by texture. However my card is five years old and I know its shader and fillrate performance are crap.

Ports should however be using an atlas for sprites, I don't see a reason not to. Walls could probably also be done, but would have to be "split" to avoid repetition. Floors and ceilings are the killer.

Share this post


Link to post
Scet said:

Ports should however be using an atlas for sprites, I don't see a reason not to. Walls could probably also be done, but would have to be "split" to avoid repetition. Floors and ceilings are the killer.


Not worth the hassle for walls. The splitting and special handling will easily kill any potential advantage and also introduce visual artifacts. For sprites, yes, it may help but don't forget that sprites normally only make up less than 5% of the entire rendering time. If you are lucky you'll gain half of these 5% with texture grouping.

Overall, on maps that have problems it's not texture changes that break them, it's having to render 20000+ primitives, each one a separate draw call. If someone wants to optimize this is the one spot they should go after so that all compatible primitives can be grouped into one batch. Everything else is just window dressing and may help with secondary issues.

Share this post


Link to post
Graf Zahl said:

each one a separate draw call.


You mean GZDoom doesn't batch at all? WTF man?

Share this post


Link to post

The code is 10 years old! And it's only the most recent generation of GFX cards where the code is truly showing its age. Even on my 3 year old GF 8600 it's still the fill rate/shader performance that's the most limiting factor in 80-90% of all cases.

And don't forget: Doom's not the kind of game that makes batching easy. There's way too much dynamic changes to the geometry that vertex buffers are quite hard to use (unless, of course, you stream them which again negates any advantage they may bring.) The rendering code is mostly immediate mode OpenGL with the exception of the flats which have an optional VBO implementation.

Share this post


Link to post

Well there are ways to modify only part of a VBO, most of the geometry in a Doom level is static. Though it can be difficult to tell which parts are static or dynamic.

Immediate mode really sucks, even plain old vertex arrays would be faster.

Share this post


Link to post

On my card they didn't make any difference whatsoever (remember: fill-rate limited!) But I had so many problems with them (either driver bugs or usability issues) that I stopped supporting them eventually.

Share this post


Link to post

Gez said:

  • Doomsday: currently undergoing complete rewrite of just about everything.
  • EDGE: I'm not sure if development continues now that Andrew has quit. SVN log shows no activity for the last five months.
  • GLBoom+: Still maintained and developed actively. I'm not sure where to see its SVN log, though, the repository won't let me and there does not seem to be a ViewVC/WebSVN thing over there, but the revision number keeps increasing.
  • GZDoom: Rendering code feature-frozen.
  • Skulltag: Same as GZDoom.
  • Vavoom: Still maintained and developed actively. The SVN log shows recent development in dynamic lights, shadow volumes, and hardware shaders.

Thank you Gez, this is the kind of summary I was expecting from somebody which has followed the scene much better and tightly than me.

I have just tried Vavoom and it looks much better than last time! Although its OpenGL support is not that good, possibly because of the SDL layer in-between?

As for what's so hard: it simply requires someone who has the know-how, the time, and the motivation to code all that. It's not more difficult than any similar project; it's just that the amount of people wanting to do that job is extremely small.

I have tried (recently or in a recent past) all the source ports you have mentioned, and I finally picked up GZDoom because its OpenGL support was much better than the others (at that time, and anyway in my opinion).

Is it today still the best one, when talking about OpenGL rendering? I will eventually give a hit to the other ports you specified above (like I did with Vavoom), but there has been a lot of FUSS about GZDoom so I really don't know where the facts are now

Share this post


Link to post
Scet said:

Do you mean texture atlases? If so, I tried to do a map viewer that used one big texture, but it didn't work so well. The Doom engine requires that textures repeat, something that really can't be done with an atlas. It is possible to fix the problem with a pixel shader. I tried that as well, but the performance was worse then just batching by texture. However my card is five years old and I know its shader and fillrate performance are crap.

Ports should however be using an atlas for sprites, I don't see a reason not to. Walls could probably also be done, but would have to be "split" to avoid repetition. Floors and ceilings are the killer.


This opens quite an interesting discussion :) - but unfortunately I am out of time now (although I am willing to contribute with my own little research on the subject), so I'll reply later!

Share this post


Link to post
legolas558 said:

Is it today still the best one, when talking about OpenGL rendering?

Depends on what you want.

For pure speed, GLBoom+ is the best; however it does not feature certain (G)ZDoom constructs that complicate rendering such as models, mirrors and portals. It does have some interesting optional rendering features, though, such as detail textures.

Vavoom's renderer is derived from Quake and it has superior light effects. But in my experience, it is also quite unstable so I don't use it much.

Doomsday has many interesting graphical features, unfortunately it is pretty much the slowest OpenGL engine for Doom. (Which is a big part of why they're rewriting it entirely.)

EDGE is kinda the underdog now. One thing I like in it is that it has a very good emulation of the spectre fuzz effect.

GZDoom is second-best for performances and can play most mods from /idgames (Boom-compatible, Legacy-compatible, ZDoom-compatible of course, Heretic/Hexen/Strife/etc.) so it's my personal favorite.

Share this post


Link to post

Scet said:
Do you mean texture atlases? If so, I tried to do a map viewer that used one big texture, but it didn't work so well. The Doom engine requires that textures repeat, something that really can't be done with an atlas.

I may be the newbie here, but if I remember correctly OpenGL can nicely repeat the texture if you properly setup one or two matrices for it.

Scet said:
It is possible to fix the problem with a pixel shader. I tried that as well, but the performance was worse then just batching by texture. However my card is five years old and I know its shader and fillrate performance are crap.

Ports should however be using an atlas for sprites, I don't see a reason not to. Walls could probably also be done, but would have to be "split" to avoid repetition. Floors and ceilings are the killer.

I still don't understand why: what's wrong with an "atlas" with proper offsets? OpenGL has a powerful texture matrix, again if I remember correctly.

Regarding my own experience: it is not something about an OpenGL FPS, but an OpenGL Orion-like game.

We needed to render a 3D view of the planet surface, as well as a "space view" of the planet as a geoid. I was able to do so using the same cached textures without too much hassle. I know that Doom is totally a different thing, but I grouped many tiny pictures in big "texture tiles" (I developed an algorithm to properly fill such tiles second the hardware's suggested texture size) and then used offsets to render each mini-texture individually.

I also got some issue with repetition, but I don't think it was of the kind that could prevent from using offset'd textures in Doom walls or floors (although I might not remember correctly and such limitation could exist).

Share this post


Link to post

Gez said:
Depends on what you want.

For pure speed, GLBoom+ is the best; however it does not feature certain (G)ZDoom constructs that complicate rendering such as models, mirrors and portals. It does have some interesting optional rendering features, though, such as detail textures.

Given that ZDoom has defined sort of a new standard for enhanced maps, this is a big limitation to me.

Vavoom's renderer is derived from Quake and it has superior light effects. But in my experience, it is also quite unstable so I don't use it much.

Yes that's something that becomes evident very quickly and causes indeed a "wow" effect

Doomsday has many interesting graphical features, unfortunately it is pretty much the slowest OpenGL engine for Doom. (Which is a big part of why they're rewriting it entirely.)

Interesting...they might come up with an original and killer renderer

EDGE is kinda the underdog now. One thing I like in it is that it has a very good emulation of the spectre fuzz effect.

When you're used to (G)ZDoom, this looks quite insufficient...although I recognize that it fulfills its proejct goals

GZDoom is second-best for performances and can play most mods from /idgames (Boom-compatible, Legacy-compatible, ZDoom-compatible of course, Heretic/Hexen/Strife/etc.) so it's my personal favorite. [/B]

If only we could come up with some next generation renderer...

Share this post


Link to post

Scet said:
Well there are ways to modify only part of a VBO, most of the geometry in a Doom level is static. Though it can be difficult to tell which parts are static or dynamic.

Immediate mode really sucks, even plain old vertex arrays would be faster.


I agree that most parts of the level could be made up of compiled lists, but it already works this way (for some engine) - doesn't it?

Share this post


Link to post

Take a ZDoom level. There are scripts, which can produce any number of effects (including random ones). The line specials can be put not just on lines, but also on monsters, weapons, items or just about anything and activated in a myriad of ways. There is absolutely no way of telling "this bit of geometry is not going to change, ever!" so you have to either assume everything is static and suffer insane glitches from hell, or assume everything is dynamic and it works correctly. What choice do you make?

Share this post


Link to post

Its not an all or nothing choice. The trick is to segment the world into logical chunks that can be drawn as one atom (everything within a subsector would be convenient though likely too small to be worthwhile, depending on your nodebuild strategy). Track changes to the atom and mark as needing an update and time stamp it. When BSP traversing the world, if not changed draw the lists for a subsector and move on. If changed, draw in immediate mode until a given time threshold is passed; at which point update the lists and revert to drawing the cached geometry for the atom.

Its a fair amount of work to implement it correctly as ideally you don't want to update an atom for trivial things like changes to texture offsets and lighting.

Texture offseting would be done by manipulating the texture matrix rather than caching the absolute offsets along with the geometry. In other words, you would normalize them into a per-atom coordinate space and storing relative offsets in the atom. If portals are required it may be beneficial to store barycentric vertex coordinates in the atom and translate the modelview matrix instead (heck this might be useful anyway (e.g., polyobjs)).

Lighting would be deferred to a shader.

You could even amortize out the effects of plane height changes with creative use of a vertex shader.

Just because everything is potentially dynamic does not preclude the possibility to cache it. It just necessitates the use of an intelligent dynamic caching scheme.

Share this post


Link to post

Gez said:
Take a ZDoom level. There are scripts, which can produce any number of effects (including random ones). The line specials can be put not just on lines, but also on monsters, weapons, items or just about anything and activated in a myriad of ways. There is absolutely no way of telling "this bit of geometry is not going to change, ever!" so you have to either assume everything is static and suffer insane glitches from hell, or assume everything is dynamic and it works correctly. What choice do you make?


Easy: half-dynamic, half-static e.g. dynamic caches! I think that DaniJ has explained this much better in the previous post.

(This is easy to design and think about, and somewhat harder to implement).

Do linedefs also modify their vertices or only translate?


DaniJ said:
Just because everything is potentially dynamic does not preclude the possibility to cache it. It just necessitates the use of an intelligent dynamic caching scheme.


100% agreeing here. Let's consider the 3D world without sprites e.g. the world of 3D planes. There is a huge amount of it which could be made up of OpenGL batchsets (aka compiled lists aka whatever), and the algorithms could even mark some batchsets as "rolling" and never cache them again e.g. render them in immediate mode because for some combination of effects they are changing very frequently. All this would be opaque to the user (if done correctly) so that only with a slow (or slown down) CPU you can detect the "calibration" hickup which properly selects immediate mode or cached mode.

The advantage are huges and such adaptive technique would theoretically behave correctly even on complex maps with many effects. Hell, we could even implement a flag which says "be careful renderer, this (sub)sector is a hotspot, never cache it", and it would work.

Next thing to do is to go lookup all these OpenGL engines and see how they are doing it :-)

Share this post


Link to post
DaniJ said:

When BSP traversing the world, if not changed draw the lists for a subsector and move on. If changed, draw in immediate mode until a given time threshold is passed; at which point update the lists and revert to drawing the cached geometry for the atom.

Its a fair amount of work to implement it correctly as ideally you don't want to update an atom for trivial things like changes to texture offsets and lighting.

Texture offseting would be done by manipulating the texture matrix rather than caching the absolute offsets along with the geometry.


GZDoom is doing all these things, albeit for flats only because the wall management is way too messy.

But there's a few things to consider:

- we want to reduce the amount of draw calls which renders the texture matrix a less appealing means to do things. If you need to update it for everything you'll negate a lot of the potential gains. Also, there's very little point in using and maintaining vertex buffers if all you do with them is draw low-vertex primitives that each need a state change. With these the maintenance overhead can easily outweigh the small gains on the renderer side.

- one thing I noticed is that mixing vertex buffers and immediatze mode is a very bad thing on ATI. It's actually slower than using immediate mode exclusively.

So in the end the entire thing is running in circles:

In order to reduce draw calls we need to orchestrate things so that they can be rendered out of a single vertex buffer. In order to use such a buffer, we need to do a lot of maintenance on that buffer, if we don't want to do that we have to resort to a streaming buffer but a streaming buffer which in turn means that precalculating is a waste of time as it's often faster to just do it each frame instead of tracking dependencies. But in order to use a streaming buffer in a way to actually reduce draw calls it needs to contain so much data that immediate mode turns out faster in the end. Well, now we are back at the start...

Share this post


Link to post

DaniJ said:
Its not an all or nothing choice. The trick is to segment the world into logical chunks that can be drawn as one atom (everything within a subsector would be convenient though likely too small to be worthwhile, depending on your nodebuild strategy). Track changes to the atom and mark as needing an update and time stamp it. When BSP traversing the world, if not changed draw the lists for a subsector and move on. If changed, draw in immediate mode until a given time threshold is passed; at which point update the lists and revert to drawing the cached geometry for the atom.

Luckily comparing changes of such lists would not require too much time (I think they are in the order of magnitude of small strings comparation), so I still think that it is perfectly feasible and no lag would happen.

Its a fair amount of work to implement it correctly as ideally you don't want to update an atom for trivial things like changes to texture offsets and lighting.

Yes but the skeleton, e.g. lists for really static subsectors, could already be implemented, isn't it?

Texture offseting would be done by manipulating the texture matrix rather than caching the absolute offsets along with the geometry. In other words, you would normalize them into a per-atom coordinate space and storing relative offsets in the atom. If portals are required it may be beneficial to store barycentric vertex coordinates in the atom and translate the modelview matrix instead (heck this might be useful anyway (e.g., polyobjs)).

Lighting would be deferred to a shader.

You could even amortize out the effects of plane height changes with creative use of a vertex shader.

You look quite ahead in the theory of this, have you been working on some 3D FPS engine?

Share this post


Link to post

Well I am a developer on the Doomsday Engine and am therefore quite familiar with the needs of an OpenGL rendering DOOM engine. :)

About a year ago I began researching techniques for what will ultimately become the renderer for Doomsday 2.0 This process involved writing several experimental renderers hardwired into the current (at the time) Doomsday release.

As I'm sure everyone is aware, Doomsday's current renderer is pretty much the slowest of all the current OpenGL rendering DOOM engines. This is because bar a few noteable exceptions we render everything in immediate mode as the renderer is now well over ten years old and has seen little change in that time.

The renderer we have planned for 2.0 is somewhat of a departure from "traditional" GL DOOM rendering and depends upon fragmenting the world into atoms which have their own BSP trees. Its a hybrid dynamic BSP/portal design.

Share this post


Link to post
DaniJ said:

This is because bar a few noteable exceptions we render everything in immediate mode as the renderer is now well over ten years old and has seen little change in that time.



If that was the reason all the other renderers would also be creepingly slow because to my knowledge none uses more modern techniques except for a few items. Even PrBoom+ uses immediate mode almost exclusively and we all know it's currently the fastest GL renderer out there.

GZDoom only uses VBO's for flats - and only on NVidia. The speed difference for straight immediate mode and VBO is marginal though - on ATI VBO is even slower.

(Disclaimer: I'm not saying that what you do is wrong but immediate mode surely is not the culprit for Doomsday's bad performance.)

Share this post


Link to post

Oh I wasn't trying to suggest that the main reason for Doomsday rendering slowly was the fact we use immediate mode. The real reason is we barely do any caching at all beyond constructing the BSP and precaching textures. All other structures are constructed JIT for every surface on every frame.

The fact is that we haven't made any effort in optimizing the construction process because with a well behaved caching scheme in place it reduces the time spent there to near insignificance.

Share this post


Link to post
entryway said:

immediate mode == scapegoat?


I wouldn't say 'scapegoat' but the experts in hardware rendering seem to think that any rendering job constists of 1000+ polygons per batch and never think about situations where this isn't the case. As such there's this widespread belief that immediate mode automatically means bad performance and that switching to vertex buffers magically resolves this. Which is of course bullshit.

Share this post


Link to post

DaniJ said:
Well I am a developer on the Doomsday Engine and am therefore quite familiar with the needs of an OpenGL rendering DOOM engine. :)

Yeah, The Doom Wiki precedes you - I hadn't seen that :)

DaniJ said:
About a year ago I began researching techniques for what will ultimately become the renderer for Doomsday 2.0 This process involved writing several experimental renderers hardwired into the current (at the time) Doomsday release.

As I'm sure everyone is aware, Doomsday's current renderer is pretty much the slowest of all the current OpenGL rendering DOOM engines. This is because bar a few noteable exceptions we render everything in immediate mode as the renderer is now well over ten years old and has seen little change in that time.

The renderer we have planned for 2.0 is somewhat of a departure from "traditional" GL DOOM rendering and depends upon fragmenting the world into atoms which have their own BSP trees. Its a hybrid dynamic BSP/portal design.


Unfortunately I have just tested doomsday and there must be some serious texture bug because on my hardware (fast-food i855GM) textures are all messed up (seems like a pitch/width issue).

I'll report this on Doomsday forums, which is a much more appropriate place - although the confirmation email has not yet arrived.

By looking at the Doomsday project it looks really good! I like very much its modularity and believe that you can really get out with something good with this new planned renderer...perhaps I'll join testing (I already have a few bugs to report) later

Share this post


Link to post

In any case, if somebody wants to test VBO efficiency on own hardware, there is a version of glboom-plus with unlocked gl_sky_detail config variable. You can set it to 16384 or something big and compare FPS for glboom-plus_immediate.exe and glboom-plus_vbo.exe. The best place to test is doom2 map12.

Formula for number of triangles for SkyDome is:
rows = 4;
columns = 4 * gl_sky_detail;
vbosize = 2 * rows * (columns * 2 + 2) + columns * 2;

So for "gl_sky_detail 32768" in config glboom-plus will render 2 359 312 triangles for sky dome.

But keep in mind, glboom-plus is trying to be tricky. If mouselook is disabled and FOV == 90 then only half of these triangles will be drawn.

btw, VBO code is disabled in glboom-plus by default (in sources), because it does not help so much.

http://prboom-plus.sf.net/prboom-plus-vbo_test.zip

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×