Multithreaded Renderer Beta [NOW IN MASTER!]

[Altazimuth]

It's live! The new DRDTeam build (about 19 hours from now) will have this officially in. Yes, I lied, I couldn't wait. Earlier today I made an important change that means that it scales far better with number of threads, both performance (faster) and CPU usage (lower). A sensible maximum at this stage is the number of physical cores your processor has. I did see some gains past that point on my 7950x but not that huge.

I haven't tested this on a processor with E and P-cores and frankly I'm scared of what'll happen if I do.

Edited May 4, 2023 by Altazimuth

[skillsaw]

Was looking to update -- did the dev build run? It's looking like the latest is still dated Apr 26.

 

https://devbuilds.drdteam.org/eternity/

[Trov]

On 5/2/2023 at 12:14 AM, Altazimuth said:

I think NUTS is largely slow due to just how many monsters are thinking, rather than being rendered. It's definitely a combo but IIRC the thinking takes up way more time, meaning that faster rendering isn't gonna help toooo much.

Now someone can try making monster thinking multithreaded too :^)

[Gez]

50 minutes ago, Tarvis said:

Now someone can try making monster thinking multithreaded too :^) 

Without breaking predictability, which is the single most essential thing for both demo and multiplayer? Chances are you'll need so much overhead that the performances will end up being worse than single-threaded...

[Darkcrafter07]

1 hour ago, Gez said:

Without breaking predictability, which is the single most essential thing for both demo and multiplayer? Chances are you'll need so much overhead that the performances will end up being worse than single-threaded...

 

I believe not that many people use Eternity, GZDoom or any other advanced source port to record demos, moreover, demo compatibilty breaks with every new version introduced and that is probably the reason why. Wait, does Eternity have a multiplayer mode at all?

[Graf Zahl]

2 hours ago, Gez said:

Without breaking predictability, which is the single most essential thing for both demo and multiplayer? Chances are you'll need so much overhead that the performances will end up being worse than single-threaded...

 

with Doom it is virtually impossible because any ticking actor can change other actors at will, e.g. putting a monster or projectile into its death state. That means you need so much synchonization that it'll probably kill any potential benefit. To allow multithreaded gameplay, actor interaction needs to be done VERY differently - but that'd make the result not feel like Doom anymore.

 

[Darkcrafter07]

2 hours ago, Graf Zahl said:

 

with Doom it is virtually impossible because any ticking actor can change other actors at will, e.g. putting a monster or projectile into its death state. That means you need so much synchonization that it'll probably kill any potential benefit. To allow multithreaded gameplay, actor interaction needs to be done VERY differently - but that'd make the result not feel like Doom anymore.

 

Maybe it could have been threaded yet, let's say if the monsters were separated one group from another and impossible to interact between each other. A need to preprocess a map would possibly arise for the engine to know what guys those are exactly that can not interact.

[Edward850]

5 hours ago, Darkcrafter07 said:

 

I believe not that many people use Eternity, GZDoom or any other advanced source port to record demos, moreover, demo compatibilty breaks with every new version introduced and that is probably the reason why. Wait, does Eternity have a multiplayer mode at all?

Eternity maintains demo compatibility between versions, has vanilla recording and playback, and does have multiplayer that it has no plans on removing.

 

2 hours ago, Darkcrafter07 said:

Maybe it could have been threaded yet, let's say if the monsters were separated one group from another and impossible to interact between each other. A need to preprocess a map would possibly arise for the engine to know what guys those are exactly that can not interact. 

The semantics of what can't interact are not obvious, and a mispredict would be expensive to reprocess. Doom's space isn't a known quantity as you don't know if two rooms are absolutely not connected (imagine any horseshoe map, do the two ends ever interact?), polyobjects outright change the arrangements of segs and portals mean distance isn't a valid quantity either.

The possible kinds of interactions are also unpredictable, it is possible for a hitscan enemy on the exact opposite side of the map to shoot a line that triggers ACS that effects the opposite side of the map drastically (spawns/kills enemies, for example), thus disrupting the potential order of tasks for a completely unrelated group of enemies. ACS as a semi-complex VM isn't a trustworthy sandbox so you can't make assumptions about what it can and can't interact with and from where long term.

 

Edited May 5, 2023 by Edward850

[Altazimuth]

11 hours ago, skillsaw said:

Was looking to update -- did the dev build run? It's looking like the latest is still dated Apr 26.

I've been trying to sort this out. Recent libpng updates have caused devbuilds to not work. I tried to fix it but failed to do so prior to the scheduled time last night. Hopefully it'll be sorted by next time. In the meantime have a manual build from me.

 

Eternity_Master_2606a1.7z

[Meerschweinmann]

I have tested the new 2606a1 release with my I7 3770K. Now i can get 141 FPS with 4 Threads :)

With the release before i had 127FPS with 3 threads (shown on previous page) and 2 to 3 frames more with 4 threads.

With the old release the sweetspot was 3 threads. Now i can use all 4 cores of my old CPU.

Whatever you have done, it is very good.

Edited May 6, 2023 by Meerschweinmann

[Altazimuth]

Thanks! Props to ceski, Edward850, and InsanityBringer for helping point me in the right direction to help alleviate stress on threads.

In other news the devbuilds are working again, so please get builds from there if you want. https://devbuilds.drdteam.org/eternity/

[Edward850]

On 5/4/2023 at 9:16 PM, Altazimuth said:

I haven't tested this on a processor with E and P-cores and frankly I'm scared of what'll happen if I do.

Tested this on an i9-12900H (20 threads, 6 P-Cores, 8 E-Cores). Multi-threading performance tops out at the number of P-Cores available (6 in this case). Windows 11 appears to be keeping Eternity off the E-cores (which is correct behavior as we have no special behavior for them and it's a foreground process), and hyperthreads do not add any extra performance. Performance at 1440P tops out at about 400FPS.

 

It's interesting that the characteristics of the multithreaded rendering seem to be effective at revealing the configuration of the CPU and even the Intel Thread Director, but it's also a good indication that the multithreading is doing everything correctly.

Edited May 7, 2023 by Edward850

[s4f3s3x]

Any update for MacOs?

[Edward850]

The only person we know who can compile Macos builds is @printz, unless someone else steps up to compile such builds.

[Meerschweinmann]

I don't want to bring bad news, but is it  possible that the multithread renderer in the DRD devbuilds has some trouble with some AMD Ryzen CPUs? I have tested the multithread renderer intensive with my good old i7 3770K and the i3 4th Gen at work with absolutely no problems.

The Ryzen 5600X from one of my sons runs the multithread renderer like a dream too.

 

But the Ryzen 4700G from my other son and my Ryzen 6800HS Notebook get some graphical glitches the more threads i activate in EE.

Mostly notable at the HUD, weapons or 2D Sprites. They flicker sometimes.

With one thread activated this flicker disappears.

Later i will test this phenomenom on the 6800HS Notebook with an older multithread build, because i think i noticed these glitches with the first DRD Devbuild.

The i3 4th Gen uses Win10.

Every other named PC uses Win11.

 

Edit: The flicker on my 6800HS notebook comes with build 2606a1. The 2023-05-01 build is good.

 

Edited May 8, 2023 by Meerschweinmann : Later tested

[NinthBurn]

I've tried multiple builds (initial, 2606a1 and the latest) and I don't have any flickering. I ran the game on my Ryzen 2700X on maps like Sunder Map09,15, Heartland Map05 and Doomium Map15, which are pretty demanding. I also tried to run in different thread configurations(3/8/12), but that didn't seem to produce any visual glitches.

[Meerschweinmann]

As i said, the Ryzen 5600X of one of my sons did not have this flicker problem. So it affects not all Ryzens in my ownership.

The two Intel i have tested runs the multithread renderer without problems.

 

But the 4700G PC and my 6800HS Notebook have the flicker. With 2 or 3 threads it is nearly absent.

With 4 or more it begins.

 

Every build after 2023-05-01 has this "problem" on the two named PCs, whether i run Heartland, selfmade maps or DOOMs original Campaign.

 

The thing is that this flickering happens so fast, that i can't make a screenshot.

My workaround is to keep the threads as low as necessary to avoid it on these two PCs.

 

None of the named machines is overclocked or something. The notebook can't do that anyway.

 

The only commonality i can see is, that NinthBurns 2700X and my sons 5600X have no integrated GPU, and my affected 4700G and 6800HS have integrated GPUs and because of that smaller caches then their non APU pendants. If that makes any difference at all.

 

Hope my english is not too scary :)

Edited May 10, 2023 by Meerschweinmann

[Altazimuth]

What does your final page of video options look like?

[Meerschweinmann]

The last page from video options says:

 

Advanced

Video Driver: SDL GL2D

 

OpenGL

GL Color Depth: 32

Texture Filtering: GL_Linear

Use Extensions: Yes

Use ARB Pixelbuffers: Yes

 

Could it be the ARB Pixelbuffers? For whatever reason this option is "no" on my i7 3770K.

[Altazimuth]

Yeah. Try turning that off. I think it might cause issues of some unknown variety.

 

[Altazimuth]

Oh god I'm reproducing it. I'll look into it.

EDIT: Fixed

Edited May 9, 2023 by Altazimuth

[Meerschweinmann]

Nice to read that i was not hallucinating :)

 

Good that you could fix it.

By the way, ARB Pixelbuffers set to "no" does not change anything with this flicker. For what is this option for?

[fraggle]

20 hours ago, Meerschweinmann said:

Hope my english is not too scary :)

Your English is perfect.

[Quasar]

3 hours ago, Meerschweinmann said:

Nice to read that i was not hallucinating :)

 

Good that you could fix it.

By the way, ARB Pixelbuffers set to "no" does not change anything with this flicker. For what is this option for?

Uses an (at the time it was implemented at least) allegedly faster way of uploading a texture to video memory.

[Meerschweinmann]

Thank you @fraggle :)

 

Thanks @Quasar for the info. I will test with and without this option to see if it makes a noticable difference in fps.

 

 

[Meerschweinmann]

I have tested the Eternity-x64-4.03.00-pre-1083-g84124aa5 Build with my i7 3770K.

 

That is an CPU with 4 physical cores and 8 Threads.

 

EE runs with commandline "-timedemo demo1" and an resolution of 2132x1200.

Every run was started 3 times to minimize measure-errors.

V-Sync off.

 

1 Thread 155 FPS
2 Thread 185 FPS
3 Thread 198 FPS
4 Thread 208 FPS
5 Thread 199 FPS
6 Thread 202 FPS
7 Thread 207 FPS
8 Thread 208 FPS

 

1 Thread ARB PB Yes 200 FPS
4 Thread ARB PB Yes 291 FPS
8 Thread ARB PB Yes 295 FPS

 

Selecting in EE more threads than the CPU has physical cores seems not necessary, but it does nothing really bad if i do so.

 

ARB Pixelbuffers brings another boost in FPS.

 

The results speak for themselves. The multithread renderer is a very nice must have feature. Thanks for that.

Edited May 9, 2023 by Meerschweinmann

[Meerschweinmann]

Now i have tested with my sons Ryzen 4700G. That is an CPU with 8 cores and 16 Threads.

 

EE runs with commandline "-timedemo demo1" and an resolution of 2866x1200.

Every run was started 3 times to minimize measure-errors.

V-Sync off.

 

1 Thread 144 FPS
2 Thread 168 FPS
3 Thread 175 FPS
4 Thread 184 FPS
5 Thread 188 FPS
6 Thread 193 FPS
7 Thread 195 FPS
8 Thread 194 FPS
12 Thread 198 FPS
16 Thread 201 FPS

 

1 Thread ARB PB Yes 143 FPS
4 Thread ARB PB Yes 179 FPS
8 Thread ARB PB Yes 186 FPS
16 Thread ARB PB Yes 194 FPS

 

Same story as with the i7 3770K. Selecting more threads then the CPU has physical cores seems not really necessary, but does not hurt anything.

Above 6 Threads FPS "stagnate". Because of that i set it to 6 and the CPU is not at 100% limit.

 

ARB Pixelbuffers with the 4700G costs some FPS. So it depends on the CPU if it is good to use it or not.

 

By the way. With the last DRD Devbuild i did not notice the flicker anymore on this PC. Wonderful!

 

Edited May 9, 2023 by Meerschweinmann

[Meerschweinmann]

I have made one last test on my Ryzen 6800HS notebook.

8 cores and 16 Threads for this CPU.

 

The FPS max out at 6 Threads and with more nothing really big happens.

But hey, it gives me 50+ frames in this case with the same CPU. 151 FPS 1Thread vs 204 FPS 6 Thread. That is great!

 

ARB Pixelbuffers costs FPS. These option seems only good for older CPUs like the 2013 released i7 3770K.

 

The flicker is gone here too with latest DRD Devbuild. Great work!

Edited May 10, 2023 by Meerschweinmann

[nobleflame]

By the way, this update is incredible. I can now play Heartland (map 07 in particular) at a consistent 144 fps on my i7 9700KF; 3070 combo. Before, it would drop to 100 and below sometimes. 

Well done, guys!

[Darkcrafter07]

Well, on my system (Ryzen 5 1600, 3.5GHz OC, AGESA updates to 2018, GTX 1063) it's nearly 5% faster than prboom-plus in software (6 threads used) and 7% slower (1 thread used).

 

But with infamous nuts.wad the real boost from 20FPS to 32FPS can be observed if at least 2 threads are activated, then it becomes just as fast as boom.

 

That's a sort of a good achievement because eternity has way more features and checks for monster logics I think, and just 2 threads give it a nice performance boost. How many CPU are out there that have at least two threads now? A lot, that's a win.