what's the state of the art in node builders?

If my priorities for nodebuilding are:

 

* works for PrBoom+

* minimises slime trails

* supports a reject map

 

What's my best option?  Preferably of those built into GZDB though I am willing to run an extra program if it will really help.

 

It sure as heck ain't zennode, I know that much (but then, zennode was shit back in the DOS days, too)

 

You might want to try

 

 

I was thinking about Zennode, too. I am creating a UDMF format map which utilizes a lot of GZDoom shenanigangs. Is ZokumBSP also recommended for non-vanilla wads?

I was about to say ZDBSP, but then I saw you want a reject map.

14 hours ago, Urthar said:

I was about to say ZDBSP, but then I saw you want a reject map.

 

I'm old enough to remember when reject maps were actually necessary :)  They probably aren't, with modern PCs.

 

On 8/23/2018 at 9:30 PM, Capellan said:

 

I'm old enough to remember when reject maps were actually necessary :)  They probably aren't, with modern PCs.

 

They are never necessary, but, here's the thing. Doom always checks the REJECT lump when doing a sight check. If the lump doesn't exist, most ports provide a dummy in memory. However it gets in memory, real lump, or a dummy provided by the port, the game always checks it.

 

Now, if it's not properly built, it provides no benefit. But, if it's even just half-ass built, it always provides *some* benefit.

 

So, you're right: it's not necessary. But, it's always there. Having a properly-built REJECT map *never* slows the game down, and almost always speeds it up. As a mapper, you can never assume what computer your *customers* are playing on. They may have an older, slower computer.

 

Typically, the more monsters you have, the more benefit a REJECT map can provide. Monsters typically do multiple sight checks per second. So if you have, say, 1,000 monsters, that may cause over 3,000 sight checks each second. Sight checks can be expensive, performance-wise. The sight-check built by Carmack in v1.0 to 1.2 of Doom was typically a lot more efficient than later sight-checks, but it had a serious flaw...an oversight, that caused it to behave badly in specific conditions, so Carmack went with a different algorithm, which is usually quite slower. Heretic uses the old algorithm, as it was based on Doom v1.2, so it has this flaw.

 

Randy Heit, the creator of ZDoom, found and fixed the flaw in the old algorithm, and decided to use this fixed algorithm in ZDoom, which provided a nice performance boost.

 

But nothing is faster than not having to do the sight check at all, which is what a proper REJECT lump provides. Specifically, it tells the engine which sectors can never possibly see into other sectors. So, when the monster is in one of these sectors, and the player is in the other sector, the REJECT lump tells the engine: "No need to do the sight-check, this monster absolutely cannot possibly see the player." When the REJECT lump is not built, it always tells the engine: "I don't know if sight is possible, so you have to do the full sight-check to determine if sight is possible."

 

Let's say that somme people are playing coop with their friends on a home LAN. And, let's say they are playing a big, complex map with a ton of monsters...maybe 4,000. As soon as one of the computers on that LAN dips below 35 frames-per-second (fps) render speed, all coop players will experience lag.

 

So, *anything* you can do to speed up the map just might make the difference between a fun experience, and a laggy mess. It's not about trying to get 400 fps vs. 350. It's about running one totally automated step at the end of your mapping process that cannot possibly slow down the game any, and will practically *always* provide a boost in speed, which just might make the difference.

 

What I'm saying is that it's not the best advice to claim that it's no longer necessary. It was never necessary. But it's required, by the mapper, or provided as a dummy by the engine. So, why not let it provide a boost in gameplay speed? In many cases, it's a very small speedup...in other cases, it's huge. I've noticed a big speedup, even using the fast, fixed 1.2 algorithm, on highly-populated, complex maps.

 

It's not often that you can get a free speedup, by using an automated tool.

 

 

I am aware of what the REJECT map does.  That's why I prefer a node builder that does it.

 

(And as far as "necessary" goes, try playing Doom on a 386 without one :) )

 

Aren't there ports that don't support Zokum?

For the record, I'm using GLBSP at the moment which seems to be working OK.  I'm getting a few minor slime trails but nothing like the gaping eyesores of zennode.

 

1 hour ago, kb1 said:

They are never necessary, but, here's the thing. Doom always checks the REJECT lump when doing a sight check. If the lump doesn't exist, most ports provide a dummy in memory. However it gets in memory, real lump, or a dummy provided by the port, the game always checks it.

 

Now, if it's not properly built, it provides no benefit. But, if it's even just half-ass built, it always provides *some* benefit.

 

[...]

 

But nothing is faster than not having to do the sight check at all, which is what a proper REJECT lump provides. Specifically, it tells the engine which sectors can never possibly see into other sectors. So, when the monster is in one of these sectors, and the player is in the other sector, the REJECT lump tells the engine: "No need to do the sight-check, this monster absolutely cannot possibly see the player." When the REJECT lump is not built, it always tells the engine: "I don't know if sight is possible, so you have to do the full sight-check to determine if sight is possible."

This got me thinking: could there be a tool, maybe just a GZDB plugin, that would allow you to make a reject table manually? For example, select a group of sectors, call it group A, select another group of sectors, group B, and then say that all sectors in group A can't see sectors in B? It would be a crappy reject table, but it would be better than none.

 

In very large, very detailed, very monster-filled maps, one must switch to ZDBSP when ZenNode starts flaking out, but that's exactly the kind of map where you need a reject table for performance, because my budget gaming laptop starts dipping well below 60fps in PrBoom+. 

 

Can you build nodes with ZDBSP but build REJECT with something else? That still might not be better than a crappy manual reject table, because a built reject table can severely bloat filesize at that point. A small, crappy, manually-built reject table might be the best?

47 minutes ago, Memfis said:

Aren't there ports that don't support Zokum?

ZokumBSP's default settings work with any port that would normally run the map if it was run through a different node builder (and sometimes even more so, to the point of squeezing in some maps into vanilla compatible that wouldn't normally be. However, if the zero header setting is set to "no zero header", it provides maps that are compatible with vanilla and chocolate doom and modern ports like GZDoom or Eternity, but break in MBF. There is a third zero header setting, where it is a zero footer instead of a header. Unlike the other two settings (no zero header allowing for slightly more optimized maps for ports that support it, and zero header running on anything normally compatible with the map) the zero footer setting has no real advantage...instead causing demos to desync.

I recommend leaving the zero header in the default setting of it being on. Even with the zero header, ZokumBSP is supposed to provide more optimized blockmaps than Zennode does.

3 hours ago, Capellan said:

I am aware of what the REJECT map does.  That's why I prefer a node builder that does it.

(And as far as "necessary" goes, try playing Doom on a 386 without one :) )

 

Never said you didn't :) I was being kinda verbose for the benefit of anyone who read it.

 

So, when you said that you're glad they're no longer necessary, you meant that it was because computers are faster now? I may have mistook your meaning. I've long been an advocate for building a proper REJECT table, though many mappers don't do it. A while back there were a few threads claiming that REJECT lumps were not necessary anymore, specifically because of the faster, fixed version of the old algorithm being used in ZDoom-derived ports. I've always countered by saying that, yes, ZDoom's sight-check is generally faster, but nothing is faster than having a good REJECT lump to begin with. Sorry if I sounded harsh (I tried not to), or misunderstood.

 

2 hours ago, Benjogami said:

This got me thinking: could there be a tool, maybe just a GZDB plugin, that would allow you to make a reject table manually? For example, select a group of sectors, call it group A, select another group of sectors, group B, and then say that all sectors in group A can't see sectors in B? It would be a crappy reject table, but it would be better than none.

 

In very large, very detailed, very monster-filled maps, one must switch to ZDBSP when ZenNode starts flaking out, but that's exactly the kind of map where you need a reject table for performance, because my budget gaming laptop starts dipping well below 60fps in PrBoom+. 

 

Can you build nodes with ZDBSP but build REJECT with something else? That still might not be better than a crappy manual reject table, because a built reject table can severely bloat filesize at that point. A small, crappy, manually-built reject table might be the best?

It's interesting that you mention that: Please check out RMB - the Reject Map Builder. It's a neat, nice concept that lets you make sectors where you're "invisible" to monsters (like in dark areas, where the monsters "can't see you"), and other neat effects.

 

The biggest issues with RMB are:

1. It's a DOS tool.
2. Any map rebuild can cause a sector-to-reject map mismatch, screwing up your effects.
3. The tool relies on sector numbers, so you have to be careful

Yes, you can build REJECT with a separate tool - it's completely independent of the nodes. Regarding your post, currently there's no support for smaller REJECT lumps. REJECT lumps are 1 bit per sector, per sector, which is very efficient on small, IWAD levels, but grows exponentially.

 

Here's the problem, as I see it: REJECT tables do save time by avoiding sight checks. But only a tiny amount of time. For tons of monsters, this does add up, and produces a combined savings that matters. But any code that would allow you to have a smaller table would most-likely negate those tiny savings, due to the overhead in the more complex logic decoding the less-than-full sized block.

 

The other issue is that, ironically, as the REJECT tables get bigger, they may even slow things down, by virtue of being able to consume memory cache-lines, wiping out other, more used data. But, don't read me wrong here: This slowdown occurs in all ports, regardless of whether or not the REJECT lump is built properly or not, because the ports are all going to have to read this memory (even if it is all zeros). So, building it properly *is* going to speed up the game. I've always felt like there was a certain Doom map size that was optimal and really shouldn't be exceeded. A lot of issues start to rear their ugly heads as we cross certain boundaries and limits. If you think about it, Doom levels *are* able to be pretty huge, without having a lot of special code in the engine to allow that. Modern games use a lot of *trickery* to make it look like they are rendering vast areas - they're not. Instead, nearby stuff is being swapped in, and far-away stuff is being swapped out, and less-detailed versions of it are being rendered instead. It's a wonderful illusion, but it requires tons of behind-the-scenes "sleight-of-hand" that Doom does not possess. Doom just does it (as long as it's not *too* big.)

 

Some could rightly argue that taking out the code that reads the REJECT lump might improve performance these days, by avoiding the cache issue, and, for large maps they may be right. But, those ports will not be able to properly render maps that used hand-crafted REJECT lumps (as built by tools like RMB), and they will not be able to play back demos recorded when the REJECT mechanism was in place. Issues, issues.

 

To answer the original question, you can use your favorite node builder for the main build, and use ZenNode just for the REJECT lump, if you wish. I don't think it builds a "perfect" REJECT lump (it takes shortcuts whcih also speeds up the calculations), but it can be instructed to just build the REJECT lump. With a clever batch file, you should be able to get both programs to run from one DoomBuilder map build request.

7 minutes ago, kb1 said:

So, when you said that you're glad they're no longer necessary, you meant that it was because computers are faster now?

 

Yeah, I expect that for modern PCs, you'd need a pretty huge map / monster count for the lack of a reject map to noticeably impact playability.  That was definitely not the case in 1994 :)

 

I used to use RMB back in the day, though obviously it's not an ideal tool today.  It appears from the GZDB interface that GLBSP generates a reject map, and it does a much better job of squishing slime trails in prboom+ than zennode, so it's what I am currently using.

Sorry for double-post:

REJECT builders are not in abundant supply, because:

1. They aren't strictly necessary.
2. They are not too hard to build, but:
3. They are difficult to optimize.

A "perfect" REJECT lump builder has to move down every pixel of every line, and "look" down every angle towards every pixel of every higher-numbered line, doing line intersection calculations, to determine which lines may be able to see each other. That directly translates to which sectors can see each other. It has to move down each line very small amounts, to be able to "see" through tiny cracks to other sectors. If you move too far down the line each iteration, you may miss a tiny crack through a sliver where you could otherwise see through to a far-away sector. Monsters would not be able to notice you peeking through a small crack.

 

A long while back,, while severely hacking up some DoomBuilder 1 code, I built a functioning "perfect" REJECT builder as well as a node builder (nothing fancy), all in Visual Basic 6 code. I optimized it to death, using every trick in the book. It was still a lengthy process for large maps. But, oh boy did it make a difference in such maps. As with a lot of my projects, it's an unrelease-able mess, because it was never designed to be for general use - my engine required a custom IWAD, my map editor assumed the existence of lots of resources not present, etc. etc. But I would be pleased to collaborate with and provide technical advice to anyone that wanted to build one.

4 minutes ago, Capellan said:

 

Yeah, I expect that for modern PCs, you'd need a pretty huge map / monster count for the lack of a reject map to noticeably impact playability.  That was definitely not the case in 1994 :)

 

I used to use RMB back in the day, though obviously it's not an ideal tool today.  It appears from the GZDB interface that GLBSP generates a reject map, and it does a much better job of squishing slime trails in prboom+ than zennode, so it's what I am currently using.

--cross-post!!--

Actually, I have to stand behind my post about the "coop on a LAN" stuff. Using Doom's original peer-to-peer networking (but with UDP today), I've had some friends setup at my place, playing 4-player co-op games. Two of my PCs are beasts, and the other two are like dual-core, low-end piece-of-poo models. Also each PC is playing a different resolution, has different hardware, etc. We play some vicious maps, and, in those games, you want mass hordes of fodder to mow down. Those games get hectic, and, there's always that one hallway that ends up full of dead bodies, blood, player corpses, you name it. And, as someone on one of those slow PCs looks down that hallway, everyone gets at least one pulse of noticeable lag.

 

If everyone is hitting...

36 fps: no problems!

35 fps: no problems!

34 fps: BAM! Lag, because one of the PCs wasn't able to report in time. This has somewhat of a cascade effect that amplifies somewhat.

 

Now, if the building of a REJECT table made even a teeny-tiny speed improvement, that particular lag might no have happened. It matters, even in this day and age. Unless you're the only consumer of your maps, you can't assume what your consumers are playing your maps on. REJECT makes your maps faster. So, REJECT makes your maps better :)

I was curious about glBSP, so I booted up GZDoomBuilder (2.3.0.3039) 64bit, and found something curious. Regardless of which node builder I saved the map with (Zennode Normal, glBSP Normal or ZDBSP Normal No Reject), the resulting Reject Table viewed in Slade appeared to be filled with zero values.

 

(Edit: Discovered I needed to force a node rebuild before saving.)

There's a bit of a difference, but glBSP definitely produces something, though with a different map I tried the glBSP reject was almost but not entirely filled with zero values.

Edited August 28, 2018 by Urthar

13 hours ago, Urthar said:

I was curious about glBSP, so I booted up GZDoomBuilder (2.3.0.3039) 64bit, and found something curious. Regardless of which node builder I saved the map with (Zennode Normal, glBSP Normal or ZDBSP Normal No Reject), the resulting Reject Table viewed in Slade appeared to be filled with zero values.

 

(Edit: Discovered I needed to force a node rebuild before saving.)

 

There's a bit of a difference, but glBSP definitely produces something, though with a different map I tried the glBSP reject was almost but not entirely filled with zero values.

Saw your edit. A quick comment:

Was that example a small-ish map? (Looks like it might be, based on the sizes).

 

Hex view is deceptive, unless you know your binary, as REJECT is bit-based, vs. byte-based. Zero is the "default" state...it's the state you get when a blank REJECT is used. It means "always do the sight-check."

 

On small maps, you can expect for there to be lots of zero bits (and bytes), especially when a single sector spans a bunch of areas, because you can expect that many other sectors can see this big sector. On hyper-detailed maps with sector detailing/borders, and just maps with lots of cubbyholes, you start to get more ones, and more chances for speedups.

 

Fun fact: The IWAD REJECT maps are pretty terrible, and can be rebuilt to be *way* more efficient, if it weren't for demo sync. I've rated efficiency as the ratio of ones to zeros, and seen IWAD maps go from 11% to upper 70's by rebuilding the REJECT maps as good as possible.

 

Another fun fact: There used to be a map editor (can't remember which) that built a default REJECT map, but built it inverted (all one's), rendering all the monsters blind.

On 8/28/2018 at 12:38 AM, kb1 said:

What I'm saying is that it's not the best advice to claim that it's no longer necessary. It was never necessary. But it's required, by the mapper, or provided as a dummy by the engine. So, why not let it provide a boost in gameplay speed? In many cases, it's a very small speedup...in other cases, it's huge. I've noticed a big speedup, even using the fast, fixed 1.2 algorithm, on highly-populated, complex maps.

 

It's not often that you can get a free speedup, by using an automated tool.

 

 

 

I didn't want to do any testing myself so I searched a bit and found some really old post about this subject with some actual numbers:

 

https://www.doomworld.com/forum/post/370100

 

So the savings are minor even with an extreme scenario on a map that seems to be tailor-made for a reject speed-up, entirely disregarding that this info is 14 years old and retrieved on far weaker systems than what is on offer as low end these days and b) no, Reject is not free! Building a good reject map takes time. I still remember that when I checked out RMB it could run for several hours on such large maps. I have no idea how it is today, but I'd say it's still several minutes to build a good one.

And what if you use a modern port that doesn't load an empty reject and just skips the check instead? Suddenly that 'free' table results in some avoidable CPU cache pollution and might actually SLOW THINGS DOWN, because that map can become quite large with rather scattered access patterns.

 

So it's not as clear cut as you make it out. That's only the case when running an engine that doesn't change how it was used originally and on hardware that does obey the same principles as what was considered state of the art in 1993. That cache thing is a real concern. Today it may actually be beneficial to have a slightly less efficient algorithm if it results in less hits of uncached memory.

 

@kb1 Yeah, the map that was almost entirely zeroes was a small vanilla map. DQM05 on the other hand is fairly detailed short map. (The Witch House from DotBoomed.)

 

Jerry.C, the statements that I made were very carefully chosen to be accurate, as the proper wording is crucial to ensure that what is being said is factual. My goals were to guide in the proper use of the faculties available to us all, not to try to prove someone wrong. But, when advice is being given that runs counter to optimal performance during gameplay, I must provide the results of my research. Besides, it's in my best interest that mappers make levels that play well :)

 

Because you raise a few points, my replies are inline:

 

21 hours ago, Jerry.C said:

 

I didn't want to do any testing myself so I searched a bit and found some really old post about this subject with some actual numbers:

https://www.doomworld.com/forum/post/370100

I have done the testing - extensively, isolating many different aspects of the Doom engine, which includes the REJECT table. That forum link leaves a lot to be desired as far as "actual numbers" go. Graf mentions what sounds like an arbitrary average 1% to 3% (300% variance, by the way) of engine time devoted to sight checking. Assuming that's true, 3% is about 1 frame per second, which is not nothing. This "analysis" seemed to be based on a single 600-monster map of a certain size, with a REJECT map of unknown population, using a modified Heretic sight-check algorithm. That's not much of an analysis.

 

In the past, I used to enjoy lots of LAN co-op play, on somewhat older PCs that ran a somewhat high resolution - let's just say that the system didn't have a lot of rendering frames-per-second to spare. We used to "zoo-ify" our maps - we would work in secret, modifying the maps, to make each other "shit bricks" when we saw the vast number of hellspawn we were faced. Of course we also provided tons of over-powered weapons and powerups. Gameplay was ridiculous, but undeniably fun.

 

And, yes, proper REJECT lump builds made significant performance improvements. And, yes, the engine did employ the fixed Doom v1.2 sight check.

 

Now I know everyone likes the numbers, and, yeah I ran numbers, and, no, I do not have a bunch of pseudo-statistics prepared to show. They don't mean anything anyway, without specific engine, specific WAD, specific REJECT lump, specific hardware, etc. Really, it should suffice to say that, worst scenario: no performance difference, and everything else is SOME level of improvement. Most of my testing consisted of those co-op sessions: Start up the game..."Ugh, lag!". Stop the game, build the REJECT, distribute the new WAD, start up the co-op session again..."Wow - this plays better...". Sure, it depends on sector count and monster count. So what? Worst = no gain (no loss either), and everything else = some gain. It's pretty simple.

 

Quote

So the savings are minor even with an extreme scenario on a map that seems to be tailor-made for a reject speed-up, entirely disregarding that this info is 14 years old and retrieved on far weaker systems than what is on offer as low end these days...

Really? Give me the change in your pocket each day, cause it's just minor. Savings are savings...what are we talking about here?

 

Quote

b) no, Reject is not free! Building a good reject map takes time. I still remember that when I checked out RMB it could run for several hours on such large maps. I have no idea how it is today, but I'd say it's still several minutes to build a good one.

Reject *is* free for the 9999 out of 10000 people playing the map, excluding the map author. The same could be said for monster placement: "Just put all the monsters in one room, cause monster placement takes time."

 

Quote

And what if you use a modern port that doesn't load an empty reject and just skips the check instead? Suddenly that 'free' table results in some avoidable CPU cache pollution and might actually SLOW THINGS DOWN, because that map can become quite large with rather scattered access patterns.

This statement is completely true, as I outlined in my previous post. Cache pollution does slow things down, and can be avoided by any port that skips reading the REJECT table.

But, what port does that? I guess someone could make one. That port would fail vanilla demo sync, and would not render any REJECT lump tricks, but, yeah, it can be done. Don't know of any, but I can't say I researched them all...

 

For all other ports, the REJECT table exists, either as a properly-built lump, a zeroed-lump, or just in memory. Regardless, the cache performance issues are identical across all these scenarios, so, you might as well get some time savings out of the deal.

 

Quote

So it's not as clear cut as you make it out. That's only the case when running an engine that doesn't change how it was used originally and on hardware that does obey the same principles as what was considered state of the art in 1993. That cache thing is a real concern. Today it may actually be beneficial to have a slightly less efficient algorithm if it results in less hits of uncached memory.

It is *exactly* as I described - I didn't leave anything out. I didn't obfuscate anything, or manipulate the truth. As discussed twice now, yes, with the machines of today, the random access memory read patterns that occur when reading the REJECT lump can cause cache misses, which can ironically have an adverse effect on performance.

 

I wonder how bad the reading of the REJECT data hurts the cache vs. how bad the reading of level data hurts the cache, when doing an actual sight-check that could have been avoided.

 

So, yes, I agree with you that some other algorithm may increase sight-check performance. But not without the demo-sync issues, and not without losing the rare manual REJECT hacks offered by tools such as RMB.

 

But, again, for present ports, for currently-known algorithms, the REJECT table is available and ready to provide a mild to significant performance increase for sector and monster-laden maps. Your port will suffer a certain amount of performance penalty, regardless of whether or not you're gaining any advantage from having a populated REJECT table...

 

...so, it's silly to not take advantage of this lump. It's like leaving a dollar bill on the ground because:

a) it hurts to bend down to pick it up

b) it's not a $20

 

Frankly, I'm tired of storing 2 copies of most WADs: The original that I playback demos with, and the fixed version I play.

 

EDIT: More info on the caching:

Doom does read the REJECT data in the most-cache-unfriendly direction (Seer-Major vs. Seen-Major). Because the thing seen most is the player, it would be better for this to be reversed. Now, because the matrix is supposed to be reversible, you *could* reverse this order in the code, and in the vast majority of WADs it would probably work, demo-sync and all. But, I'm sure there are some REJECT lumps which are not exactly equal when read 90 degrees out.

 

But, nothing's stopping anyone from rotating the table in memory before first use, and rotating the check as well. *That* REJECT table would benefit a lot from having parts of it sitting in cache lines, as different monsters look for the player.

 

As yet another option, modern CPUs let you read from main memory without initiating a burst-cache read, if you use a special read instruction. The read itself is, of course, slow, but you don't evict other cache data, which basically removes any cache-based penalty from having a REJECT table.

 

So, all in all, my advice is to do your players a favor, and always build a REJECT table before final release if you can, for all maps with monsters, and especially before recording demos on such maps. It can't hurt, and except for the most simple of cases, it will always help to some degree (unless you're playing some mythical port that *incorrectly* doesn't check the REJECT table).

Edited August 30, 2018 by kb1 : clarity, and more info

Tell me, what kind of computer are you using?

 

Because where I live, I have to take out some huge UDMF-monstrosities for any of this to have even the slightest bit of relevance - and for those I cannot build REJECT anyway - not that it'd help, because GZDoom dumps REJECT entirely if there's portals in the map, which these days seems to be normal for port-specific projects.

 

But when it comes to anything vanilla or even most Boom stuff I have to look hard to find anything that ever dips below 60 fps with either GZDoom or PrBoom+ - let alone 35 fps - on playsim related load. 99% of the time it's the rendering that causes frame rate drops.

To be honest, I have a hard time finding maps not making use of extensive DECORATE where the entire per-tic think time exceeds 2 ms per frame in GZDoom, which unfortunately is the only port capable of displaying such metrics - I'd love to see numbers from a simpler ports for comparing.

 

5 hours ago, kb1 said:

This "analysis" seemed to be based on a single 600-monster map of a certain size, with a REJECT map of unknown population, using a modified Heretic sight-check algorithm. That's not much of an analysis.

 

Have you even read the linked post? I'll help you out - here's the relevant quote:

 

That was done with Doom's original code. ZDoom uses Hexen's sight checking code which is significantly faster so the gain by a valid REJECT lump is even reduced more.

 

We should also note that this analysis was lacking one crucial component: 5% of how much? If it was 5% of 20ms it would be relevant, but this is unlikely. If it was 5% of 8 ms, which I'd consider reasonable for a system of this age, we'd end up at 0.4 ms sight checking time, and a saving potential of roughly 0.2 ms

 

So with the Hexen algorithm the sight checking time will probably be reduced to less than half because from what I read it scales a lot better to large maps.

And as of now, all the 3 major ports (i.e. PrBoom+, GZDoom and Eternity), which are the only ones relevant when talking about strong limit removal are actually using a variation of the Hexen algorithm for sight checking.

 

And my bad for not testing myself, but I rectified that mistake now.

So trying to get some real numbers? It's hard because most maps I've thrown at GZDoom's "stat sight" report an average time of 0.0 or 0.1ms per tic for sight checking i.e. it's merely background noise. Even the largest map out there - ZDCMP2 - reports an average of 0.2 ms on my laptop equipped with a 3 year old 2.5 GHz Core i5 Dual Core, which is as run-of-the-mill as CPUs come these days. Remember, that is total sight checking time, not just the checks that would get shortcut by REJECT.

So it's these 0.2 ms that can theoretically be saved by REJECT. Let's say we have a success rate of 50% and we end up at 0.1 ms. On the largest map that ever was made - and a map with mostly contained areas where REJECT would really be beneficial it amounts to close than nothing!

 

Wanna talk about the 'classic' sight check algorithm? Ok, but then we have to get down to maps of the size of Vrack3, maybe. Currently I cannot measure sight check on this because it just reports 0.0, so even with an old claim that on such large maps the BSP based check was 5 times slower it'd still be 5x background noise.

I can also run this map with software rendering at a constant 60 fps on the same system at full HD resolution so what does it matter anyway? This is one of the most detailed Boom maps out there and it hardly breaks a sweat on any account.

 

There's also no need to test wide open maps like Frozen Time. These may put the renderer to its limits but are a textbook case for bad REJECT performance because it'd mostly be 0s with that area where every sector can see into every other one.

 

So you can say what you want. If you save 50% of nothing you end up with no improvement. The time spent in sight checking is totally irrelevant in the big picture when everything, including high resolution rendering gets factored in.

 

In case you are still not convinced and point to older, slower computers - sure, they still exist in smaller numbers, but they'd have far more trouble keeping up with all the rest of the needed work and the sight checking part would still be irrelevant. Thinking of it, it was irrelevant 14 years ago and since then computers only have gotten faster. So in the end you can twist it and turn it as much as you want - the simple matter of fact is that the time saving potential here is not high enough to warrant the hassle of spending several minutes building a large resource. This only scales if you run on an old clunker system that should have been retired 10 years ago.

 

 

@Jerry.C

The entire purpose of my posts is to respectfully help the OP make good decisions about map making. My conclusions are backed by a whole lot of personal observation and experience, with results that can be repeated at will. I don't have to read a bunch of posts to know what I have experienced and seen with my own eyes. "Oh, Jerry.C said it...I must have be hallucinating?" Are you kidding me?

 

Your entire purpose seems to be to fight the science, and to promote the abandonment of a technique designed to improve gameplay performance. And you are going about it in an abusive sarcastic manner.

 

You claim not to be able to find a scenario where a game dips below 60 fps (even though I described such a scenario.) Then, you mention 2 ms./tic think time, as if that's nothing. 28ms./tic is all you have. Divide that again if you're trying to run interpolated. Subtract enough time for 12 network packets, if you're running 4-player P2P co-op. Subtract rendering time. Subtract sound updates. Subtract other overhead. Even if 2 ms. was some fixed constant (it's not), that's a lot of time in a real-time game. *Any* time savings help. Now, yes, as a port developer, it is your job to balance optimization against other factors, like code manageability. And, you should make sure that your optimizations actually do provide a worth-while optimization.

 

You proceed to report some metric that's telling you the entire sight-check time, inclusive of REJECT checks. And then you add some completely fabricated "let's divide that by 2, and suggest that that is what the REJECT lump is doing for us...saving 0.1 ms". Maybe sight-check time was 60 seconds without REJECT, and REJECT saved you 59.9999 seconds? Ever consider that?

 

Your test summarizes as follows: "This big-ass BOOM map gets tons of fps, so REJECT is insignificant." There's no science there at all. Who cares when you're getting 100 fps? Optimizations matter when you're otherwise running too slow, not when you've got time to spare. 

 

You throw in some unfriendly, sarcastic, and frankly abusive "let me help you read...say what you want...if you're not convinced..." jazz. I never stooped to using any of those tactics on you.

 

You round out your post with assumptions on which computers people are using, and your personal opinions on how many savings are worth it, and when people should throw out their computers.

 

But why? It seems very important to you, to ensure that people do not use every tool available to them.

 

You included lots of numbers in this latest post, none of which paint enough of a picture to make any sort of determination of what is happening:

• How many sight checks were performed?
• How many hit vs. missed?
• How many used the REJECT table to miss?
• How many blocks had to be traversed?
• What was the state of the cache before, during, and after these sight checks?
• What average frame rates were you getting during these tests?

If you have meaningful results to report, and you are honestly trying to guide people towards making decisions that will help enrich their lives, then by all means please do so. But if your goal is to play Debate Club, go play with someone who doesn't know better.

 

Until you experience the situation I wrote about above (which I quote below), you will not understand:

"...yes, proper REJECT lump builds made significant performance improvements (in our co-op games, over empty REJECT builds)."

 

On a lot of maps, my gameplay experience was significantly improved after the proper building of the REJECT table. There's simply no denying that. Sure, I can pull out my little calipers and my stopwatch and measure this or that. Or, I can run the tool, and play a smoother game and have a good time.

 

What in the world are you fighting about? Better is better. Everyone else knows what I'm talking about. Make your maps as slow as you want them - no one's stopping you.

 

@Capellan From what I remember, DoomBuilder and friends have config files that support the use of different nodebuilders. These files are text files, and they can be modified, and new files can be added to support other nodebuilders. DoomBuilder generates a DOS command using a combination of components of the config file, and of internal variables (like map filename). This is combined into a DOS command which is called to initiate the build. DoomBuilder monitors the output of these tools to know when the build succeeds or fails.

 

The point is, with some work, you should be able to make a new config that allows the output of one tool to feed the input of another tool, to provide a multi-step build, if desired. For example, one tool could build the nodes, and another tool the REJECT lump. This should be do-able. I can't instruct you how to go about it at this time, but the config files should hint at what the proper commands look like. Good luck!

 

5 hours ago, kb1 said:

@Jerry.C

The entire purpose of my posts is to respectfully help the OP make good decisions about map making. My conclusions are backed by a whole lot of personal observation and experience, with results that can be repeated at will. I don't have to read a bunch of posts to know what I have experienced and seen with my own eyes. "Oh, Jerry.C said it...I must have be hallucinating?" Are you kidding me?

 

So what you have seen and what I have seen is not the same.

What I have seen is that I can run nearly every Boom compatible map out there without any sort of performance issues on a less than optimal system. So your supposed "performance improvement" doesn't really improve things at all on any reasonable computer these days.

It apparently didn't even improve things 14 years ago - too bad that I do not own such an old computer to prove it.

 

 

5 hours ago, kb1 said:

 

Your entire purpose seems to be to fight the science, and to promote the abandonment of a technique designed to improve gameplay performance. And you are going about it in an abusive sarcastic manner.

 

I see how you are trying to play this. You set yourself up as the all-knowing expert, you talk a lot but say little, but if someone posts actual numbers they are just summarily dismissed because they do not match what you claim.

So, if you want to keep up the impression that REJECT actually is beneficial, post something that proves it and not lengthy walls of text where the lack of information is hidden in the mass of words.

 

 

5 hours ago, kb1 said:

 

You claim not to be able to find a scenario where a game dips below 60 fps (even though I described such a scenario.) Then, you mention 2 ms./tic think time, as if that's nothing. 28ms./tic is all you have. Divide that again if you're trying to run interpolated. Subtract enough time for 12 network packets, if you're running 4-player P2P co-op. Subtract rendering time. Subtract sound updates. Subtract other overhead. Even if 2 ms. was some fixed constant (it's not), that's a lot of time in a real-time game. *Any* time savings help. Now, yes, as a port developer, it is your job to balance optimization against other factors, like code manageability. And, you should make sure that your optimizations actually do provide a worth-while optimization.

 

It's clear that you do not follow through with the math. If you have 2 ms of think time on an average computer you have 4 ms of think time on a low end computer - or if you get even lower you end up on a system where the map cannot be played and no magic is going to help.

And you keep talking about 28 ms - as if that had any relevance these days! The magic number is not 28 ms but 16 ms for 60 fps interpolated gameplay.

Also, think time is not sight checking time - it is running the entire monster AI and movement logic as well. Sight checking is only a minor part of it - if you read my previous post you'll see that even on the largest map I tested I was unable to get GZDoom to something higher than 0.2 ms. Yes: 0.2 ms! That's the entire time spent in the sight checking code, and this is the entire time that can be saved by providing a REJECT table - but even here you still have to factor in the effect on the CPU cache. This map has roughly 16000 sectors, meaning a REJECT table of 32 megabytes! Fortunately the ZDoom family of ports is smart enough to add a simple check to the sight checking code that only accesses REJECT if it is there, so these 32 MB never hit the cache. I wonder what would be worse: Not being able to shortcut the sight checks or the constant scattered hits on this large block of memory that would otherwise pollute the CPU cache like crazy. Unfortunately I cannot test it because nobody has written an UDMF-capable REJECT builder.

 

Speaking of smaller maps, if the sight checking hovers far below 0.1 ms per frame, there's really no need to investigate. If you start with nothing all you can save is nothing. And speaking of other ports, unfortunately they lack the user-side profiling capabilities so I'm stuck with GZDoom for such measurements.

 

 

5 hours ago, kb1 said:

 

You proceed to report some metric that's telling you the entire sight-check time, inclusive of REJECT checks. And then you add some completely fabricated "let's divide that by 2, and suggest that that is what the REJECT lump is doing for us...saving 0.1 ms". Maybe sight-check time was 60 seconds without REJECT, and REJECT saved you 59.9999 seconds? Ever consider that?

 

Yes, I considered that, but it's ludicrous outside of fabricated test cases. But let's assume this could happen. Even then you couldn't save more time than was actually spent - which for the largest map around was 0.2 ms! If you spend 0.2 ms in a certain section of code, no magic in the world will save you more than those 0.2 ms. On top of that you also need some basic understanding what causes sight checks to be performed. Most are done by active monsters near the player. For which REJECT naturally does not shortcut.

 

 

 

5 hours ago, kb1 said:

 

Your test summarizes as follows: "This big-ass BOOM map gets tons of fps, so REJECT is insignificant." There's no science there at all. Who cares when you're getting 100 fps? Optimizations matter when you're otherwise running too slow, not when you've got time to spare. 

 

Indeed. The point I was trying to make is that you really need some very, very old and very, verx weak computer for this to become relevant. and that very, very old and very, very weak computer will not only increase its sight checking time but everything so in the end the relation of stuff is still the same, i.e. very low.

 

Let's be clear: REJECT made a lot of sense in 1993 when an average computer had trouble rendering 320x200 at a stable 35 fps. In that scenario saving a handful of percent was extremely important and every cycle indeed mattered. But you can twist it like you want: My first Pentium 90 already played the largest maps of the early years at stable 35 fps - none of which actually contained a REJECT table, save a very few exceptions - and over the years processing power increased a lot more than Doom map complexity so we never ever got close again to the cost-benefit-ratio REJECT provided initially, mainly because the advent of higher resolutions put increasingly more time on the renderer side than the gameplay side.

 

 

5 hours ago, kb1 said:

 

You round out your post with assumptions on which computers people are using, and your personal opinions on how many savings are worth it, and when people should throw out their computers.

 

Those assumptions are based on real-world numbers from various sources, among them Steam's hardware survey or GZDoom's own survey or what can generally be read in the tech press. All those come back the same: It is a vanishingly small minority who still use computers that may fall victim to what you claim here.

 

 

5 hours ago, kb1 said:

 

But why? It seems very important to you, to ensure that people do not use every tool available to them.

 

You included lots of numbers in this latest post, none of which paint enough of a picture to make any sort of determination of what is happening:

• How many sight checks were performed?
• How many hit vs. missed?
• How many used the REJECT table to miss?
• How many blocks had to be traversed?
• What was the state of the cache before, during, and after these sight checks?
• What average frame rates were you getting during these tests?

 

All that would matter if the time was in any way significant.

One thing I learned about optimizing is to optimize where it actually matters. It makes no sense doing fine grained performance analysis where some rough measurement clearly shows that it's insignificant. And the rough checks I did yesterday clearly show that it is indeed insignificant. If I cannot manage to get this part of code to spend more than 0.2 ms per frame, which is 1.25% of the critical time interval of 16 ms. Optimizing this code cannot save more than 1.25% even in the best case scenario. And even then it would only be relevant if the total frame time spent would be slightly below those critical 16 ms.

No matter what, if a map runs bad on a system, you won't make it be able to run well by shaving off one or two percent of execution time. If you are in such dire straits you have to make bigger compromises, like reducing the screen resolution to save time on the rendering.

 

 

5 hours ago, kb1 said:

If you have meaningful results to report, and you are honestly trying to guide people towards making decisions that will help enrich their lives, then by all means please do so. But if your goal is to play Debate Club, go play with someone who doesn't know better.

 

Unlike you I have meaningful results. I already posted them and they clearly show that even on extremely large maps the time being spent in the code that can be shortcut is mostly insignficant.

Which is very much unlike this:

 

5 hours ago, kb1 said:

 

Until you experience the situation I wrote about above (which I quote below), you will not understand:

"...yes, proper REJECT lump builds made significant performance improvements (in our co-op games, over empty REJECT builds)."

 

If you make such claims, where's the map being used, the port being used and the hardware specs? You accuse others of being unspecific and here we got a random claim that isn't backed by any information whatsoever. I absolutely cannot respond to it because I know zip about the scenario. I totally have to trust that you were right in your assessment of the situation.

So please, practice what you preach and back your claim with some actual numbers! In particular, that it is indeed REJECT that made all the difference and not some random occurence with the network that just happened to coincide with the state of REJECT.

 

 

5 hours ago, kb1 said:

 

On a lot of maps, my gameplay experience was significantly improved after the proper building of the REJECT table. There's simply no denying that. Sure, I can pull out my little calipers and my stopwatch and measure this or that. Or, I can run the tool, and play a smoother game and have a good time.

 

There's no denying this? Tell me, why do I have to trust your empirical "proof" that apparently just comes from subjective observation? Again:Post the numbers!

Without numbers to support it, all you make is a completely unsubstantiated claim about your personal impression but it's backed by nothing.

 

 

 

5 hours ago, kb1 said:

 

What in the world are you fighting about? Better is better. Everyone else knows what I'm talking about. Make your maps as slow as you want them - no one's stopping you.

 

I am not fighting anything. But what I do not like is that you make some broad claims about performance improvements but have nothing whatsoever to show. You never made a techical analysis, all you said is based on personal impressions and we are forced to trust your personal impressions.

Even Graf's 14 year old post went deeper with that. He said "5% on a large map of its time spent on sight checking", that's at least something to start from and it shows that he actually took the time to run the code through some profiling.

 

So for the third time in a row: Post some technical proof that REJECT really helps!

 

 

 

 

 

Glad someone finally said it. :)

 

Btw hi graf. 

Jerry.C: I have no interest in proving anyone wrong - that's your thing. I was just offering advice based on my experience.

 

If you truly have 4 computers (need 4 for co-op) that can play *any* non-UDMF map without dipping below 60 fps (added blood and gore effects, blood and blast wall decals, thousands of monsters, rain and fog effects, 3d floors, whatever), put them on eBay - I'll give you $10k for each of them.

 

For the rest of us not blessed with super-computers, maintaining at least 35 fps is paramount for avoiding lag in a multiplayer environment. In these frequent scenarios, any technology that helps maintain this minimal performance requirement is welcome in my book. REJECT has proven to be such technology.

 

It's amazing: Go back and read how you describe me: "I see how you are trying to play this. You set yourself up as the all-knowing expert". I did none of that. Again, Debate Club 101: When your science fails, attack the people.

 

You say "if someone posts actual numbers they are just summarily dismissed because they do not match what you claim." I agreed with the numbers you posted, and your notes about cache pollution. My goodness!

 

Here's the kicker:

"Tell me, why do I have to trust your empirical "proof" that apparently just comes from subjective observation?"

 

and:

"...you make some broad claims about performance improvements but have nothing whatsoever to show. You never made a techical analysis, all you said is based on personal impressions and we are forced to trust your personal impressions."

 

Actually, yeah, as I said, I did do technical analysis - lot's of it. And I also did lots of less technical observations.

 

Why do you have to trust my empirical proof? You don't. You're not forced to trust anything.

 

So now, the issue is that I might be lying? For what purpose, actually? "Yes, my master plan unfolds! All of DoomWorld must build REJECT maps! Doom gameplay will be forever...faster? My stock in hard drive manufacturers will skyrocket!! People of all walks of life will remember me as the guy who convinced the world to use...a lump designed (by Carmack, no less) to do exactly what my experience says!" You got me, man, my secret is out.

 

Are all suggestions made to you met with such utter disbelief?

 

Friend: "Hey, Jerry.C, let's go to the this new pizza place - I was there last week, and it was awesome!"

Jerry.C: "I see how you are trying to play this. You set yourself up as a pizza expert. why do I have to trust your empirical "proof" that apparently just comes from subjective observation?"

 

Sounds like a lonely existence.

 

So, now, instead of offering some friendly advice based on experience, you expect me to stop what I'm doing, and set up and run a set of test scenarios, chart all the results, and submit them to you (numbers which you probably won't believe anyway), before I am allowed to share my experience with Capellan and offer some advice?  Eat shit.

 

19 hours ago, jmickle66666666 said:

just use your actual account, graf. hiding behind a fake one is tedious, and you're really bad at it

Are you sure?

I gotta ask: Did I act appropriately here? I mean, I have had a few situations come up like this before, in which I am trying to offer some friendly advice, where things escalate to the point where I am being accused of some devious act...I swear I don't understand what motivations are at work here. It often ends up with threads being closed, posts deleted, or some other less-than-desirable outcome, where I'm left wondering what happened, and, more importantly, not knowing what I could have done to prevent it, other than allowing myself to be dishonored, and, worse, allowing non-truths to proliferate towards my friends.

 

I provided the results of an observation I've made in the past, and I'm being asked to prove it, not out of curiosity, but as if I'm some common liar. I don't appreciate the insinuation.

 

Sure, if and when I happen to be in a position where I am studying the effects of the REJECT lump, and I have collected good, comparable data, I will be more than happy to publish it, along with any conclusions I may arrive at. I find such studies interesting, as I imagine others would as well. But I'll be damned if I'm going to feel like I need to prove everything I say, before I say it. "I like Doom. Here's my receipts..."

 

So, good forum readers of Doomworld - I have two questions:

1. What motivates people to act like this? I just don't experience ordeals like this, outside of a forum. This really sucks, since I am otherwise very fond of the Doomworld forums, and especially the people here.
2. How should I handle confrontations like this in the future? I do not want to pollute threads with this type of argumental garbage, as it can turn from being interesting to downright disrespectful. I try to respond positively, respectfully, and truthfully to all questions I have some knowledge on, and I hope to be treated with the same courtesy. Yet, sometimes it takes a few rounds of posts to realize that the tone has taken a different direction, and by then, there's usually a wall of text with unanswered questions. It takes a long time to try to respond clearly and accurately. It is discouraging to have that effort "rewarded" with sarcasm, condescension, and plain rudeness.

 

Does anyone have any advice on how to handle such confrontations in a cleaner, quicker, more-respectful way? Thanks in advance, and for your time.

 

(Sorry to Capellan, for carrying-on, while trying to answer your questions).

8 hours ago, kb1 said:

Yet, sometimes it takes a few rounds of posts to realize that the tone has taken a different direction, and by then, there's usually a wall of text with unanswered questions. It takes a long time to try to respond clearly and accurately. It is discouraging to have that effort "rewarded" with sarcasm, condescension, and plain rudeness.

 

Then don't reply to posts like that. Every time this happens you paint yourself as a martyr who is just here to talk about Doom and who is completely randomly confronted with "a wall of text" and hostile people starting arguments. But of course the only thing connecting all these incidents is you. You post your own pedantic walls of text and then when people want to argue with points in it, or don't want the thread to get derailed, you post 10 more walls of text arguing why your first wall of text was appropriate. Just don't post them! Don't get drawn into these arguments! If people dont like what you have to say, leave the thread! It's not hard!

You're right, of course. I should be able to state my peace once, and have faith that people will know that I'm motivated by honesty. Just like your reply - honest, and to the point. without throwing any punches. This is a lesson I struggle with a lot. Thanks.