Bad sector border rendering

[Blastfrog]

This isn't about editing, and although it's engine related, it isn't about any specific engine, hence why I'm posting it here in general rather than source ports.

I've noticed when playing PrBoom+ that when I walk over some sector boundaries, it pops outward slightly (to my left if on the right and vice-versa) as the texture above/beneath is distorted. I've also noticed this in the 0.4 alpha of vanilla Doom. What is causing this?

[bradharding]

Sodaholic said:

This isn't about editing, and although it's engine related, it isn't about any specific engine, hence why I'm posting it here in general rather than source ports.

I've noticed when playing PrBoom+ that when I walk over some sector boundaries, it pops outward slightly (to my left if on the right and vice-versa) as the texture above/beneath is distorted. I've also noticed this in the 0.4 alpha of vanilla Doom. What is causing this?

I believe this issue is in R_ScaleFromGlobalAngle().

fixed_t R_ScaleFromGlobalAngle (angle_t visangle)
{
    fixed_t		scale;
    angle_t		anglea;
    angle_t		angleb;
    int			sinea;
    int			sineb;
    fixed_t		num;
    int			den;

    anglea = ANG90 + (visangle-viewangle);
    angleb = ANG90 + (visangle-rw_normalangle);

    // both sines are allways positive
    sinea = finesine[anglea>>ANGLETOFINESHIFT];	
    sineb = finesine[angleb>>ANGLETOFINESHIFT];
    num = FixedMul(projection,sineb)<<detailshift;
    den = FixedMul(rw_distance,sinea);

    if (den > num>>16)
    {
	scale = FixedDiv (num, den);

	if (scale > 64*FRACUNIT)
	    scale = 64*FRACUNIT;
	else if (scale < 256)
	    scale = 256;
    }
    else
	scale = 64*FRACUNIT;

    return scale;
}

Increasing each 64 to 1024 appears to resolve the issue, which I've done so in DOOM RETRO.

[LogicDeLuxe]

Sodaholic said:

when I walk over some sector boundaries, it pops outward slightly (to my left if on the right and vice-versa)

The most extreme example of this can be seen in Dark Forces at any sector boundary.

[Dr. Sean]

bradharding said:

Increasing each 64 to 1024 appears to resolve the issue, which I've done so DOOM RETRO.

Doing so may possibly lead to integer overflows when the scale value returned by the function is used, and in particular, rendering when the player walks underneath 2s midtextures. In my experience, 256 was a better value than 1024. Eventually this issue was the catalyst prompting me to rewrite Odamex's renderer to eliminate angle-based scale calculations entirely.

[entryway]

Dr. Sean said:

Doing so may possibly lead to integer overflows when the scale value returned by the function is used, and in particular, rendering when the player walks underneath 2s midtextures. In my experience, 256 was a better value than 1024.

256 still causes visual issues sometimes, at least with prboom codebase
http://sourceforge.net/p/prboom-plus/bugs/209/