Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Quasar

I know floats are inexact but damn...

Recommended Posts

If EE is compiled using GCC 4.2.1.20070719 on platform i386-unknown-openbsd, the following check fails:

if(vbscreen.getVirtualAspectRatio() <= 4.0/3.0)
   return;
With the exception of modes 320x200 and 640x400, which are treated specially, the function in question basically does this:
   return static_cast<double>(width) / height;
Regardless of the input screen dimensions, if the aspect ratio is 4/3, such as in modes 640x480 or 1024x768, the comparison always fails, leading to a frequent crash and other side effects.

I have to believe this isn't just floating point inaccuracy, because the value in question 4/3 is, in ideal real math, 1.3333... - this should not be getting rounded UP at all, rather if anything I'd expect the binary float truncation to have a value less than real 4/3. Either the folded constant value in the EXE is less than 4/3 and the function is accurate, or the constant is correct and the function returns a value > 4/3.

Anybody else ever ran into this?

Share this post


Link to post

Can you post the body of getVirtualAspectRatio() ? If you are truly returning double(width=640) / height=480 then I would certainly be worried. Though, round up can happen, particularly when a double for such a number as 1.333... is truncated to a float. So I would add an epsilon, or change your representation (seems too error prone and may lead to subtle (or not) bugs).

Share this post


Link to post

Hint: Never EVER compare floats for equality. Since by their very nature some values will get rounded you'll always get into situations where such a comparison will fail.


4/3 can neither be represented exactly as a decimal nor a binary value.

Share this post


Link to post

Is there a common practice for float comparison margin of error, or is it a case-by-case situation?

Share this post


Link to post

Would adding 1e-x (where x is an exponent) to 4.0/3.0 fix the problem? Which is the best x for double comparison margin of error?

Share this post


Link to post
marineController said:

Maybe Dooms fixed point can help you: http://pastebin.com/0NnHAxDp



No, not at all. Imprecisions come from numbers not being representable as binary values. Fixing the number of digits doesn't change anything about this problem. Besides, there is no case where 32 bit fixed point can be more precise than a 64 bit double.

Share this post


Link to post

If you want to test such things, try multiplications instead, they will be exact.

Share this post


Link to post
Graf Zahl said:

No, not at all. Imprecisions come from numbers not being representable as binary values. Fixing the number of digits doesn't change anything about this problem. Besides, there is no case where 32 bit fixed point can be more precise than a 64 bit double.

Yes it does, because integer division doesn't round, it chops.

4 * FRACUNIT / 3 will *always* be equal to any other integer division of numbers with the same ratio. 640 * FRACUNIT / 480 will give the same result. Try it yourself if you don't buy the chart he posted. Either way, I've already replaced the floating point aspect ratio methods with fixed-point ones, resulting in this problem being repaired.

Integer math also isn't subject to the "false precision" problem, which seems to be the cause of this particular failure. 4/3 stored in the executable has the form of a 64-bit serialized IEEE-754 double precision number, but 4/3 calculated at runtime on an Intel has 80 bits of precision. The calculated value always compares as greater than the constant simply because it has more bits set (somewhat meaninglessly) to 1.

Share this post


Link to post
printz said:

So if you have 801x600, it will register as widescreen? :-/

I already told you we don't support use of weird modes like that, just because it's technically possible. So yes, it will.

Share this post


Link to post

I'd be interested to hear why (not that I'm doubting you have a valid reason).

Share this post


Link to post
ducon said:

If you want to test such things, try multiplications instead, they will be exact.


This.

Instead of testing a/b <= c/d as floats, test a*d <= c*b as ints. Note that if for some reason one of the denominators is negative, you need to flip the inequality.

Share this post


Link to post
DaniJ said:

I'd be interested to hear why (not that I'm doubting you have a valid reason).

Well, where in the region of deformed screen shapes between 4:3 and 16:10 do I declare the boundary to have been crossed into "widescreen" territory? I could always set the boundary at 16:10 or wider, I suppose, but then somebody will complain no doubt about their 16:11 custom windowed resolution not being treated as widescreen.

It's kind of arbitrary.

Share this post


Link to post

I'm not sure I understand the desire to classify an aspect ratio as "widescreen" or not. As you say, its rather arbitrary. In Doomsday we don't classify, all we care about is whether the aspect ratio is 4:3 and/or whether to stretch the render or not.

Share this post


Link to post

The pixel buffer resolution tells you the aspect ratio? I thought it was the actual physical size of the viewing surface.

It would be best to have an option that says you want either 4:3 or whatever else. I know my Wii does this, even though the analog component cables are naturally 4:3, you can view it like it was 16:9.

Share this post


Link to post
GhostlyDeath said:

It would be best to have an option that says you want either 4:3 or whatever else. I know my Wii does this, even though the analog component cables are naturally 4:3, you can view it like it was 16:9.

While I do recommend that any source port supporting widescreen/tallscreen to have anamorphic options. It is still recommended to detect automatically based on the virtual aspect ratio. This is slightly harder than just comparing the ratio between the width and height since some widescreen monitors are not exactly 16:9, 16:10, (or recently somewhere around 17:10). Having to manually select is the only option on TVs since the virtual aspect ratio isn't the same as the physical (IIRC it's not even 4:3).

Here's an ACS approximation of the function ZDoom uses to determine the ratio if it's of any use. (Although it looks like that page still needs to be updated for 17:10.)

Share this post


Link to post

Just guessing, but the 4.0/3.0 could have been rounded by the compiler because it was not told to use (double).
Rounding is likely because the floating point unit has more register bits than are stored in memory.
The compiler must store the 4/3 constant, so it would round the low bits to 000.
The function result would still be in the floating point registers with all its digits.
The double function result would continue with more digits, so the double function result is larger.

Edit: This is the same thing that Quasar was saying, but I did not read it that way the first time.

if( fabs( vbscreen.getVirtualAspectRatio() - (4.0/3.0)) < 0.01 )
return;

Share this post


Link to post
Quasar said:

4/3 stored in the executable has the form of a 64-bit serialized IEEE-754 double precision number, but 4/3 calculated at runtime on an Intel has 80 bits of precision. The calculated value always compares as greater than the constant simply because it has more bits set (somewhat meaninglessly) to 1.


That is fucking madness. I thought IEEE-754 was supposed to fix all that, but I guess not.

Some other people have asked if there's like an equivalence tolerance, like if it's within .00001 difference then the numbers are equal. There is, it's called an epsilon value, and it's a big pain in the ass. You may as well just typedef a struct with the whole & fraction parts separate.... oh hey fixed point....

Share this post


Link to post
Ladna said:

There is, it's called an epsilon value, and it's a big pain in the ass.

Well, in lack of time or in lack of need, I can just go with a gross value.

Share this post


Link to post
Blzut3 said:

Here's an ACS approximation of the function ZDoom uses to determine the ratio if it's of any use. (Although it looks like that page still needs to be updated for 17:10.)


Does ZDoom only ever display at 4:3, 5:4, 16:9, 16:10 or perhaps 17:10 in spite of whatever resolution you've set? Surely this doesn't work for the hypothetical bizarre windowed modes.

edit: perhaps to answer my own question, it seems ZDoom doesn't allow bizarre resolutions at all, even in Windowed modes.

Share this post


Link to post

It might clear up some issues in this thread if I explain that the code in question is to determine whether or not screen patches will be drawn onto a centered 4:3 subscreen of a widescreen video buffer.

It has *nothing* to do with the first-person renderer.

Share this post


Link to post
Siggi said:

Does ZDoom only ever display at 4:3, 5:4, 16:9, 16:10 or perhaps 17:10 in spite of whatever resolution you've set? Surely this doesn't work for the hypothetical bizarre windowed modes.

Yes, ZDoom will only display within a predefined set of aspect ratios. This does work out somewhat better that allowing bizarre ratios since mod authors can design UI elements for the supported ratio. (Even adding 17:10 for netbooks resulting in a large number of groans.)

Technically there's nothing stopping ZDoom from rendering at an odd resolution, but it will be corrected as if it was 4:3. Unless it's close enough to one of the supported wide/tallscreen ratios. (Note that there are a lot of wide screen resolutions that are not exactly 16:9 or 16:10, but only off by a few pixels. Likewise 1024x600 isn't exactly 17:10, but it's a close enough approximation.)

Share this post


Link to post
Quasar said:

4/3 stored in the executable has the form of a 64-bit serialized IEEE-754 double precision number, but 4/3 calculated at runtime on an Intel has 80 bits of precision. The calculated value always compares as greater than the constant simply because it has more bits set (somewhat meaninglessly) to 1.

To avoid this problem, you can use _FPU_GETCW/FPU_SETCW on GCC to set the number of bits of precision to run the FPU registers at. Visual C++ has _control87, but its runtime already sets the FPU to use double-sized registers, so you don't need to use it for that compiler.

Share this post


Link to post
Moriarti said:

To avoid this problem, you can use _FPU_GETCW/FPU_SETCW on GCC to set the number of bits of precision to run the FPU registers at.

You need to #include <fpu_control.h> for these, and only apply to the x86 architecture.

However it is still best to use epsilons when comparing floating point values.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×