Re: CVS commit: src/tests/lib/libm

[Tetsuya Isaki <isaki%pastel-flower.jp@localhost>]

At Sat, 24 Jan 2015 10:35:01 +0000,
David Laight wrote:
> > > > Log Message:
> > > > In the exp2_values test case, provide separate expected return values
> > > > for the float case, reflecting the actual exp2f() argument value after
> > > > rounding to float precision.  Fixes PR lib/49256.  Thanks to Makoto
> > > > Kamada and Tetsuya Isaki for the analysis.
> > > 
> > > The reason I left the tests failing is that the results should be more
> > > accurate than the values that are actually returned.
> > > 
> > > Changing the 'expected' values so that the tests pass is just wrong.
> ...
> > > I think the values ought to be accurate to one or two counts on the lsb
> > > of the mantissa.
> > 
> > It is not an acuuracy problem of result, is an implicit type cast
> > problem of argument.
> > 
> > exp2()'s argument is double and exp2f()'s is float.
> > 
> > As you know,
> > exp2(7.7) means exp2((double)7.7) and
> > exp2f(7.7) means exp2f((float)7.7).
> > 
> > (double)7.7 = 0x1.ecccccccccccdp+2 in "%a" format
> > (float)7.7  = 0x1.ecccccp+2 in "%a" format
> 
> I'd forgetten to allow for that difference, and the change didn't
> mention it either.
> I'd expect (double)exp2f(f) to differ from exp2((double)f) by only
> 1 or 2 counts in the mantissa. You definitely want to use the
> lattet calculation to get the check value.
> 
> It is 'interesting' that (float)7.7 gets truncated rather than
> rounded. I'd expect 7.7f to be 0x1eccccdp+2, which would bring
> the result into the old bounds.

"%a"'s mantissa for float is 24 bit long (6 digit hex), but
float's mantissa is 23 bit long.  Therefore the LSB of this
format is 'invalid' bit, not 'valid and zero'.

(On the other hand, double's mantissa is 52 bit and 13 digit
hex of "%a" format is also 54 bit.  double does not have such
a trailing invalid bit.)

7.7 is represented as 0x1.ecccccc...p+2.
In float, ULP=0 Guard=0 as below, so it is truncated.

 bit 17    20  21    24  25
  ... 1 1 0 0   1 1 0 0   1 1 0 0 ...
                    ^ ^   ^
                  ULP G   R S

In double, on the other hand, ULP=0 Guard=1 and Round=1, so
it is rounded up.

 bit 47    50  51    54  55
  ... 1 1 0 0   1 1 0 0   1 1 0 0 ...
                      ^   ^ ^
                     ULP  G R S
---
Tetsuya Isaki <isaki%pastel-flower.jp@localhost / isaki%NetBSD.org@localhost>