Port-amd64 archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Changing x87 precision to full 63bit as default



On 12 Nov, 2013, at 14:40 , tsugutomo.enami%jp.sony.com@localhost wrote:

> And result was
> % ./t
> 3002399751580332.0000000000000000, 3002399751580331.0000000000000000, 
> 1.0000000000000004
> % ./t setprec
> prec 3 -> 2
> 3002399751580332.0000000000000000, 3002399751580331.0000000000000000, 
> 1.0000000000000002
> 
> Compiling with -ffloat-store and/or -fexcess-precision=standard was
> some result.

Not that there doesn't have to be some solution for this issue,
and not that it is relevant, but I was curious about which of
those answers is in fact closer to the actual value of
3002399751580332/3002399751580331.  When I compute that with dc
carrying a bunch of extra digits what I get is

    $ dc
    32k
    3002399751580332 3002399751580331 / p
    1.00000000000000033306690738754692

which, assuming dc isn't broken, is actually closer to the
"wrong" answer of 1.0000000000000004 than it is to the "correct"
answer of 1.0000000000000002.  Of course the rounding of the
answers to 16 digits exaggerates the difference; the correct value
of the ratio is almost exactly in the middle of those two values,
but when I print them with many more digits the precise value
is still closer to the "wrong" answer (in the 23rd decimal place).

The definition of "correct" and "wrong" being applied in this case
is bizarre.

Dennis Ferguson


Home | Main Index | Thread Index | Old Index