[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

**To**:**Rhialto <rhialto%falu.nl@localhost>****Subject**:**Re: long double losing mantissa bits****From**:**Taylor R Campbell <riastradh%NetBSD.org@localhost>**- Date: Tue, 19 Jan 2021 22:48:00 +0000

> Date: Tue, 19 Jan 2021 22:16:32 +0000 > From: Taylor R Campbell <riastradh%NetBSD.org@localhost> > > This is not an integer, so conversion to uint64_t rounds it to > nearest, with ties to even, so you get ufrac = 0x80000000000000 = > 36028797018963968 as shown in the result. Oops -- one more correction: conversion to uint64_t rounds toward zero, not to nearest. The result happens to coincide in this case. > It looks like you have an off-by-one error in your exponent handling. > If you want to scale the fractional part into an integer, you need to > multiply by 2^57, not by 2^56. Another possibility is that you expected round-to-odd or something instead of toward-zero or to-nearest -- or perhaps the code works as intended but the test cases are buggy.

**References**:**Re: long double losing mantissa bits***From:*Taylor R Campbell

- Prev by Date:
**Re: long double losing mantissa bits** - Next by Date:
**Re: long double losing mantissa bits** - Previous by Thread:
**Re: long double losing mantissa bits** - Next by Thread:
**Re: long double losing mantissa bits** - Indexes: