Re: lib/57250: dtoa mishandles infinite doubles on 32bit big endian machines

[David Holland <dholland-bugs%netbsd.org@localhost>]

The following reply was made to PR lib/57250; it has been noted by GNATS.

From: David Holland <dholland-bugs%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: lib/57250: dtoa mishandles infinite doubles on 32bit big endian
 machines
Date: Sun, 2 Apr 2023 00:35:34 +0000

 On Tue, Feb 28, 2023 at 03:05:00PM +0000, martin%NetBSD.org@localhost wrote:
  > compiling and invoking it with 1e309 shows strange output on 32bit big
  > endian machines:
  > 
  > test: errno: Result too large or too small
  > test: infinite
  > Infinity0000000000000000000000000000000000000000000000000000000[...]
  > 
  > while the expected output is:
  > 
  > [/tmp] martin@thirdstage > ./test 1e309
  > test: infinite
  > inf
  > [/tmp] martin@thirdstage > 
  > 
  > The failure happens at least on sparc, powerpc and evbearmv7hf-eb.
  > It works on 64bit big endian (sparc64) and all little endian machines I
  > tested.
 
 Tracing through all the baling wire, it seems that there are two
 different copies of the printf logic. One begins at line 1062 of
 vfwprintf.c; in this one we get "inf" if calling __ldtoa returns
 INT_MAX in expt and a string that doesn't begin with 'N'. The other
 begins at line 1174, and in that one we get "inf" if isinf() returns
 true.
 
 They're selected based on #ifdef WIDE_DOUBLE, which seems from the
 logic to be meant to be set if long double is different from double.
 Checking which copy of the code is in use is probably a good idea; I
 don't have any of the effected machines to try.
 
 It appears that the non-WIDE_DOUBLE code is lacking support for %a and
 %A, so printing with that is a quick way to check.
 
 With WIDE_DOUBLE the ldtoa code (ldtoa.c and gdtoa.c) seems to only be
 able to produce the string "Infinity" on its own, and tracing through
 all the magic number signalling that leads to producing INT_MAX it
 _looks_ correct, so getting a string of zeros after it suggests some
 other path.
 
 The most plausible hypothesis at the moment seems to be that the
 failing platforms are not using WIDE_DOUBLE and also isinf() is
 broken, so they go through the logic at line 1259 using the magic
 exponent return of 9999 and print a ton of zeros at line 1520. How
 many zeros are you getting?
 
 -- 
 David A. Holland
 dholland%netbsd.org@localhost