Subject: floating-point bugs in kernel, gcc?
To: None <mellon@toccata.fugue.com>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: port-pmax
Date: 06/21/1995 17:53:32
NetBSD/pmax's software floating-point doesn't handle gradual
underflow (denorms) properly.   Results that should be denorms
seem to be getting stuck at the minimum normalized positive
number.

	From an Ultrix ECOFF binary running on NetBSD/pmax:
	
	Diagnosis resumes after milestone Number 90          Page: 7
	
	Testing powers Z^i for small Integers Z and i.
	... no discrepancis found.
	
	Seeking Underflow thresholds UfThold and E0.
	
	FLAW:  Underflow can stick at an allegedly positive
	value PseudoZero that prints out as 2.22508e-308 .
	Since comparison denies Z = 0, evaluating (Z + Z) / Z should be safe.
	What the machine gets for (Z + Z) / Z is  2.00000000000000000e+00 .
	This is O.K., provided Over/Underflow has NOT just been signaled.
	Smallest strictly positive number found is E0 = 2.22507e-308 .
	Since comparison denies Z = 0, evaluating (Z + Z) / Z should be safe.
	What the machine gets for (Z + Z) / Z is  2.00000000000000000e+00 .
	This is O.K., provided Over/Underflow has NOT just been signaled.

This must  be a kernel software-denorm bug, since the Ultrix binary
running on Ultrix doesn't produce the error.


I suspect that this denorm glitch may be behind other buggy FP
behaviour, for example, as strtod.c breaking on NetBSD/pmax when
compiled with -O.  To reproduce this, compile and run enquire
from gcc 2.x.*.  (It goes into an infinte loop.)


NB: if I ``cross-compile'' the netBSD strtod.c on Ultrix with gcc
2.6.3, getting an ECOFF .o file (and also infinity.c, flt_rounds.c)
and link *those* objects into paranioa, I get an Ultrix binary that
works precisely as the Ultrix binary linked with native Ultrix
libraries does.  So it's unlikely to be just a gcc codegen bug.

Then again, it could  be a codegen bug in libm.a, or buggy
FP code in the gas or cc1...

Any bright ideas would be appreciated.  I don't have any more time
to chase this particular bug.

--Jonathan