NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

port-alpha/52520: Wrong double to uint64_t conversion

>Number:         52520
>Category:       port-alpha
>Synopsis:       Wrong double to uint64_t conversion
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    port-alpha-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Sep 02 00:40:00 +0000 2017
>Originator:     coypu
>Release:        NetBSD 8.99.2
NetBSD 8.99.2 NetBSD 8.99.2 (GENERIC-$Revision: 1.376 $) #2: Mon Aug 28 13:00:23 CEST 2017 alpha

the following test case asserts if built with -mieee, but not without:

#include <assert.h>
#include <inttypes.h>
#include <stdint.h>

main (void)
        long double even = 9223372036854775808.000000; /* 2^63 */
        uint64_t unsigned_even = even;

        assert(unsigned_even % 2 == 0);

        return 0;

This took a while to make sense of, and appears to be a kernel bug.
Here is my reasoning to help a non-expert along:

2^63: valid float, valid uint64_t, invalid int64_t.

  Because we are building with -mieee, it is also effectively built with -mfp-trap-mode=su

  The code generated is the following:
    cvttq/svc  $f10,$f11

  According to the alpha manual, "V" means that we trap on integer overflow.
  CVTTQ probably thinks in terms of signed integers, so (I think) unsigned 2^63 traps.

  The kernel must complete it, but it does so incorrectly.   We get unsigned_even == 2^63 - 1.

note this bug came from MPFR's testsuite. other things may fail too.


Home | Main Index | Thread Index | Old Index