[src/trunk]: src/lib/libc/rpc Avoid undefined behavior in the definition of L...

[kamil <kamil%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/9644e122c97d
branches:  trunk
changeset: 324890:9644e122c97d
user:      kamil <kamil%NetBSD.org@localhost>
date:      Wed Jul 25 23:59:08 2018 +0000

description:
Avoid undefined behavior in the definition of LAST_FRAG in xdr_rec.c

Do not change the signedness bit with a left shift operation.
Switch to unsigned integer to prevent this in the LAST_FRAG symbol.

xdr_rec.c:559:39, left shift of 1 by 31 places cannot be represented in type 'int'
xdr_rec.c:572:26, left shift of 1 by 31 places cannot be represented in type 'int'
xdr_rec.c:573:25, left shift of 1 by 31 places cannot be represented in type 'int'
xdr_rec.c:632:37, left shift of 1 by 31 places cannot be represented in type 'int'
xdr_rec.c:711:32, left shift of 1 by 31 places cannot be represented in type 'int'
xdr_rec.c:722:28, left shift of 1 by 31 places cannot be represented in type 'int'

Detected with micro-UBSan in the user mode.

diffstat:

 lib/libc/rpc/xdr_rec.c |  6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diffs (27 lines):

diff -r c7bceb01ca84 -r 9644e122c97d lib/libc/rpc/xdr_rec.c
--- a/lib/libc/rpc/xdr_rec.c    Wed Jul 25 23:52:38 2018 +0000
+++ b/lib/libc/rpc/xdr_rec.c    Wed Jul 25 23:59:08 2018 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: xdr_rec.c,v 1.36 2015/03/26 11:31:57 justin Exp $      */
+/*     $NetBSD: xdr_rec.c,v 1.37 2018/07/25 23:59:08 kamil Exp $       */
 
 /*
  * Copyright (c) 2010, Oracle America, Inc.
@@ -37,7 +37,7 @@
 static char *sccsid = "@(#)xdr_rec.c 1.21 87/08/11 Copyr 1984 Sun Micro";
 static char *sccsid = "@(#)xdr_rec.c   2.2 88/08/01 4.0 RPCSRC";
 #else
-__RCSID("$NetBSD: xdr_rec.c,v 1.36 2015/03/26 11:31:57 justin Exp $");
+__RCSID("$NetBSD: xdr_rec.c,v 1.37 2018/07/25 23:59:08 kamil Exp $");
 #endif
 #endif
 
@@ -121,7 +121,7 @@
  * meet the needs of xdr and rpc based on tcp.
  */
 
-#define LAST_FRAG ((uint32_t)(1 << 31))
+#define LAST_FRAG ((uint32_t)(1U << 31))
 
 typedef struct rec_strm {
        char *tcp_handle;