[src/trunk]: src/usr.bin/gzip Correct Undefined Behavior in gzip(1)

[kamil <kamil%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/7f99d24e2848
branches:  trunk
changeset: 323350:7f99d24e2848
user:      kamil <kamil%NetBSD.org@localhost>
date:      Tue Jun 12 00:42:17 2018 +0000

description:
Correct Undefined Behavior in gzip(1)

Unportable left shift reported with MKSANITIZER=yes USE_SANITIZER=undefined:

# progress -zf ./games.tgz  tar -xp -C "./" -f -
/public/src.git/usr.bin/gzip/gzip.c:2126:33: runtime error: left shift of 251 by 24 places cannot be represented in type 'int'
100% |****************************************************************************************************************| 44500 KiB  119.69 MiB/s    00:00 ETA


Refactor the following code into something that is more clear
and fix signed integer shift, by casting all buf[] elements to
(unsigned int):

unsigned char buf[8];
uint32_t usize;
[...]
else {
    usize = buf[4] | buf[5] << 8 |
            buf[6] << 16 | buf[7] << 24;
[...]

New version:

    usize = buf[4];
    usize |= (unsigned int)buf[5] << 8;
    usize |= (unsigned int)buf[6] << 16;
    usize |= (unsigned int)buf[7] << 24;

Only the "<< 24" part needs explicit cast, but for consistency make the
integer promotion explicit and clear to a code reader.

Sponsored by <The NetBSD Foundation>

diffstat:

 usr.bin/gzip/gzip.c |  16 ++++++++++------
 1 files changed, 10 insertions(+), 6 deletions(-)

diffs (39 lines):

diff -r 875def549af8 -r 7f99d24e2848 usr.bin/gzip/gzip.c
--- a/usr.bin/gzip/gzip.c       Tue Jun 12 00:19:17 2018 +0000
+++ b/usr.bin/gzip/gzip.c       Tue Jun 12 00:42:17 2018 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: gzip.c,v 1.112 2017/08/23 13:04:17 christos Exp $      */
+/*     $NetBSD: gzip.c,v 1.113 2018/06/12 00:42:17 kamil Exp $ */
 
 /*
  * Copyright (c) 1997, 1998, 2003, 2004, 2006, 2008, 2009, 2010, 2011, 2015, 2017
@@ -31,7 +31,7 @@
 #ifndef lint
 __COPYRIGHT("@(#) Copyright (c) 1997, 1998, 2003, 2004, 2006, 2008,\
  2009, 2010, 2011, 2015, 2017 Matthew R. Green.  All rights reserved.");
-__RCSID("$NetBSD: gzip.c,v 1.112 2017/08/23 13:04:17 christos Exp $");
+__RCSID("$NetBSD: gzip.c,v 1.113 2018/06/12 00:42:17 kamil Exp $");
 #endif /* not lint */
 
 /*
@@ -2118,12 +2118,16 @@
                                maybe_warnx("read of uncompressed size");
 
                        else {
-                               usize = buf[4] | buf[5] << 8 |
-                                       buf[6] << 16 | buf[7] << 24;
+                               usize = buf[4];
+                               usize |= (unsigned int)buf[5] << 8;
+                               usize |= (unsigned int)buf[6] << 16;
+                               usize |= (unsigned int)buf[7] << 24;
                                in = (off_t)usize;
 #ifndef SMALL
-                               crc = buf[0] | buf[1] << 8 |
-                                     buf[2] << 16 | buf[3] << 24;
+                               crc = buf[0];
+                               crc |= (unsigned int)buf[1] << 8;
+                               crc |= (unsigned int)buf[2] << 16;
+                               crc |= (unsigned int)buf[3] << 24;
 #endif
                        }
                }