NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: bin/47374: Possible Integer Overflow in msort.c

The following reply was made to PR bin/47374; it has been noted by GNATS.

From: David Laight <>
Subject: Re: bin/47374: Possible Integer Overflow in msort.c
Date: Sat, 29 Dec 2012 20:27:00 +0000

 On Sat, Dec 29, 2012 at 02:45:01PM +0000, wrote:
 > >Number:         47374
 > >Category:       bin
 > >Synopsis:       Possible Integer Overflow in msort.c
 > >Description:
 > This came up during a discussion with Dhruv Matani (@dhruvbird) over 
 > Twitter. There seems to be a chance of an integer overflow in the 
 > implementation of merge sort in src/usr.bin/sort. 
 > The for loop in the insert function in msort.c has the potential of an 
 > integer overflow (during the calculation of mid).
 > static int
 > insert(struct mfile **flist, struct mfile *rec, int ttop, int delete)
 > {
 >     int mid, top = ttop, bot = 0, cmpv = 1;
 >     for (mid = top / 2; bot + 1 != top; mid = (bot + top) / 2) {
 No it doesn't.
 The calculated number is an index into an in-memory array of records.
 There is no way that the array is going to have more than MAXINT/2
 I can't remember off-hand what actually limits the array (it might
 be the number of files in the merge), but it can't be anywhere
 near MAXINT/2.
 David Laight:

Home | Main Index | Thread Index | Old Index