NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: bin/47374: Possible Integer Overflow in msort.c

On Sat, Dec 29, 2012 at 02:45:01PM +0000, wrote:
> >Number:         47374
> >Category:       bin
> >Synopsis:       Possible Integer Overflow in msort.c
> >Description:
> This came up during a discussion with Dhruv Matani (@dhruvbird) over Twitter. 
> There seems to be a chance of an integer overflow in the implementation of 
> merge sort in src/usr.bin/sort. 
> The for loop in the insert function in msort.c has the potential of an 
> integer overflow (during the calculation of mid).
> static int
> insert(struct mfile **flist, struct mfile *rec, int ttop, int delete)
> {
>     int mid, top = ttop, bot = 0, cmpv = 1;
>     for (mid = top / 2; bot + 1 != top; mid = (bot + top) / 2) {

No it doesn't.
The calculated number is an index into an in-memory array of records.
There is no way that the array is going to have more than MAXINT/2

I can't remember off-hand what actually limits the array (it might
be the number of files in the merge), but it can't be anywhere
near MAXINT/2.


David Laight:

Home | Main Index | Thread Index | Old Index