Subject: Re: bin/10625: /usr/bin/cmp is unable to compare rather large files
To: NetBSD Bugs and PR posting List <netbsd-bugs@NetBSD.ORG>
From: R. C. Dowdeswell <elric@mabelode.imrryr.org>
List: netbsd-bugs
Date: 07/26/2000 18:45:44
On 964109402 seconds since the Beginning of the UNIX epoch
Greg A. Woods wrote:
>
>[ On Tuesday, July 18, 2000 at 01:21:35 (-0700), uklaus@hgb-leipzig.de wrote: ]
>> Subject: bin/10625: /usr/bin/cmp is unable to compare rather large files
>>
>> <root@hgb1> ~# ls -lai /wd0/Backup/media48.tar.gz /wd1/Backup/media48.tar.gz 
>>    7942 -rw-r--r--  1 root  daemon  1409578717 Jul 12 08:16 /wd0/Backup/media48.tar.gz
>> 4551315 -rw-r--r--  1 root  daemon  1409578717 Jul 12 08:16 /wd1/Backup/media48.tar.gz
>> <root@hgb1> ~# cmp /wd0/Backup/media48.tar.gz /wd1/Backup/media48.tar.gz 
>> cmp: /wd1/Backup/media48.tar.gz: Cannot allocate memory
>
>because of BAD non-unix-like programming practices....
>
>The damn program uses mmap()!
>
>If read() isn't fast enough then something's wrong in the land of *BSD....
>
>(Of course if mmap() is broken and can't handle paging transparently
>through any reasonable number of *VERY* large files then something's
>very wrong with it too....)

Well, mmap() by its very operation can't handle files that are too
large, especially on 32bit arches.  Mmap(2) will map the entire
file into the processes address space, which is limited.  Looks
like the above files together total 2.6GB and cmp only uses two
mmaps so it is conceivable that problems might occur.

The issue is not whether read(2) or mmap(2) is used, but rather
the fact that cmp tries to get both files in their entirety into
its process address space.  One would have the same problem if
one tried to compare these two files with only two read(2)s.  :-)

In fact perusing the source code (prior to being fixed), it will
only fail for large files that are not too large on 32bit
architechtures.  :-)

 == Roland Dowdeswell                      http://www.Imrryr.ORG/~elric/  ==
 == The Unofficial NetBSD Web Pages        http://www.Imrryr.ORG/NetBSD/  ==
 == The NetBSD Project                            http://www.NetBSD.ORG/  ==