Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
To: Simon Burge <simonb@netbsd.org>
From: David Brownlee <abs@mono.org>
List: netbsd-bugs
Date: 10/26/1999 12:00:56
	Maybe it could call getrlimit and base the size of the limited
	buffer on that (1/8th?)


		David/absolute

On Tue, 26 Oct 1999, Simon Burge wrote:

> Havard.Eidnes@runit.sintef.no wrote:
> 
> > > With both this and bin/8461, seems like a very strong case to turn off
> > > HAVE_MMAP.  Any objections?
> > 
> > Gah, even doing that appears not to be sufficient to cure the
> > problem:
> 
> It seems that under some circumstances that grep wants to save a very
> large chunk of the current file in a buffer, and that five times that
> buffer is allocated when the buffers are expanded.  So when it tries to
> look in:
> 
> 	 532947  72608 -rwxr-xr-x    1 root     wheel     37136160 Oct 25 11:05 ./compile/SERVER_1/netbsd.gdb
> 
> it'll _could_ try to allocate up to 175MB...  A simple reproduction is:
> 
> 	% dd if=/dev/zero of=/tmp/foo bs=1m count=8
> 	% limit data 8192
> 	% grep foo /tmp/foo
> 	grep: memory exhausted
> 
> The following patch limits the size of the save buffer to 2MB, which
> means up to 10MB will be allocated.  I'm really not sure what the
> implications of this are though - simple tests still seem to work ok.
> 
> I'll send something to the BUG bugs list and see what they say about
> this.
> 
> Simon.
> --
> Index: grep.c
> ===================================================================
> RCS file: /cvsroot/gnusrc/gnu/dist/grep/src/grep.c,v
> retrieving revision 1.4
> diff -p -u -r1.4 grep.c
> --- grep.c	1999/04/06 16:54:55	1.4
> +++ grep.c	1999/10/26 00:55:55
> @@ -334,6 +334,9 @@ fillbuf (save, stats)
>    caddr_t maddr;
>  #endif
>  
> +  /* limit save area to 2MB so that memory doesn't overflow on large files */
> +  if (save > 2 * 1024 * 1024)
> +    save = 2 * 1024 * 1024;
>    if (save > bufsalloc)
>      {
>        char *nubuffer;
>