Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
To: Todd Whitesel <toddpw@best.com>
From: David Brownlee <abs@mono.org>
List: netbsd-bugs
Date: 10/28/1999 00:47:56
On Wed, 27 Oct 1999, Todd Whitesel wrote:

> > 	Maybe it could call getrlimit and base the size of the limited
> > 	buffer on that (1/8th?)
> 
> Ugh. These are all cheesy heuristics.
> 
> It would be much better if there was a simple system-specific enhancement
> requiring HAVE_MMAP, where it picks an address way out in the process space
> somewhere and starts mmap-ing the growable buffer there. When it needs to
> grow the buffer, it just mmap's some more memory on the end of it.
> 
> This is just using sparse addressing to get the benefits of segments...
> 
> Admittedly I didn't follow this entire thread, so I'm assuming grep really
> needs all that memory, and we should let it mmap away until it hits an rlimit.
> 
	As I understand it this is only used for grep as memory to store
	a single newline when searching through files.

	Arguably any file with several megabytes of data without a newline
	is pathalogical (in a text sense), though a grep failing to find
	a string in a binary file because it was split across an arbitrary
	block boundary is a bad thing...

		David/absolute