[ On Thursday, October 28, 1999 at 10:39:38 (+1000), Simon Burge wrote: ]
> Subject: Re: bin/8681: grep may bomb out with "memory exhausted" 
>
> This is what the maintainer of grep has to say:
> 
> 	GNU grep needs to buffer a line of input in main memory.  If your
> 	input lines are too long to fit into your main memory, then the right
> 	thing to do is to get more memory.  See the section `Memory Usage' in
> 	the GNU coding standards for more.
> 
> Personally, I'd be happy to put the size limit in and put a note in the
> man page stating "you may have problems if you are searching for a regex
> that is greater than 4MB, or if you want to see a line greater than 2MB
> long that has a match".  This doesn't seem unreasonable to me...

Personally I'd say the GNU Grep maintainer(s) has(have) a brain blockage
if that's his(their) attitude!  ;-)

Why not simply fail the match for the partial line, optionally print a
warning message to stderr, flush the input buffer (perhaps giving back
as much of it as it had grown beyond the statistical line size), and
start reading again pretending that there had been a newline at the
point where available memory ran out?  This might not do very nice
things to the system, but it should make 'grep' continue to work under
adverse conditions....

Assuming the problem is with an RE that does span the majority of a line
up to the newline then perhaps the documentation should mention that the
'out-of-memory' warning indicates the user has chosen a sub-optimal RE
for the data he or she is searching and it might be prudent to try
re-writing the RE so that it didn't try to span an entire line (i.e.
didn't anchor at "$").

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>