Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
To: Simon Burge <simonb@netbsd.org>
From: David Brownlee <abs@mono.org>
List: netbsd-bugs
Date: 10/26/1999 12:00:56
Maybe it could call getrlimit and base the size of the limited
buffer on that (1/8th?)
David/absolute
On Tue, 26 Oct 1999, Simon Burge wrote:
> Havard.Eidnes@runit.sintef.no wrote:
>
> > > With both this and bin/8461, seems like a very strong case to turn off
> > > HAVE_MMAP. Any objections?
> >
> > Gah, even doing that appears not to be sufficient to cure the
> > problem:
>
> It seems that under some circumstances that grep wants to save a very
> large chunk of the current file in a buffer, and that five times that
> buffer is allocated when the buffers are expanded. So when it tries to
> look in:
>
> 532947 72608 -rwxr-xr-x 1 root wheel 37136160 Oct 25 11:05 ./compile/SERVER_1/netbsd.gdb
>
> it'll _could_ try to allocate up to 175MB... A simple reproduction is:
>
> % dd if=/dev/zero of=/tmp/foo bs=1m count=8
> % limit data 8192
> % grep foo /tmp/foo
> grep: memory exhausted
>
> The following patch limits the size of the save buffer to 2MB, which
> means up to 10MB will be allocated. I'm really not sure what the
> implications of this are though - simple tests still seem to work ok.
>
> I'll send something to the BUG bugs list and see what they say about
> this.
>
> Simon.
> --
> Index: grep.c
> ===================================================================
> RCS file: /cvsroot/gnusrc/gnu/dist/grep/src/grep.c,v
> retrieving revision 1.4
> diff -p -u -r1.4 grep.c
> --- grep.c 1999/04/06 16:54:55 1.4
> +++ grep.c 1999/10/26 00:55:55
> @@ -334,6 +334,9 @@ fillbuf (save, stats)
> caddr_t maddr;
> #endif
>
> + /* limit save area to 2MB so that memory doesn't overflow on large files */
> + if (save > 2 * 1024 * 1024)
> + save = 2 * 1024 * 1024;
> if (save > bufsalloc)
> {
> char *nubuffer;
>