Re: brk(3)/break(2) does not fail for very high adresses

[Nicolas Joly <njoly%pasteur.fr@localhost>]

On Mon, Nov 23, 2009 at 07:26:19PM +0100, Joerg Sonnenberger wrote:
> On Mon, Nov 23, 2009 at 05:32:09PM +0100, Nicolas Joly wrote:
> > On Mon, Nov 23, 2009 at 05:25:01PM +0100, Joerg Sonnenberger wrote:
> > > On Mon, Nov 23, 2009 at 05:10:36PM +0100, Nicolas Joly wrote:
> > > > Is the attached patch ok ? It make break(2) fail, when the rounded
> > > > adress wrap.
> > > 
> > > Why not modify the existing check to do (new == 0 || new > old)?
> > 
> > It could fail early without mutex enter/exit calls.
> 
> Yeah, but I don't know if optimising for the error case is really an
> improvement.

Ok. Here is an updated version which address Joerg concern.

Thanks.

-- 
Nicolas Joly

Biological Software and Databanks.
Institut Pasteur, Paris.

Index: sys/uvm/uvm_unix.c
===================================================================
RCS file: /cvsroot/src/sys/uvm/uvm_unix.c,v
retrieving revision 1.41
diff -u -p -r1.41 uvm_unix.c
--- sys/uvm/uvm_unix.c  4 Mar 2009 21:52:38 -0000       1.41
+++ sys/uvm/uvm_unix.c  25 Nov 2009 10:58:40 -0000
@@ -86,7 +86,8 @@ sys_obreak(struct lwp *l, const struct s
        mutex_enter(&p->p_auxlock);
        old = (vaddr_t)vm->vm_daddr;
        new = round_page((vaddr_t)SCARG(uap, nsize));
-       if ((new - old) > p->p_rlimit[RLIMIT_DATA].rlim_cur && new > old) {
+       if ((SCARG(uap, nsize) && new == 0) ||
+           ((new - old) > p->p_rlimit[RLIMIT_DATA].rlim_cur && new > old)) {
                mutex_exit(&p->p_auxlock);
                return (ENOMEM);
        }