brk(3)/break(2) does not fail for very high adresses

[Nicolas Joly <njoly%pasteur.fr@localhost>]

Hi,

While still working on compat linux regression tests, i noticed a
problem where native brk(3) does not fail for very high addresses such
as (void *)-1 (... really the last PAGE_SIZE-1 adresses).

I tracked it to the break(2) syscall (uvm_unix.c:sys_obreak()), where

          new = round_page((vaddr_t)SCARG(uap, nsize));

make the new value wrap, coming back to 0; and then defeat the check
for exceeding process limit.

 28043      1 test CALL  break(0xffffffffffffffff)
 28043      1 test RET   break 0

Is the attached patch ok ? It make break(2) fail, when the rounded
adress wrap.

Thanks.

-- 
Nicolas Joly

Biological Software and Databanks.
Institut Pasteur, Paris.

Index: sys/uvm/uvm_unix.c
===================================================================
RCS file: /cvsroot/src/sys/uvm/uvm_unix.c,v
retrieving revision 1.41
diff -u -p -r1.41 uvm_unix.c
--- sys/uvm/uvm_unix.c  4 Mar 2009 21:52:38 -0000       1.41
+++ sys/uvm/uvm_unix.c  23 Nov 2009 14:53:48 -0000
@@ -83,9 +83,13 @@ sys_obreak(struct lwp *l, const struct s
        vaddr_t new, old;
        int error;
 
+       new = round_page((vaddr_t)SCARG(uap, nsize));
+       /* Detect overflow */
+       if (SCARG(uap, nsize) && new == 0)
+               return (ENOMEM);
+
        mutex_enter(&p->p_auxlock);
        old = (vaddr_t)vm->vm_daddr;
-       new = round_page((vaddr_t)SCARG(uap, nsize));
        if ((new - old) > p->p_rlimit[RLIMIT_DATA].rlim_cur && new > old) {
                mutex_exit(&p->p_auxlock);
                return (ENOMEM);