As many of you know, I've been working for some time on an
implementation of scheduler activations for NetBSD, and a
POSIX-compatible thread library built on top of it.

I believe that the MI kernel code is stable enough to merge into the
main source tree. However, there is a certain amount of MD work needed
to make each platform build and run with the new code. I have done
this work on the i386 and alpha, and powerpc code is forthcoming. I
would like to ask for volunteers to port the nathanw_sa branch to
other platforms.

Attached to this message is my (rough) set of directions for porting;
porters are also encouraged to look at the i386 and alpha code. I
would like to coordinate the work on the branch, so please contact me
if wou would like to work on a particular architecture.

   - Nathan



How to Add nathanw_sa branch support to an additional port.

I) Administrivia
 
 0) Tell me that you're working on it!

 1) Branch the architecture directories:
   cvs rtag nathanw_sa_base syssrc/sys/arch/PORT
   cvs rtag -b nathanw_sa syssrc/sys/arch/PORT

II) LWP support

 2) Add mcontext support if it doesn't already exist. 
  - sys/arch/PORT/include/mcontext.h should define mcontext_t.
  - sys/arch/PORT/PORT/machdep.c should define cpu_getmcontext() and
    cpu_setmcontext() (prototypes in sys/sys/ucontext.h). The copying
    of data from the trapframe to the mcontext should be similar to
    the existing code in sendsig() and sys__sigreturn14().

 3) Split struct mdproc in sys/arch/PORT/include/proc.h into the
    lwp-specific part, struct mdlwp, and the proc-specific part,
    struct mdproc.  

    Generally, syscall entry pointers go in mdproc; some flags, such
    as per-process AST flags go in mdproc (since they're used by
    signals as well as scheduling); everything else goes in
    mdlwp. struct mdproc may be empty.

 4) Slow and grungy part: Audit all uses of struct proc * and curproc,
    and figure out whether they're trying to refer to the process state
    or execution state, and change appropriately.

    System calls (such as those in sys/arch/PORT/PORT/sys_machdep.c)
    change signature to sys_foo(struct lwp *, void *, register_t
    *). Often the easiest way to convert a syscall is just to add a
    "struct proc *p = l->l_proc;" to the local variable declarations.

    Uses of curproc should remain curproc if they refer to the current
    LWP. If they refer to the current process, change them to
    curproc->l_proc. If they are used in a context where curproc could
    be NULL (such as interrupt context), use something like:

    "l = curproc; p = l ? l->l_proc : NULL;"

    to avoid the l->l_proc dereference when l is NULL.

    open/close/poll/ioctl entry points keep proc * in their signature.

    References to "p->p_md" change to "l->l_md".
    References to "p->p_addr" change to "l->l_addr".

    Replace P_* flags with L_* flags where appropriate. See <sys/lwp.h>
    for a list.

    References to proc0 need to be split into references to proc0 and
    lwp0. Be especially careful about this in locore.s.

    Add "#include <sys/lwp.h>" to files modified to use l->l_*.


 5) PORT/userret.h: Add code to check for P_WEXIT and L_SA_UPCALL to
    userret(). XXX this is apparantly MI, but all of the userret path
    is MD...

 6) Adjuct vm_machdep.c:cpu_exit() to take a second int argument ("proc") and
    to call switch_exit() if it is nonzero, and switch_lwp_exit() if it
    is zero.
 
 7) Modify cpu_switch() to return 0 if no better LWP was found to
    switch to and 1 if another LWP was switched to and switched back to
    this one.


 8) Implement the following functions:
    (locations are just suggestions)

   cpu_preempt(struct lwp *current, struct lwp *new) (locore.s)
        Variant of cpu_switch, but instead of picking a new LWP
        from the run queue, switch to the indicated new LWP.

   switch_lwp_exit(struct lwp *l) (locore.s)
        Clone of switch_exit() but calls lwp_exit2() instead of exit2(). 

   cpu_upcall() (machdep.c)
        Copies out upcall arguments and sets up trapframe for return
        through upcall trampoline, much like sendsig().

   cpu_stashcontext() (machdep.c)
        Puts a ucontext_t representing the current user-level state on
        the user stack, below the currently used area.


   These two can be textually copied from the i386 or alpha trap.c; they
   could be #included from somewhere, but I find #including functions
   distasteful:

   startlwp()  (machdep.c; it's only MD because userret is)
        Takes a ucontext_t * argument (from the lwp_uc_pool), calls
        cpu_setmcontext() in it, returns the ucontext_t to the pool,
        and calls userret().

   upcallret() (ditto)
        Takes a struct lwp * as an argument and calls userret().

   cpu_setfunc() (parallels cpu_fork() in purpose, so it should probably
       go in vm_machdep.c)
       Set the given LWP to start at the given function, via
       proc_trampoline, similar to the way cpu_fork() arranges to call
       func(arg). 

At this point a kernel should build and run normally.

III) Non-SA userlevel support:

 10) In libc/arch/PORT/gen/makecontext.c, implement makecontext(3).
 11) In libc/arch/PORT/{gen,sys}, implement asm stubs for
     resumecontext(), swapcontext(), and getcontext()
 12) In libc/arch/PORT/gen/_lwp.c, implement _lwp_makecontext().


III) pthread support

 13) On the target machine, install the new headers ("make includes" in
     the sa-branch sys/ and include/). Build and install sa-branch libc.

 14) In the pthread/arch/PORT directory (this should be per-CPU, not
     per-platform), implement _getcontext_u(), _setcontext_u(),
     _swapcontext_u() (in _context_u.S), and pthread__switch(),
     pthread__locked_switch(), and pthread__upcall_switch() (in
     pthread_switch.S). In pthread_md, define pthread__sp() as an
     inline function which returns the value of the stack pointer, and
     pthread___uc_sp() as a macro that takes a ucontext_t and extracts
     the stack pointer from it.