I've finally kludged kernel profiling into working.  (I've done a full
kernel re-build with profiling turned on, and got the same binaries as
with a non- profiling kernel._)

It turns out that if I builda both locore.o and kern_synch.o without
profililng, profiling works. (I don t recall if all the chagnes to
<mips/profile.h> are in the tree; the machine is off for reasons
beyond my control.)

I double-checked that if trap.o and locore_mips3.o are built with
profiling enabled (-pg / -DGPROF), then profillng still woriks.
It's just locore.o and kern_synch.o

That strongly suggests some kind of stack-layout dependency between
the context-switch code in kern_synch.c and locore.S. I didnt' see
anything obvious last night, though.

I'd be _very_ happy if someone can see anything wrong. 
If we get really lucky, it may also fix the kernel div traps when
using  gas from binutils-2.9.1 which I consistently see on a 5000/150.

Meanwhile, the bad news is, pmap_zero_page() still looks like a real
pig for big compile workloads -- ~30% of kernel time -- , followed by
mips3_flushDcache.

thanks once again to everyone who offered opinions.