> Is this a strongarm optimised kernel?  I find stability to increase with
> a non SA optimised kernel, however I've still had the odd segv in
> places.  I'm going to try stressing my cats without the SA optimisation
> and the updated bcopyinout.S

Yes, it is SA optimized.  Turning off SA optimizations would make the 
kernel less efficient, thus if it is a cache issue, more likely that the 
page in question would get flushed out by the internal rototil that 
happens in all caches -- that would explain why you see this less often.

Looking at the log for pmap.c we see:

        netbsd-1-6: 1.97.0.4

revision 1.97.4.5
date: 2002/12/07 20:44:23;  author: he;  state: Exp;  lines: +120 -50
Pull up revision 1.111 (requested by thorpej in ticket #714):
  Use a pool cache for PT-PTs.
----------------------------
revision 1.97.4.4
date: 2002/12/07 20:43:02;  author: he;  state: Exp;  lines: +129 -3
Pull up revision 1.105 (requested by thorpej in ticket #714):
  Add code, conditional on PMAP_ALIAS_DEBUG, which can be
  used to hunt for virtual aliases between managed (pmap_enter)
  and unmanaged (pmap_kenter_pa) mappings.
----------------------------
revision 1.97.4.3
date: 2002/11/21 20:15:23;  author: he;  state: Exp;  lines: +12 -16
Pull up revision 1.110 (requested by thorpej in ticket #712):
  Do cached memory access to L1 tables, making sure to
  write-back the cache after any L1 table modifications.
----------------------------
revision 1.97.4.2
date: 2002/11/16 10:03:47;  author: he;  state: Exp;  lines: +9 -19
Pull up revision 1.107 (requested by thorpej in ticket #662):
  Clean up some warts in pmap_protect().
----------------------------
revision 1.97.4.1
date: 2002/07/31 00:23:17;  author: lukem;  state: Exp;  lines: +13 -4
Pull up revision 1.100 (requested by thorpej in ticket #587):
Don't use pmap_kenter_pa() in pmap_map(); doing so causes an assertion
failure in pmap_kenter_pa().
----------------------------

Corresponding to patches:

revision 1.111
date: 2002/08/21 21:22:52;  author: thorpej;  state: Exp;  lines: +120 -50
Use a pool cache for PT-PTs.
----------------------------
revision 1.110
date: 2002/08/21 18:34:31;  author: thorpej;  state: Exp;  lines: +12 -16
Do cached memory access to L1 tables, making sure to write-back the
cache after any L1 table modifications.
----------------------------
revision 1.107
date: 2002/08/10 00:11:51;  author: thorpej;  state: Exp;  lines: +9 -19
Clean up some warts in pmap_protect().
----------------------------
revision 1.105
date: 2002/08/09 18:22:59;  author: thorpej;  state: Exp;  lines: +129 -3
Add some code, conditional on PMAP_ALIAS_DEBUG, that can be used to
hunt for virtual aliases between managed (pmap_enter) and non-managed
(pmap_kenter_pa) mappings.
----------------------------
revision 1.100
date: 2002/07/30 16:07:23;  author: thorpej;  state: Exp;  lines: +13 -4
Don't use pmap_kenter_pa() in pmap_map(); doing so causes an assertion
failure in pmap_kenter_pa().
----------------------------

But we haven't pulled up

revision 1.122
date: 2002/11/12 22:14:21;  author: chris;  state: Exp;  lines: +23 -22
Tweak a few minor things:
when looking to reenable caching, only do so if all the pages aren't 
already
cached.
Convert some ints to unsigned int.  (scarily this actually shows the 
biggest
decrease in timing for my benchmark, I guess the compiler can optimise 
better)
----------------------------
revision 1.115
date: 2002/08/24 03:10:40;  author: thorpej;  state: Exp;  lines: +13 -9
In pmap_map_in_l1() and pmap_unmap_in_l1(), make sure that the VA
that is passed in is already aligned to a 4M super-section.
----------------------------
revision 1.114
date: 2002/08/24 02:50:53;  author: thorpej;  state: Exp;  lines: +13 -7
When we allocate a PTP, make sure the offset we specify is for
the 4M super-section that the PTP will map, not some random 1M
chunk of it.  This gives the PTP hint code a much better chance
to working properly, and allows us to tidy up the code that
flushes a PTP from the cache in pmap_destroy().
----------------------------
revision 1.113
date: 2002/08/24 02:16:31;  author: thorpej;  state: Exp;  lines: +117 -23
Enable caching on kernel and user page tables.  This saves having
to do uncached memory access during VM operations (which can be
quite expensive on some CPUs).

We currently write-back PTEs as soon as they're modified; there is
some room for optimization (to write them back in larger chunks).
For PTEs in the APTE space (i.e. PTEs for pmaps that describe another
process's address space), PTEs must also be evicted from the cache
complete (PTEs in PTE space will be evicted durint a context switch).
----------------------------
revision 1.112
date: 2002/08/22 01:13:55;  author: thorpej;  state: Exp;  lines: +9 -2
* Add PTE_SYNC() and PTE_SYNC_RANGE() macros.  These don't actually do
  anything yet.
* Use PTE_SYNC() and PTE_SYNC_RANGE() in some obvious places, i.e.
  where vtopte() is used.
----------------------------
revision 1.108
date: 2002/08/10 00:48:35;  author: thorpej;  state: Exp;  lines: +19 -19
Tidy up pmap_clean_page() a little, and reenable some code that was
disabled previously: Skip cleaning mappings which are read-only, because
the pmap (now) does clean pages on a r/w -> r/o transition.
----------------------------
revision 1.104
date: 2002/08/06 21:43:51;  author: thorpej;  state: Exp;  lines: +46 -18
- pmap_remove(): unmap the PTEs *after* we have finished with the
  page tables.
- pmap_enter(): if making a mapping for the same PA rw->ro, write-back
  the cache before doing so.
- pmap_clearbit(): if revoking REF on a page, make sure to wbinv the
  cache if the page has write permission, else inv the cache if the page's
  PTE is valid (XXX we actually wbinv in this case, as well, due to lack
  of idcache_inv_range()).  Only flush the TLB if the PTE changed.
----------------------------

Most of these I don't think are relevant, but I'd want to look at 1.104 
and 1.108 in more detail: these two might be the source of the problem.

R.