CVS commit: src/sys/arch/aarch64

["Nick Hudson" <skrll%netbsd.org@localhost>]

Module Name:    src
Committed By:   skrll
Date:           Wed Aug 12 13:19:35 UTC 2020

Modified Files:
        src/sys/arch/aarch64/aarch64: copyinout.S cpu_machdep.c cpuswitch.S
            db_machdep.c fusu.S genassym.cf idle_machdep.S locore.S vectors.S
        src/sys/arch/aarch64/include: cpu.h proc.h

Log Message:
Part II of ad's aarch64 performance improvements (cpu_switch.S bugs are
all mine)

- Use tpidr_el1 to hold curlwp and not curcpu, because curlwp is accessed
  much more often by MI code.  It also makes curlwp preemption safe and
  allows aarch64_curlwp() to be a const function (curcpu must be volatile).

- Make ASTs operate per-LWP rather than per-CPU, otherwise sometimes LWPs
  can see spurious ASTs (which doesn't cause a problem, it just means some
  time may be wasted).

- Use plain stores to set/clear ASTs.  Make sure ASTs are always set on the
  same CPU as the target LWP, and delivered via IPI if posted from a remote
  CPU so that they are resolved quickly.

- Add some cache line padding to struct cpu_info, to match x86.

- Add a memory barrier in a couple of places where ci_curlwp is set.  This
  is needed whenever an LWP that is resuming on the CPU could hold an
  adaptive mutex.  The barrier needs to drain the CPU's store buffer, so
  that the update to ci_curlwp becomes globally visible before the LWP can
  resume and call mutex_exit().  By my reading of the ARM docs it looks like
  the instruction I used will do the right thing, but I'm not 100% sure.


To generate a diff of this commit:
cvs rdiff -u -r1.14 -r1.15 src/sys/arch/aarch64/aarch64/copyinout.S
cvs rdiff -u -r1.10 -r1.11 src/sys/arch/aarch64/aarch64/cpu_machdep.c
cvs rdiff -u -r1.24 -r1.25 src/sys/arch/aarch64/aarch64/cpuswitch.S
cvs rdiff -u -r1.25 -r1.26 src/sys/arch/aarch64/aarch64/db_machdep.c
cvs rdiff -u -r1.9 -r1.10 src/sys/arch/aarch64/aarch64/fusu.S
cvs rdiff -u -r1.29 -r1.30 src/sys/arch/aarch64/aarch64/genassym.cf
cvs rdiff -u -r1.5 -r1.6 src/sys/arch/aarch64/aarch64/idle_machdep.S
cvs rdiff -u -r1.69 -r1.70 src/sys/arch/aarch64/aarch64/locore.S
cvs rdiff -u -r1.17 -r1.18 src/sys/arch/aarch64/aarch64/vectors.S
cvs rdiff -u -r1.25 -r1.26 src/sys/arch/aarch64/include/cpu.h
cvs rdiff -u -r1.7 -r1.8 src/sys/arch/aarch64/include/proc.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.