PR/45975 CVS commit: [netbsd-6] src/sys

["Jeff Rizzo" <riz%netbsd.org@localhost>]

The following reply was made to PR port-xen/45975; it has been noted by GNATS.

From: "Jeff Rizzo" <riz%netbsd.org@localhost>
To: gnats-bugs%gnats.NetBSD.org@localhost
Cc: 
Subject: PR/45975 CVS commit: [netbsd-6] src/sys
Date: Wed, 22 Feb 2012 18:56:50 +0000

 Module Name:   src
 Committed By:  riz
 Date:          Wed Feb 22 18:56:49 UTC 2012
 
 Modified Files:
        src/sys/arch/x86/include [netbsd-6]: cpu.h pmap.h
        src/sys/arch/x86/x86 [netbsd-6]: cpu.c pmap.c
        src/sys/arch/xen/include [netbsd-6]: hypervisor.h intr.h
        src/sys/arch/xen/x86 [netbsd-6]: cpu.c x86_xpmap.c xen_ipi.c xen_pmap.c
        src/sys/uvm [netbsd-6]: uvm_km.c uvm_kmguard.c uvm_map.c
 
 Log Message:
 Pull up following revision(s) (requested by bouyer in ticket #29):
        sys/arch/xen/x86/x86_xpmap.c: revision 1.39
        sys/arch/xen/include/hypervisor.h: revision 1.37
        sys/arch/xen/include/intr.h: revision 1.34
        sys/arch/xen/x86/xen_ipi.c: revision 1.10
        sys/arch/x86/x86/cpu.c: revision 1.97
        sys/arch/x86/include/cpu.h: revision 1.48
        sys/uvm/uvm_map.c: revision 1.315
        sys/arch/x86/x86/pmap.c: revision 1.165
        sys/arch/xen/x86/cpu.c: revision 1.81
        sys/arch/x86/x86/pmap.c: revision 1.167
        sys/arch/xen/x86/cpu.c: revision 1.82
        sys/arch/x86/x86/pmap.c: revision 1.168
        sys/arch/xen/x86/xen_pmap.c: revision 1.17
        sys/uvm/uvm_km.c: revision 1.122
        sys/uvm/uvm_kmguard.c: revision 1.10
        sys/arch/x86/include/pmap.h: revision 1.50
 Apply patch proposed in PR port-xen/45975 (this does not solve the exact
 problem reported here but is part of the solution):
 xen_kpm_sync() is not working as expected,
 leading to races between CPUs.
 1 the check (xpq_cpu != &x86_curcpu) is always false because we
   have different x86_curcpu symbols with different addresses in the kernel.
   Fortunably, all addresses dissaemble to the same code.
   Because of this we always use the code intended for bootstrap, which doesn't
   use cross-calls or lock.
 2 once 1 above is fixed, xen_kpm_sync() will use xcalls to sync other CPUs,
   which cause it to sleep and pmap.c doesn't like that. It triggers this
   KASSERT() in pmap_unmap_ptes():
   KASSERT(pmap->pm_ncsw == curlwp->l_ncsw);
 3 pmap->pm_cpus is not safe for the purpose of xen_kpm_sync(), which
   needs to know on which CPU a pmap is loaded *now*:
   pmap->pm_cpus is cleared before cpu_load_pmap() is called to switch
   to a new pmap, leaving a window where a pmap is still in a CPU's
   ci_kpm_pdir but not in pm_cpus. As a virtual CPU may be preempted
   by the hypervisor at any time, it can be large enough to let another
   CPU free the PTP and reuse it as a normal page.
 To fix 2), avoid cross-calls and IPIs completely, and instead
 use a mutex to update all CPU's ci_kpm_pdir from the local CPU.
 It's safe because we just need to update the table page, a tlbflush IPI will
 happen later. As a side effect, we don't need a different code for bootstrap,
 fixing 1). The mutex added to struct cpu needs a small headers reorganisation.
 to fix 3), introduce a pm_xen_ptp_cpus which is updated from
 cpu_pmap_load(), whith the ci_kpm_mtx mutex held. Checking it with
 ci_kpm_mtx held will avoid overwriting the wrong pmap's ci_kpm_pdir.
 While there I removed the unused pmap_is_active() function;
 and added some more details to DIAGNOSTIC panics.
 When using uvm_km_pgremove_intrsafe() make sure mappings are removed
 before returning the pages to the free pool. Otherwise, under Xen,
 a page which still has a writable mapping could be allocated for
 a PDP by another CPU and the hypervisor would refuse it (this is
 PR port-xen/45975).
 For this, move the pmap_kremove() calls inside uvm_km_pgremove_intrsafe(),
 and do pmap_kremove()/uvm_pagefree() in batch of (at most) 16 entries
 (as suggested by Chuck Silvers on tech-kern@, see also
 http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012727.html and
 followups).
 Avoid early use of xen_kpm_sync(); locks are not available at this time.
 Don't call cpu_init() twice.
 Makes LOCKDEBUG kernels boot again
 Revert pmap_pte_flush() -> xpq_flush_queue() in previous.
 
 
 To generate a diff of this commit:
 cvs rdiff -u -r1.47 -r1.47.2.1 src/sys/arch/x86/include/cpu.h
 cvs rdiff -u -r1.49 -r1.49.2.1 src/sys/arch/x86/include/pmap.h
 cvs rdiff -u -r1.96 -r1.96.8.1 src/sys/arch/x86/x86/cpu.c
 cvs rdiff -u -r1.164 -r1.164.2.1 src/sys/arch/x86/x86/pmap.c
 cvs rdiff -u -r1.36.2.1 -r1.36.2.2 src/sys/arch/xen/include/hypervisor.h
 cvs rdiff -u -r1.33 -r1.33.8.1 src/sys/arch/xen/include/intr.h
 cvs rdiff -u -r1.80 -r1.80.2.1 src/sys/arch/xen/x86/cpu.c
 cvs rdiff -u -r1.38 -r1.38.2.1 src/sys/arch/xen/x86/x86_xpmap.c
 cvs rdiff -u -r1.9 -r1.9.2.1 src/sys/arch/xen/x86/xen_ipi.c
 cvs rdiff -u -r1.16 -r1.16.2.1 src/sys/arch/xen/x86/xen_pmap.c
 cvs rdiff -u -r1.120 -r1.120.2.1 src/sys/uvm/uvm_km.c
 cvs rdiff -u -r1.9 -r1.9.2.1 src/sys/uvm/uvm_kmguard.c
 cvs rdiff -u -r1.313 -r1.313.2.1 src/sys/uvm/uvm_map.c
 
 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.