NetBSD/i386 handling of CR2 vs. VirtualBox "raw" mode

["Valeriy E. Ushakov" <uwe%stderr.spb.ru@localhost>]

I've been playing with NetBSD/i386 under VirtualBox (on non VT-x host).

VirtualBox "raw" mode executes guest code by patching it and running
it as-is.  Currently there are several problems that prevent netbsd
from running under raw mode.  I've kludged around the first two (add a
4-byte nop in Xspllower and disable x86_patch) and the kernel would
boot all the way, but userland processes would get random segfaults.

It turns out that that last problem is caused by the fact that trap()
reads CR2 very late.  Since in vbox raw mode real CR2 is also an
"emulated" CR2, so to say, real CR2 is already clobbered, so by the
time we read it (as "emulated" CR2) in trap() the page fault address
we read is wrong (it's usually an address of a field in cpu_info, like
ntraps, that trap vector code touches on its way to call trap()).

The case can be argued both ways.

On one hand, netbsd is lazy about reading CR2 since any fault between
the time we took the original fault and the time we read CR2 would be
a double fault and hence a panic.  Thus one can argue that vbox should
just emulate CR2 properly in raw mode.

On the other hand, reading CR2 immediately when page fault happens
seems like a good hygiene and arguably if a kernel bug causes a double
fault then original fault address (read from CR2 early and kept
around) might be helpful for debugging the kernel bug.

Since trap is a bit of a kitchen sink used to handle ~all traps and
since my i386 asm fu is weak I did a quick and dirty kludge that saves
CR2 in a variable and changed trap() to use that instead of rcr2().
That makes netbsd boot.  I haven't given it a thorough workout yet,
though.

Ideally, I think trap() should be refactored and the piece to handle
page faults should be called with fault address as an argument (read
from CR2 early in the trap vector).  That might be a good idea
regardless of vbox issue. 

Opinions?


PS: NetBSD seems to work as-is under --recompile-supervisor that uses
full interperter for the kernel.

-uwe