Re: Running XEN under a Linux AMD64 DOM0

[Christoph Egger <Christoph_Egger%gmx.de@localhost>]

Luke S Crawford wrote:
> Carl Brewer <carl%bl.echidna.id.au@localhost> writes:
> > As a general suggestion, would you advise amd64-current or
> > i386-current for a xen dom0 server with 6GB of RAM with HVM-aware
> > cpu's to get the best of a modern Intel C2D server?  Core2duo 8200 is
> > hvm aware and I believe i386-cuurent is PAE capable also?  There'd be
> > little/no performance impact from using HVM?
>
> Uh, just saying, HVM mode takes a pretty significant performance hit when
> it comes to I/O (file or network)  -  there's not much overhead if all you
> do is cpu/memory access, as the intel vt takes care of that;

Intel VT has troubles with booting HVM guests, in particular when it 
comes to big real mode emulation. Most of them have been resolved
in Xen 3.2.0 / 3.2.1 by emulating all instructions available in real
mode in the Xen hypervisor. The most used bootloader (GRUB) works,
but some other ones (like the FreeBSD bootloader) has problems.

The real performance impact comes from the world switches. A world
switch happens when the CPU leaves guest mode (VMEXIT) does
some emulation and switches back into guest mode (VMRUN).

Each VMEXIT and each VMRUN costs about 1000-2000 cycles. The 
VMEXIT/VMRUN happens on IO, on guest context switches and some misc
stuff (SMI/NMI intercepts, Machine check intercepts, etc.).
The impact on IO is the largest, because hw emulation is most expensive.
The second largest impact come from guest context switches
(CR3 emulation). The cost of shadow paging comes in mind here.

Using PV drivers in HVM guests eliminates the largest performance
impact. The second largest perf. impact can be eliminated with proper
hw support: Replace "shadow paging" with "nested paging" which has been
introduced in AMD Barcelona. With this the CPU can handle CR3 accesses
within the guest mode, so there's no need for a world switch.

Summary: Using PV drivers and Nested Paging you can reach in a HVM
guest up to 97% of native performance.

> but once you start hitting disk or network, I believe Xen emulates real 
> hardware using
> code pulled from qemu.  

You believe it, I know it. For each HVM guest runs a seperate qemu
process in the Dom0.

On a world switch, the CPU leaves the guest
and is in the hypervisor, makes a context switch into Dom0 and schedules
the qemu process. When qemu finished, the CPU makes a context switch
into the hypervisor and then enters into guest mode.


I hope, the technical aspects are understood and that this mail is not
interpreted as covert advertisment for AMD ;-)


Christoph