I/O performance in hvm domU

[Linus Nordberg <linus%nordberg.se@localhost>]

Hi,

I'm running FreeBSD (6.3) in a hvm domain on NetBSD (5.0.1) and suffer
from pretty bad I/O performance.  I realize that disk I/O should be
punished for going through qemu but would like to hear if my figures
make sense.

I've been running bonnie on both the domU and the dom0, using a file
system on the same disk device on the dom0 as the one on which the files
backing the domU disks resides.  The amount of data used for testing has
been chosen to be equal to the amount of RAM available to the domains,
2GB and 256MB for the domU and the dom0 respectively.

Write performance in domU is 11% and 9% of that for dom0 for char and
block writes, respectively.  For disk reads these figures are 36% and
44% and for random seek performance it's 23%.  The "rewrite" figure is
actually slightly better in domU than dom0 (103%).

Any thoughts on this?  Does it seem reasonable to you?  I can tell that
running ./configure && make throws me back in time about 15 years and in
space to a Slowaris box.  It's very depressing.

hvm configuration and typical output from running bonnie shown below.

Thanks,
Linus N


hvm config
----------
#  -*- mode: python; -*-
# Based on /usr/pkg/share/examples/xen/hvm
kernel = "/usr/pkg/lib/xen/boot/hvmloader"
builder='hvm'
memory = 2048
name = "banksy"
vif = [ 'mac=00:16:3e:00:00:87, bridge=bridge0, type=ioemu' ]
device_model = '/usr/pkg/libexec/qemu-dm'
disk = [
     'file:/domU/banksy/ad0,ioemu:hda,w',
     'file:/domU/banksy/ad1,ioemu:hdb,w'
]
boot='c'
vnc = 1
vncdisplay = 0
usb=1
usbdevice='tablet'


bonnie in hvm domU
------------------
[root@banksy /u/tmp]# dmesg | egrep real\ mem
real memory  = 2147446784 (2047 MB)
[root@banksy /u/tmp]# df -h .
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad1s1h    9.7G     57M    8.9G     1%    /u
[root@banksy /u/tmp]# bonnie -s 2048
File './Bonnie.76622', size: 2147483648
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...Sep  1 16:43:08 banksy ntpd[569]: time reset +1.095584 s
done
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         2048 15999 15.4  9930  4.7 27065 20.0 30324 37.9 49356 24.9 199.8  1.7


bonnie in dom0
--------------
-bash-4.0# dmesg | egrep total\ memor
total memory = 256 MB
-bash-4.0# bonnie -s 256
File './Bonnie.7288', size: 268435456
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 2...Seeker 3...Seeker 1...start 'em...done...done...done...
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
          256 103968 51.7 106710 24.2 26210  9.4 99043 54.6 116068 21.5 939.3  
4.9