cache sync on zvols - other problems?

[Greg Troxel <gdt%lexort.com@localhost>]

This message is likely of interest only to those running both xen and
zfs.

Recently I added support for DIOCCACHESYNC in zvols.  When the ioctl is
called, it does a zil_commit.  This is basically copying what FreeBSD
does execpt their cache flush ioctl has a different name.

With this, I no longer see xen complaining about an error return from
the ioctl.

However:

  a n9-amd64 domU seems fine

  a n11-amd64 (11 means current) is ok, but after overnight, it is very
  very sluggish in disk io and the dom0's top shows
       0 root      96    0     0K   71M CPU/11    408:43 99.76% 99.76% [system]
  and this does not seem to recover

I will continue investigating, but I wonder if anyone running current
on a dom0 with disks on zvols is seeing this.

My underlying disk for zfs is an oldisk sandisk "3D" 1T ssd on sata.
That should be not super fast by today's standards, but far from
terrible.


Can anyone explain when xen decides to call DIOCCACHESYNC?  Is that when
there is a corresponding cache flush disk command from the domU?

My guess is that cache flushing is pretty expensive and that a scheme
that does flushes when soundness says is going to be trouble, but I
really am unclear on the details.