Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monné <roger.pau%citrix.com@localhost>]

On 28/11/12 20:40, Toby Karyadi wrote:
> On 11/28/12 10:29 AM, Thor Lancelot Simon wrote:
>> On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
>>> On 28/11/12 14:26, Thor Lancelot Simon wrote:
>>>> I am assuming there's some reason we want this, rather than handling
>>>> such requests in-kernel.  But what is it?
>>> I should have marked this as "experimental", or something like this. I'm
>>> not sure if switching to Qdisk for all image file backends is needed.
>>> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
>>>
>>> "loop device had problems with flushing dirty pages (specifically, doing
>>> a large number of writes to an NFS-backed image don't result in the OOM
>>> killer going berserk)."
>>>
>>> I'm not sure if NetBSD is in the same situation, but if I remember
>>> correctly (haven't tried that in a long time), trying to use a disk file
>>> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
>>> issue is: http://gnats.netbsd.org/40726.
>> It seems highly unlikely to me that a problem with the loop device driver
>> on Linux is tremendously relevant to NetBSD.
>>
>> But, even if there is some problem with vnd backed by NFS, that hardly
>> seems like it would be a good reason to make a change that reduces I/O
>> throughput for the *non* NFS-backed case by at least 20%.  Why would one
>> keep disk images for guests, as a general rule, on NFS, rather than simply
>> doing the NFS mounts on the guests themselves, or using a more sensible
>> protocol like iSCSI?  I have to assume most folks using files as disks are
>> storing them on local filesystems on the dom0, and wrecking performance
>> for that case to solve a problem with NFS that may or may not be hypothetical
>> seems like a very, very bad trade-off.
>>
>> Thor
> Yeah, I usually don't pipe up, but I strongly dislike being forced to 
> use qemu to read raw disk file instead of being able to use vnd on 
> netbsd dom0.

I'm not a fan of Qemu either, but I don't know an easy way to detect if
an image is in a NFS share, or if an image is in qcow, qcow2 or raw. I'm
open to a solution that only launches Qemu if image format is different
than raw or if the image file is in a NFS share.

> There was/is? a bug in xentools 4.1 in libxl that prevents using vnd for 
> raw disk file. The problem boils down to libxl not notifying xenbackendd 
> (via xenstored) to call /usr/pkg/etc/xen/scripts/block at the right time 
> such that vnconfig doesn't get called when it's time to shutdown the 
> domU, and hence you end up with a bunch of vnd devices even after the 
> corresponding domU's have been shutdown. I created a patch to fix that: 
> http://mail-index.netbsd.org/port-xen/2012/05/29/msg007252.html 
> Additionally I put in an experimental feature that allows specifying 
> other types of backend aside from phy: or file:, and as long as the 
> /usr/pkg/etc/xen/scripts/block sh script is modified accordingly, the 
> domU will just get the appropriate device in dom0. Then we can use, for 
> example, vnd:, or iscsi: and from the POV of the domU, it's just another 
> 'physical' device that the dom0 manages. I think the way xenbackendd 
> works, in conjunction with xenstored, is pretty flexible and decoupled, 
> at least with xen 3.3 and 4.1; I don't know what the story is for xen 
> 4.2. Was there a rumor of retiring xenbackendd?

Yes, xenbackendd is deprecated and hotplug scripts are called from libxl
directly (that's true for both NetBSD and Linux).

> I'm also concerned about the trend, which to me is more of putting more 
> and more policy in libxl, as opposed to functionality. I saw this back 
> when I was trying to fix the bug above, hopefully this trend is no 
> longer true. For example, I had the impression (I don't remember the 
> details) that the code that parses the disk configuration to a certain 
> degree decides whether the disk is a block device or a regular file 
> based on the prefix (phy: vs file:), which is not true in netbsd dom0, 
> because the file path is then used to setup a vnd device, which is from 
> libxl's POV is just a block device. Most probably it's born out of the 
> fact that libxl is developed in a mostly linux environment. I do 
> appreciate Roger's effort in improving xen support in netbsd, but 
> hopefully some BSDism would travel back to xen as well.

Yes, most libxl developers are Linux users, so most of the functionality
in libxl is designed with Linux in mind. The only way to change that is
to get BSD developers to contribute to libxl and the Xen project in
general, if we are not active players in this game, we should not expect
that BSD will be taken into account.

As an example, libxl is now working in NetBSD without any external
patches, and AFAIK all functionality is working, and Linux developers
are very careful to try not to break this compatibility.

From my POV, I think your patches to add new functionality to disk
backends should be rebased against xen-unstable and posted to xen-devel,
so that they are added to upstream Xen.

Having a bunch of out-of-tree patches in pkgsrc is not the way to move
forward, because we are not able to test NetBSD with latest Xen. Being
able to test xen-unstable easily will allow us to find bugs and develop
patches before Xen is released, thus keeping this pkgsrc patch queue
minimal.