Re: Fatal crash on xen4.2 HVM + qemu-xen dm + NFS

[Stefano Stabellini <stefano.stabellini@eu.citrix.com>]

On Tue, 22 Jan 2013, Alex Bligh wrote:
> Stefano,
> 
> --On 22 January 2013 16:09:21 +0000 Stefano Stabellini 
> <stefano.stabellini@eu.citrix.com> wrote:
> 
> > Looking more closely at xen_disk's flush operations,
> > even if the semantics of the old BLKIF_OP_WRITE_BARRIER is confused, the
> > implementation of it in xen_disk is strickly a superset of the flushes
> > performed by the new BLKIF_OP_FLUSH_DISKCACHE.
> >
> > So unless blkfront issues fewer cache flushes when using
> > BLKIF_OP_WRITE_BARRIER, I am tempted to say that even with the old flush
> > operation, it might be safe to open the file write-back.
> 
> Either writeback or 'user specifiable' would be my preference. Writethrough
> has known performance problems with various disk formats, "notably qcow2"
> (see qemu manpage). And "none" (aka O_DIRECT) appears to be dangerous.
> 
> I suspect the best possible answer is a
>   cache=[qemu-cache-identifier]
> config key, which gets put in xenstore, and which xen_disk.c then
> interprets using the same routine QEMU does itself for cache= on the
> command line, then uses exactly those BDEV flags.

that would be nice to have


> For completeness one could also add an emulcache= option and just
> pass that straight through to the qemu command line for the emulated
> drives.

In qemu-xen it should be already possible to change the cache setting
for the IDE disk by passing the right command line option to QEMU. Not
ideal, but it would work.


> I had a quick look at this on the train and it appears that to do it
> properly requires fiddling with lex file and possibly xenstore, i.e.
> is not completely trivial.
> 
> An alternative more disgusting arrangement would be to overload one
> of the existing options.
> 
> What's the right approach here, and have I any hope of getting a patch
> into 4.2.2? Not a disaster if not as I already need to maintain a local
> tree due to the absence of HVM live migrate on qemu-upstream.

Backports are only for bug fixes. However it might be possible to
backport a simple patch that in case of opening failure with O_DIRECT,
tries again without it (see appended). I don't think that would work
for you though, unless you manage to mount NFS in a way that would
return EINVAL on open(O_DIRECT), like it used to.

diff --git a/hw/xen_disk.c b/hw/xen_disk.c
index 33a5531..d6d71fe 100644
--- a/hw/xen_disk.c
+++ b/hw/xen_disk.c
@@ -659,9 +659,16 @@ static int blk_init(struct XenDevice *xendev)
        if (blkdev->bs) {
            if (bdrv_open2(blkdev->bs, blkdev->filename, qflags,
                            bdrv_find_format(blkdev->fileproto)) != 0) {
-               bdrv_delete(blkdev->bs);
-               blkdev->bs = NULL;
-           }
+                /* try without O_DIRECT */
+                xen_be_printf(&blkdev->xendev, 0, "opening %s with O_DIRECT failed, trying write-through.\n",
+                        blkdev->filename);
+                qflags &= BDRV_O_NOCACHE;
+                if (bdrv_open2(blkdev->bs, blkdev->filename, qflags,
+                            bdrv_find_format(blkdev->fileproto)) != 0) {
+                    bdrv_delete(blkdev->bs);
+                    blkdev->bs = NULL;
+                }
+            }
        }
        if (!blkdev->bs)
            return -1;