Re: [PATCHv3] QEMU(upstream): Disable xen's use of O_DIRECT by default as it results in crashes.

From: Paolo Bonzini <pbonzini@redhat.com>
To: Alex Bligh <alex@alex.org.uk>
Cc: Ian Campbell <Ian.Campbell@citrix.com>,
	Stefano Stabellini <stefano.stabellini@eu.citrix.com>,
	George Dunlap <George.Dunlap@eu.citrix.com>,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	qemu-devel@nongnu.org, xen-devel <xen-devel@lists.xen.org>,
	Anthony Liguori <anthony@codemonkey.ws>
Subject: Re: [PATCHv3] QEMU(upstream): Disable xen's use of O_DIRECT by default as it results in crashes.
Date: Mon, 18 Mar 2013 17:19:14 +0100	[thread overview]
Message-ID: <51473E82.1020806__14117.7736958072$1363623716$gmane$org@redhat.com> (raw)
In-Reply-To: <A2FA46AE3DD746AD97DC3137@Ximines.local>

Il 18/03/2013 16:40, Alex Bligh ha scritto:
> Paolo,
> 
> --On 18 March 2013 15:49:48 +0100 Paolo Bonzini <pbonzini@redhat.com>
> wrote:
> 
>>>> The problem is that the target's page cache might host image data from a
>>>> previous run.

I remembered this incorrectly, sorry.  It's not from a previous run,
it's from the beginning of this run.  See
http://wiki.qemu.org/Migration/Storage for more information.

A VM has a disk backed by NFS. It runs on node A, at which point pages
are introduced to the page cache. It then migrates to node B, which
entails starting the VM on node B while it is still running on node A.
Closing has yet to happen on node A, but the file is already open on
node B; anything that is cached on node B will never be invalidated.

Thus, any changes done to the disk on node A during migration may not
become visible on node B.

>>> Disabling migration seems a bit excessive when migration isn't disabled
>>> with cache=unsafe (AFAIK)
>>
>> It is not QEMU's task.  There are cases where the cache= options are
>> unnecessary or ignored.  But indeed libvirt warns (or blocks, I don't
>> remember) in this case.
> 
> Well we are kvm user too, and I understand hat kvm (as opposed to
> libvirt) does not error or warn if you live migrate with cache=writeback.
> 
> I've no problem if xl or libvirt or whatever error or warn. My usage
> is API based, rather than xl / libvirt based.

What makes libvirt not an API (just like libxl)?

>> If libxl does migration without O_DIRECT, then that's a bug in libxl.
>> What about blkback?  IIRC it uses bios, so it also bypasses the page
>> cache.
> 
> Possibly a bug in xl rather than libxl, but as no emulated devices
> use O_DIRECT, that bug is already there, and isn't in QEMU.

blkback is the in-kernel PV device, it's not an emulated device.

>>> Stefano did ack the patch, and for a one line change it's been
>>> through a pretty extensive discussion on xen-devel ...
>>
>> It may be a one-line change, but it completely changes the paths that
>> I/O goes through.  Apparently the discussion was not enough.
> 
> What would you suggest?

Nothing except fixing the bug in the kernel.  No one has yet explained
why blkback is not susceptible to the same bug.

Paolo