On Fri, Jul 17, 2020 at 08:48:01AM +0100, Paul Durrant wrote: > > -----Original Message----- > > From: Brian Marcotte > > Sent: 16 July 2020 21:24 > > To: Paul Durrant > > Cc: paul@xen.org; 'Jules' ; xen-devel@lists.xenproject.org; > > oleksandr_grytsov@epam.com; wl@xen.org > > Subject: Re: [EXTERNAL] [Xen-devel] XEN Qdisk Ceph rbd support broken? > > > > > Your issue stems from the auto-creation code in xen-block: > > > > > > The "aio:rbd:rbd/machine.disk0" string is generated by libxl and does > > > look a little odd and will fool the parser there, but the error you see > > > after modifying the string appears to be because QEMU's QMP block > > > device instantiation code is objecting to a missing parameter. Older > > > QEMUs circumvented that code which is almost certainly why you don't > > > see the issue with versions 2 or 3. > > > > Xen 4.13 and 4.14 includes QEMU 4 and 5. They don't work with Ceph/RBD. > > > > Are you saying that xl/libxl is doing the right thing and the problem > > needs to be fixed in QEMU? > > Unfortunately, from what you describe, it sounds like there is a problem in both. To get something going, you could bring a domain > up paused and then try manually adding your rbd device using the QMP shell. > > It would be useful if a toolstack maintainer could take a look at this issue in the near future. > Hi, I did start working on a solution some time ago and produced a patch for QEMU (attached) which would allow QEMU to parse the aio:rdb:... string from xenstore. But I ran into an other issue when I tried with nbd (Network Block Device), QEMU would connect twice to the NBD server and the server I had didn't like it. Maybe Ceph would allow two connections to the same disk? The two connections issue is less likely to happen on older QEMU because it would delay the second connection until the guest connect to the PV backend, so after the emulated disk has been unplugged (and thus the first connection disconnected). Anyway, it would be better to upgrade libxl to be able to create a QEMU PV backend via QMP or qemu's command line rather than via xenstore, but I don't think I have time to work on it just yet. But I feel like we are going to have the same issue that QEMU will try to connect twice to the Ceph server where this wasn't likely to happen before. Jules, Brian, could you maybe try the attach QEMU patch and see if that works? Cheers, -- Anthony PERARD