RE: qemu and Xen ABI-unstable libs

[Paul Durrant <xadimgnik@gmail.com>]

> -----Original Message-----
> From: Roger Pau Monné <roger.pau@citrix.com>
> Sent: 21 September 2020 11:16
> To: paul@xen.org
> Cc: 'Ian Jackson' <iwj@xenproject.org>; 'Debian folks: Michael Tokarev' <mjt@tls.msk.ru>; 'Hans van
> Kranenburg' <hans@knorrie.org>; 'Xen upstream folks with an interest: Andrew Cooper'
> <Andrew.Cooper3@citrix.com>; pkg-xen-devel@lists.alioth.debian.org; xen-devel@lists.xenproject.org;
> 'My Xen upstream tools co-maintainer: Wei Liu' <wl@xen.org>
> Subject: Re: qemu and Xen ABI-unstable libs
> 
> On Mon, Sep 21, 2020 at 08:36:55AM +0100, Paul Durrant wrote:
> > > -----Original Message-----
> > > From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Ian Jackson
> > > Sent: 18 September 2020 17:39
> > > To: Debian folks: Michael Tokarev <mjt@tls.msk.ru>; Hans van Kranenburg <hans@knorrie.org>; Xen
> > > upstream folks with an interest: Andrew Cooper <Andrew.Cooper3@citrix.com>; Roger Pau Monné
> > > <roger.pau@citrix.com>
> > > Cc: pkg-xen-devel@lists.alioth.debian.org; xen-devel@lists.xenproject.org; My Xen upstream tools
> co-
> > > maintainer: Wei Liu <wl@xen.org>
> > > Subject: RFC: qemu and Xen ABI-unstable libs
> > >
> > > Hi all.  Michael Tokarev has been looking into the problem that qemu
> > > is using Xen libraries with usntable ABIs.  We did an experiment to
> > > see which abi-unstable symbols qemu links to, by suppressing libxc
> > > from the link line.  The results are below.[1]
> > >
> > > Things are not looking too bad.  After some discussion on #xendevel I
> > > have tried to summarise the situation for each of the troublesome
> > > symbols.
> > >
> > > Also, we discovered that upstream qemu does not link against any
> > > abi-unstable Xen libraries if PCI passthrough is disabled.
> > >
> > > Please would my Xen colleages correct me if I have made any mistakes.
> > > Michael, I hope this is helpful and clear.
> > >
> > >
> > > In order from easy to hard:
> > >
> > >
> > > xc_domain_shutdown
> > >
> > > This call in qemu needs to be replaced with a call to the existing
> > > function xendevicemodel_shutdown in libxendevicemodel.  I think it is
> > > likely that this call is fixed in qemu upstream.
> > >
> >
> > I just pulled QEMU master and it appears that destroy_hvm_domain() is still calling
> xc_domain_shutdown().
> >
> > >
> > > xc_get_hvm_param
> > >
> > > There are three references in qemu's
> > > xen_get_default_ioreq_server_info, relating to ioreq servers.  These
> > > uses (and perhaps surrounding code at this function's call site)
> > > should be replaced by use of xendevicemodel_create_ioreq_server
> > > etc. from libxendevicemodel.  I think it is likely that this call is
> > > fixed in qemu upstream.
> > >
> >
> > These references are in compat code for Xen < 4.6. Use of (non-default) ioreq server has been
> present in the code for a long time.
> > We can remove them by retiring the compat code.
> >
> > >
> > > xc_physdev_map_pirq
> > > xc_physdev_map_pirq_msi
> > > xc_physdev_unmap_pirq
> > >
> > > These are all small wrappers for the PHYSDEVOP_map_pirq hypercall.
> > > PHYSDEVOP is already reasonably abi-stable at the hypervisor level (in
> > > theory it's versioned, but changing it would break all dom0's).
> >
> > The hypercalls are non-tools and directly called from the Linux kernel code so they are ABI.
> >
> > > These calls could just be provided as-is by a new stable abi
> > > entrypoint.  We think this should probably go in libxendevicemodel.
> > >
> >
> > Rather than simply moving this calls into libxendevicemodel, we should think about their
> interactions with calls such as
> > xc_domain_bind_pt_pci_irq() below and maybe have a stable library that actually provides a better
> API/ABI for interrupt
> > mapping/triggering although...
> 
> I've thought the same when speaking with Ian about this, as (for HVM
> passthrough) we use the physdev op to obtain a pirq from a physical
> device interrupt source (a MSI entry in the QEMU case, because the
> legacy interrupt is bound by the toolstack IIRC) and then use that
> pirq to bind it to a guest lapic vector.
> 
> I think in a sense such physical interrupt abstraction (the pirq) is
> helpful in order to simplify the binding, as you don't end up with a
> hypercall with a massive number of parameters to identify both the
> source and destination interrupt data. It's also helpful when the
> guest changes the interrupt binding, as you then only update the guest
> side and keep using the same pirq.
> 
> We might want however to have an interface more specific to
> passthrough, such that the pirqs (or maybe we could just call them
> handles) returned by such interface can only be used with guest
> specific bind hypercalls?

Yes, that sounds sensible: we have functions to allocare/free a pirq handle and then functions to bind/unbind such a handle to guest vector.

> 
> > I've long felt PCI pass-through should not be done by QEMU anyway (not least because we currently
> > have no mechanism for PCI pass-through to PVH domains).
> 
> Having xenpt in tree would be fine IMO. Now we have all the proper
> infrastructure in place to allow different pci devices to be handled
> by different emulators IIRC, which is all that's required for this to
> work correctly.

To be useful we need a way of selecting pass-through emulator in the toolstack and we also need to re-visit moving the PCI hotplug controller implementation into Xen (and augmenting the ioreq server API to add some sort of unplug request callback). I'll give it some thought.

  Paul

> 
> Thanks, Roger.