Re: [RFC PATCH 07/12] hvmloader: allocate MMCONFIG area in the MMIO hole + minor code refactoring

[Paul Durrant <Paul.Durrant@citrix.com>]

> -----Original Message-----
> From: Alexey G [mailto:x1917x@gmail.com]
> Sent: 21 March 2018 22:50
> To: Roger Pau Monne <roger.pau@citrix.com>
> Cc: xen-devel@lists.xenproject.org; Andrew Cooper
> <Andrew.Cooper3@citrix.com>; Ian Jackson <Ian.Jackson@citrix.com>; Jan
> Beulich <jbeulich@suse.com>; Wei Liu <wei.liu2@citrix.com>; Paul Durrant
> <Paul.Durrant@citrix.com>; Anthony Perard <anthony.perard@citrix.com>;
> Stefano Stabellini <sstabellini@kernel.org>
> Subject: Re: [Xen-devel] [RFC PATCH 07/12] hvmloader: allocate MMCONFIG
> area in the MMIO hole + minor code refactoring
> 
> On Wed, 21 Mar 2018 17:15:04 +0000
> Roger Pau Monné <roger.pau@citrix.com> wrote:
> [...]
> >> Above scenario makes it obvious that at least for QEMU the MMIO->PCI
> >> conf translation is a redundant step. Why not to allow specifying
> >> for DM whether it prefers to receive MMCONFIG accesses as native
> >> (MMIO ones) or as translated PCI conf ioreqs?
> >
> >You are just adding an extra level of complexity to an interface
> >that's fairly simple. You register a PCI device using
> >XEN_DMOP_IO_RANGE_PCI and you get IOREQ_TYPE_PCI_CONFIG ioreqs.
> 
> Yes, and it is still needed as we have two distinct (and not equal)
> interfaces to PCI conf space. Apart from 0..FFh range overlapping they
> can be considered very different interfaces. And whether it is a real
> system or emulated -- we can use either one of these two interfaces or
> both.
> 
> For QEMU zero changes are needed to support MMCONFIG MMIO accesses
> if
> they come as MMIO ioreqs. It's just what its MMCONFIG emulation code
> expects.
> Anyway, for (kind of vague) users of the multiple ioreq servers
> capability we can enable MMIO translation to PCI conf ioreqs. Note that
> actually this is an extra step, not forwarding trapped MMCONFIG MMIO
> accesses to the selected device model as is.
> 
> >Getting both IOREQ_TYPE_PCI_CONFIG and IOREQ_TYPE_COPY for PCI
> config
> >space access is misleading.
> 
> These are very different accesses, both in transport and capabilities.
> 
> >In both cases Xen would have to do the MCFG access decoding in order
> >to figure out which IOREQ server will handle the request. At which
> >point the only step that you avoid is the reconstruction of the memory
> >access from the IOREQ_TYPE_PCI_CONFIG which is trivial.
> 
> The "reconstruction of the memory access" you mentioned won't be easy
> actually. The thing is, address_space_read/write is not all what we
> need.
> 
> In order to translate PCI conf ioreqs back to emulated MMIO ops, we
> need to be an involved party, mainly to know where MMCONFIG area is
> located so we can construct the address within its range from BDF.
> This piece of information is destroyed in the process of MMIO ioreq
> translation to PCI conf type.
> 
> The code which parse PCI conf ioreqs in xen-hvm.c doesn't know anything
> about the current emulated MMCONFIG state. The correct way to have this
> info is to participate in its emulation. As we don't participate, we
> have no other way than trying to gain backdoor access to PCIHost fields
> via things like object_resolve_*(). This solution is cumbersome and
> ugly but will work... and may break anytime due to changes in QEMU.
> 
> QEMU maintainers will grin while looking at all this I'm afraid --
> trapped MMIO accesses which are translated to PCI conf accesses which
> in turn translated back to emulated MMIO accesses upon receiving, along
> with tedious attempts to gain access to MMCONFIG-related info as we're
> not invited to the MMCONFIG emulation party.
> 
> The more I think about it, the more I like the existing
> map_io_range_to_ioreq_server() approach. :( It works without doing
> anything, no hacks, no new interfaces, both MMCONFIG and CF8/CFC are
> working as expected. There is a problem to make it compatible with
> the specific multiple ioreq servers feature, but providing a new
> dmop/hypercall (which you suggest is a must have thing to trap MMCONFIG
> MMIO to give QEMU only the freedom to tell where it is located) allows
> to solve this problem in any possible way, either MMIO -> PCI conf
> translation or anything else.
> 

I don't think we even want QEMU to have the freedom to say where the MMCONFIG areas are located, do we? QEMU is not in charge of the guest memory map and it is not responsible for the building the MCFG table, Xen is. So it should be Xen that decides where the MMCONFIG area goes for each registered PCI device and it should be Xen that adds that to the MCFG table. It should be Xen that handles the MMCONFIG MMIO accesses and these should be forwarded to QEMU as PCI config IOREQs.
Now, it may be that we need to introduce a Xen specific mechanism into QEMU to then route those config space transactions to the device models but that would be an improvement over the current cf8/cfc hackery anyway.

  Paul

> >> We can still route either ioreq
> >> type to multiple device emulators accordingly.
> >
> >It's exactly the same that's done for IO space PCI config space
> >addresses. QEMU gets an IOREQ_TYPE_PCI_CONFIG and it replays the IO
> >space access using do_outp and cpu_ioreq_pio.
> 
> ...And it is completely limited to basic PCI conf space. I don't know
> the context of this line in xen-hvm.c:
> 
> val = (1u << 31) | ((req->addr & 0x0f00) << 16) | ((sbdf & 0xffff) << 8)
>        | (req->addr & 0xfc);
> 
> but seems like current QEMU versions do not expect anything similar to
> AMD ECS-style accesses for 0CF8h. It is limited to basic PCI conf only.
> 
> >If you think using IOREQ_TYPE_COPY for MCFG accesses is such a benefit
> >for QEMU, why not just translate the IOREQ_TYPE_PCI_CONFIG into
> >IOREQ_TYPE_COPY in handle_ioreq and dispatch it using
> >cpu_ioreq_move?
> 
> Answered above, we need to somehow have access to the info which don't
> belong to us for this step.
> 
> >Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel