Re: [RFC PATCH 07/12] hvmloader: allocate MMCONFIG area in the MMIO hole + minor code refactoring

[Alexey G <x1917x@gmail.com>]

On Thu, 22 Mar 2018 09:29:44 +0000
Paul Durrant <Paul.Durrant@citrix.com> wrote:

>> -----Original Message-----
[...]
>> >In both cases Xen would have to do the MCFG access decoding in order
>> >to figure out which IOREQ server will handle the request. At which
>> >point the only step that you avoid is the reconstruction of the
>> >memory access from the IOREQ_TYPE_PCI_CONFIG which is trivial.  
>> 
>> The "reconstruction of the memory access" you mentioned won't be easy
>> actually. The thing is, address_space_read/write is not all what we
>> need.
>> 
>> In order to translate PCI conf ioreqs back to emulated MMIO ops, we
>> need to be an involved party, mainly to know where MMCONFIG area is
>> located so we can construct the address within its range from BDF.
>> This piece of information is destroyed in the process of MMIO ioreq
>> translation to PCI conf type.
>> 
>> The code which parse PCI conf ioreqs in xen-hvm.c doesn't know
>> anything about the current emulated MMCONFIG state. The correct way
>> to have this info is to participate in its emulation. As we don't
>> participate, we have no other way than trying to gain backdoor
>> access to PCIHost fields via things like object_resolve_*(). This
>> solution is cumbersome and ugly but will work... and may break
>> anytime due to changes in QEMU.
>> 
>> QEMU maintainers will grin while looking at all this I'm afraid --
>> trapped MMIO accesses which are translated to PCI conf accesses which
>> in turn translated back to emulated MMIO accesses upon receiving,
>> along with tedious attempts to gain access to MMCONFIG-related info
>> as we're not invited to the MMCONFIG emulation party.
>> 
>> The more I think about it, the more I like the existing
>> map_io_range_to_ioreq_server() approach. :( It works without doing
>> anything, no hacks, no new interfaces, both MMCONFIG and CF8/CFC are
>> working as expected. There is a problem to make it compatible with
>> the specific multiple ioreq servers feature, but providing a new
>> dmop/hypercall (which you suggest is a must have thing to trap
>> MMCONFIG MMIO to give QEMU only the freedom to tell where it is
>> located) allows to solve this problem in any possible way, either
>> MMIO -> PCI conf translation or anything else.
>>   
>
>I don't think we even want QEMU to have the freedom to say where the
>MMCONFIG areas are located, do we? QEMU is not in charge of the guest
>memory map and it is not responsible for the building the MCFG table,
>Xen is. So it should be Xen that decides where the MMCONFIG area goes
>for each registered PCI device and it should be Xen that adds that to
>the MCFG table. It should be Xen that handles the MMCONFIG MMIO
>accesses and these should be forwarded to QEMU as PCI config IOREQs.
>Now, it may be that we need to introduce a Xen specific mechanism into
>QEMU to then route those config space transactions to the device
>models but that would be an improvement over the current cf8/cfc
>hackery anyway.

Well, MMCONFIG is a chipset-specific thing. We probably can't simply
abstract its usage, merely providing ACPI MCFG table for it.

Its layout must correspond to the emulated PCI conf space where the
majority of devices belong to QEMU. Although we can track all QEMU's
usage of emulated/PT PCI devices and build this layout themselves, this
design may introduce multiple issues. For QEMU handling of such PCI
conf ioreq without knowing anything about MMCONFIG will become worse --
previously he at least knew that those belong to the MMCONFIG range he
emulates, but in case of PCI conf ioreqs situation gets a bit more
complicated -- either CF8/CFC workaround or manual lookup of the target
device from rather isolated xen-hvm.c. Feasible, yes, but will look like
a dirty hack -- doing part of QEMU's internal job.

These are merely inconveniences, main problem here at the moment is
OVMF. OVMF does MMCONFIG relocation by writing to PCIEXBAR he knows
about on Q35, followed by using it at the address he expects. This is
something I want to address in subsequent patches.

>  Paul
>
>> >> We can still route either ioreq
>> >> type to multiple device emulators accordingly.  
>> >
>> >It's exactly the same that's done for IO space PCI config space
>> >addresses. QEMU gets an IOREQ_TYPE_PCI_CONFIG and it replays the IO
>> >space access using do_outp and cpu_ioreq_pio.  
>> 
>> ...And it is completely limited to basic PCI conf space. I don't know
>> the context of this line in xen-hvm.c:
>> 
>> val = (1u << 31) | ((req->addr & 0x0f00) << 16) | ((sbdf & 0xffff)
>> << 8) | (req->addr & 0xfc);
>> 
>> but seems like current QEMU versions do not expect anything similar
>> to AMD ECS-style accesses for 0CF8h. It is limited to basic PCI conf
>> only. 
>> >If you think using IOREQ_TYPE_COPY for MCFG accesses is such a
>> >benefit for QEMU, why not just translate the IOREQ_TYPE_PCI_CONFIG
>> >into IOREQ_TYPE_COPY in handle_ioreq and dispatch it using
>> >cpu_ioreq_move?  
>> 
>> Answered above, we need to somehow have access to the info which
>> don't belong to us for this step.
>>   
>> >Thanks, Roger.  


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel