Re: Regression PCI passthrough from 4.5.5 to 4.6.0-rc1

["Roger Pau Monné" <roger.pau@citrix.com>]

[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]

On Mon, Aug 21, 2017 at 11:09:01AM +0200, Andreas Kinzler wrote:
> Hello Roger & Jan,
> 
> > > > Could you please try the patch below and paste the output you get on
> > > > the Xen console?
> > > Output is in attached file. Does it help?
> > Yes, thanks. I'm quite sure the problem is that MSIX table entries are
> > being unmasked before MSIX is enabled, and so Xen is not able to snoop
> > those writes.
> > Just to confirm, can you try the following two debug patches? One is
> > for the hypervisor, the other is for QEMU.
> 
> Attached ZIP file contains hypervisor and qemu log.

Thanks, so the guest is indeed unmasking the MSIX table entries before
MSIX is enabled, and QEMU only registers the entries with Xen once MSIX
is enabled:

Trying to interleave the Xen and QEMU log correctly, I think the
following is the current flow:

[00:05.0] pci_msix_write: Write to MSIX table entry 0 CTRL, masked: 0
[...]
[00:05.0] xen_pt_msixctrl_reg_write: Enabling MSIX, setting up entries
[00:05.0] xen_pt_msix_update_one: Setting up MSIX vector 0 PIRQ: -1 Masked: 0
[00:05.0] msi_msix_setup: Mapping PIRQ for MSIX entry 0
[00:05.0] msi_msix_update: Updating MSI-X with pirq 55 gvec 0x81 gflags 0x1301 (entry: 0)
(XEN) 0000:02:00.0 added entry 0 to msi_list
(XEN) 0000:02:00.0 added to msixtbl list
[...]
[00:05.0] xen_pt_msixctrl_reg_write: enable MSI-X
(XEN) MSIX ctrl write. Enabled: 1 Maskall: 0. Configured entries:
(XEN) 0 host_masked: 0 guest_masked: 1

Xen never detects the MSIX table entry unmask because it happens
before the MSIX is bound to the guest, so the Xen msixtbl handlers are
not aware of this memory region.

The two possible solutions I see to this are:

 - Make QEMU setup the vectors when the table entries are unmasked,
   even if MSIX is not enabled.
 - Provide an hypercall so QEMU can unmask MSIX vectors on behalf of
   the guest. This would be used to unmask the entries if MSIX is
   enabled with table entries already unmasked.

The patch I've sent earlier (which I'm also attaching to this email)
implements the first solution for QEMU.

Can you please give it a try?

Thanks.

[-- Attachment #2: msix_bind.patch --]
[-- Type: text/plain, Size: 789 bytes --]

diff --git a/hw/xen/xen_pt_msi.c b/hw/xen/xen_pt_msi.c
index ff9a79f5d2..dfb8d64654 100644
--- a/hw/xen/xen_pt_msi.c
+++ b/hw/xen/xen_pt_msi.c
@@ -451,8 +451,12 @@ static void pci_msix_write(void *opaque, hwaddr addr,
         }
 
         entry->updated = true;
-    } else if (msix->enabled && entry->updated &&
-               !(val & PCI_MSIX_ENTRY_CTRL_MASKBIT)) {
+    } else if (entry->updated && !(val & PCI_MSIX_ENTRY_CTRL_MASKBIT)) {
+        /*
+         * NB: always register the entries with Xen when the write to the MSIX
+         * table happens, or else Xen won't be able to correctly snoop the
+         * entry control register write, and will fails to unmask the vector.
+         */
         const volatile uint32_t *vec_ctrl;
 
         /*
-- 
2.11.0 (Apple Git-81)


[-- Attachment #3: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel