[Qemu-devel] [PULL 0/1] VFIO fixes 2016-09-15

[Qemu-devel] [PULL 0/1] VFIO fixes 2016-09-15

[Alex Williamson]

The following changes since commit 9f16390cd3cdd85fa20739f07120f4d697c11837:

  Merge remote-tracking branch 'remotes/kraxel/tags/pull-usb-20160914-1' into
staging (2016-09-15 14:56:36 +0100)

are available in the git repository at:


  git://github.com/awilliam/qemu-vfio.git tags/vfio-fixes-20160915.0

for you to fetch changes up to 6d17a018d09801a2b18133a4febd81433bb0cf85:

  vfio/pci: Fix regression in MSI routing configuration (2016-09-15 10:41:36
-0600)

----------------------------------------------------------------
VFIO fixes 2016-09-15

Fix a 2.7.0 regression affecting POWER8 systems in relation to EEH,
possibly extending to subtle changes for other devices and archs.
(David Gibson)

----------------------------------------------------------------
David Gibson (1):
      vfio/pci: Fix regression in MSI routing configuration

 hw/vfio/pci.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

[Qemu-devel] [PULL 1/1] vfio/pci: Fix regression in MSI routing configuration

[Alex Williamson]

From: David Gibson <david@gibson.dropbear.id.au>

d1f6af6 "kvm-irqchip: simplify kvm_irqchip_add_msi_route" was a cleanup
of kvmchip routing configuration, that was mostly intended for x86.
However, it also contains a subtle change in behaviour which breaks EEH[1]
error recovery on certain VFIO passthrough devices on spapr guests.  So far
it's only been seen on a BCM5719 NIC on a POWER8 server, but there may be
other hardware with the same problem.  It's also possible there could be
circumstances where it causes a bug on x86 as well, though I don't know of
any obvious candidates.

Prior to d1f6af6, both vfio_msix_vector_do_use() and
vfio_add_kvm_msi_virq() used msg == NULL as a special flag to mark this
as the "dummy" vector used to make the host hardware state sync with the
guest expected hardware state in terms of MSI configuration.

Specifically that flag caused vfio_add_kvm_msi_virq() to become a no-op,
meaning the dummy irq would always be delivered via qemu. d1f6af6 changed
vfio_add_kvm_msi_virq() so it takes a vector number instead of the msg
parameter, and determines the correct message itself.  The test for !msg
was removed, and not replaced with anything there or in the caller.

With an spapr guest which has a VFIO device, if an EEH error occurs on the
host hardware, then the device will be isolated then reset.  This is a
combination of host and guest action, mediated by some EEH related
hypercalls.  I haven't fully traced the mechanics, but somehow installing
the kvm irqchip route for the dummy irq on the BCM5719 means that after EEH
reset and recovery, at least some irqs are no longer delivered to the
guest.

In particular, the guest never gets the link up event, and so the NIC is
effectively dead.

[1] EEH (Enhanced Error Handling) is an IBM POWER server specific PCI-*
    error reporting and recovery mechanism.  The concept is somewhat
    similar to PCI-E AER, but the details are different.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1373802

Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Gavin Shan <gwshan@au1.ibm.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Cc: qemu-stable@nongnu.org
Fixes: d1f6af6a17a6 ("kvm-irqchip: simplify kvm_irqchip_add_msi_route")
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/vfio/pci.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 7bfa17c..a5a620a 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -496,7 +496,9 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
             vfio_update_kvm_msi_virq(vector, *msg, pdev);
         }
     } else {
-        vfio_add_kvm_msi_virq(vdev, vector, nr, true);
+        if (msg) {
+            vfio_add_kvm_msi_virq(vdev, vector, nr, true);
+        }
     }
 
     /*

Re: [Qemu-devel] [PULL 0/1] VFIO fixes 2016-09-15

[Peter Maydell]

On 15 September 2016 at 19:24, Alex Williamson
<alex.williamson@redhat.com> wrote:
> The following changes since commit 9f16390cd3cdd85fa20739f07120f4d697c11837:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/pull-usb-20160914-1' into
> staging (2016-09-15 14:56:36 +0100)
>
> are available in the git repository at:
>
>
>   git://github.com/awilliam/qemu-vfio.git tags/vfio-fixes-20160915.0
>
> for you to fetch changes up to 6d17a018d09801a2b18133a4febd81433bb0cf85:
>
>   vfio/pci: Fix regression in MSI routing configuration (2016-09-15 10:41:36
> -0600)
>
> ----------------------------------------------------------------
> VFIO fixes 2016-09-15
>
> Fix a 2.7.0 regression affecting POWER8 systems in relation to EEH,
> possibly extending to subtle changes for other devices and archs.
> (David Gibson)
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM