From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52078) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcW4P-0005Yp-Mg for qemu-devel@nongnu.org; Mon, 30 Mar 2015 05:33:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YcW4I-0007Vc-NS for qemu-devel@nongnu.org; Mon, 30 Mar 2015 05:33:21 -0400 Received: from e28smtp08.in.ibm.com ([122.248.162.8]:52737) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcW4I-0007Ue-4N for qemu-devel@nongnu.org; Mon, 30 Mar 2015 05:33:14 -0400 Received: from /spool/local by e28smtp08.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 30 Mar 2015 15:03:10 +0530 Date: Mon, 30 Mar 2015 20:32:54 +1100 From: Gavin Shan Message-ID: <20150330093254.GA4716@shangw> References: <1427348102-5879-1-git-send-email-gwshan@linux.vnet.ibm.com> <1427348102-5879-2-git-send-email-gwshan@linux.vnet.ibm.com> <20150327060024.GF2900@voom.fritz.box> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150327060024.GF2900@voom.fritz.box> Subject: Re: [Qemu-devel] [PATCH v3 1/2] VFIO: Clear stale MSIx table during EEH reset Reply-To: Gavin Shan List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: David Gibson Cc: qemu-devel@nongnu.org, Gavin Shan , agraf@suse.de, alex.williamson@redhat.com, qemu-ppc@nongnu.org On Fri, Mar 27, 2015 at 05:00:25PM +1100, David Gibson wrote: >On Thu, Mar 26, 2015 at 04:35:01PM +1100, Gavin Shan wrote: >> The PCI device MSIx table is cleaned out in hardware after EEH PE >> reset. However, we still hold the stale MSIx entries in QEMU, which >> should be cleared accordingly. Otherwise, we will run into another >> (recursive) EEH error and the PCI devices contained in the PE have >> to be offlined exceptionally. >> >> The patch introduces function vfio_eeh_pe_reset(), which is called >> by sPAPR when asserting hot or fundamental reset, to clear stale MSIx >> table before EEH PE reset so that MSIx table could be restored properly >> after EEH PE reset. >> >> Signed-off-by: Gavin Shan >> --- >> hw/ppc/spapr_pci_vfio.c | 13 +++++++++---- >> hw/vfio/Makefile.objs | 6 +++++- >> hw/vfio/pci-stub.c | 16 ++++++++++++++++ >> hw/vfio/pci.c | 36 ++++++++++++++++++++++++++++++++++++ >> include/hw/vfio/vfio.h | 2 ++ >> 5 files changed, 68 insertions(+), 5 deletions(-) >> create mode 100644 hw/vfio/pci-stub.c >> >> diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c >> index 99a1be5..6fa3afe 100644 >> --- a/hw/ppc/spapr_pci_vfio.c >> +++ b/hw/ppc/spapr_pci_vfio.c >> @@ -151,19 +151,24 @@ static int spapr_phb_vfio_eeh_reset(sPAPRPHBState *sphb, int option) >> switch (option) { >> case RTAS_SLOT_RESET_DEACTIVATE: >> op.op = VFIO_EEH_PE_RESET_DEACTIVATE; >> + ret = vfio_container_ioctl(&svphb->phb.iommu_as, >> + svphb->iommugroupid, >> + VFIO_EEH_PE_OP, &op); >> break; >> case RTAS_SLOT_RESET_HOT: >> - op.op = VFIO_EEH_PE_RESET_HOT; >> + ret = vfio_eeh_pe_reset(&svphb->phb.iommu_as, >> + svphb->iommugroupid, >> + VFIO_EEH_PE_RESET_HOT); >> break; >> case RTAS_SLOT_RESET_FUNDAMENTAL: >> - op.op = VFIO_EEH_PE_RESET_FUNDAMENTAL; >> + ret = vfio_eeh_pe_reset(&svphb->phb.iommu_as, >> + svphb->iommugroupid, >> + VFIO_EEH_PE_RESET_FUNDAMENTAL); >> break; >> default: >> return RTAS_OUT_PARAM_ERROR; >> } >> >> - ret = vfio_container_ioctl(&svphb->phb.iommu_as, svphb->iommugroupid, >> - VFIO_EEH_PE_OP, &op); >> if (ret < 0) { >> return RTAS_OUT_HW_ERROR; >> } >> diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs >> index e31f30e..1b8a065 100644 >> --- a/hw/vfio/Makefile.objs >> +++ b/hw/vfio/Makefile.objs >> @@ -1,4 +1,8 @@ >> ifeq ($(CONFIG_LINUX), y) >> obj-$(CONFIG_SOFTMMU) += common.o >> -obj-$(CONFIG_PCI) += pci.o >> +ifeq ($(CONFIG_PCI), y) >> +obj-y += pci.o >> +else >> +obj-y += pci-stub.o >> +endif >> endif >> diff --git a/hw/vfio/pci-stub.c b/hw/vfio/pci-stub.c >> new file mode 100644 >> index 0000000..f317c1e >> --- /dev/null >> +++ b/hw/vfio/pci-stub.c >> @@ -0,0 +1,16 @@ >> +/* >> + * To include the file on !CONFIG_PCI >> + * >> + * This work is licensed under the terms of the GNU GPL, version 2. See >> + * the COPYING file in the top-level directory. >> + */ >> + >> +#include >> + >> +#include "exec/memory.h" >> +#include "hw/vfio/vfio.h" >> + >> +int vfio_eeh_pe_reset(AddressSpace *as, int32_t groupid, uint32_t option) >> +{ >> + return -1; >> +} > >This doesn't seem quite right. AFAICT the only caller of >vfio_eeh_pe_reset() is in spapr_pci_vfio.c, which is only built if >CONFIG_PCI is enabled. So if there needed to be !PCI stubs, I'd >expect them further up the call stack. > Or we simply drop the stub for !CONFIG_PCI if Alex.W agree. Alex, what's your opnion? Thanks, Gavin >> +int vfio_eeh_pe_reset(AddressSpace *as, int32_t groupid, uint32_t option) >> +{ >> + VFIOGroup *group; >> + VFIODevice *vbasedev; >> + VFIOPCIDevice *vdev; >> + struct vfio_eeh_pe_op op = { >> + .argsz = sizeof(op), >> + .op = option >> + }; >> + >> + group = vfio_get_group(groupid, as); >> + if (!group) { >> + error_report("vfio: group %d not found\n", groupid); >> + return -1; >> + } >> + >> + /* >> + * The MSIx table will be cleaned out by reset. We need >> + * disable it so that it can be reenabled properly. Also, >> + * the cached MSIx table should be cleared as it's not >> + * reflecting the contents in hardware. >> + */ >> + QLIST_FOREACH(vbasedev, &group->device_list, next) { >> + vdev = container_of(vbasedev, VFIOPCIDevice, vbasedev); >> + if (msix_enabled(&vdev->pdev)) { >> + vfio_disable_msix(vdev); >> + } >> + >> + msix_reset(&vdev->pdev); >> + } >> + >> + vfio_put_group(group); >> + >> + return vfio_container_ioctl(as, groupid, VFIO_EEH_PE_OP, &op); >> +} > >This is much better than the vfio_eeh_event stuff(). > >-- >David Gibson | I'll have my music baroque, and my code >david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ > | _way_ _around_! >http://www.ozlabs.org/~dgibson