[PATCH More PCIFront]

[PATCH More PCIFront]

[Konrad Rzeszutek Wilk]

This set of patches makes the Xen PCI frontend driver work
with MSI and MSI-X interrupts. In essence I've split the patches
in two camps: enabling MSI/MSI-X and disabling. I've also sprinkled
some other fixes amongst them.

The first patch is .. umm, one I already posted, but figured it might
be easier for Jeremy to ingest this whole e-mail thread than hunt a
specific one down:
 [PATCH 1/9] Fix compile error: error: too many arguments to function 'pci_frontend_enable_msi'

The next one removes the first implementation I did for turning MSI on:
 [PATCH 2/9] Revert "To enable MSI devices in a non-privileged PV domain use pci_frontend_enable_msi."

And the next three make it possible to have MSI/MSI-X enabled with a more
general approach:
 [PATCH 3/9] Alter 'xen_create_msi_irq' to respect an PIRQ override when in non-privileged PV mode.
 [PATCH 4/9] Have pci_frontend_enable_[msi|msix] save PIRQ values in a int array.
 [PATCH 5/9] Pass PIRQ values obtained from pci_frontend_enable_[msi|msix] in xen_create_msi_irq.

A tiny little fix to make it more obvious which device is failing in the field:
 [PATCH 6/9] In xen-pcifront.c updated printk(KERN_ERR ... to dev_err macro.

And the next three for disabling MSI/MSI-X support:
 [PATCH 7/9] Copy-n-paste arch_teardown_msi_irqs from msi.c to io_apic.c.
 [PATCH 8/9] Provide a 'xen_teardown_msi_dev' function to be called by 'arch_teardown_msi_irqs'.
 [PATCH 9/9] In pci_frontend_disable_msi do not alter the dev->irq.

With that, I've gotten MSI to work properly. MSI-X _should_ work as well, but
I do not have the hardware on hand to test this yet.

There is one issue for guest domains:
 - 4GB or more hangs it. When running xenctx it looks to be stuck in:
  [<ffffffff816b9043>] panic+0x1c0 
  [<ffffffff81013335>] xen_swiotlb_fixup+0x123 
  A bit of instrumentation shows tat xen_create_continugous_region fails:
	xen_swiotlb_fixup: messed up: ffff880020000000, size: 67108864, rc:-12

  So you don't even get:

 PCI: Warning: Cannot find a gap in the 32bit address range
 PCI: Unassigned devices with 32bit resource registers may break!


And for dom0:
 - No support yet for 'reassign_device' which page aligns the BARs
   on PCI devices.

 - Haven't tested FLR or PCI AER.

[PATCH 1/9] Fix compile error: error: too many arguments to function 'pci_frontend_enable_msi'

[Konrad Rzeszutek Wilk]

Did not have the same set of arguments for the static inline
function when CONFIG_XEN_PCIDEV_FRONTEND was not defined.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/xen/pci.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 57ac283..b72c858 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -41,7 +41,10 @@ int pci_frontend_enable_msix(struct pci_dev *dev,
 			     struct msix_entry *entries, int nvec);
 void pci_frontend_disable_msix(struct pci_dev *dev);
 #else
-static inline int pci_frontend_enable_msi(struct pci_dev *dev) { return -1; }
+static inline int pci_frontend_enable_msi(struct pci_dev *dev, int *pirq)
+{
+	return -1;
+}
 static inline void pci_frontend_disable_msi(struct pci_dev *dev) { }
 static inline int pci_frontend_enable_msix(struct pci_dev *dev,
 					   struct msix_entry *entries, int nvec)
-- 
1.6.2.5

[PATCH 2/9] Revert "To enable MSI devices in a non-privileged PV domain use pci_frontend_enable_msi."

[Konrad Rzeszutek Wilk]

This reverts commit e01bb1c35107412b74f7027ebf1b86a0243f1cd3.

Lets do it a better way.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/xen/events.c |   19 -------------------
 1 files changed, 0 insertions(+), 19 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 344c778..48b806a 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -39,7 +39,6 @@
 #include <asm/sync_bitops.h>
 #include <asm/xen/hypercall.h>
 #include <asm/xen/hypervisor.h>
-#include <asm/xen/pci.h>
 
 #include <xen/xen-ops.h>
 #include <xen/events.h>
@@ -715,24 +714,6 @@ int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc, int type)
 			irq = -1;
 			goto out;
 		}
-	} else {
-		int pirq = 0;
-		/* TODO: MSI-X not yet supported. */
-		if (type == PCI_CAP_ID_MSIX) {
-			dev_warn(&dev->dev, "pcifront: MSI-X not supported.\n");
-			rc = -ENODEV;
-		} else
-			rc = pci_frontend_enable_msi(dev, &pirq);
-		if (rc) {
-			printk(KERN_WARNING "xen map irq failed %d\n", rc);
-			dynamic_irq_cleanup(irq);
-			irq = -1;
-			goto out;
-		}
-		map_irq.pirq = pirq;
-		map_irq.index = 0;
-		dev_info(&dev->dev, "Allocated IRQ %d (vector: %d)\n",
-				     irq, map_irq.pirq);
 	}
 	irq_info[irq] = mk_pirq_info(0, map_irq.pirq, map_irq.index);
 	if (domid)
-- 
1.6.2.5

[PATCH 3/9] Alter 'xen_create_msi_irq' to respect an PIRQ override when in non-privileged PV mode.

[Konrad Rzeszutek Wilk]

Only privileged domains can get PCI INTx/MSI PIRQ values for a domain. As such,
'xen_create_msi_irq' in non-privileged mode needs to only do kernel IRQ book-keeping.
But, it still needs a PIRQ value to bind to its local IRQ - this PIRQ value is now
being passed in as an argument.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/xen/pci.h |    4 ++--
 arch/x86/xen/pci.c             |    2 +-
 drivers/xen/events.c           |   24 ++++++++++++++++--------
 3 files changed, 19 insertions(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index b72c858..4774ae0 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -5,7 +5,7 @@
 int xen_register_gsi(u32 gsi, int triggering, int polarity);
 int xen_create_msi_irq(struct pci_dev *dev,
 			struct msi_desc *msidesc,
-			int type);
+			int type, int pirq_override);
 int xen_destroy_irq(int irq);
 #else
 static inline int xen_register_gsi(u32 gsi, int triggering, int polarity)
@@ -15,7 +15,7 @@ static inline int xen_register_gsi(u32 gsi, int triggering, int polarity)
 
 static inline int xen_create_msi_irq(struct pci_dev *dev,
 				struct msi_desc *msidesc,
-				int type)
+				int type, int pirq_override)
 {
 	return -1;
 }
diff --git a/arch/x86/xen/pci.c b/arch/x86/xen/pci.c
index 44d91ad..00b8b09 100644
--- a/arch/x86/xen/pci.c
+++ b/arch/x86/xen/pci.c
@@ -94,7 +94,7 @@ int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
 	struct msi_desc *msidesc;
 
 	list_for_each_entry(msidesc, &dev->msi_list, list) {
-		irq = xen_create_msi_irq(dev, msidesc, type);
+		irq = xen_create_msi_irq(dev, msidesc, type, 0 /* for now. */);
 		if (irq < 0)
 			return -1;
 
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 48b806a..b791031 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -662,7 +662,8 @@ static int get_domid_for_dev(char *nodename, char *path, char *val, void *data)
 	return 0;
 }
 
-int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc, int type)
+int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,
+		       int type, int pirq_override)
 {
 	int irq = 0;
 	struct physdev_map_pirq map_irq;
@@ -703,17 +704,24 @@ int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc, int type)
 	if (irq == -1)
 		goto out;
 
-	if (xen_initial_domain()) {
+	/* Only the privileged domain can do this. For non-priv PV domains
+	 * we have to make a call to pci_frontend_* before so that the priv
+	 * domain can do it for us. The 'pirq_override' is its return value. */
+
+	if (xen_initial_domain())
 		rc = HYPERVISOR_physdev_op(PHYSDEVOP_map_pirq, &map_irq);
-		if (rc) {
+	else {
+		rc = pirq_override ? 0 : -ENODEV;
+		map_irq.pirq = pirq_override;
+	}
+	if (rc) {
 
-			printk(KERN_WARNING "xen map irq failed %d\n", rc);
+		printk(KERN_WARNING "xen map irq failed %d\n", rc);
 
-			dynamic_irq_cleanup(irq);
+		dynamic_irq_cleanup(irq);
 
-			irq = -1;
-			goto out;
-		}
+		irq = -1;
+		goto out;
 	}
 	irq_info[irq] = mk_pirq_info(0, map_irq.pirq, map_irq.index);
 	if (domid)
-- 
1.6.2.5

[PATCH 4/9] Have pci_frontend_enable_[msi|msix] save PIRQ values in a int array.

[Konrad Rzeszutek Wilk]

Previously the PIRQ values obtained from do_pci_op would be saved in the
dev->irq (for MSI) or entries[i]->vector (for MSI-X). Unify it to save
those values in an array passed in as an argument.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/xen/pci.h |    8 ++++----
 drivers/pci/xen-pcifront.c     |   26 +++++++++++++++-----------
 2 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 4774ae0..8adb33a 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -35,19 +35,19 @@ static inline int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
 #endif
 #if defined(CONFIG_PCI_MSI) && defined(CONFIG_XEN_PCIDEV_FRONTEND)
 /* Defined in drivers/pci/xen-pcifront.c */
-int pci_frontend_enable_msi(struct pci_dev *dev, int *pirq);
+int pci_frontend_enable_msi(struct pci_dev *dev, int **vectors);
 void pci_frontend_disable_msi(struct pci_dev *dev);
 int pci_frontend_enable_msix(struct pci_dev *dev,
-			     struct msix_entry *entries, int nvec);
+			     int **vectors, int nvec);
 void pci_frontend_disable_msix(struct pci_dev *dev);
 #else
-static inline int pci_frontend_enable_msi(struct pci_dev *dev, int *pirq)
+static inline int pci_frontend_enable_msi(struct pci_dev *dev, int **vectors)
 {
 	return -1;
 }
 static inline void pci_frontend_disable_msi(struct pci_dev *dev) { }
 static inline int pci_frontend_enable_msix(struct pci_dev *dev,
-					   struct msix_entry *entries, int nvec)
+					   int **vectors, int nvec)
 {
 	return -1;
 }
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index adef0c3..1120034 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -12,6 +12,8 @@
 #include <xen/page.h>
 #include <linux/spinlock.h>
 #include <linux/pci.h>
+#include <asm/xen/pci.h>
+#include <linux/msi.h>
 #include <xen/xenbus.h>
 #include <xen/interface/io/pciif.h>
 #include <linux/interrupt.h>
@@ -255,8 +257,7 @@ struct pci_ops pcifront_bus_ops = {
 
 #ifdef CONFIG_PCI_MSI
 int pci_frontend_enable_msix(struct pci_dev *dev,
-		struct msix_entry *entries,
-		int nvec)
+		int **vector, int nvec)
 {
 	int err;
 	int i;
@@ -269,24 +270,28 @@ int pci_frontend_enable_msix(struct pci_dev *dev,
 	};
 	struct pcifront_sd *sd = dev->bus->sysdata;
 	struct pcifront_device *pdev = pcifront_get_pdev(sd);
+	struct msi_desc *entry;
 
 	if (nvec > SH_INFO_MAX_VEC) {
 		printk(KERN_ERR "too much vector for pci frontend%x\n", nvec);
 		return -EINVAL;
 	}
 
-	for (i = 0; i < nvec; i++) {
-		op.msix_entries[i].entry = entries[i].entry;
-		op.msix_entries[i].vector = entries[i].vector;
+	i = 0;
+	list_for_each_entry(entry, &dev->msi_list, list) {
+		op.msix_entries[i].entry = entry->msi_attrib.entry_nr;
+		/* Vector is useless at this point. */
+		op.msix_entries[i].vector = -1;
+		i++;
 	}
 
 	err = do_pci_op(pdev, &op);
 
-	if (!err) {
-		if (!op.value) {
+	if (likely(!err)) {
+		if (likely(!op.value)) {
 			/* we get the result */
 			for (i = 0; i < nvec; i++)
-				entries[i].vector = op.msix_entries[i].vector;
+				*vector[i] = op.msix_entries[i].vector;
 			return 0;
 		} else {
 			printk(KERN_DEBUG "enable msix get value %x\n",
@@ -320,7 +325,7 @@ void pci_frontend_disable_msix(struct pci_dev *dev)
 }
 EXPORT_SYMBOL_GPL(pci_frontend_disable_msix);
 
-int pci_frontend_enable_msi(struct pci_dev *dev, int *pirq)
+int pci_frontend_enable_msi(struct pci_dev *dev, int **vector)
 {
 	int err;
 	struct xen_pci_op op = {
@@ -332,10 +337,9 @@ int pci_frontend_enable_msi(struct pci_dev *dev, int *pirq)
 	struct pcifront_sd *sd = dev->bus->sysdata;
 	struct pcifront_device *pdev = pcifront_get_pdev(sd);
 
-	*pirq = -1;
 	err = do_pci_op(pdev, &op);
 	if (likely(!err)) {
-		*pirq = op.value;
+		*vector[0] = op.value;
 	} else {
 		printk(KERN_ERR "pci frontend enable msi failed for dev %x:%x \n",
 				op.bus, op.devfn);
-- 
1.6.2.5

[PATCH 5/9] Pass PIRQ values obtained from pci_frontend_enable_[msi|msix] in xen_create_msi_irq.

[Konrad Rzeszutek Wilk]

When running in non-privileged PV domain, we will obtain an array of
PIRQ values to be passed in to xen_create_msi_irq. For privileged PV domains
we don't use those values, so we can safely ignore them.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/xen/pci.c |   25 +++++++++++++++++++++----
 1 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/arch/x86/xen/pci.c b/arch/x86/xen/pci.c
index 00b8b09..dc02241 100644
--- a/arch/x86/xen/pci.c
+++ b/arch/x86/xen/pci.c
@@ -90,22 +90,39 @@ void __init xen_setup_pirqs(void)
 #ifdef CONFIG_PCI_MSI
 int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
 {
-	int irq, ret;
+	int irq, ret, i;
 	struct msi_desc *msidesc;
+	int *v;
 
+	v = kzalloc(sizeof(int) * min(1, nvec), GFP_KERNEL);
+	if (!v)
+		return -ENOMEM;
+
+	if (!xen_initial_domain()) {
+		if (type == PCI_CAP_ID_MSIX)
+			ret = pci_frontend_enable_msix(dev, &v, nvec);
+		else
+			ret = pci_frontend_enable_msi(dev, &v);
+		if (ret)
+			goto error;
+	}
+	i = 0;
 	list_for_each_entry(msidesc, &dev->msi_list, list) {
-		irq = xen_create_msi_irq(dev, msidesc, type, 0 /* for now. */);
+		irq = xen_create_msi_irq(dev, msidesc, type, v[i]);
 		if (irq < 0)
 			return -1;
 
 		ret = set_irq_msi(irq, msidesc);
 		if (ret)
-			goto error;
+			goto error_while;
 	}
+	kfree(v);
 	return 0;
 
-error:
+error_while:
 	xen_destroy_irq(irq);
+error:
+	kfree(v);
 	return ret;
 }
 #endif
-- 
1.6.2.5

[PATCH 6/9] In xen-pcifront.c updated printk(KERN_ERR ... to dev_err macro.

[Konrad Rzeszutek Wilk]

Added much more verbose information to aid in troubleshooting
in the field.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/xen-pcifront.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index 1120034..ca95b23 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -273,7 +273,8 @@ int pci_frontend_enable_msix(struct pci_dev *dev,
 	struct msi_desc *entry;
 
 	if (nvec > SH_INFO_MAX_VEC) {
-		printk(KERN_ERR "too much vector for pci frontend%x\n", nvec);
+		dev_err(&dev->dev, "too much vector for pci frontend: %x."
+				   " Increase SH_INFO_MAX_VEC.\n", nvec);
 		return -EINVAL;
 	}
 
@@ -299,7 +300,7 @@ int pci_frontend_enable_msix(struct pci_dev *dev,
 			return op.value;
 		}
 	} else {
-		printk(KERN_ERR "enable msix get err %x\n", err);
+		dev_err(&dev->dev, "enable msix get err %x\n", err);
 		return err;
 	}
 }
@@ -321,7 +322,7 @@ void pci_frontend_disable_msix(struct pci_dev *dev)
 
 	/* What should do for error ? */
 	if (err)
-		printk(KERN_ERR "pci_disable_msix get err %x\n", err);
+		dev_err(&dev->dev, "pci_disable_msix get err %x\n", err);
 }
 EXPORT_SYMBOL_GPL(pci_frontend_disable_msix);
 
@@ -341,8 +342,8 @@ int pci_frontend_enable_msi(struct pci_dev *dev, int **vector)
 	if (likely(!err)) {
 		*vector[0] = op.value;
 	} else {
-		printk(KERN_ERR "pci frontend enable msi failed for dev %x:%x \n",
-				op.bus, op.devfn);
+		dev_err(&dev->dev, "pci frontend enable msi failed for dev "
+				   "%x:%x \n", op.bus, op.devfn);
 		err = -EINVAL;
 	}
 	return err;
-- 
1.6.2.5

[PATCH 7/9] Copy-n-paste arch_teardown_msi_irqs from msi.c to io_apic.c.

[Konrad Rzeszutek Wilk]

In preparation for non-privileged domains to disable PCI devices'
MSI/MSIx, we need to augment arch_teardown_msi_irqs to make
a call to the privileged domain (patch to follow).

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/pci.h     |    1 +
 arch/x86/kernel/apic/io_apic.c |   14 ++++++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
index 70458d6..1a61dbb 100644
--- a/arch/x86/include/asm/pci.h
+++ b/arch/x86/include/asm/pci.h
@@ -93,6 +93,7 @@ extern void pci_iommu_alloc(void);
 
 /* MSI arch hook */
 #define arch_setup_msi_irqs arch_setup_msi_irqs
+#define arch_teardown_msi_irqs arch_teardown_msi_irqs
 
 #define PCI_DMA_BUS_IS_PHYS (dma_ops->is_phys)
 
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 77151ce..0cc2dde 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3572,6 +3572,20 @@ void arch_teardown_msi_irq(unsigned int irq)
 		destroy_irq(irq);
 }
 
+void arch_teardown_msi_irqs(struct pci_dev *dev)
+{
+	struct msi_desc *entry;
+
+	list_for_each_entry(entry, &dev->msi_list, list) {
+		int i, nvec;
+		if (entry->irq == 0)
+			continue;
+		nvec = 1 << entry->msi_attrib.multiple;
+		for (i = 0; i < nvec; i++)
+			arch_teardown_msi_irq(entry->irq + i);
+	}
+}
+
 #if defined (CONFIG_DMAR) || defined (CONFIG_INTR_REMAP)
 #ifdef CONFIG_SMP
 static int dmar_msi_set_affinity(unsigned int irq, const struct cpumask *mask)
-- 
1.6.2.5

[PATCH 8/9] Provide a 'xen_teardown_msi_dev' function to be called by 'arch_teardown_msi_irqs'.

[Konrad Rzeszutek Wilk]

The 'xen_teardown_msi_dev' will call pci_frontend_disable_[msi|msix] when
running in non-privileged PV mode. In all other cases it is effectively a NOP.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 arch/x86/include/asm/xen/pci.h |    2 ++
 arch/x86/kernel/apic/io_apic.c |    5 +++++
 arch/x86/xen/pci.c             |   16 ++++++++++++++++
 3 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/arch/x86/include/asm/xen/pci.h b/arch/x86/include/asm/xen/pci.h
index 8adb33a..6c022c8 100644
--- a/arch/x86/include/asm/xen/pci.h
+++ b/arch/x86/include/asm/xen/pci.h
@@ -7,6 +7,7 @@ int xen_create_msi_irq(struct pci_dev *dev,
 			struct msi_desc *msidesc,
 			int type, int pirq_override);
 int xen_destroy_irq(int irq);
+void xen_teardown_msi_dev(struct pci_dev *dev);
 #else
 static inline int xen_register_gsi(u32 gsi, int triggering, int polarity)
 {
@@ -23,6 +24,7 @@ static inline int xen_destroy_irq(int irq)
 {
 	return -1;
 }
+static inline void xen_teardown_msi_dev(struct pci_dev *dev) { }
 #endif
 
 #if defined(CONFIG_PCI_MSI) && defined(CONFIG_XEN_DOM0_PCI)
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 0cc2dde..b9a0e67 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -3576,6 +3576,11 @@ void arch_teardown_msi_irqs(struct pci_dev *dev)
 {
 	struct msi_desc *entry;
 
+	/* If we are non-privileged PV domain, we have to
+	* to call xen_teardown_msi_dev first. */
+	if (xen_domain())
+		xen_teardown_msi_dev(dev);
+
 	list_for_each_entry(entry, &dev->msi_list, list) {
 		int i, nvec;
 		if (entry->irq == 0)
diff --git a/arch/x86/xen/pci.c b/arch/x86/xen/pci.c
index dc02241..b572f26 100644
--- a/arch/x86/xen/pci.c
+++ b/arch/x86/xen/pci.c
@@ -125,4 +125,20 @@ error:
 	kfree(v);
 	return ret;
 }
+
+void xen_teardown_msi_dev(struct pci_dev *dev)
+{
+	/* Only do this when were are in non-privileged mode.*/
+	if (!xen_initial_domain()) {
+		struct msi_desc *msidesc;
+
+		msidesc = list_entry(dev->msi_list.next, struct msi_desc, list);
+		if (msidesc->msi_attrib.is_msix)
+			pci_frontend_disable_msix(dev);
+		else
+			pci_frontend_disable_msi(dev);
+	}
+
+}
+
 #endif
-- 
1.6.2.5

[PATCH 9/9] In pci_frontend_disable_msi do not alter the dev->irq.

[Konrad Rzeszutek Wilk]

The MSI subsystem keeps track of the IRQ number before enabling
MSI and it will write the old IRQ once it has completed disabling
MSI functionality of the device.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/pci/xen-pcifront.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index ca95b23..cc3b51b 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -368,9 +368,7 @@ void pci_frontend_disable_msi(struct pci_dev *dev)
 		printk(KERN_DEBUG "get no response from backend for disable MSI\n");
 		return;
 	}
-	if (likely(!err))
-		dev->irq = op.value;
-	else
+	if (err)
 		/* how can pciback notify us fail? */
 		printk(KERN_DEBUG "get fake response frombackend \n");
 }
-- 
1.6.2.5