[PATCH 1/4 RFC] fsl/msi: have msiir register address absolute rather than offset

[PATCH 1/4 RFC] fsl/msi: have msiir register address absolute rather than offset

[Bharat Bhushan]

Having absolute address simplifies the code and also removes the
confusion around feature->msiir_offset and msi_data->msiir_offset.

Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
---
 arch/powerpc/sysdev/fsl_msi.c | 9 +++------
 arch/powerpc/sysdev/fsl_msi.h | 2 +-
 2 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c
index 4bbb4b8..ec3161b 100644
--- a/arch/powerpc/sysdev/fsl_msi.c
+++ b/arch/powerpc/sysdev/fsl_msi.c
@@ -157,7 +157,7 @@ static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
 	if (reg && (len == sizeof(u64)))
 		address = be64_to_cpup(reg);
 	else
-		address = fsl_pci_immrbar_base(hose) + msi_data->msiir_offset;
+		address = msi_data->msiir;
 
 	msg->address_lo = lower_32_bits(address);
 	msg->address_hi = upper_32_bits(address);
@@ -430,18 +430,15 @@ static int fsl_of_msi_probe(struct platform_device *dev)
 				dev->dev.of_node->full_name);
 			goto error_out;
 		}
-		msi->msiir_offset =
-			features->msiir_offset + (res.start & 0xfffff);
 
 		/*
 		 * First read the MSIIR/MSIIR1 offset from dts
 		 * On failure use the hardcode MSIIR offset
 		 */
 		if (of_address_to_resource(dev->dev.of_node, 1, &msiir))
-			msi->msiir_offset = features->msiir_offset +
-					    (res.start & MSIIR_OFFSET_MASK);
+			msi->msiir = res.start + features->msiir_offset;
 		else
-			msi->msiir_offset = msiir.start & MSIIR_OFFSET_MASK;
+			msi->msiir = msiir.start;
 	}
 
 	msi->feature = features->fsl_pic_ip;
diff --git a/arch/powerpc/sysdev/fsl_msi.h b/arch/powerpc/sysdev/fsl_msi.h
index 420cfcb..9b0ab84 100644
--- a/arch/powerpc/sysdev/fsl_msi.h
+++ b/arch/powerpc/sysdev/fsl_msi.h
@@ -34,7 +34,7 @@ struct fsl_msi {
 
 	unsigned long cascade_irq;
 
-	u32 msiir_offset; /* Offset of MSIIR, relative to start of CCSR */
+	phys_addr_t msiir; /* MSIIR Address in CCSR */
 	u32 ibs_shift; /* Shift of interrupt bit select */
 	u32 srs_shift; /* Shift of the shared interrupt register select */
 	void __iomem *msi_regs;
-- 
1.9.3

[PATCH 2/4 RFC] fsl/msi: Move fsl, msi mode specific MSI device search out of main loop

[Bharat Bhushan]

Moving out the specific MSI device search out of main loop. And now
the specific msi device search is placed with other "fsl.msi" specific
code in same function.
This is in preparation to MSI bank partitioning.

Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
---
 arch/powerpc/sysdev/fsl_msi.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c
index ec3161b..32ba1e3 100644
--- a/arch/powerpc/sysdev/fsl_msi.c
+++ b/arch/powerpc/sysdev/fsl_msi.c
@@ -178,7 +178,8 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
 	unsigned int virq;
 	struct msi_desc *entry;
 	struct msi_msg msg;
-	struct fsl_msi *msi_data;
+	struct fsl_msi *msi_data = NULL;
+	bool specific_msi_bank = false;
 
 	if (type == PCI_CAP_ID_MSIX)
 		pr_debug("fslmsi: MSI-X untested, trying anyway.\n");
@@ -199,12 +200,9 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
 				hose->dn->full_name, np->phandle);
 			return -EINVAL;
 		}
-	}
-
-	list_for_each_entry(entry, &pdev->msi_list, list) {
 		/*
-		 * Loop over all the MSI devices until we find one that has an
-		 * available interrupt.
+		 * Loop over all the MSI devices till we find
+		 * specific MSI device.
 		 */
 		list_for_each_entry(msi_data, &msi_head, list) {
 			/*
@@ -215,12 +213,33 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
 			 * has the additional benefit of skipping over MSI
 			 * nodes that are not mapped in the PAMU.
 			 */
-			if (phandle && (phandle != msi_data->phandle))
-				continue;
+			if (phandle == msi_data->phandle) {
+				specific_msi_bank = true;
+				break;
+			}
+		}
 
+		if (!specific_msi_bank) {
+			dev_err(&pdev->dev,
+				"No specific MSI device found for node %s\n",
+				hose->dn->full_name);
+			return -EINVAL;
+		}
+	}
+
+	list_for_each_entry(entry, &pdev->msi_list, list) {
+		if (specific_msi_bank) {
 			hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
-			if (hwirq >= 0)
-				break;
+		} else {
+			/*
+			 * Loop over all the MSI devices until we find one that has an
+			 * available interrupt.
+			 */
+			list_for_each_entry(msi_data, &msi_head, list) {
+				hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
+				if (hwirq >= 0)
+					break;
+			}
 		}
 
 		if (hwirq < 0) {
-- 
1.9.3

[PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel owned devices

[Bharat Bhushan]

With this patch a "context" can allocate a MSI bank and use the
allocated MSI-bank for the devices in that "context".

kernel/host "context" is "NULL", So all devices owned by kernel
will share a MSI bank allocated with "context = NULL.

This patch is in direction to have separate MSI bank for kernel
context and userspace/VM context. We do not want two software
context (kernel and VMs) to share a MSI bank for safe/reliable
interrupts with full isolation. Follow up patch will add interface
to allocate a MSI bank for userspace/VM context.

NOTE: This RFC patch allows only one MSI bank to be allocated for
kernel context. Which seems to be sufficient to me. But if we see this
is limiting some real usecase scanerio then this limitation can be
removed

One issue which still need to addressed is when to free kernel
context allocated MSI bank? Say all MSI capable devices are assigned
to VM/userspace then there is no need to have any MSI bank reserved
for kernel context.

Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
---
 arch/powerpc/sysdev/fsl_msi.c | 88 ++++++++++++++++++++++++++++++++++++++-----
 arch/powerpc/sysdev/fsl_msi.h |  4 ++
 2 files changed, 83 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c
index 32ba1e3..027aeeb 100644
--- a/arch/powerpc/sysdev/fsl_msi.c
+++ b/arch/powerpc/sysdev/fsl_msi.c
@@ -142,6 +142,79 @@ static void fsl_teardown_msi_irqs(struct pci_dev *pdev)
 	return;
 }
 
+/*
+ * Allocate a MSI Bank for the requested "context".
+ * NULL "context" means that this request is to allocate
+ * MSI bank for kernel owned devices. And currently we
+ * assume that one MSI bank is sufficient for kernel.
+ */
+static struct fsl_msi *fsl_msi_allocate_msi_bank(void *context)
+{
+	struct fsl_msi *msi_data;
+
+	/* Kernel context (NULL) can reserve only one msi bank */
+	if (!context) {
+		list_for_each_entry(msi_data, &msi_head, list) {
+			if ((msi_data->reserved == MSI_RESERVED) &&
+			    (msi_data->context == NULL))
+				return NULL;
+		}
+	}
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if (msi_data->reserved == MSI_FREE) {
+			msi_data->reserved = MSI_RESERVED;
+			msi_data->context = context;
+			return msi_data;
+		}
+	}
+
+	return NULL;
+}
+
+/* FIXME: Assumption that host kernel will allocate only one MSI bank */
+ __attribute__ ((unused)) static int fsl_msi_free_msi_bank(void *context)
+{
+	struct fsl_msi *msi_data;
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if ((msi_data->reserved == MSI_RESERVED) &&
+		     (msi_data->context == context)) {
+			msi_data->reserved = MSI_FREE;
+			msi_data->context = NULL;
+			return 0;
+		}
+	}
+	return -ENODEV;
+}
+
+/*  This API returns the allocated MSI bank of "context"
+ *  to which "pdev" device belongs.
+ *  All kernel owned devices have NULL context. All devices
+ *  in same "context" will share the allocated MSI bank.
+ *
+ *  Note: If no MSI bank allocated to kernel context then
+ *  we allocate a MSI bank here.
+ */
+static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev)
+{
+	struct fsl_msi *msi_data = NULL;
+	void *context = NULL;
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if ((msi_data->reserved == MSI_RESERVED) &&
+		    (msi_data->context == context))
+			return msi_data;
+	}
+
+	/* If no MSI bank allocated for kernel owned device, allocate one */
+	msi_data = fsl_msi_allocate_msi_bank(NULL);
+	if (msi_data)
+		return msi_data;
+
+	return NULL;
+}
+
 static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
 				struct msi_msg *msg,
 				struct fsl_msi *fsl_msi_data)
@@ -174,7 +247,7 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
 	struct pci_controller *hose = pci_bus_to_host(pdev->bus);
 	struct device_node *np;
 	phandle phandle = 0;
-	int rc, hwirq = -ENOMEM;
+	int rc = -ENODEV, hwirq = -ENOMEM;
 	unsigned int virq;
 	struct msi_desc *entry;
 	struct msi_msg msg;
@@ -231,15 +304,12 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
 		if (specific_msi_bank) {
 			hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
 		} else {
-			/*
-			 * Loop over all the MSI devices until we find one that has an
-			 * available interrupt.
-			 */
-			list_for_each_entry(msi_data, &msi_head, list) {
-				hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
-				if (hwirq >= 0)
-					break;
+			msi_data = fsl_msi_get_reserved_msi_bank(pdev);
+			if (!msi_data) {
+				dev_err(&pdev->dev, "No MSI Bank allocated\n");
+				goto out_free;
 			}
+			hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
 		}
 
 		if (hwirq < 0) {
diff --git a/arch/powerpc/sysdev/fsl_msi.h b/arch/powerpc/sysdev/fsl_msi.h
index 9b0ab84..c69702b 100644
--- a/arch/powerpc/sysdev/fsl_msi.h
+++ b/arch/powerpc/sysdev/fsl_msi.h
@@ -46,6 +46,10 @@ struct fsl_msi {
 	struct list_head list;          /* support multiple MSI banks */
 
 	phandle phandle;
+#define MSI_FREE		0
+#define MSI_RESERVED		1
+        int reserved;
+        void *context;
 };
 
 #endif /* _POWERPC_SYSDEV_FSL_MSI_H */
-- 
1.9.3

[PATCH 4/4 RFC] fsl/msi: Add interface to reserve/free msi bank

[Bharat Bhushan]

This patch allows a context (different from kernel context)
to reserve a MSI bank for itself. And then the devices in the
context will share the MSI bank.

VFIO meta driver is one of typical user of these APIs. It will
reserve a MSI bank for MSI interrupt support of direct assignment
PCI devices to a Guest. Patches for same will follow this patch.

Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
---
 arch/powerpc/include/asm/device.h  |   2 +
 arch/powerpc/include/asm/fsl_msi.h |  26 ++++++
 arch/powerpc/sysdev/fsl_msi.c      | 169 +++++++++++++++++++++++++++++++------
 arch/powerpc/sysdev/fsl_msi.h      |   1 +
 4 files changed, 173 insertions(+), 25 deletions(-)
 create mode 100644 arch/powerpc/include/asm/fsl_msi.h

diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
index 38faede..1c2bfd7 100644
--- a/arch/powerpc/include/asm/device.h
+++ b/arch/powerpc/include/asm/device.h
@@ -40,6 +40,8 @@ struct dev_archdata {
 #ifdef CONFIG_FAIL_IOMMU
 	int fail_iommu;
 #endif
+
+	void *context;
 };
 
 struct pdev_archdata {
diff --git a/arch/powerpc/include/asm/fsl_msi.h b/arch/powerpc/include/asm/fsl_msi.h
new file mode 100644
index 0000000..e9041c2
--- /dev/null
+++ b/arch/powerpc/include/asm/fsl_msi.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2014 Freescale Semiconductor, Inc. All rights reserved.
+ *
+ * Author: Bharat Bhushan <bharat.bhushan@freescale.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#ifndef _POWERPC_FSL_MSI_H
+#define _POWERPC_FSL_MSI_H
+
+extern int fsl_msi_set_msi_bank_region(struct iommu_domain *domain,
+				       void *context, int win,
+				       dma_addr_t iova, int prot);
+extern int fsl_msi_clear_msi_bank_region(struct iommu_domain *domain,
+					 struct iommu_group *iommu_group,
+					 int win, dma_addr_t iova);
+extern struct fsl_msi *fsl_msi_reserve_msi_bank(void *context);
+extern int fsl_msi_unreserve_msi_bank(void *context);
+extern int fsl_msi_set_msi_bank_in_dev(struct device *dev, void *data);
+
+#endif /* _POWERPC_FSL_MSI_H */
diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c
index 027aeeb..75cd196 100644
--- a/arch/powerpc/sysdev/fsl_msi.c
+++ b/arch/powerpc/sysdev/fsl_msi.c
@@ -25,6 +25,7 @@
 #include <asm/ppc-pci.h>
 #include <asm/mpic.h>
 #include <asm/fsl_hcalls.h>
+#include <linux/iommu.h>
 
 #include "fsl_msi.h"
 #include "fsl_pci.h"
@@ -172,22 +173,6 @@ static struct fsl_msi *fsl_msi_allocate_msi_bank(void *context)
 	return NULL;
 }
 
-/* FIXME: Assumption that host kernel will allocate only one MSI bank */
- __attribute__ ((unused)) static int fsl_msi_free_msi_bank(void *context)
-{
-	struct fsl_msi *msi_data;
-
-	list_for_each_entry(msi_data, &msi_head, list) {
-		if ((msi_data->reserved == MSI_RESERVED) &&
-		     (msi_data->context == context)) {
-			msi_data->reserved = MSI_FREE;
-			msi_data->context = NULL;
-			return 0;
-		}
-	}
-	return -ENODEV;
-}
-
 /*  This API returns the allocated MSI bank of "context"
  *  to which "pdev" device belongs.
  *  All kernel owned devices have NULL context. All devices
@@ -200,6 +185,12 @@ static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev)
 {
 	struct fsl_msi *msi_data = NULL;
 	void *context = NULL;
+	struct device *dev = &pdev->dev;
+
+	/* Device assigned to userspace if there is valid context */
+	if (dev->archdata.context) {
+		context = dev->archdata.context;
+	}
 
 	list_for_each_entry(msi_data, &msi_head, list) {
 		if ((msi_data->reserved == MSI_RESERVED) &&
@@ -208,13 +199,133 @@ static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev)
 	}
 
 	/* If no MSI bank allocated for kernel owned device, allocate one */
-	msi_data = fsl_msi_allocate_msi_bank(NULL);
-	if (msi_data)
-		return msi_data;
+	if (!context) {
+		msi_data = fsl_msi_allocate_msi_bank(NULL);
+		if (msi_data)
+			return msi_data;
+	}
 
 	return NULL;
 }
 
+/* API to set "context" to which the device belongs */
+int fsl_msi_set_msi_bank_in_dev(struct device *dev, void *data)
+{
+	dev->archdata.context = data;
+	return 0;
+}
+
+/*  This API Allows a MSI bank to be reserved for a "context".
+ *  All devices in same "context" will share the allocated
+ *  MSI bank.
+ *  Typically this function will be called from meta
+ *  driver like VFIO with a valid "context".
+ */
+struct fsl_msi *fsl_msi_reserve_msi_bank(void *context)
+{
+	struct fsl_msi *msi_data;
+
+	if (!context)
+		return NULL;
+
+	/* Check if msi-bank already allocated for the context */
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if (msi_data->reserved == MSI_FREE)
+			continue;
+
+		if (context == msi_data->context)
+			return msi_data;
+	}
+
+	msi_data = fsl_msi_allocate_msi_bank(context);
+	return msi_data;
+}
+
+/* Free reserved MSI bank for a given valid context */
+int fsl_msi_unreserve_msi_bank(void *context)
+{
+	struct fsl_msi *msi_data;
+
+	if (!context)
+		return -EINVAL;
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if ((context == msi_data->context) &&
+		    (msi_data->reserved == MSI_RESERVED)) {
+			msi_data->reserved = MSI_FREE;
+			msi_data->context = NULL;
+			return 0;
+		}
+	}
+	return -ENODEV;
+}
+
+static int is_msi_bank_reserved(struct fsl_msi *msi)
+{
+	return msi->reserved != MSI_FREE;
+}
+
+/*
+ * This function configures PAMU window for MSI page with
+ * given iova. Also same iova will be used as "msi-address"
+ * when configuring msi-message in the devices using this
+ * msi bank.
+ */
+int fsl_msi_set_msi_bank_region(struct iommu_domain *domain,
+				void *context , int win,
+				dma_addr_t iova, int prot)
+{
+	struct fsl_msi *msi_data;
+	dma_addr_t addr;
+	u64 size;
+	int ret;
+
+	if (!context)
+		return -EINVAL;
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if (msi_data->reserved == MSI_FREE)
+			continue;
+
+		if (context != msi_data->context)
+			continue;
+
+		size = PAGE_SIZE;
+		addr = msi_data->msiir & ~(size - 1);
+		ret = iommu_domain_window_enable(domain, win, addr, size, prot);
+		if (ret) {
+			pr_err("%s Error: unable to map msi region\n", __func__);
+			return ret;
+		}
+		msi_data->iova = iova | (msi_data->msiir & (size - 1));
+		return 0;
+	}
+
+	return -ENODEV;
+}
+
+/* This allows to undo what is done in fsl_msi_set_msi_bank_region() */
+int fsl_msi_clear_msi_bank_region(struct iommu_domain *domain, void *context,
+				  int win)
+{
+	struct fsl_msi *msi_data;
+
+	if (!context)
+		return -EINVAL;
+
+	list_for_each_entry(msi_data, &msi_head, list) {
+		if (msi_data->reserved == MSI_FREE)
+			continue;
+
+		if (context == msi_data->context) {
+			iommu_domain_window_disable(domain, win);
+			msi_data->iova = 0;
+			return 0;
+		}
+	}
+	return -ENODEV;
+}
+
 static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
 				struct msi_msg *msg,
 				struct fsl_msi *fsl_msi_data)
@@ -225,12 +336,17 @@ static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
 	int len;
 	const __be64 *reg;
 
-	/* If the msi-address-64 property exists, then use it */
-	reg = of_get_property(hose->dn, "msi-address-64", &len);
-	if (reg && (len == sizeof(u64)))
-		address = be64_to_cpup(reg);
-	else
-		address = msi_data->msiir;
+	if (pdev->dev.archdata.context) {
+		address = msi_data->iova;
+	} else {
+		/* If the msi-address-64 property exists, then use it */
+		reg = of_get_property(hose->dn, "msi-address-64", &len);
+		if (reg && (len == sizeof(u64)))
+			address = be64_to_cpup(reg);
+		else
+			address = fsl_pci_immrbar_base(hose) +
+					(msi_data->msiir & 0xfffff);
+	}
 
 	msg->address_lo = lower_32_bits(address);
 	msg->address_hi = upper_32_bits(address);
@@ -401,6 +517,9 @@ static int fsl_of_msi_remove(struct platform_device *ofdev)
 	struct fsl_msi *msi = platform_get_drvdata(ofdev);
 	int virq, i;
 
+	if (is_msi_bank_reserved(msi))
+		return -EBUSY;
+
 	if (msi->list.prev != NULL)
 		list_del(&msi->list);
 	for (i = 0; i < NR_MSI_REG_MAX; i++) {
diff --git a/arch/powerpc/sysdev/fsl_msi.h b/arch/powerpc/sysdev/fsl_msi.h
index c69702b..7dc6f35 100644
--- a/arch/powerpc/sysdev/fsl_msi.h
+++ b/arch/powerpc/sysdev/fsl_msi.h
@@ -50,6 +50,7 @@ struct fsl_msi {
 #define MSI_RESERVED		1
         int reserved;
         void *context;
+	dma_addr_t iova;
 };
 
 #endif /* _POWERPC_SYSDEV_FSL_MSI_H */
-- 
1.9.3

Re: [PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel owned devices

[Scott Wood]

On Tue, 2015-03-03 at 10:47 +0530, Bharat Bhushan wrote:
> With this patch a "context" can allocate a MSI bank and use the
> allocated MSI-bank for the devices in that "context".
> 
> kernel/host "context" is "NULL", So all devices owned by kernel
> will share a MSI bank allocated with "context = NULL.
> 
> This patch is in direction to have separate MSI bank for kernel
> context and userspace/VM context. We do not want two software
> context (kernel and VMs) to share a MSI bank for safe/reliable
> interrupts with full isolation. Follow up patch will add interface
> to allocate a MSI bank for userspace/VM context.
> 
> NOTE: This RFC patch allows only one MSI bank to be allocated for
> kernel context. Which seems to be sufficient to me. But if we see this
> is limiting some real usecase scanerio then this limitation can be
> removed
> 
> One issue which still need to addressed is when to free kernel
> context allocated MSI bank? Say all MSI capable devices are assigned
> to VM/userspace then there is no need to have any MSI bank reserved
> for kernel context.
> 
> Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
> ---
>  arch/powerpc/sysdev/fsl_msi.c | 88 ++++++++++++++++++++++++++++++++++++++-----
>  arch/powerpc/sysdev/fsl_msi.h |  4 ++
>  2 files changed, 83 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c
> index 32ba1e3..027aeeb 100644
> --- a/arch/powerpc/sysdev/fsl_msi.c
> +++ b/arch/powerpc/sysdev/fsl_msi.c
> @@ -142,6 +142,79 @@ static void fsl_teardown_msi_irqs(struct pci_dev *pdev)
>  	return;
>  }
>  
> +/*
> + * Allocate a MSI Bank for the requested "context".
> + * NULL "context" means that this request is to allocate
> + * MSI bank for kernel owned devices. And currently we
> + * assume that one MSI bank is sufficient for kernel.
> + */
> +static struct fsl_msi *fsl_msi_allocate_msi_bank(void *context)
> +{
> +	struct fsl_msi *msi_data;
> +
> +	/* Kernel context (NULL) can reserve only one msi bank */
> +	if (!context) {
> +		list_for_each_entry(msi_data, &msi_head, list) {
> +			if ((msi_data->reserved == MSI_RESERVED) &&
> +			    (msi_data->context == NULL))
> +				return NULL;
> +		}

Unnecessary parentheses

Is there any reason why the kernel bank needs to participate in this
mechanism at all?  Set it aside at MSI driver init, and don't put it on
the list.  I know you've previously said that you want to support
configs where the kernel doesn't get any banks, but that can just be a
boot option that tells the MSI code to not set aside a bank for the
kernel.  It would simplify the code.

With the patchset as is, how would one indicate whether kernel devices
should get a bank?  Specifically, when the kernel does have an
MSI-capable device but we'd prefer to use legacy interrupts to keep MSIs
available to VFIO.

> +	}
> +
> +	list_for_each_entry(msi_data, &msi_head, list) {
> +		if (msi_data->reserved == MSI_FREE) {
> +			msi_data->reserved = MSI_RESERVED;
> +			msi_data->context = context;
> +			return msi_data;
> +		}
> +	}

What prevents races from parallel callers?

> +/* FIXME: Assumption that host kernel will allocate only one MSI bank */

It's not a FIXME if we think the limitation is not burdensome.

> + __attribute__ ((unused)) static int fsl_msi_free_msi_bank(void *context)

static int __maybe_unused fsl_msi_free_msi_bank(void *context)

Why are you adding this function then removing it in the next patch?

> +	/* If no MSI bank allocated for kernel owned device, allocate one */
> +	msi_data = fsl_msi_allocate_msi_bank(NULL);
> +	if (msi_data)
> +		return msi_data;
> +
> +	return NULL;

return fsl_msi_allocate_msi_bank(NULL);


> +}
> +
>  static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
>  				struct msi_msg *msg,
>  				struct fsl_msi *fsl_msi_data)
> @@ -174,7 +247,7 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
>  	struct pci_controller *hose = pci_bus_to_host(pdev->bus);
>  	struct device_node *np;
>  	phandle phandle = 0;
> -	int rc, hwirq = -ENOMEM;
> +	int rc = -ENODEV, hwirq = -ENOMEM;

Initialize rc when you detect the error instead of here (it'd be OK to
initialize it to zero here, to mitigate potential uninitialized value
bugs).

>  	unsigned int virq;
>  	struct msi_desc *entry;
>  	struct msi_msg msg;
> @@ -231,15 +304,12 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
>  		if (specific_msi_bank) {
>  			hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
>  		} else {
> -			/*
> -			 * Loop over all the MSI devices until we find one that has an
> -			 * available interrupt.
> -			 */
> -			list_for_each_entry(msi_data, &msi_head, list) {
> -				hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
> -				if (hwirq >= 0)
> -					break;
> +			msi_data = fsl_msi_get_reserved_msi_bank(pdev);
> +			if (!msi_data) {
> +				dev_err(&pdev->dev, "No MSI Bank allocated\n");
> +				goto out_free;

Is this really an error?  Seems like dev_info() would be more
appropriate to indicate the absence of a resource where you can fall
back to another option.

> diff --git a/arch/powerpc/sysdev/fsl_msi.h b/arch/powerpc/sysdev/fsl_msi.h
> index 9b0ab84..c69702b 100644
> --- a/arch/powerpc/sysdev/fsl_msi.h
> +++ b/arch/powerpc/sysdev/fsl_msi.h
> @@ -46,6 +46,10 @@ struct fsl_msi {
>  	struct list_head list;          /* support multiple MSI banks */
>  
>  	phandle phandle;
> +#define MSI_FREE		0
> +#define MSI_RESERVED		1
> +        int reserved;
> +        void *context;

Whitespace

How about just "bool reserved"?  Or if the kernel bank is kept off the
list, just check context for NULL.

-Scott

Re: [PATCH 4/4 RFC] fsl/msi: Add interface to reserve/free msi bank

[Scott Wood]

On Tue, 2015-03-03 at 10:47 +0530, Bharat Bhushan wrote:
> This patch allows a context (different from kernel context)
> to reserve a MSI bank for itself. And then the devices in the
> context will share the MSI bank.
> 
> VFIO meta driver is one of typical user of these APIs. It will
> reserve a MSI bank for MSI interrupt support of direct assignment
> PCI devices to a Guest. Patches for same will follow this patch.
> 
> Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com>
> ---
>  arch/powerpc/include/asm/device.h  |   2 +
>  arch/powerpc/include/asm/fsl_msi.h |  26 ++++++
>  arch/powerpc/sysdev/fsl_msi.c      | 169 +++++++++++++++++++++++++++++++------
>  arch/powerpc/sysdev/fsl_msi.h      |   1 +
>  4 files changed, 173 insertions(+), 25 deletions(-)
>  create mode 100644 arch/powerpc/include/asm/fsl_msi.h
> 
> diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
> index 38faede..1c2bfd7 100644
> --- a/arch/powerpc/include/asm/device.h
> +++ b/arch/powerpc/include/asm/device.h
> @@ -40,6 +40,8 @@ struct dev_archdata {
>  #ifdef CONFIG_FAIL_IOMMU
>  	int fail_iommu;
>  #endif
> +
> +	void *context;
>  };

This needs a comment and probably a more specific name.

> @@ -200,6 +185,12 @@ static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev)
>  {
>  	struct fsl_msi *msi_data = NULL;
>  	void *context = NULL;
> +	struct device *dev = &pdev->dev;
> +
> +	/* Device assigned to userspace if there is valid context */
> +	if (dev->archdata.context) {
> +		context = dev->archdata.context;
> +	}

No {}

>  
>  	list_for_each_entry(msi_data, &msi_head, list) {
>  		if ((msi_data->reserved == MSI_RESERVED) &&
> @@ -208,13 +199,133 @@ static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev)
>  	}
>  
>  	/* If no MSI bank allocated for kernel owned device, allocate one */
> -	msi_data = fsl_msi_allocate_msi_bank(NULL);
> -	if (msi_data)
> -		return msi_data;
> +	if (!context) {
> +		msi_data = fsl_msi_allocate_msi_bank(NULL);
> +		if (msi_data)
> +			return msi_data;
> +	}
>  
>  	return NULL;
>  }
>  
> +/* API to set "context" to which the device belongs */
> +int fsl_msi_set_msi_bank_in_dev(struct device *dev, void *data)
> +{
> +	dev->archdata.context = data;
> +	return 0;
> +}

Do we really need "msi" to appear twice in every function name?

> +
> +/*  This API Allows a MSI bank to be reserved for a "context".
> + *  All devices in same "context" will share the allocated
> + *  MSI bank.
> + *  Typically this function will be called from meta
> + *  driver like VFIO with a valid "context".
> + */
> +struct fsl_msi *fsl_msi_reserve_msi_bank(void *context)
> +{
> +	struct fsl_msi *msi_data;
> +
> +	if (!context)
> +		return NULL;
> +
> +	/* Check if msi-bank already allocated for the context */
> +	list_for_each_entry(msi_data, &msi_head, list) {
> +		if (msi_data->reserved == MSI_FREE)
> +			continue;
> +
> +		if (context == msi_data->context)
> +			return msi_data;

The reserved == MSI_FREE check doesn't add anything because if it's free
then the context check will fail.

> +static int is_msi_bank_reserved(struct fsl_msi *msi)

s/int/bool/


> +/*
> + * This function configures PAMU window for MSI page with
> + * given iova. Also same iova will be used as "msi-address"
> + * when configuring msi-message in the devices using this
> + * msi bank.
> + */
> +int fsl_msi_set_msi_bank_region(struct iommu_domain *domain,
> +				void *context , int win,

Whitespace

> @@ -225,12 +336,17 @@ static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq,
>  	int len;
>  	const __be64 *reg;
>  
> -	/* If the msi-address-64 property exists, then use it */
> -	reg = of_get_property(hose->dn, "msi-address-64", &len);
> -	if (reg && (len == sizeof(u64)))
> -		address = be64_to_cpup(reg);
> -	else
> -		address = msi_data->msiir;
> +	if (pdev->dev.archdata.context) {
> +		address = msi_data->iova;
> +	} else {
> +		/* If the msi-address-64 property exists, then use it */
> +		reg = of_get_property(hose->dn, "msi-address-64", &len);
> +		if (reg && (len == sizeof(u64)))
> +			address = be64_to_cpup(reg);
> +		else
> +			address = fsl_pci_immrbar_base(hose) +
> +					(msi_data->msiir & 0xfffff);
> +	}

You removed the call to fsl_pci_immrbar_base in patch 1/4 and are
reintroducing it here.  If this call is needed, how does it work in
between those two patches?

> @@ -401,6 +517,9 @@ static int fsl_of_msi_remove(struct platform_device *ofdev)
>  	struct fsl_msi *msi = platform_get_drvdata(ofdev);
>  	int virq, i;
>  
> +	if (is_msi_bank_reserved(msi))
> +		return -EBUSY;

As I mentioned in discussion of a prior version of this, the driver core
ignores error codes in .remove().  Since there's no reason to remove
this piece of platform infrastructure, and the remove path has probably
never been tested, I'd just replace the whole thing with BUG().

-Scott

RE: [PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel owned devices

[Bharat.Bhushan]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV29vZCBTY290dC1CMDc0
MjENCj4gU2VudDogVGh1cnNkYXksIE1hcmNoIDEyLCAyMDE1IDQ6NTMgQU0NCj4gVG86IEJodXNo
YW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogbGludXhwcGMtZGV2QG96bGFicy5vcmcNCj4gU3ViamVj
dDogUmU6IFtQQVRDSCAzLzQgUkZDXSBmc2wvbXNpOiBBZGQgTVNJIGJhbmsgYWxsb2NhdGlvbiBm
b3Iga2VybmVsDQo+IG93bmVkIGRldmljZXMNCj4gDQo+IE9uIFR1ZSwgMjAxNS0wMy0wMyBhdCAx
MDo0NyArMDUzMCwgQmhhcmF0IEJodXNoYW4gd3JvdGU6DQo+ID4gV2l0aCB0aGlzIHBhdGNoIGEg
ImNvbnRleHQiIGNhbiBhbGxvY2F0ZSBhIE1TSSBiYW5rIGFuZCB1c2UgdGhlDQo+ID4gYWxsb2Nh
dGVkIE1TSS1iYW5rIGZvciB0aGUgZGV2aWNlcyBpbiB0aGF0ICJjb250ZXh0Ii4NCj4gPg0KPiA+
IGtlcm5lbC9ob3N0ICJjb250ZXh0IiBpcyAiTlVMTCIsIFNvIGFsbCBkZXZpY2VzIG93bmVkIGJ5
IGtlcm5lbCB3aWxsDQo+ID4gc2hhcmUgYSBNU0kgYmFuayBhbGxvY2F0ZWQgd2l0aCAiY29udGV4
dCA9IE5VTEwuDQo+ID4NCj4gPiBUaGlzIHBhdGNoIGlzIGluIGRpcmVjdGlvbiB0byBoYXZlIHNl
cGFyYXRlIE1TSSBiYW5rIGZvciBrZXJuZWwNCj4gPiBjb250ZXh0IGFuZCB1c2Vyc3BhY2UvVk0g
Y29udGV4dC4gV2UgZG8gbm90IHdhbnQgdHdvIHNvZnR3YXJlIGNvbnRleHQNCj4gPiAoa2VybmVs
IGFuZCBWTXMpIHRvIHNoYXJlIGEgTVNJIGJhbmsgZm9yIHNhZmUvcmVsaWFibGUgaW50ZXJydXB0
cyB3aXRoDQo+ID4gZnVsbCBpc29sYXRpb24uIEZvbGxvdyB1cCBwYXRjaCB3aWxsIGFkZCBpbnRl
cmZhY2UgdG8gYWxsb2NhdGUgYSBNU0kNCj4gPiBiYW5rIGZvciB1c2Vyc3BhY2UvVk0gY29udGV4
dC4NCj4gPg0KPiA+IE5PVEU6IFRoaXMgUkZDIHBhdGNoIGFsbG93cyBvbmx5IG9uZSBNU0kgYmFu
ayB0byBiZSBhbGxvY2F0ZWQgZm9yDQo+ID4ga2VybmVsIGNvbnRleHQuIFdoaWNoIHNlZW1zIHRv
IGJlIHN1ZmZpY2llbnQgdG8gbWUuIEJ1dCBpZiB3ZSBzZWUgdGhpcw0KPiA+IGlzIGxpbWl0aW5n
IHNvbWUgcmVhbCB1c2VjYXNlIHNjYW5lcmlvIHRoZW4gdGhpcyBsaW1pdGF0aW9uIGNhbiBiZQ0K
PiA+IHJlbW92ZWQNCj4gPg0KPiA+IE9uZSBpc3N1ZSB3aGljaCBzdGlsbCBuZWVkIHRvIGFkZHJl
c3NlZCBpcyB3aGVuIHRvIGZyZWUga2VybmVsIGNvbnRleHQNCj4gPiBhbGxvY2F0ZWQgTVNJIGJh
bms/IFNheSBhbGwgTVNJIGNhcGFibGUgZGV2aWNlcyBhcmUgYXNzaWduZWQgdG8NCj4gPiBWTS91
c2Vyc3BhY2UgdGhlbiB0aGVyZSBpcyBubyBuZWVkIHRvIGhhdmUgYW55IE1TSSBiYW5rIHJlc2Vy
dmVkIGZvcg0KPiA+IGtlcm5lbCBjb250ZXh0Lg0KPiA+DQo+ID4gU2lnbmVkLW9mZi1ieTogQmhh
cmF0IEJodXNoYW4gPEJoYXJhdC5CaHVzaGFuQGZyZWVzY2FsZS5jb20+DQo+ID4gLS0tDQo+ID4g
IGFyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX21zaS5jIHwgODgNCj4gPiArKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKy0tLS0tDQo+ID4gIGFyY2gvcG93ZXJwYy9zeXNkZXYvZnNs
X21zaS5oIHwgIDQgKysNCj4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA4MyBpbnNlcnRpb25zKCspLCA5
IGRlbGV0aW9ucygtKQ0KPiA+DQo+ID4gZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9zeXNkZXYv
ZnNsX21zaS5jDQo+ID4gYi9hcmNoL3Bvd2VycGMvc3lzZGV2L2ZzbF9tc2kuYyBpbmRleCAzMmJh
MWUzLi4wMjdhZWViIDEwMDY0NA0KPiA+IC0tLSBhL2FyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX21z
aS5jDQo+ID4gKysrIGIvYXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfbXNpLmMNCj4gPiBAQCAtMTQy
LDYgKzE0Miw3OSBAQCBzdGF0aWMgdm9pZCBmc2xfdGVhcmRvd25fbXNpX2lycXMoc3RydWN0IHBj
aV9kZXYNCj4gKnBkZXYpDQo+ID4gIAlyZXR1cm47DQo+ID4gIH0NCj4gPg0KPiA+ICsvKg0KPiA+
ICsgKiBBbGxvY2F0ZSBhIE1TSSBCYW5rIGZvciB0aGUgcmVxdWVzdGVkICJjb250ZXh0Ii4NCj4g
PiArICogTlVMTCAiY29udGV4dCIgbWVhbnMgdGhhdCB0aGlzIHJlcXVlc3QgaXMgdG8gYWxsb2Nh
dGUNCj4gPiArICogTVNJIGJhbmsgZm9yIGtlcm5lbCBvd25lZCBkZXZpY2VzLiBBbmQgY3VycmVu
dGx5IHdlDQo+ID4gKyAqIGFzc3VtZSB0aGF0IG9uZSBNU0kgYmFuayBpcyBzdWZmaWNpZW50IGZv
ciBrZXJuZWwuDQo+ID4gKyAqLw0KPiA+ICtzdGF0aWMgc3RydWN0IGZzbF9tc2kgKmZzbF9tc2lf
YWxsb2NhdGVfbXNpX2Jhbmsodm9pZCAqY29udGV4dCkgew0KPiA+ICsJc3RydWN0IGZzbF9tc2kg
Km1zaV9kYXRhOw0KPiA+ICsNCj4gPiArCS8qIEtlcm5lbCBjb250ZXh0IChOVUxMKSBjYW4gcmVz
ZXJ2ZSBvbmx5IG9uZSBtc2kgYmFuayAqLw0KPiA+ICsJaWYgKCFjb250ZXh0KSB7DQo+ID4gKwkJ
bGlzdF9mb3JfZWFjaF9lbnRyeShtc2lfZGF0YSwgJm1zaV9oZWFkLCBsaXN0KSB7DQo+ID4gKwkJ
CWlmICgobXNpX2RhdGEtPnJlc2VydmVkID09IE1TSV9SRVNFUlZFRCkgJiYNCj4gPiArCQkJICAg
IChtc2lfZGF0YS0+Y29udGV4dCA9PSBOVUxMKSkNCj4gPiArCQkJCXJldHVybiBOVUxMOw0KPiA+
ICsJCX0NCj4gDQo+IFVubmVjZXNzYXJ5IHBhcmVudGhlc2VzDQo+IA0KPiBJcyB0aGVyZSBhbnkg
cmVhc29uIHdoeSB0aGUga2VybmVsIGJhbmsgbmVlZHMgdG8gcGFydGljaXBhdGUgaW4gdGhpcw0K
PiBtZWNoYW5pc20gYXQgYWxsPyAgU2V0IGl0IGFzaWRlIGF0IE1TSSBkcml2ZXIgaW5pdCwgYW5k
IGRvbid0IHB1dCBpdCBvbg0KPiB0aGUgbGlzdC4gIEkga25vdyB5b3UndmUgcHJldmlvdXNseSBz
YWlkIHRoYXQgeW91IHdhbnQgdG8gc3VwcG9ydCBjb25maWdzDQo+IHdoZXJlIHRoZSBrZXJuZWwg
ZG9lc24ndCBnZXQgYW55IGJhbmtzLCBidXQgdGhhdCBjYW4ganVzdCBiZSBhIGJvb3QNCj4gb3B0
aW9uIHRoYXQgdGVsbHMgdGhlIE1TSSBjb2RlIHRvIG5vdCBzZXQgYXNpZGUgYSBiYW5rIGZvciB0
aGUga2VybmVsLg0KPiBJdCB3b3VsZCBzaW1wbGlmeSB0aGUgY29kZS4NCg0KSSB0aGluayB3ZSBj
YW5ub3QgY29tcGxldGVseSByZW1vdmUgdGhlIE1TSSBiYW5rIGZyb20ga2VybmVsIHByYWN0aWNh
bGx5LiBTbyBJIGNhbiBtYWtlIHRoZSBrZXJuZWwgbXNpIGJhbmsgb3V0IG9mIGxpc3QuDQoNCj4g
DQo+IFdpdGggdGhlIHBhdGNoc2V0IGFzIGlzLCBob3cgd291bGQgb25lIGluZGljYXRlIHdoZXRo
ZXIga2VybmVsIGRldmljZXMNCj4gc2hvdWxkIGdldCBhIGJhbms/DQoNCkZvciBrZXJuZWwgb3du
ZWQgZGV2aWNlLCBpbiBmc2xfc2V0dXBfbXNpX2lycXMoKSB3ZSBjaGVjayBpZiB0aGVyZSBpcyBh
IHJlc2VydmVkIE1TSSBiYW5rLCBpZiBub3QgdGhlbiByZXNlcnZlIGEgbXNpIGJhbmsuIElmIHJl
c2VydmUgZmFpbHMgdGhlbiBzZXR1cF9tc2lfaXJxKCkgZmFpbHMuIEkgdGhpbmsgdGhlcmUgaXMg
bm8gZmFsbGJhY2sgdG8gTGVnYWN5IGludGVycnVwdC4NCg0KPiAgU3BlY2lmaWNhbGx5LCB3aGVu
IHRoZSBrZXJuZWwgZG9lcyBoYXZlIGFuIE1TSS0NCj4gY2FwYWJsZSBkZXZpY2UgYnV0IHdlJ2Qg
cHJlZmVyIHRvIHVzZSBsZWdhY3kgaW50ZXJydXB0cyB0byBrZWVwIE1TSXMNCj4gYXZhaWxhYmxl
IHRvIFZGSU8uDQoNCkRvIHdlIHdhbnQgdGhpcz8NCg0KPiANCj4gPiArCX0NCj4gPiArDQo+ID4g
KwlsaXN0X2Zvcl9lYWNoX2VudHJ5KG1zaV9kYXRhLCAmbXNpX2hlYWQsIGxpc3QpIHsNCj4gPiAr
CQlpZiAobXNpX2RhdGEtPnJlc2VydmVkID09IE1TSV9GUkVFKSB7DQo+ID4gKwkJCW1zaV9kYXRh
LT5yZXNlcnZlZCA9IE1TSV9SRVNFUlZFRDsNCj4gPiArCQkJbXNpX2RhdGEtPmNvbnRleHQgPSBj
b250ZXh0Ow0KPiA+ICsJCQlyZXR1cm4gbXNpX2RhdGE7DQo+ID4gKwkJfQ0KPiA+ICsJfQ0KPiAN
Cj4gV2hhdCBwcmV2ZW50cyByYWNlcyBmcm9tIHBhcmFsbGVsIGNhbGxlcnM/DQoNCldpbGwgYWRk
IGEgbG9jay4NCg0KPiANCj4gPiArLyogRklYTUU6IEFzc3VtcHRpb24gdGhhdCBob3N0IGtlcm5l
bCB3aWxsIGFsbG9jYXRlIG9ubHkgb25lIE1TSSBiYW5rDQo+ID4gKyovDQo+IA0KPiBJdCdzIG5v
dCBhIEZJWE1FIGlmIHdlIHRoaW5rIHRoZSBsaW1pdGF0aW9uIGlzIG5vdCBidXJkZW5zb21lLg0K
DQpPaw0KDQo+IA0KPiA+ICsgX19hdHRyaWJ1dGVfXyAoKHVudXNlZCkpIHN0YXRpYyBpbnQgZnNs
X21zaV9mcmVlX21zaV9iYW5rKHZvaWQNCj4gPiArICpjb250ZXh0KQ0KPiANCj4gc3RhdGljIGlu
dCBfX21heWJlX3VudXNlZCBmc2xfbXNpX2ZyZWVfbXNpX2Jhbmsodm9pZCAqY29udGV4dCkNCj4g
DQo+IFdoeSBhcmUgeW91IGFkZGluZyB0aGlzIGZ1bmN0aW9uIHRoZW4gcmVtb3ZpbmcgaXQgaW4g
dGhlIG5leHQgcGF0Y2g/DQoNCldpbGwgZml4IHRoaXMNCg0KPiANCj4gPiArCS8qIElmIG5vIE1T
SSBiYW5rIGFsbG9jYXRlZCBmb3Iga2VybmVsIG93bmVkIGRldmljZSwgYWxsb2NhdGUgb25lDQo+
ICovDQo+ID4gKwltc2lfZGF0YSA9IGZzbF9tc2lfYWxsb2NhdGVfbXNpX2JhbmsoTlVMTCk7DQo+
ID4gKwlpZiAobXNpX2RhdGEpDQo+ID4gKwkJcmV0dXJuIG1zaV9kYXRhOw0KPiA+ICsNCj4gPiAr
CXJldHVybiBOVUxMOw0KPiANCj4gcmV0dXJuIGZzbF9tc2lfYWxsb2NhdGVfbXNpX2JhbmsoTlVM
TCk7DQoNCk9rDQoNCj4gDQo+IA0KPiA+ICt9DQo+ID4gKw0KPiA+ICBzdGF0aWMgdm9pZCBmc2xf
Y29tcG9zZV9tc2lfbXNnKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgaHdpcnEsDQo+ID4gIAkJ
CQlzdHJ1Y3QgbXNpX21zZyAqbXNnLA0KPiA+ICAJCQkJc3RydWN0IGZzbF9tc2kgKmZzbF9tc2lf
ZGF0YSkNCj4gPiBAQCAtMTc0LDcgKzI0Nyw3IEBAIHN0YXRpYyBpbnQgZnNsX3NldHVwX21zaV9p
cnFzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LA0KPiBpbnQgbnZlYywgaW50IHR5cGUpDQo+ID4gIAlz
dHJ1Y3QgcGNpX2NvbnRyb2xsZXIgKmhvc2UgPSBwY2lfYnVzX3RvX2hvc3QocGRldi0+YnVzKTsN
Cj4gPiAgCXN0cnVjdCBkZXZpY2Vfbm9kZSAqbnA7DQo+ID4gIAlwaGFuZGxlIHBoYW5kbGUgPSAw
Ow0KPiA+IC0JaW50IHJjLCBod2lycSA9IC1FTk9NRU07DQo+ID4gKwlpbnQgcmMgPSAtRU5PREVW
LCBod2lycSA9IC1FTk9NRU07DQo+IA0KPiBJbml0aWFsaXplIHJjIHdoZW4geW91IGRldGVjdCB0
aGUgZXJyb3IgaW5zdGVhZCBvZiBoZXJlIChpdCdkIGJlIE9LIHRvDQo+IGluaXRpYWxpemUgaXQg
dG8gemVybyBoZXJlLCB0byBtaXRpZ2F0ZSBwb3RlbnRpYWwgdW5pbml0aWFsaXplZCB2YWx1ZQ0K
PiBidWdzKS4NCg0KT2sNCg0KPiANCj4gPiAgCXVuc2lnbmVkIGludCB2aXJxOw0KPiA+ICAJc3Ry
dWN0IG1zaV9kZXNjICplbnRyeTsNCj4gPiAgCXN0cnVjdCBtc2lfbXNnIG1zZzsNCj4gPiBAQCAt
MjMxLDE1ICszMDQsMTIgQEAgc3RhdGljIGludCBmc2xfc2V0dXBfbXNpX2lycXMoc3RydWN0IHBj
aV9kZXYNCj4gKnBkZXYsIGludCBudmVjLCBpbnQgdHlwZSkNCj4gPiAgCQlpZiAoc3BlY2lmaWNf
bXNpX2JhbmspIHsNCj4gPiAgCQkJaHdpcnEgPSBtc2lfYml0bWFwX2FsbG9jX2h3aXJxcygmbXNp
X2RhdGEtPmJpdG1hcCwgMSk7DQo+ID4gIAkJfSBlbHNlIHsNCj4gPiAtCQkJLyoNCj4gPiAtCQkJ
ICogTG9vcCBvdmVyIGFsbCB0aGUgTVNJIGRldmljZXMgdW50aWwgd2UgZmluZCBvbmUgdGhhdA0K
PiBoYXMgYW4NCj4gPiAtCQkJICogYXZhaWxhYmxlIGludGVycnVwdC4NCj4gPiAtCQkJICovDQo+
ID4gLQkJCWxpc3RfZm9yX2VhY2hfZW50cnkobXNpX2RhdGEsICZtc2lfaGVhZCwgbGlzdCkgew0K
PiA+IC0JCQkJaHdpcnEgPSBtc2lfYml0bWFwX2FsbG9jX2h3aXJxcygmbXNpX2RhdGEtDQo+ID5i
aXRtYXAsIDEpOw0KPiA+IC0JCQkJaWYgKGh3aXJxID49IDApDQo+ID4gLQkJCQkJYnJlYWs7DQo+
ID4gKwkJCW1zaV9kYXRhID0gZnNsX21zaV9nZXRfcmVzZXJ2ZWRfbXNpX2JhbmsocGRldik7DQo+
ID4gKwkJCWlmICghbXNpX2RhdGEpIHsNCj4gPiArCQkJCWRldl9lcnIoJnBkZXYtPmRldiwgIk5v
IE1TSSBCYW5rIGFsbG9jYXRlZFxuIik7DQo+ID4gKwkJCQlnb3RvIG91dF9mcmVlOw0KPiANCj4g
SXMgdGhpcyByZWFsbHkgYW4gZXJyb3I/ICBTZWVtcyBsaWtlIGRldl9pbmZvKCkgd291bGQgYmUg
bW9yZSBhcHByb3ByaWF0ZQ0KPiB0byBpbmRpY2F0ZSB0aGUgYWJzZW5jZSBvZiBhIHJlc291cmNl
IHdoZXJlIHlvdSBjYW4gZmFsbCBiYWNrIHRvIGFub3RoZXINCj4gb3B0aW9uLg0KDQpUaGVyZSBp
cyBubyBmYWxsYmFjayBpbiBmc2xfc2V0dXBfbXNpX2lycXMoKSwgV2UgaGF2ZSB0byBlcnJvciBv
dXQgZnJvbSBmc2xfc2V0dXBfbXNpX2lycXMoKSwgbm8/DQoNCj4gDQo+ID4gZGlmZiAtLWdpdCBh
L2FyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX21zaS5oDQo+ID4gYi9hcmNoL3Bvd2VycGMvc3lzZGV2
L2ZzbF9tc2kuaCBpbmRleCA5YjBhYjg0Li5jNjk3MDJiIDEwMDY0NA0KPiA+IC0tLSBhL2FyY2gv
cG93ZXJwYy9zeXNkZXYvZnNsX21zaS5oDQo+ID4gKysrIGIvYXJjaC9wb3dlcnBjL3N5c2Rldi9m
c2xfbXNpLmgNCj4gPiBAQCAtNDYsNiArNDYsMTAgQEAgc3RydWN0IGZzbF9tc2kgew0KPiA+ICAJ
c3RydWN0IGxpc3RfaGVhZCBsaXN0OyAgICAgICAgICAvKiBzdXBwb3J0IG11bHRpcGxlIE1TSSBi
YW5rcyAqLw0KPiA+DQo+ID4gIAlwaGFuZGxlIHBoYW5kbGU7DQo+ID4gKyNkZWZpbmUgTVNJX0ZS
RUUJCTANCj4gPiArI2RlZmluZSBNU0lfUkVTRVJWRUQJCTENCj4gPiArICAgICAgICBpbnQgcmVz
ZXJ2ZWQ7DQo+ID4gKyAgICAgICAgdm9pZCAqY29udGV4dDsNCj4gDQo+IFdoaXRlc3BhY2UNCj4g
DQo+IEhvdyBhYm91dCBqdXN0ICJib29sIHJlc2VydmVkIj8gIE9yIGlmIHRoZSBrZXJuZWwgYmFu
ayBpcyBrZXB0IG9mZiB0aGUNCj4gbGlzdCwganVzdCBjaGVjayBjb250ZXh0IGZvciBOVUxMLg0K
DQpJIHdpbGwgdHJ5IHRvIG1ha2Uga2VybmVsIGJhbmsgb3V0IG9mIGxpc3QgYW5kIHNlZSBob3cg
bXVjaCB0aGlzIHNpbXBsaWZpZXMgdGhlIGNvZGUuDQoNClRoYW5rcw0KLUJoYXJhdA0KDQo+IA0K
PiAtU2NvdHQNCj4gDQoNCg==

RE: [PATCH 4/4 RFC] fsl/msi: Add interface to reserve/free msi bank

[Bharat.Bhushan]

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV29vZCBTY290dC1CMDc0
MjENCj4gU2VudDogVGh1cnNkYXksIE1hcmNoIDEyLCAyMDE1IDU6NDggQU0NCj4gVG86IEJodXNo
YW4gQmhhcmF0LVI2NTc3Nw0KPiBDYzogbGludXhwcGMtZGV2QG96bGFicy5vcmcNCj4gU3ViamVj
dDogUmU6IFtQQVRDSCA0LzQgUkZDXSBmc2wvbXNpOiBBZGQgaW50ZXJmYWNlIHRvIHJlc2VydmUv
ZnJlZSBtc2kNCj4gYmFuaw0KPiANCj4gT24gVHVlLCAyMDE1LTAzLTAzIGF0IDEwOjQ3ICswNTMw
LCBCaGFyYXQgQmh1c2hhbiB3cm90ZToNCj4gPiBUaGlzIHBhdGNoIGFsbG93cyBhIGNvbnRleHQg
KGRpZmZlcmVudCBmcm9tIGtlcm5lbCBjb250ZXh0KSB0byByZXNlcnZlDQo+ID4gYSBNU0kgYmFu
ayBmb3IgaXRzZWxmLiBBbmQgdGhlbiB0aGUgZGV2aWNlcyBpbiB0aGUgY29udGV4dCB3aWxsIHNo
YXJlDQo+ID4gdGhlIE1TSSBiYW5rLg0KPiA+DQo+ID4gVkZJTyBtZXRhIGRyaXZlciBpcyBvbmUg
b2YgdHlwaWNhbCB1c2VyIG9mIHRoZXNlIEFQSXMuIEl0IHdpbGwgcmVzZXJ2ZQ0KPiA+IGEgTVNJ
IGJhbmsgZm9yIE1TSSBpbnRlcnJ1cHQgc3VwcG9ydCBvZiBkaXJlY3QgYXNzaWdubWVudCBQQ0kg
ZGV2aWNlcw0KPiA+IHRvIGEgR3Vlc3QuIFBhdGNoZXMgZm9yIHNhbWUgd2lsbCBmb2xsb3cgdGhp
cyBwYXRjaC4NCj4gPg0KPiA+IFNpZ25lZC1vZmYtYnk6IEJoYXJhdCBCaHVzaGFuIDxCaGFyYXQu
Qmh1c2hhbkBmcmVlc2NhbGUuY29tPg0KPiA+IC0tLQ0KPiA+ICBhcmNoL3Bvd2VycGMvaW5jbHVk
ZS9hc20vZGV2aWNlLmggIHwgICAyICsNCj4gPiAgYXJjaC9wb3dlcnBjL2luY2x1ZGUvYXNtL2Zz
bF9tc2kuaCB8ICAyNiArKysrKysNCj4gPiAgYXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfbXNpLmMg
ICAgICB8IDE2OQ0KPiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tDQo+ID4g
IGFyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX21zaS5oICAgICAgfCAgIDEgKw0KPiA+ICA0IGZpbGVz
IGNoYW5nZWQsIDE3MyBpbnNlcnRpb25zKCspLCAyNSBkZWxldGlvbnMoLSkgIGNyZWF0ZSBtb2Rl
DQo+ID4gMTAwNjQ0IGFyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9mc2xfbXNpLmgNCj4gPg0KPiA+
IGRpZmYgLS1naXQgYS9hcmNoL3Bvd2VycGMvaW5jbHVkZS9hc20vZGV2aWNlLmgNCj4gPiBiL2Fy
Y2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9kZXZpY2UuaA0KPiA+IGluZGV4IDM4ZmFlZGUuLjFjMmJm
ZDcgMTAwNjQ0DQo+ID4gLS0tIGEvYXJjaC9wb3dlcnBjL2luY2x1ZGUvYXNtL2RldmljZS5oDQo+
ID4gKysrIGIvYXJjaC9wb3dlcnBjL2luY2x1ZGUvYXNtL2RldmljZS5oDQo+ID4gQEAgLTQwLDYg
KzQwLDggQEAgc3RydWN0IGRldl9hcmNoZGF0YSB7ICAjaWZkZWYgQ09ORklHX0ZBSUxfSU9NTVUN
Cj4gPiAgCWludCBmYWlsX2lvbW11Ow0KPiA+ICAjZW5kaWYNCj4gPiArDQo+ID4gKwl2b2lkICpj
b250ZXh0Ow0KPiA+ICB9Ow0KPiANCj4gVGhpcyBuZWVkcyBhIGNvbW1lbnQgYW5kIHByb2JhYmx5
IGEgbW9yZSBzcGVjaWZpYyBuYW1lLg0KDQpPaw0KDQo+IA0KPiA+IEBAIC0yMDAsNiArMTg1LDEy
IEBAIHN0YXRpYyBzdHJ1Y3QgZnNsX21zaQ0KPiA+ICpmc2xfbXNpX2dldF9yZXNlcnZlZF9tc2lf
YmFuayhzdHJ1Y3QgcGNpX2RldiAqcGRldikgIHsNCj4gPiAgCXN0cnVjdCBmc2xfbXNpICptc2lf
ZGF0YSA9IE5VTEw7DQo+ID4gIAl2b2lkICpjb250ZXh0ID0gTlVMTDsNCj4gPiArCXN0cnVjdCBk
ZXZpY2UgKmRldiA9ICZwZGV2LT5kZXY7DQo+ID4gKw0KPiA+ICsJLyogRGV2aWNlIGFzc2lnbmVk
IHRvIHVzZXJzcGFjZSBpZiB0aGVyZSBpcyB2YWxpZCBjb250ZXh0ICovDQo+ID4gKwlpZiAoZGV2
LT5hcmNoZGF0YS5jb250ZXh0KSB7DQo+ID4gKwkJY29udGV4dCA9IGRldi0+YXJjaGRhdGEuY29u
dGV4dDsNCj4gPiArCX0NCj4gDQo+IE5vIHt9DQoNCk9rLCBsZWZ0b3ZlciBvZiBwcmludCBkZWJ1
Z2dpbmcgOikNCg0KPiANCj4gPg0KPiA+ICAJbGlzdF9mb3JfZWFjaF9lbnRyeShtc2lfZGF0YSwg
Jm1zaV9oZWFkLCBsaXN0KSB7DQo+ID4gIAkJaWYgKChtc2lfZGF0YS0+cmVzZXJ2ZWQgPT0gTVNJ
X1JFU0VSVkVEKSAmJiBAQCAtMjA4LDEzDQo+ICsxOTksMTMzIEBADQo+ID4gc3RhdGljIHN0cnVj
dCBmc2xfbXNpICpmc2xfbXNpX2dldF9yZXNlcnZlZF9tc2lfYmFuayhzdHJ1Y3QgcGNpX2Rldg0K
PiAqcGRldikNCj4gPiAgCX0NCj4gPg0KPiA+ICAJLyogSWYgbm8gTVNJIGJhbmsgYWxsb2NhdGVk
IGZvciBrZXJuZWwgb3duZWQgZGV2aWNlLCBhbGxvY2F0ZSBvbmUNCj4gKi8NCj4gPiAtCW1zaV9k
YXRhID0gZnNsX21zaV9hbGxvY2F0ZV9tc2lfYmFuayhOVUxMKTsNCj4gPiAtCWlmIChtc2lfZGF0
YSkNCj4gPiAtCQlyZXR1cm4gbXNpX2RhdGE7DQo+ID4gKwlpZiAoIWNvbnRleHQpIHsNCj4gPiAr
CQltc2lfZGF0YSA9IGZzbF9tc2lfYWxsb2NhdGVfbXNpX2JhbmsoTlVMTCk7DQo+ID4gKwkJaWYg
KG1zaV9kYXRhKQ0KPiA+ICsJCQlyZXR1cm4gbXNpX2RhdGE7DQo+ID4gKwl9DQo+ID4NCj4gPiAg
CXJldHVybiBOVUxMOw0KPiA+ICB9DQo+ID4NCj4gPiArLyogQVBJIHRvIHNldCAiY29udGV4dCIg
dG8gd2hpY2ggdGhlIGRldmljZSBiZWxvbmdzICovIGludA0KPiA+ICtmc2xfbXNpX3NldF9tc2lf
YmFua19pbl9kZXYoc3RydWN0IGRldmljZSAqZGV2LCB2b2lkICpkYXRhKSB7DQo+ID4gKwlkZXYt
PmFyY2hkYXRhLmNvbnRleHQgPSBkYXRhOw0KPiA+ICsJcmV0dXJuIDA7DQo+ID4gK30NCj4gDQo+
IERvIHdlIHJlYWxseSBuZWVkICJtc2kiIHRvIGFwcGVhciB0d2ljZSBpbiBldmVyeSBmdW5jdGlv
biBuYW1lPw0KDQpObywgd2lsbCByZW1vdmUgbGF0ZXIgIm1zaSIuDQoNCj4gDQo+ID4gKw0KPiA+
ICsvKiAgVGhpcyBBUEkgQWxsb3dzIGEgTVNJIGJhbmsgdG8gYmUgcmVzZXJ2ZWQgZm9yIGEgImNv
bnRleHQiLg0KPiA+ICsgKiAgQWxsIGRldmljZXMgaW4gc2FtZSAiY29udGV4dCIgd2lsbCBzaGFy
ZSB0aGUgYWxsb2NhdGVkDQo+ID4gKyAqICBNU0kgYmFuay4NCj4gPiArICogIFR5cGljYWxseSB0
aGlzIGZ1bmN0aW9uIHdpbGwgYmUgY2FsbGVkIGZyb20gbWV0YQ0KPiA+ICsgKiAgZHJpdmVyIGxp
a2UgVkZJTyB3aXRoIGEgdmFsaWQgImNvbnRleHQiLg0KPiA+ICsgKi8NCj4gPiArc3RydWN0IGZz
bF9tc2kgKmZzbF9tc2lfcmVzZXJ2ZV9tc2lfYmFuayh2b2lkICpjb250ZXh0KSB7DQo+ID4gKwlz
dHJ1Y3QgZnNsX21zaSAqbXNpX2RhdGE7DQo+ID4gKw0KPiA+ICsJaWYgKCFjb250ZXh0KQ0KPiA+
ICsJCXJldHVybiBOVUxMOw0KPiA+ICsNCj4gPiArCS8qIENoZWNrIGlmIG1zaS1iYW5rIGFscmVh
ZHkgYWxsb2NhdGVkIGZvciB0aGUgY29udGV4dCAqLw0KPiA+ICsJbGlzdF9mb3JfZWFjaF9lbnRy
eShtc2lfZGF0YSwgJm1zaV9oZWFkLCBsaXN0KSB7DQo+ID4gKwkJaWYgKG1zaV9kYXRhLT5yZXNl
cnZlZCA9PSBNU0lfRlJFRSkNCj4gPiArCQkJY29udGludWU7DQo+ID4gKw0KPiA+ICsJCWlmIChj
b250ZXh0ID09IG1zaV9kYXRhLT5jb250ZXh0KQ0KPiA+ICsJCQlyZXR1cm4gbXNpX2RhdGE7DQo+
IA0KPiBUaGUgcmVzZXJ2ZWQgPT0gTVNJX0ZSRUUgY2hlY2sgZG9lc24ndCBhZGQgYW55dGhpbmcg
YmVjYXVzZSBpZiBpdCdzIGZyZWUNCj4gdGhlbiB0aGUgY29udGV4dCBjaGVjayB3aWxsIGZhaWwu
DQoNCm9rDQoNCj4gDQo+ID4gK3N0YXRpYyBpbnQgaXNfbXNpX2JhbmtfcmVzZXJ2ZWQoc3RydWN0
IGZzbF9tc2kgKm1zaSkNCj4gDQo+IHMvaW50L2Jvb2wvDQoNCk9rDQoNCj4gDQo+IA0KPiA+ICsv
Kg0KPiA+ICsgKiBUaGlzIGZ1bmN0aW9uIGNvbmZpZ3VyZXMgUEFNVSB3aW5kb3cgZm9yIE1TSSBw
YWdlIHdpdGgNCj4gPiArICogZ2l2ZW4gaW92YS4gQWxzbyBzYW1lIGlvdmEgd2lsbCBiZSB1c2Vk
IGFzICJtc2ktYWRkcmVzcyINCj4gPiArICogd2hlbiBjb25maWd1cmluZyBtc2ktbWVzc2FnZSBp
biB0aGUgZGV2aWNlcyB1c2luZyB0aGlzDQo+ID4gKyAqIG1zaSBiYW5rLg0KPiA+ICsgKi8NCj4g
PiAraW50IGZzbF9tc2lfc2V0X21zaV9iYW5rX3JlZ2lvbihzdHJ1Y3QgaW9tbXVfZG9tYWluICpk
b21haW4sDQo+ID4gKwkJCQl2b2lkICpjb250ZXh0ICwgaW50IHdpbiwNCj4gDQo+IFdoaXRlc3Bh
Y2UNCj4gDQo+ID4gQEAgLTIyNSwxMiArMzM2LDE3IEBAIHN0YXRpYyB2b2lkIGZzbF9jb21wb3Nl
X21zaV9tc2coc3RydWN0IHBjaV9kZXYNCj4gKnBkZXYsIGludCBod2lycSwNCj4gPiAgCWludCBs
ZW47DQo+ID4gIAljb25zdCBfX2JlNjQgKnJlZzsNCj4gPg0KPiA+IC0JLyogSWYgdGhlIG1zaS1h
ZGRyZXNzLTY0IHByb3BlcnR5IGV4aXN0cywgdGhlbiB1c2UgaXQgKi8NCj4gPiAtCXJlZyA9IG9m
X2dldF9wcm9wZXJ0eShob3NlLT5kbiwgIm1zaS1hZGRyZXNzLTY0IiwgJmxlbik7DQo+ID4gLQlp
ZiAocmVnICYmIChsZW4gPT0gc2l6ZW9mKHU2NCkpKQ0KPiA+IC0JCWFkZHJlc3MgPSBiZTY0X3Rv
X2NwdXAocmVnKTsNCj4gPiAtCWVsc2UNCj4gPiAtCQlhZGRyZXNzID0gbXNpX2RhdGEtPm1zaWly
Ow0KPiA+ICsJaWYgKHBkZXYtPmRldi5hcmNoZGF0YS5jb250ZXh0KSB7DQo+ID4gKwkJYWRkcmVz
cyA9IG1zaV9kYXRhLT5pb3ZhOw0KPiA+ICsJfSBlbHNlIHsNCj4gPiArCQkvKiBJZiB0aGUgbXNp
LWFkZHJlc3MtNjQgcHJvcGVydHkgZXhpc3RzLCB0aGVuIHVzZSBpdCAqLw0KPiA+ICsJCXJlZyA9
IG9mX2dldF9wcm9wZXJ0eShob3NlLT5kbiwgIm1zaS1hZGRyZXNzLTY0IiwgJmxlbik7DQo+ID4g
KwkJaWYgKHJlZyAmJiAobGVuID09IHNpemVvZih1NjQpKSkNCj4gPiArCQkJYWRkcmVzcyA9IGJl
NjRfdG9fY3B1cChyZWcpOw0KPiA+ICsJCWVsc2UNCj4gPiArCQkJYWRkcmVzcyA9IGZzbF9wY2lf
aW1tcmJhcl9iYXNlKGhvc2UpICsNCj4gPiArCQkJCQkobXNpX2RhdGEtPm1zaWlyICYgMHhmZmZm
Zik7DQo+ID4gKwl9DQo+IA0KPiBZb3UgcmVtb3ZlZCB0aGUgY2FsbCB0byBmc2xfcGNpX2ltbXJi
YXJfYmFzZSBpbiBwYXRjaCAxLzQgYW5kIGFyZQ0KPiByZWludHJvZHVjaW5nIGl0IGhlcmUuICBJ
ZiB0aGlzIGNhbGwgaXMgbmVlZGVkLCBob3cgZG9lcyBpdCB3b3JrIGluDQo+IGJldHdlZW4gdGhv
c2UgdHdvIHBhdGNoZXM/DQoNCk5vdCBuZWVkZWQsIHdpbGwgdXNlIHRoZSBtc2lfZGF0YS0+bXNp
aXIgYXMgcGVyIHBhdGNoIDEvNDsNCg0KPiANCj4gPiBAQCAtNDAxLDYgKzUxNyw5IEBAIHN0YXRp
YyBpbnQgZnNsX29mX21zaV9yZW1vdmUoc3RydWN0IHBsYXRmb3JtX2RldmljZQ0KPiAqb2ZkZXYp
DQo+ID4gIAlzdHJ1Y3QgZnNsX21zaSAqbXNpID0gcGxhdGZvcm1fZ2V0X2RydmRhdGEob2ZkZXYp
Ow0KPiA+ICAJaW50IHZpcnEsIGk7DQo+ID4NCj4gPiArCWlmIChpc19tc2lfYmFua19yZXNlcnZl
ZChtc2kpKQ0KPiA+ICsJCXJldHVybiAtRUJVU1k7DQo+IA0KPiBBcyBJIG1lbnRpb25lZCBpbiBk
aXNjdXNzaW9uIG9mIGEgcHJpb3IgdmVyc2lvbiBvZiB0aGlzLCB0aGUgZHJpdmVyIGNvcmUNCj4g
aWdub3JlcyBlcnJvciBjb2RlcyBpbiAucmVtb3ZlKCkuICBTaW5jZSB0aGVyZSdzIG5vIHJlYXNv
biB0byByZW1vdmUgdGhpcw0KPiBwaWVjZSBvZiBwbGF0Zm9ybSBpbmZyYXN0cnVjdHVyZSwgYW5k
IHRoZSByZW1vdmUgcGF0aCBoYXMgcHJvYmFibHkgbmV2ZXINCj4gYmVlbiB0ZXN0ZWQsIEknZCBq
dXN0IHJlcGxhY2UgdGhlIHdob2xlIHRoaW5nIHdpdGggQlVHKCkuDQoNCk9rLA0KDQpUaGFua3Mg
YSBsb3QNCi1CaGFyYXQNCg0KPiANCj4gLVNjb3R0DQo+IA0KDQo=

Re: [PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel owned devices

[Scott Wood]

On Thu, 2015-03-12 at 10:46 -0500, Bhushan Bharat-R65777 wrote:
> 
> 
> > -----Original Message-----
> > From: Wood Scott-B07421
> > Sent: Thursday, March 12, 2015 4:53 AM
> > To: Bhushan Bharat-R65777
> > Cc: linuxppc-dev@ozlabs.org
> > Subject: Re: [PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel
> > owned devices
> > 
> > With the patchset as is, how would one indicate whether kernel devices
> > should get a bank?
> 
> For kernel owned device, in fsl_setup_msi_irqs() we check if there is
> a reserved MSI bank, if not then reserve a msi bank. If reserve fails
> then setup_msi_irq() fails. I think there is no fallback to Legacy
> interrupt.

If enabling MSI fails, it's up to the driver to fall back to legacy
interrupts (grep drivers for pci_enable_msi for examples).

> >  Specifically, when the kernel does have an MSI-
> > capable device but we'd prefer to use legacy interrupts to keep MSIs
> > available to VFIO.
> 
> Do we want this?

You'd previously raised concern about wanting to give all MSI banks to
VFIO.  The kernel might still have PCIe devices that are not
performance-critical.  That said, I'm not going to nack the patchset if
it requires the kernel to allocate a bank for itself.

> > > @@ -231,15 +304,12 @@ static int fsl_setup_msi_irqs(struct pci_dev
> > *pdev, int nvec, int type)
> > >  		if (specific_msi_bank) {
> > >  			hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1);
> > >  		} else {
> > > -			/*
> > > -			 * Loop over all the MSI devices until we find one that
> > has an
> > > -			 * available interrupt.
> > > -			 */
> > > -			list_for_each_entry(msi_data, &msi_head, list) {
> > > -				hwirq = msi_bitmap_alloc_hwirqs(&msi_data-
> > >bitmap, 1);
> > > -				if (hwirq >= 0)
> > > -					break;
> > > +			msi_data = fsl_msi_get_reserved_msi_bank(pdev);
> > > +			if (!msi_data) {
> > > +				dev_err(&pdev->dev, "No MSI Bank allocated\n");
> > > +				goto out_free;
> > 
> > Is this really an error?  Seems like dev_info() would be more appropriate
> > to indicate the absence of a resource where you can fall back to another
> > option.
> 
> There is no fallback in fsl_setup_msi_irqs(), We have to error out from fsl_setup_msi_irqs(), no?

I meant as far as the user is concerned, not whether you return an error from the function.

-Scott