Linux-PCI Archive on lore.kernel.org
 help / color / Atom feed
* [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
@ 2019-11-20  3:45 Z.q. Hou
  2019-11-20  3:45 ` [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure Z.q. Hou
                   ` (13 more replies)
  0 siblings, 14 replies; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

This patch set is to recode the Mobiveil driver and add
PCIe support for NXP Layerscape series SoCs integrated
Mobiveil's PCIe Gen4 controller.

Hou Zhiqiang (12):
  PCI: mobiveil: Re-abstract the private structure
  PCI: mobiveil: Move the host initialization into a routine
  PCI: mobiveil: Collect the interrupt related operations into a routine
  PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver
  PCI: mobiveil: Add callback function for interrupt initialization
  PCI: mobiveil: Add callback function for link up check
  PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host
  PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors
  dt-bindings: PCI: Add NXP Layerscape SoCs PCIe Gen4 controller
  PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs
  arm64: dts: lx2160a: Add PCIe controller DT nodes
  arm64: defconfig: Enable CONFIG_PCIE_LAYERSCAPE_GEN4

 .../bindings/pci/layerscape-pcie-gen4.txt     |  52 ++
 MAINTAINERS                                   |  10 +-
 .../arm64/boot/dts/freescale/fsl-lx2160a.dtsi | 163 ++++++
 arch/arm64/configs/defconfig                  |   1 +
 drivers/pci/controller/Kconfig                |  11 +-
 drivers/pci/controller/Makefile               |   2 +-
 drivers/pci/controller/mobiveil/Kconfig       |  34 ++
 drivers/pci/controller/mobiveil/Makefile      |   5 +
 .../mobiveil/pcie-layerscape-gen4.c           | 274 +++++++++
 .../pcie-mobiveil-host.c}                     | 544 ++++--------------
 .../controller/mobiveil/pcie-mobiveil-plat.c  |  60 ++
 .../pci/controller/mobiveil/pcie-mobiveil.c   | 230 ++++++++
 .../pci/controller/mobiveil/pcie-mobiveil.h   | 226 ++++++++
 13 files changed, 1157 insertions(+), 455 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt
 create mode 100644 drivers/pci/controller/mobiveil/Kconfig
 create mode 100644 drivers/pci/controller/mobiveil/Makefile
 create mode 100644 drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
 rename drivers/pci/controller/{pcie-mobiveil.c => mobiveil/pcie-mobiveil-host.c} (54%)
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.c
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
@ 2019-11-20  3:45 ` Z.q. Hou
  2020-01-13 10:09   ` Andrew Murray
  2019-11-20  3:45 ` [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine Z.q. Hou
                   ` (12 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The Mobiveil PCIe controller can work in either Root Complex
mode or Endpoint mode. So introduce a new structure root_port,
and abstract the RC related members into it.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 drivers/pci/controller/pcie-mobiveil.c | 99 ++++++++++++++++----------
 1 file changed, 60 insertions(+), 39 deletions(-)

diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
index 3a696ca45bfa..5fd26e376af2 100644
--- a/drivers/pci/controller/pcie-mobiveil.c
+++ b/drivers/pci/controller/pcie-mobiveil.c
@@ -3,7 +3,10 @@
  * PCIe host controller driver for Mobiveil PCIe Host controller
  *
  * Copyright (c) 2018 Mobiveil Inc.
+ * Copyright 2019 NXP
+ *
  * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
+ * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
  */
 
 #include <linux/delay.h>
@@ -138,22 +141,27 @@ struct mobiveil_msi {			/* MSI information */
 	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
 };
 
+struct root_port {
+	char root_bus_nr;
+	void __iomem *config_axi_slave_base;	/* endpoint config base */
+	struct resource *ob_io_res;
+	int irq;
+	raw_spinlock_t intx_mask_lock;
+	struct irq_domain *intx_domain;
+	struct mobiveil_msi msi;
+	struct pci_host_bridge *bridge;
+};
+
 struct mobiveil_pcie {
 	struct platform_device *pdev;
-	void __iomem *config_axi_slave_base;	/* endpoint config base */
 	void __iomem *csr_axi_slave_base;	/* root port config base */
 	void __iomem *apb_csr_base;	/* MSI register base */
 	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
-	struct irq_domain *intx_domain;
-	raw_spinlock_t intx_mask_lock;
-	int irq;
 	int apio_wins;
 	int ppio_wins;
 	int ob_wins_configured;		/* configured outbound windows */
 	int ib_wins_configured;		/* configured inbound windows */
-	struct resource *ob_io_res;
-	char root_bus_nr;
-	struct mobiveil_msi msi;
+	struct root_port rp;
 };
 
 /*
@@ -281,16 +289,17 @@ static bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
 static bool mobiveil_pcie_valid_device(struct pci_bus *bus, unsigned int devfn)
 {
 	struct mobiveil_pcie *pcie = bus->sysdata;
+	struct root_port *rp = &pcie->rp;
 
 	/* Only one device down on each root port */
-	if ((bus->number == pcie->root_bus_nr) && (devfn > 0))
+	if ((bus->number == rp->root_bus_nr) && (devfn > 0))
 		return false;
 
 	/*
 	 * Do not read more than one device on the bus directly
 	 * attached to RC
 	 */
-	if ((bus->primary == pcie->root_bus_nr) && (PCI_SLOT(devfn) > 0))
+	if ((bus->primary == rp->root_bus_nr) && (PCI_SLOT(devfn) > 0))
 		return false;
 
 	return true;
@@ -304,13 +313,14 @@ static void __iomem *mobiveil_pcie_map_bus(struct pci_bus *bus,
 					   unsigned int devfn, int where)
 {
 	struct mobiveil_pcie *pcie = bus->sysdata;
+	struct root_port *rp = &pcie->rp;
 	u32 value;
 
 	if (!mobiveil_pcie_valid_device(bus, devfn))
 		return NULL;
 
 	/* RC config access */
-	if (bus->number == pcie->root_bus_nr)
+	if (bus->number == rp->root_bus_nr)
 		return pcie->csr_axi_slave_base + where;
 
 	/*
@@ -325,7 +335,7 @@ static void __iomem *mobiveil_pcie_map_bus(struct pci_bus *bus,
 
 	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_PEX_WIN_L(WIN_NUM_0));
 
-	return pcie->config_axi_slave_base + where;
+	return rp->config_axi_slave_base + where;
 }
 
 static struct pci_ops mobiveil_pcie_ops = {
@@ -339,7 +349,8 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
 	struct irq_chip *chip = irq_desc_get_chip(desc);
 	struct mobiveil_pcie *pcie = irq_desc_get_handler_data(desc);
 	struct device *dev = &pcie->pdev->dev;
-	struct mobiveil_msi *msi = &pcie->msi;
+	struct root_port *rp = &pcie->rp;
+	struct mobiveil_msi *msi = &rp->msi;
 	u32 msi_data, msi_addr_lo, msi_addr_hi;
 	u32 intr_status, msi_status;
 	unsigned long shifted_status;
@@ -365,7 +376,7 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
 		shifted_status >>= PAB_INTX_START;
 		do {
 			for_each_set_bit(bit, &shifted_status, PCI_NUM_INTX) {
-				virq = irq_find_mapping(pcie->intx_domain,
+				virq = irq_find_mapping(rp->intx_domain,
 							bit + 1);
 				if (virq)
 					generic_handle_irq(virq);
@@ -424,15 +435,16 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
 	struct device *dev = &pcie->pdev->dev;
 	struct platform_device *pdev = pcie->pdev;
 	struct device_node *node = dev->of_node;
+	struct root_port *rp = &pcie->rp;
 	struct resource *res;
 
 	/* map config resource */
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
 					   "config_axi_slave");
-	pcie->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
-	if (IS_ERR(pcie->config_axi_slave_base))
-		return PTR_ERR(pcie->config_axi_slave_base);
-	pcie->ob_io_res = res;
+	rp->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
+	if (IS_ERR(rp->config_axi_slave_base))
+		return PTR_ERR(rp->config_axi_slave_base);
+	rp->ob_io_res = res;
 
 	/* map csr resource */
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
@@ -455,9 +467,9 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
 	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
 		pcie->ppio_wins = MAX_PIO_WINDOWS;
 
-	pcie->irq = platform_get_irq(pdev, 0);
-	if (pcie->irq <= 0) {
-		dev_err(dev, "failed to map IRQ: %d\n", pcie->irq);
+	rp->irq = platform_get_irq(pdev, 0);
+	if (rp->irq <= 0) {
+		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
 		return -ENODEV;
 	}
 
@@ -564,9 +576,9 @@ static int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
 static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
 {
 	phys_addr_t msg_addr = pcie->pcie_reg_base;
-	struct mobiveil_msi *msi = &pcie->msi;
+	struct mobiveil_msi *msi = &pcie->rp.msi;
 
-	pcie->msi.num_of_vectors = PCI_NUM_MSI;
+	msi->num_of_vectors = PCI_NUM_MSI;
 	msi->msi_pages_phys = (phys_addr_t)msg_addr;
 
 	writel_relaxed(lower_32_bits(msg_addr),
@@ -579,7 +591,8 @@ static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
 
 static int mobiveil_host_init(struct mobiveil_pcie *pcie)
 {
-	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+	struct root_port *rp = &pcie->rp;
+	struct pci_host_bridge *bridge = rp->bridge;
 	u32 value, pab_ctrl, type;
 	struct resource_entry *win;
 
@@ -629,8 +642,8 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
 	 */
 
 	/* config outbound translation window */
-	program_ob_windows(pcie, WIN_NUM_0, pcie->ob_io_res->start, 0,
-			   CFG_WINDOW_TYPE, resource_size(pcie->ob_io_res));
+	program_ob_windows(pcie, WIN_NUM_0, rp->ob_io_res->start, 0,
+			   CFG_WINDOW_TYPE, resource_size(rp->ob_io_res));
 
 	/* memory inbound translation window */
 	program_ib_windows(pcie, WIN_NUM_0, 0, 0, MEM_WINDOW_TYPE, IB_WIN_SIZE);
@@ -667,32 +680,36 @@ static void mobiveil_mask_intx_irq(struct irq_data *data)
 {
 	struct irq_desc *desc = irq_to_desc(data->irq);
 	struct mobiveil_pcie *pcie;
+	struct root_port *rp;
 	unsigned long flags;
 	u32 mask, shifted_val;
 
 	pcie = irq_desc_get_chip_data(desc);
+	rp = &pcie->rp;
 	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
-	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
+	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
 	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
 	shifted_val &= ~mask;
 	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
-	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
+	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
 }
 
 static void mobiveil_unmask_intx_irq(struct irq_data *data)
 {
 	struct irq_desc *desc = irq_to_desc(data->irq);
 	struct mobiveil_pcie *pcie;
+	struct root_port *rp;
 	unsigned long flags;
 	u32 shifted_val, mask;
 
 	pcie = irq_desc_get_chip_data(desc);
+	rp = &pcie->rp;
 	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
-	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
+	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
 	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
 	shifted_val |= mask;
 	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
-	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
+	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
 }
 
 static struct irq_chip intx_irq_chip = {
@@ -760,7 +777,7 @@ static int mobiveil_irq_msi_domain_alloc(struct irq_domain *domain,
 					 unsigned int nr_irqs, void *args)
 {
 	struct mobiveil_pcie *pcie = domain->host_data;
-	struct mobiveil_msi *msi = &pcie->msi;
+	struct mobiveil_msi *msi = &pcie->rp.msi;
 	unsigned long bit;
 
 	WARN_ON(nr_irqs != 1);
@@ -787,7 +804,7 @@ static void mobiveil_irq_msi_domain_free(struct irq_domain *domain,
 {
 	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
 	struct mobiveil_pcie *pcie = irq_data_get_irq_chip_data(d);
-	struct mobiveil_msi *msi = &pcie->msi;
+	struct mobiveil_msi *msi = &pcie->rp.msi;
 
 	mutex_lock(&msi->lock);
 
@@ -808,9 +825,9 @@ static int mobiveil_allocate_msi_domains(struct mobiveil_pcie *pcie)
 {
 	struct device *dev = &pcie->pdev->dev;
 	struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
-	struct mobiveil_msi *msi = &pcie->msi;
+	struct mobiveil_msi *msi = &pcie->rp.msi;
 
-	mutex_init(&pcie->msi.lock);
+	mutex_init(&msi->lock);
 	msi->dev_domain = irq_domain_add_linear(NULL, msi->num_of_vectors,
 						&msi_domain_ops, pcie);
 	if (!msi->dev_domain) {
@@ -834,18 +851,19 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
 {
 	struct device *dev = &pcie->pdev->dev;
 	struct device_node *node = dev->of_node;
+	struct root_port *rp = &pcie->rp;
 	int ret;
 
 	/* setup INTx */
-	pcie->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
-						  &intx_domain_ops, pcie);
+	rp->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
+						&intx_domain_ops, pcie);
 
-	if (!pcie->intx_domain) {
+	if (!rp->intx_domain) {
 		dev_err(dev, "Failed to get a INTx IRQ domain\n");
 		return -ENOMEM;
 	}
 
-	raw_spin_lock_init(&pcie->intx_mask_lock);
+	raw_spin_lock_init(&rp->intx_mask_lock);
 
 	/* setup MSI */
 	ret = mobiveil_allocate_msi_domains(pcie);
@@ -862,6 +880,7 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
 	struct pci_bus *child;
 	struct pci_host_bridge *bridge;
 	struct device *dev = &pdev->dev;
+	struct root_port *rp;
 	int ret;
 
 	/* allocate the PCIe port */
@@ -870,6 +889,8 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	pcie = pci_host_bridge_priv(bridge);
+	rp = &pcie->rp;
+	rp->bridge = bridge;
 
 	pcie->pdev = pdev;
 
@@ -904,12 +925,12 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
 		return ret;
 	}
 
-	irq_set_chained_handler_and_data(pcie->irq, mobiveil_pcie_isr, pcie);
+	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
 
 	/* Initialize bridge */
 	bridge->dev.parent = dev;
 	bridge->sysdata = pcie;
-	bridge->busnr = pcie->root_bus_nr;
+	bridge->busnr = rp->root_bus_nr;
 	bridge->ops = &mobiveil_pcie_ops;
 	bridge->map_irq = of_irq_parse_and_map_pci;
 	bridge->swizzle_irq = pci_common_swizzle;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
  2019-11-20  3:45 ` [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure Z.q. Hou
@ 2019-11-20  3:45 ` Z.q. Hou
  2020-01-13 10:19   ` Andrew Murray
  2019-11-20  3:45 ` [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations " Z.q. Hou
                   ` (11 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Move the host initialization related operations into a new
routine to make it can be reused by other incoming platform's
PCIe host driver, in which the Mobiveil GPEX is integrated.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 drivers/pci/controller/pcie-mobiveil.c | 38 +++++++++++++++-----------
 1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
index 5fd26e376af2..97f682ca7c7a 100644
--- a/drivers/pci/controller/pcie-mobiveil.c
+++ b/drivers/pci/controller/pcie-mobiveil.c
@@ -873,27 +873,15 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
 	return 0;
 }
 
-static int mobiveil_pcie_probe(struct platform_device *pdev)
+int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
 {
-	struct mobiveil_pcie *pcie;
+	struct root_port *rp = &pcie->rp;
+	struct pci_host_bridge *bridge = rp->bridge;
+	struct device *dev = &pcie->pdev->dev;
 	struct pci_bus *bus;
 	struct pci_bus *child;
-	struct pci_host_bridge *bridge;
-	struct device *dev = &pdev->dev;
-	struct root_port *rp;
 	int ret;
 
-	/* allocate the PCIe port */
-	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
-	if (!bridge)
-		return -ENOMEM;
-
-	pcie = pci_host_bridge_priv(bridge);
-	rp = &pcie->rp;
-	rp->bridge = bridge;
-
-	pcie->pdev = pdev;
-
 	ret = mobiveil_pcie_parse_dt(pcie);
 	if (ret) {
 		dev_err(dev, "Parsing DT failed, ret: %x\n", ret);
@@ -956,6 +944,24 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
 	return 0;
 }
 
+static int mobiveil_pcie_probe(struct platform_device *pdev)
+{
+	struct mobiveil_pcie *pcie;
+	struct pci_host_bridge *bridge;
+	struct device *dev = &pdev->dev;
+
+	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
+	if (!bridge)
+		return -ENOMEM;
+
+	pcie = pci_host_bridge_priv(bridge);
+	pcie->rp.bridge = bridge;
+
+	pcie->pdev = pdev;
+
+	return mobiveil_pcie_host_probe(pcie);
+}
+
 static const struct of_device_id mobiveil_pcie_of_match[] = {
 	{.compatible = "mbvl,gpex40-pcie",},
 	{},
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations into a routine
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
  2019-11-20  3:45 ` [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure Z.q. Hou
  2019-11-20  3:45 ` [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine Z.q. Hou
@ 2019-11-20  3:45 ` " Z.q. Hou
  2020-01-13 10:34   ` Andrew Murray
  2019-11-20  3:45 ` [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver Z.q. Hou
                   ` (10 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Collect the interrupt initialization related operations into
a new routine to make it more readable.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 drivers/pci/controller/pcie-mobiveil.c | 65 +++++++++++++++++---------
 1 file changed, 42 insertions(+), 23 deletions(-)

diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
index 97f682ca7c7a..512b27a0536e 100644
--- a/drivers/pci/controller/pcie-mobiveil.c
+++ b/drivers/pci/controller/pcie-mobiveil.c
@@ -454,12 +454,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
 		return PTR_ERR(pcie->csr_axi_slave_base);
 	pcie->pcie_reg_base = res->start;
 
-	/* map MSI config resource */
-	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
-	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
-	if (IS_ERR(pcie->apb_csr_base))
-		return PTR_ERR(pcie->apb_csr_base);
-
 	/* read the number of windows requested */
 	if (of_property_read_u32(node, "apio-wins", &pcie->apio_wins))
 		pcie->apio_wins = MAX_PIO_WINDOWS;
@@ -467,12 +461,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
 	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
 		pcie->ppio_wins = MAX_PIO_WINDOWS;
 
-	rp->irq = platform_get_irq(pdev, 0);
-	if (rp->irq <= 0) {
-		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
-		return -ENODEV;
-	}
-
 	return 0;
 }
 
@@ -618,9 +606,6 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
 	pab_ctrl |= (1 << AMBA_PIO_ENABLE_SHIFT) | (1 << PEX_PIO_ENABLE_SHIFT);
 	mobiveil_csr_writel(pcie, pab_ctrl, PAB_CTRL);
 
-	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK | PAB_INTP_MSI_MASK),
-			    PAB_INTP_AMBA_MISC_ENB);
-
 	/*
 	 * program PIO Enable Bit to 1 and Config Window Enable Bit to 1 in
 	 * PAB_AXI_PIO_CTRL Register
@@ -670,9 +655,6 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
 	value |= (PCI_CLASS_BRIDGE_PCI << 16);
 	mobiveil_csr_writel(pcie, value, PAB_INTP_AXI_PIO_CLASS);
 
-	/* setup MSI hardware registers */
-	mobiveil_pcie_enable_msi(pcie);
-
 	return 0;
 }
 
@@ -873,6 +855,46 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
 	return 0;
 }
 
+static int mobiveil_pcie_interrupt_init(struct mobiveil_pcie *pcie)
+{
+	struct platform_device *pdev = pcie->pdev;
+	struct device *dev = &pdev->dev;
+	struct root_port *rp = &pcie->rp;
+	struct resource *res;
+	int ret;
+
+	/* map MSI config resource */
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
+	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
+	if (IS_ERR(pcie->apb_csr_base))
+		return PTR_ERR(pcie->apb_csr_base);
+
+	/* setup MSI hardware registers */
+	mobiveil_pcie_enable_msi(pcie);
+
+	rp->irq = platform_get_irq(pdev, 0);
+	if (rp->irq <= 0) {
+		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
+		return -ENODEV;
+	}
+
+	/* initialize the IRQ domains */
+	ret = mobiveil_pcie_init_irq_domain(pcie);
+	if (ret) {
+		dev_err(dev, "Failed creating IRQ Domain\n");
+		return ret;
+	}
+
+	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
+
+	/* Enable interrupts */
+	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK | PAB_INTP_MSI_MASK),
+			    PAB_INTP_AMBA_MISC_ENB);
+
+
+	return 0;
+}
+
 int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
 {
 	struct root_port *rp = &pcie->rp;
@@ -906,15 +928,12 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
 		return ret;
 	}
 
-	/* initialize the IRQ domains */
-	ret = mobiveil_pcie_init_irq_domain(pcie);
+	ret = mobiveil_pcie_interrupt_init(pcie);
 	if (ret) {
-		dev_err(dev, "Failed creating IRQ Domain\n");
+		dev_err(dev, "Interrupt init failed\n");
 		return ret;
 	}
 
-	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
-
 	/* Initialize bridge */
 	bridge->dev.parent = dev;
 	bridge->sysdata = pcie;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (2 preceding siblings ...)
  2019-11-20  3:45 ` [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations " Z.q. Hou
@ 2019-11-20  3:45 ` Z.q. Hou
  2020-01-13 11:05   ` Andrew Murray
  2019-11-20  3:45 ` [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization Z.q. Hou
                   ` (9 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Modularize the Mobiveil PCIe host driver according to the
abstraction of Root Complex and Endpoint and move it into
a new directory.

Split the RC related routines into pcie-mobiveil-host.c,
and common routines into pcie-mobiveil.c, move the macro
definitions and function declarations into pcie-mobiveil.h,
and the Mobiveil platform reference code into
pcie-mobiveil-plat.c. So that it is easy to reuse the
extracted routines to add a new host driver, which
integrated Mobiveil PCIe GPEX IP.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 MAINTAINERS                                   |   2 +-
 drivers/pci/controller/Kconfig                |  11 +-
 drivers/pci/controller/Makefile               |   2 +-
 drivers/pci/controller/mobiveil/Kconfig       |  24 ++
 drivers/pci/controller/mobiveil/Makefile      |   4 +
 .../pcie-mobiveil-host.c}                     | 398 +-----------------
 .../controller/mobiveil/pcie-mobiveil-plat.c  |  60 +++
 .../pci/controller/mobiveil/pcie-mobiveil.c   | 227 ++++++++++
 .../pci/controller/mobiveil/pcie-mobiveil.h   | 178 ++++++++
 9 files changed, 497 insertions(+), 409 deletions(-)
 create mode 100644 drivers/pci/controller/mobiveil/Kconfig
 create mode 100644 drivers/pci/controller/mobiveil/Makefile
 rename drivers/pci/controller/{pcie-mobiveil.c => mobiveil/pcie-mobiveil-host.c} (61%)
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.c
 create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 3f7f8cdbc471..a4ad99619e53 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12612,7 +12612,7 @@ M:	Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
 L:	linux-pci@vger.kernel.org
 S:	Supported
 F:	Documentation/devicetree/bindings/pci/mobiveil-pcie.txt
-F:	drivers/pci/controller/pcie-mobiveil.c
+F:	drivers/pci/controller/mobiveil/pcie-mobiveil*
 
 PCI DRIVER FOR MVEBU (Marvell Armada 370 and Armada XP SOC support)
 M:	Thomas Petazzoni <thomas.petazzoni@bootlin.com>
diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig
index f5de9119e8d3..74fd332755ae 100644
--- a/drivers/pci/controller/Kconfig
+++ b/drivers/pci/controller/Kconfig
@@ -241,16 +241,6 @@ config PCIE_MEDIATEK
 	  Say Y here if you want to enable PCIe controller support on
 	  MediaTek SoCs.
 
-config PCIE_MOBIVEIL
-	bool "Mobiveil AXI PCIe controller"
-	depends on ARCH_ZYNQMP || COMPILE_TEST
-	depends on OF
-	depends on PCI_MSI_IRQ_DOMAIN
-	help
-	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
-	  Soft IP. It has up to 8 outbound and inbound windows
-	  for address translation and it is a PCIe Gen4 IP.
-
 config PCIE_TANGO_SMP8759
 	bool "Tango SMP8759 PCIe controller (DANGEROUS)"
 	depends on ARCH_TANGO && PCI_MSI && OF
@@ -289,4 +279,5 @@ config PCI_HYPERV_INTERFACE
 	  have a common interface with the Hyper-V PCI frontend driver.
 
 source "drivers/pci/controller/dwc/Kconfig"
+source "drivers/pci/controller/mobiveil/Kconfig"
 endmenu
diff --git a/drivers/pci/controller/Makefile b/drivers/pci/controller/Makefile
index a2a22c9d91af..44414cfd45ea 100644
--- a/drivers/pci/controller/Makefile
+++ b/drivers/pci/controller/Makefile
@@ -27,11 +27,11 @@ obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
 obj-$(CONFIG_PCIE_ROCKCHIP_EP) += pcie-rockchip-ep.o
 obj-$(CONFIG_PCIE_ROCKCHIP_HOST) += pcie-rockchip-host.o
 obj-$(CONFIG_PCIE_MEDIATEK) += pcie-mediatek.o
-obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
 obj-$(CONFIG_PCIE_TANGO_SMP8759) += pcie-tango.o
 obj-$(CONFIG_VMD) += vmd.o
 # pcie-hisi.o quirks are needed even without CONFIG_PCIE_DW
 obj-y				+= dwc/
+obj-y				+= mobiveil/
 
 
 # The following drivers are for devices that use the generic ACPI
diff --git a/drivers/pci/controller/mobiveil/Kconfig b/drivers/pci/controller/mobiveil/Kconfig
new file mode 100644
index 000000000000..64343c07bfed
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/Kconfig
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: GPL-2.0
+
+menu "Mobiveil PCIe Core Support"
+	depends on PCI
+
+config PCIE_MOBIVEIL
+	bool
+
+config PCIE_MOBIVEIL_HOST
+	bool
+	depends on PCI_MSI_IRQ_DOMAIN
+	select PCIE_MOBIVEIL
+
+config PCIE_MOBIVEIL_PLAT
+	bool "Mobiveil AXI PCIe controller"
+	depends on ARCH_ZYNQMP || COMPILE_TEST
+	depends on OF
+	select PCIE_MOBIVEIL_HOST
+	help
+	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
+	  Soft IP. It has up to 8 outbound and inbound windows
+	  for address translation and it is a PCIe Gen4 IP.
+
+endmenu
diff --git a/drivers/pci/controller/mobiveil/Makefile b/drivers/pci/controller/mobiveil/Makefile
new file mode 100644
index 000000000000..9fb6d1c6504d
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
+obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
+obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
similarity index 61%
rename from drivers/pci/controller/pcie-mobiveil.c
rename to drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
index 512b27a0536e..2cc424e78d33 100644
--- a/drivers/pci/controller/pcie-mobiveil.c
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
@@ -9,7 +9,6 @@
  * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
  */
 
-#include <linux/delay.h>
 #include <linux/init.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
@@ -26,265 +25,7 @@
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 
-#include "../pci.h"
-
-/* register offsets and bit positions */
-
-/*
- * translation tables are grouped into windows, each window registers are
- * grouped into blocks of 4 or 16 registers each
- */
-#define PAB_REG_BLOCK_SIZE		16
-#define PAB_EXT_REG_BLOCK_SIZE		4
-
-#define PAB_REG_ADDR(offset, win)	\
-	(offset + (win * PAB_REG_BLOCK_SIZE))
-#define PAB_EXT_REG_ADDR(offset, win)	\
-	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
-
-#define LTSSM_STATUS			0x0404
-#define  LTSSM_STATUS_L0_MASK		0x3f
-#define  LTSSM_STATUS_L0		0x2d
-
-#define PAB_CTRL			0x0808
-#define  AMBA_PIO_ENABLE_SHIFT		0
-#define  PEX_PIO_ENABLE_SHIFT		1
-#define  PAGE_SEL_SHIFT			13
-#define  PAGE_SEL_MASK			0x3f
-#define  PAGE_LO_MASK			0x3ff
-#define  PAGE_SEL_OFFSET_SHIFT		10
-
-#define PAB_AXI_PIO_CTRL		0x0840
-#define  APIO_EN_MASK			0xf
-
-#define PAB_PEX_PIO_CTRL		0x08c0
-#define  PIO_ENABLE_SHIFT		0
-
-#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
-#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
-#define  PAB_INTP_INTX_MASK		0x01e0
-#define  PAB_INTP_MSI_MASK		0x8
-
-#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
-#define  WIN_ENABLE_SHIFT		0
-#define  WIN_TYPE_SHIFT			1
-#define  WIN_TYPE_MASK			0x3
-#define  WIN_SIZE_MASK			0xfffffc00
-
-#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0, win)
-
-#define PAB_EXT_AXI_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0x80a0, win)
-#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
-#define  AXI_WINDOW_ALIGN_MASK		3
-
-#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8, win)
-#define  PAB_BUS_SHIFT			24
-#define  PAB_DEVICE_SHIFT		19
-#define  PAB_FUNCTION_SHIFT		16
-
-#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac, win)
-#define PAB_INTP_AXI_PIO_CLASS		0x474
-
-#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0, win)
-#define  AMAP_CTRL_EN_SHIFT		0
-#define  AMAP_CTRL_TYPE_SHIFT		1
-#define  AMAP_CTRL_TYPE_MASK		3
-
-#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0, win)
-#define PAB_EXT_PEX_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0xb4a0, win)
-#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
-#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8, win)
-#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac, win)
-
-/* starting offset of INTX bits in status register */
-#define PAB_INTX_START			5
-
-/* supported number of MSI interrupts */
-#define PCI_NUM_MSI			16
-
-/* MSI registers */
-#define MSI_BASE_LO_OFFSET		0x04
-#define MSI_BASE_HI_OFFSET		0x08
-#define MSI_SIZE_OFFSET			0x0c
-#define MSI_ENABLE_OFFSET		0x14
-#define MSI_STATUS_OFFSET		0x18
-#define MSI_DATA_OFFSET			0x20
-#define MSI_ADDR_L_OFFSET		0x24
-#define MSI_ADDR_H_OFFSET		0x28
-
-/* outbound and inbound window definitions */
-#define WIN_NUM_0			0
-#define WIN_NUM_1			1
-#define CFG_WINDOW_TYPE			0
-#define IO_WINDOW_TYPE			1
-#define MEM_WINDOW_TYPE			2
-#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
-#define MAX_PIO_WINDOWS			8
-
-/* Parameters for the waiting for link up routine */
-#define LINK_WAIT_MAX_RETRIES		10
-#define LINK_WAIT_MIN			90000
-#define LINK_WAIT_MAX			100000
-
-#define PAGED_ADDR_BNDRY		0xc00
-#define OFFSET_TO_PAGE_ADDR(off)	\
-	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
-#define OFFSET_TO_PAGE_IDX(off)		\
-	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
-
-struct mobiveil_msi {			/* MSI information */
-	struct mutex lock;		/* protect bitmap variable */
-	struct irq_domain *msi_domain;
-	struct irq_domain *dev_domain;
-	phys_addr_t msi_pages_phys;
-	int num_of_vectors;
-	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
-};
-
-struct root_port {
-	char root_bus_nr;
-	void __iomem *config_axi_slave_base;	/* endpoint config base */
-	struct resource *ob_io_res;
-	int irq;
-	raw_spinlock_t intx_mask_lock;
-	struct irq_domain *intx_domain;
-	struct mobiveil_msi msi;
-	struct pci_host_bridge *bridge;
-};
-
-struct mobiveil_pcie {
-	struct platform_device *pdev;
-	void __iomem *csr_axi_slave_base;	/* root port config base */
-	void __iomem *apb_csr_base;	/* MSI register base */
-	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
-	int apio_wins;
-	int ppio_wins;
-	int ob_wins_configured;		/* configured outbound windows */
-	int ib_wins_configured;		/* configured inbound windows */
-	struct root_port rp;
-};
-
-/*
- * mobiveil_pcie_sel_page - routine to access paged register
- *
- * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are paged,
- * for this scheme to work extracted higher 6 bits of the offset will be
- * written to pg_sel field of PAB_CTRL register and rest of the lower 10
- * bits enabled with PAGED_ADDR_BNDRY are used as offset of the register.
- */
-static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8 pg_idx)
-{
-	u32 val;
-
-	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
-	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
-	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
-
-	writel(val, pcie->csr_axi_slave_base + PAB_CTRL);
-}
-
-static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32 off)
-{
-	if (off < PAGED_ADDR_BNDRY) {
-		/* For directly accessed registers, clear the pg_sel field */
-		mobiveil_pcie_sel_page(pcie, 0);
-		return pcie->csr_axi_slave_base + off;
-	}
-
-	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
-	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off);
-}
-
-static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
-{
-	if ((uintptr_t)addr & (size - 1)) {
-		*val = 0;
-		return PCIBIOS_BAD_REGISTER_NUMBER;
-	}
-
-	switch (size) {
-	case 4:
-		*val = readl(addr);
-		break;
-	case 2:
-		*val = readw(addr);
-		break;
-	case 1:
-		*val = readb(addr);
-		break;
-	default:
-		*val = 0;
-		return PCIBIOS_BAD_REGISTER_NUMBER;
-	}
-
-	return PCIBIOS_SUCCESSFUL;
-}
-
-static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
-{
-	if ((uintptr_t)addr & (size - 1))
-		return PCIBIOS_BAD_REGISTER_NUMBER;
-
-	switch (size) {
-	case 4:
-		writel(val, addr);
-		break;
-	case 2:
-		writew(val, addr);
-		break;
-	case 1:
-		writeb(val, addr);
-		break;
-	default:
-		return PCIBIOS_BAD_REGISTER_NUMBER;
-	}
-
-	return PCIBIOS_SUCCESSFUL;
-}
-
-static u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size)
-{
-	void *addr;
-	u32 val;
-	int ret;
-
-	addr = mobiveil_pcie_comp_addr(pcie, off);
-
-	ret = mobiveil_pcie_read(addr, size, &val);
-	if (ret)
-		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
-
-	return val;
-}
-
-static void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
-			       size_t size)
-{
-	void *addr;
-	int ret;
-
-	addr = mobiveil_pcie_comp_addr(pcie, off);
-
-	ret = mobiveil_pcie_write(addr, size, val);
-	if (ret)
-		dev_err(&pcie->pdev->dev, "write CSR address failed\n");
-}
-
-static u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
-{
-	return mobiveil_csr_read(pcie, off, 0x4);
-}
-
-static void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val, u32 off)
-{
-	mobiveil_csr_write(pcie, val, off, 0x4);
-}
-
-static bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
-{
-	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
-		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
-}
+#include "pcie-mobiveil.h"
 
 static bool mobiveil_pcie_valid_device(struct pci_bus *bus, unsigned int devfn)
 {
@@ -464,103 +205,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
 	return 0;
 }
 
-static void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
-			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
-{
-	u32 value;
-	u64 size64 = ~(size - 1);
-
-	if (win_num >= pcie->ppio_wins) {
-		dev_err(&pcie->pdev->dev,
-			"ERROR: max inbound windows reached !\n");
-		return;
-	}
-
-	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
-	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT | WIN_SIZE_MASK);
-	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 << AMAP_CTRL_EN_SHIFT |
-		 (lower_32_bits(size64) & WIN_SIZE_MASK);
-	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
-
-	mobiveil_csr_writel(pcie, upper_32_bits(size64),
-			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
-
-	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
-			    PAB_PEX_AMAP_AXI_WIN(win_num));
-	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
-			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
-
-	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
-			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
-	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
-			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
-
-	pcie->ib_wins_configured++;
-}
-
-/*
- * routine to program the outbound windows
- */
-static void program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
-			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
-{
-	u32 value;
-	u64 size64 = ~(size - 1);
-
-	if (win_num >= pcie->apio_wins) {
-		dev_err(&pcie->pdev->dev,
-			"ERROR: max outbound windows reached !\n");
-		return;
-	}
-
-	/*
-	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
-	 * to 4 KB in PAB_AXI_AMAP_CTRL register
-	 */
-	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
-	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
-	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
-		 (lower_32_bits(size64) & WIN_SIZE_MASK);
-	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
-
-	mobiveil_csr_writel(pcie, upper_32_bits(size64),
-			    PAB_EXT_AXI_AMAP_SIZE(win_num));
-
-	/*
-	 * program AXI window base with appropriate value in
-	 * PAB_AXI_AMAP_AXI_WIN0 register
-	 */
-	mobiveil_csr_writel(pcie,
-			    lower_32_bits(cpu_addr) & (~AXI_WINDOW_ALIGN_MASK),
-			    PAB_AXI_AMAP_AXI_WIN(win_num));
-	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
-			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
-
-	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
-			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
-	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
-			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
-
-	pcie->ob_wins_configured++;
-}
-
-static int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
-{
-	int retries;
-
-	/* check if the link is up or not */
-	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
-		if (mobiveil_pcie_link_up(pcie))
-			return 0;
-
-		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
-	}
-
-	dev_err(&pcie->pdev->dev, "link never came up\n");
-
-	return -ETIMEDOUT;
-}
-
 static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
 {
 	phys_addr_t msg_addr = pcie->pcie_reg_base;
@@ -962,43 +606,3 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
 
 	return 0;
 }
-
-static int mobiveil_pcie_probe(struct platform_device *pdev)
-{
-	struct mobiveil_pcie *pcie;
-	struct pci_host_bridge *bridge;
-	struct device *dev = &pdev->dev;
-
-	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
-	if (!bridge)
-		return -ENOMEM;
-
-	pcie = pci_host_bridge_priv(bridge);
-	pcie->rp.bridge = bridge;
-
-	pcie->pdev = pdev;
-
-	return mobiveil_pcie_host_probe(pcie);
-}
-
-static const struct of_device_id mobiveil_pcie_of_match[] = {
-	{.compatible = "mbvl,gpex40-pcie",},
-	{},
-};
-
-MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
-
-static struct platform_driver mobiveil_pcie_driver = {
-	.probe = mobiveil_pcie_probe,
-	.driver = {
-		.name = "mobiveil-pcie",
-		.of_match_table = mobiveil_pcie_of_match,
-		.suppress_bind_attrs = true,
-	},
-};
-
-builtin_platform_driver(mobiveil_pcie_driver);
-
-MODULE_LICENSE("GPL v2");
-MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
-MODULE_AUTHOR("Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>");
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
new file mode 100644
index 000000000000..64c85f852869
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Mobiveil PCIe Host controller
+ *
+ * Copyright (c) 2018 Mobiveil Inc.
+ * Copyright 2019 NXP
+ *
+ * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
+ * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_pci.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include "pcie-mobiveil.h"
+
+static int mobiveil_pcie_probe(struct platform_device *pdev)
+{
+	struct mobiveil_pcie *pcie;
+	struct pci_host_bridge *bridge;
+	struct device *dev = &pdev->dev;
+
+	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
+	if (!bridge)
+		return -ENOMEM;
+
+	pcie = pci_host_bridge_priv(bridge);
+	pcie->rp.bridge = bridge;
+
+	pcie->pdev = pdev;
+
+	return mobiveil_pcie_host_probe(pcie);
+}
+
+static const struct of_device_id mobiveil_pcie_of_match[] = {
+	{.compatible = "mbvl,gpex40-pcie",},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
+
+static struct platform_driver mobiveil_pcie_driver = {
+	.probe = mobiveil_pcie_probe,
+	.driver = {
+		.name = "mobiveil-pcie",
+		.of_match_table = mobiveil_pcie_of_match,
+		.suppress_bind_attrs = true,
+	},
+};
+
+builtin_platform_driver(mobiveil_pcie_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
+MODULE_AUTHOR("Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>");
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
new file mode 100644
index 000000000000..2773f823c9ea
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe host controller driver for Mobiveil PCIe Host controller
+ *
+ * Copyright (c) 2018 Mobiveil Inc.
+ * Copyright 2019 NXP
+ *
+ * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
+ * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
+ */
+
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+
+#include "pcie-mobiveil.h"
+
+/*
+ * mobiveil_pcie_sel_page - routine to access paged register
+ *
+ * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are paged,
+ * for this scheme to work extracted higher 6 bits of the offset will be
+ * written to pg_sel field of PAB_CTRL register and rest of the lower 10
+ * bits enabled with PAGED_ADDR_BNDRY are used as offset of the register.
+ */
+static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8 pg_idx)
+{
+	u32 val;
+
+	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
+	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
+	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
+
+	writel(val, pcie->csr_axi_slave_base + PAB_CTRL);
+}
+
+static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32 off)
+{
+	if (off < PAGED_ADDR_BNDRY) {
+		/* For directly accessed registers, clear the pg_sel field */
+		mobiveil_pcie_sel_page(pcie, 0);
+		return pcie->csr_axi_slave_base + off;
+	}
+
+	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
+	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off);
+}
+
+static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
+{
+	if ((uintptr_t)addr & (size - 1)) {
+		*val = 0;
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+
+	switch (size) {
+	case 4:
+		*val = readl(addr);
+		break;
+	case 2:
+		*val = readw(addr);
+		break;
+	case 1:
+		*val = readb(addr);
+		break;
+	default:
+		*val = 0;
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
+{
+	if ((uintptr_t)addr & (size - 1))
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+
+	switch (size) {
+	case 4:
+		writel(val, addr);
+		break;
+	case 2:
+		writew(val, addr);
+		break;
+	case 1:
+		writeb(val, addr);
+		break;
+	default:
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size)
+{
+	void *addr;
+	u32 val;
+	int ret;
+
+	addr = mobiveil_pcie_comp_addr(pcie, off);
+
+	ret = mobiveil_pcie_read(addr, size, &val);
+	if (ret)
+		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
+
+	return val;
+}
+
+void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
+			       size_t size)
+{
+	void *addr;
+	int ret;
+
+	addr = mobiveil_pcie_comp_addr(pcie, off);
+
+	ret = mobiveil_pcie_write(addr, size, val);
+	if (ret)
+		dev_err(&pcie->pdev->dev, "write CSR address failed\n");
+}
+
+bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
+{
+	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
+		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
+}
+
+void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
+			u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
+{
+	u32 value;
+	u64 size64 = ~(size - 1);
+
+	if (win_num >= pcie->ppio_wins) {
+		dev_err(&pcie->pdev->dev,
+			"ERROR: max inbound windows reached !\n");
+		return;
+	}
+
+	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
+	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT | WIN_SIZE_MASK);
+	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 << AMAP_CTRL_EN_SHIFT |
+		 (lower_32_bits(size64) & WIN_SIZE_MASK);
+	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
+
+	mobiveil_csr_writel(pcie, upper_32_bits(size64),
+			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
+
+	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
+			    PAB_PEX_AMAP_AXI_WIN(win_num));
+	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
+			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
+
+	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
+			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
+	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
+			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
+
+	pcie->ib_wins_configured++;
+}
+
+/*
+ * routine to program the outbound windows
+ */
+void program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
+			u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
+{
+	u32 value;
+	u64 size64 = ~(size - 1);
+
+	if (win_num >= pcie->apio_wins) {
+		dev_err(&pcie->pdev->dev,
+			"ERROR: max outbound windows reached !\n");
+		return;
+	}
+
+	/*
+	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
+	 * to 4 KB in PAB_AXI_AMAP_CTRL register
+	 */
+	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
+	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
+	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
+		 (lower_32_bits(size64) & WIN_SIZE_MASK);
+	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
+
+	mobiveil_csr_writel(pcie, upper_32_bits(size64),
+			    PAB_EXT_AXI_AMAP_SIZE(win_num));
+
+	/*
+	 * program AXI window base with appropriate value in
+	 * PAB_AXI_AMAP_AXI_WIN0 register
+	 */
+	mobiveil_csr_writel(pcie,
+			    lower_32_bits(cpu_addr) & (~AXI_WINDOW_ALIGN_MASK),
+			    PAB_AXI_AMAP_AXI_WIN(win_num));
+	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
+			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
+
+	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
+			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
+	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
+			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
+
+	pcie->ob_wins_configured++;
+}
+
+int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
+{
+	int retries;
+
+	/* check if the link is up or not */
+	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
+		if (mobiveil_pcie_link_up(pcie))
+			return 0;
+
+		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
+	}
+
+	dev_err(&pcie->pdev->dev, "link never came up\n");
+
+	return -ETIMEDOUT;
+}
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
new file mode 100644
index 000000000000..e3148078e9dd
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * PCIe host controller driver for Mobiveil PCIe Host controller
+ *
+ * Copyright (c) 2018 Mobiveil Inc.
+ * Copyright 2019 NXP
+ *
+ * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
+ * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
+ */
+
+#ifndef _PCIE_MOBIVEIL_H
+#define _PCIE_MOBIVEIL_H
+
+#include <linux/pci.h>
+#include <linux/irq.h>
+#include <linux/msi.h>
+#include "../../pci.h"
+
+/* register offsets and bit positions */
+
+/*
+ * translation tables are grouped into windows, each window registers are
+ * grouped into blocks of 4 or 16 registers each
+ */
+#define PAB_REG_BLOCK_SIZE		16
+#define PAB_EXT_REG_BLOCK_SIZE		4
+
+#define PAB_REG_ADDR(offset, win)	\
+	(offset + (win * PAB_REG_BLOCK_SIZE))
+#define PAB_EXT_REG_ADDR(offset, win)	\
+	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
+
+#define LTSSM_STATUS			0x0404
+#define  LTSSM_STATUS_L0_MASK		0x3f
+#define  LTSSM_STATUS_L0		0x2d
+
+#define PAB_CTRL			0x0808
+#define  AMBA_PIO_ENABLE_SHIFT		0
+#define  PEX_PIO_ENABLE_SHIFT		1
+#define  PAGE_SEL_SHIFT			13
+#define  PAGE_SEL_MASK			0x3f
+#define  PAGE_LO_MASK			0x3ff
+#define  PAGE_SEL_OFFSET_SHIFT		10
+
+#define PAB_AXI_PIO_CTRL		0x0840
+#define  APIO_EN_MASK			0xf
+
+#define PAB_PEX_PIO_CTRL		0x08c0
+#define  PIO_ENABLE_SHIFT		0
+
+#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
+#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
+#define  PAB_INTP_INTX_MASK		0x01e0
+#define  PAB_INTP_MSI_MASK		0x8
+
+#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
+#define  WIN_ENABLE_SHIFT		0
+#define  WIN_TYPE_SHIFT			1
+#define  WIN_TYPE_MASK			0x3
+#define  WIN_SIZE_MASK			0xfffffc00
+
+#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0, win)
+
+#define PAB_EXT_AXI_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0x80a0, win)
+#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
+#define  AXI_WINDOW_ALIGN_MASK		3
+
+#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8, win)
+#define  PAB_BUS_SHIFT			24
+#define  PAB_DEVICE_SHIFT		19
+#define  PAB_FUNCTION_SHIFT		16
+
+#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac, win)
+#define PAB_INTP_AXI_PIO_CLASS		0x474
+
+#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0, win)
+#define  AMAP_CTRL_EN_SHIFT		0
+#define  AMAP_CTRL_TYPE_SHIFT		1
+#define  AMAP_CTRL_TYPE_MASK		3
+
+#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0, win)
+#define PAB_EXT_PEX_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0xb4a0, win)
+#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
+#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8, win)
+#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac, win)
+
+/* starting offset of INTX bits in status register */
+#define PAB_INTX_START			5
+
+/* supported number of MSI interrupts */
+#define PCI_NUM_MSI			16
+
+/* MSI registers */
+#define MSI_BASE_LO_OFFSET		0x04
+#define MSI_BASE_HI_OFFSET		0x08
+#define MSI_SIZE_OFFSET			0x0c
+#define MSI_ENABLE_OFFSET		0x14
+#define MSI_STATUS_OFFSET		0x18
+#define MSI_DATA_OFFSET			0x20
+#define MSI_ADDR_L_OFFSET		0x24
+#define MSI_ADDR_H_OFFSET		0x28
+
+/* outbound and inbound window definitions */
+#define WIN_NUM_0			0
+#define WIN_NUM_1			1
+#define CFG_WINDOW_TYPE			0
+#define IO_WINDOW_TYPE			1
+#define MEM_WINDOW_TYPE			2
+#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
+#define MAX_PIO_WINDOWS			8
+
+/* Parameters for the waiting for link up routine */
+#define LINK_WAIT_MAX_RETRIES		10
+#define LINK_WAIT_MIN			90000
+#define LINK_WAIT_MAX			100000
+
+#define PAGED_ADDR_BNDRY		0xc00
+#define OFFSET_TO_PAGE_ADDR(off)	\
+	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
+#define OFFSET_TO_PAGE_IDX(off)		\
+	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
+
+struct mobiveil_msi {			/* MSI information */
+	struct mutex lock;		/* protect bitmap variable */
+	struct irq_domain *msi_domain;
+	struct irq_domain *dev_domain;
+	phys_addr_t msi_pages_phys;
+	int num_of_vectors;
+	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
+};
+
+struct root_port {
+	char root_bus_nr;
+	void __iomem *config_axi_slave_base;	/* endpoint config base */
+	struct resource *ob_io_res;
+	int irq;
+	raw_spinlock_t intx_mask_lock;
+	struct irq_domain *intx_domain;
+	struct mobiveil_msi msi;
+	struct pci_host_bridge *bridge;
+};
+
+struct mobiveil_pcie {
+	struct platform_device *pdev;
+	void __iomem *csr_axi_slave_base;	/* root port config base */
+	void __iomem *apb_csr_base;	/* MSI register base */
+	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
+	int apio_wins;
+	int ppio_wins;
+	int ob_wins_configured;		/* configured outbound windows */
+	int ib_wins_configured;		/* configured inbound windows */
+	struct root_port rp;
+};
+
+int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie);
+bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie);
+int mobiveil_bringup_link(struct mobiveil_pcie *pcie);
+void program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
+			u64 pci_addr, u32 type, u64 size);
+void program_ib_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
+			u64 pci_addr, u32 type, u64 size);
+u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size);
+void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
+			size_t size);
+
+static inline u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
+{
+	return mobiveil_csr_read(pcie, off, 0x4);
+}
+
+static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
+				       u32 off)
+{
+	mobiveil_csr_write(pcie, val, off, 0x4);
+}
+
+#endif /* _PCIE_MOBIVEIL_H */
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (3 preceding siblings ...)
  2019-11-20  3:45 ` [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver Z.q. Hou
@ 2019-11-20  3:45 ` Z.q. Hou
  2020-01-13 11:19   ` Andrew Murray
  2019-11-20  3:45 ` [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check Z.q. Hou
                   ` (8 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The Mobiveil GPEX internal MSI/INTx controller may not be used
by other platforms in which the Mobiveil GPEX is integrated.
This patch is to allow these platforms to implement their
specific interrupt initialization.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 3 +++
 drivers/pci/controller/mobiveil/pcie-mobiveil.h      | 7 +++++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
index 2cc424e78d33..3cd93df6fe6e 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
@@ -507,6 +507,9 @@ static int mobiveil_pcie_interrupt_init(struct mobiveil_pcie *pcie)
 	struct resource *res;
 	int ret;
 
+	if (rp->ops->interrupt_init)
+		return rp->ops->interrupt_init(pcie);
+
 	/* map MSI config resource */
 	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
 	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
index e3148078e9dd..18d85806a7fc 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -130,10 +130,17 @@ struct mobiveil_msi {			/* MSI information */
 	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
 };
 
+struct mobiveil_pcie;
+
+struct mobiveil_rp_ops {
+	int (*interrupt_init)(struct mobiveil_pcie *pcie);
+};
+
 struct root_port {
 	char root_bus_nr;
 	void __iomem *config_axi_slave_base;	/* endpoint config base */
 	struct resource *ob_io_res;
+	struct mobiveil_rp_ops *ops;
 	int irq;
 	raw_spinlock_t intx_mask_lock;
 	struct irq_domain *intx_domain;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (4 preceding siblings ...)
  2019-11-20  3:45 ` [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization Z.q. Hou
@ 2019-11-20  3:45 ` Z.q. Hou
  2020-01-13 11:22   ` Andrew Murray
  2019-11-20  3:46 ` [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host Z.q. Hou
                   ` (7 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:45 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The platforms, in which the Mobiveil GPEX is integrated,
may have their specific mechanism to check link up status.
This patch is to enable these platforms to implement theirs.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
---
V9:
 - New patch splited from the #1 of V8 patches to make it easy to review.

 drivers/pci/controller/mobiveil/pcie-mobiveil.c | 3 +++
 drivers/pci/controller/mobiveil/pcie-mobiveil.h | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
index 2773f823c9ea..b9ed2d95641c 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.c
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
@@ -125,6 +125,9 @@ void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
 
 bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
 {
+	if (pcie->ops->link_up)
+		return pcie->ops->link_up(pcie);
+
 	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
 		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
 }
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
index 18d85806a7fc..95d2e7c809b8 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -148,6 +148,10 @@ struct root_port {
 	struct pci_host_bridge *bridge;
 };
 
+struct mobiveil_pab_ops {
+	int (*link_up)(struct mobiveil_pcie *pcie);
+};
+
 struct mobiveil_pcie {
 	struct platform_device *pdev;
 	void __iomem *csr_axi_slave_base;	/* root port config base */
@@ -157,6 +161,7 @@ struct mobiveil_pcie {
 	int ppio_wins;
 	int ob_wins_configured;		/* configured outbound windows */
 	int ib_wins_configured;		/* configured inbound windows */
+	const struct mobiveil_pab_ops *ops;
 	struct root_port rp;
 };
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (5 preceding siblings ...)
  2019-11-20  3:45 ` [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2020-01-13 11:26   ` Andrew Murray
  2019-11-20  3:46 ` [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors Z.q. Hou
                   ` (6 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Make the mobiveil_host_init() function can be used to re-init
host controller's PAB and GPEX CSR register block, as NXP
integrated Mobiveil IP has to reset and then re-init the PAB
and GPEX CSR registers upon hot-reset.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
---
V9:
 - No change

 .../controller/mobiveil/pcie-mobiveil-host.c  | 19 ++++++++++++-------
 .../pci/controller/mobiveil/pcie-mobiveil.h   |  1 +
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
index 3cd93df6fe6e..9bc3da036720 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
@@ -221,18 +221,23 @@ static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
 	writel_relaxed(1, pcie->apb_csr_base + MSI_ENABLE_OFFSET);
 }
 
-static int mobiveil_host_init(struct mobiveil_pcie *pcie)
+int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit)
 {
 	struct root_port *rp = &pcie->rp;
 	struct pci_host_bridge *bridge = rp->bridge;
 	u32 value, pab_ctrl, type;
 	struct resource_entry *win;
 
-	/* setup bus numbers */
-	value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
-	value &= 0xff000000;
-	value |= 0x00ff0100;
-	mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
+	pcie->ib_wins_configured = 0;
+	pcie->ob_wins_configured = 0;
+
+	if (!reinit) {
+		/* setup bus numbers */
+		value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
+		value &= 0xff000000;
+		value |= 0x00ff0100;
+		mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
+	}
 
 	/*
 	 * program Bus Master Enable Bit in Command Register in PAB Config
@@ -569,7 +574,7 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
 	 * configure all inbound and outbound windows and prepare the RC for
 	 * config access
 	 */
-	ret = mobiveil_host_init(pcie);
+	ret = mobiveil_host_init(pcie, false);
 	if (ret) {
 		dev_err(dev, "Failed to initialize host\n");
 		return ret;
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
index 95d2e7c809b8..37116c2a19fe 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -166,6 +166,7 @@ struct mobiveil_pcie {
 };
 
 int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie);
+int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit);
 bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie);
 int mobiveil_bringup_link(struct mobiveil_pcie *pcie);
 void program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (6 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2020-01-13 11:31   ` Andrew Murray
  2019-11-20  3:46 ` [PATCHv9 09/12] dt-bindings: PCI: Add NXP Layerscape SoCs PCIe Gen4 controller Z.q. Hou
                   ` (5 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

There are some 8-bit and 16-bit registers in PCIe configuration
space, so add these accessors accordingly.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
---
V9:
 - No change

 .../pci/controller/mobiveil/pcie-mobiveil.h   | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
index 37116c2a19fe..750a7fd95bc1 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -182,10 +182,33 @@ static inline u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
 	return mobiveil_csr_read(pcie, off, 0x4);
 }
 
+static inline u32 mobiveil_csr_readw(struct mobiveil_pcie *pcie, u32 off)
+{
+	return mobiveil_csr_read(pcie, off, 0x2);
+}
+
+static inline u32 mobiveil_csr_readb(struct mobiveil_pcie *pcie, u32 off)
+{
+	return mobiveil_csr_read(pcie, off, 0x1);
+}
+
+
 static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
 				       u32 off)
 {
 	mobiveil_csr_write(pcie, val, off, 0x4);
 }
 
+static inline void mobiveil_csr_writew(struct mobiveil_pcie *pcie, u32 val,
+				       u32 off)
+{
+	mobiveil_csr_write(pcie, val, off, 0x2);
+}
+
+static inline void mobiveil_csr_writeb(struct mobiveil_pcie *pcie, u32 val,
+				       u32 off)
+{
+	mobiveil_csr_write(pcie, val, off, 0x1);
+}
+
 #endif /* _PCIE_MOBIVEIL_H */
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 09/12] dt-bindings: PCI: Add NXP Layerscape SoCs PCIe Gen4 controller
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (7 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2019-11-20  3:46 ` [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs Z.q. Hou
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Add PCIe Gen4 controller DT bindings of NXP Layerscape SoCs.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Rob Herring <robh@kernel.org>
---
V9:
 - No change

 .../bindings/pci/layerscape-pcie-gen4.txt     | 52 +++++++++++++++++++
 MAINTAINERS                                   |  8 +++
 2 files changed, 60 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt

diff --git a/Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt b/Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt
new file mode 100644
index 000000000000..b40fb5d15d3d
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt
@@ -0,0 +1,52 @@
+NXP Layerscape PCIe Gen4 controller
+
+This PCIe controller is based on the Mobiveil PCIe IP and thus inherits all
+the common properties defined in mobiveil-pcie.txt.
+
+Required properties:
+- compatible: should contain the platform identifier such as:
+  "fsl,lx2160a-pcie"
+- reg: base addresses and lengths of the PCIe controller register blocks.
+  "csr_axi_slave": Bridge config registers
+  "config_axi_slave": PCIe controller registers
+- interrupts: A list of interrupt outputs of the controller. Must contain an
+  entry for each entry in the interrupt-names property.
+- interrupt-names: It could include the following entries:
+  "intr": The interrupt that is asserted for controller interrupts
+  "aer": Asserted for aer interrupt when chip support the aer interrupt with
+	 none MSI/MSI-X/INTx mode,but there is interrupt line for aer.
+  "pme": Asserted for pme interrupt when chip support the pme interrupt with
+	 none MSI/MSI-X/INTx mode,but there is interrupt line for pme.
+- dma-coherent: Indicates that the hardware IP block can ensure the coherency
+  of the data transferred from/to the IP block. This can avoid the software
+  cache flush/invalid actions, and improve the performance significantly.
+- msi-parent : See the generic MSI binding described in
+  Documentation/devicetree/bindings/interrupt-controller/msi.txt.
+
+Example:
+
+	pcie@3400000 {
+		compatible = "fsl,lx2160a-pcie";
+		reg = <0x00 0x03400000 0x0 0x00100000   /* controller registers */
+		       0x80 0x00000000 0x0 0x00001000>; /* configuration space */
+		reg-names = "csr_axi_slave", "config_axi_slave";
+		interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+			     <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+			     <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+		interrupt-names = "aer", "pme", "intr";
+		#address-cells = <3>;
+		#size-cells = <2>;
+		device_type = "pci";
+		apio-wins = <8>;
+		ppio-wins = <8>;
+		dma-coherent;
+		bus-range = <0x0 0xff>;
+		msi-parent = <&its>;
+		ranges = <0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>;
+		#interrupt-cells = <1>;
+		interrupt-map-mask = <0 0 0 7>;
+		interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
+				<0000 0 0 2 &gic 0 0 GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
+				<0000 0 0 3 &gic 0 0 GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>,
+				<0000 0 0 4 &gic 0 0 GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>;
+	};
diff --git a/MAINTAINERS b/MAINTAINERS
index a4ad99619e53..2f68f71896c7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12569,6 +12569,14 @@ L:	linux-arm-kernel@lists.infradead.org
 S:	Maintained
 F:	drivers/pci/controller/dwc/*layerscape*
 
+PCI DRIVER FOR NXP LAYERSCAPE GEN4 CONTROLLER
+M:	Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
+L:	linux-pci@vger.kernel.org
+L:	linux-arm-kernel@lists.infradead.org
+S:	Maintained
+F:	Documentation/devicetree/bindings/pci/layerscape-pcie-gen4.txt
+F:	drivers/pci/controller/mobibeil/pcie-layerscape-gen4.c
+
 PCI DRIVER FOR GENERIC OF HOSTS
 M:	Will Deacon <will@kernel.org>
 L:	linux-pci@vger.kernel.org
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (8 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 09/12] dt-bindings: PCI: Add NXP Layerscape SoCs PCIe Gen4 controller Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2020-01-13 12:02   ` Andrew Murray
  2019-11-20  3:46 ` [PATCHv9 11/12] arm64: dts: lx2160a: Add PCIe controller DT nodes Z.q. Hou
                   ` (3 subsequent siblings)
  13 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

This PCIe controller is based on the Mobiveil GPEX IP, which is
compatible with the PCI Express™ Base Specification, Revision 4.0.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
---
 drivers/pci/controller/mobiveil/Kconfig       |  10 +
 drivers/pci/controller/mobiveil/Makefile      |   1 +
 .../mobiveil/pcie-layerscape-gen4.c           | 274 ++++++++++++++++++
 .../pci/controller/mobiveil/pcie-mobiveil.h   |  16 +-
 4 files changed, 299 insertions(+), 2 deletions(-)
 create mode 100644 drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c

diff --git a/drivers/pci/controller/mobiveil/Kconfig b/drivers/pci/controller/mobiveil/Kconfig
index 64343c07bfed..c823be8dab1c 100644
--- a/drivers/pci/controller/mobiveil/Kconfig
+++ b/drivers/pci/controller/mobiveil/Kconfig
@@ -21,4 +21,14 @@ config PCIE_MOBIVEIL_PLAT
 	  Soft IP. It has up to 8 outbound and inbound windows
 	  for address translation and it is a PCIe Gen4 IP.
 
+config PCIE_LAYERSCAPE_GEN4
+	bool "Freescale Layerscape PCIe Gen4 controller"
+	depends on PCI
+	depends on OF && (ARM64 || ARCH_LAYERSCAPE)
+	depends on PCI_MSI_IRQ_DOMAIN
+	select PCIE_MOBIVEIL_HOST
+	help
+	  Say Y here if you want PCIe Gen4 controller support on
+	  Layerscape SoCs. The PCIe controller can work in RC or
+	  EP mode according to RCW[HOST_AGT_PEX] setting.
 endmenu
diff --git a/drivers/pci/controller/mobiveil/Makefile b/drivers/pci/controller/mobiveil/Makefile
index 9fb6d1c6504d..99d879de32d6 100644
--- a/drivers/pci/controller/mobiveil/Makefile
+++ b/drivers/pci/controller/mobiveil/Makefile
@@ -2,3 +2,4 @@
 obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
 obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
 obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
+obj-$(CONFIG_PCIE_LAYERSCAPE_GEN4) += pcie-layerscape-gen4.o
diff --git a/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
new file mode 100644
index 000000000000..6c0d3e2532db
--- /dev/null
+++ b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
@@ -0,0 +1,274 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe Gen4 host controller driver for NXP Layerscape SoCs
+ *
+ * Copyright 2019 NXP
+ *
+ * Author: Zhiqiang Hou <Zhiqiang.Hou@nxp.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/interrupt.h>
+#include <linux/init.h>
+#include <linux/of_pci.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/resource.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
+
+#include "pcie-mobiveil.h"
+
+/* LUT and PF control registers */
+#define PCIE_LUT_OFF			0x80000
+#define PCIE_PF_OFF			0xc0000
+#define PCIE_PF_INT_STAT		0x18
+#define PF_INT_STAT_PABRST		BIT(31)
+
+#define PCIE_PF_DBG			0x7fc
+#define PF_DBG_LTSSM_MASK		0x3f
+#define PF_DBG_LTSSM_L0			0x2d /* L0 state */
+#define PF_DBG_WE			BIT(31)
+#define PF_DBG_PABR			BIT(27)
+
+#define to_ls_pcie_g4(x)		platform_get_drvdata((x)->pdev)
+
+struct ls_pcie_g4 {
+	struct mobiveil_pcie pci;
+	struct delayed_work dwork;
+	int irq;
+};
+
+static inline u32 ls_pcie_g4_lut_readl(struct ls_pcie_g4 *pcie, u32 off)
+{
+	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
+}
+
+static inline void ls_pcie_g4_lut_writel(struct ls_pcie_g4 *pcie,
+					 u32 off, u32 val)
+{
+	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
+}
+
+static inline u32 ls_pcie_g4_pf_readl(struct ls_pcie_g4 *pcie, u32 off)
+{
+	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off);
+}
+
+static inline void ls_pcie_g4_pf_writel(struct ls_pcie_g4 *pcie,
+					u32 off, u32 val)
+{
+	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off);
+}
+
+static bool ls_pcie_g4_is_bridge(struct ls_pcie_g4 *pcie)
+{
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+	u32 header_type;
+
+	header_type = mobiveil_csr_readb(mv_pci, PCI_HEADER_TYPE);
+	header_type &= 0x7f;
+
+	return header_type == PCI_HEADER_TYPE_BRIDGE;
+}
+
+static int ls_pcie_g4_link_up(struct mobiveil_pcie *pci)
+{
+	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(pci);
+	u32 state;
+
+	state = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
+	state =	state & PF_DBG_LTSSM_MASK;
+
+	if (state == PF_DBG_LTSSM_L0)
+		return 1;
+
+	return 0;
+}
+
+static void ls_pcie_g4_disable_interrupt(struct ls_pcie_g4 *pcie)
+{
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+
+	mobiveil_csr_writel(mv_pci, 0, PAB_INTP_AMBA_MISC_ENB);
+}
+
+static void ls_pcie_g4_enable_interrupt(struct ls_pcie_g4 *pcie)
+{
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+	u32 val;
+
+	/* Clear the interrupt status */
+	mobiveil_csr_writel(mv_pci, 0xffffffff, PAB_INTP_AMBA_MISC_STAT);
+
+	val = PAB_INTP_INTX_MASK | PAB_INTP_MSI | PAB_INTP_RESET |
+	      PAB_INTP_PCIE_UE | PAB_INTP_IE_PMREDI | PAB_INTP_IE_EC;
+	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_ENB);
+}
+
+static void ls_pcie_g4_reinit_hw(struct ls_pcie_g4 *pcie)
+{
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+	struct device *dev = &mv_pci->pdev->dev;
+	u32 val, act_stat;
+	int to = 100;
+
+	/* Poll for pab_csb_reset to set and PAB activity to clear */
+	do {
+		usleep_range(10, 15);
+		val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_INT_STAT);
+		act_stat = mobiveil_csr_readl(mv_pci, PAB_ACTIVITY_STAT);
+	} while (((val & PF_INT_STAT_PABRST) == 0 || act_stat) && to--);
+	if (to < 0) {
+		dev_err(dev, "Poll PABRST&PABACT timeout\n");
+		return;
+	}
+
+	/* clear PEX_RESET bit in PEX_PF0_DBG register */
+	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
+	val |= PF_DBG_WE;
+	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
+
+	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
+	val |= PF_DBG_PABR;
+	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
+
+	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
+	val &= ~PF_DBG_WE;
+	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
+
+	mobiveil_host_init(mv_pci, true);
+
+	to = 100;
+	while (!ls_pcie_g4_link_up(mv_pci) && to--)
+		usleep_range(200, 250);
+	if (to < 0)
+		dev_err(dev, "PCIe link training timeout\n");
+}
+
+static irqreturn_t ls_pcie_g4_isr(int irq, void *dev_id)
+{
+	struct ls_pcie_g4 *pcie = (struct ls_pcie_g4 *)dev_id;
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+	u32 val;
+
+	val = mobiveil_csr_readl(mv_pci, PAB_INTP_AMBA_MISC_STAT);
+	if (!val)
+		return IRQ_NONE;
+
+	if (val & PAB_INTP_RESET) {
+		ls_pcie_g4_disable_interrupt(pcie);
+		schedule_delayed_work(&pcie->dwork, msecs_to_jiffies(1));
+	}
+
+	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_STAT);
+
+	return IRQ_HANDLED;
+}
+
+static int ls_pcie_g4_interrupt_init(struct mobiveil_pcie *mv_pci)
+{
+	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(mv_pci);
+	struct platform_device *pdev = mv_pci->pdev;
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	pcie->irq = platform_get_irq_byname(pdev, "intr");
+	if (pcie->irq < 0) {
+		dev_err(dev, "Can't get 'intr' IRQ, errno = %d\n", pcie->irq);
+		return pcie->irq;
+	}
+	ret = devm_request_irq(dev, pcie->irq, ls_pcie_g4_isr,
+			       IRQF_SHARED, pdev->name, pcie);
+	if (ret) {
+		dev_err(dev, "Can't register PCIe IRQ, errno = %d\n", ret);
+		return  ret;
+	}
+
+	return 0;
+}
+
+static void ls_pcie_g4_reset(struct work_struct *work)
+{
+	struct delayed_work *dwork = container_of(work, struct delayed_work,
+						  work);
+	struct ls_pcie_g4 *pcie = container_of(dwork, struct ls_pcie_g4, dwork);
+	struct mobiveil_pcie *mv_pci = &pcie->pci;
+	u16 ctrl;
+
+	ctrl = mobiveil_csr_readw(mv_pci, PCI_BRIDGE_CONTROL);
+	ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
+	mobiveil_csr_writew(mv_pci, ctrl, PCI_BRIDGE_CONTROL);
+	ls_pcie_g4_reinit_hw(pcie);
+	ls_pcie_g4_enable_interrupt(pcie);
+}
+
+static struct mobiveil_rp_ops ls_pcie_g4_rp_ops = {
+	.interrupt_init = ls_pcie_g4_interrupt_init,
+};
+
+static const struct mobiveil_pab_ops ls_pcie_g4_pab_ops = {
+	.link_up = ls_pcie_g4_link_up,
+};
+
+static int __init ls_pcie_g4_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct pci_host_bridge *bridge;
+	struct mobiveil_pcie *mv_pci;
+	struct ls_pcie_g4 *pcie;
+	struct device_node *np = dev->of_node;
+	int ret;
+
+	if (!of_parse_phandle(np, "msi-parent", 0)) {
+		dev_err(dev, "Failed to find msi-parent\n");
+		return -EINVAL;
+	}
+
+	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
+	if (!bridge)
+		return -ENOMEM;
+
+	pcie = pci_host_bridge_priv(bridge);
+	mv_pci = &pcie->pci;
+
+	mv_pci->pdev = pdev;
+	mv_pci->ops = &ls_pcie_g4_pab_ops;
+	mv_pci->rp.ops = &ls_pcie_g4_rp_ops;
+	mv_pci->rp.bridge = bridge;
+
+	platform_set_drvdata(pdev, pcie);
+
+	INIT_DELAYED_WORK(&pcie->dwork, ls_pcie_g4_reset);
+
+	ret = mobiveil_pcie_host_probe(mv_pci);
+	if (ret) {
+		dev_err(dev, "Fail to probe\n");
+		return  ret;
+	}
+
+	if (!ls_pcie_g4_is_bridge(pcie))
+		return -ENODEV;
+
+	ls_pcie_g4_enable_interrupt(pcie);
+
+	return 0;
+}
+
+static const struct of_device_id ls_pcie_g4_of_match[] = {
+	{ .compatible = "fsl,lx2160a-pcie", },
+	{ },
+};
+
+static struct platform_driver ls_pcie_g4_driver = {
+	.driver = {
+		.name = "layerscape-pcie-gen4",
+		.of_match_table = ls_pcie_g4_of_match,
+		.suppress_bind_attrs = true,
+	},
+};
+
+builtin_platform_driver_probe(ls_pcie_g4_driver, ls_pcie_g4_probe);
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
index 750a7fd95bc1..c57a68d2bac4 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
@@ -43,6 +43,8 @@
 #define  PAGE_LO_MASK			0x3ff
 #define  PAGE_SEL_OFFSET_SHIFT		10
 
+#define PAB_ACTIVITY_STAT		0x81c
+
 #define PAB_AXI_PIO_CTRL		0x0840
 #define  APIO_EN_MASK			0xf
 
@@ -51,8 +53,18 @@
 
 #define PAB_INTP_AMBA_MISC_ENB		0x0b0c
 #define PAB_INTP_AMBA_MISC_STAT		0x0b1c
-#define  PAB_INTP_INTX_MASK		0x01e0
-#define  PAB_INTP_MSI_MASK		0x8
+#define  PAB_INTP_RESET			BIT(1)
+#define  PAB_INTP_MSI			BIT(3)
+#define  PAB_INTP_INTA			BIT(5)
+#define  PAB_INTP_INTB			BIT(6)
+#define  PAB_INTP_INTC			BIT(7)
+#define  PAB_INTP_INTD			BIT(8)
+#define  PAB_INTP_PCIE_UE		BIT(9)
+#define  PAB_INTP_IE_PMREDI		BIT(29)
+#define  PAB_INTP_IE_EC			BIT(30)
+#define  PAB_INTP_MSI_MASK		PAB_INTP_MSI
+#define  PAB_INTP_INTX_MASK		(PAB_INTP_INTA | PAB_INTP_INTB |\
+					PAB_INTP_INTC | PAB_INTP_INTD)
 
 #define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
 #define  WIN_ENABLE_SHIFT		0
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 11/12] arm64: dts: lx2160a: Add PCIe controller DT nodes
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (9 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2019-11-20  3:46 ` [PATCHv9 12/12] arm64: defconfig: Enable CONFIG_PCIE_LAYERSCAPE_GEN4 Z.q. Hou
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

The LX2160A integrated 6 PCIe Gen4 controllers.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
---
V9:
 - No change

 .../arm64/boot/dts/freescale/fsl-lx2160a.dtsi | 163 ++++++++++++++++++
 1 file changed, 163 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
index e883fe0fc1b7..ecc5bd90aa5d 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
@@ -1055,5 +1055,168 @@
 				};
 			};
 		};
+
+		pcie@3400000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03400000 0x0 0x00100000   /* controller registers */
+			       0x80 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <8>;
+			ppio-wins = <8>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		pcie@3500000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03500000 0x0 0x00100000   /* controller registers */
+			       0x88 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <8>;
+			ppio-wins = <8>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0x88 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 117 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		pcie@3600000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03600000 0x0 0x00100000   /* controller registers */
+			       0x90 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <256>;
+			ppio-wins = <24>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0x90 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 122 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		pcie@3700000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03700000 0x0 0x00100000   /* controller registers */
+			       0x98 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <8>;
+			ppio-wins = <8>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0x98 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 124 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		pcie@3800000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03800000 0x0 0x00100000   /* controller registers */
+			       0xa0 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 128 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <256>;
+			ppio-wins = <24>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0xa0 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 129 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 130 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 132 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
+		pcie@3900000 {
+			compatible = "fsl,lx2160a-pcie";
+			reg = <0x00 0x03900000 0x0 0x00100000   /* controller registers */
+			       0xa8 0x00000000 0x0 0x00001000>; /* configuration space */
+			reg-names = "csr_axi_slave", "config_axi_slave";
+			interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>, /* AER interrupt */
+				     <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */
+				     <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
+			interrupt-names = "aer", "pme", "intr";
+			#address-cells = <3>;
+			#size-cells = <2>;
+			device_type = "pci";
+			dma-coherent;
+			apio-wins = <8>;
+			ppio-wins = <8>;
+			bus-range = <0x0 0xff>;
+			ranges = <0x82000000 0x0 0x40000000 0xa8 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */
+			msi-parent = <&its>;
+			#interrupt-cells = <1>;
+			interrupt-map-mask = <0 0 0 7>;
+			interrupt-map = <0000 0 0 1 &gic 0 0 GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 2 &gic 0 0 GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 3 &gic 0 0 GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>,
+					<0000 0 0 4 &gic 0 0 GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>;
+			status = "disabled";
+		};
+
 	};
 };
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* [PATCHv9 12/12] arm64: defconfig: Enable CONFIG_PCIE_LAYERSCAPE_GEN4
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (10 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 11/12] arm64: dts: lx2160a: Add PCIe controller DT nodes Z.q. Hou
@ 2019-11-20  3:46 ` Z.q. Hou
  2019-11-20  9:57 ` [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Russell King - ARM Linux admin
  2019-12-13 18:37 ` Olof Johansson
  13 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20  3:46 UTC (permalink / raw)
  To: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray
  Cc: Mingkai Hu, M.h. Lian, Xiaowei Bao, Z.q. Hou

From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Enable the PCIe Gen4 controller driver for Layerscape SoCs.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
---
V9:
 - No change

 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 7fa92defb964..0e51207b5ed5 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -199,6 +199,7 @@ CONFIG_PCI_HOST_THUNDER_PEM=y
 CONFIG_PCI_HOST_THUNDER_ECAM=y
 CONFIG_PCIE_ROCKCHIP_HOST=m
 CONFIG_PCI_LAYERSCAPE=y
+CONFIG_PCIE_LAYERSCAPE_GEN4=y
 CONFIG_PCI_HISI=y
 CONFIG_PCIE_QCOM=y
 CONFIG_PCIE_ARMADA_8K=y
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (11 preceding siblings ...)
  2019-11-20  3:46 ` [PATCHv9 12/12] arm64: defconfig: Enable CONFIG_PCIE_LAYERSCAPE_GEN4 Z.q. Hou
@ 2019-11-20  9:57 ` Russell King - ARM Linux admin
  2019-11-20 10:30   ` Z.q. Hou
  2019-12-13 18:37 ` Olof Johansson
  13 siblings, 1 reply; 55+ messages in thread
From: Russell King - ARM Linux admin @ 2019-11-20  9:57 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray, M.h. Lian, Xiaowei Bao, Mingkai Hu

On Wed, Nov 20, 2019 at 03:45:17AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> This patch set is to recode the Mobiveil driver and add
> PCIe support for NXP Layerscape series SoCs integrated
> Mobiveil's PCIe Gen4 controller.

How many PCIe cards have been tested to work/don't work with this?

I need:

PCI: mobiveil: ls_pcie_g4: fix SError when accessing config space
PCI: mobiveil: ls_pcie_g4: add Workaround for A-011451
PCI: mobiveil: ls_pcie_g4: add Workaround for A-011577

to successfully boot with a Mellanox card plugged in with a previous
revision of these patches.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2019-11-20  9:57 ` [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Russell King - ARM Linux admin
@ 2019-11-20 10:30   ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2019-11-20 10:30 UTC (permalink / raw)
  To: Russell King - ARM Linux admin
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, andrew.murray, M.h. Lian, Xiaowei Bao, Mingkai Hu

Hi Russell,

> -----Original Message-----
> From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> Sent: 2019年11月20日 17:57
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com;
> andrew.murray@arm.com; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei
> Bao <xiaowei.bao@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>
> Subject: Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4
> driver for NXP Layerscape SoCs
> 
> On Wed, Nov 20, 2019 at 03:45:17AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > This patch set is to recode the Mobiveil driver and add PCIe support
> > for NXP Layerscape series SoCs integrated Mobiveil's PCIe Gen4
> > controller.
> 
> How many PCIe cards have been tested to work/don't work with this?
> 
> I need:
> 
> PCI: mobiveil: ls_pcie_g4: fix SError when accessing config space
> PCI: mobiveil: ls_pcie_g4: add Workaround for A-011451
> PCI: mobiveil: ls_pcie_g4: add Workaround for A-011577
> 
> to successfully boot with a Mellanox card plugged in with a previous revision
> of these patches.
>

Yes, we need to apply these NXP internal maintained workarounds on top of
this series. I only tested Intel e1000e NIC with this patch set + these 3
workarounds.

Thanks,
Zhiqiang
 
> --
> RMK's Patch system:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
> .armlinux.org.uk%2Fdeveloper%2Fpatches%2F&amp;data=02%7C01%7Czhiq
> iang.hou%40nxp.com%7C69f6fb1f4fd44f3fca3808d76da01440%7C686ea1d
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637098406606503361&amp;sd
> ata=wOLWzKfZZoiP%2FZpTOw5zr4enpuNImz45RM8Hy80aUdI%3D&amp;res
> erved=0
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps
> up According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
                   ` (12 preceding siblings ...)
  2019-11-20  9:57 ` [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Russell King - ARM Linux admin
@ 2019-12-13 18:37 ` Olof Johansson
  2019-12-17  2:50   ` Z.q. Hou
  13 siblings, 1 reply; 55+ messages in thread
From: Olof Johansson @ 2019-12-13 18:37 UTC (permalink / raw)
  To: Z.q. Hou, bhelgaas
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, robh+dt,
	arnd, mark.rutland, l.subrahmanya, shawnguo, m.karthikeyan,
	Leo Li, lorenzo.pieralisi, catalin.marinas, will.deacon,
	andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi!

On Tue, Nov 19, 2019 at 7:45 PM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
>
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
>
> This patch set is to recode the Mobiveil driver and add
> PCIe support for NXP Layerscape series SoCs integrated
> Mobiveil's PCIe Gen4 controller.

Can we get a respin for this on top of the 5.5 merge window material?
Given that it's a bunch of refactorings, many of them don't apply on
top of the material that was merged.

I'd love to see these go in sooner rather than later so I can start
getting -next running on ls2160a here.


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2019-12-13 18:37 ` Olof Johansson
@ 2019-12-17  2:50   ` Z.q. Hou
  2020-01-10 15:33     ` Lorenzo Pieralisi
  0 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2019-12-17  2:50 UTC (permalink / raw)
  To: Olof Johansson, bhelgaas, lorenzo.pieralisi
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, robh+dt,
	arnd, mark.rutland, l.subrahmanya, shawnguo, m.karthikeyan,
	Leo Li, catalin.marinas, will.deacon, andrew.murray, Mingkai Hu,
	M.h. Lian, Xiaowei Bao

Hi Lorenzo,

The v9 patches have addressed the comments from Andrew, and it has been dried about 1 month, can you help to apply them?

Thanks,
Zhiqiang

> -----Original Message-----
> From: Olof Johansson <olof@lixom.net>
> Sent: 2019年12月14日 2:37
> To: Z.q. Hou <zhiqiang.hou@nxp.com>; bhelgaas@google.com
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> robh+dt@kernel.org; arnd@arndb.de; mark.rutland@arm.com;
> l.subrahmanya@mobiveil.co.in; shawnguo@kernel.org;
> m.karthikeyan@mobiveil.co.in; Leo Li <leoyang.li@nxp.com>;
> lorenzo.pieralisi@arm.com; catalin.marinas@arm.com;
> will.deacon@arm.com; andrew.murray@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4
> driver for NXP Layerscape SoCs
> 
> Hi!
> 
> On Tue, Nov 19, 2019 at 7:45 PM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> >
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > This patch set is to recode the Mobiveil driver and add PCIe support
> > for NXP Layerscape series SoCs integrated Mobiveil's PCIe Gen4
> > controller.
> 
> Can we get a respin for this on top of the 5.5 merge window material?
> Given that it's a bunch of refactorings, many of them don't apply on top of
> the material that was merged.
> 
> I'd love to see these go in sooner rather than later so I can start getting -next
> running on ls2160a here.
> 
> 
> -Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2019-12-17  2:50   ` Z.q. Hou
@ 2020-01-10 15:33     ` Lorenzo Pieralisi
  2020-01-10 17:05       ` Olof Johansson
  0 siblings, 1 reply; 55+ messages in thread
From: Lorenzo Pieralisi @ 2020-01-10 15:33 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: Olof Johansson, bhelgaas, linux-pci, linux-arm-kernel,
	devicetree, linux-kernel, robh+dt, arnd, mark.rutland,
	l.subrahmanya, shawnguo, m.karthikeyan, Leo Li, catalin.marinas,
	will.deacon, andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Tue, Dec 17, 2019 at 02:50:15AM +0000, Z.q. Hou wrote:
> Hi Lorenzo,
> 
> The v9 patches have addressed the comments from Andrew, and it has
> been dried about 1 month, can you help to apply them?

We shall have a look beginning of next week, sorry for the delay
in getting back to you.

Lorenzo

> Thanks,
> Zhiqiang
> 
> > -----Original Message-----
> > From: Olof Johansson <olof@lixom.net>
> > Sent: 2019年12月14日 2:37
> > To: Z.q. Hou <zhiqiang.hou@nxp.com>; bhelgaas@google.com
> > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> > robh+dt@kernel.org; arnd@arndb.de; mark.rutland@arm.com;
> > l.subrahmanya@mobiveil.co.in; shawnguo@kernel.org;
> > m.karthikeyan@mobiveil.co.in; Leo Li <leoyang.li@nxp.com>;
> > lorenzo.pieralisi@arm.com; catalin.marinas@arm.com;
> > will.deacon@arm.com; andrew.murray@arm.com; Mingkai Hu
> > <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> > <xiaowei.bao@nxp.com>
> > Subject: Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4
> > driver for NXP Layerscape SoCs
> > 
> > Hi!
> > 
> > On Tue, Nov 19, 2019 at 7:45 PM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > >
> > > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > >
> > > This patch set is to recode the Mobiveil driver and add PCIe support
> > > for NXP Layerscape series SoCs integrated Mobiveil's PCIe Gen4
> > > controller.
> > 
> > Can we get a respin for this on top of the 5.5 merge window material?
> > Given that it's a bunch of refactorings, many of them don't apply on top of
> > the material that was merged.
> > 
> > I'd love to see these go in sooner rather than later so I can start getting -next
> > running on ls2160a here.
> > 
> > 
> > -Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-01-10 15:33     ` Lorenzo Pieralisi
@ 2020-01-10 17:05       ` Olof Johansson
  2020-02-06 10:57         ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Olof Johansson @ 2020-01-10 17:05 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Z.q. Hou, bhelgaas, linux-pci, linux-arm-kernel, devicetree,
	linux-kernel, robh+dt, arnd, mark.rutland, l.subrahmanya,
	shawnguo, m.karthikeyan, Leo Li, catalin.marinas, will.deacon,
	andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Fri, Jan 10, 2020 at 7:33 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Tue, Dec 17, 2019 at 02:50:15AM +0000, Z.q. Hou wrote:
> > Hi Lorenzo,
> >
> > The v9 patches have addressed the comments from Andrew, and it has
> > been dried about 1 month, can you help to apply them?
>
> We shall have a look beginning of next week, sorry for the delay
> in getting back to you.

Note that the patch set no longer applies since the refactorings
conflict with new development by others.

Zhiqiang, can you rebase and post a new version of the patch set?


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
  2019-11-20  3:45 ` [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure Z.q. Hou
@ 2020-01-13 10:09   ` Andrew Murray
  2020-02-06 11:04     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 10:09 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:23AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The Mobiveil PCIe controller can work in either Root Complex
> mode or Endpoint mode. So introduce a new structure root_port,
> and abstract the RC related members into it.

The first sentence explains the trigger for this work, the second
explains what you are changing, it would be helpful to also describe
why you need to make this change. You could do this by extending the
last sentence, e.g.

"So introduce a new structure root_port, and abstract the RC
 related members into it such that it can be used by both ..."

As this series doesn't actually add a EP driver, this abstraction
isn't needed now - but it is nice to have - it may be helpful to explain
this.

The email subject could also more precisely explain what this patch
does.


> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.

Indeed, it's much nicer to review - thanks.


> 
>  drivers/pci/controller/pcie-mobiveil.c | 99 ++++++++++++++++----------
>  1 file changed, 60 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
> index 3a696ca45bfa..5fd26e376af2 100644
> --- a/drivers/pci/controller/pcie-mobiveil.c
> +++ b/drivers/pci/controller/pcie-mobiveil.c
> @@ -3,7 +3,10 @@
>   * PCIe host controller driver for Mobiveil PCIe Host controller
>   *
>   * Copyright (c) 2018 Mobiveil Inc.
> + * Copyright 2019 NXP
> + *
>   * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

As per my previous feedback, I'm not sure the value of using the term refactor
or a synonym of it. And I certaintly don't want to encourage anyone that
modifies this file to add a similar tag when the information is easily visible
via GIT and the get_maintainers script.

>   */
>  
>  #include <linux/delay.h>
> @@ -138,22 +141,27 @@ struct mobiveil_msi {			/* MSI information */
>  	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
>  };
>  
> +struct root_port {
> +	char root_bus_nr;
> +	void __iomem *config_axi_slave_base;	/* endpoint config base */
> +	struct resource *ob_io_res;
> +	int irq;
> +	raw_spinlock_t intx_mask_lock;
> +	struct irq_domain *intx_domain;
> +	struct mobiveil_msi msi;
> +	struct pci_host_bridge *bridge;
> +};

Please prefix with mobiveil given we have mobiveil related structures
inside it.

(Also on your respin, please rebase as per Olof's feedback).

Thanks,

Andrew Murray

> +
>  struct mobiveil_pcie {
>  	struct platform_device *pdev;
> -	void __iomem *config_axi_slave_base;	/* endpoint config base */
>  	void __iomem *csr_axi_slave_base;	/* root port config base */
>  	void __iomem *apb_csr_base;	/* MSI register base */
>  	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> -	struct irq_domain *intx_domain;
> -	raw_spinlock_t intx_mask_lock;
> -	int irq;
>  	int apio_wins;
>  	int ppio_wins;
>  	int ob_wins_configured;		/* configured outbound windows */
>  	int ib_wins_configured;		/* configured inbound windows */
> -	struct resource *ob_io_res;
> -	char root_bus_nr;
> -	struct mobiveil_msi msi;
> +	struct root_port rp;
>  };
>  
>  /*
> @@ -281,16 +289,17 @@ static bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
>  static bool mobiveil_pcie_valid_device(struct pci_bus *bus, unsigned int devfn)
>  {
>  	struct mobiveil_pcie *pcie = bus->sysdata;
> +	struct root_port *rp = &pcie->rp;
>  
>  	/* Only one device down on each root port */
> -	if ((bus->number == pcie->root_bus_nr) && (devfn > 0))
> +	if ((bus->number == rp->root_bus_nr) && (devfn > 0))
>  		return false;
>  
>  	/*
>  	 * Do not read more than one device on the bus directly
>  	 * attached to RC
>  	 */
> -	if ((bus->primary == pcie->root_bus_nr) && (PCI_SLOT(devfn) > 0))
> +	if ((bus->primary == rp->root_bus_nr) && (PCI_SLOT(devfn) > 0))
>  		return false;
>  
>  	return true;
> @@ -304,13 +313,14 @@ static void __iomem *mobiveil_pcie_map_bus(struct pci_bus *bus,
>  					   unsigned int devfn, int where)
>  {
>  	struct mobiveil_pcie *pcie = bus->sysdata;
> +	struct root_port *rp = &pcie->rp;
>  	u32 value;
>  
>  	if (!mobiveil_pcie_valid_device(bus, devfn))
>  		return NULL;
>  
>  	/* RC config access */
> -	if (bus->number == pcie->root_bus_nr)
> +	if (bus->number == rp->root_bus_nr)
>  		return pcie->csr_axi_slave_base + where;
>  
>  	/*
> @@ -325,7 +335,7 @@ static void __iomem *mobiveil_pcie_map_bus(struct pci_bus *bus,
>  
>  	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_PEX_WIN_L(WIN_NUM_0));
>  
> -	return pcie->config_axi_slave_base + where;
> +	return rp->config_axi_slave_base + where;
>  }
>  
>  static struct pci_ops mobiveil_pcie_ops = {
> @@ -339,7 +349,8 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
>  	struct irq_chip *chip = irq_desc_get_chip(desc);
>  	struct mobiveil_pcie *pcie = irq_desc_get_handler_data(desc);
>  	struct device *dev = &pcie->pdev->dev;
> -	struct mobiveil_msi *msi = &pcie->msi;
> +	struct root_port *rp = &pcie->rp;
> +	struct mobiveil_msi *msi = &rp->msi;
>  	u32 msi_data, msi_addr_lo, msi_addr_hi;
>  	u32 intr_status, msi_status;
>  	unsigned long shifted_status;
> @@ -365,7 +376,7 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
>  		shifted_status >>= PAB_INTX_START;
>  		do {
>  			for_each_set_bit(bit, &shifted_status, PCI_NUM_INTX) {
> -				virq = irq_find_mapping(pcie->intx_domain,
> +				virq = irq_find_mapping(rp->intx_domain,
>  							bit + 1);
>  				if (virq)
>  					generic_handle_irq(virq);
> @@ -424,15 +435,16 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
>  	struct device *dev = &pcie->pdev->dev;
>  	struct platform_device *pdev = pcie->pdev;
>  	struct device_node *node = dev->of_node;
> +	struct root_port *rp = &pcie->rp;
>  	struct resource *res;
>  
>  	/* map config resource */
>  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
>  					   "config_axi_slave");
> -	pcie->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> -	if (IS_ERR(pcie->config_axi_slave_base))
> -		return PTR_ERR(pcie->config_axi_slave_base);
> -	pcie->ob_io_res = res;
> +	rp->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> +	if (IS_ERR(rp->config_axi_slave_base))
> +		return PTR_ERR(rp->config_axi_slave_base);
> +	rp->ob_io_res = res;
>  
>  	/* map csr resource */
>  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> @@ -455,9 +467,9 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
>  	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
>  		pcie->ppio_wins = MAX_PIO_WINDOWS;
>  
> -	pcie->irq = platform_get_irq(pdev, 0);
> -	if (pcie->irq <= 0) {
> -		dev_err(dev, "failed to map IRQ: %d\n", pcie->irq);
> +	rp->irq = platform_get_irq(pdev, 0);
> +	if (rp->irq <= 0) {
> +		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
>  		return -ENODEV;
>  	}
>  
> @@ -564,9 +576,9 @@ static int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
>  static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
>  {
>  	phys_addr_t msg_addr = pcie->pcie_reg_base;
> -	struct mobiveil_msi *msi = &pcie->msi;
> +	struct mobiveil_msi *msi = &pcie->rp.msi;
>  
> -	pcie->msi.num_of_vectors = PCI_NUM_MSI;
> +	msi->num_of_vectors = PCI_NUM_MSI;
>  	msi->msi_pages_phys = (phys_addr_t)msg_addr;
>  
>  	writel_relaxed(lower_32_bits(msg_addr),
> @@ -579,7 +591,8 @@ static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
>  
>  static int mobiveil_host_init(struct mobiveil_pcie *pcie)
>  {
> -	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> +	struct root_port *rp = &pcie->rp;
> +	struct pci_host_bridge *bridge = rp->bridge;
>  	u32 value, pab_ctrl, type;
>  	struct resource_entry *win;
>  
> @@ -629,8 +642,8 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
>  	 */
>  
>  	/* config outbound translation window */
> -	program_ob_windows(pcie, WIN_NUM_0, pcie->ob_io_res->start, 0,
> -			   CFG_WINDOW_TYPE, resource_size(pcie->ob_io_res));
> +	program_ob_windows(pcie, WIN_NUM_0, rp->ob_io_res->start, 0,
> +			   CFG_WINDOW_TYPE, resource_size(rp->ob_io_res));
>  
>  	/* memory inbound translation window */
>  	program_ib_windows(pcie, WIN_NUM_0, 0, 0, MEM_WINDOW_TYPE, IB_WIN_SIZE);
> @@ -667,32 +680,36 @@ static void mobiveil_mask_intx_irq(struct irq_data *data)
>  {
>  	struct irq_desc *desc = irq_to_desc(data->irq);
>  	struct mobiveil_pcie *pcie;
> +	struct root_port *rp;
>  	unsigned long flags;
>  	u32 mask, shifted_val;
>  
>  	pcie = irq_desc_get_chip_data(desc);
> +	rp = &pcie->rp;
>  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
>  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
>  	shifted_val &= ~mask;
>  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
>  }
>  
>  static void mobiveil_unmask_intx_irq(struct irq_data *data)
>  {
>  	struct irq_desc *desc = irq_to_desc(data->irq);
>  	struct mobiveil_pcie *pcie;
> +	struct root_port *rp;
>  	unsigned long flags;
>  	u32 shifted_val, mask;
>  
>  	pcie = irq_desc_get_chip_data(desc);
> +	rp = &pcie->rp;
>  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
>  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
>  	shifted_val |= mask;
>  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
>  }
>  
>  static struct irq_chip intx_irq_chip = {
> @@ -760,7 +777,7 @@ static int mobiveil_irq_msi_domain_alloc(struct irq_domain *domain,
>  					 unsigned int nr_irqs, void *args)
>  {
>  	struct mobiveil_pcie *pcie = domain->host_data;
> -	struct mobiveil_msi *msi = &pcie->msi;
> +	struct mobiveil_msi *msi = &pcie->rp.msi;
>  	unsigned long bit;
>  
>  	WARN_ON(nr_irqs != 1);
> @@ -787,7 +804,7 @@ static void mobiveil_irq_msi_domain_free(struct irq_domain *domain,
>  {
>  	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
>  	struct mobiveil_pcie *pcie = irq_data_get_irq_chip_data(d);
> -	struct mobiveil_msi *msi = &pcie->msi;
> +	struct mobiveil_msi *msi = &pcie->rp.msi;
>  
>  	mutex_lock(&msi->lock);
>  
> @@ -808,9 +825,9 @@ static int mobiveil_allocate_msi_domains(struct mobiveil_pcie *pcie)
>  {
>  	struct device *dev = &pcie->pdev->dev;
>  	struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
> -	struct mobiveil_msi *msi = &pcie->msi;
> +	struct mobiveil_msi *msi = &pcie->rp.msi;
>  
> -	mutex_init(&pcie->msi.lock);
> +	mutex_init(&msi->lock);
>  	msi->dev_domain = irq_domain_add_linear(NULL, msi->num_of_vectors,
>  						&msi_domain_ops, pcie);
>  	if (!msi->dev_domain) {
> @@ -834,18 +851,19 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
>  {
>  	struct device *dev = &pcie->pdev->dev;
>  	struct device_node *node = dev->of_node;
> +	struct root_port *rp = &pcie->rp;
>  	int ret;
>  
>  	/* setup INTx */
> -	pcie->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> -						  &intx_domain_ops, pcie);
> +	rp->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> +						&intx_domain_ops, pcie);
>  
> -	if (!pcie->intx_domain) {
> +	if (!rp->intx_domain) {
>  		dev_err(dev, "Failed to get a INTx IRQ domain\n");
>  		return -ENOMEM;
>  	}
>  
> -	raw_spin_lock_init(&pcie->intx_mask_lock);
> +	raw_spin_lock_init(&rp->intx_mask_lock);
>  
>  	/* setup MSI */
>  	ret = mobiveil_allocate_msi_domains(pcie);
> @@ -862,6 +880,7 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
>  	struct pci_bus *child;
>  	struct pci_host_bridge *bridge;
>  	struct device *dev = &pdev->dev;
> +	struct root_port *rp;
>  	int ret;
>  
>  	/* allocate the PCIe port */
> @@ -870,6 +889,8 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
>  		return -ENOMEM;
>  
>  	pcie = pci_host_bridge_priv(bridge);
> +	rp = &pcie->rp;
> +	rp->bridge = bridge;
>  
>  	pcie->pdev = pdev;
>  
> @@ -904,12 +925,12 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
>  		return ret;
>  	}
>  
> -	irq_set_chained_handler_and_data(pcie->irq, mobiveil_pcie_isr, pcie);
> +	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
>  
>  	/* Initialize bridge */
>  	bridge->dev.parent = dev;
>  	bridge->sysdata = pcie;
> -	bridge->busnr = pcie->root_bus_nr;
> +	bridge->busnr = rp->root_bus_nr;
>  	bridge->ops = &mobiveil_pcie_ops;
>  	bridge->map_irq = of_irq_parse_and_map_pci;
>  	bridge->swizzle_irq = pci_common_swizzle;
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine
  2019-11-20  3:45 ` [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine Z.q. Hou
@ 2020-01-13 10:19   ` Andrew Murray
  2020-02-06 11:14     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 10:19 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:30AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Move the host initialization related operations into a new
> routine to make it can be reused by other incoming platform's

s/to make/such that/

'function' is probably a better word than 'routine'.


> PCIe host driver, in which the Mobiveil GPEX is integrated.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
>  drivers/pci/controller/pcie-mobiveil.c | 38 +++++++++++++++-----------
>  1 file changed, 22 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
> index 5fd26e376af2..97f682ca7c7a 100644
> --- a/drivers/pci/controller/pcie-mobiveil.c
> +++ b/drivers/pci/controller/pcie-mobiveil.c
> @@ -873,27 +873,15 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
>  	return 0;
>  }
>  
> -static int mobiveil_pcie_probe(struct platform_device *pdev)
> +int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)

This is no longer static - but do you need to add a header file somewhere?


>  {
> -	struct mobiveil_pcie *pcie;
> +	struct root_port *rp = &pcie->rp;
> +	struct pci_host_bridge *bridge = rp->bridge;
> +	struct device *dev = &pcie->pdev->dev;
>  	struct pci_bus *bus;
>  	struct pci_bus *child;
> -	struct pci_host_bridge *bridge;
> -	struct device *dev = &pdev->dev;
> -	struct root_port *rp;
>  	int ret;
>  
> -	/* allocate the PCIe port */
> -	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> -	if (!bridge)
> -		return -ENOMEM;
> -
> -	pcie = pci_host_bridge_priv(bridge);
> -	rp = &pcie->rp;
> -	rp->bridge = bridge;
> -
> -	pcie->pdev = pdev;
> -
>  	ret = mobiveil_pcie_parse_dt(pcie);
>  	if (ret) {
>  		dev_err(dev, "Parsing DT failed, ret: %x\n", ret);
> @@ -956,6 +944,24 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
>  	return 0;
>  }
>  
> +static int mobiveil_pcie_probe(struct platform_device *pdev)
> +{
> +	struct mobiveil_pcie *pcie;
> +	struct pci_host_bridge *bridge;
> +	struct device *dev = &pdev->dev;
> +
> +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));

You've lost the comment that was above this.

Thanks,

Andrew Murray

> +	if (!bridge)
> +		return -ENOMEM;
> +
> +	pcie = pci_host_bridge_priv(bridge);
> +	pcie->rp.bridge = bridge;
> +
> +	pcie->pdev = pdev;
> +
> +	return mobiveil_pcie_host_probe(pcie);
> +}
> +
>  static const struct of_device_id mobiveil_pcie_of_match[] = {
>  	{.compatible = "mbvl,gpex40-pcie",},
>  	{},
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations into a routine
  2019-11-20  3:45 ` [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations " Z.q. Hou
@ 2020-01-13 10:34   ` Andrew Murray
  2020-02-06 11:30     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 10:34 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:37AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Collect the interrupt initialization related operations into
> a new routine to make it more readable.

I prefer the word 'function' instead of routine. Also indicate why, not only
is it nicer but it is in preparation for EP support.

> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
>  drivers/pci/controller/pcie-mobiveil.c | 65 +++++++++++++++++---------
>  1 file changed, 42 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/pcie-mobiveil.c
> index 97f682ca7c7a..512b27a0536e 100644
> --- a/drivers/pci/controller/pcie-mobiveil.c
> +++ b/drivers/pci/controller/pcie-mobiveil.c
> @@ -454,12 +454,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
>  		return PTR_ERR(pcie->csr_axi_slave_base);
>  	pcie->pcie_reg_base = res->start;
>  
> -	/* map MSI config resource */
> -	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
> -	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
> -	if (IS_ERR(pcie->apb_csr_base))
> -		return PTR_ERR(pcie->apb_csr_base);
> -
>  	/* read the number of windows requested */
>  	if (of_property_read_u32(node, "apio-wins", &pcie->apio_wins))
>  		pcie->apio_wins = MAX_PIO_WINDOWS;
> @@ -467,12 +461,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
>  	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
>  		pcie->ppio_wins = MAX_PIO_WINDOWS;
>  
> -	rp->irq = platform_get_irq(pdev, 0);
> -	if (rp->irq <= 0) {
> -		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> -		return -ENODEV;
> -	}
> -
>  	return 0;
>  }
>  
> @@ -618,9 +606,6 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
>  	pab_ctrl |= (1 << AMBA_PIO_ENABLE_SHIFT) | (1 << PEX_PIO_ENABLE_SHIFT);
>  	mobiveil_csr_writel(pcie, pab_ctrl, PAB_CTRL);
>  
> -	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK | PAB_INTP_MSI_MASK),
> -			    PAB_INTP_AMBA_MISC_ENB);
> -
>  	/*
>  	 * program PIO Enable Bit to 1 and Config Window Enable Bit to 1 in
>  	 * PAB_AXI_PIO_CTRL Register
> @@ -670,9 +655,6 @@ static int mobiveil_host_init(struct mobiveil_pcie *pcie)
>  	value |= (PCI_CLASS_BRIDGE_PCI << 16);
>  	mobiveil_csr_writel(pcie, value, PAB_INTP_AXI_PIO_CLASS);
>  
> -	/* setup MSI hardware registers */
> -	mobiveil_pcie_enable_msi(pcie);
> -
>  	return 0;
>  }
>  
> @@ -873,6 +855,46 @@ static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
>  	return 0;
>  }
>  
> +static int mobiveil_pcie_interrupt_init(struct mobiveil_pcie *pcie)
> +{
> +	struct platform_device *pdev = pcie->pdev;
> +	struct device *dev = &pdev->dev;
> +	struct root_port *rp = &pcie->rp;
> +	struct resource *res;
> +	int ret;
> +
> +	/* map MSI config resource */
> +	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
> +	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
> +	if (IS_ERR(pcie->apb_csr_base))
> +		return PTR_ERR(pcie->apb_csr_base);
> +
> +	/* setup MSI hardware registers */
> +	mobiveil_pcie_enable_msi(pcie);

Does this need to come after mobiveil_pcie_init_irq_domain - given that
this function sets up the irq domain for MSI?

Thanks,

Andrew Murray

> +
> +	rp->irq = platform_get_irq(pdev, 0);
> +	if (rp->irq <= 0) {
> +		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> +		return -ENODEV;
> +	}
> +
> +	/* initialize the IRQ domains */
> +	ret = mobiveil_pcie_init_irq_domain(pcie);
> +	if (ret) {
> +		dev_err(dev, "Failed creating IRQ Domain\n");
> +		return ret;
> +	}
> +
> +	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> +
> +	/* Enable interrupts */
> +	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK | PAB_INTP_MSI_MASK),
> +			    PAB_INTP_AMBA_MISC_ENB);
> +
> +
> +	return 0;
> +}
> +
>  int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
>  {
>  	struct root_port *rp = &pcie->rp;
> @@ -906,15 +928,12 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
>  		return ret;
>  	}
>  
> -	/* initialize the IRQ domains */
> -	ret = mobiveil_pcie_init_irq_domain(pcie);
> +	ret = mobiveil_pcie_interrupt_init(pcie);
>  	if (ret) {
> -		dev_err(dev, "Failed creating IRQ Domain\n");
> +		dev_err(dev, "Interrupt init failed\n");
>  		return ret;
>  	}
>  
> -	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> -
>  	/* Initialize bridge */
>  	bridge->dev.parent = dev;
>  	bridge->sysdata = pcie;
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver
  2019-11-20  3:45 ` [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver Z.q. Hou
@ 2020-01-13 11:05   ` Andrew Murray
  2020-02-06 12:25     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 11:05 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:43AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Modularize the Mobiveil PCIe host driver according to the
> abstraction of Root Complex and Endpoint and move it into
> a new directory.
> 
> Split the RC related routines into pcie-mobiveil-host.c,
> and common routines into pcie-mobiveil.c, move the macro
> definitions and function declarations into pcie-mobiveil.h,
> and the Mobiveil platform reference code into
> pcie-mobiveil-plat.c. So that it is easy to reuse the
> extracted routines to add a new host driver, which
> integrated Mobiveil PCIe GPEX IP.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Reviewed-by: Andrew Murray <andrew.murray@arm.com>

> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
>  MAINTAINERS                                   |   2 +-
>  drivers/pci/controller/Kconfig                |  11 +-
>  drivers/pci/controller/Makefile               |   2 +-
>  drivers/pci/controller/mobiveil/Kconfig       |  24 ++
>  drivers/pci/controller/mobiveil/Makefile      |   4 +
>  .../pcie-mobiveil-host.c}                     | 398 +-----------------
>  .../controller/mobiveil/pcie-mobiveil-plat.c  |  60 +++
>  .../pci/controller/mobiveil/pcie-mobiveil.c   | 227 ++++++++++
>  .../pci/controller/mobiveil/pcie-mobiveil.h   | 178 ++++++++
>  9 files changed, 497 insertions(+), 409 deletions(-)
>  create mode 100644 drivers/pci/controller/mobiveil/Kconfig
>  create mode 100644 drivers/pci/controller/mobiveil/Makefile
>  rename drivers/pci/controller/{pcie-mobiveil.c => mobiveil/pcie-mobiveil-host.c} (61%)
>  create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
>  create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.c
>  create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 3f7f8cdbc471..a4ad99619e53 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12612,7 +12612,7 @@ M:	Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
>  L:	linux-pci@vger.kernel.org
>  S:	Supported
>  F:	Documentation/devicetree/bindings/pci/mobiveil-pcie.txt
> -F:	drivers/pci/controller/pcie-mobiveil.c
> +F:	drivers/pci/controller/mobiveil/pcie-mobiveil*
>  
>  PCI DRIVER FOR MVEBU (Marvell Armada 370 and Armada XP SOC support)
>  M:	Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig
> index f5de9119e8d3..74fd332755ae 100644
> --- a/drivers/pci/controller/Kconfig
> +++ b/drivers/pci/controller/Kconfig
> @@ -241,16 +241,6 @@ config PCIE_MEDIATEK
>  	  Say Y here if you want to enable PCIe controller support on
>  	  MediaTek SoCs.
>  
> -config PCIE_MOBIVEIL
> -	bool "Mobiveil AXI PCIe controller"
> -	depends on ARCH_ZYNQMP || COMPILE_TEST
> -	depends on OF
> -	depends on PCI_MSI_IRQ_DOMAIN
> -	help
> -	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
> -	  Soft IP. It has up to 8 outbound and inbound windows
> -	  for address translation and it is a PCIe Gen4 IP.
> -
>  config PCIE_TANGO_SMP8759
>  	bool "Tango SMP8759 PCIe controller (DANGEROUS)"
>  	depends on ARCH_TANGO && PCI_MSI && OF
> @@ -289,4 +279,5 @@ config PCI_HYPERV_INTERFACE
>  	  have a common interface with the Hyper-V PCI frontend driver.
>  
>  source "drivers/pci/controller/dwc/Kconfig"
> +source "drivers/pci/controller/mobiveil/Kconfig"
>  endmenu
> diff --git a/drivers/pci/controller/Makefile b/drivers/pci/controller/Makefile
> index a2a22c9d91af..44414cfd45ea 100644
> --- a/drivers/pci/controller/Makefile
> +++ b/drivers/pci/controller/Makefile
> @@ -27,11 +27,11 @@ obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
>  obj-$(CONFIG_PCIE_ROCKCHIP_EP) += pcie-rockchip-ep.o
>  obj-$(CONFIG_PCIE_ROCKCHIP_HOST) += pcie-rockchip-host.o
>  obj-$(CONFIG_PCIE_MEDIATEK) += pcie-mediatek.o
> -obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
>  obj-$(CONFIG_PCIE_TANGO_SMP8759) += pcie-tango.o
>  obj-$(CONFIG_VMD) += vmd.o
>  # pcie-hisi.o quirks are needed even without CONFIG_PCIE_DW
>  obj-y				+= dwc/
> +obj-y				+= mobiveil/
>  
>  
>  # The following drivers are for devices that use the generic ACPI
> diff --git a/drivers/pci/controller/mobiveil/Kconfig b/drivers/pci/controller/mobiveil/Kconfig
> new file mode 100644
> index 000000000000..64343c07bfed
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/Kconfig
> @@ -0,0 +1,24 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +menu "Mobiveil PCIe Core Support"
> +	depends on PCI
> +
> +config PCIE_MOBIVEIL
> +	bool
> +
> +config PCIE_MOBIVEIL_HOST
> +	bool
> +	depends on PCI_MSI_IRQ_DOMAIN
> +	select PCIE_MOBIVEIL
> +
> +config PCIE_MOBIVEIL_PLAT
> +	bool "Mobiveil AXI PCIe controller"
> +	depends on ARCH_ZYNQMP || COMPILE_TEST
> +	depends on OF
> +	select PCIE_MOBIVEIL_HOST
> +	help
> +	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
> +	  Soft IP. It has up to 8 outbound and inbound windows
> +	  for address translation and it is a PCIe Gen4 IP.
> +
> +endmenu
> diff --git a/drivers/pci/controller/mobiveil/Makefile b/drivers/pci/controller/mobiveil/Makefile
> new file mode 100644
> index 000000000000..9fb6d1c6504d
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
> +obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
> +obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
> diff --git a/drivers/pci/controller/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> similarity index 61%
> rename from drivers/pci/controller/pcie-mobiveil.c
> rename to drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> index 512b27a0536e..2cc424e78d33 100644
> --- a/drivers/pci/controller/pcie-mobiveil.c
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> @@ -9,7 +9,6 @@
>   * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
>   */
>  
> -#include <linux/delay.h>
>  #include <linux/init.h>
>  #include <linux/interrupt.h>
>  #include <linux/irq.h>
> @@ -26,265 +25,7 @@
>  #include <linux/platform_device.h>
>  #include <linux/slab.h>
>  
> -#include "../pci.h"
> -
> -/* register offsets and bit positions */
> -
> -/*
> - * translation tables are grouped into windows, each window registers are
> - * grouped into blocks of 4 or 16 registers each
> - */
> -#define PAB_REG_BLOCK_SIZE		16
> -#define PAB_EXT_REG_BLOCK_SIZE		4
> -
> -#define PAB_REG_ADDR(offset, win)	\
> -	(offset + (win * PAB_REG_BLOCK_SIZE))
> -#define PAB_EXT_REG_ADDR(offset, win)	\
> -	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
> -
> -#define LTSSM_STATUS			0x0404
> -#define  LTSSM_STATUS_L0_MASK		0x3f
> -#define  LTSSM_STATUS_L0		0x2d
> -
> -#define PAB_CTRL			0x0808
> -#define  AMBA_PIO_ENABLE_SHIFT		0
> -#define  PEX_PIO_ENABLE_SHIFT		1
> -#define  PAGE_SEL_SHIFT			13
> -#define  PAGE_SEL_MASK			0x3f
> -#define  PAGE_LO_MASK			0x3ff
> -#define  PAGE_SEL_OFFSET_SHIFT		10
> -
> -#define PAB_AXI_PIO_CTRL		0x0840
> -#define  APIO_EN_MASK			0xf
> -
> -#define PAB_PEX_PIO_CTRL		0x08c0
> -#define  PIO_ENABLE_SHIFT		0
> -
> -#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> -#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> -#define  PAB_INTP_INTX_MASK		0x01e0
> -#define  PAB_INTP_MSI_MASK		0x8
> -
> -#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
> -#define  WIN_ENABLE_SHIFT		0
> -#define  WIN_TYPE_SHIFT			1
> -#define  WIN_TYPE_MASK			0x3
> -#define  WIN_SIZE_MASK			0xfffffc00
> -
> -#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0, win)
> -
> -#define PAB_EXT_AXI_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0x80a0, win)
> -#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
> -#define  AXI_WINDOW_ALIGN_MASK		3
> -
> -#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8, win)
> -#define  PAB_BUS_SHIFT			24
> -#define  PAB_DEVICE_SHIFT		19
> -#define  PAB_FUNCTION_SHIFT		16
> -
> -#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac, win)
> -#define PAB_INTP_AXI_PIO_CLASS		0x474
> -
> -#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0, win)
> -#define  AMAP_CTRL_EN_SHIFT		0
> -#define  AMAP_CTRL_TYPE_SHIFT		1
> -#define  AMAP_CTRL_TYPE_MASK		3
> -
> -#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0, win)
> -#define PAB_EXT_PEX_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0xb4a0, win)
> -#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
> -#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8, win)
> -#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac, win)
> -
> -/* starting offset of INTX bits in status register */
> -#define PAB_INTX_START			5
> -
> -/* supported number of MSI interrupts */
> -#define PCI_NUM_MSI			16
> -
> -/* MSI registers */
> -#define MSI_BASE_LO_OFFSET		0x04
> -#define MSI_BASE_HI_OFFSET		0x08
> -#define MSI_SIZE_OFFSET			0x0c
> -#define MSI_ENABLE_OFFSET		0x14
> -#define MSI_STATUS_OFFSET		0x18
> -#define MSI_DATA_OFFSET			0x20
> -#define MSI_ADDR_L_OFFSET		0x24
> -#define MSI_ADDR_H_OFFSET		0x28
> -
> -/* outbound and inbound window definitions */
> -#define WIN_NUM_0			0
> -#define WIN_NUM_1			1
> -#define CFG_WINDOW_TYPE			0
> -#define IO_WINDOW_TYPE			1
> -#define MEM_WINDOW_TYPE			2
> -#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
> -#define MAX_PIO_WINDOWS			8
> -
> -/* Parameters for the waiting for link up routine */
> -#define LINK_WAIT_MAX_RETRIES		10
> -#define LINK_WAIT_MIN			90000
> -#define LINK_WAIT_MAX			100000
> -
> -#define PAGED_ADDR_BNDRY		0xc00
> -#define OFFSET_TO_PAGE_ADDR(off)	\
> -	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
> -#define OFFSET_TO_PAGE_IDX(off)		\
> -	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
> -
> -struct mobiveil_msi {			/* MSI information */
> -	struct mutex lock;		/* protect bitmap variable */
> -	struct irq_domain *msi_domain;
> -	struct irq_domain *dev_domain;
> -	phys_addr_t msi_pages_phys;
> -	int num_of_vectors;
> -	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
> -};
> -
> -struct root_port {
> -	char root_bus_nr;
> -	void __iomem *config_axi_slave_base;	/* endpoint config base */
> -	struct resource *ob_io_res;
> -	int irq;
> -	raw_spinlock_t intx_mask_lock;
> -	struct irq_domain *intx_domain;
> -	struct mobiveil_msi msi;
> -	struct pci_host_bridge *bridge;
> -};
> -
> -struct mobiveil_pcie {
> -	struct platform_device *pdev;
> -	void __iomem *csr_axi_slave_base;	/* root port config base */
> -	void __iomem *apb_csr_base;	/* MSI register base */
> -	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> -	int apio_wins;
> -	int ppio_wins;
> -	int ob_wins_configured;		/* configured outbound windows */
> -	int ib_wins_configured;		/* configured inbound windows */
> -	struct root_port rp;
> -};
> -
> -/*
> - * mobiveil_pcie_sel_page - routine to access paged register
> - *
> - * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are paged,
> - * for this scheme to work extracted higher 6 bits of the offset will be
> - * written to pg_sel field of PAB_CTRL register and rest of the lower 10
> - * bits enabled with PAGED_ADDR_BNDRY are used as offset of the register.
> - */
> -static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8 pg_idx)
> -{
> -	u32 val;
> -
> -	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
> -	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
> -	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
> -
> -	writel(val, pcie->csr_axi_slave_base + PAB_CTRL);
> -}
> -
> -static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32 off)
> -{
> -	if (off < PAGED_ADDR_BNDRY) {
> -		/* For directly accessed registers, clear the pg_sel field */
> -		mobiveil_pcie_sel_page(pcie, 0);
> -		return pcie->csr_axi_slave_base + off;
> -	}
> -
> -	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
> -	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off);
> -}
> -
> -static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
> -{
> -	if ((uintptr_t)addr & (size - 1)) {
> -		*val = 0;
> -		return PCIBIOS_BAD_REGISTER_NUMBER;
> -	}
> -
> -	switch (size) {
> -	case 4:
> -		*val = readl(addr);
> -		break;
> -	case 2:
> -		*val = readw(addr);
> -		break;
> -	case 1:
> -		*val = readb(addr);
> -		break;
> -	default:
> -		*val = 0;
> -		return PCIBIOS_BAD_REGISTER_NUMBER;
> -	}
> -
> -	return PCIBIOS_SUCCESSFUL;
> -}
> -
> -static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
> -{
> -	if ((uintptr_t)addr & (size - 1))
> -		return PCIBIOS_BAD_REGISTER_NUMBER;
> -
> -	switch (size) {
> -	case 4:
> -		writel(val, addr);
> -		break;
> -	case 2:
> -		writew(val, addr);
> -		break;
> -	case 1:
> -		writeb(val, addr);
> -		break;
> -	default:
> -		return PCIBIOS_BAD_REGISTER_NUMBER;
> -	}
> -
> -	return PCIBIOS_SUCCESSFUL;
> -}
> -
> -static u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size)
> -{
> -	void *addr;
> -	u32 val;
> -	int ret;
> -
> -	addr = mobiveil_pcie_comp_addr(pcie, off);
> -
> -	ret = mobiveil_pcie_read(addr, size, &val);
> -	if (ret)
> -		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
> -
> -	return val;
> -}
> -
> -static void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> -			       size_t size)
> -{
> -	void *addr;
> -	int ret;
> -
> -	addr = mobiveil_pcie_comp_addr(pcie, off);
> -
> -	ret = mobiveil_pcie_write(addr, size, val);
> -	if (ret)
> -		dev_err(&pcie->pdev->dev, "write CSR address failed\n");
> -}
> -
> -static u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
> -{
> -	return mobiveil_csr_read(pcie, off, 0x4);
> -}
> -
> -static void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val, u32 off)
> -{
> -	mobiveil_csr_write(pcie, val, off, 0x4);
> -}
> -
> -static bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
> -{
> -	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
> -		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
> -}
> +#include "pcie-mobiveil.h"
>  
>  static bool mobiveil_pcie_valid_device(struct pci_bus *bus, unsigned int devfn)
>  {
> @@ -464,103 +205,6 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
>  	return 0;
>  }
>  
> -static void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
> -			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> -{
> -	u32 value;
> -	u64 size64 = ~(size - 1);
> -
> -	if (win_num >= pcie->ppio_wins) {
> -		dev_err(&pcie->pdev->dev,
> -			"ERROR: max inbound windows reached !\n");
> -		return;
> -	}
> -
> -	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
> -	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT | WIN_SIZE_MASK);
> -	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 << AMAP_CTRL_EN_SHIFT |
> -		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> -	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
> -
> -	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> -			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
> -
> -	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
> -			    PAB_PEX_AMAP_AXI_WIN(win_num));
> -	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> -			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
> -
> -	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> -			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
> -	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> -			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
> -
> -	pcie->ib_wins_configured++;
> -}
> -
> -/*
> - * routine to program the outbound windows
> - */
> -static void program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
> -			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> -{
> -	u32 value;
> -	u64 size64 = ~(size - 1);
> -
> -	if (win_num >= pcie->apio_wins) {
> -		dev_err(&pcie->pdev->dev,
> -			"ERROR: max outbound windows reached !\n");
> -		return;
> -	}
> -
> -	/*
> -	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
> -	 * to 4 KB in PAB_AXI_AMAP_CTRL register
> -	 */
> -	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
> -	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
> -	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
> -		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> -	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
> -
> -	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> -			    PAB_EXT_AXI_AMAP_SIZE(win_num));
> -
> -	/*
> -	 * program AXI window base with appropriate value in
> -	 * PAB_AXI_AMAP_AXI_WIN0 register
> -	 */
> -	mobiveil_csr_writel(pcie,
> -			    lower_32_bits(cpu_addr) & (~AXI_WINDOW_ALIGN_MASK),
> -			    PAB_AXI_AMAP_AXI_WIN(win_num));
> -	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> -			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
> -
> -	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> -			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
> -	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> -			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
> -
> -	pcie->ob_wins_configured++;
> -}
> -
> -static int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
> -{
> -	int retries;
> -
> -	/* check if the link is up or not */
> -	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
> -		if (mobiveil_pcie_link_up(pcie))
> -			return 0;
> -
> -		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
> -	}
> -
> -	dev_err(&pcie->pdev->dev, "link never came up\n");
> -
> -	return -ETIMEDOUT;
> -}
> -
>  static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
>  {
>  	phys_addr_t msg_addr = pcie->pcie_reg_base;
> @@ -962,43 +606,3 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
>  
>  	return 0;
>  }
> -
> -static int mobiveil_pcie_probe(struct platform_device *pdev)
> -{
> -	struct mobiveil_pcie *pcie;
> -	struct pci_host_bridge *bridge;
> -	struct device *dev = &pdev->dev;
> -
> -	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> -	if (!bridge)
> -		return -ENOMEM;
> -
> -	pcie = pci_host_bridge_priv(bridge);
> -	pcie->rp.bridge = bridge;
> -
> -	pcie->pdev = pdev;
> -
> -	return mobiveil_pcie_host_probe(pcie);
> -}
> -
> -static const struct of_device_id mobiveil_pcie_of_match[] = {
> -	{.compatible = "mbvl,gpex40-pcie",},
> -	{},
> -};
> -
> -MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
> -
> -static struct platform_driver mobiveil_pcie_driver = {
> -	.probe = mobiveil_pcie_probe,
> -	.driver = {
> -		.name = "mobiveil-pcie",
> -		.of_match_table = mobiveil_pcie_of_match,
> -		.suppress_bind_attrs = true,
> -	},
> -};
> -
> -builtin_platform_driver(mobiveil_pcie_driver);
> -
> -MODULE_LICENSE("GPL v2");
> -MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
> -MODULE_AUTHOR("Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>");
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> new file mode 100644
> index 000000000000..64c85f852869
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> @@ -0,0 +1,60 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * PCIe host controller driver for Mobiveil PCIe Host controller
> + *
> + * Copyright (c) 2018 Mobiveil Inc.
> + * Copyright 2019 NXP
> + *
> + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>

Same feedback as before with the recode tag.

> + */
> +
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +
> +#include "pcie-mobiveil.h"
> +
> +static int mobiveil_pcie_probe(struct platform_device *pdev)
> +{
> +	struct mobiveil_pcie *pcie;
> +	struct pci_host_bridge *bridge;
> +	struct device *dev = &pdev->dev;
> +
> +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> +	if (!bridge)
> +		return -ENOMEM;
> +
> +	pcie = pci_host_bridge_priv(bridge);
> +	pcie->rp.bridge = bridge;
> +
> +	pcie->pdev = pdev;
> +
> +	return mobiveil_pcie_host_probe(pcie);
> +}
> +
> +static const struct of_device_id mobiveil_pcie_of_match[] = {
> +	{.compatible = "mbvl,gpex40-pcie",},
> +	{},
> +};
> +
> +MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
> +
> +static struct platform_driver mobiveil_pcie_driver = {
> +	.probe = mobiveil_pcie_probe,
> +	.driver = {
> +		.name = "mobiveil-pcie",
> +		.of_match_table = mobiveil_pcie_of_match,
> +		.suppress_bind_attrs = true,
> +	},
> +};
> +
> +builtin_platform_driver(mobiveil_pcie_driver);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
> +MODULE_AUTHOR("Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>");
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> new file mode 100644
> index 000000000000..2773f823c9ea
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> @@ -0,0 +1,227 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * PCIe host controller driver for Mobiveil PCIe Host controller
> + *
> + * Copyright (c) 2018 Mobiveil Inc.
> + * Copyright 2019 NXP
> + *
> + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +
> +#include "pcie-mobiveil.h"
> +
> +/*
> + * mobiveil_pcie_sel_page - routine to access paged register
> + *
> + * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are paged,
> + * for this scheme to work extracted higher 6 bits of the offset will be
> + * written to pg_sel field of PAB_CTRL register and rest of the lower 10
> + * bits enabled with PAGED_ADDR_BNDRY are used as offset of the register.
> + */
> +static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8 pg_idx)
> +{
> +	u32 val;
> +
> +	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
> +	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
> +	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
> +
> +	writel(val, pcie->csr_axi_slave_base + PAB_CTRL);
> +}
> +
> +static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32 off)
> +{
> +	if (off < PAGED_ADDR_BNDRY) {
> +		/* For directly accessed registers, clear the pg_sel field */
> +		mobiveil_pcie_sel_page(pcie, 0);
> +		return pcie->csr_axi_slave_base + off;
> +	}
> +
> +	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
> +	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off);
> +}
> +
> +static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
> +{
> +	if ((uintptr_t)addr & (size - 1)) {
> +		*val = 0;
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +	}
> +
> +	switch (size) {
> +	case 4:
> +		*val = readl(addr);
> +		break;
> +	case 2:
> +		*val = readw(addr);
> +		break;
> +	case 1:
> +		*val = readb(addr);
> +		break;
> +	default:
> +		*val = 0;
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +	}
> +
> +	return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
> +{
> +	if ((uintptr_t)addr & (size - 1))
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> +	switch (size) {
> +	case 4:
> +		writel(val, addr);
> +		break;
> +	case 2:
> +		writew(val, addr);
> +		break;
> +	case 1:
> +		writeb(val, addr);
> +		break;
> +	default:
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +	}
> +
> +	return PCIBIOS_SUCCESSFUL;
> +}
> +
> +u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size)
> +{
> +	void *addr;
> +	u32 val;
> +	int ret;
> +
> +	addr = mobiveil_pcie_comp_addr(pcie, off);
> +
> +	ret = mobiveil_pcie_read(addr, size, &val);
> +	if (ret)
> +		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
> +
> +	return val;
> +}
> +
> +void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> +			       size_t size)
> +{
> +	void *addr;
> +	int ret;
> +
> +	addr = mobiveil_pcie_comp_addr(pcie, off);
> +
> +	ret = mobiveil_pcie_write(addr, size, val);
> +	if (ret)
> +		dev_err(&pcie->pdev->dev, "write CSR address failed\n");
> +}
> +
> +bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
> +{
> +	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
> +		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
> +}
> +
> +void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
> +			u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> +{
> +	u32 value;
> +	u64 size64 = ~(size - 1);
> +
> +	if (win_num >= pcie->ppio_wins) {
> +		dev_err(&pcie->pdev->dev,
> +			"ERROR: max inbound windows reached !\n");
> +		return;
> +	}
> +
> +	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
> +	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT | WIN_SIZE_MASK);
> +	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 << AMAP_CTRL_EN_SHIFT |
> +		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> +	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
> +
> +	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> +			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
> +
> +	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
> +			    PAB_PEX_AMAP_AXI_WIN(win_num));
> +	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> +			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
> +
> +	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> +			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
> +	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> +			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
> +
> +	pcie->ib_wins_configured++;
> +}
> +
> +/*
> + * routine to program the outbound windows
> + */
> +void program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
> +			u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> +{
> +	u32 value;
> +	u64 size64 = ~(size - 1);
> +
> +	if (win_num >= pcie->apio_wins) {
> +		dev_err(&pcie->pdev->dev,
> +			"ERROR: max outbound windows reached !\n");
> +		return;
> +	}
> +
> +	/*
> +	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
> +	 * to 4 KB in PAB_AXI_AMAP_CTRL register
> +	 */
> +	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
> +	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
> +	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
> +		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> +	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
> +
> +	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> +			    PAB_EXT_AXI_AMAP_SIZE(win_num));
> +
> +	/*
> +	 * program AXI window base with appropriate value in
> +	 * PAB_AXI_AMAP_AXI_WIN0 register
> +	 */
> +	mobiveil_csr_writel(pcie,
> +			    lower_32_bits(cpu_addr) & (~AXI_WINDOW_ALIGN_MASK),
> +			    PAB_AXI_AMAP_AXI_WIN(win_num));
> +	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> +			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
> +
> +	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> +			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
> +	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> +			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
> +
> +	pcie->ob_wins_configured++;
> +}
> +
> +int mobiveil_bringup_link(struct mobiveil_pcie *pcie)
> +{
> +	int retries;
> +
> +	/* check if the link is up or not */
> +	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
> +		if (mobiveil_pcie_link_up(pcie))
> +			return 0;
> +
> +		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
> +	}
> +
> +	dev_err(&pcie->pdev->dev, "link never came up\n");
> +
> +	return -ETIMEDOUT;
> +}
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> new file mode 100644
> index 000000000000..e3148078e9dd
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -0,0 +1,178 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * PCIe host controller driver for Mobiveil PCIe Host controller
> + *
> + * Copyright (c) 2018 Mobiveil Inc.
> + * Copyright 2019 NXP
> + *
> + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> + */
> +
> +#ifndef _PCIE_MOBIVEIL_H
> +#define _PCIE_MOBIVEIL_H
> +
> +#include <linux/pci.h>
> +#include <linux/irq.h>
> +#include <linux/msi.h>
> +#include "../../pci.h"
> +
> +/* register offsets and bit positions */
> +
> +/*
> + * translation tables are grouped into windows, each window registers are
> + * grouped into blocks of 4 or 16 registers each
> + */
> +#define PAB_REG_BLOCK_SIZE		16
> +#define PAB_EXT_REG_BLOCK_SIZE		4
> +
> +#define PAB_REG_ADDR(offset, win)	\
> +	(offset + (win * PAB_REG_BLOCK_SIZE))
> +#define PAB_EXT_REG_ADDR(offset, win)	\
> +	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
> +
> +#define LTSSM_STATUS			0x0404
> +#define  LTSSM_STATUS_L0_MASK		0x3f
> +#define  LTSSM_STATUS_L0		0x2d
> +
> +#define PAB_CTRL			0x0808
> +#define  AMBA_PIO_ENABLE_SHIFT		0
> +#define  PEX_PIO_ENABLE_SHIFT		1
> +#define  PAGE_SEL_SHIFT			13
> +#define  PAGE_SEL_MASK			0x3f
> +#define  PAGE_LO_MASK			0x3ff
> +#define  PAGE_SEL_OFFSET_SHIFT		10
> +
> +#define PAB_AXI_PIO_CTRL		0x0840
> +#define  APIO_EN_MASK			0xf
> +
> +#define PAB_PEX_PIO_CTRL		0x08c0
> +#define  PIO_ENABLE_SHIFT		0
> +
> +#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> +#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> +#define  PAB_INTP_INTX_MASK		0x01e0
> +#define  PAB_INTP_MSI_MASK		0x8
> +
> +#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
> +#define  WIN_ENABLE_SHIFT		0
> +#define  WIN_TYPE_SHIFT			1
> +#define  WIN_TYPE_MASK			0x3
> +#define  WIN_SIZE_MASK			0xfffffc00
> +
> +#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0, win)
> +
> +#define PAB_EXT_AXI_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0x80a0, win)
> +#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
> +#define  AXI_WINDOW_ALIGN_MASK		3
> +
> +#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8, win)
> +#define  PAB_BUS_SHIFT			24
> +#define  PAB_DEVICE_SHIFT		19
> +#define  PAB_FUNCTION_SHIFT		16
> +
> +#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac, win)
> +#define PAB_INTP_AXI_PIO_CLASS		0x474
> +
> +#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0, win)
> +#define  AMAP_CTRL_EN_SHIFT		0
> +#define  AMAP_CTRL_TYPE_SHIFT		1
> +#define  AMAP_CTRL_TYPE_MASK		3
> +
> +#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0, win)
> +#define PAB_EXT_PEX_AMAP_AXI_WIN(win)	PAB_EXT_REG_ADDR(0xb4a0, win)
> +#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
> +#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8, win)
> +#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac, win)
> +
> +/* starting offset of INTX bits in status register */
> +#define PAB_INTX_START			5
> +
> +/* supported number of MSI interrupts */
> +#define PCI_NUM_MSI			16
> +
> +/* MSI registers */
> +#define MSI_BASE_LO_OFFSET		0x04
> +#define MSI_BASE_HI_OFFSET		0x08
> +#define MSI_SIZE_OFFSET			0x0c
> +#define MSI_ENABLE_OFFSET		0x14
> +#define MSI_STATUS_OFFSET		0x18
> +#define MSI_DATA_OFFSET			0x20
> +#define MSI_ADDR_L_OFFSET		0x24
> +#define MSI_ADDR_H_OFFSET		0x28
> +
> +/* outbound and inbound window definitions */
> +#define WIN_NUM_0			0
> +#define WIN_NUM_1			1
> +#define CFG_WINDOW_TYPE			0
> +#define IO_WINDOW_TYPE			1
> +#define MEM_WINDOW_TYPE			2
> +#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
> +#define MAX_PIO_WINDOWS			8
> +
> +/* Parameters for the waiting for link up routine */
> +#define LINK_WAIT_MAX_RETRIES		10
> +#define LINK_WAIT_MIN			90000
> +#define LINK_WAIT_MAX			100000
> +
> +#define PAGED_ADDR_BNDRY		0xc00
> +#define OFFSET_TO_PAGE_ADDR(off)	\
> +	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
> +#define OFFSET_TO_PAGE_IDX(off)		\
> +	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
> +
> +struct mobiveil_msi {			/* MSI information */
> +	struct mutex lock;		/* protect bitmap variable */
> +	struct irq_domain *msi_domain;
> +	struct irq_domain *dev_domain;
> +	phys_addr_t msi_pages_phys;
> +	int num_of_vectors;
> +	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
> +};
> +
> +struct root_port {
> +	char root_bus_nr;
> +	void __iomem *config_axi_slave_base;	/* endpoint config base */
> +	struct resource *ob_io_res;
> +	int irq;
> +	raw_spinlock_t intx_mask_lock;
> +	struct irq_domain *intx_domain;
> +	struct mobiveil_msi msi;
> +	struct pci_host_bridge *bridge;
> +};
> +
> +struct mobiveil_pcie {
> +	struct platform_device *pdev;
> +	void __iomem *csr_axi_slave_base;	/* root port config base */
> +	void __iomem *apb_csr_base;	/* MSI register base */
> +	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> +	int apio_wins;
> +	int ppio_wins;
> +	int ob_wins_configured;		/* configured outbound windows */
> +	int ib_wins_configured;		/* configured inbound windows */
> +	struct root_port rp;
> +};
> +
> +int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie);
> +bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie);
> +int mobiveil_bringup_link(struct mobiveil_pcie *pcie);
> +void program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
> +			u64 pci_addr, u32 type, u64 size);
> +void program_ib_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
> +			u64 pci_addr, u32 type, u64 size);
> +u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t size);
> +void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> +			size_t size);
> +
> +static inline u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
> +{
> +	return mobiveil_csr_read(pcie, off, 0x4);
> +}
> +
> +static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
> +				       u32 off)
> +{
> +	mobiveil_csr_write(pcie, val, off, 0x4);
> +}
> +
> +#endif /* _PCIE_MOBIVEIL_H */
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization
  2019-11-20  3:45 ` [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization Z.q. Hou
@ 2020-01-13 11:19   ` Andrew Murray
  2020-02-06 13:25     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 11:19 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:50AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The Mobiveil GPEX internal MSI/INTx controller may not be used
> by other platforms in which the Mobiveil GPEX is integrated.
> This patch is to allow these platforms to implement their
> specific interrupt initialization.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
>  drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 3 +++
>  drivers/pci/controller/mobiveil/pcie-mobiveil.h      | 7 +++++++
>  2 files changed, 10 insertions(+)
> 
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> index 2cc424e78d33..3cd93df6fe6e 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> @@ -507,6 +507,9 @@ static int mobiveil_pcie_interrupt_init(struct mobiveil_pcie *pcie)
>  	struct resource *res;
>  	int ret;
>  
> +	if (rp->ops->interrupt_init)
> +		return rp->ops->interrupt_init(pcie);
> +

This may be cleaner if you have a helper function named
"mobiveil_pcie_interrupt_init" where it either calls interrupt_init if present
or calls this current function renamed to "mobiveil_pcie_integrated_interrupt_init"
or similar.

A bit like the DWC dw_pcie_rd_own_conf function.

Thanks,

Andrew Murray

>  	/* map MSI config resource */
>  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "apb_csr");
>  	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> index e3148078e9dd..18d85806a7fc 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -130,10 +130,17 @@ struct mobiveil_msi {			/* MSI information */
>  	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
>  };
>  
> +struct mobiveil_pcie;
> +
> +struct mobiveil_rp_ops {
> +	int (*interrupt_init)(struct mobiveil_pcie *pcie);
> +};
> +
>  struct root_port {
>  	char root_bus_nr;
>  	void __iomem *config_axi_slave_base;	/* endpoint config base */
>  	struct resource *ob_io_res;
> +	struct mobiveil_rp_ops *ops;
>  	int irq;
>  	raw_spinlock_t intx_mask_lock;
>  	struct irq_domain *intx_domain;
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check
  2019-11-20  3:45 ` [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check Z.q. Hou
@ 2020-01-13 11:22   ` Andrew Murray
  2020-02-06 13:25     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 11:22 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:45:57AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> The platforms, in which the Mobiveil GPEX is integrated,
> may have their specific mechanism to check link up status.
> This patch is to enable these platforms to implement theirs.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> ---
> V9:
>  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
>  drivers/pci/controller/mobiveil/pcie-mobiveil.c | 3 +++
>  drivers/pci/controller/mobiveil/pcie-mobiveil.h | 5 +++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> index 2773f823c9ea..b9ed2d95641c 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> @@ -125,6 +125,9 @@ void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
>  
>  bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)
>  {
> +	if (pcie->ops->link_up)
> +		return pcie->ops->link_up(pcie);
> +
>  	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
>  		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;

On the previous patch I suggested that we don't mix up the link_up logic
with the logic that decides which function to call. In this case the link_up
logic is trivial. So this is probably OK.

Reviewed-by: Andrew Murray <andrew.murray@arm.com>

>  }
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> index 18d85806a7fc..95d2e7c809b8 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -148,6 +148,10 @@ struct root_port {
>  	struct pci_host_bridge *bridge;
>  };
>  
> +struct mobiveil_pab_ops {
> +	int (*link_up)(struct mobiveil_pcie *pcie);
> +};
> +
>  struct mobiveil_pcie {
>  	struct platform_device *pdev;
>  	void __iomem *csr_axi_slave_base;	/* root port config base */
> @@ -157,6 +161,7 @@ struct mobiveil_pcie {
>  	int ppio_wins;
>  	int ob_wins_configured;		/* configured outbound windows */
>  	int ib_wins_configured;		/* configured inbound windows */
> +	const struct mobiveil_pab_ops *ops;
>  	struct root_port rp;
>  };
>  
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host
  2019-11-20  3:46 ` [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host Z.q. Hou
@ 2020-01-13 11:26   ` Andrew Murray
  2020-02-06 13:27     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 11:26 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:46:03AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Make the mobiveil_host_init() function can be used to re-init

Perhaps reword to "Allow the mobiveil_host_init() function to be
used to ...

> host controller's PAB and GPEX CSR register block, as NXP
> integrated Mobiveil IP has to reset and then re-init the PAB
> and GPEX CSR registers upon hot-reset.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> ---
> V9:
>  - No change
> 
>  .../controller/mobiveil/pcie-mobiveil-host.c  | 19 ++++++++++++-------
>  .../pci/controller/mobiveil/pcie-mobiveil.h   |  1 +
>  2 files changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> index 3cd93df6fe6e..9bc3da036720 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> @@ -221,18 +221,23 @@ static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)
>  	writel_relaxed(1, pcie->apb_csr_base + MSI_ENABLE_OFFSET);
>  }
>  
> -static int mobiveil_host_init(struct mobiveil_pcie *pcie)
> +int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit)
>  {
>  	struct root_port *rp = &pcie->rp;
>  	struct pci_host_bridge *bridge = rp->bridge;
>  	u32 value, pab_ctrl, type;
>  	struct resource_entry *win;
>  
> -	/* setup bus numbers */
> -	value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
> -	value &= 0xff000000;
> -	value |= 0x00ff0100;
> -	mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
> +	pcie->ib_wins_configured = 0;
> +	pcie->ob_wins_configured = 0;

This works so long as the number of bridge->windows never reduces. I
think this assumption holds true.

Thanks,

Andrew Murray

> +
> +	if (!reinit) {
> +		/* setup bus numbers */
> +		value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
> +		value &= 0xff000000;
> +		value |= 0x00ff0100;
> +		mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
> +	}
>  
>  	/*
>  	 * program Bus Master Enable Bit in Command Register in PAB Config
> @@ -569,7 +574,7 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
>  	 * configure all inbound and outbound windows and prepare the RC for
>  	 * config access
>  	 */
> -	ret = mobiveil_host_init(pcie);
> +	ret = mobiveil_host_init(pcie, false);
>  	if (ret) {
>  		dev_err(dev, "Failed to initialize host\n");
>  		return ret;
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> index 95d2e7c809b8..37116c2a19fe 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -166,6 +166,7 @@ struct mobiveil_pcie {
>  };
>  
>  int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie);
> +int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit);
>  bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie);
>  int mobiveil_bringup_link(struct mobiveil_pcie *pcie);
>  void program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors
  2019-11-20  3:46 ` [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors Z.q. Hou
@ 2020-01-13 11:31   ` Andrew Murray
  2020-02-06 13:45     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 11:31 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:46:10AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> There are some 8-bit and 16-bit registers in PCIe configuration
> space, so add these accessors accordingly.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> ---
> V9:
>  - No change
> 
>  .../pci/controller/mobiveil/pcie-mobiveil.h   | 23 +++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> index 37116c2a19fe..750a7fd95bc1 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -182,10 +182,33 @@ static inline u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off)
>  	return mobiveil_csr_read(pcie, off, 0x4);
>  }
>  
> +static inline u32 mobiveil_csr_readw(struct mobiveil_pcie *pcie, u32 off)
> +{
> +	return mobiveil_csr_read(pcie, off, 0x2);
> +}
> +
> +static inline u32 mobiveil_csr_readb(struct mobiveil_pcie *pcie, u32 off)
> +{
> +	return mobiveil_csr_read(pcie, off, 0x1);
> +}

Do you think the above two return types should reflect the size of the access?

Thanks,

Andrew Murray

> +
> +
>  static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
>  				       u32 off)
>  {
>  	mobiveil_csr_write(pcie, val, off, 0x4);
>  }
>  
> +static inline void mobiveil_csr_writew(struct mobiveil_pcie *pcie, u32 val,
> +				       u32 off)
> +{
> +	mobiveil_csr_write(pcie, val, off, 0x2);
> +}
> +
> +static inline void mobiveil_csr_writeb(struct mobiveil_pcie *pcie, u32 val,
> +				       u32 off)
> +{
> +	mobiveil_csr_write(pcie, val, off, 0x1);
> +}
> +
>  #endif /* _PCIE_MOBIVEIL_H */
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs
  2019-11-20  3:46 ` [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs Z.q. Hou
@ 2020-01-13 12:02   ` Andrew Murray
  2020-02-06 13:45     ` Z.q. Hou
  0 siblings, 1 reply; 55+ messages in thread
From: Andrew Murray @ 2020-01-13 12:02 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Wed, Nov 20, 2019 at 03:46:23AM +0000, Z.q. Hou wrote:
> From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> This PCIe controller is based on the Mobiveil GPEX IP, which is
> compatible with the PCI Express™ Base Specification, Revision 4.0.
> 
> Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> ---
>  drivers/pci/controller/mobiveil/Kconfig       |  10 +
>  drivers/pci/controller/mobiveil/Makefile      |   1 +
>  .../mobiveil/pcie-layerscape-gen4.c           | 274 ++++++++++++++++++
>  .../pci/controller/mobiveil/pcie-mobiveil.h   |  16 +-
>  4 files changed, 299 insertions(+), 2 deletions(-)
>  create mode 100644 drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> 
> diff --git a/drivers/pci/controller/mobiveil/Kconfig b/drivers/pci/controller/mobiveil/Kconfig
> index 64343c07bfed..c823be8dab1c 100644
> --- a/drivers/pci/controller/mobiveil/Kconfig
> +++ b/drivers/pci/controller/mobiveil/Kconfig
> @@ -21,4 +21,14 @@ config PCIE_MOBIVEIL_PLAT
>  	  Soft IP. It has up to 8 outbound and inbound windows
>  	  for address translation and it is a PCIe Gen4 IP.
>  
> +config PCIE_LAYERSCAPE_GEN4
> +	bool "Freescale Layerscape PCIe Gen4 controller"
> +	depends on PCI
> +	depends on OF && (ARM64 || ARCH_LAYERSCAPE)
> +	depends on PCI_MSI_IRQ_DOMAIN
> +	select PCIE_MOBIVEIL_HOST
> +	help
> +	  Say Y here if you want PCIe Gen4 controller support on
> +	  Layerscape SoCs. The PCIe controller can work in RC or
> +	  EP mode according to RCW[HOST_AGT_PEX] setting.

I think you can remove the last sentence - it doesn't give any value
to users of KConfig.


>  endmenu
> diff --git a/drivers/pci/controller/mobiveil/Makefile b/drivers/pci/controller/mobiveil/Makefile
> index 9fb6d1c6504d..99d879de32d6 100644
> --- a/drivers/pci/controller/mobiveil/Makefile
> +++ b/drivers/pci/controller/mobiveil/Makefile
> @@ -2,3 +2,4 @@
>  obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
>  obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
>  obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
> +obj-$(CONFIG_PCIE_LAYERSCAPE_GEN4) += pcie-layerscape-gen4.o
> diff --git a/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> new file mode 100644
> index 000000000000..6c0d3e2532db
> --- /dev/null
> +++ b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> @@ -0,0 +1,274 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * PCIe Gen4 host controller driver for NXP Layerscape SoCs
> + *
> + * Copyright 2019 NXP
> + *
> + * Author: Zhiqiang Hou <Zhiqiang.Hou@nxp.com>
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/interrupt.h>
> +#include <linux/init.h>
> +#include <linux/of_pci.h>
> +#include <linux/of_platform.h>
> +#include <linux/of_irq.h>
> +#include <linux/of_address.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/resource.h>
> +#include <linux/mfd/syscon.h>
> +#include <linux/regmap.h>
> +
> +#include "pcie-mobiveil.h"
> +
> +/* LUT and PF control registers */
> +#define PCIE_LUT_OFF			0x80000
> +#define PCIE_PF_OFF			0xc0000
> +#define PCIE_PF_INT_STAT		0x18
> +#define PF_INT_STAT_PABRST		BIT(31)
> +
> +#define PCIE_PF_DBG			0x7fc
> +#define PF_DBG_LTSSM_MASK		0x3f
> +#define PF_DBG_LTSSM_L0			0x2d /* L0 state */
> +#define PF_DBG_WE			BIT(31)
> +#define PF_DBG_PABR			BIT(27)
> +
> +#define to_ls_pcie_g4(x)		platform_get_drvdata((x)->pdev)
> +
> +struct ls_pcie_g4 {
> +	struct mobiveil_pcie pci;
> +	struct delayed_work dwork;
> +	int irq;
> +};
> +
> +static inline u32 ls_pcie_g4_lut_readl(struct ls_pcie_g4 *pcie, u32 off)
> +{
> +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
> +}
> +
> +static inline void ls_pcie_g4_lut_writel(struct ls_pcie_g4 *pcie,
> +					 u32 off, u32 val)
> +{
> +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
> +}
> +
> +static inline u32 ls_pcie_g4_pf_readl(struct ls_pcie_g4 *pcie, u32 off)
> +{
> +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off);
> +}
> +
> +static inline void ls_pcie_g4_pf_writel(struct ls_pcie_g4 *pcie,
> +					u32 off, u32 val)
> +{
> +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off);
> +}
> +
> +static bool ls_pcie_g4_is_bridge(struct ls_pcie_g4 *pcie)
> +{
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +	u32 header_type;
> +
> +	header_type = mobiveil_csr_readb(mv_pci, PCI_HEADER_TYPE);
> +	header_type &= 0x7f;
> +
> +	return header_type == PCI_HEADER_TYPE_BRIDGE;
> +}
> +
> +static int ls_pcie_g4_link_up(struct mobiveil_pcie *pci)
> +{
> +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(pci);
> +	u32 state;
> +
> +	state = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> +	state =	state & PF_DBG_LTSSM_MASK;
> +
> +	if (state == PF_DBG_LTSSM_L0)
> +		return 1;
> +
> +	return 0;
> +}
> +
> +static void ls_pcie_g4_disable_interrupt(struct ls_pcie_g4 *pcie)
> +{
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +
> +	mobiveil_csr_writel(mv_pci, 0, PAB_INTP_AMBA_MISC_ENB);
> +}
> +
> +static void ls_pcie_g4_enable_interrupt(struct ls_pcie_g4 *pcie)
> +{
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +	u32 val;
> +
> +	/* Clear the interrupt status */
> +	mobiveil_csr_writel(mv_pci, 0xffffffff, PAB_INTP_AMBA_MISC_STAT);
> +
> +	val = PAB_INTP_INTX_MASK | PAB_INTP_MSI | PAB_INTP_RESET |
> +	      PAB_INTP_PCIE_UE | PAB_INTP_IE_PMREDI | PAB_INTP_IE_EC;
> +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_ENB);
> +}
> +
> +static void ls_pcie_g4_reinit_hw(struct ls_pcie_g4 *pcie)
> +{
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +	struct device *dev = &mv_pci->pdev->dev;
> +	u32 val, act_stat;
> +	int to = 100;
> +
> +	/* Poll for pab_csb_reset to set and PAB activity to clear */
> +	do {
> +		usleep_range(10, 15);
> +		val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_INT_STAT);
> +		act_stat = mobiveil_csr_readl(mv_pci, PAB_ACTIVITY_STAT);
> +	} while (((val & PF_INT_STAT_PABRST) == 0 || act_stat) && to--);
> +	if (to < 0) {
> +		dev_err(dev, "Poll PABRST&PABACT timeout\n");
> +		return;

If a timeout happens here - the caller has no idea this has happened
and yet the following work doesn't get done. Isn't this a problem?

> +	}
> +
> +	/* clear PEX_RESET bit in PEX_PF0_DBG register */
> +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> +	val |= PF_DBG_WE;
> +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> +
> +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> +	val |= PF_DBG_PABR;
> +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> +
> +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> +	val &= ~PF_DBG_WE;
> +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> +
> +	mobiveil_host_init(mv_pci, true);

Can mobiveil_host_init fail?

> +
> +	to = 100;
> +	while (!ls_pcie_g4_link_up(mv_pci) && to--)
> +		usleep_range(200, 250);
> +	if (to < 0)
> +		dev_err(dev, "PCIe link training timeout\n");
> +}
> +
> +static irqreturn_t ls_pcie_g4_isr(int irq, void *dev_id)
> +{
> +	struct ls_pcie_g4 *pcie = (struct ls_pcie_g4 *)dev_id;
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +	u32 val;
> +
> +	val = mobiveil_csr_readl(mv_pci, PAB_INTP_AMBA_MISC_STAT);
> +	if (!val)
> +		return IRQ_NONE;
> +
> +	if (val & PAB_INTP_RESET) {

Can you explain why this is needed (perhaps also in the cover letter)?

> +		ls_pcie_g4_disable_interrupt(pcie);
> +		schedule_delayed_work(&pcie->dwork, msecs_to_jiffies(1));
> +	}
> +
> +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_STAT);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static int ls_pcie_g4_interrupt_init(struct mobiveil_pcie *mv_pci)
> +{
> +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(mv_pci);
> +	struct platform_device *pdev = mv_pci->pdev;
> +	struct device *dev = &pdev->dev;
> +	int ret;
> +
> +	pcie->irq = platform_get_irq_byname(pdev, "intr");
> +	if (pcie->irq < 0) {
> +		dev_err(dev, "Can't get 'intr' IRQ, errno = %d\n", pcie->irq);
> +		return pcie->irq;
> +	}
> +	ret = devm_request_irq(dev, pcie->irq, ls_pcie_g4_isr,
> +			       IRQF_SHARED, pdev->name, pcie);
> +	if (ret) {
> +		dev_err(dev, "Can't register PCIe IRQ, errno = %d\n", ret);
> +		return  ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static void ls_pcie_g4_reset(struct work_struct *work)
> +{
> +	struct delayed_work *dwork = container_of(work, struct delayed_work,
> +						  work);
> +	struct ls_pcie_g4 *pcie = container_of(dwork, struct ls_pcie_g4, dwork);
> +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> +	u16 ctrl;
> +
> +	ctrl = mobiveil_csr_readw(mv_pci, PCI_BRIDGE_CONTROL);
> +	ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
> +	mobiveil_csr_writew(mv_pci, ctrl, PCI_BRIDGE_CONTROL);
> +	ls_pcie_g4_reinit_hw(pcie);
> +	ls_pcie_g4_enable_interrupt(pcie);
> +}
> +
> +static struct mobiveil_rp_ops ls_pcie_g4_rp_ops = {
> +	.interrupt_init = ls_pcie_g4_interrupt_init,
> +};
> +
> +static const struct mobiveil_pab_ops ls_pcie_g4_pab_ops = {
> +	.link_up = ls_pcie_g4_link_up,
> +};
> +
> +static int __init ls_pcie_g4_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct pci_host_bridge *bridge;
> +	struct mobiveil_pcie *mv_pci;
> +	struct ls_pcie_g4 *pcie;
> +	struct device_node *np = dev->of_node;
> +	int ret;
> +
> +	if (!of_parse_phandle(np, "msi-parent", 0)) {
> +		dev_err(dev, "Failed to find msi-parent\n");
> +		return -EINVAL;
> +	}
> +
> +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> +	if (!bridge)
> +		return -ENOMEM;
> +
> +	pcie = pci_host_bridge_priv(bridge);
> +	mv_pci = &pcie->pci;
> +
> +	mv_pci->pdev = pdev;
> +	mv_pci->ops = &ls_pcie_g4_pab_ops;
> +	mv_pci->rp.ops = &ls_pcie_g4_rp_ops;
> +	mv_pci->rp.bridge = bridge;
> +
> +	platform_set_drvdata(pdev, pcie);
> +
> +	INIT_DELAYED_WORK(&pcie->dwork, ls_pcie_g4_reset);
> +
> +	ret = mobiveil_pcie_host_probe(mv_pci);
> +	if (ret) {
> +		dev_err(dev, "Fail to probe\n");
> +		return  ret;
> +	}
> +
> +	if (!ls_pcie_g4_is_bridge(pcie))

Is this a check that could apply to all host bridge drivers and thus live
in mobiveil_pcie_host_probe?

> +		return -ENODEV;
> +
> +	ls_pcie_g4_enable_interrupt(pcie);

Is there an issue here in that we enable interrupts *after* telling the kernel
about our controller? (Same applies for bailing if the IP isn't a bridge).

Thanks,

Andrew Murray

> +
> +	return 0;
> +}
> +
> +static const struct of_device_id ls_pcie_g4_of_match[] = {
> +	{ .compatible = "fsl,lx2160a-pcie", },
> +	{ },
> +};
> +
> +static struct platform_driver ls_pcie_g4_driver = {
> +	.driver = {
> +		.name = "layerscape-pcie-gen4",
> +		.of_match_table = ls_pcie_g4_of_match,
> +		.suppress_bind_attrs = true,
> +	},
> +};
> +
> +builtin_platform_driver_probe(ls_pcie_g4_driver, ls_pcie_g4_probe);
> diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> index 750a7fd95bc1..c57a68d2bac4 100644
> --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> @@ -43,6 +43,8 @@
>  #define  PAGE_LO_MASK			0x3ff
>  #define  PAGE_SEL_OFFSET_SHIFT		10
>  
> +#define PAB_ACTIVITY_STAT		0x81c
> +
>  #define PAB_AXI_PIO_CTRL		0x0840
>  #define  APIO_EN_MASK			0xf
>  
> @@ -51,8 +53,18 @@
>  
>  #define PAB_INTP_AMBA_MISC_ENB		0x0b0c
>  #define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> -#define  PAB_INTP_INTX_MASK		0x01e0
> -#define  PAB_INTP_MSI_MASK		0x8
> +#define  PAB_INTP_RESET			BIT(1)
> +#define  PAB_INTP_MSI			BIT(3)
> +#define  PAB_INTP_INTA			BIT(5)
> +#define  PAB_INTP_INTB			BIT(6)
> +#define  PAB_INTP_INTC			BIT(7)
> +#define  PAB_INTP_INTD			BIT(8)
> +#define  PAB_INTP_PCIE_UE		BIT(9)
> +#define  PAB_INTP_IE_PMREDI		BIT(29)
> +#define  PAB_INTP_IE_EC			BIT(30)
> +#define  PAB_INTP_MSI_MASK		PAB_INTP_MSI
> +#define  PAB_INTP_INTX_MASK		(PAB_INTP_INTA | PAB_INTP_INTB |\
> +					PAB_INTP_INTC | PAB_INTP_INTD)
>  
>  #define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
>  #define  WIN_ENABLE_SHIFT		0
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-01-10 17:05       ` Olof Johansson
@ 2020-02-06 10:57         ` Z.q. Hou
  2020-02-10 15:12           ` Olof Johansson
  0 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 10:57 UTC (permalink / raw)
  To: Olof Johansson, Lorenzo Pieralisi
  Cc: bhelgaas, linux-pci, linux-arm-kernel, devicetree, linux-kernel,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, catalin.marinas, will.deacon,
	andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Olof,

Thanks a lot for your comments!
And sorry for my delay respond!

> -----Original Message-----
> From: Olof Johansson <olof@lixom.net>
> Sent: 2020年1月11日 1:06
> To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Cc: Z.q. Hou <zhiqiang.hou@nxp.com>; bhelgaas@google.com;
> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> robh+dt@kernel.org; arnd@arndb.de; mark.rutland@arm.com;
> l.subrahmanya@mobiveil.co.in; shawnguo@kernel.org;
> m.karthikeyan@mobiveil.co.in; Leo Li <leoyang.li@nxp.com>;
> catalin.marinas@arm.com; will.deacon@arm.com; andrew.murray@arm.com;
> Mingkai Hu <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>;
> Xiaowei Bao <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4
> driver for NXP Layerscape SoCs
> 
> On Fri, Jan 10, 2020 at 7:33 AM Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> wrote:
> >
> > On Tue, Dec 17, 2019 at 02:50:15AM +0000, Z.q. Hou wrote:
> > > Hi Lorenzo,
> > >
> > > The v9 patches have addressed the comments from Andrew, and it has
> > > been dried about 1 month, can you help to apply them?
> >
> > We shall have a look beginning of next week, sorry for the delay in
> > getting back to you.
> 
> Note that the patch set no longer applies since the refactorings conflict with
> new development by others.
> 
> Zhiqiang, can you rebase and post a new version of the patch set?

Yes, I will rebase the patches to the latest code base.

Thanks,
Zhiqiang

 
> 
> -Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
  2020-01-13 10:09   ` Andrew Murray
@ 2020-02-06 11:04     ` Z.q. Hou
  2020-02-06 11:27       ` Andrew Murray
  0 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 11:04 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!
Sorry for my delay respond!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 18:10
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
> 
> On Wed, Nov 20, 2019 at 03:45:23AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The Mobiveil PCIe controller can work in either Root Complex mode or
> > Endpoint mode. So introduce a new structure root_port, and abstract
> > the RC related members into it.
> 
> The first sentence explains the trigger for this work, the second explains what
> you are changing, it would be helpful to also describe why you need to make
> this change. You could do this by extending the last sentence, e.g.
> 
> "So introduce a new structure root_port, and abstract the RC  related
> members into it such that it can be used by both ..."
> 
> As this series doesn't actually add a EP driver, this abstraction isn't needed
> now - but it is nice to have - it may be helpful to explain this.
> 
> The email subject could also more precisely explain what this patch does.

Thanks for the good suggestions! Will change in v10.

> 
> 
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> 
> Indeed, it's much nicer to review - thanks.
> 
> 
> >
> >  drivers/pci/controller/pcie-mobiveil.c | 99
> > ++++++++++++++++----------
> >  1 file changed, 60 insertions(+), 39 deletions(-)
> >
> > diff --git a/drivers/pci/controller/pcie-mobiveil.c
> > b/drivers/pci/controller/pcie-mobiveil.c
> > index 3a696ca45bfa..5fd26e376af2 100644
> > --- a/drivers/pci/controller/pcie-mobiveil.c
> > +++ b/drivers/pci/controller/pcie-mobiveil.c
> > @@ -3,7 +3,10 @@
> >   * PCIe host controller driver for Mobiveil PCIe Host controller
> >   *
> >   * Copyright (c) 2018 Mobiveil Inc.
> > + * Copyright 2019 NXP
> > + *
> >   * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> As per my previous feedback, I'm not sure the value of using the term refactor
> or a synonym of it. And I certaintly don't want to encourage anyone that
> modifies this file to add a similar tag when the information is easily visible via
> GIT and the get_maintainers script.

Will remove the recode tag in v10.

> 
> >   */
> >
> >  #include <linux/delay.h>
> > @@ -138,22 +141,27 @@ struct mobiveil_msi {			/* MSI
> information */
> >  	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);  };
> >
> > +struct root_port {
> > +	char root_bus_nr;
> > +	void __iomem *config_axi_slave_base;	/* endpoint config base */
> > +	struct resource *ob_io_res;
> > +	int irq;
> > +	raw_spinlock_t intx_mask_lock;
> > +	struct irq_domain *intx_domain;
> > +	struct mobiveil_msi msi;
> > +	struct pci_host_bridge *bridge;
> > +};
> 
> Please prefix with mobiveil given we have mobiveil related structures inside it.

Do you mean s/root_port/mobiveil_root_port/ ?

Thanks,
Zhiqiang

> 
> (Also on your respin, please rebase as per Olof's feedback).
> 
> Thanks,
> 
> Andrew Murray
> 
> > +
> >  struct mobiveil_pcie {
> >  	struct platform_device *pdev;
> > -	void __iomem *config_axi_slave_base;	/* endpoint config base */
> >  	void __iomem *csr_axi_slave_base;	/* root port config base */
> >  	void __iomem *apb_csr_base;	/* MSI register base */
> >  	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> > -	struct irq_domain *intx_domain;
> > -	raw_spinlock_t intx_mask_lock;
> > -	int irq;
> >  	int apio_wins;
> >  	int ppio_wins;
> >  	int ob_wins_configured;		/* configured outbound windows */
> >  	int ib_wins_configured;		/* configured inbound windows */
> > -	struct resource *ob_io_res;
> > -	char root_bus_nr;
> > -	struct mobiveil_msi msi;
> > +	struct root_port rp;
> >  };
> >
> >  /*
> > @@ -281,16 +289,17 @@ static bool mobiveil_pcie_link_up(struct
> > mobiveil_pcie *pcie)  static bool mobiveil_pcie_valid_device(struct
> > pci_bus *bus, unsigned int devfn)  {
> >  	struct mobiveil_pcie *pcie = bus->sysdata;
> > +	struct root_port *rp = &pcie->rp;
> >
> >  	/* Only one device down on each root port */
> > -	if ((bus->number == pcie->root_bus_nr) && (devfn > 0))
> > +	if ((bus->number == rp->root_bus_nr) && (devfn > 0))
> >  		return false;
> >
> >  	/*
> >  	 * Do not read more than one device on the bus directly
> >  	 * attached to RC
> >  	 */
> > -	if ((bus->primary == pcie->root_bus_nr) && (PCI_SLOT(devfn) > 0))
> > +	if ((bus->primary == rp->root_bus_nr) && (PCI_SLOT(devfn) > 0))
> >  		return false;
> >
> >  	return true;
> > @@ -304,13 +313,14 @@ static void __iomem
> *mobiveil_pcie_map_bus(struct pci_bus *bus,
> >  					   unsigned int devfn, int where)  {
> >  	struct mobiveil_pcie *pcie = bus->sysdata;
> > +	struct root_port *rp = &pcie->rp;
> >  	u32 value;
> >
> >  	if (!mobiveil_pcie_valid_device(bus, devfn))
> >  		return NULL;
> >
> >  	/* RC config access */
> > -	if (bus->number == pcie->root_bus_nr)
> > +	if (bus->number == rp->root_bus_nr)
> >  		return pcie->csr_axi_slave_base + where;
> >
> >  	/*
> > @@ -325,7 +335,7 @@ static void __iomem
> *mobiveil_pcie_map_bus(struct
> > pci_bus *bus,
> >
> >  	mobiveil_csr_writel(pcie, value,
> PAB_AXI_AMAP_PEX_WIN_L(WIN_NUM_0));
> >
> > -	return pcie->config_axi_slave_base + where;
> > +	return rp->config_axi_slave_base + where;
> >  }
> >
> >  static struct pci_ops mobiveil_pcie_ops = { @@ -339,7 +349,8 @@
> > static void mobiveil_pcie_isr(struct irq_desc *desc)
> >  	struct irq_chip *chip = irq_desc_get_chip(desc);
> >  	struct mobiveil_pcie *pcie = irq_desc_get_handler_data(desc);
> >  	struct device *dev = &pcie->pdev->dev;
> > -	struct mobiveil_msi *msi = &pcie->msi;
> > +	struct root_port *rp = &pcie->rp;
> > +	struct mobiveil_msi *msi = &rp->msi;
> >  	u32 msi_data, msi_addr_lo, msi_addr_hi;
> >  	u32 intr_status, msi_status;
> >  	unsigned long shifted_status;
> > @@ -365,7 +376,7 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
> >  		shifted_status >>= PAB_INTX_START;
> >  		do {
> >  			for_each_set_bit(bit, &shifted_status, PCI_NUM_INTX) {
> > -				virq = irq_find_mapping(pcie->intx_domain,
> > +				virq = irq_find_mapping(rp->intx_domain,
> >  							bit + 1);
> >  				if (virq)
> >  					generic_handle_irq(virq);
> > @@ -424,15 +435,16 @@ static int mobiveil_pcie_parse_dt(struct
> mobiveil_pcie *pcie)
> >  	struct device *dev = &pcie->pdev->dev;
> >  	struct platform_device *pdev = pcie->pdev;
> >  	struct device_node *node = dev->of_node;
> > +	struct root_port *rp = &pcie->rp;
> >  	struct resource *res;
> >
> >  	/* map config resource */
> >  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> >  					   "config_axi_slave");
> > -	pcie->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> > -	if (IS_ERR(pcie->config_axi_slave_base))
> > -		return PTR_ERR(pcie->config_axi_slave_base);
> > -	pcie->ob_io_res = res;
> > +	rp->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> > +	if (IS_ERR(rp->config_axi_slave_base))
> > +		return PTR_ERR(rp->config_axi_slave_base);
> > +	rp->ob_io_res = res;
> >
> >  	/* map csr resource */
> >  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, @@
> -455,9
> > +467,9 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
> >  	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
> >  		pcie->ppio_wins = MAX_PIO_WINDOWS;
> >
> > -	pcie->irq = platform_get_irq(pdev, 0);
> > -	if (pcie->irq <= 0) {
> > -		dev_err(dev, "failed to map IRQ: %d\n", pcie->irq);
> > +	rp->irq = platform_get_irq(pdev, 0);
> > +	if (rp->irq <= 0) {
> > +		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> >  		return -ENODEV;
> >  	}
> >
> > @@ -564,9 +576,9 @@ static int mobiveil_bringup_link(struct
> > mobiveil_pcie *pcie)  static void mobiveil_pcie_enable_msi(struct
> > mobiveil_pcie *pcie)  {
> >  	phys_addr_t msg_addr = pcie->pcie_reg_base;
> > -	struct mobiveil_msi *msi = &pcie->msi;
> > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> >
> > -	pcie->msi.num_of_vectors = PCI_NUM_MSI;
> > +	msi->num_of_vectors = PCI_NUM_MSI;
> >  	msi->msi_pages_phys = (phys_addr_t)msg_addr;
> >
> >  	writel_relaxed(lower_32_bits(msg_addr),
> > @@ -579,7 +591,8 @@ static void mobiveil_pcie_enable_msi(struct
> > mobiveil_pcie *pcie)
> >
> >  static int mobiveil_host_init(struct mobiveil_pcie *pcie)  {
> > -	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > +	struct root_port *rp = &pcie->rp;
> > +	struct pci_host_bridge *bridge = rp->bridge;
> >  	u32 value, pab_ctrl, type;
> >  	struct resource_entry *win;
> >
> > @@ -629,8 +642,8 @@ static int mobiveil_host_init(struct mobiveil_pcie
> *pcie)
> >  	 */
> >
> >  	/* config outbound translation window */
> > -	program_ob_windows(pcie, WIN_NUM_0, pcie->ob_io_res->start, 0,
> > -			   CFG_WINDOW_TYPE, resource_size(pcie->ob_io_res));
> > +	program_ob_windows(pcie, WIN_NUM_0, rp->ob_io_res->start, 0,
> > +			   CFG_WINDOW_TYPE, resource_size(rp->ob_io_res));
> >
> >  	/* memory inbound translation window */
> >  	program_ib_windows(pcie, WIN_NUM_0, 0, 0, MEM_WINDOW_TYPE,
> > IB_WIN_SIZE); @@ -667,32 +680,36 @@ static void
> > mobiveil_mask_intx_irq(struct irq_data *data)  {
> >  	struct irq_desc *desc = irq_to_desc(data->irq);
> >  	struct mobiveil_pcie *pcie;
> > +	struct root_port *rp;
> >  	unsigned long flags;
> >  	u32 mask, shifted_val;
> >
> >  	pcie = irq_desc_get_chip_data(desc);
> > +	rp = &pcie->rp;
> >  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> > -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> > +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
> >  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
> >  	shifted_val &= ~mask;
> >  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> > -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> > +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
> >  }
> >
> >  static void mobiveil_unmask_intx_irq(struct irq_data *data)  {
> >  	struct irq_desc *desc = irq_to_desc(data->irq);
> >  	struct mobiveil_pcie *pcie;
> > +	struct root_port *rp;
> >  	unsigned long flags;
> >  	u32 shifted_val, mask;
> >
> >  	pcie = irq_desc_get_chip_data(desc);
> > +	rp = &pcie->rp;
> >  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> > -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> > +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
> >  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
> >  	shifted_val |= mask;
> >  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> > -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> > +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
> >  }
> >
> >  static struct irq_chip intx_irq_chip = { @@ -760,7 +777,7 @@ static
> > int mobiveil_irq_msi_domain_alloc(struct irq_domain *domain,
> >  					 unsigned int nr_irqs, void *args)  {
> >  	struct mobiveil_pcie *pcie = domain->host_data;
> > -	struct mobiveil_msi *msi = &pcie->msi;
> > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> >  	unsigned long bit;
> >
> >  	WARN_ON(nr_irqs != 1);
> > @@ -787,7 +804,7 @@ static void mobiveil_irq_msi_domain_free(struct
> > irq_domain *domain,  {
> >  	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
> >  	struct mobiveil_pcie *pcie = irq_data_get_irq_chip_data(d);
> > -	struct mobiveil_msi *msi = &pcie->msi;
> > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> >
> >  	mutex_lock(&msi->lock);
> >
> > @@ -808,9 +825,9 @@ static int mobiveil_allocate_msi_domains(struct
> > mobiveil_pcie *pcie)  {
> >  	struct device *dev = &pcie->pdev->dev;
> >  	struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
> > -	struct mobiveil_msi *msi = &pcie->msi;
> > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> >
> > -	mutex_init(&pcie->msi.lock);
> > +	mutex_init(&msi->lock);
> >  	msi->dev_domain = irq_domain_add_linear(NULL,
> msi->num_of_vectors,
> >  						&msi_domain_ops, pcie);
> >  	if (!msi->dev_domain) {
> > @@ -834,18 +851,19 @@ static int mobiveil_pcie_init_irq_domain(struct
> > mobiveil_pcie *pcie)  {
> >  	struct device *dev = &pcie->pdev->dev;
> >  	struct device_node *node = dev->of_node;
> > +	struct root_port *rp = &pcie->rp;
> >  	int ret;
> >
> >  	/* setup INTx */
> > -	pcie->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> > -						  &intx_domain_ops, pcie);
> > +	rp->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> > +						&intx_domain_ops, pcie);
> >
> > -	if (!pcie->intx_domain) {
> > +	if (!rp->intx_domain) {
> >  		dev_err(dev, "Failed to get a INTx IRQ domain\n");
> >  		return -ENOMEM;
> >  	}
> >
> > -	raw_spin_lock_init(&pcie->intx_mask_lock);
> > +	raw_spin_lock_init(&rp->intx_mask_lock);
> >
> >  	/* setup MSI */
> >  	ret = mobiveil_allocate_msi_domains(pcie);
> > @@ -862,6 +880,7 @@ static int mobiveil_pcie_probe(struct
> platform_device *pdev)
> >  	struct pci_bus *child;
> >  	struct pci_host_bridge *bridge;
> >  	struct device *dev = &pdev->dev;
> > +	struct root_port *rp;
> >  	int ret;
> >
> >  	/* allocate the PCIe port */
> > @@ -870,6 +889,8 @@ static int mobiveil_pcie_probe(struct
> platform_device *pdev)
> >  		return -ENOMEM;
> >
> >  	pcie = pci_host_bridge_priv(bridge);
> > +	rp = &pcie->rp;
> > +	rp->bridge = bridge;
> >
> >  	pcie->pdev = pdev;
> >
> > @@ -904,12 +925,12 @@ static int mobiveil_pcie_probe(struct
> platform_device *pdev)
> >  		return ret;
> >  	}
> >
> > -	irq_set_chained_handler_and_data(pcie->irq, mobiveil_pcie_isr, pcie);
> > +	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> >
> >  	/* Initialize bridge */
> >  	bridge->dev.parent = dev;
> >  	bridge->sysdata = pcie;
> > -	bridge->busnr = pcie->root_bus_nr;
> > +	bridge->busnr = rp->root_bus_nr;
> >  	bridge->ops = &mobiveil_pcie_ops;
> >  	bridge->map_irq = of_irq_parse_and_map_pci;
> >  	bridge->swizzle_irq = pci_common_swizzle;
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine
  2020-01-13 10:19   ` Andrew Murray
@ 2020-02-06 11:14     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 11:14 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 18:19
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a
> routine
> 
> On Wed, Nov 20, 2019 at 03:45:30AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Move the host initialization related operations into a new routine to
> > make it can be reused by other incoming platform's
> 
> s/to make/such that/
> 
> 'function' is probably a better word than 'routine'.
> 

Will change in v10.

> 
> > PCIe host driver, in which the Mobiveil GPEX is integrated.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> >
> >  drivers/pci/controller/pcie-mobiveil.c | 38
> > +++++++++++++++-----------
> >  1 file changed, 22 insertions(+), 16 deletions(-)
> >
> > diff --git a/drivers/pci/controller/pcie-mobiveil.c
> > b/drivers/pci/controller/pcie-mobiveil.c
> > index 5fd26e376af2..97f682ca7c7a 100644
> > --- a/drivers/pci/controller/pcie-mobiveil.c
> > +++ b/drivers/pci/controller/pcie-mobiveil.c
> > @@ -873,27 +873,15 @@ static int mobiveil_pcie_init_irq_domain(struct
> mobiveil_pcie *pcie)
> >  	return 0;
> >  }
> >
> > -static int mobiveil_pcie_probe(struct platform_device *pdev)
> > +int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
> 
> This is no longer static - but do you need to add a header file somewhere?

The function mobiveil_pcie_probe() is still static below, the
mobiveil_pcie_host_probe() is new introduced one, it is now only used in this
.c file, so I'll change it to a static function in v10.

> 
> 
> >  {
> > -	struct mobiveil_pcie *pcie;
> > +	struct root_port *rp = &pcie->rp;
> > +	struct pci_host_bridge *bridge = rp->bridge;
> > +	struct device *dev = &pcie->pdev->dev;
> >  	struct pci_bus *bus;
> >  	struct pci_bus *child;
> > -	struct pci_host_bridge *bridge;
> > -	struct device *dev = &pdev->dev;
> > -	struct root_port *rp;
> >  	int ret;
> >
> > -	/* allocate the PCIe port */
> > -	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> > -	if (!bridge)
> > -		return -ENOMEM;
> > -
> > -	pcie = pci_host_bridge_priv(bridge);
> > -	rp = &pcie->rp;
> > -	rp->bridge = bridge;
> > -
> > -	pcie->pdev = pdev;
> > -
> >  	ret = mobiveil_pcie_parse_dt(pcie);
> >  	if (ret) {
> >  		dev_err(dev, "Parsing DT failed, ret: %x\n", ret); @@ -956,6
> > +944,24 @@ static int mobiveil_pcie_probe(struct platform_device *pdev)
> >  	return 0;
> >  }
> >
> > +static int mobiveil_pcie_probe(struct platform_device *pdev) {
> > +	struct mobiveil_pcie *pcie;
> > +	struct pci_host_bridge *bridge;
> > +	struct device *dev = &pdev->dev;
> > +
> > +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> 
> You've lost the comment that was above this.

Will add it back.

Thanks,
Zhiqiang

> 
> Thanks,
> 
> Andrew Murray
> 
> > +	if (!bridge)
> > +		return -ENOMEM;
> > +
> > +	pcie = pci_host_bridge_priv(bridge);
> > +	pcie->rp.bridge = bridge;
> > +
> > +	pcie->pdev = pdev;
> > +
> > +	return mobiveil_pcie_host_probe(pcie); }
> > +
> >  static const struct of_device_id mobiveil_pcie_of_match[] = {
> >  	{.compatible = "mbvl,gpex40-pcie",},
> >  	{},
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
  2020-02-06 11:04     ` Z.q. Hou
@ 2020-02-06 11:27       ` Andrew Murray
  0 siblings, 0 replies; 55+ messages in thread
From: Andrew Murray @ 2020-02-06 11:27 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: Andrew Murray, linux-pci, linux-arm-kernel, devicetree,
	linux-kernel, bhelgaas, robh+dt, arnd, mark.rutland,
	l.subrahmanya, shawnguo, m.karthikeyan, Leo Li,
	lorenzo.pieralisi, catalin.marinas, will.deacon, Mingkai Hu,
	M.h. Lian, Xiaowei Bao

On Thu, Feb 06, 2020 at 11:04:29AM +0000, Z.q. Hou wrote:
> Hi Andrew,
> 
> Thanks a lot for your comments!
> Sorry for my delay respond!
> 
> > -----Original Message-----
> > From: Andrew Murray <andrew.murray@arm.com>
> > Sent: 2020年1月13日 18:10
> > To: Z.q. Hou <zhiqiang.hou@nxp.com>
> > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> > bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> > mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> > shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> > <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> > catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> > <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> > <xiaowei.bao@nxp.com>
> > Subject: Re: [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure
> > 
> > On Wed, Nov 20, 2019 at 03:45:23AM +0000, Z.q. Hou wrote:
> > > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > >
> > > The Mobiveil PCIe controller can work in either Root Complex mode or
> > > Endpoint mode. So introduce a new structure root_port, and abstract
> > > the RC related members into it.
> > 
> > The first sentence explains the trigger for this work, the second explains what
> > you are changing, it would be helpful to also describe why you need to make
> > this change. You could do this by extending the last sentence, e.g.
> > 
> > "So introduce a new structure root_port, and abstract the RC  related
> > members into it such that it can be used by both ..."
> > 
> > As this series doesn't actually add a EP driver, this abstraction isn't needed
> > now - but it is nice to have - it may be helpful to explain this.
> > 
> > The email subject could also more precisely explain what this patch does.
> 
> Thanks for the good suggestions! Will change in v10.
> 
> > 
> > 
> > >
> > > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > > ---
> > > V9:
> > >  - New patch splited from the #1 of V8 patches to make it easy to review.
> > 
> > Indeed, it's much nicer to review - thanks.
> > 
> > 
> > >
> > >  drivers/pci/controller/pcie-mobiveil.c | 99
> > > ++++++++++++++++----------
> > >  1 file changed, 60 insertions(+), 39 deletions(-)
> > >
> > > diff --git a/drivers/pci/controller/pcie-mobiveil.c
> > > b/drivers/pci/controller/pcie-mobiveil.c
> > > index 3a696ca45bfa..5fd26e376af2 100644
> > > --- a/drivers/pci/controller/pcie-mobiveil.c
> > > +++ b/drivers/pci/controller/pcie-mobiveil.c
> > > @@ -3,7 +3,10 @@
> > >   * PCIe host controller driver for Mobiveil PCIe Host controller
> > >   *
> > >   * Copyright (c) 2018 Mobiveil Inc.
> > > + * Copyright 2019 NXP
> > > + *
> > >   * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > > + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > 
> > As per my previous feedback, I'm not sure the value of using the term refactor
> > or a synonym of it. And I certaintly don't want to encourage anyone that
> > modifies this file to add a similar tag when the information is easily visible via
> > GIT and the get_maintainers script.
> 
> Will remove the recode tag in v10.
> 
> > 
> > >   */
> > >
> > >  #include <linux/delay.h>
> > > @@ -138,22 +141,27 @@ struct mobiveil_msi {			/* MSI
> > information */
> > >  	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);  };
> > >
> > > +struct root_port {
> > > +	char root_bus_nr;
> > > +	void __iomem *config_axi_slave_base;	/* endpoint config base */
> > > +	struct resource *ob_io_res;
> > > +	int irq;
> > > +	raw_spinlock_t intx_mask_lock;
> > > +	struct irq_domain *intx_domain;
> > > +	struct mobiveil_msi msi;
> > > +	struct pci_host_bridge *bridge;
> > > +};
> > 
> > Please prefix with mobiveil given we have mobiveil related structures inside it.
> 
> Do you mean s/root_port/mobiveil_root_port/ ?

Yes!

Thanks,

Andrew Murray

> 
> Thanks,
> Zhiqiang
> 
> > 
> > (Also on your respin, please rebase as per Olof's feedback).
> > 
> > Thanks,
> > 
> > Andrew Murray
> > 
> > > +
> > >  struct mobiveil_pcie {
> > >  	struct platform_device *pdev;
> > > -	void __iomem *config_axi_slave_base;	/* endpoint config base */
> > >  	void __iomem *csr_axi_slave_base;	/* root port config base */
> > >  	void __iomem *apb_csr_base;	/* MSI register base */
> > >  	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> > > -	struct irq_domain *intx_domain;
> > > -	raw_spinlock_t intx_mask_lock;
> > > -	int irq;
> > >  	int apio_wins;
> > >  	int ppio_wins;
> > >  	int ob_wins_configured;		/* configured outbound windows */
> > >  	int ib_wins_configured;		/* configured inbound windows */
> > > -	struct resource *ob_io_res;
> > > -	char root_bus_nr;
> > > -	struct mobiveil_msi msi;
> > > +	struct root_port rp;
> > >  };
> > >
> > >  /*
> > > @@ -281,16 +289,17 @@ static bool mobiveil_pcie_link_up(struct
> > > mobiveil_pcie *pcie)  static bool mobiveil_pcie_valid_device(struct
> > > pci_bus *bus, unsigned int devfn)  {
> > >  	struct mobiveil_pcie *pcie = bus->sysdata;
> > > +	struct root_port *rp = &pcie->rp;
> > >
> > >  	/* Only one device down on each root port */
> > > -	if ((bus->number == pcie->root_bus_nr) && (devfn > 0))
> > > +	if ((bus->number == rp->root_bus_nr) && (devfn > 0))
> > >  		return false;
> > >
> > >  	/*
> > >  	 * Do not read more than one device on the bus directly
> > >  	 * attached to RC
> > >  	 */
> > > -	if ((bus->primary == pcie->root_bus_nr) && (PCI_SLOT(devfn) > 0))
> > > +	if ((bus->primary == rp->root_bus_nr) && (PCI_SLOT(devfn) > 0))
> > >  		return false;
> > >
> > >  	return true;
> > > @@ -304,13 +313,14 @@ static void __iomem
> > *mobiveil_pcie_map_bus(struct pci_bus *bus,
> > >  					   unsigned int devfn, int where)  {
> > >  	struct mobiveil_pcie *pcie = bus->sysdata;
> > > +	struct root_port *rp = &pcie->rp;
> > >  	u32 value;
> > >
> > >  	if (!mobiveil_pcie_valid_device(bus, devfn))
> > >  		return NULL;
> > >
> > >  	/* RC config access */
> > > -	if (bus->number == pcie->root_bus_nr)
> > > +	if (bus->number == rp->root_bus_nr)
> > >  		return pcie->csr_axi_slave_base + where;
> > >
> > >  	/*
> > > @@ -325,7 +335,7 @@ static void __iomem
> > *mobiveil_pcie_map_bus(struct
> > > pci_bus *bus,
> > >
> > >  	mobiveil_csr_writel(pcie, value,
> > PAB_AXI_AMAP_PEX_WIN_L(WIN_NUM_0));
> > >
> > > -	return pcie->config_axi_slave_base + where;
> > > +	return rp->config_axi_slave_base + where;
> > >  }
> > >
> > >  static struct pci_ops mobiveil_pcie_ops = { @@ -339,7 +349,8 @@
> > > static void mobiveil_pcie_isr(struct irq_desc *desc)
> > >  	struct irq_chip *chip = irq_desc_get_chip(desc);
> > >  	struct mobiveil_pcie *pcie = irq_desc_get_handler_data(desc);
> > >  	struct device *dev = &pcie->pdev->dev;
> > > -	struct mobiveil_msi *msi = &pcie->msi;
> > > +	struct root_port *rp = &pcie->rp;
> > > +	struct mobiveil_msi *msi = &rp->msi;
> > >  	u32 msi_data, msi_addr_lo, msi_addr_hi;
> > >  	u32 intr_status, msi_status;
> > >  	unsigned long shifted_status;
> > > @@ -365,7 +376,7 @@ static void mobiveil_pcie_isr(struct irq_desc *desc)
> > >  		shifted_status >>= PAB_INTX_START;
> > >  		do {
> > >  			for_each_set_bit(bit, &shifted_status, PCI_NUM_INTX) {
> > > -				virq = irq_find_mapping(pcie->intx_domain,
> > > +				virq = irq_find_mapping(rp->intx_domain,
> > >  							bit + 1);
> > >  				if (virq)
> > >  					generic_handle_irq(virq);
> > > @@ -424,15 +435,16 @@ static int mobiveil_pcie_parse_dt(struct
> > mobiveil_pcie *pcie)
> > >  	struct device *dev = &pcie->pdev->dev;
> > >  	struct platform_device *pdev = pcie->pdev;
> > >  	struct device_node *node = dev->of_node;
> > > +	struct root_port *rp = &pcie->rp;
> > >  	struct resource *res;
> > >
> > >  	/* map config resource */
> > >  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> > >  					   "config_axi_slave");
> > > -	pcie->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> > > -	if (IS_ERR(pcie->config_axi_slave_base))
> > > -		return PTR_ERR(pcie->config_axi_slave_base);
> > > -	pcie->ob_io_res = res;
> > > +	rp->config_axi_slave_base = devm_pci_remap_cfg_resource(dev, res);
> > > +	if (IS_ERR(rp->config_axi_slave_base))
> > > +		return PTR_ERR(rp->config_axi_slave_base);
> > > +	rp->ob_io_res = res;
> > >
> > >  	/* map csr resource */
> > >  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, @@
> > -455,9
> > > +467,9 @@ static int mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
> > >  	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
> > >  		pcie->ppio_wins = MAX_PIO_WINDOWS;
> > >
> > > -	pcie->irq = platform_get_irq(pdev, 0);
> > > -	if (pcie->irq <= 0) {
> > > -		dev_err(dev, "failed to map IRQ: %d\n", pcie->irq);
> > > +	rp->irq = platform_get_irq(pdev, 0);
> > > +	if (rp->irq <= 0) {
> > > +		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> > >  		return -ENODEV;
> > >  	}
> > >
> > > @@ -564,9 +576,9 @@ static int mobiveil_bringup_link(struct
> > > mobiveil_pcie *pcie)  static void mobiveil_pcie_enable_msi(struct
> > > mobiveil_pcie *pcie)  {
> > >  	phys_addr_t msg_addr = pcie->pcie_reg_base;
> > > -	struct mobiveil_msi *msi = &pcie->msi;
> > > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> > >
> > > -	pcie->msi.num_of_vectors = PCI_NUM_MSI;
> > > +	msi->num_of_vectors = PCI_NUM_MSI;
> > >  	msi->msi_pages_phys = (phys_addr_t)msg_addr;
> > >
> > >  	writel_relaxed(lower_32_bits(msg_addr),
> > > @@ -579,7 +591,8 @@ static void mobiveil_pcie_enable_msi(struct
> > > mobiveil_pcie *pcie)
> > >
> > >  static int mobiveil_host_init(struct mobiveil_pcie *pcie)  {
> > > -	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > > +	struct root_port *rp = &pcie->rp;
> > > +	struct pci_host_bridge *bridge = rp->bridge;
> > >  	u32 value, pab_ctrl, type;
> > >  	struct resource_entry *win;
> > >
> > > @@ -629,8 +642,8 @@ static int mobiveil_host_init(struct mobiveil_pcie
> > *pcie)
> > >  	 */
> > >
> > >  	/* config outbound translation window */
> > > -	program_ob_windows(pcie, WIN_NUM_0, pcie->ob_io_res->start, 0,
> > > -			   CFG_WINDOW_TYPE, resource_size(pcie->ob_io_res));
> > > +	program_ob_windows(pcie, WIN_NUM_0, rp->ob_io_res->start, 0,
> > > +			   CFG_WINDOW_TYPE, resource_size(rp->ob_io_res));
> > >
> > >  	/* memory inbound translation window */
> > >  	program_ib_windows(pcie, WIN_NUM_0, 0, 0, MEM_WINDOW_TYPE,
> > > IB_WIN_SIZE); @@ -667,32 +680,36 @@ static void
> > > mobiveil_mask_intx_irq(struct irq_data *data)  {
> > >  	struct irq_desc *desc = irq_to_desc(data->irq);
> > >  	struct mobiveil_pcie *pcie;
> > > +	struct root_port *rp;
> > >  	unsigned long flags;
> > >  	u32 mask, shifted_val;
> > >
> > >  	pcie = irq_desc_get_chip_data(desc);
> > > +	rp = &pcie->rp;
> > >  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> > > -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> > > +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
> > >  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
> > >  	shifted_val &= ~mask;
> > >  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> > > -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> > > +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
> > >  }
> > >
> > >  static void mobiveil_unmask_intx_irq(struct irq_data *data)  {
> > >  	struct irq_desc *desc = irq_to_desc(data->irq);
> > >  	struct mobiveil_pcie *pcie;
> > > +	struct root_port *rp;
> > >  	unsigned long flags;
> > >  	u32 shifted_val, mask;
> > >
> > >  	pcie = irq_desc_get_chip_data(desc);
> > > +	rp = &pcie->rp;
> > >  	mask = 1 << ((data->hwirq + PAB_INTX_START) - 1);
> > > -	raw_spin_lock_irqsave(&pcie->intx_mask_lock, flags);
> > > +	raw_spin_lock_irqsave(&rp->intx_mask_lock, flags);
> > >  	shifted_val = mobiveil_csr_readl(pcie, PAB_INTP_AMBA_MISC_ENB);
> > >  	shifted_val |= mask;
> > >  	mobiveil_csr_writel(pcie, shifted_val, PAB_INTP_AMBA_MISC_ENB);
> > > -	raw_spin_unlock_irqrestore(&pcie->intx_mask_lock, flags);
> > > +	raw_spin_unlock_irqrestore(&rp->intx_mask_lock, flags);
> > >  }
> > >
> > >  static struct irq_chip intx_irq_chip = { @@ -760,7 +777,7 @@ static
> > > int mobiveil_irq_msi_domain_alloc(struct irq_domain *domain,
> > >  					 unsigned int nr_irqs, void *args)  {
> > >  	struct mobiveil_pcie *pcie = domain->host_data;
> > > -	struct mobiveil_msi *msi = &pcie->msi;
> > > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> > >  	unsigned long bit;
> > >
> > >  	WARN_ON(nr_irqs != 1);
> > > @@ -787,7 +804,7 @@ static void mobiveil_irq_msi_domain_free(struct
> > > irq_domain *domain,  {
> > >  	struct irq_data *d = irq_domain_get_irq_data(domain, virq);
> > >  	struct mobiveil_pcie *pcie = irq_data_get_irq_chip_data(d);
> > > -	struct mobiveil_msi *msi = &pcie->msi;
> > > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> > >
> > >  	mutex_lock(&msi->lock);
> > >
> > > @@ -808,9 +825,9 @@ static int mobiveil_allocate_msi_domains(struct
> > > mobiveil_pcie *pcie)  {
> > >  	struct device *dev = &pcie->pdev->dev;
> > >  	struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
> > > -	struct mobiveil_msi *msi = &pcie->msi;
> > > +	struct mobiveil_msi *msi = &pcie->rp.msi;
> > >
> > > -	mutex_init(&pcie->msi.lock);
> > > +	mutex_init(&msi->lock);
> > >  	msi->dev_domain = irq_domain_add_linear(NULL,
> > msi->num_of_vectors,
> > >  						&msi_domain_ops, pcie);
> > >  	if (!msi->dev_domain) {
> > > @@ -834,18 +851,19 @@ static int mobiveil_pcie_init_irq_domain(struct
> > > mobiveil_pcie *pcie)  {
> > >  	struct device *dev = &pcie->pdev->dev;
> > >  	struct device_node *node = dev->of_node;
> > > +	struct root_port *rp = &pcie->rp;
> > >  	int ret;
> > >
> > >  	/* setup INTx */
> > > -	pcie->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> > > -						  &intx_domain_ops, pcie);
> > > +	rp->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
> > > +						&intx_domain_ops, pcie);
> > >
> > > -	if (!pcie->intx_domain) {
> > > +	if (!rp->intx_domain) {
> > >  		dev_err(dev, "Failed to get a INTx IRQ domain\n");
> > >  		return -ENOMEM;
> > >  	}
> > >
> > > -	raw_spin_lock_init(&pcie->intx_mask_lock);
> > > +	raw_spin_lock_init(&rp->intx_mask_lock);
> > >
> > >  	/* setup MSI */
> > >  	ret = mobiveil_allocate_msi_domains(pcie);
> > > @@ -862,6 +880,7 @@ static int mobiveil_pcie_probe(struct
> > platform_device *pdev)
> > >  	struct pci_bus *child;
> > >  	struct pci_host_bridge *bridge;
> > >  	struct device *dev = &pdev->dev;
> > > +	struct root_port *rp;
> > >  	int ret;
> > >
> > >  	/* allocate the PCIe port */
> > > @@ -870,6 +889,8 @@ static int mobiveil_pcie_probe(struct
> > platform_device *pdev)
> > >  		return -ENOMEM;
> > >
> > >  	pcie = pci_host_bridge_priv(bridge);
> > > +	rp = &pcie->rp;
> > > +	rp->bridge = bridge;
> > >
> > >  	pcie->pdev = pdev;
> > >
> > > @@ -904,12 +925,12 @@ static int mobiveil_pcie_probe(struct
> > platform_device *pdev)
> > >  		return ret;
> > >  	}
> > >
> > > -	irq_set_chained_handler_and_data(pcie->irq, mobiveil_pcie_isr, pcie);
> > > +	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> > >
> > >  	/* Initialize bridge */
> > >  	bridge->dev.parent = dev;
> > >  	bridge->sysdata = pcie;
> > > -	bridge->busnr = pcie->root_bus_nr;
> > > +	bridge->busnr = rp->root_bus_nr;
> > >  	bridge->ops = &mobiveil_pcie_ops;
> > >  	bridge->map_irq = of_irq_parse_and_map_pci;
> > >  	bridge->swizzle_irq = pci_common_swizzle;
> > > --
> > > 2.17.1
> > >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations into a routine
  2020-01-13 10:34   ` Andrew Murray
@ 2020-02-06 11:30     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 11:30 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 18:35
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related
> operations into a routine
> 
> On Wed, Nov 20, 2019 at 03:45:37AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Collect the interrupt initialization related operations into a new
> > routine to make it more readable.
> 
> I prefer the word 'function' instead of routine. Also indicate why, not only is it
> nicer but it is in preparation for EP support.

Will replace the 'routine' with 'function', it is only used in RC mode, not used in EP mode.

> 
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> >
> >  drivers/pci/controller/pcie-mobiveil.c | 65
> > +++++++++++++++++---------
> >  1 file changed, 42 insertions(+), 23 deletions(-)
> >
> > diff --git a/drivers/pci/controller/pcie-mobiveil.c
> > b/drivers/pci/controller/pcie-mobiveil.c
> > index 97f682ca7c7a..512b27a0536e 100644
> > --- a/drivers/pci/controller/pcie-mobiveil.c
> > +++ b/drivers/pci/controller/pcie-mobiveil.c
> > @@ -454,12 +454,6 @@ static int mobiveil_pcie_parse_dt(struct
> mobiveil_pcie *pcie)
> >  		return PTR_ERR(pcie->csr_axi_slave_base);
> >  	pcie->pcie_reg_base = res->start;
> >
> > -	/* map MSI config resource */
> > -	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> "apb_csr");
> > -	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
> > -	if (IS_ERR(pcie->apb_csr_base))
> > -		return PTR_ERR(pcie->apb_csr_base);
> > -
> >  	/* read the number of windows requested */
> >  	if (of_property_read_u32(node, "apio-wins", &pcie->apio_wins))
> >  		pcie->apio_wins = MAX_PIO_WINDOWS;
> > @@ -467,12 +461,6 @@ static int mobiveil_pcie_parse_dt(struct
> mobiveil_pcie *pcie)
> >  	if (of_property_read_u32(node, "ppio-wins", &pcie->ppio_wins))
> >  		pcie->ppio_wins = MAX_PIO_WINDOWS;
> >
> > -	rp->irq = platform_get_irq(pdev, 0);
> > -	if (rp->irq <= 0) {
> > -		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> > -		return -ENODEV;
> > -	}
> > -
> >  	return 0;
> >  }
> >
> > @@ -618,9 +606,6 @@ static int mobiveil_host_init(struct mobiveil_pcie
> *pcie)
> >  	pab_ctrl |= (1 << AMBA_PIO_ENABLE_SHIFT) | (1 <<
> PEX_PIO_ENABLE_SHIFT);
> >  	mobiveil_csr_writel(pcie, pab_ctrl, PAB_CTRL);
> >
> > -	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK |
> PAB_INTP_MSI_MASK),
> > -			    PAB_INTP_AMBA_MISC_ENB);
> > -
> >  	/*
> >  	 * program PIO Enable Bit to 1 and Config Window Enable Bit to 1 in
> >  	 * PAB_AXI_PIO_CTRL Register
> > @@ -670,9 +655,6 @@ static int mobiveil_host_init(struct mobiveil_pcie
> *pcie)
> >  	value |= (PCI_CLASS_BRIDGE_PCI << 16);
> >  	mobiveil_csr_writel(pcie, value, PAB_INTP_AXI_PIO_CLASS);
> >
> > -	/* setup MSI hardware registers */
> > -	mobiveil_pcie_enable_msi(pcie);
> > -
> >  	return 0;
> >  }
> >
> > @@ -873,6 +855,46 @@ static int mobiveil_pcie_init_irq_domain(struct
> mobiveil_pcie *pcie)
> >  	return 0;
> >  }
> >
> > +static int mobiveil_pcie_interrupt_init(struct mobiveil_pcie *pcie) {
> > +	struct platform_device *pdev = pcie->pdev;
> > +	struct device *dev = &pdev->dev;
> > +	struct root_port *rp = &pcie->rp;
> > +	struct resource *res;
> > +	int ret;
> > +
> > +	/* map MSI config resource */
> > +	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> "apb_csr");
> > +	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res);
> > +	if (IS_ERR(pcie->apb_csr_base))
> > +		return PTR_ERR(pcie->apb_csr_base);
> > +
> > +	/* setup MSI hardware registers */
> > +	mobiveil_pcie_enable_msi(pcie);
> 
> Does this need to come after mobiveil_pcie_init_irq_domain - given that this
> function sets up the irq domain for MSI?

No, I don't think so, because I didn't change the relative order of the 2 functions
you mentioned.

Thanks,
Zhiqiang

> 
> Thanks,
> 
> Andrew Murray
> 
> > +
> > +	rp->irq = platform_get_irq(pdev, 0);
> > +	if (rp->irq <= 0) {
> > +		dev_err(dev, "failed to map IRQ: %d\n", rp->irq);
> > +		return -ENODEV;
> > +	}
> > +
> > +	/* initialize the IRQ domains */
> > +	ret = mobiveil_pcie_init_irq_domain(pcie);
> > +	if (ret) {
> > +		dev_err(dev, "Failed creating IRQ Domain\n");
> > +		return ret;
> > +	}
> > +
> > +	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> > +
> > +	/* Enable interrupts */
> > +	mobiveil_csr_writel(pcie, (PAB_INTP_INTX_MASK |
> PAB_INTP_MSI_MASK),
> > +			    PAB_INTP_AMBA_MISC_ENB);
> > +
> > +
> > +	return 0;
> > +}
> > +
> >  int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)  {
> >  	struct root_port *rp = &pcie->rp;
> > @@ -906,15 +928,12 @@ int mobiveil_pcie_host_probe(struct
> mobiveil_pcie *pcie)
> >  		return ret;
> >  	}
> >
> > -	/* initialize the IRQ domains */
> > -	ret = mobiveil_pcie_init_irq_domain(pcie);
> > +	ret = mobiveil_pcie_interrupt_init(pcie);
> >  	if (ret) {
> > -		dev_err(dev, "Failed creating IRQ Domain\n");
> > +		dev_err(dev, "Interrupt init failed\n");
> >  		return ret;
> >  	}
> >
> > -	irq_set_chained_handler_and_data(rp->irq, mobiveil_pcie_isr, pcie);
> > -
> >  	/* Initialize bridge */
> >  	bridge->dev.parent = dev;
> >  	bridge->sysdata = pcie;
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver
  2020-01-13 11:05   ` Andrew Murray
@ 2020-02-06 12:25     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 12:25 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your review!

B.R,
Zhiqiang

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 19:06
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe
> Host Bridge IP driver
> 
> On Wed, Nov 20, 2019 at 03:45:43AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Modularize the Mobiveil PCIe host driver according to the abstraction
> > of Root Complex and Endpoint and move it into a new directory.
> >
> > Split the RC related routines into pcie-mobiveil-host.c, and common
> > routines into pcie-mobiveil.c, move the macro definitions and function
> > declarations into pcie-mobiveil.h, and the Mobiveil platform reference
> > code into pcie-mobiveil-plat.c. So that it is easy to reuse the
> > extracted routines to add a new host driver, which integrated Mobiveil
> > PCIe GPEX IP.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Reviewed-by: Andrew Murray <andrew.murray@arm.com>
> 
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> >
> >  MAINTAINERS                                   |   2 +-
> >  drivers/pci/controller/Kconfig                |  11 +-
> >  drivers/pci/controller/Makefile               |   2 +-
> >  drivers/pci/controller/mobiveil/Kconfig       |  24 ++
> >  drivers/pci/controller/mobiveil/Makefile      |   4 +
> >  .../pcie-mobiveil-host.c}                     | 398 +-----------------
> >  .../controller/mobiveil/pcie-mobiveil-plat.c  |  60 +++
> >  .../pci/controller/mobiveil/pcie-mobiveil.c   | 227 ++++++++++
> >  .../pci/controller/mobiveil/pcie-mobiveil.h   | 178 ++++++++
> >  9 files changed, 497 insertions(+), 409 deletions(-)  create mode
> > 100644 drivers/pci/controller/mobiveil/Kconfig
> >  create mode 100644 drivers/pci/controller/mobiveil/Makefile
> >  rename drivers/pci/controller/{pcie-mobiveil.c =>
> > mobiveil/pcie-mobiveil-host.c} (61%)  create mode 100644
> > drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> >  create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.c
> >  create mode 100644 drivers/pci/controller/mobiveil/pcie-mobiveil.h
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS index
> > 3f7f8cdbc471..a4ad99619e53 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -12612,7 +12612,7 @@ M:	Hou Zhiqiang
> <Zhiqiang.Hou@nxp.com>
> >  L:	linux-pci@vger.kernel.org
> >  S:	Supported
> >  F:	Documentation/devicetree/bindings/pci/mobiveil-pcie.txt
> > -F:	drivers/pci/controller/pcie-mobiveil.c
> > +F:	drivers/pci/controller/mobiveil/pcie-mobiveil*
> >
> >  PCI DRIVER FOR MVEBU (Marvell Armada 370 and Armada XP SOC
> support)
> >  M:	Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > diff --git a/drivers/pci/controller/Kconfig
> > b/drivers/pci/controller/Kconfig index f5de9119e8d3..74fd332755ae
> > 100644
> > --- a/drivers/pci/controller/Kconfig
> > +++ b/drivers/pci/controller/Kconfig
> > @@ -241,16 +241,6 @@ config PCIE_MEDIATEK
> >  	  Say Y here if you want to enable PCIe controller support on
> >  	  MediaTek SoCs.
> >
> > -config PCIE_MOBIVEIL
> > -	bool "Mobiveil AXI PCIe controller"
> > -	depends on ARCH_ZYNQMP || COMPILE_TEST
> > -	depends on OF
> > -	depends on PCI_MSI_IRQ_DOMAIN
> > -	help
> > -	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
> > -	  Soft IP. It has up to 8 outbound and inbound windows
> > -	  for address translation and it is a PCIe Gen4 IP.
> > -
> >  config PCIE_TANGO_SMP8759
> >  	bool "Tango SMP8759 PCIe controller (DANGEROUS)"
> >  	depends on ARCH_TANGO && PCI_MSI && OF @@ -289,4 +279,5 @@
> config
> > PCI_HYPERV_INTERFACE
> >  	  have a common interface with the Hyper-V PCI frontend driver.
> >
> >  source "drivers/pci/controller/dwc/Kconfig"
> > +source "drivers/pci/controller/mobiveil/Kconfig"
> >  endmenu
> > diff --git a/drivers/pci/controller/Makefile
> > b/drivers/pci/controller/Makefile index a2a22c9d91af..44414cfd45ea
> > 100644
> > --- a/drivers/pci/controller/Makefile
> > +++ b/drivers/pci/controller/Makefile
> > @@ -27,11 +27,11 @@ obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
> >  obj-$(CONFIG_PCIE_ROCKCHIP_EP) += pcie-rockchip-ep.o
> >  obj-$(CONFIG_PCIE_ROCKCHIP_HOST) += pcie-rockchip-host.o
> >  obj-$(CONFIG_PCIE_MEDIATEK) += pcie-mediatek.o
> > -obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
> >  obj-$(CONFIG_PCIE_TANGO_SMP8759) += pcie-tango.o
> >  obj-$(CONFIG_VMD) += vmd.o
> >  # pcie-hisi.o quirks are needed even without CONFIG_PCIE_DW
> >  obj-y				+= dwc/
> > +obj-y				+= mobiveil/
> >
> >
> >  # The following drivers are for devices that use the generic ACPI
> > diff --git a/drivers/pci/controller/mobiveil/Kconfig
> > b/drivers/pci/controller/mobiveil/Kconfig
> > new file mode 100644
> > index 000000000000..64343c07bfed
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/Kconfig
> > @@ -0,0 +1,24 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +menu "Mobiveil PCIe Core Support"
> > +	depends on PCI
> > +
> > +config PCIE_MOBIVEIL
> > +	bool
> > +
> > +config PCIE_MOBIVEIL_HOST
> > +	bool
> > +	depends on PCI_MSI_IRQ_DOMAIN
> > +	select PCIE_MOBIVEIL
> > +
> > +config PCIE_MOBIVEIL_PLAT
> > +	bool "Mobiveil AXI PCIe controller"
> > +	depends on ARCH_ZYNQMP || COMPILE_TEST
> > +	depends on OF
> > +	select PCIE_MOBIVEIL_HOST
> > +	help
> > +	  Say Y here if you want to enable support for the Mobiveil AXI PCIe
> > +	  Soft IP. It has up to 8 outbound and inbound windows
> > +	  for address translation and it is a PCIe Gen4 IP.
> > +
> > +endmenu
> > diff --git a/drivers/pci/controller/mobiveil/Makefile
> > b/drivers/pci/controller/mobiveil/Makefile
> > new file mode 100644
> > index 000000000000..9fb6d1c6504d
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/Makefile
> > @@ -0,0 +1,4 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
> > +obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
> > +obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
> > diff --git a/drivers/pci/controller/pcie-mobiveil.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > similarity index 61%
> > rename from drivers/pci/controller/pcie-mobiveil.c
> > rename to drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > index 512b27a0536e..2cc424e78d33 100644
> > --- a/drivers/pci/controller/pcie-mobiveil.c
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > @@ -9,7 +9,6 @@
> >   * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >   */
> >
> > -#include <linux/delay.h>
> >  #include <linux/init.h>
> >  #include <linux/interrupt.h>
> >  #include <linux/irq.h>
> > @@ -26,265 +25,7 @@
> >  #include <linux/platform_device.h>
> >  #include <linux/slab.h>
> >
> > -#include "../pci.h"
> > -
> > -/* register offsets and bit positions */
> > -
> > -/*
> > - * translation tables are grouped into windows, each window registers
> > are
> > - * grouped into blocks of 4 or 16 registers each
> > - */
> > -#define PAB_REG_BLOCK_SIZE		16
> > -#define PAB_EXT_REG_BLOCK_SIZE		4
> > -
> > -#define PAB_REG_ADDR(offset, win)	\
> > -	(offset + (win * PAB_REG_BLOCK_SIZE))
> > -#define PAB_EXT_REG_ADDR(offset, win)	\
> > -	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
> > -
> > -#define LTSSM_STATUS			0x0404
> > -#define  LTSSM_STATUS_L0_MASK		0x3f
> > -#define  LTSSM_STATUS_L0		0x2d
> > -
> > -#define PAB_CTRL			0x0808
> > -#define  AMBA_PIO_ENABLE_SHIFT		0
> > -#define  PEX_PIO_ENABLE_SHIFT		1
> > -#define  PAGE_SEL_SHIFT			13
> > -#define  PAGE_SEL_MASK			0x3f
> > -#define  PAGE_LO_MASK			0x3ff
> > -#define  PAGE_SEL_OFFSET_SHIFT		10
> > -
> > -#define PAB_AXI_PIO_CTRL		0x0840
> > -#define  APIO_EN_MASK			0xf
> > -
> > -#define PAB_PEX_PIO_CTRL		0x08c0
> > -#define  PIO_ENABLE_SHIFT		0
> > -
> > -#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> > -#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> > -#define  PAB_INTP_INTX_MASK		0x01e0
> > -#define  PAB_INTP_MSI_MASK		0x8
> > -
> > -#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
> > -#define  WIN_ENABLE_SHIFT		0
> > -#define  WIN_TYPE_SHIFT			1
> > -#define  WIN_TYPE_MASK			0x3
> > -#define  WIN_SIZE_MASK			0xfffffc00
> > -
> > -#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0,
> win)
> > -
> > -#define PAB_EXT_AXI_AMAP_AXI_WIN(win)
> 	PAB_EXT_REG_ADDR(0x80a0, win)
> > -#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
> > -#define  AXI_WINDOW_ALIGN_MASK		3
> > -
> > -#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8,
> win)
> > -#define  PAB_BUS_SHIFT			24
> > -#define  PAB_DEVICE_SHIFT		19
> > -#define  PAB_FUNCTION_SHIFT		16
> > -
> > -#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac,
> win)
> > -#define PAB_INTP_AXI_PIO_CLASS		0x474
> > -
> > -#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0, win)
> > -#define  AMAP_CTRL_EN_SHIFT		0
> > -#define  AMAP_CTRL_TYPE_SHIFT		1
> > -#define  AMAP_CTRL_TYPE_MASK		3
> > -
> > -#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0,
> win)
> > -#define PAB_EXT_PEX_AMAP_AXI_WIN(win)
> 	PAB_EXT_REG_ADDR(0xb4a0, win)
> > -#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
> > -#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8,
> win)
> > -#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac,
> win)
> > -
> > -/* starting offset of INTX bits in status register */
> > -#define PAB_INTX_START			5
> > -
> > -/* supported number of MSI interrupts */
> > -#define PCI_NUM_MSI			16
> > -
> > -/* MSI registers */
> > -#define MSI_BASE_LO_OFFSET		0x04
> > -#define MSI_BASE_HI_OFFSET		0x08
> > -#define MSI_SIZE_OFFSET			0x0c
> > -#define MSI_ENABLE_OFFSET		0x14
> > -#define MSI_STATUS_OFFSET		0x18
> > -#define MSI_DATA_OFFSET			0x20
> > -#define MSI_ADDR_L_OFFSET		0x24
> > -#define MSI_ADDR_H_OFFSET		0x28
> > -
> > -/* outbound and inbound window definitions */
> > -#define WIN_NUM_0			0
> > -#define WIN_NUM_1			1
> > -#define CFG_WINDOW_TYPE			0
> > -#define IO_WINDOW_TYPE			1
> > -#define MEM_WINDOW_TYPE			2
> > -#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
> > -#define MAX_PIO_WINDOWS			8
> > -
> > -/* Parameters for the waiting for link up routine */
> > -#define LINK_WAIT_MAX_RETRIES		10
> > -#define LINK_WAIT_MIN			90000
> > -#define LINK_WAIT_MAX			100000
> > -
> > -#define PAGED_ADDR_BNDRY		0xc00
> > -#define OFFSET_TO_PAGE_ADDR(off)	\
> > -	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
> > -#define OFFSET_TO_PAGE_IDX(off)		\
> > -	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
> > -
> > -struct mobiveil_msi {			/* MSI information */
> > -	struct mutex lock;		/* protect bitmap variable */
> > -	struct irq_domain *msi_domain;
> > -	struct irq_domain *dev_domain;
> > -	phys_addr_t msi_pages_phys;
> > -	int num_of_vectors;
> > -	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);
> > -};
> > -
> > -struct root_port {
> > -	char root_bus_nr;
> > -	void __iomem *config_axi_slave_base;	/* endpoint config base */
> > -	struct resource *ob_io_res;
> > -	int irq;
> > -	raw_spinlock_t intx_mask_lock;
> > -	struct irq_domain *intx_domain;
> > -	struct mobiveil_msi msi;
> > -	struct pci_host_bridge *bridge;
> > -};
> > -
> > -struct mobiveil_pcie {
> > -	struct platform_device *pdev;
> > -	void __iomem *csr_axi_slave_base;	/* root port config base */
> > -	void __iomem *apb_csr_base;	/* MSI register base */
> > -	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> > -	int apio_wins;
> > -	int ppio_wins;
> > -	int ob_wins_configured;		/* configured outbound windows */
> > -	int ib_wins_configured;		/* configured inbound windows */
> > -	struct root_port rp;
> > -};
> > -
> > -/*
> > - * mobiveil_pcie_sel_page - routine to access paged register
> > - *
> > - * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are
> > paged,
> > - * for this scheme to work extracted higher 6 bits of the offset will
> > be
> > - * written to pg_sel field of PAB_CTRL register and rest of the lower
> > 10
> > - * bits enabled with PAGED_ADDR_BNDRY are used as offset of the
> register.
> > - */
> > -static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8
> > pg_idx) -{
> > -	u32 val;
> > -
> > -	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
> > -	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
> > -	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
> > -
> > -	writel(val, pcie->csr_axi_slave_base + PAB_CTRL);
> > -}
> > -
> > -static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32
> > off) -{
> > -	if (off < PAGED_ADDR_BNDRY) {
> > -		/* For directly accessed registers, clear the pg_sel field */
> > -		mobiveil_pcie_sel_page(pcie, 0);
> > -		return pcie->csr_axi_slave_base + off;
> > -	}
> > -
> > -	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
> > -	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off);
> > -}
> > -
> > -static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
> > -{
> > -	if ((uintptr_t)addr & (size - 1)) {
> > -		*val = 0;
> > -		return PCIBIOS_BAD_REGISTER_NUMBER;
> > -	}
> > -
> > -	switch (size) {
> > -	case 4:
> > -		*val = readl(addr);
> > -		break;
> > -	case 2:
> > -		*val = readw(addr);
> > -		break;
> > -	case 1:
> > -		*val = readb(addr);
> > -		break;
> > -	default:
> > -		*val = 0;
> > -		return PCIBIOS_BAD_REGISTER_NUMBER;
> > -	}
> > -
> > -	return PCIBIOS_SUCCESSFUL;
> > -}
> > -
> > -static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
> > -{
> > -	if ((uintptr_t)addr & (size - 1))
> > -		return PCIBIOS_BAD_REGISTER_NUMBER;
> > -
> > -	switch (size) {
> > -	case 4:
> > -		writel(val, addr);
> > -		break;
> > -	case 2:
> > -		writew(val, addr);
> > -		break;
> > -	case 1:
> > -		writeb(val, addr);
> > -		break;
> > -	default:
> > -		return PCIBIOS_BAD_REGISTER_NUMBER;
> > -	}
> > -
> > -	return PCIBIOS_SUCCESSFUL;
> > -}
> > -
> > -static u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off,
> > size_t size) -{
> > -	void *addr;
> > -	u32 val;
> > -	int ret;
> > -
> > -	addr = mobiveil_pcie_comp_addr(pcie, off);
> > -
> > -	ret = mobiveil_pcie_read(addr, size, &val);
> > -	if (ret)
> > -		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
> > -
> > -	return val;
> > -}
> > -
> > -static void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> > -			       size_t size)
> > -{
> > -	void *addr;
> > -	int ret;
> > -
> > -	addr = mobiveil_pcie_comp_addr(pcie, off);
> > -
> > -	ret = mobiveil_pcie_write(addr, size, val);
> > -	if (ret)
> > -		dev_err(&pcie->pdev->dev, "write CSR address failed\n");
> > -}
> > -
> > -static u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32 off) -{
> > -	return mobiveil_csr_read(pcie, off, 0x4);
> > -}
> > -
> > -static void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
> > u32 off) -{
> > -	mobiveil_csr_write(pcie, val, off, 0x4);
> > -}
> > -
> > -static bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie) -{
> > -	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
> > -		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
> > -}
> > +#include "pcie-mobiveil.h"
> >
> >  static bool mobiveil_pcie_valid_device(struct pci_bus *bus, unsigned
> > int devfn)  { @@ -464,103 +205,6 @@ static int
> > mobiveil_pcie_parse_dt(struct mobiveil_pcie *pcie)
> >  	return 0;
> >  }
> >
> > -static void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
> > -			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> > -{
> > -	u32 value;
> > -	u64 size64 = ~(size - 1);
> > -
> > -	if (win_num >= pcie->ppio_wins) {
> > -		dev_err(&pcie->pdev->dev,
> > -			"ERROR: max inbound windows reached !\n");
> > -		return;
> > -	}
> > -
> > -	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
> > -	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT |
> WIN_SIZE_MASK);
> > -	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 <<
> AMAP_CTRL_EN_SHIFT |
> > -		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> > -	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
> > -
> > -	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> > -			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
> > -
> > -	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
> > -			    PAB_PEX_AMAP_AXI_WIN(win_num));
> > -	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> > -			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
> > -
> > -	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> > -			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
> > -	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> > -			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
> > -
> > -	pcie->ib_wins_configured++;
> > -}
> > -
> > -/*
> > - * routine to program the outbound windows
> > - */
> > -static void program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
> > -			       u64 cpu_addr, u64 pci_addr, u32 type, u64 size)
> > -{
> > -	u32 value;
> > -	u64 size64 = ~(size - 1);
> > -
> > -	if (win_num >= pcie->apio_wins) {
> > -		dev_err(&pcie->pdev->dev,
> > -			"ERROR: max outbound windows reached !\n");
> > -		return;
> > -	}
> > -
> > -	/*
> > -	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
> > -	 * to 4 KB in PAB_AXI_AMAP_CTRL register
> > -	 */
> > -	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
> > -	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
> > -	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
> > -		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> > -	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
> > -
> > -	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> > -			    PAB_EXT_AXI_AMAP_SIZE(win_num));
> > -
> > -	/*
> > -	 * program AXI window base with appropriate value in
> > -	 * PAB_AXI_AMAP_AXI_WIN0 register
> > -	 */
> > -	mobiveil_csr_writel(pcie,
> > -			    lower_32_bits(cpu_addr) &
> (~AXI_WINDOW_ALIGN_MASK),
> > -			    PAB_AXI_AMAP_AXI_WIN(win_num));
> > -	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> > -			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
> > -
> > -	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> > -			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
> > -	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> > -			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
> > -
> > -	pcie->ob_wins_configured++;
> > -}
> > -
> > -static int mobiveil_bringup_link(struct mobiveil_pcie *pcie) -{
> > -	int retries;
> > -
> > -	/* check if the link is up or not */
> > -	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
> > -		if (mobiveil_pcie_link_up(pcie))
> > -			return 0;
> > -
> > -		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
> > -	}
> > -
> > -	dev_err(&pcie->pdev->dev, "link never came up\n");
> > -
> > -	return -ETIMEDOUT;
> > -}
> > -
> >  static void mobiveil_pcie_enable_msi(struct mobiveil_pcie *pcie)  {
> >  	phys_addr_t msg_addr = pcie->pcie_reg_base; @@ -962,43 +606,3 @@
> int
> > mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie)
> >
> >  	return 0;
> >  }
> > -
> > -static int mobiveil_pcie_probe(struct platform_device *pdev) -{
> > -	struct mobiveil_pcie *pcie;
> > -	struct pci_host_bridge *bridge;
> > -	struct device *dev = &pdev->dev;
> > -
> > -	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> > -	if (!bridge)
> > -		return -ENOMEM;
> > -
> > -	pcie = pci_host_bridge_priv(bridge);
> > -	pcie->rp.bridge = bridge;
> > -
> > -	pcie->pdev = pdev;
> > -
> > -	return mobiveil_pcie_host_probe(pcie);
> > -}
> > -
> > -static const struct of_device_id mobiveil_pcie_of_match[] = {
> > -	{.compatible = "mbvl,gpex40-pcie",},
> > -	{},
> > -};
> > -
> > -MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
> > -
> > -static struct platform_driver mobiveil_pcie_driver = {
> > -	.probe = mobiveil_pcie_probe,
> > -	.driver = {
> > -		.name = "mobiveil-pcie",
> > -		.of_match_table = mobiveil_pcie_of_match,
> > -		.suppress_bind_attrs = true,
> > -	},
> > -};
> > -
> > -builtin_platform_driver(mobiveil_pcie_driver);
> > -
> > -MODULE_LICENSE("GPL v2");
> > -MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
> > -MODULE_AUTHOR("Subrahmanya Lingappa
> <l.subrahmanya@mobiveil.co.in>");
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> > new file mode 100644
> > index 000000000000..64c85f852869
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-plat.c
> > @@ -0,0 +1,60 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * PCIe host controller driver for Mobiveil PCIe Host controller
> > + *
> > + * Copyright (c) 2018 Mobiveil Inc.
> > + * Copyright 2019 NXP
> > + *
> > + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> 
> Same feedback as before with the recode tag.
> 
> > + */
> > +
> > +#include <linux/init.h>
> > +#include <linux/kernel.h>
> > +#include <linux/module.h>
> > +#include <linux/of_pci.h>
> > +#include <linux/pci.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/slab.h>
> > +
> > +#include "pcie-mobiveil.h"
> > +
> > +static int mobiveil_pcie_probe(struct platform_device *pdev) {
> > +	struct mobiveil_pcie *pcie;
> > +	struct pci_host_bridge *bridge;
> > +	struct device *dev = &pdev->dev;
> > +
> > +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> > +	if (!bridge)
> > +		return -ENOMEM;
> > +
> > +	pcie = pci_host_bridge_priv(bridge);
> > +	pcie->rp.bridge = bridge;
> > +
> > +	pcie->pdev = pdev;
> > +
> > +	return mobiveil_pcie_host_probe(pcie); }
> > +
> > +static const struct of_device_id mobiveil_pcie_of_match[] = {
> > +	{.compatible = "mbvl,gpex40-pcie",},
> > +	{},
> > +};
> > +
> > +MODULE_DEVICE_TABLE(of, mobiveil_pcie_of_match);
> > +
> > +static struct platform_driver mobiveil_pcie_driver = {
> > +	.probe = mobiveil_pcie_probe,
> > +	.driver = {
> > +		.name = "mobiveil-pcie",
> > +		.of_match_table = mobiveil_pcie_of_match,
> > +		.suppress_bind_attrs = true,
> > +	},
> > +};
> > +
> > +builtin_platform_driver(mobiveil_pcie_driver);
> > +
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_DESCRIPTION("Mobiveil PCIe host controller driver");
> > +MODULE_AUTHOR("Subrahmanya Lingappa
> <l.subrahmanya@mobiveil.co.in>");
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > new file mode 100644
> > index 000000000000..2773f823c9ea
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > @@ -0,0 +1,227 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * PCIe host controller driver for Mobiveil PCIe Host controller
> > + *
> > + * Copyright (c) 2018 Mobiveil Inc.
> > + * Copyright 2019 NXP
> > + *
> > + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>  */
> > +
> > +#include <linux/delay.h>
> > +#include <linux/init.h>
> > +#include <linux/kernel.h>
> > +#include <linux/pci.h>
> > +#include <linux/platform_device.h>
> > +
> > +#include "pcie-mobiveil.h"
> > +
> > +/*
> > + * mobiveil_pcie_sel_page - routine to access paged register
> > + *
> > + * Registers whose address greater than PAGED_ADDR_BNDRY (0xc00) are
> > +paged,
> > + * for this scheme to work extracted higher 6 bits of the offset will
> > +be
> > + * written to pg_sel field of PAB_CTRL register and rest of the lower
> > +10
> > + * bits enabled with PAGED_ADDR_BNDRY are used as offset of the
> register.
> > + */
> > +static void mobiveil_pcie_sel_page(struct mobiveil_pcie *pcie, u8
> > +pg_idx) {
> > +	u32 val;
> > +
> > +	val = readl(pcie->csr_axi_slave_base + PAB_CTRL);
> > +	val &= ~(PAGE_SEL_MASK << PAGE_SEL_SHIFT);
> > +	val |= (pg_idx & PAGE_SEL_MASK) << PAGE_SEL_SHIFT;
> > +
> > +	writel(val, pcie->csr_axi_slave_base + PAB_CTRL); }
> > +
> > +static void *mobiveil_pcie_comp_addr(struct mobiveil_pcie *pcie, u32
> > +off) {
> > +	if (off < PAGED_ADDR_BNDRY) {
> > +		/* For directly accessed registers, clear the pg_sel field */
> > +		mobiveil_pcie_sel_page(pcie, 0);
> > +		return pcie->csr_axi_slave_base + off;
> > +	}
> > +
> > +	mobiveil_pcie_sel_page(pcie, OFFSET_TO_PAGE_IDX(off));
> > +	return pcie->csr_axi_slave_base + OFFSET_TO_PAGE_ADDR(off); }
> > +
> > +static int mobiveil_pcie_read(void __iomem *addr, int size, u32 *val)
> > +{
> > +	if ((uintptr_t)addr & (size - 1)) {
> > +		*val = 0;
> > +		return PCIBIOS_BAD_REGISTER_NUMBER;
> > +	}
> > +
> > +	switch (size) {
> > +	case 4:
> > +		*val = readl(addr);
> > +		break;
> > +	case 2:
> > +		*val = readw(addr);
> > +		break;
> > +	case 1:
> > +		*val = readb(addr);
> > +		break;
> > +	default:
> > +		*val = 0;
> > +		return PCIBIOS_BAD_REGISTER_NUMBER;
> > +	}
> > +
> > +	return PCIBIOS_SUCCESSFUL;
> > +}
> > +
> > +static int mobiveil_pcie_write(void __iomem *addr, int size, u32 val)
> > +{
> > +	if ((uintptr_t)addr & (size - 1))
> > +		return PCIBIOS_BAD_REGISTER_NUMBER;
> > +
> > +	switch (size) {
> > +	case 4:
> > +		writel(val, addr);
> > +		break;
> > +	case 2:
> > +		writew(val, addr);
> > +		break;
> > +	case 1:
> > +		writeb(val, addr);
> > +		break;
> > +	default:
> > +		return PCIBIOS_BAD_REGISTER_NUMBER;
> > +	}
> > +
> > +	return PCIBIOS_SUCCESSFUL;
> > +}
> > +
> > +u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t
> > +size) {
> > +	void *addr;
> > +	u32 val;
> > +	int ret;
> > +
> > +	addr = mobiveil_pcie_comp_addr(pcie, off);
> > +
> > +	ret = mobiveil_pcie_read(addr, size, &val);
> > +	if (ret)
> > +		dev_err(&pcie->pdev->dev, "read CSR address failed\n");
> > +
> > +	return val;
> > +}
> > +
> > +void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> > +			       size_t size)
> > +{
> > +	void *addr;
> > +	int ret;
> > +
> > +	addr = mobiveil_pcie_comp_addr(pcie, off);
> > +
> > +	ret = mobiveil_pcie_write(addr, size, val);
> > +	if (ret)
> > +		dev_err(&pcie->pdev->dev, "write CSR address failed\n"); }
> > +
> > +bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie) {
> > +	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
> > +		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0; }
> > +
> > +void program_ib_windows(struct mobiveil_pcie *pcie, int win_num,
> > +			u64 cpu_addr, u64 pci_addr, u32 type, u64 size) {
> > +	u32 value;
> > +	u64 size64 = ~(size - 1);
> > +
> > +	if (win_num >= pcie->ppio_wins) {
> > +		dev_err(&pcie->pdev->dev,
> > +			"ERROR: max inbound windows reached !\n");
> > +		return;
> > +	}
> > +
> > +	value = mobiveil_csr_readl(pcie, PAB_PEX_AMAP_CTRL(win_num));
> > +	value &= ~(AMAP_CTRL_TYPE_MASK << AMAP_CTRL_TYPE_SHIFT |
> WIN_SIZE_MASK);
> > +	value |= type << AMAP_CTRL_TYPE_SHIFT | 1 <<
> AMAP_CTRL_EN_SHIFT |
> > +		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> > +	mobiveil_csr_writel(pcie, value, PAB_PEX_AMAP_CTRL(win_num));
> > +
> > +	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> > +			    PAB_EXT_PEX_AMAP_SIZEN(win_num));
> > +
> > +	mobiveil_csr_writel(pcie, lower_32_bits(cpu_addr),
> > +			    PAB_PEX_AMAP_AXI_WIN(win_num));
> > +	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> > +			    PAB_EXT_PEX_AMAP_AXI_WIN(win_num));
> > +
> > +	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> > +			    PAB_PEX_AMAP_PEX_WIN_L(win_num));
> > +	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> > +			    PAB_PEX_AMAP_PEX_WIN_H(win_num));
> > +
> > +	pcie->ib_wins_configured++;
> > +}
> > +
> > +/*
> > + * routine to program the outbound windows  */ void
> > +program_ob_windows(struct mobiveil_pcie *pcie, int win_num,
> > +			u64 cpu_addr, u64 pci_addr, u32 type, u64 size) {
> > +	u32 value;
> > +	u64 size64 = ~(size - 1);
> > +
> > +	if (win_num >= pcie->apio_wins) {
> > +		dev_err(&pcie->pdev->dev,
> > +			"ERROR: max outbound windows reached !\n");
> > +		return;
> > +	}
> > +
> > +	/*
> > +	 * program Enable Bit to 1, Type Bit to (00) base 2, AXI Window Size Bit
> > +	 * to 4 KB in PAB_AXI_AMAP_CTRL register
> > +	 */
> > +	value = mobiveil_csr_readl(pcie, PAB_AXI_AMAP_CTRL(win_num));
> > +	value &= ~(WIN_TYPE_MASK << WIN_TYPE_SHIFT | WIN_SIZE_MASK);
> > +	value |= 1 << WIN_ENABLE_SHIFT | type << WIN_TYPE_SHIFT |
> > +		 (lower_32_bits(size64) & WIN_SIZE_MASK);
> > +	mobiveil_csr_writel(pcie, value, PAB_AXI_AMAP_CTRL(win_num));
> > +
> > +	mobiveil_csr_writel(pcie, upper_32_bits(size64),
> > +			    PAB_EXT_AXI_AMAP_SIZE(win_num));
> > +
> > +	/*
> > +	 * program AXI window base with appropriate value in
> > +	 * PAB_AXI_AMAP_AXI_WIN0 register
> > +	 */
> > +	mobiveil_csr_writel(pcie,
> > +			    lower_32_bits(cpu_addr) &
> (~AXI_WINDOW_ALIGN_MASK),
> > +			    PAB_AXI_AMAP_AXI_WIN(win_num));
> > +	mobiveil_csr_writel(pcie, upper_32_bits(cpu_addr),
> > +			    PAB_EXT_AXI_AMAP_AXI_WIN(win_num));
> > +
> > +	mobiveil_csr_writel(pcie, lower_32_bits(pci_addr),
> > +			    PAB_AXI_AMAP_PEX_WIN_L(win_num));
> > +	mobiveil_csr_writel(pcie, upper_32_bits(pci_addr),
> > +			    PAB_AXI_AMAP_PEX_WIN_H(win_num));
> > +
> > +	pcie->ob_wins_configured++;
> > +}
> > +
> > +int mobiveil_bringup_link(struct mobiveil_pcie *pcie) {
> > +	int retries;
> > +
> > +	/* check if the link is up or not */
> > +	for (retries = 0; retries < LINK_WAIT_MAX_RETRIES; retries++) {
> > +		if (mobiveil_pcie_link_up(pcie))
> > +			return 0;
> > +
> > +		usleep_range(LINK_WAIT_MIN, LINK_WAIT_MAX);
> > +	}
> > +
> > +	dev_err(&pcie->pdev->dev, "link never came up\n");
> > +
> > +	return -ETIMEDOUT;
> > +}
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > new file mode 100644
> > index 000000000000..e3148078e9dd
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -0,0 +1,178 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * PCIe host controller driver for Mobiveil PCIe Host controller
> > + *
> > + * Copyright (c) 2018 Mobiveil Inc.
> > + * Copyright 2019 NXP
> > + *
> > + * Author: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > + * Recode: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>  */
> > +
> > +#ifndef _PCIE_MOBIVEIL_H
> > +#define _PCIE_MOBIVEIL_H
> > +
> > +#include <linux/pci.h>
> > +#include <linux/irq.h>
> > +#include <linux/msi.h>
> > +#include "../../pci.h"
> > +
> > +/* register offsets and bit positions */
> > +
> > +/*
> > + * translation tables are grouped into windows, each window registers
> > +are
> > + * grouped into blocks of 4 or 16 registers each  */
> > +#define PAB_REG_BLOCK_SIZE		16
> > +#define PAB_EXT_REG_BLOCK_SIZE		4
> > +
> > +#define PAB_REG_ADDR(offset, win)	\
> > +	(offset + (win * PAB_REG_BLOCK_SIZE))
> > +#define PAB_EXT_REG_ADDR(offset, win)	\
> > +	(offset + (win * PAB_EXT_REG_BLOCK_SIZE))
> > +
> > +#define LTSSM_STATUS			0x0404
> > +#define  LTSSM_STATUS_L0_MASK		0x3f
> > +#define  LTSSM_STATUS_L0		0x2d
> > +
> > +#define PAB_CTRL			0x0808
> > +#define  AMBA_PIO_ENABLE_SHIFT		0
> > +#define  PEX_PIO_ENABLE_SHIFT		1
> > +#define  PAGE_SEL_SHIFT			13
> > +#define  PAGE_SEL_MASK			0x3f
> > +#define  PAGE_LO_MASK			0x3ff
> > +#define  PAGE_SEL_OFFSET_SHIFT		10
> > +
> > +#define PAB_AXI_PIO_CTRL		0x0840
> > +#define  APIO_EN_MASK			0xf
> > +
> > +#define PAB_PEX_PIO_CTRL		0x08c0
> > +#define  PIO_ENABLE_SHIFT		0
> > +
> > +#define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> > +#define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> > +#define  PAB_INTP_INTX_MASK		0x01e0
> > +#define  PAB_INTP_MSI_MASK		0x8
> > +
> > +#define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0, win)
> > +#define  WIN_ENABLE_SHIFT		0
> > +#define  WIN_TYPE_SHIFT			1
> > +#define  WIN_TYPE_MASK			0x3
> > +#define  WIN_SIZE_MASK			0xfffffc00
> > +
> > +#define PAB_EXT_AXI_AMAP_SIZE(win)	PAB_EXT_REG_ADDR(0xbaf0,
> win)
> > +
> > +#define PAB_EXT_AXI_AMAP_AXI_WIN(win)
> 	PAB_EXT_REG_ADDR(0x80a0, win)
> > +#define PAB_AXI_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x0ba4, win)
> > +#define  AXI_WINDOW_ALIGN_MASK		3
> > +
> > +#define PAB_AXI_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x0ba8,
> win)
> > +#define  PAB_BUS_SHIFT			24
> > +#define  PAB_DEVICE_SHIFT		19
> > +#define  PAB_FUNCTION_SHIFT		16
> > +
> > +#define PAB_AXI_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x0bac,
> win)
> > +#define PAB_INTP_AXI_PIO_CLASS		0x474
> > +
> > +#define PAB_PEX_AMAP_CTRL(win)		PAB_REG_ADDR(0x4ba0,
> win)
> > +#define  AMAP_CTRL_EN_SHIFT		0
> > +#define  AMAP_CTRL_TYPE_SHIFT		1
> > +#define  AMAP_CTRL_TYPE_MASK		3
> > +
> > +#define PAB_EXT_PEX_AMAP_SIZEN(win)	PAB_EXT_REG_ADDR(0xbef0,
> win)
> > +#define PAB_EXT_PEX_AMAP_AXI_WIN(win)
> 	PAB_EXT_REG_ADDR(0xb4a0, win)
> > +#define PAB_PEX_AMAP_AXI_WIN(win)	PAB_REG_ADDR(0x4ba4, win)
> > +#define PAB_PEX_AMAP_PEX_WIN_L(win)	PAB_REG_ADDR(0x4ba8,
> win)
> > +#define PAB_PEX_AMAP_PEX_WIN_H(win)	PAB_REG_ADDR(0x4bac,
> win)
> > +
> > +/* starting offset of INTX bits in status register */
> > +#define PAB_INTX_START			5
> > +
> > +/* supported number of MSI interrupts */
> > +#define PCI_NUM_MSI			16
> > +
> > +/* MSI registers */
> > +#define MSI_BASE_LO_OFFSET		0x04
> > +#define MSI_BASE_HI_OFFSET		0x08
> > +#define MSI_SIZE_OFFSET			0x0c
> > +#define MSI_ENABLE_OFFSET		0x14
> > +#define MSI_STATUS_OFFSET		0x18
> > +#define MSI_DATA_OFFSET			0x20
> > +#define MSI_ADDR_L_OFFSET		0x24
> > +#define MSI_ADDR_H_OFFSET		0x28
> > +
> > +/* outbound and inbound window definitions */
> > +#define WIN_NUM_0			0
> > +#define WIN_NUM_1			1
> > +#define CFG_WINDOW_TYPE			0
> > +#define IO_WINDOW_TYPE			1
> > +#define MEM_WINDOW_TYPE			2
> > +#define IB_WIN_SIZE			((u64)256 * 1024 * 1024 * 1024)
> > +#define MAX_PIO_WINDOWS			8
> > +
> > +/* Parameters for the waiting for link up routine */
> > +#define LINK_WAIT_MAX_RETRIES		10
> > +#define LINK_WAIT_MIN			90000
> > +#define LINK_WAIT_MAX			100000
> > +
> > +#define PAGED_ADDR_BNDRY		0xc00
> > +#define OFFSET_TO_PAGE_ADDR(off)	\
> > +	((off & PAGE_LO_MASK) | PAGED_ADDR_BNDRY)
> > +#define OFFSET_TO_PAGE_IDX(off)		\
> > +	((off >> PAGE_SEL_OFFSET_SHIFT) & PAGE_SEL_MASK)
> > +
> > +struct mobiveil_msi {			/* MSI information */
> > +	struct mutex lock;		/* protect bitmap variable */
> > +	struct irq_domain *msi_domain;
> > +	struct irq_domain *dev_domain;
> > +	phys_addr_t msi_pages_phys;
> > +	int num_of_vectors;
> > +	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI); };
> > +
> > +struct root_port {
> > +	char root_bus_nr;
> > +	void __iomem *config_axi_slave_base;	/* endpoint config base */
> > +	struct resource *ob_io_res;
> > +	int irq;
> > +	raw_spinlock_t intx_mask_lock;
> > +	struct irq_domain *intx_domain;
> > +	struct mobiveil_msi msi;
> > +	struct pci_host_bridge *bridge;
> > +};
> > +
> > +struct mobiveil_pcie {
> > +	struct platform_device *pdev;
> > +	void __iomem *csr_axi_slave_base;	/* root port config base */
> > +	void __iomem *apb_csr_base;	/* MSI register base */
> > +	phys_addr_t pcie_reg_base;	/* Physical PCIe Controller Base */
> > +	int apio_wins;
> > +	int ppio_wins;
> > +	int ob_wins_configured;		/* configured outbound windows */
> > +	int ib_wins_configured;		/* configured inbound windows */
> > +	struct root_port rp;
> > +};
> > +
> > +int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie); bool
> > +mobiveil_pcie_link_up(struct mobiveil_pcie *pcie); int
> > +mobiveil_bringup_link(struct mobiveil_pcie *pcie); void
> > +program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64
> cpu_addr,
> > +			u64 pci_addr, u32 type, u64 size); void
> program_ib_windows(struct
> > +mobiveil_pcie *pcie, int win_num, u64 cpu_addr,
> > +			u64 pci_addr, u32 type, u64 size);
> > +u32 mobiveil_csr_read(struct mobiveil_pcie *pcie, u32 off, size_t
> > +size); void mobiveil_csr_write(struct mobiveil_pcie *pcie, u32 val, u32 off,
> > +			size_t size);
> > +
> > +static inline u32 mobiveil_csr_readl(struct mobiveil_pcie *pcie, u32
> > +off) {
> > +	return mobiveil_csr_read(pcie, off, 0x4); }
> > +
> > +static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
> > +				       u32 off)
> > +{
> > +	mobiveil_csr_write(pcie, val, off, 0x4); }
> > +
> > +#endif /* _PCIE_MOBIVEIL_H */
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization
  2020-01-13 11:19   ` Andrew Murray
@ 2020-02-06 13:25     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 13:25 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 19:20
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 05/12] PCI: mobiveil: Add callback function for
> interrupt initialization
> 
> On Wed, Nov 20, 2019 at 03:45:50AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The Mobiveil GPEX internal MSI/INTx controller may not be used by
> > other platforms in which the Mobiveil GPEX is integrated.
> > This patch is to allow these platforms to implement their specific
> > interrupt initialization.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> >
> >  drivers/pci/controller/mobiveil/pcie-mobiveil-host.c | 3 +++
> >  drivers/pci/controller/mobiveil/pcie-mobiveil.h      | 7 +++++++
> >  2 files changed, 10 insertions(+)
> >
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > index 2cc424e78d33..3cd93df6fe6e 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > @@ -507,6 +507,9 @@ static int mobiveil_pcie_interrupt_init(struct
> mobiveil_pcie *pcie)
> >  	struct resource *res;
> >  	int ret;
> >
> > +	if (rp->ops->interrupt_init)
> > +		return rp->ops->interrupt_init(pcie);
> > +
> 
> This may be cleaner if you have a helper function named
> "mobiveil_pcie_interrupt_init" where it either calls interrupt_init if present or
> calls this current function renamed to
> "mobiveil_pcie_integrated_interrupt_init"
> or similar.
> 
> A bit like the DWC dw_pcie_rd_own_conf function.

Good suggestion! Will change in v10.

Thanks,
Zhiqiang

> 
> Thanks,
> 
> Andrew Murray
> 
> >  	/* map MSI config resource */
> >  	res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> "apb_csr");
> >  	pcie->apb_csr_base = devm_pci_remap_cfg_resource(dev, res); diff
> > --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > index e3148078e9dd..18d85806a7fc 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -130,10 +130,17 @@ struct mobiveil_msi {			/* MSI
> information */
> >  	DECLARE_BITMAP(msi_irq_in_use, PCI_NUM_MSI);  };
> >
> > +struct mobiveil_pcie;
> > +
> > +struct mobiveil_rp_ops {
> > +	int (*interrupt_init)(struct mobiveil_pcie *pcie); };
> > +
> >  struct root_port {
> >  	char root_bus_nr;
> >  	void __iomem *config_axi_slave_base;	/* endpoint config base */
> >  	struct resource *ob_io_res;
> > +	struct mobiveil_rp_ops *ops;
> >  	int irq;
> >  	raw_spinlock_t intx_mask_lock;
> >  	struct irq_domain *intx_domain;
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check
  2020-01-13 11:22   ` Andrew Murray
@ 2020-02-06 13:25     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 13:25 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your review!

B.R,
Zhiqiang

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 19:22
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up
> check
> 
> On Wed, Nov 20, 2019 at 03:45:57AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > The platforms, in which the Mobiveil GPEX is integrated, may have
> > their specific mechanism to check link up status.
> > This patch is to enable these platforms to implement theirs.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > ---
> > V9:
> >  - New patch splited from the #1 of V8 patches to make it easy to review.
> >
> >  drivers/pci/controller/mobiveil/pcie-mobiveil.c | 3 +++
> > drivers/pci/controller/mobiveil/pcie-mobiveil.h | 5 +++++
> >  2 files changed, 8 insertions(+)
> >
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > index 2773f823c9ea..b9ed2d95641c 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.c
> > @@ -125,6 +125,9 @@ void mobiveil_csr_write(struct mobiveil_pcie
> > *pcie, u32 val, u32 off,
> >
> >  bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie)  {
> > +	if (pcie->ops->link_up)
> > +		return pcie->ops->link_up(pcie);
> > +
> >  	return (mobiveil_csr_readl(pcie, LTSSM_STATUS) &
> >  		LTSSM_STATUS_L0_MASK) == LTSSM_STATUS_L0;
> 
> On the previous patch I suggested that we don't mix up the link_up logic with
> the logic that decides which function to call. In this case the link_up logic is
> trivial. So this is probably OK.
> 
> Reviewed-by: Andrew Murray <andrew.murray@arm.com>
> 
> >  }
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > index 18d85806a7fc..95d2e7c809b8 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -148,6 +148,10 @@ struct root_port {
> >  	struct pci_host_bridge *bridge;
> >  };
> >
> > +struct mobiveil_pab_ops {
> > +	int (*link_up)(struct mobiveil_pcie *pcie); };
> > +
> >  struct mobiveil_pcie {
> >  	struct platform_device *pdev;
> >  	void __iomem *csr_axi_slave_base;	/* root port config base */
> > @@ -157,6 +161,7 @@ struct mobiveil_pcie {
> >  	int ppio_wins;
> >  	int ob_wins_configured;		/* configured outbound windows */
> >  	int ib_wins_configured;		/* configured inbound windows */
> > +	const struct mobiveil_pab_ops *ops;
> >  	struct root_port rp;
> >  };
> >
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host
  2020-01-13 11:26   ` Andrew Murray
@ 2020-02-06 13:27     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 13:27 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 19:27
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be
> used to re-init host
> 
> On Wed, Nov 20, 2019 at 03:46:03AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > Make the mobiveil_host_init() function can be used to re-init
> 
> Perhaps reword to "Allow the mobiveil_host_init() function to be used to ...
> 

Thanks a lot for your suggestion, will change in v10.

Thanks,
Zhiqiang

> > host controller's PAB and GPEX CSR register block, as NXP integrated
> > Mobiveil IP has to reset and then re-init the PAB and GPEX CSR
> > registers upon hot-reset.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > ---
> > V9:
> >  - No change
> >
> >  .../controller/mobiveil/pcie-mobiveil-host.c  | 19 ++++++++++++-------
> >  .../pci/controller/mobiveil/pcie-mobiveil.h   |  1 +
> >  2 files changed, 13 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > index 3cd93df6fe6e..9bc3da036720 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
> > @@ -221,18 +221,23 @@ static void mobiveil_pcie_enable_msi(struct
> mobiveil_pcie *pcie)
> >  	writel_relaxed(1, pcie->apb_csr_base + MSI_ENABLE_OFFSET);  }
> >
> > -static int mobiveil_host_init(struct mobiveil_pcie *pcie)
> > +int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit)
> >  {
> >  	struct root_port *rp = &pcie->rp;
> >  	struct pci_host_bridge *bridge = rp->bridge;
> >  	u32 value, pab_ctrl, type;
> >  	struct resource_entry *win;
> >
> > -	/* setup bus numbers */
> > -	value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
> > -	value &= 0xff000000;
> > -	value |= 0x00ff0100;
> > -	mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
> > +	pcie->ib_wins_configured = 0;
> > +	pcie->ob_wins_configured = 0;
> 
> This works so long as the number of bridge->windows never reduces. I think
> this assumption holds true.
> 
> Thanks,
> 
> Andrew Murray
> 
> > +
> > +	if (!reinit) {
> > +		/* setup bus numbers */
> > +		value = mobiveil_csr_readl(pcie, PCI_PRIMARY_BUS);
> > +		value &= 0xff000000;
> > +		value |= 0x00ff0100;
> > +		mobiveil_csr_writel(pcie, value, PCI_PRIMARY_BUS);
> > +	}
> >
> >  	/*
> >  	 * program Bus Master Enable Bit in Command Register in PAB Config
> > @@ -569,7 +574,7 @@ int mobiveil_pcie_host_probe(struct mobiveil_pcie
> *pcie)
> >  	 * configure all inbound and outbound windows and prepare the RC for
> >  	 * config access
> >  	 */
> > -	ret = mobiveil_host_init(pcie);
> > +	ret = mobiveil_host_init(pcie, false);
> >  	if (ret) {
> >  		dev_err(dev, "Failed to initialize host\n");
> >  		return ret;
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > index 95d2e7c809b8..37116c2a19fe 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -166,6 +166,7 @@ struct mobiveil_pcie {  };
> >
> >  int mobiveil_pcie_host_probe(struct mobiveil_pcie *pcie);
> > +int mobiveil_host_init(struct mobiveil_pcie *pcie, bool reinit);
> >  bool mobiveil_pcie_link_up(struct mobiveil_pcie *pcie);  int
> > mobiveil_bringup_link(struct mobiveil_pcie *pcie);  void
> > program_ob_windows(struct mobiveil_pcie *pcie, int win_num, u64
> > cpu_addr,
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors
  2020-01-13 11:31   ` Andrew Murray
@ 2020-02-06 13:45     ` Z.q. Hou
  0 siblings, 0 replies; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 13:45 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 19:32
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register
> accessors
> 
> On Wed, Nov 20, 2019 at 03:46:10AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > There are some 8-bit and 16-bit registers in PCIe configuration space,
> > so add these accessors accordingly.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> > Reviewed-by: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in>
> > ---
> > V9:
> >  - No change
> >
> >  .../pci/controller/mobiveil/pcie-mobiveil.h   | 23
> +++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> >
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > index 37116c2a19fe..750a7fd95bc1 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -182,10 +182,33 @@ static inline u32 mobiveil_csr_readl(struct
> mobiveil_pcie *pcie, u32 off)
> >  	return mobiveil_csr_read(pcie, off, 0x4);  }
> >
> > +static inline u32 mobiveil_csr_readw(struct mobiveil_pcie *pcie, u32
> > +off) {
> > +	return mobiveil_csr_read(pcie, off, 0x2); }
> > +
> > +static inline u32 mobiveil_csr_readb(struct mobiveil_pcie *pcie, u32
> > +off) {
> > +	return mobiveil_csr_read(pcie, off, 0x1); }
> 
> Do you think the above two return types should reflect the size of the access?

Will change in v10.

Thanks,
Zhiqiang

> 
> Thanks,
> 
> Andrew Murray
> 
> > +
> > +
> >  static inline void mobiveil_csr_writel(struct mobiveil_pcie *pcie, u32 val,
> >  				       u32 off)
> >  {
> >  	mobiveil_csr_write(pcie, val, off, 0x4);  }
> >
> > +static inline void mobiveil_csr_writew(struct mobiveil_pcie *pcie, u32 val,
> > +				       u32 off)
> > +{
> > +	mobiveil_csr_write(pcie, val, off, 0x2); }
> > +
> > +static inline void mobiveil_csr_writeb(struct mobiveil_pcie *pcie, u32 val,
> > +				       u32 off)
> > +{
> > +	mobiveil_csr_write(pcie, val, off, 0x1); }
> > +
> >  #endif /* _PCIE_MOBIVEIL_H */
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* RE: [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs
  2020-01-13 12:02   ` Andrew Murray
@ 2020-02-06 13:45     ` Z.q. Hou
  2020-02-06 14:29       ` Andrew Murray
  0 siblings, 1 reply; 55+ messages in thread
From: Z.q. Hou @ 2020-02-06 13:45 UTC (permalink / raw)
  To: Andrew Murray
  Cc: linux-pci, linux-arm-kernel, devicetree, linux-kernel, bhelgaas,
	robh+dt, arnd, mark.rutland, l.subrahmanya, shawnguo,
	m.karthikeyan, Leo Li, lorenzo.pieralisi, catalin.marinas,
	will.deacon, Mingkai Hu, M.h. Lian, Xiaowei Bao

Hi Andrew,

Thanks a lot for your comments!

> -----Original Message-----
> From: Andrew Murray <andrew.murray@arm.com>
> Sent: 2020年1月13日 20:03
> To: Z.q. Hou <zhiqiang.hou@nxp.com>
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> <xiaowei.bao@nxp.com>
> Subject: Re: [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP
> Layerscape SoCs
> 
> On Wed, Nov 20, 2019 at 03:46:23AM +0000, Z.q. Hou wrote:
> > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> >
> > This PCIe controller is based on the Mobiveil GPEX IP, which is
> > compatible with the PCI Express™ Base Specification, Revision 4.0.
> >
> > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> > ---
> >  drivers/pci/controller/mobiveil/Kconfig       |  10 +
> >  drivers/pci/controller/mobiveil/Makefile      |   1 +
> >  .../mobiveil/pcie-layerscape-gen4.c           | 274
> ++++++++++++++++++
> >  .../pci/controller/mobiveil/pcie-mobiveil.h   |  16 +-
> >  4 files changed, 299 insertions(+), 2 deletions(-)  create mode
> > 100644 drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> >
> > diff --git a/drivers/pci/controller/mobiveil/Kconfig
> > b/drivers/pci/controller/mobiveil/Kconfig
> > index 64343c07bfed..c823be8dab1c 100644
> > --- a/drivers/pci/controller/mobiveil/Kconfig
> > +++ b/drivers/pci/controller/mobiveil/Kconfig
> > @@ -21,4 +21,14 @@ config PCIE_MOBIVEIL_PLAT
> >  	  Soft IP. It has up to 8 outbound and inbound windows
> >  	  for address translation and it is a PCIe Gen4 IP.
> >
> > +config PCIE_LAYERSCAPE_GEN4
> > +	bool "Freescale Layerscape PCIe Gen4 controller"
> > +	depends on PCI
> > +	depends on OF && (ARM64 || ARCH_LAYERSCAPE)
> > +	depends on PCI_MSI_IRQ_DOMAIN
> > +	select PCIE_MOBIVEIL_HOST
> > +	help
> > +	  Say Y here if you want PCIe Gen4 controller support on
> > +	  Layerscape SoCs. The PCIe controller can work in RC or
> > +	  EP mode according to RCW[HOST_AGT_PEX] setting.
> 
> I think you can remove the last sentence - it doesn't give any value to users of
> KConfig.

OK, will remove it in v10.

> 
> 
> >  endmenu
> > diff --git a/drivers/pci/controller/mobiveil/Makefile
> > b/drivers/pci/controller/mobiveil/Makefile
> > index 9fb6d1c6504d..99d879de32d6 100644
> > --- a/drivers/pci/controller/mobiveil/Makefile
> > +++ b/drivers/pci/controller/mobiveil/Makefile
> > @@ -2,3 +2,4 @@
> >  obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
> >  obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
> >  obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
> > +obj-$(CONFIG_PCIE_LAYERSCAPE_GEN4) += pcie-layerscape-gen4.o
> > diff --git a/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > new file mode 100644
> > index 000000000000..6c0d3e2532db
> > --- /dev/null
> > +++ b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > @@ -0,0 +1,274 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * PCIe Gen4 host controller driver for NXP Layerscape SoCs
> > + *
> > + * Copyright 2019 NXP
> > + *
> > + * Author: Zhiqiang Hou <Zhiqiang.Hou@nxp.com>  */
> > +
> > +#include <linux/kernel.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/init.h>
> > +#include <linux/of_pci.h>
> > +#include <linux/of_platform.h>
> > +#include <linux/of_irq.h>
> > +#include <linux/of_address.h>
> > +#include <linux/pci.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/resource.h>
> > +#include <linux/mfd/syscon.h>
> > +#include <linux/regmap.h>
> > +
> > +#include "pcie-mobiveil.h"
> > +
> > +/* LUT and PF control registers */
> > +#define PCIE_LUT_OFF			0x80000
> > +#define PCIE_PF_OFF			0xc0000
> > +#define PCIE_PF_INT_STAT		0x18
> > +#define PF_INT_STAT_PABRST		BIT(31)
> > +
> > +#define PCIE_PF_DBG			0x7fc
> > +#define PF_DBG_LTSSM_MASK		0x3f
> > +#define PF_DBG_LTSSM_L0			0x2d /* L0 state */
> > +#define PF_DBG_WE			BIT(31)
> > +#define PF_DBG_PABR			BIT(27)
> > +
> > +#define to_ls_pcie_g4(x)		platform_get_drvdata((x)->pdev)
> > +
> > +struct ls_pcie_g4 {
> > +	struct mobiveil_pcie pci;
> > +	struct delayed_work dwork;
> > +	int irq;
> > +};
> > +
> > +static inline u32 ls_pcie_g4_lut_readl(struct ls_pcie_g4 *pcie, u32
> > +off) {
> > +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
> > +}
> > +
> > +static inline void ls_pcie_g4_lut_writel(struct ls_pcie_g4 *pcie,
> > +					 u32 off, u32 val)
> > +{
> > +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off); }
> > +
> > +static inline u32 ls_pcie_g4_pf_readl(struct ls_pcie_g4 *pcie, u32
> > +off) {
> > +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off); }
> > +
> > +static inline void ls_pcie_g4_pf_writel(struct ls_pcie_g4 *pcie,
> > +					u32 off, u32 val)
> > +{
> > +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off); }
> > +
> > +static bool ls_pcie_g4_is_bridge(struct ls_pcie_g4 *pcie) {
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +	u32 header_type;
> > +
> > +	header_type = mobiveil_csr_readb(mv_pci, PCI_HEADER_TYPE);
> > +	header_type &= 0x7f;
> > +
> > +	return header_type == PCI_HEADER_TYPE_BRIDGE; }
> > +
> > +static int ls_pcie_g4_link_up(struct mobiveil_pcie *pci) {
> > +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(pci);
> > +	u32 state;
> > +
> > +	state = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > +	state =	state & PF_DBG_LTSSM_MASK;
> > +
> > +	if (state == PF_DBG_LTSSM_L0)
> > +		return 1;
> > +
> > +	return 0;
> > +}
> > +
> > +static void ls_pcie_g4_disable_interrupt(struct ls_pcie_g4 *pcie) {
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +
> > +	mobiveil_csr_writel(mv_pci, 0, PAB_INTP_AMBA_MISC_ENB); }
> > +
> > +static void ls_pcie_g4_enable_interrupt(struct ls_pcie_g4 *pcie) {
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +	u32 val;
> > +
> > +	/* Clear the interrupt status */
> > +	mobiveil_csr_writel(mv_pci, 0xffffffff, PAB_INTP_AMBA_MISC_STAT);
> > +
> > +	val = PAB_INTP_INTX_MASK | PAB_INTP_MSI | PAB_INTP_RESET |
> > +	      PAB_INTP_PCIE_UE | PAB_INTP_IE_PMREDI | PAB_INTP_IE_EC;
> > +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_ENB); }
> > +
> > +static void ls_pcie_g4_reinit_hw(struct ls_pcie_g4 *pcie) {
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +	struct device *dev = &mv_pci->pdev->dev;
> > +	u32 val, act_stat;
> > +	int to = 100;
> > +
> > +	/* Poll for pab_csb_reset to set and PAB activity to clear */
> > +	do {
> > +		usleep_range(10, 15);
> > +		val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_INT_STAT);
> > +		act_stat = mobiveil_csr_readl(mv_pci, PAB_ACTIVITY_STAT);
> > +	} while (((val & PF_INT_STAT_PABRST) == 0 || act_stat) && to--);
> > +	if (to < 0) {
> > +		dev_err(dev, "Poll PABRST&PABACT timeout\n");
> > +		return;
> 
> If a timeout happens here - the caller has no idea this has happened and yet
> the following work doesn't get done. Isn't this a problem?

Will change the return value type to 'int' in v10, so that the caller can know the fail.

> 
> > +	}
> > +
> > +	/* clear PEX_RESET bit in PEX_PF0_DBG register */
> > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > +	val |= PF_DBG_WE;
> > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > +
> > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > +	val |= PF_DBG_PABR;
> > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > +
> > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > +	val &= ~PF_DBG_WE;
> > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > +
> > +	mobiveil_host_init(mv_pci, true);
> 
> Can mobiveil_host_init fail?

It should not fail, only register programming operations were left in this function.

> 
> > +
> > +	to = 100;
> > +	while (!ls_pcie_g4_link_up(mv_pci) && to--)
> > +		usleep_range(200, 250);
> > +	if (to < 0)
> > +		dev_err(dev, "PCIe link training timeout\n"); }
> > +
> > +static irqreturn_t ls_pcie_g4_isr(int irq, void *dev_id) {
> > +	struct ls_pcie_g4 *pcie = (struct ls_pcie_g4 *)dev_id;
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +	u32 val;
> > +
> > +	val = mobiveil_csr_readl(mv_pci, PAB_INTP_AMBA_MISC_STAT);
> > +	if (!val)
> > +		return IRQ_NONE;
> > +
> > +	if (val & PAB_INTP_RESET) {
> 
> Can you explain why this is needed (perhaps also in the cover letter)?

The hot reset will result in the RC crash, so need the ISR to reset the RC.

> 
> > +		ls_pcie_g4_disable_interrupt(pcie);
> > +		schedule_delayed_work(&pcie->dwork, msecs_to_jiffies(1));
> > +	}
> > +
> > +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_STAT);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static int ls_pcie_g4_interrupt_init(struct mobiveil_pcie *mv_pci) {
> > +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(mv_pci);
> > +	struct platform_device *pdev = mv_pci->pdev;
> > +	struct device *dev = &pdev->dev;
> > +	int ret;
> > +
> > +	pcie->irq = platform_get_irq_byname(pdev, "intr");
> > +	if (pcie->irq < 0) {
> > +		dev_err(dev, "Can't get 'intr' IRQ, errno = %d\n", pcie->irq);
> > +		return pcie->irq;
> > +	}
> > +	ret = devm_request_irq(dev, pcie->irq, ls_pcie_g4_isr,
> > +			       IRQF_SHARED, pdev->name, pcie);
> > +	if (ret) {
> > +		dev_err(dev, "Can't register PCIe IRQ, errno = %d\n", ret);
> > +		return  ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void ls_pcie_g4_reset(struct work_struct *work) {
> > +	struct delayed_work *dwork = container_of(work, struct delayed_work,
> > +						  work);
> > +	struct ls_pcie_g4 *pcie = container_of(dwork, struct ls_pcie_g4, dwork);
> > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > +	u16 ctrl;
> > +
> > +	ctrl = mobiveil_csr_readw(mv_pci, PCI_BRIDGE_CONTROL);
> > +	ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
> > +	mobiveil_csr_writew(mv_pci, ctrl, PCI_BRIDGE_CONTROL);
> > +	ls_pcie_g4_reinit_hw(pcie);
> > +	ls_pcie_g4_enable_interrupt(pcie);
> > +}
> > +
> > +static struct mobiveil_rp_ops ls_pcie_g4_rp_ops = {
> > +	.interrupt_init = ls_pcie_g4_interrupt_init, };
> > +
> > +static const struct mobiveil_pab_ops ls_pcie_g4_pab_ops = {
> > +	.link_up = ls_pcie_g4_link_up,
> > +};
> > +
> > +static int __init ls_pcie_g4_probe(struct platform_device *pdev) {
> > +	struct device *dev = &pdev->dev;
> > +	struct pci_host_bridge *bridge;
> > +	struct mobiveil_pcie *mv_pci;
> > +	struct ls_pcie_g4 *pcie;
> > +	struct device_node *np = dev->of_node;
> > +	int ret;
> > +
> > +	if (!of_parse_phandle(np, "msi-parent", 0)) {
> > +		dev_err(dev, "Failed to find msi-parent\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> > +	if (!bridge)
> > +		return -ENOMEM;
> > +
> > +	pcie = pci_host_bridge_priv(bridge);
> > +	mv_pci = &pcie->pci;
> > +
> > +	mv_pci->pdev = pdev;
> > +	mv_pci->ops = &ls_pcie_g4_pab_ops;
> > +	mv_pci->rp.ops = &ls_pcie_g4_rp_ops;
> > +	mv_pci->rp.bridge = bridge;
> > +
> > +	platform_set_drvdata(pdev, pcie);
> > +
> > +	INIT_DELAYED_WORK(&pcie->dwork, ls_pcie_g4_reset);
> > +
> > +	ret = mobiveil_pcie_host_probe(mv_pci);
> > +	if (ret) {
> > +		dev_err(dev, "Fail to probe\n");
> > +		return  ret;
> > +	}
> > +
> > +	if (!ls_pcie_g4_is_bridge(pcie))
> 
> Is this a check that could apply to all host bridge drivers and thus live in
> mobiveil_pcie_host_probe?

Yes, will do in v10.

> 
> > +		return -ENODEV;
> > +
> > +	ls_pcie_g4_enable_interrupt(pcie);
> 
> Is there an issue here in that we enable interrupts *after* telling the kernel
> about our controller? (Same applies for bailing if the IP isn't a bridge).

Andrew, I don't understand the issue, can you help to explain?

Thanks,
Zhiqiang

> 
> Thanks,
> 
> Andrew Murray
> 
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct of_device_id ls_pcie_g4_of_match[] = {
> > +	{ .compatible = "fsl,lx2160a-pcie", },
> > +	{ },
> > +};
> > +
> > +static struct platform_driver ls_pcie_g4_driver = {
> > +	.driver = {
> > +		.name = "layerscape-pcie-gen4",
> > +		.of_match_table = ls_pcie_g4_of_match,
> > +		.suppress_bind_attrs = true,
> > +	},
> > +};
> > +
> > +builtin_platform_driver_probe(ls_pcie_g4_driver, ls_pcie_g4_probe);
> > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > index 750a7fd95bc1..c57a68d2bac4 100644
> > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > @@ -43,6 +43,8 @@
> >  #define  PAGE_LO_MASK			0x3ff
> >  #define  PAGE_SEL_OFFSET_SHIFT		10
> >
> > +#define PAB_ACTIVITY_STAT		0x81c
> > +
> >  #define PAB_AXI_PIO_CTRL		0x0840
> >  #define  APIO_EN_MASK			0xf
> >
> > @@ -51,8 +53,18 @@
> >
> >  #define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> >  #define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> > -#define  PAB_INTP_INTX_MASK		0x01e0
> > -#define  PAB_INTP_MSI_MASK		0x8
> > +#define  PAB_INTP_RESET			BIT(1)
> > +#define  PAB_INTP_MSI			BIT(3)
> > +#define  PAB_INTP_INTA			BIT(5)
> > +#define  PAB_INTP_INTB			BIT(6)
> > +#define  PAB_INTP_INTC			BIT(7)
> > +#define  PAB_INTP_INTD			BIT(8)
> > +#define  PAB_INTP_PCIE_UE		BIT(9)
> > +#define  PAB_INTP_IE_PMREDI		BIT(29)
> > +#define  PAB_INTP_IE_EC			BIT(30)
> > +#define  PAB_INTP_MSI_MASK		PAB_INTP_MSI
> > +#define  PAB_INTP_INTX_MASK		(PAB_INTP_INTA |
> PAB_INTP_INTB |\
> > +					PAB_INTP_INTC | PAB_INTP_INTD)
> >
> >  #define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0,
> win)
> >  #define  WIN_ENABLE_SHIFT		0
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs
  2020-02-06 13:45     ` Z.q. Hou
@ 2020-02-06 14:29       ` Andrew Murray
  0 siblings, 0 replies; 55+ messages in thread
From: Andrew Murray @ 2020-02-06 14:29 UTC (permalink / raw)
  To: Z.q. Hou
  Cc: Andrew Murray, linux-pci, linux-arm-kernel, devicetree,
	linux-kernel, bhelgaas, robh+dt, arnd, mark.rutland,
	l.subrahmanya, shawnguo, m.karthikeyan, Leo Li,
	lorenzo.pieralisi, catalin.marinas, will.deacon, Mingkai Hu,
	M.h. Lian, Xiaowei Bao

On Thu, Feb 06, 2020 at 01:45:57PM +0000, Z.q. Hou wrote:
> Hi Andrew,
> 
> Thanks a lot for your comments!
> 
> > -----Original Message-----
> > From: Andrew Murray <andrew.murray@arm.com>
> > Sent: 2020年1月13日 20:03
> > To: Z.q. Hou <zhiqiang.hou@nxp.com>
> > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> > bhelgaas@google.com; robh+dt@kernel.org; arnd@arndb.de;
> > mark.rutland@arm.com; l.subrahmanya@mobiveil.co.in;
> > shawnguo@kernel.org; m.karthikeyan@mobiveil.co.in; Leo Li
> > <leoyang.li@nxp.com>; lorenzo.pieralisi@arm.com;
> > catalin.marinas@arm.com; will.deacon@arm.com; Mingkai Hu
> > <mingkai.hu@nxp.com>; M.h. Lian <minghuan.lian@nxp.com>; Xiaowei Bao
> > <xiaowei.bao@nxp.com>
> > Subject: Re: [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP
> > Layerscape SoCs
> > 
> > On Wed, Nov 20, 2019 at 03:46:23AM +0000, Z.q. Hou wrote:
> > > From: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > >
> > > This PCIe controller is based on the Mobiveil GPEX IP, which is
> > > compatible with the PCI Express™ Base Specification, Revision 4.0.
> > >
> > > Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
> > > Reviewed-by: Minghuan Lian <Minghuan.Lian@nxp.com>
> > > ---
> > >  drivers/pci/controller/mobiveil/Kconfig       |  10 +
> > >  drivers/pci/controller/mobiveil/Makefile      |   1 +
> > >  .../mobiveil/pcie-layerscape-gen4.c           | 274
> > ++++++++++++++++++
> > >  .../pci/controller/mobiveil/pcie-mobiveil.h   |  16 +-
> > >  4 files changed, 299 insertions(+), 2 deletions(-)  create mode
> > > 100644 drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > >
> > > diff --git a/drivers/pci/controller/mobiveil/Kconfig
> > > b/drivers/pci/controller/mobiveil/Kconfig
> > > index 64343c07bfed..c823be8dab1c 100644
> > > --- a/drivers/pci/controller/mobiveil/Kconfig
> > > +++ b/drivers/pci/controller/mobiveil/Kconfig
> > > @@ -21,4 +21,14 @@ config PCIE_MOBIVEIL_PLAT
> > >  	  Soft IP. It has up to 8 outbound and inbound windows
> > >  	  for address translation and it is a PCIe Gen4 IP.
> > >
> > > +config PCIE_LAYERSCAPE_GEN4
> > > +	bool "Freescale Layerscape PCIe Gen4 controller"
> > > +	depends on PCI
> > > +	depends on OF && (ARM64 || ARCH_LAYERSCAPE)
> > > +	depends on PCI_MSI_IRQ_DOMAIN
> > > +	select PCIE_MOBIVEIL_HOST
> > > +	help
> > > +	  Say Y here if you want PCIe Gen4 controller support on
> > > +	  Layerscape SoCs. The PCIe controller can work in RC or
> > > +	  EP mode according to RCW[HOST_AGT_PEX] setting.
> > 
> > I think you can remove the last sentence - it doesn't give any value to users of
> > KConfig.
> 
> OK, will remove it in v10.
> 
> > 
> > 
> > >  endmenu
> > > diff --git a/drivers/pci/controller/mobiveil/Makefile
> > > b/drivers/pci/controller/mobiveil/Makefile
> > > index 9fb6d1c6504d..99d879de32d6 100644
> > > --- a/drivers/pci/controller/mobiveil/Makefile
> > > +++ b/drivers/pci/controller/mobiveil/Makefile
> > > @@ -2,3 +2,4 @@
> > >  obj-$(CONFIG_PCIE_MOBIVEIL) += pcie-mobiveil.o
> > >  obj-$(CONFIG_PCIE_MOBIVEIL_HOST) += pcie-mobiveil-host.o
> > >  obj-$(CONFIG_PCIE_MOBIVEIL_PLAT) += pcie-mobiveil-plat.o
> > > +obj-$(CONFIG_PCIE_LAYERSCAPE_GEN4) += pcie-layerscape-gen4.o
> > > diff --git a/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > > b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > > new file mode 100644
> > > index 000000000000..6c0d3e2532db
> > > --- /dev/null
> > > +++ b/drivers/pci/controller/mobiveil/pcie-layerscape-gen4.c
> > > @@ -0,0 +1,274 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * PCIe Gen4 host controller driver for NXP Layerscape SoCs
> > > + *
> > > + * Copyright 2019 NXP
> > > + *
> > > + * Author: Zhiqiang Hou <Zhiqiang.Hou@nxp.com>  */
> > > +
> > > +#include <linux/kernel.h>
> > > +#include <linux/interrupt.h>
> > > +#include <linux/init.h>
> > > +#include <linux/of_pci.h>
> > > +#include <linux/of_platform.h>
> > > +#include <linux/of_irq.h>
> > > +#include <linux/of_address.h>
> > > +#include <linux/pci.h>
> > > +#include <linux/platform_device.h>
> > > +#include <linux/resource.h>
> > > +#include <linux/mfd/syscon.h>
> > > +#include <linux/regmap.h>
> > > +
> > > +#include "pcie-mobiveil.h"
> > > +
> > > +/* LUT and PF control registers */
> > > +#define PCIE_LUT_OFF			0x80000
> > > +#define PCIE_PF_OFF			0xc0000
> > > +#define PCIE_PF_INT_STAT		0x18
> > > +#define PF_INT_STAT_PABRST		BIT(31)
> > > +
> > > +#define PCIE_PF_DBG			0x7fc
> > > +#define PF_DBG_LTSSM_MASK		0x3f
> > > +#define PF_DBG_LTSSM_L0			0x2d /* L0 state */
> > > +#define PF_DBG_WE			BIT(31)
> > > +#define PF_DBG_PABR			BIT(27)
> > > +
> > > +#define to_ls_pcie_g4(x)		platform_get_drvdata((x)->pdev)
> > > +
> > > +struct ls_pcie_g4 {
> > > +	struct mobiveil_pcie pci;
> > > +	struct delayed_work dwork;
> > > +	int irq;
> > > +};
> > > +
> > > +static inline u32 ls_pcie_g4_lut_readl(struct ls_pcie_g4 *pcie, u32
> > > +off) {
> > > +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off);
> > > +}
> > > +
> > > +static inline void ls_pcie_g4_lut_writel(struct ls_pcie_g4 *pcie,
> > > +					 u32 off, u32 val)
> > > +{
> > > +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_LUT_OFF + off); }
> > > +
> > > +static inline u32 ls_pcie_g4_pf_readl(struct ls_pcie_g4 *pcie, u32
> > > +off) {
> > > +	return ioread32(pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off); }
> > > +
> > > +static inline void ls_pcie_g4_pf_writel(struct ls_pcie_g4 *pcie,
> > > +					u32 off, u32 val)
> > > +{
> > > +	iowrite32(val, pcie->pci.csr_axi_slave_base + PCIE_PF_OFF + off); }
> > > +
> > > +static bool ls_pcie_g4_is_bridge(struct ls_pcie_g4 *pcie) {
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +	u32 header_type;
> > > +
> > > +	header_type = mobiveil_csr_readb(mv_pci, PCI_HEADER_TYPE);
> > > +	header_type &= 0x7f;
> > > +
> > > +	return header_type == PCI_HEADER_TYPE_BRIDGE; }
> > > +
> > > +static int ls_pcie_g4_link_up(struct mobiveil_pcie *pci) {
> > > +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(pci);
> > > +	u32 state;
> > > +
> > > +	state = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > > +	state =	state & PF_DBG_LTSSM_MASK;
> > > +
> > > +	if (state == PF_DBG_LTSSM_L0)
> > > +		return 1;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static void ls_pcie_g4_disable_interrupt(struct ls_pcie_g4 *pcie) {
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +
> > > +	mobiveil_csr_writel(mv_pci, 0, PAB_INTP_AMBA_MISC_ENB); }
> > > +
> > > +static void ls_pcie_g4_enable_interrupt(struct ls_pcie_g4 *pcie) {
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +	u32 val;
> > > +
> > > +	/* Clear the interrupt status */
> > > +	mobiveil_csr_writel(mv_pci, 0xffffffff, PAB_INTP_AMBA_MISC_STAT);
> > > +
> > > +	val = PAB_INTP_INTX_MASK | PAB_INTP_MSI | PAB_INTP_RESET |
> > > +	      PAB_INTP_PCIE_UE | PAB_INTP_IE_PMREDI | PAB_INTP_IE_EC;
> > > +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_ENB); }
> > > +
> > > +static void ls_pcie_g4_reinit_hw(struct ls_pcie_g4 *pcie) {
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +	struct device *dev = &mv_pci->pdev->dev;
> > > +	u32 val, act_stat;
> > > +	int to = 100;
> > > +
> > > +	/* Poll for pab_csb_reset to set and PAB activity to clear */
> > > +	do {
> > > +		usleep_range(10, 15);
> > > +		val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_INT_STAT);
> > > +		act_stat = mobiveil_csr_readl(mv_pci, PAB_ACTIVITY_STAT);
> > > +	} while (((val & PF_INT_STAT_PABRST) == 0 || act_stat) && to--);
> > > +	if (to < 0) {
> > > +		dev_err(dev, "Poll PABRST&PABACT timeout\n");
> > > +		return;
> > 
> > If a timeout happens here - the caller has no idea this has happened and yet
> > the following work doesn't get done. Isn't this a problem?
> 
> Will change the return value type to 'int' in v10, so that the caller can know the fail.
> 
> > 
> > > +	}
> > > +
> > > +	/* clear PEX_RESET bit in PEX_PF0_DBG register */
> > > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > > +	val |= PF_DBG_WE;
> > > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > > +
> > > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > > +	val |= PF_DBG_PABR;
> > > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > > +
> > > +	val = ls_pcie_g4_pf_readl(pcie, PCIE_PF_DBG);
> > > +	val &= ~PF_DBG_WE;
> > > +	ls_pcie_g4_pf_writel(pcie, PCIE_PF_DBG, val);
> > > +
> > > +	mobiveil_host_init(mv_pci, true);
> > 
> > Can mobiveil_host_init fail?
> 
> It should not fail, only register programming operations were left in this function.
> 
> > 
> > > +
> > > +	to = 100;
> > > +	while (!ls_pcie_g4_link_up(mv_pci) && to--)
> > > +		usleep_range(200, 250);
> > > +	if (to < 0)
> > > +		dev_err(dev, "PCIe link training timeout\n"); }
> > > +
> > > +static irqreturn_t ls_pcie_g4_isr(int irq, void *dev_id) {
> > > +	struct ls_pcie_g4 *pcie = (struct ls_pcie_g4 *)dev_id;
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +	u32 val;
> > > +
> > > +	val = mobiveil_csr_readl(mv_pci, PAB_INTP_AMBA_MISC_STAT);
> > > +	if (!val)
> > > +		return IRQ_NONE;
> > > +
> > > +	if (val & PAB_INTP_RESET) {
> > 
> > Can you explain why this is needed (perhaps also in the cover letter)?
> 
> The hot reset will result in the RC crash, so need the ISR to reset the RC.
> 
> > 
> > > +		ls_pcie_g4_disable_interrupt(pcie);
> > > +		schedule_delayed_work(&pcie->dwork, msecs_to_jiffies(1));
> > > +	}
> > > +
> > > +	mobiveil_csr_writel(mv_pci, val, PAB_INTP_AMBA_MISC_STAT);
> > > +
> > > +	return IRQ_HANDLED;
> > > +}
> > > +
> > > +static int ls_pcie_g4_interrupt_init(struct mobiveil_pcie *mv_pci) {
> > > +	struct ls_pcie_g4 *pcie = to_ls_pcie_g4(mv_pci);
> > > +	struct platform_device *pdev = mv_pci->pdev;
> > > +	struct device *dev = &pdev->dev;
> > > +	int ret;
> > > +
> > > +	pcie->irq = platform_get_irq_byname(pdev, "intr");
> > > +	if (pcie->irq < 0) {
> > > +		dev_err(dev, "Can't get 'intr' IRQ, errno = %d\n", pcie->irq);
> > > +		return pcie->irq;
> > > +	}
> > > +	ret = devm_request_irq(dev, pcie->irq, ls_pcie_g4_isr,
> > > +			       IRQF_SHARED, pdev->name, pcie);
> > > +	if (ret) {
> > > +		dev_err(dev, "Can't register PCIe IRQ, errno = %d\n", ret);
> > > +		return  ret;
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static void ls_pcie_g4_reset(struct work_struct *work) {
> > > +	struct delayed_work *dwork = container_of(work, struct delayed_work,
> > > +						  work);
> > > +	struct ls_pcie_g4 *pcie = container_of(dwork, struct ls_pcie_g4, dwork);
> > > +	struct mobiveil_pcie *mv_pci = &pcie->pci;
> > > +	u16 ctrl;
> > > +
> > > +	ctrl = mobiveil_csr_readw(mv_pci, PCI_BRIDGE_CONTROL);
> > > +	ctrl &= ~PCI_BRIDGE_CTL_BUS_RESET;
> > > +	mobiveil_csr_writew(mv_pci, ctrl, PCI_BRIDGE_CONTROL);
> > > +	ls_pcie_g4_reinit_hw(pcie);
> > > +	ls_pcie_g4_enable_interrupt(pcie);
> > > +}
> > > +
> > > +static struct mobiveil_rp_ops ls_pcie_g4_rp_ops = {
> > > +	.interrupt_init = ls_pcie_g4_interrupt_init, };
> > > +
> > > +static const struct mobiveil_pab_ops ls_pcie_g4_pab_ops = {
> > > +	.link_up = ls_pcie_g4_link_up,
> > > +};
> > > +
> > > +static int __init ls_pcie_g4_probe(struct platform_device *pdev) {
> > > +	struct device *dev = &pdev->dev;
> > > +	struct pci_host_bridge *bridge;
> > > +	struct mobiveil_pcie *mv_pci;
> > > +	struct ls_pcie_g4 *pcie;
> > > +	struct device_node *np = dev->of_node;
> > > +	int ret;
> > > +
> > > +	if (!of_parse_phandle(np, "msi-parent", 0)) {
> > > +		dev_err(dev, "Failed to find msi-parent\n");
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie));
> > > +	if (!bridge)
> > > +		return -ENOMEM;
> > > +
> > > +	pcie = pci_host_bridge_priv(bridge);
> > > +	mv_pci = &pcie->pci;
> > > +
> > > +	mv_pci->pdev = pdev;
> > > +	mv_pci->ops = &ls_pcie_g4_pab_ops;
> > > +	mv_pci->rp.ops = &ls_pcie_g4_rp_ops;
> > > +	mv_pci->rp.bridge = bridge;
> > > +
> > > +	platform_set_drvdata(pdev, pcie);
> > > +
> > > +	INIT_DELAYED_WORK(&pcie->dwork, ls_pcie_g4_reset);
> > > +
> > > +	ret = mobiveil_pcie_host_probe(mv_pci);
> > > +	if (ret) {
> > > +		dev_err(dev, "Fail to probe\n");
> > > +		return  ret;
> > > +	}
> > > +
> > > +	if (!ls_pcie_g4_is_bridge(pcie))
> > 
> > Is this a check that could apply to all host bridge drivers and thus live in
> > mobiveil_pcie_host_probe?
> 
> Yes, will do in v10.
> 
> > 
> > > +		return -ENODEV;
> > > +
> > > +	ls_pcie_g4_enable_interrupt(pcie);
> > 
> > Is there an issue here in that we enable interrupts *after* telling the kernel
> > about our controller? (Same applies for bailing if the IP isn't a bridge).
> 
> Andrew, I don't understand the issue, can you help to explain?

If I recall correctly mobiveil_pcie_host_probe tells the kernel there is a PCI host
bridge and allows it to start enumerating the tree - at this point surely interrupts
may be expected - however we don't enable them until after this point. I'd assume
we'd need to get the hardware into a state where it can handle interrupts before
telling the kernel it can use this host bridge.

Likewise is ls_pcie_g4_is_bridge returns false we fail the probe yet the kernel may
already be enumerating the bus.

Thanks,

Andrew Murray

> 
> Thanks,
> Zhiqiang
> 
> > 
> > Thanks,
> > 
> > Andrew Murray
> > 
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static const struct of_device_id ls_pcie_g4_of_match[] = {
> > > +	{ .compatible = "fsl,lx2160a-pcie", },
> > > +	{ },
> > > +};
> > > +
> > > +static struct platform_driver ls_pcie_g4_driver = {
> > > +	.driver = {
> > > +		.name = "layerscape-pcie-gen4",
> > > +		.of_match_table = ls_pcie_g4_of_match,
> > > +		.suppress_bind_attrs = true,
> > > +	},
> > > +};
> > > +
> > > +builtin_platform_driver_probe(ls_pcie_g4_driver, ls_pcie_g4_probe);
> > > diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > > b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > > index 750a7fd95bc1..c57a68d2bac4 100644
> > > --- a/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > > +++ b/drivers/pci/controller/mobiveil/pcie-mobiveil.h
> > > @@ -43,6 +43,8 @@
> > >  #define  PAGE_LO_MASK			0x3ff
> > >  #define  PAGE_SEL_OFFSET_SHIFT		10
> > >
> > > +#define PAB_ACTIVITY_STAT		0x81c
> > > +
> > >  #define PAB_AXI_PIO_CTRL		0x0840
> > >  #define  APIO_EN_MASK			0xf
> > >
> > > @@ -51,8 +53,18 @@
> > >
> > >  #define PAB_INTP_AMBA_MISC_ENB		0x0b0c
> > >  #define PAB_INTP_AMBA_MISC_STAT		0x0b1c
> > > -#define  PAB_INTP_INTX_MASK		0x01e0
> > > -#define  PAB_INTP_MSI_MASK		0x8
> > > +#define  PAB_INTP_RESET			BIT(1)
> > > +#define  PAB_INTP_MSI			BIT(3)
> > > +#define  PAB_INTP_INTA			BIT(5)
> > > +#define  PAB_INTP_INTB			BIT(6)
> > > +#define  PAB_INTP_INTC			BIT(7)
> > > +#define  PAB_INTP_INTD			BIT(8)
> > > +#define  PAB_INTP_PCIE_UE		BIT(9)
> > > +#define  PAB_INTP_IE_PMREDI		BIT(29)
> > > +#define  PAB_INTP_IE_EC			BIT(30)
> > > +#define  PAB_INTP_MSI_MASK		PAB_INTP_MSI
> > > +#define  PAB_INTP_INTX_MASK		(PAB_INTP_INTA |
> > PAB_INTP_INTB |\
> > > +					PAB_INTP_INTC | PAB_INTP_INTD)
> > >
> > >  #define PAB_AXI_AMAP_CTRL(win)		PAB_REG_ADDR(0x0ba0,
> > win)
> > >  #define  WIN_ENABLE_SHIFT		0
> > > --
> > > 2.17.1
> > >

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-06 10:57         ` Z.q. Hou
@ 2020-02-10 15:12           ` Olof Johansson
  2020-02-10 15:22             ` Russell King - ARM Linux admin
  2020-02-10 15:33             ` Lorenzo Pieralisi
  0 siblings, 2 replies; 55+ messages in thread
From: Olof Johansson @ 2020-02-10 15:12 UTC (permalink / raw)
  To: Z.q. Hou, bhelgaas
  Cc: Lorenzo Pieralisi, linux-pci, linux-arm-kernel, devicetree,
	linux-kernel, robh+dt, arnd, mark.rutland, l.subrahmanya,
	shawnguo, m.karthikeyan, Leo Li, catalin.marinas, will.deacon,
	andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
>
> Hi Olof,
>
> Thanks a lot for your comments!
> And sorry for my delay respond!

Actually, they apply with only minor conflicts on top of current -next.

Bjorn, any chance we can get you to pick these up pretty soon? They
enable full use of a promising ARM developer system, the SolidRun
HoneyComb, and would be quite valuable for me and others to be able to
use with mainline or -next without any additional patches applied --
which this patchset achieves.

I know there are pending revisions based on feedback. I'll leave it up
to you and others to determine if that can be done with incremental
patches on top, or if it should be fixed before the initial patchset
is applied. But all in all, it's holding up adaption by me and surely
others of a very interesting platform -- I'm looking to replace my
aging MacchiatoBin with one of these and would need PCIe/NVMe to work
before I do.


Thanks!


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 15:12           ` Olof Johansson
@ 2020-02-10 15:22             ` Russell King - ARM Linux admin
  2020-02-10 15:28               ` Olof Johansson
  2020-02-10 15:33             ` Lorenzo Pieralisi
  1 sibling, 1 reply; 55+ messages in thread
From: Russell King - ARM Linux admin @ 2020-02-10 15:22 UTC (permalink / raw)
  To: Olof Johansson
  Cc: Z.q. Hou, bhelgaas, mark.rutland, devicetree, Lorenzo Pieralisi,
	m.karthikeyan, arnd, linux-pci, l.subrahmanya, will.deacon,
	linux-kernel, Leo Li, M.h. Lian, robh+dt, Mingkai Hu,
	catalin.marinas, Xiaowei Bao, andrew.murray, shawnguo,
	linux-arm-kernel

On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> >
> > Hi Olof,
> >
> > Thanks a lot for your comments!
> > And sorry for my delay respond!
> 
> Actually, they apply with only minor conflicts on top of current -next.
> 
> Bjorn, any chance we can get you to pick these up pretty soon? They
> enable full use of a promising ARM developer system, the SolidRun
> HoneyComb, and would be quite valuable for me and others to be able to
> use with mainline or -next without any additional patches applied --
> which this patchset achieves.
> 
> I know there are pending revisions based on feedback. I'll leave it up
> to you and others to determine if that can be done with incremental
> patches on top, or if it should be fixed before the initial patchset
> is applied. But all in all, it's holding up adaption by me and surely
> others of a very interesting platform -- I'm looking to replace my
> aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> before I do.

If you're going to be using NVMe, make sure you use a power-fail safe
version; I've already had one instance where ext4 failed to mount
because of a corrupted journal using an XPG SX8200 after the Honeycomb
Serror'd, and then I powered it down after a few hours before later
booting it back up.

EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
EXT4-fs (nvme0n1p2): write access will be enabled during recovery
JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
EXT4-fs (nvme0n1p2): error loading journal

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 15:22             ` Russell King - ARM Linux admin
@ 2020-02-10 15:28               ` Olof Johansson
  2020-02-10 16:15                 ` Russell King - ARM Linux admin
  2020-02-10 18:41                 ` Li Yang
  0 siblings, 2 replies; 55+ messages in thread
From: Olof Johansson @ 2020-02-10 15:28 UTC (permalink / raw)
  To: Russell King - ARM Linux admin
  Cc: Z.q. Hou, bhelgaas, mark.rutland, devicetree, Lorenzo Pieralisi,
	m.karthikeyan, arnd, linux-pci, l.subrahmanya, will.deacon,
	linux-kernel, Leo Li, M.h. Lian, robh+dt, Mingkai Hu,
	catalin.marinas, Xiaowei Bao, andrew.murray, shawnguo,
	linux-arm-kernel

On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > >
> > > Hi Olof,
> > >
> > > Thanks a lot for your comments!
> > > And sorry for my delay respond!
> >
> > Actually, they apply with only minor conflicts on top of current -next.
> >
> > Bjorn, any chance we can get you to pick these up pretty soon? They
> > enable full use of a promising ARM developer system, the SolidRun
> > HoneyComb, and would be quite valuable for me and others to be able to
> > use with mainline or -next without any additional patches applied --
> > which this patchset achieves.
> >
> > I know there are pending revisions based on feedback. I'll leave it up
> > to you and others to determine if that can be done with incremental
> > patches on top, or if it should be fixed before the initial patchset
> > is applied. But all in all, it's holding up adaption by me and surely
> > others of a very interesting platform -- I'm looking to replace my
> > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > before I do.
>
> If you're going to be using NVMe, make sure you use a power-fail safe
> version; I've already had one instance where ext4 failed to mount
> because of a corrupted journal using an XPG SX8200 after the Honeycomb
> Serror'd, and then I powered it down after a few hours before later
> booting it back up.
>
> EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> EXT4-fs (nvme0n1p2): error loading journal

Hmm, using btrfs on mine, not sure if the exposure is similar or not.

Do you know if the SErr was due to a known issue and/or if it's
something that's fixed in production silicon?

(I still can't enable SMMU since across a warm reboot it fails
*completely*, with nothing coming up and working. NXP folks, you
listening? :)


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 15:12           ` Olof Johansson
  2020-02-10 15:22             ` Russell King - ARM Linux admin
@ 2020-02-10 15:33             ` Lorenzo Pieralisi
  1 sibling, 0 replies; 55+ messages in thread
From: Lorenzo Pieralisi @ 2020-02-10 15:33 UTC (permalink / raw)
  To: Olof Johansson
  Cc: Z.q. Hou, bhelgaas, linux-pci, linux-arm-kernel, devicetree,
	linux-kernel, robh+dt, arnd, mark.rutland, l.subrahmanya,
	shawnguo, m.karthikeyan, Leo Li, catalin.marinas, will.deacon,
	andrew.murray, Mingkai Hu, M.h. Lian, Xiaowei Bao

On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> >
> > Hi Olof,
> >
> > Thanks a lot for your comments!
> > And sorry for my delay respond!
> 
> Actually, they apply with only minor conflicts on top of current -next.
> 
> Bjorn, any chance we can get you to pick these up pretty soon? They
> enable full use of a promising ARM developer system, the SolidRun
> HoneyComb, and would be quite valuable for me and others to be able to
> use with mainline or -next without any additional patches applied --
> which this patchset achieves.
> 
> I know there are pending revisions based on feedback. I'll leave it up
> to you and others to determine if that can be done with incremental
> patches on top, or if it should be fixed before the initial patchset
> is applied. But all in all, it's holding up adaption by me and surely
> others of a very interesting platform -- I'm looking to replace my
> aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> before I do.

We should be able to merge them for v5.7, I don't know when they
will land in -next.

Thanks,
Lorenzo

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 15:28               ` Olof Johansson
@ 2020-02-10 16:15                 ` Russell King - ARM Linux admin
  2020-02-10 17:20                   ` Russell King - ARM Linux admin
  2020-02-10 18:33                   ` Olof Johansson
  2020-02-10 18:41                 ` Li Yang
  1 sibling, 2 replies; 55+ messages in thread
From: Russell King - ARM Linux admin @ 2020-02-10 16:15 UTC (permalink / raw)
  To: Olof Johansson
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon, linux-kernel,
	Leo Li, M.h. Lian, robh+dt, Xiaowei Bao, catalin.marinas,
	bhelgaas, andrew.murray, shawnguo, Mingkai Hu, linux-arm-kernel

On Mon, Feb 10, 2020 at 04:28:23PM +0100, Olof Johansson wrote:
> On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
> <linux@armlinux.org.uk> wrote:
> >
> > On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > > >
> > > > Hi Olof,
> > > >
> > > > Thanks a lot for your comments!
> > > > And sorry for my delay respond!
> > >
> > > Actually, they apply with only minor conflicts on top of current -next.
> > >
> > > Bjorn, any chance we can get you to pick these up pretty soon? They
> > > enable full use of a promising ARM developer system, the SolidRun
> > > HoneyComb, and would be quite valuable for me and others to be able to
> > > use with mainline or -next without any additional patches applied --
> > > which this patchset achieves.
> > >
> > > I know there are pending revisions based on feedback. I'll leave it up
> > > to you and others to determine if that can be done with incremental
> > > patches on top, or if it should be fixed before the initial patchset
> > > is applied. But all in all, it's holding up adaption by me and surely
> > > others of a very interesting platform -- I'm looking to replace my
> > > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > > before I do.
> >
> > If you're going to be using NVMe, make sure you use a power-fail safe
> > version; I've already had one instance where ext4 failed to mount
> > because of a corrupted journal using an XPG SX8200 after the Honeycomb
> > Serror'd, and then I powered it down after a few hours before later
> > booting it back up.
> >
> > EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> > EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> > JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> > EXT4-fs (nvme0n1p2): error loading journal
> 
> Hmm, using btrfs on mine, not sure if the exposure is similar or not.

As I understand the problem, it isn't a filesystem issue.  It's a data
integrity issue with the NVMe over power fail, how they cache the data,
and ultimately write it to the nand flash.

Have a read of:

https://www.kingston.com/en/solutions/servers-data-centers/ssd-power-loss-protection

As NVMe and SSD are basically the same underlying technology (the host
interface is different) and the issues I've heard, and now experienced
with my NVMe, I think the above is a good pointer to the problems of
flash mass storage.

As I understand it, the problem occurs when the mapping table has not
been written back to flash, power is lost without the Standby Immediate
command being sent, and there is no way for the firmware to quickly
save the table.  On subsequent power up, the firmware has to
reconstruct the mapping table, and depending on how that is done,
incorrect (old?) data may be returned for some blocks.

That can happen to any blocks on the drive, which means any data can
be at risk from a power loss event, whether that is a power failure
or after a crash.

> Do you know if the SErr was due to a known issue and/or if it's
> something that's fixed in production silicon?

The SError is triggered by something on the PCIe side of things; if I
leave the Mellanox PCIe card out, then I don't get them.  The errata
patches I have merged into my tree help a bit, turning the code from
being unable to boot without a SError with the card plugged in, to
being able to boot and last a while - but the SErrors still eventually
come, maybe taking a few days... and that's without the Mellanox
ethernet interface being up.

> (I still can't enable SMMU since across a warm reboot it fails
> *completely*, with nothing coming up and working. NXP folks, you
> listening? :)

Is it just a warm reboot?  I thought I saw SMMU activity on a cold
boot as well, implying that there were devices active that Linux
did not know about.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 16:15                 ` Russell King - ARM Linux admin
@ 2020-02-10 17:20                   ` Russell King - ARM Linux admin
  2020-02-10 18:33                   ` Olof Johansson
  1 sibling, 0 replies; 55+ messages in thread
From: Russell King - ARM Linux admin @ 2020-02-10 17:20 UTC (permalink / raw)
  To: Olof Johansson
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon, linux-kernel,
	Leo Li, M.h. Lian, robh+dt, Xiaowei Bao, catalin.marinas,
	bhelgaas, andrew.murray, shawnguo, Mingkai Hu, linux-arm-kernel

On Mon, Feb 10, 2020 at 04:15:53PM +0000, Russell King - ARM Linux admin wrote:
> On Mon, Feb 10, 2020 at 04:28:23PM +0100, Olof Johansson wrote:
> > On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
> > <linux@armlinux.org.uk> wrote:
> > >
> > > On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > > > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > > > >
> > > > > Hi Olof,
> > > > >
> > > > > Thanks a lot for your comments!
> > > > > And sorry for my delay respond!
> > > >
> > > > Actually, they apply with only minor conflicts on top of current -next.
> > > >
> > > > Bjorn, any chance we can get you to pick these up pretty soon? They
> > > > enable full use of a promising ARM developer system, the SolidRun
> > > > HoneyComb, and would be quite valuable for me and others to be able to
> > > > use with mainline or -next without any additional patches applied --
> > > > which this patchset achieves.
> > > >
> > > > I know there are pending revisions based on feedback. I'll leave it up
> > > > to you and others to determine if that can be done with incremental
> > > > patches on top, or if it should be fixed before the initial patchset
> > > > is applied. But all in all, it's holding up adaption by me and surely
> > > > others of a very interesting platform -- I'm looking to replace my
> > > > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > > > before I do.
> > >
> > > If you're going to be using NVMe, make sure you use a power-fail safe
> > > version; I've already had one instance where ext4 failed to mount
> > > because of a corrupted journal using an XPG SX8200 after the Honeycomb
> > > Serror'd, and then I powered it down after a few hours before later
> > > booting it back up.
> > >
> > > EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> > > EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> > > JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> > > EXT4-fs (nvme0n1p2): error loading journal
> > 
> > Hmm, using btrfs on mine, not sure if the exposure is similar or not.
> 
> As I understand the problem, it isn't a filesystem issue.  It's a data
> integrity issue with the NVMe over power fail, how they cache the data,
> and ultimately write it to the nand flash.
> 
> Have a read of:
> 
> https://www.kingston.com/en/solutions/servers-data-centers/ssd-power-loss-protection

This was the link I was actually looking for:

http://industrial.adata.com/en/technology/92

but there's also:

http://industrial.adata.com/en/technology/26

ADATA make the XPG SX8200:

NVME Identify Controller:
vid       : 0x1cc1
ssvid     : 0x1cc1
mn        : ADATA SX8200PNP
fr        : R0906I

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 16:15                 ` Russell King - ARM Linux admin
  2020-02-10 17:20                   ` Russell King - ARM Linux admin
@ 2020-02-10 18:33                   ` Olof Johansson
  1 sibling, 0 replies; 55+ messages in thread
From: Olof Johansson @ 2020-02-10 18:33 UTC (permalink / raw)
  To: Russell King - ARM Linux admin
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon, linux-kernel,
	Leo Li, M.h. Lian, robh+dt, Xiaowei Bao, catalin.marinas,
	bhelgaas, andrew.murray, shawnguo, Mingkai Hu, linux-arm-kernel,
	honeycomb-users

[cc:ing honeycomb-users, didn't think of that earlier]

On Mon, Feb 10, 2020 at 5:16 PM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Mon, Feb 10, 2020 at 04:28:23PM +0100, Olof Johansson wrote:
> > On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
> > <linux@armlinux.org.uk> wrote:
> > >
> > > On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > > > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > > > >
> > > > > Hi Olof,
> > > > >
> > > > > Thanks a lot for your comments!
> > > > > And sorry for my delay respond!
> > > >
> > > > Actually, they apply with only minor conflicts on top of current -next.
> > > >
> > > > Bjorn, any chance we can get you to pick these up pretty soon? They
> > > > enable full use of a promising ARM developer system, the SolidRun
> > > > HoneyComb, and would be quite valuable for me and others to be able to
> > > > use with mainline or -next without any additional patches applied --
> > > > which this patchset achieves.
> > > >
> > > > I know there are pending revisions based on feedback. I'll leave it up
> > > > to you and others to determine if that can be done with incremental
> > > > patches on top, or if it should be fixed before the initial patchset
> > > > is applied. But all in all, it's holding up adaption by me and surely
> > > > others of a very interesting platform -- I'm looking to replace my
> > > > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > > > before I do.
> > >
> > > If you're going to be using NVMe, make sure you use a power-fail safe
> > > version; I've already had one instance where ext4 failed to mount
> > > because of a corrupted journal using an XPG SX8200 after the Honeycomb
> > > Serror'd, and then I powered it down after a few hours before later
> > > booting it back up.
> > >
> > > EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> > > EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> > > JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> > > EXT4-fs (nvme0n1p2): error loading journal
> >
> > Hmm, using btrfs on mine, not sure if the exposure is similar or not.
>
> As I understand the problem, it isn't a filesystem issue.  It's a data
> integrity issue with the NVMe over power fail, how they cache the data,
> and ultimately write it to the nand flash.
>
> Have a read of:
>
> https://www.kingston.com/en/solutions/servers-data-centers/ssd-power-loss-protection
>
> As NVMe and SSD are basically the same underlying technology (the host
> interface is different) and the issues I've heard, and now experienced
> with my NVMe, I think the above is a good pointer to the problems of
> flash mass storage.
>
> As I understand it, the problem occurs when the mapping table has not
> been written back to flash, power is lost without the Standby Immediate
> command being sent, and there is no way for the firmware to quickly
> save the table.  On subsequent power up, the firmware has to
> reconstruct the mapping table, and depending on how that is done,
> incorrect (old?) data may be returned for some blocks.
>
> That can happen to any blocks on the drive, which means any data can
> be at risk from a power loss event, whether that is a power failure
> or after a crash.

Makes me suspect if there's some board-level power/reset sequencing
issue, or if there's a problem with one card going down disabling
others. I haven't read the specs enough to know what's expected
behavior but I've seen similar issues on other platforms so take it
with a grain of salt.

> > Do you know if the SErr was due to a known issue and/or if it's
> > something that's fixed in production silicon?
>
> The SError is triggered by something on the PCIe side of things; if I
> leave the Mellanox PCIe card out, then I don't get them.  The errata
> patches I have merged into my tree help a bit, turning the code from
> being unable to boot without a SError with the card plugged in, to
> being able to boot and last a while - but the SErrors still eventually
> come, maybe taking a few days... and that's without the Mellanox
> ethernet interface being up.
>
> > (I still can't enable SMMU since across a warm reboot it fails
> > *completely*, with nothing coming up and working. NXP folks, you
> > listening? :)
>
> Is it just a warm reboot?  I thought I saw SMMU activity on a cold
> boot as well, implying that there were devices active that Linux
> did not know about.

Yeah, 100% reproducible on warm reboot -- every single time. Not on
cold boot though (100% success rate as far as I remember). I boot with
kernel on NVMe on PCIe, native 1GbE for networking. u-boot from SD
card.

This is with the SolidRun u-boot from GitHub.


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 15:28               ` Olof Johansson
  2020-02-10 16:15                 ` Russell King - ARM Linux admin
@ 2020-02-10 18:41                 ` Li Yang
  2020-02-10 19:48                   ` Li Yang
  2020-02-11 12:13                   ` Laurentiu Tudor
  1 sibling, 2 replies; 55+ messages in thread
From: Li Yang @ 2020-02-10 18:41 UTC (permalink / raw)
  To: Olof Johansson, Laurentiu Tudor
  Cc: Russell King - ARM Linux admin, Z.q. Hou, bhelgaas, mark.rutland,
	devicetree, Lorenzo Pieralisi, m.karthikeyan, arnd, linux-pci,
	l.subrahmanya, will.deacon, linux-kernel, M.h. Lian, robh+dt,
	Mingkai Hu, catalin.marinas, Xiaowei Bao, andrew.murray,
	shawnguo, linux-arm-kernel

On Mon, Feb 10, 2020 at 9:32 AM Olof Johansson <olof@lixom.net> wrote:
>
> On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
> <linux@armlinux.org.uk> wrote:
> >
> > On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > > >
> > > > Hi Olof,
> > > >
> > > > Thanks a lot for your comments!
> > > > And sorry for my delay respond!
> > >
> > > Actually, they apply with only minor conflicts on top of current -next.
> > >
> > > Bjorn, any chance we can get you to pick these up pretty soon? They
> > > enable full use of a promising ARM developer system, the SolidRun
> > > HoneyComb, and would be quite valuable for me and others to be able to
> > > use with mainline or -next without any additional patches applied --
> > > which this patchset achieves.
> > >
> > > I know there are pending revisions based on feedback. I'll leave it up
> > > to you and others to determine if that can be done with incremental
> > > patches on top, or if it should be fixed before the initial patchset
> > > is applied. But all in all, it's holding up adaption by me and surely
> > > others of a very interesting platform -- I'm looking to replace my
> > > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > > before I do.
> >
> > If you're going to be using NVMe, make sure you use a power-fail safe
> > version; I've already had one instance where ext4 failed to mount
> > because of a corrupted journal using an XPG SX8200 after the Honeycomb
> > Serror'd, and then I powered it down after a few hours before later
> > booting it back up.
> >
> > EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> > EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> > JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> > EXT4-fs (nvme0n1p2): error loading journal
>
> Hmm, using btrfs on mine, not sure if the exposure is similar or not.
>
> Do you know if the SErr was due to a known issue and/or if it's
> something that's fixed in production silicon?
>
> (I still can't enable SMMU since across a warm reboot it fails
> *completely*, with nothing coming up and working. NXP folks, you
> listening? :)

This is a known issue about DPAA2 MC bus not working well with SMMU
based IO mapping.  Adding Laurentiu to the chain who has been looking
into this issue.

Regards,
Leo

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 18:41                 ` Li Yang
@ 2020-02-10 19:48                   ` Li Yang
  2020-02-11 12:13                   ` Laurentiu Tudor
  1 sibling, 0 replies; 55+ messages in thread
From: Li Yang @ 2020-02-10 19:48 UTC (permalink / raw)
  To: Olof Johansson, Laurentiu Tudor
  Cc: Russell King - ARM Linux admin, Z.q. Hou, bhelgaas, mark.rutland,
	devicetree, Lorenzo Pieralisi, m.karthikeyan, arnd, linux-pci,
	l.subrahmanya, will.deacon, linux-kernel, M.h. Lian, robh+dt,
	Mingkai Hu, catalin.marinas, Xiaowei Bao, andrew.murray,
	shawnguo, linux-arm-kernel

On Mon, Feb 10, 2020 at 12:41 PM Li Yang <leoyang.li@nxp.com> wrote:
>
> On Mon, Feb 10, 2020 at 9:32 AM Olof Johansson <olof@lixom.net> wrote:
> >
> > On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
> > <linux@armlinux.org.uk> wrote:
> > >
> > > On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
> > > > On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
> > > > >
> > > > > Hi Olof,
> > > > >
> > > > > Thanks a lot for your comments!
> > > > > And sorry for my delay respond!
> > > >
> > > > Actually, they apply with only minor conflicts on top of current -next.
> > > >
> > > > Bjorn, any chance we can get you to pick these up pretty soon? They
> > > > enable full use of a promising ARM developer system, the SolidRun
> > > > HoneyComb, and would be quite valuable for me and others to be able to
> > > > use with mainline or -next without any additional patches applied --
> > > > which this patchset achieves.
> > > >
> > > > I know there are pending revisions based on feedback. I'll leave it up
> > > > to you and others to determine if that can be done with incremental
> > > > patches on top, or if it should be fixed before the initial patchset
> > > > is applied. But all in all, it's holding up adaption by me and surely
> > > > others of a very interesting platform -- I'm looking to replace my
> > > > aging MacchiatoBin with one of these and would need PCIe/NVMe to work
> > > > before I do.
> > >
> > > If you're going to be using NVMe, make sure you use a power-fail safe
> > > version; I've already had one instance where ext4 failed to mount
> > > because of a corrupted journal using an XPG SX8200 after the Honeycomb
> > > Serror'd, and then I powered it down after a few hours before later
> > > booting it back up.
> > >
> > > EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
> > > EXT4-fs (nvme0n1p2): write access will be enabled during recovery
> > > JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
> > > EXT4-fs (nvme0n1p2): error loading journal
> >
> > Hmm, using btrfs on mine, not sure if the exposure is similar or not.
> >
> > Do you know if the SErr was due to a known issue and/or if it's
> > something that's fixed in production silicon?
> >
> > (I still can't enable SMMU since across a warm reboot it fails
> > *completely*, with nothing coming up and working. NXP folks, you
> > listening? :)
>
> This is a known issue about DPAA2 MC bus not working well with SMMU
> based IO mapping.  Adding Laurentiu to the chain who has been looking
> into this issue.

Forgot to mention that you can workaround the issue by setting
CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT=n or adding
"arm-smmu.disable_bypass=0" to boot parameters.

Regards,
Leo

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-10 18:41                 ` Li Yang
  2020-02-10 19:48                   ` Li Yang
@ 2020-02-11 12:13                   ` Laurentiu Tudor
  2020-02-11 13:04                     ` Robin Murphy
  1 sibling, 1 reply; 55+ messages in thread
From: Laurentiu Tudor @ 2020-02-11 12:13 UTC (permalink / raw)
  To: Li Yang, Olof Johansson
  Cc: Russell King - ARM Linux admin, Z.q. Hou, bhelgaas, mark.rutland,
	devicetree, Lorenzo Pieralisi, m.karthikeyan, arnd, linux-pci,
	l.subrahmanya, will.deacon, linux-kernel, M.h. Lian, robh+dt,
	Mingkai Hu, catalin.marinas, Xiaowei Bao, andrew.murray,
	shawnguo, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3051 bytes --]



On 10.02.2020 20:41, Li Yang wrote:
> On Mon, Feb 10, 2020 at 9:32 AM Olof Johansson <olof@lixom.net> wrote:
>>
>> On Mon, Feb 10, 2020 at 4:23 PM Russell King - ARM Linux admin
>> <linux@armlinux.org.uk> wrote:
>>>
>>> On Mon, Feb 10, 2020 at 04:12:30PM +0100, Olof Johansson wrote:
>>>> On Thu, Feb 6, 2020 at 11:57 AM Z.q. Hou <zhiqiang.hou@nxp.com> wrote:
>>>>>
>>>>> Hi Olof,
>>>>>
>>>>> Thanks a lot for your comments!
>>>>> And sorry for my delay respond!
>>>>
>>>> Actually, they apply with only minor conflicts on top of current -next.
>>>>
>>>> Bjorn, any chance we can get you to pick these up pretty soon? They
>>>> enable full use of a promising ARM developer system, the SolidRun
>>>> HoneyComb, and would be quite valuable for me and others to be able to
>>>> use with mainline or -next without any additional patches applied --
>>>> which this patchset achieves.
>>>>
>>>> I know there are pending revisions based on feedback. I'll leave it up
>>>> to you and others to determine if that can be done with incremental
>>>> patches on top, or if it should be fixed before the initial patchset
>>>> is applied. But all in all, it's holding up adaption by me and surely
>>>> others of a very interesting platform -- I'm looking to replace my
>>>> aging MacchiatoBin with one of these and would need PCIe/NVMe to work
>>>> before I do.
>>>
>>> If you're going to be using NVMe, make sure you use a power-fail safe
>>> version; I've already had one instance where ext4 failed to mount
>>> because of a corrupted journal using an XPG SX8200 after the Honeycomb
>>> Serror'd, and then I powered it down after a few hours before later
>>> booting it back up.
>>>
>>> EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
>>> EXT4-fs (nvme0n1p2): write access will be enabled during recovery
>>> JBD2: journal transaction 80849 on nvme0n1p2-8 is corrupt.
>>> EXT4-fs (nvme0n1p2): error loading journal
>>
>> Hmm, using btrfs on mine, not sure if the exposure is similar or not.
>>
>> Do you know if the SErr was due to a known issue and/or if it's
>> something that's fixed in production silicon?
>>
>> (I still can't enable SMMU since across a warm reboot it fails
>> *completely*, with nothing coming up and working. NXP folks, you
>> listening? :)
> 
> This is a known issue about DPAA2 MC bus not working well with SMMU
> based IO mapping.  Adding Laurentiu to the chain who has been looking
> into this issue.

Yes, I'm closely following the issue. I actually have a workaround 
(attached) but haven't submitted as it will probably raise a lot of 
eyebrows. In the mean time I'm following some discussions [1][2][3] on 
the iommu list which seem to try to tackle what appears to be a similar 
issue but with framebuffers. My hope is that we will be able to leverage 
whatever turns out.
In the mean time, can you try the workaround Leo suggested?

[1] https://patchwork.kernel.org/patch/11327667/
[2] https://patchwork.kernel.org/patch/10967729/
[3] https://patchwork.kernel.org/cover/11279577/

---
Best Regards, Laurentiu

[-- Attachment #2: 0001-arm64-dts-lx2160a-add-iommus-property-for-mc-node.patch --]
[-- Type: text/x-patch, Size: 1023 bytes --]

From 75dcd4a7bdf51db65dc5553a255b277f9d126e30 Mon Sep 17 00:00:00 2001
From: Laurentiu Tudor <laurentiu.tudor@nxp.com>
Date: Tue, 19 Nov 2019 17:01:39 +0200
Subject: [PATCH 1/2] arm64: dts: lx2160a: add iommus property for mc node
Content-Type: text/plain; charset="us-ascii"

Enable SMMU management for the MC firmware by adding the required
iommus property in the device tree node.

Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
---
 arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
index b032f3890c8c..f46f0d0905b5 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
@@ -868,6 +868,7 @@
 			msi-parent = <&its>;
 			/* iommu-map property is fixed up by u-boot */
 			iommu-map = <0 &smmu 0 0>;
+			iommus = <&smmu 0x4000>;
 			dma-coherent;
 			#address-cells = <3>;
 			#size-cells = <1>;
-- 
2.17.1


[-- Attachment #3: 0002-bus-fsl-mc-make-mc-work-with-smmu-disable-bypass-on.patch --]
[-- Type: text/x-patch, Size: 4146 bytes --]

From 46ccd2291e259c906b449f789ee62e03598fe4d7 Mon Sep 17 00:00:00 2001
From: Laurentiu Tudor <laurentiu.tudor@nxp.com>
Date: Tue, 1 Oct 2019 16:22:48 +0300
Subject: [PATCH 2/2] bus: fsl-mc: make mc work with smmu disable bypass on
Content-Type: text/plain; charset="us-ascii"

Since this commit [1] booting kernel on MC based chips started to
fail because this firmware starts before the kernel and as soon as
SMMU is probed it starts to trigger contiguous faults.
This is a workaround that allows MC firmware to run with SMMU
in disable bypass mode. It consists of the following steps:
 1. pause the firmware at early boot to get a chance to setup SMMU
 2. request direct mapping for MC device
 3. resume the firmware
The workaround relies on the fact that no state is lost when
pausing / resuming firmware, as per the docs.
With this patch, platforms with MC firmware can now boot without
having to change the default config to set:
  CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT=n

[1] 954a03be033 ("iommu/arm-smmu: Break insecure users by disabling bypass by default")

Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
---
 drivers/bus/fsl-mc/fsl-mc-bus.c | 51 +++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/drivers/bus/fsl-mc/fsl-mc-bus.c b/drivers/bus/fsl-mc/fsl-mc-bus.c
index fec394a28891..f9bc9c384ab5 100644
--- a/drivers/bus/fsl-mc/fsl-mc-bus.c
+++ b/drivers/bus/fsl-mc/fsl-mc-bus.c
@@ -18,6 +18,8 @@
 #include <linux/bitops.h>
 #include <linux/msi.h>
 #include <linux/dma-mapping.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
 
 #include "fsl-mc-private.h"
 
@@ -889,6 +891,12 @@ static int get_mc_addr_translation_ranges(struct device *dev,
 	return 0;
 }
 
+#define FSL_MC_GCR1	0x0
+#define GCR1_P1_STOP	BIT(31)
+
+static u32 boot_gcr1;
+static void __iomem *fsl_mc_regs;
+
 /**
  * fsl_mc_bus_probe - callback invoked when the root MC bus is being
  * added
@@ -906,6 +914,19 @@ static int fsl_mc_bus_probe(struct platform_device *pdev)
 	struct mc_version mc_version;
 	struct resource res;
 
+	/*
+	 * The MC firmware requires full access to the whole address space plus
+	 * it has no way of dealing with any kind of address translation, so
+	 * request direct mapping for it.
+	 */
+	error = iommu_request_dm_for_dev(&pdev->dev);
+	if (error)
+		dev_warn(&pdev->dev, "iommu_request_dm_for_dev() failed: %d\n",
+			 error);
+
+	/* Resume the firmware */
+	writel(boot_gcr1 & ~GCR1_P1_STOP, fsl_mc_regs + FSL_MC_GCR1);
+
 	mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
 	if (!mc)
 		return -ENOMEM;
@@ -990,6 +1011,13 @@ static int fsl_mc_bus_remove(struct platform_device *pdev)
 	if (!fsl_mc_is_root_dprc(&mc->root_mc_bus_dev->dev))
 		return -EINVAL;
 
+	/*
+	 * Pause back the firmware so that it doesn't trigger faults by the
+	 * time SMMU gets brought down.
+	 */
+	writel(boot_gcr1 | GCR1_P1_STOP, fsl_mc_regs + FSL_MC_GCR1);
+	iounmap(fsl_mc_regs);
+
 	fsl_mc_device_remove(mc->root_mc_bus_dev);
 
 	fsl_destroy_mc_io(mc->root_mc_bus_dev->mc_io);
@@ -1018,6 +1046,8 @@ static struct platform_driver fsl_mc_bus_driver = {
 static int __init fsl_mc_bus_driver_init(void)
 {
 	int error;
+	struct device_node *np;
+	struct resource res;
 
 	error = bus_register(&fsl_mc_bus_type);
 	if (error < 0) {
@@ -1039,9 +1069,30 @@ static int __init fsl_mc_bus_driver_init(void)
 	if (error < 0)
 		goto error_cleanup_dprc_driver;
 
+	np = of_find_matching_node(NULL, fsl_mc_bus_match_table);
+	if (!of_device_is_available(np))
+		goto error_cleanup_dprc_driver;
+	error = of_address_to_resource(np, 1, &res);
+	if (error)
+		goto error_cleanup_dprc_driver;
+	fsl_mc_regs = ioremap(res.start, resource_size(&res));
+	if (!fsl_mc_regs) {
+		error = -ENXIO;
+		goto error_cleanup_dprc_driver;
+	}
+
+	boot_gcr1 = readl(fsl_mc_regs + FSL_MC_GCR1);
+	/*
+	 * If found running, pause MC firmware in order to get a chance
+	 * to setup SMMU for it.
+	 */
+	if (!(boot_gcr1 & GCR1_P1_STOP))
+		writel(boot_gcr1 | GCR1_P1_STOP,  fsl_mc_regs + FSL_MC_GCR1);
+
 	return 0;
 
 error_cleanup_dprc_driver:
+	iounmap(fsl_mc_regs);
 	dprc_driver_exit();
 
 error_cleanup_driver:
-- 
2.17.1


^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-11 12:13                   ` Laurentiu Tudor
@ 2020-02-11 13:04                     ` Robin Murphy
  2020-02-11 13:55                       ` Laurentiu Tudor
  2020-02-11 14:48                       ` Olof Johansson
  0 siblings, 2 replies; 55+ messages in thread
From: Robin Murphy @ 2020-02-11 13:04 UTC (permalink / raw)
  To: Laurentiu Tudor, Li Yang, Olof Johansson
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon,
	Russell King - ARM Linux admin, linux-kernel, M.h. Lian, robh+dt,
	Xiaowei Bao, catalin.marinas, bhelgaas, andrew.murray, shawnguo,
	Mingkai Hu, linux-arm-kernel

On 2020-02-11 12:13 pm, Laurentiu Tudor wrote:
[...]
>> This is a known issue about DPAA2 MC bus not working well with SMMU
>> based IO mapping.  Adding Laurentiu to the chain who has been looking
>> into this issue.
> 
> Yes, I'm closely following the issue. I actually have a workaround 
> (attached) but haven't submitted as it will probably raise a lot of 
> eyebrows. In the mean time I'm following some discussions [1][2][3] on 
> the iommu list which seem to try to tackle what appears to be a similar 
> issue but with framebuffers. My hope is that we will be able to leverage 
> whatever turns out.

Indeed it's more general than framebuffers - in fact there was a 
specific requirement from the IORT side to accommodate network/storage 
controllers with in-memory firmware/configuration data/whatever set up 
by the bootloader that want to be handed off 'live' to Linux because the 
overhead of stopping and restarting them is impractical. Thus this DPAA2 
setup is very much within scope of the desired solution, so please feel 
free to join in (particularly on the DT parts) :)

As for right now, note that your patch would only be a partial 
mitigation to slightly reduce the fault window but not remove it 
entirely. To be robust the SMMU driver *has* to know about live streams 
before the first arm_smmu_reset() - hence the need for generic firmware 
bindings - so doing anything from the MC driver is already too late (and 
indeed the current iommu_request_dm_for_dev() mechanism is itself a 
microcosm of the same problem).

> In the mean time, can you try the workaround Leo suggested?

Agreed, I'd imagine the command-line option is probably the best choice 
for these platforms, since it's likely to be easier to set that by 
default in the bootloader than faff with rebuilding generic kernel configs.

Robin.

> [1] https://patchwork.kernel.org/patch/11327667/
> [2] https://patchwork.kernel.org/patch/10967729/
> [3] https://patchwork.kernel.org/cover/11279577/
> 
> ---
> Best Regards, Laurentiu
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-11 13:04                     ` Robin Murphy
@ 2020-02-11 13:55                       ` Laurentiu Tudor
  2020-02-11 14:51                         ` Robin Murphy
  2020-02-11 14:48                       ` Olof Johansson
  1 sibling, 1 reply; 55+ messages in thread
From: Laurentiu Tudor @ 2020-02-11 13:55 UTC (permalink / raw)
  To: Robin Murphy, Li Yang, Olof Johansson
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon,
	Russell King - ARM Linux admin, linux-kernel, M.h. Lian, robh+dt,
	Xiaowei Bao, catalin.marinas, bhelgaas, andrew.murray, shawnguo,
	Mingkai Hu, linux-arm-kernel, Diana Madalina Craciun



On 11.02.2020 15:04, Robin Murphy wrote:
> On 2020-02-11 12:13 pm, Laurentiu Tudor wrote:
> [...]
>>> This is a known issue about DPAA2 MC bus not working well with SMMU
>>> based IO mapping.  Adding Laurentiu to the chain who has been looking
>>> into this issue.
>>
>> Yes, I'm closely following the issue. I actually have a workaround 
>> (attached) but haven't submitted as it will probably raise a lot of 
>> eyebrows. In the mean time I'm following some discussions [1][2][3] on 
>> the iommu list which seem to try to tackle what appears to be a 
>> similar issue but with framebuffers. My hope is that we will be able 
>> to leverage whatever turns out.
> 
> Indeed it's more general than framebuffers - in fact there was a 
> specific requirement from the IORT side to accommodate network/storage 
> controllers with in-memory firmware/configuration data/whatever set up 
> by the bootloader that want to be handed off 'live' to Linux because the 
> overhead of stopping and restarting them is impractical. Thus this DPAA2 
> setup is very much within scope of the desired solution, so please feel 
> free to join in (particularly on the DT parts) :)

Will sure do. Seems that the 2nd approach (the one with list of 
compatibles in arm-smmu) fits really well with our scenario. Will this 
be the way to go forward?

> As for right now, note that your patch would only be a partial 
> mitigation to slightly reduce the fault window but not remove it 
> entirely. To be robust the SMMU driver *has* to know about live streams 
> before the first arm_smmu_reset() - hence the need for generic firmware 
> bindings - so doing anything from the MC driver is already too late (and 
> indeed the current iommu_request_dm_for_dev() mechanism is itself a 
> microcosm of the same problem).

I think you might have missed in the patch that it pauses the firmware 
at early boot, in its driver init and it resumes it only after 
iommu_request_dm_for_dev() has completed. :)

---
Best Regards, Laurentiu

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-11 13:04                     ` Robin Murphy
  2020-02-11 13:55                       ` Laurentiu Tudor
@ 2020-02-11 14:48                       ` Olof Johansson
  2020-02-11 15:14                         ` Laurentiu Tudor
  1 sibling, 1 reply; 55+ messages in thread
From: Olof Johansson @ 2020-02-11 14:48 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Laurentiu Tudor, Li Yang, mark.rutland, devicetree,
	Lorenzo Pieralisi, arnd, m.karthikeyan, linux-pci, Z.q. Hou,
	l.subrahmanya, will.deacon, Russell King - ARM Linux admin,
	linux-kernel, M.h. Lian, robh+dt, Xiaowei Bao, catalin.marinas,
	bhelgaas, andrew.murray, shawnguo, Mingkai Hu, linux-arm-kernel

On Tue, Feb 11, 2020 at 5:04 AM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2020-02-11 12:13 pm, Laurentiu Tudor wrote:
> [...]
> >> This is a known issue about DPAA2 MC bus not working well with SMMU
> >> based IO mapping.  Adding Laurentiu to the chain who has been looking
> >> into this issue.
> >
> > Yes, I'm closely following the issue. I actually have a workaround
> > (attached) but haven't submitted as it will probably raise a lot of
> > eyebrows. In the mean time I'm following some discussions [1][2][3] on
> > the iommu list which seem to try to tackle what appears to be a similar
> > issue but with framebuffers. My hope is that we will be able to leverage
> > whatever turns out.
>
> Indeed it's more general than framebuffers - in fact there was a
> specific requirement from the IORT side to accommodate network/storage
> controllers with in-memory firmware/configuration data/whatever set up
> by the bootloader that want to be handed off 'live' to Linux because the
> overhead of stopping and restarting them is impractical. Thus this DPAA2
> setup is very much within scope of the desired solution, so please feel
> free to join in (particularly on the DT parts) :)

That's a real problem that nees a solution, but that's not what's
happening here, since cold boots works fine.

Isn't it a whole lot more likely that something isn't
reset/reinitialized properly in u-boot, such that there is lingering
state in the setup, causing this?

> As for right now, note that your patch would only be a partial
> mitigation to slightly reduce the fault window but not remove it
> entirely. To be robust the SMMU driver *has* to know about live streams
> before the first arm_smmu_reset() - hence the need for generic firmware
> bindings - so doing anything from the MC driver is already too late (and
> indeed the current iommu_request_dm_for_dev() mechanism is itself a
> microcosm of the same problem).

This is more likely a live stream that's left behind from the previous
kernel (there are some error messages about being unable to detach
domains, but the errors make it hard to tell what driver didn't unbind
enough).

*BUT*, even with that bug, the system should reboot reliably and come
up clean. So, something isn't clearing up the state *on boot*.

> > In the mean time, can you try the workaround Leo suggested?
>
> Agreed, I'd imagine the command-line option is probably the best choice
> for these platforms, since it's likely to be easier to set that by
> default in the bootloader than faff with rebuilding generic kernel configs.

For the generic user, definitely. I'll give it a go later this week
when I have a bit more spare time with the device physically present.


-Olof

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-11 13:55                       ` Laurentiu Tudor
@ 2020-02-11 14:51                         ` Robin Murphy
  0 siblings, 0 replies; 55+ messages in thread
From: Robin Murphy @ 2020-02-11 14:51 UTC (permalink / raw)
  To: Laurentiu Tudor, Li Yang, Olof Johansson
  Cc: mark.rutland, devicetree, Lorenzo Pieralisi, arnd, m.karthikeyan,
	linux-pci, Z.q. Hou, l.subrahmanya, will.deacon,
	Russell King - ARM Linux admin, linux-kernel, M.h. Lian, robh+dt,
	Xiaowei Bao, catalin.marinas, bhelgaas, andrew.murray, shawnguo,
	Mingkai Hu, linux-arm-kernel, Diana Madalina Craciun

On 11/02/2020 1:55 pm, Laurentiu Tudor wrote:
> 
> 
> On 11.02.2020 15:04, Robin Murphy wrote:
>> On 2020-02-11 12:13 pm, Laurentiu Tudor wrote:
>> [...]
>>>> This is a known issue about DPAA2 MC bus not working well with SMMU
>>>> based IO mapping.  Adding Laurentiu to the chain who has been looking
>>>> into this issue.
>>>
>>> Yes, I'm closely following the issue. I actually have a workaround 
>>> (attached) but haven't submitted as it will probably raise a lot of 
>>> eyebrows. In the mean time I'm following some discussions [1][2][3] 
>>> on the iommu list which seem to try to tackle what appears to be a 
>>> similar issue but with framebuffers. My hope is that we will be able 
>>> to leverage whatever turns out.
>>
>> Indeed it's more general than framebuffers - in fact there was a 
>> specific requirement from the IORT side to accommodate network/storage 
>> controllers with in-memory firmware/configuration data/whatever set up 
>> by the bootloader that want to be handed off 'live' to Linux because 
>> the overhead of stopping and restarting them is impractical. Thus this 
>> DPAA2 setup is very much within scope of the desired solution, so 
>> please feel free to join in (particularly on the DT parts) :)
> 
> Will sure do. Seems that the 2nd approach (the one with list of 
> compatibles in arm-smmu) fits really well with our scenario. Will this 
> be the way to go forward?

I'm hoping that Thierry's proposal can be made to work out, since it's 
closer to how the ACPI version should work, which means we would be able 
to do a lot more in shared common code rather than baking magic 
knowledge and duplicated functionality into individual IOMMU drivers.

>> As for right now, note that your patch would only be a partial 
>> mitigation to slightly reduce the fault window but not remove it 
>> entirely. To be robust the SMMU driver *has* to know about live 
>> streams before the first arm_smmu_reset() - hence the need for generic 
>> firmware bindings - so doing anything from the MC driver is already 
>> too late (and indeed the current iommu_request_dm_for_dev() mechanism 
>> is itself a microcosm of the same problem).
> 
> I think you might have missed in the patch that it pauses the firmware 
> at early boot, in its driver init and it resumes it only after 
> iommu_request_dm_for_dev() has completed. :)

Ah, from the context I missed that that was non-modular and relying on 
initcall trickery... fair enough, in that case I'll downgrade my "it's 
insufficient" to "it's ugly and somewhat fragile" :P

Thanks,
Robin.

^ permalink raw reply	[flat|nested] 55+ messages in thread

* Re: [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs
  2020-02-11 14:48                       ` Olof Johansson
@ 2020-02-11 15:14                         ` Laurentiu Tudor
  0 siblings, 0 replies; 55+ messages in thread
From: Laurentiu Tudor @ 2020-02-11 15:14 UTC (permalink / raw)
  To: Olof Johansson, Robin Murphy
  Cc: Li Yang, mark.rutland, devicetree, Lorenzo Pieralisi, arnd,
	m.karthikeyan, linux-pci, Z.q. Hou, l.subrahmanya, will.deacon,
	Russell King - ARM Linux admin, linux-kernel, M.h. Lian, robh+dt,
	Xiaowei Bao, catalin.marinas, bhelgaas, andrew.murray, shawnguo,
	Mingkai Hu, linux-arm-kernel



On 11.02.2020 16:48, Olof Johansson wrote:
> On Tue, Feb 11, 2020 at 5:04 AM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2020-02-11 12:13 pm, Laurentiu Tudor wrote:
>> [...]
>>>> This is a known issue about DPAA2 MC bus not working well with SMMU
>>>> based IO mapping.  Adding Laurentiu to the chain who has been looking
>>>> into this issue.
>>>
>>> Yes, I'm closely following the issue. I actually have a workaround
>>> (attached) but haven't submitted as it will probably raise a lot of
>>> eyebrows. In the mean time I'm following some discussions [1][2][3] on
>>> the iommu list which seem to try to tackle what appears to be a similar
>>> issue but with framebuffers. My hope is that we will be able to leverage
>>> whatever turns out.
>>
>> Indeed it's more general than framebuffers - in fact there was a
>> specific requirement from the IORT side to accommodate network/storage
>> controllers with in-memory firmware/configuration data/whatever set up
>> by the bootloader that want to be handed off 'live' to Linux because the
>> overhead of stopping and restarting them is impractical. Thus this DPAA2
>> setup is very much within scope of the desired solution, so please feel
>> free to join in (particularly on the DT parts) :)
> 
> That's a real problem that nees a solution, but that's not what's
> happening here, since cold boots works fine.
> 
> Isn't it a whole lot more likely that something isn't
> reset/reinitialized properly in u-boot, such that there is lingering
> state in the setup, causing this?

Ok, so this is completely something else. I don't think our u-boots are 
designed to run in ways other than coming from hard reset.

>> As for right now, note that your patch would only be a partial
>> mitigation to slightly reduce the fault window but not remove it
>> entirely. To be robust the SMMU driver *has* to know about live streams
>> before the first arm_smmu_reset() - hence the need for generic firmware
>> bindings - so doing anything from the MC driver is already too late (and
>> indeed the current iommu_request_dm_for_dev() mechanism is itself a
>> microcosm of the same problem).
> 
> This is more likely a live stream that's left behind from the previous
> kernel (there are some error messages about being unable to detach
> domains, but the errors make it hard to tell what driver didn't unbind
> enough).

I also noticed those messages. Perhaps our PCI driver doesn't do all the 
required cleanup.

> *BUT*, even with that bug, the system should reboot reliably and come
> up clean. So, something isn't clearing up the state *on boot*.

We do test some kexec based "soft-reset" scenarios, didn't hit your 
issue but instead we hit this:

https://lkml.org/lkml/2018/9/21/1066

Can you please provide some more info on your scenario?

---
Best Regards, Laurentiu

^ permalink raw reply	[flat|nested] 55+ messages in thread

end of thread, back to index

Thread overview: 55+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-20  3:45 [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Z.q. Hou
2019-11-20  3:45 ` [PATCHv9 01/12] PCI: mobiveil: Re-abstract the private structure Z.q. Hou
2020-01-13 10:09   ` Andrew Murray
2020-02-06 11:04     ` Z.q. Hou
2020-02-06 11:27       ` Andrew Murray
2019-11-20  3:45 ` [PATCHv9 02/12] PCI: mobiveil: Move the host initialization into a routine Z.q. Hou
2020-01-13 10:19   ` Andrew Murray
2020-02-06 11:14     ` Z.q. Hou
2019-11-20  3:45 ` [PATCHv9 03/12] PCI: mobiveil: Collect the interrupt related operations " Z.q. Hou
2020-01-13 10:34   ` Andrew Murray
2020-02-06 11:30     ` Z.q. Hou
2019-11-20  3:45 ` [PATCHv9 04/12] PCI: mobiveil: Modularize the Mobiveil PCIe Host Bridge IP driver Z.q. Hou
2020-01-13 11:05   ` Andrew Murray
2020-02-06 12:25     ` Z.q. Hou
2019-11-20  3:45 ` [PATCHv9 05/12] PCI: mobiveil: Add callback function for interrupt initialization Z.q. Hou
2020-01-13 11:19   ` Andrew Murray
2020-02-06 13:25     ` Z.q. Hou
2019-11-20  3:45 ` [PATCHv9 06/12] PCI: mobiveil: Add callback function for link up check Z.q. Hou
2020-01-13 11:22   ` Andrew Murray
2020-02-06 13:25     ` Z.q. Hou
2019-11-20  3:46 ` [PATCHv9 07/12] PCI: mobiveil: Make mobiveil_host_init() can be used to re-init host Z.q. Hou
2020-01-13 11:26   ` Andrew Murray
2020-02-06 13:27     ` Z.q. Hou
2019-11-20  3:46 ` [PATCHv9 08/12] PCI: mobiveil: Add 8-bit and 16-bit CSR register accessors Z.q. Hou
2020-01-13 11:31   ` Andrew Murray
2020-02-06 13:45     ` Z.q. Hou
2019-11-20  3:46 ` [PATCHv9 09/12] dt-bindings: PCI: Add NXP Layerscape SoCs PCIe Gen4 controller Z.q. Hou
2019-11-20  3:46 ` [PATCHv9 10/12] PCI: mobiveil: Add PCIe Gen4 RC driver for NXP Layerscape SoCs Z.q. Hou
2020-01-13 12:02   ` Andrew Murray
2020-02-06 13:45     ` Z.q. Hou
2020-02-06 14:29       ` Andrew Murray
2019-11-20  3:46 ` [PATCHv9 11/12] arm64: dts: lx2160a: Add PCIe controller DT nodes Z.q. Hou
2019-11-20  3:46 ` [PATCHv9 12/12] arm64: defconfig: Enable CONFIG_PCIE_LAYERSCAPE_GEN4 Z.q. Hou
2019-11-20  9:57 ` [PATCHv9 00/12] PCI: Recode Mobiveil driver and add PCIe Gen4 driver for NXP Layerscape SoCs Russell King - ARM Linux admin
2019-11-20 10:30   ` Z.q. Hou
2019-12-13 18:37 ` Olof Johansson
2019-12-17  2:50   ` Z.q. Hou
2020-01-10 15:33     ` Lorenzo Pieralisi
2020-01-10 17:05       ` Olof Johansson
2020-02-06 10:57         ` Z.q. Hou
2020-02-10 15:12           ` Olof Johansson
2020-02-10 15:22             ` Russell King - ARM Linux admin
2020-02-10 15:28               ` Olof Johansson
2020-02-10 16:15                 ` Russell King - ARM Linux admin
2020-02-10 17:20                   ` Russell King - ARM Linux admin
2020-02-10 18:33                   ` Olof Johansson
2020-02-10 18:41                 ` Li Yang
2020-02-10 19:48                   ` Li Yang
2020-02-11 12:13                   ` Laurentiu Tudor
2020-02-11 13:04                     ` Robin Murphy
2020-02-11 13:55                       ` Laurentiu Tudor
2020-02-11 14:51                         ` Robin Murphy
2020-02-11 14:48                       ` Olof Johansson
2020-02-11 15:14                         ` Laurentiu Tudor
2020-02-10 15:33             ` Lorenzo Pieralisi

Linux-PCI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \
		linux-pci@vger.kernel.org
	public-inbox-index linux-pci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git