linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05
@ 2015-08-17 11:55 Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 1/6] PCI: designware: move calculation of bus addresses to DRA7xx Zhou Wang
                   ` (5 more replies)
  0 siblings, 6 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

This patchset adds PCIe host support for HiSilicon SoC Hip05. The PCIe hosts
use PCIe IP core from Synopsys, So this driver is base on designware PCIe driver.

Hip05 is an ARMv8 architecture SoC. It should be able to use ARM64 PCIe API in
designeware PCIe driver. So this patch also adds ARM64 support for designware
pcie.

This patchset is based on v4.2-rc1.

Change from v6:
- Add Pratyush's Acked-by for 1/6 and 2/6.
- Add James' Tested-by for 3/6.

Change from v5:
- Merge 1/6 in this series, discussion about this can be found in [1]

Change from v4:
- Change the author of 1/5 to Gabriele.
- Modify problems in 3/5 pointed by Bjorn.
- Modify spelling problems in 4/5.

Change from v3:
- Change 1/5 to what Gabriele suggested.
- Use win->__res.start to get *_mod_base in 2/5, this fix a bug in v3 series.

Change from v2:
- Move struct pci_dev *dev and struct pci_sys_data *sys in
  pcibios_align_resource in 1/5.
- Add Gabriele's codes in 2/5 which delete unnecessary information parse and
  use of_pci_get_host_bridge_resources for both ARM32 and ARM64.
- Add maintainer patch 5/5.

Change from RFC v1:
- Add 1/4 patch by Arnd which removes align_resource callback in ARM
  pcibios_align_resource.
- Change head file in pcie-designware from asm/hardirq.h to linux/hardirq.h.
- Set pp->root_bus_nr = 0 in dra7xx, exynos, imx6, keystone, layerscape,
  spear13xx.
- Remove unnecessary parentheses of some macros in pcie-hisi.
- Use macro to replace some magic values.
- Merge two loops together and add some comments about it in context_config
  function in pcie-hisi.
- Modify some value of items in pcie node example in binding document. 

Change from RFC:
- delete dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
  merge related operations into dw_pcie_host_init.

Link of v6:
- http://www.spinics.net/lists/linux-pci/msg43669.html
Link of v5:
- http://www.spinics.net/lists/devicetree/msg87959.html
Link of v4:
- http://www.spinics.net/lists/arm-kernel/msg433050.html
Link of v3:
- http://www.spinics.net/lists/linux-pci/msg42539.html
Link of v2:
- http://www.spinics.net/lists/linux-pci/msg41844.html
Link of RFC v1:
- http://www.spinics.net/lists/linux-pci/msg41305.html
Link of RFC:
- http://www.spinics.net/lists/linux-pci/msg40434.html

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-July/359741.html

Zhou Wang (4):
  PCI: designware: Add ARM64 support
  PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05
  Documentation: DT: Add HiSilicon PCIe host binding
  MAINTAINERS: Add pcie-hisi maintainer

gabriele paoloni (2):
  PCI: designware: move calculation of bus addresses to DRA7xx
  ARM/PCI: remove align_resource in pci_sys_data

 .../devicetree/bindings/pci/hisilicon-pcie.txt     |  46 ++++
 MAINTAINERS                                        |   7 +
 arch/arm/include/asm/mach/pci.h                    |   5 -
 arch/arm/kernel/bios32.c                           |  12 +-
 drivers/pci/host/Kconfig                           |   8 +
 drivers/pci/host/Makefile                          |   1 +
 drivers/pci/host/pci-dra7xx.c                      |  14 ++
 drivers/pci/host/pci-exynos.c                      |   2 +-
 drivers/pci/host/pci-imx6.c                        |   2 +-
 drivers/pci/host/pci-keystone-dw.c                 |   2 +-
 drivers/pci/host/pci-keystone.c                    |   2 +-
 drivers/pci/host/pci-layerscape.c                  |   2 +-
 drivers/pci/host/pcie-designware.c                 | 236 ++++++-------------
 drivers/pci/host/pcie-designware.h                 |  14 +-
 drivers/pci/host/pcie-hisi.c                       | 254 +++++++++++++++++++++
 drivers/pci/host/pcie-spear13xx.c                  |   2 +-
 16 files changed, 425 insertions(+), 184 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
 create mode 100644 drivers/pci/host/pcie-hisi.c

-- 
1.9.1

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 1/6] PCI: designware: move calculation of bus addresses to DRA7xx
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 2/6] ARM/PCI: remove align_resource in pci_sys_data Zhou Wang
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

From: gabriele paoloni <gabriele.paoloni@huawei.com>

Commit f4c55c5a3f7f "PCI: designware: Program ATU with untranslated
address" added the calculation of PCI BUS addresses in designware,
storing them in new fields added in "struct pcie_port". This
calculation is done for every designware user even if is only
applicable to DRA7xx.
This patch moves the calculation of the bus addresses to the DRA7xx
driver and is needed to allow the rework of designware to use
the new DT parsing API.

Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Acked-by: Pratyush Anand <pratyush.anand@gmail.com>
---
 drivers/pci/host/pci-dra7xx.c      | 13 +++++++++++++
 drivers/pci/host/pcie-designware.c | 15 ++++-----------
 2 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 80db09e..18ae7ff 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -61,6 +61,7 @@
 
 #define	PCIECTRL_DRA7XX_CONF_PHY_CS			0x010C
 #define	LINK_UP						BIT(16)
+#define	CPU_TO_BUS_ADDR					0x0FFFFFFF
 
 struct dra7xx_pcie {
 	void __iomem		*base;
@@ -139,6 +140,18 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
 static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
 	dw_pcie_setup_rc(pp);
+
+	if (pp->io_mod_base)
+		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->mem_mod_base)
+		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->cfg0_mod_base) {
+		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	}
+
 	dra7xx_pcie_establish_link(pp);
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		dw_pcie_msi_init(pp);
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 69486be..c5d407c 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -366,14 +366,10 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	struct of_pci_range range;
 	struct of_pci_range_parser parser;
 	struct resource *cfg_res;
-	u32 val, na, ns;
+	u32 val, ns;
 	const __be32 *addrp;
 	int i, index, ret;
 
-	/* Find the address cell size and the number of cells in order to get
-	 * the untranslated address.
-	 */
-	of_property_read_u32(np, "#address-cells", &na);
 	ns = of_n_size_cells(np);
 
 	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
@@ -416,8 +412,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->io_base = range.cpu_addr;
 
 			/* Find the untranslated IO space address */
-			pp->io_mod_base = of_read_number(parser.range -
-							 parser.np + na, ns);
+			pp->io_mod_base = range.cpu_addr;
 		}
 		if (restype == IORESOURCE_MEM) {
 			of_pci_range_to_resource(&range, np, &pp->mem);
@@ -426,8 +421,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->mem_bus_addr = range.pci_addr;
 
 			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = of_read_number(parser.range -
-							  parser.np + na, ns);
+			pp->mem_mod_base = range.cpu_addr;
 		}
 		if (restype == 0) {
 			of_pci_range_to_resource(&range, np, &pp->cfg);
@@ -437,8 +431,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
 
 			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = of_read_number(parser.range -
-							   parser.np + na, ns);
+			pp->cfg0_mod_base = range.cpu_addr;
 			pp->cfg1_mod_base = pp->cfg0_mod_base +
 					    pp->cfg0_size;
 		}
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 2/6] ARM/PCI: remove align_resource in pci_sys_data
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 1/6] PCI: designware: move calculation of bus addresses to DRA7xx Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 3/6] PCI: designware: Add ARM64 support Zhou Wang
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

From: gabriele paoloni <gabriele.paoloni@huawei.com>

This patch is needed in order to unify the PCIe designware framework for ARM and
ARM64 architectures. In the PCIe designware unification process we are calling
pci_create_root_bus() passing a "sysdata" parameter that is the same for both
ARM and ARM64 and is of type "struct pcie_port*". In the ARM case this will
cause a problem with the function pcibios_align_resource(); in fact this will
cast "dev->sysdata" to "struct pci_sys_data*", whereas designware had passed a
"struct pcie_port*" pointer.

This patch solves the issue by removing "align_resource" from "pci_sys_data"
struct and defining a static global function pointer in "bios32.c"

Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Acked-by: Pratyush Anand <pratyush.anand@gmail.com>
---
 arch/arm/include/asm/mach/pci.h |  5 -----
 arch/arm/kernel/bios32.c        | 12 ++++++++----
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index 28b9bb3..8a4e4de 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -58,11 +58,6 @@ struct pci_sys_data {
 					/* IRQ mapping				*/
 	int		(*map_irq)(const struct pci_dev *, u8, u8);
 					/* Resource alignement requirements	*/
-	resource_size_t (*align_resource)(struct pci_dev *dev,
-					  const struct resource *res,
-					  resource_size_t start,
-					  resource_size_t size,
-					  resource_size_t align);
 	void		*private_data;	/* platform controller private data	*/
 };
 
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index fcbbbb1..4cdc64d 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -17,6 +17,11 @@
 #include <asm/mach/pci.h>
 
 static int debug_pci;
+static resource_size_t (*align_resource)(struct pci_dev *dev,
+		  const struct resource *res,
+		  resource_size_t start,
+		  resource_size_t size,
+		  resource_size_t align) = NULL;
 
 #ifdef CONFIG_PCI_MSI
 struct msi_controller *pcibios_msi_controller(struct pci_dev *dev)
@@ -468,7 +473,7 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
 		sys->busnr   = busnr;
 		sys->swizzle = hw->swizzle;
 		sys->map_irq = hw->map_irq;
-		sys->align_resource = hw->align_resource;
+		align_resource = hw->align_resource;
 		INIT_LIST_HEAD(&sys->resources);
 
 		if (hw->private_data)
@@ -589,7 +594,6 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
 	struct pci_dev *dev = data;
-	struct pci_sys_data *sys = dev->sysdata;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO && start & 0x300)
@@ -597,8 +601,8 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 
 	start = (start + align - 1) & ~(align - 1);
 
-	if (sys->align_resource)
-		return sys->align_resource(dev, res, start, size, align);
+	if (align_resource)
+		return align_resource(dev, res, start, size, align);
 
 	return start;
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 1/6] PCI: designware: move calculation of bus addresses to DRA7xx Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 2/6] ARM/PCI: remove align_resource in pci_sys_data Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  2015-08-19 12:20   ` Zhou Wang
  2015-08-19 12:54   ` Lucas Stach
  2015-08-17 11:55 ` [PATCH v7 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
                   ` (2 subsequent siblings)
  5 siblings, 2 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
move related operations to dw_pcie_host_init.

In past, we use:
pci_common_init_dev
	-> pcibios_init_hw
		-> hw->scan (dw_pcie_scan_bus)
to pass 0 to root_bus_nr in struct pcie_port. This patch set pp->root_bus_nr = 0
in each PCIe host driver which is based on pcie-designware.

This patch also try to use of_pci_get_host_bridge_resources for ARM32 and ARM64
according to the suggestion for Gabriele[1]

Finally this patch reverts commit f4c55c5a3f7f "PCI: designware: Program ATU
with untranslated address" based on 1/6 in this series. we delete *_mod_base in
pcie-designware. This was discussed in [2]

I have compiled the driver with multi_v7_defconfig. However, I don't have
ARM32 PCIe related board to do test. It will be appreciated if someone could
help to test it.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Tested-By: James Morse <james.morse@arm.com>

[1] http://www.spinics.net/lists/linux-pci/msg42194.html
[2] http://www.spinics.net/lists/arm-kernel/msg436779.html
---
 drivers/pci/host/pci-dra7xx.c      |  15 +--
 drivers/pci/host/pci-exynos.c      |   2 +-
 drivers/pci/host/pci-imx6.c        |   2 +-
 drivers/pci/host/pci-keystone-dw.c |   2 +-
 drivers/pci/host/pci-keystone.c    |   2 +-
 drivers/pci/host/pci-layerscape.c  |   2 +-
 drivers/pci/host/pcie-designware.c | 229 ++++++++++++-------------------------
 drivers/pci/host/pcie-designware.h |  14 +--
 drivers/pci/host/pcie-spear13xx.c  |   2 +-
 9 files changed, 95 insertions(+), 175 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 18ae7ff..1268c69 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
 	dw_pcie_setup_rc(pp);
 
-	if (pp->io_mod_base)
-		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->io_base)
+		pp->io_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->mem_mod_base)
-		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->mem_base)
+		pp->mem_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->cfg0_mod_base) {
-		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
-		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->cfg0_base) {
+		pp->cfg0_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_base &= CPU_TO_BUS_ADDR;
 	}
 
 	dra7xx_pcie_establish_link(pp);
@@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
 
 	pp = &dra7xx->pp;
 	pp->dev = dev;
+	pp->root_bus_nr = 0;
 	pp->ops = &dra7xx_pcie_host_ops;
 
 	pp->irq = platform_get_irq(pdev, 1);
diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-exynos.c
index f9f468d..9771bb0 100644
--- a/drivers/pci/host/pci-exynos.c
+++ b/drivers/pci/host/pci-exynos.c
@@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct pcie_port *pp,
 		}
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = 0;
 	pp->ops = &exynos_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
index 233a196..bec256c 100644
--- a/drivers/pci/host/pci-imx6.c
+++ b/drivers/pci/host/pci-imx6.c
@@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
 		}
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = 0;
 	pp->ops = &imx6_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
index f34892e..b1e4135 100644
--- a/drivers/pci/host/pci-keystone-dw.c
+++ b/drivers/pci/host/pci-keystone-dw.c
@@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
 void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
 {
 	struct pcie_port *pp = &ks_pcie->pp;
-	u32 start = pp->mem.start, end = pp->mem.end;
+	u32 start = pp->mem->start, end = pp->mem->end;
 	int i, tr_size;
 
 	/* Disable BARs for inbound access */
diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
index 734da58..8113832 100644
--- a/drivers/pci/host/pci-keystone.c
+++ b/drivers/pci/host/pci-keystone.c
@@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
 			return ret;
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = 0;
 	pp->ops = &keystone_pcie_host_ops;
 	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
 	if (ret) {
diff --git a/drivers/pci/host/pci-layerscape.c b/drivers/pci/host/pci-layerscape.c
index b2328ea1..79ff08c 100644
--- a/drivers/pci/host/pci-layerscape.c
+++ b/drivers/pci/host/pci-layerscape.c
@@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
 	pp = &pcie->pp;
 	pp->dev = pcie->dev;
 	pp->dbi_base = pcie->dbi;
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = 0;
 	pp->ops = &ls_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index c5d407c..e71a88e 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -11,6 +11,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/hardirq.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kernel.h>
@@ -69,16 +70,7 @@
 #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
 #define PCIE_ATU_UPPER_TARGET		0x91C
 
-static struct hw_pci dw_pci;
-
-static unsigned long global_io_offset;
-
-static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
-{
-	BUG_ON(!sys->private_data);
-
-	return sys->private_data;
-}
+static struct pci_ops dw_pcie_ops;
 
 int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
 {
@@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
 static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
 {
 	int irq, pos0, i;
-	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
+	struct pcie_port *pp = desc->dev->bus->sysdata;
 
 	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
 				       order_base_2(no_irqs));
@@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
 {
 	int irq, pos;
 	struct msi_msg msg;
-	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
+	struct pcie_port *pp = pdev->bus->sysdata;
 
 	if (desc->msi_attrib.is_msix)
 		return -EINVAL;
@@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
 {
 	struct irq_data *data = irq_get_irq_data(irq);
 	struct msi_desc *msi = irq_data_get_msi(data);
-	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
+	struct pcie_port *pp = msi->dev->bus->sysdata;
 
 	clear_irq_range(pp, irq, 1, data->hwirq);
 }
@@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
 {
 	struct device_node *np = pp->dev->of_node;
 	struct platform_device *pdev = to_platform_device(pp->dev);
-	struct of_pci_range range;
-	struct of_pci_range_parser parser;
+	struct pci_bus *bus;
 	struct resource *cfg_res;
-	u32 val, ns;
-	const __be32 *addrp;
-	int i, index, ret;
-
-	ns = of_n_size_cells(np);
+	LIST_HEAD(res);
+	u32 val;
+	int i, ret;
+	struct resource_entry *win;
 
 	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
 	if (cfg_res) {
@@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
 		pp->cfg1_size = resource_size(cfg_res)/2;
 		pp->cfg0_base = cfg_res->start;
 		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
-
-		/* Find the untranslated configuration space address */
-		index = of_property_match_string(np, "reg-names", "config");
-		addrp = of_get_address(np, index, NULL, NULL);
-		pp->cfg0_mod_base = of_read_number(addrp, ns);
-		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
 	} else {
 		dev_err(pp->dev, "missing *config* reg space\n");
 	}
 
-	if (of_pci_range_parser_init(&parser, np)) {
-		dev_err(pp->dev, "missing ranges property\n");
-		return -EINVAL;
-	}
+	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
+	if (ret)
+		return ret;
 
 	/* Get the I/O and memory ranges from DT */
-	for_each_of_pci_range(&parser, &range) {
-		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
-
-		if (restype == IORESOURCE_IO) {
-			of_pci_range_to_resource(&range, np, &pp->io);
-			pp->io.name = "I/O";
-			pp->io.start = max_t(resource_size_t,
-					     PCIBIOS_MIN_IO,
-					     range.pci_addr + global_io_offset);
-			pp->io.end = min_t(resource_size_t,
-					   IO_SPACE_LIMIT,
-					   range.pci_addr + range.size
-					   + global_io_offset - 1);
-			pp->io_size = resource_size(&pp->io);
-			pp->io_bus_addr = range.pci_addr;
-			pp->io_base = range.cpu_addr;
-
-			/* Find the untranslated IO space address */
-			pp->io_mod_base = range.cpu_addr;
-		}
-		if (restype == IORESOURCE_MEM) {
-			of_pci_range_to_resource(&range, np, &pp->mem);
-			pp->mem.name = "MEM";
-			pp->mem_size = resource_size(&pp->mem);
-			pp->mem_bus_addr = range.pci_addr;
-
-			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = range.cpu_addr;
-		}
-		if (restype == 0) {
-			of_pci_range_to_resource(&range, np, &pp->cfg);
-			pp->cfg0_size = resource_size(&pp->cfg)/2;
-			pp->cfg1_size = resource_size(&pp->cfg)/2;
-			pp->cfg0_base = pp->cfg.start;
-			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
-
-			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = range.cpu_addr;
-			pp->cfg1_mod_base = pp->cfg0_mod_base +
-					    pp->cfg0_size;
+	resource_list_for_each_entry(win, &res) {
+		switch (resource_type(win->res)) {
+		case IORESOURCE_IO:
+			pp->io = win->res;
+			pp->io->name = "I/O";
+			pp->io_size = resource_size(pp->io);
+			pp->io_bus_addr = pp->io->start - win->offset;
+			ret = pci_remap_iospace(pp->io, pp->io_base);
+			if (ret) {
+				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
+					 ret, pp->io);
+				continue;
+			}
+			break;
+		case IORESOURCE_MEM:
+			pp->mem = win->res;
+			pp->mem->name = "MEM";
+			pp->mem_size = resource_size(pp->mem);
+			pp->mem_bus_addr = pp->mem->start - win->offset;
+			break;
+		case 0:
+			pp->cfg = win->res;
+			pp->cfg0_size = resource_size(pp->cfg)/2;
+			pp->cfg1_size = resource_size(pp->cfg)/2;
+			pp->cfg0_base = pp->cfg->start;
+			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
+			break;
+		case IORESOURCE_BUS:
+			pp->busn = win->res;
+			break;
+		default:
+			continue;
 		}
 	}
 
-	ret = of_pci_parse_bus_range(np, &pp->busn);
-	if (ret < 0) {
-		pp->busn.name = np->name;
-		pp->busn.start = 0;
-		pp->busn.end = 0xff;
-		pp->busn.flags = IORESOURCE_BUS;
-		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
-			ret, &pp->busn);
-	}
-
 	if (!pp->dbi_base) {
-		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
-					resource_size(&pp->cfg));
+		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
+					resource_size(pp->cfg));
 		if (!pp->dbi_base) {
 			dev_err(pp->dev, "error with ioremap\n");
 			return -ENOMEM;
 		}
 	}
 
-	pp->mem_base = pp->mem.start;
+	pp->mem_base = pp->mem->start;
 
 	if (!pp->va_cfg0_base) {
 		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 
 	if (!pp->ops->rd_other_conf)
 		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
-					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
+					  PCIE_ATU_TYPE_MEM, pp->mem_base,
 					  pp->mem_bus_addr, pp->mem_size);
 
 	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
@@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	val |= PORT_LOGIC_SPEED_CHANGE;
 	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
 
-#ifdef CONFIG_PCI_MSI
+	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
+				  pp, &res);
+	if (!bus)
+		return -ENOMEM;
+
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+	bus->msi = container_of(&pp->irq_domain, struct msi_controller, domain);
+#else
 	dw_pcie_msi_chip.dev = pp->dev;
-	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
+	bus->msi = &dw_pcie_msi_chip;
 #endif
 
-	dw_pci.nr_controllers = 1;
-	dw_pci.private_data = (void **)&pp;
+	pci_scan_child_bus(bus);
+	if (pp->ops->scan_bus)
+		pp->ops->scan_bus(pp);
+
+#ifdef CONFIG_ARM
+	/* support old dtbs that incorrectly describe IRQs */
+	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
+#endif
 
-	pci_common_init_dev(pp->dev, &dw_pci);
+	pci_assign_unassigned_bus_resources(bus);
+	pci_bus_add_devices(bus);
 
 	return 0;
 }
@@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
 static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 			int size, u32 *val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
@@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
 			int where, int size, u32 val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
@@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
 	.write = dw_pcie_wr_conf,
 };
 
-static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
-{
-	struct pcie_port *pp;
-
-	pp = sys_to_pcie(sys);
-
-	if (global_io_offset < SZ_1M && pp->io_size > 0) {
-		sys->io_offset = global_io_offset - pp->io_bus_addr;
-		pci_ioremap_io(global_io_offset, pp->io_base);
-		global_io_offset += SZ_64K;
-		pci_add_resource_offset(&sys->resources, &pp->io,
-					sys->io_offset);
-	}
-
-	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
-	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
-	pci_add_resource(&sys->resources, &pp->busn);
-
-	return 1;
-}
-
-static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
-{
-	struct pci_bus *bus;
-	struct pcie_port *pp = sys_to_pcie(sys);
-
-	pp->root_bus_nr = sys->busnr;
-	bus = pci_scan_root_bus(pp->dev, sys->busnr,
-				  &dw_pcie_ops, sys, &sys->resources);
-	if (!bus)
-		return NULL;
-
-	if (bus && pp->ops->scan_bus)
-		pp->ops->scan_bus(pp);
-
-	return bus;
-}
-
-static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
-{
-	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
-	int irq;
-
-	irq = of_irq_parse_and_map_pci(dev, slot, pin);
-	if (!irq)
-		irq = pp->irq;
-
-	return irq;
-}
-
-static struct hw_pci dw_pci = {
-	.setup		= dw_pcie_setup,
-	.scan		= dw_pcie_scan_bus,
-	.map_irq	= dw_pcie_map_irq,
-};
-
 void dw_pcie_setup_rc(struct pcie_port *pp)
 {
 	u32 val;
diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
index d0bbd27..264c969 100644
--- a/drivers/pci/host/pcie-designware.h
+++ b/drivers/pci/host/pcie-designware.h
@@ -27,25 +27,21 @@ struct pcie_port {
 	u8			root_bus_nr;
 	void __iomem		*dbi_base;
 	u64			cfg0_base;
-	u64			cfg0_mod_base;
 	void __iomem		*va_cfg0_base;
 	u32			cfg0_size;
 	u64			cfg1_base;
-	u64			cfg1_mod_base;
 	void __iomem		*va_cfg1_base;
 	u32			cfg1_size;
-	u64			io_base;
-	u64			io_mod_base;
+	resource_size_t		io_base;
 	phys_addr_t		io_bus_addr;
 	u32			io_size;
 	u64			mem_base;
-	u64			mem_mod_base;
 	phys_addr_t		mem_bus_addr;
 	u32			mem_size;
-	struct resource		cfg;
-	struct resource		io;
-	struct resource		mem;
-	struct resource		busn;
+	struct resource		*cfg;
+	struct resource		*io;
+	struct resource		*mem;
+	struct resource		*busn;
 	int			irq;
 	u32			lanes;
 	struct pcie_host_ops	*ops;
diff --git a/drivers/pci/host/pcie-spear13xx.c b/drivers/pci/host/pcie-spear13xx.c
index c49fbdc..03eb204 100644
--- a/drivers/pci/host/pcie-spear13xx.c
+++ b/drivers/pci/host/pcie-spear13xx.c
@@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct pcie_port *pp,
 		return ret;
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = 0;
 	pp->ops = &spear13xx_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
                   ` (2 preceding siblings ...)
  2015-08-17 11:55 ` [PATCH v7 3/6] PCI: designware: Add ARM64 support Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 5/6] Documentation: DT: Add HiSilicon PCIe host binding Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 6/6] MAINTAINERS: Add pcie-hisi maintainer Zhou Wang
  5 siblings, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

This patch adds PCIe host support for HiSilicon SoC Hip05.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
---
 drivers/pci/host/Kconfig     |   8 ++
 drivers/pci/host/Makefile    |   1 +
 drivers/pci/host/pcie-hisi.c | 254 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 263 insertions(+)
 create mode 100644 drivers/pci/host/pcie-hisi.c

diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index c132bdd..cd2b80b 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -145,4 +145,12 @@ config PCIE_IPROC_BCMA
 	  Say Y here if you want to use the Broadcom iProc PCIe controller
 	  through the BCMA bus interface
 
+config PCI_HISI
+	depends on OF && ARM64
+	bool "HiSilicon SoC HIP05 PCIe controller"
+	select PCIEPORTBUS
+	select PCIE_DW
+	help
+	  Say Y here if you want PCIe controller support on HiSilicon HIP05 SoC
+
 endmenu
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 140d66f..ea1dbf2 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -17,3 +17,4 @@ obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
 obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o
 obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
 obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
+obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
new file mode 100644
index 0000000..7dcacf1
--- /dev/null
+++ b/drivers/pci/host/pcie-hisi.c
@@ -0,0 +1,254 @@
+/*
+ * PCIe host controller driver for HiSilicon Hip05 SoC
+ *
+ * Copyright (C) 2015 HiSilicon Co., Ltd. http://www.hisilicon.com
+ *
+ * Author: Zhou Wang <wangzhou1@hisilicon.com>
+ *         Dacai Zhu <zhudacai@hisilicon.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_pci.h>
+#include <linux/platform_device.h>
+
+#include "pcie-designware.h"
+
+#define PCIE_SUBCTRL_MODE_REG                           0x2800
+#define PCIE_SUBCTRL_SYS_STATE4_REG                     0x6818
+#define PCIE_SLV_DBI_MODE                               0x0
+#define PCIE_SLV_SYSCTRL_MODE                           0x1
+#define PCIE_SLV_CONTENT_MODE                           0x2
+#define PCIE_SLV_MSI_ASID                               0x10
+#define PCIE_LTSSM_LINKUP_STATE                         0x11
+#define PCIE_LTSSM_STATE_MASK                           0x3F
+#define PCIE_MSI_ASID_ENABLE                            (0x1 << 12)
+#define PCIE_MSI_ASID_VALUE                             (0x1 << 16)
+#define PCIE_MSI_TRANS_ENABLE                           (0x1 << 12)
+#define PCIE_MSI_TRANS_REG                              0x1c8
+#define PCIE_MSI_LOW_ADDRESS                            0x1b4
+#define PCIE_MSI_HIGH_ADDRESS                           0x1c4
+#define PCIE_MSI_ADDRESS_VAL                            0xb7010040
+
+#define to_hisi_pcie(x)	container_of(x, struct hisi_pcie, pp)
+
+struct hisi_pcie {
+	void __iomem *subctrl_base;
+	void __iomem *reg_base;
+	struct msi_controller *msi;
+	u32 port_id;
+	struct pcie_port pp;
+};
+
+static inline void hisi_pcie_subctrl_writel(struct hisi_pcie *pcie,
+					    u32 val, u32 reg)
+{
+	writel(val, pcie->subctrl_base + reg);
+}
+
+static inline u32 hisi_pcie_subctrl_readl(struct hisi_pcie *pcie, u32 reg)
+{
+	return readl(pcie->subctrl_base + reg);
+}
+
+static inline void hisi_pcie_apb_writel(struct hisi_pcie *pcie,
+					u32 val, u32 reg)
+{
+	writel(val, pcie->reg_base + reg);
+}
+
+static inline u32 hisi_pcie_apb_readl(struct hisi_pcie *pcie, u32 reg)
+{
+	return readl(pcie->reg_base + reg);
+}
+
+/*
+ * Change mode to indicate the same reg_base to base of PCIe host configure
+ * registers, base of RC configure space or base of vmid/asid context table
+ */
+static void hisi_pcie_change_apb_mode(struct hisi_pcie *pcie, u32 mode)
+{
+	u32 val;
+	u32 bit_mask;
+	u32 bit_shift;
+	u32 port_id = pcie->port_id;
+	u32 reg = PCIE_SUBCTRL_MODE_REG + 0x100 * port_id;
+
+	if ((port_id == 1) || (port_id == 2)) {
+		bit_mask = 0xc;
+		bit_shift = 0x2;
+	} else {
+		bit_mask = 0x6;
+		bit_shift = 0x1;
+	}
+
+	val = hisi_pcie_subctrl_readl(pcie, reg);
+	val = (val & (~bit_mask)) | (mode << bit_shift);
+	hisi_pcie_subctrl_writel(pcie, val, reg);
+}
+
+/* Configure vmid/asid table in PCIe host */
+static void hisi_pcie_config_context(struct hisi_pcie *pcie)
+{
+	int i;
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
+
+	/*
+	 * init vmid and asid tables for all PCIe devices as 0
+	 * vmid table: 0 ~ 0x3ff, asid table: 0x400 ~ 0x7ff
+	 */
+	for (i = 0; i < 0x800; i++)
+		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
+
+	hisi_pcie_apb_writel(pcie, PCIE_MSI_ADDRESS_VAL, PCIE_MSI_LOW_ADDRESS);
+	hisi_pcie_apb_writel(pcie, 0x0, PCIE_MSI_HIGH_ADDRESS);
+	hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
+			     PCIE_SLV_MSI_ASID);
+	hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
+}
+
+static int hisi_pcie_link_up(struct pcie_port *pp)
+{
+	u32 val;
+	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
+
+	val = hisi_pcie_subctrl_readl(hisi_pcie, PCIE_SUBCTRL_SYS_STATE4_REG +
+				      0x100 * hisi_pcie->port_id);
+
+	return ((val & PCIE_LTSSM_STATE_MASK) == PCIE_LTSSM_LINKUP_STATE);
+}
+
+static
+int hisi_pcie_msi_host_init(struct pcie_port *pp, struct msi_controller *chip)
+{
+	struct device_node *msi_node;
+	struct irq_domain *irq_domain;
+	struct device_node *np = pp->dev->of_node;
+
+	msi_node = of_parse_phandle(np, "msi-parent", 0);
+	if (!msi_node) {
+		dev_err(pp->dev, "failed to find msi-parent\n");
+		return -ENODEV;
+	}
+
+	irq_domain = irq_find_host(msi_node);
+	if (!irq_domain) {
+		dev_err(pp->dev, "failed to find irq domain\n");
+		return -ENODEV;
+	}
+
+	pp->irq_domain = irq_domain;
+
+	return 0;
+}
+
+static struct pcie_host_ops hisi_pcie_host_ops = {
+	.link_up = hisi_pcie_link_up,
+	.msi_host_init = hisi_pcie_msi_host_init,
+};
+
+static int __init hisi_add_pcie_port(struct pcie_port *pp,
+				     struct platform_device *pdev)
+{
+	int ret;
+	u32 port_id;
+	struct resource busn;
+	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
+
+	if (of_property_read_u32(pdev->dev.of_node, "port-id", &port_id)) {
+		dev_err(&pdev->dev, "failed to read port-id\n");
+		return -EINVAL;
+	}
+	if (port_id > 3) {
+		dev_err(&pdev->dev, "Invalid port-id: %d\n", port_id);
+		return -EINVAL;
+	}
+
+	hisi_pcie->port_id = port_id;
+
+	if (of_pci_parse_bus_range(pdev->dev.of_node, &busn)) {
+		dev_err(&pdev->dev, "failed to parse bus-ranges\n");
+		return -EINVAL;
+	}
+
+	pp->root_bus_nr = busn.start;
+	pp->ops = &hisi_pcie_host_ops;
+
+	hisi_pcie_config_context(hisi_pcie);
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to initialize host\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int __init hisi_pcie_probe(struct platform_device *pdev)
+{
+	struct hisi_pcie *hisi_pcie;
+	struct pcie_port *pp;
+	struct resource *reg;
+	struct resource *subctrl;
+	int ret;
+
+	hisi_pcie = devm_kzalloc(&pdev->dev, sizeof(*hisi_pcie), GFP_KERNEL);
+	if (!hisi_pcie)
+		return -ENOMEM;
+
+	pp = &hisi_pcie->pp;
+	pp->dev = &pdev->dev;
+
+	subctrl = platform_get_resource_byname(pdev, IORESOURCE_MEM, "subctrl");
+	hisi_pcie->subctrl_base = devm_ioremap_nocache(&pdev->dev,
+					subctrl->start, resource_size(subctrl));
+	if (IS_ERR(hisi_pcie->subctrl_base)) {
+		dev_err(pp->dev, "cannot get subctrl base\n");
+		return PTR_ERR(hisi_pcie->subctrl_base);
+	}
+
+	reg = platform_get_resource_byname(pdev, IORESOURCE_MEM, "rc_dbi");
+	hisi_pcie->reg_base = devm_ioremap_resource(&pdev->dev, reg);
+	if (IS_ERR(hisi_pcie->reg_base)) {
+		dev_err(pp->dev, "cannot get rc_dbi base\n");
+		return PTR_ERR(hisi_pcie->reg_base);
+	}
+
+	hisi_pcie->pp.dbi_base = hisi_pcie->reg_base;
+
+	ret = hisi_add_pcie_port(pp, pdev);
+	if (ret)
+		return ret;
+
+	platform_set_drvdata(pdev, hisi_pcie);
+
+	return 0;
+}
+
+static const struct of_device_id hisi_pcie_of_match[] = {
+	{.compatible = "hisilicon,hip05-pcie",},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, hisi_pcie_of_match);
+
+static struct platform_driver hisi_pcie_driver = {
+	.probe  = hisi_pcie_probe,
+	.driver = {
+		   .name = "hisi-pcie",
+		   .of_match_table = hisi_pcie_of_match,
+	},
+};
+
+module_platform_driver(hisi_pcie_driver);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 5/6] Documentation: DT: Add HiSilicon PCIe host binding
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
                   ` (3 preceding siblings ...)
  2015-08-17 11:55 ` [PATCH v7 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  2015-08-17 11:55 ` [PATCH v7 6/6] MAINTAINERS: Add pcie-hisi maintainer Zhou Wang
  5 siblings, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

This patch adds related DTS binding document for HiSilicon PCIe host driver.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
---
 .../devicetree/bindings/pci/hisilicon-pcie.txt     | 46 ++++++++++++++++++++++
 1 file changed, 46 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/hisilicon-pcie.txt

diff --git a/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
new file mode 100644
index 0000000..2afc9d1
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
@@ -0,0 +1,46 @@
+HiSilicon PCIe host bridge DT description
+
+HiSilicon PCIe host controller is based on Designware PCI core.
+It shares common functions with PCIe Designware core driver and inherits
+common properties defined in
+Documentation/devicetree/bindings/pci/designware-pci.txt.
+
+Additional properties are described here:
+
+Required properties:
+- compatible: Should contain "hisilicon,hip05-pcie".
+- reg: Should contain rc_dbi, subctrl, config registers location and length.
+- reg-names: Must include the following entries:
+  "rc_dbi": controller configuration registers;
+  "subctrl": whole PCIe hosts configuration registers;
+  "config": PCIe configuration space registers.
+- msi-parent: Should be its_pcie which is an ITS receiving MSI interrupts.
+- port-id: Should be 0, 1, 2 or 3.
+
+Optional properties:
+- status: Either "ok" or "disabled".
+- dma-coherent: Present if DMA operations are coherent.
+
+Example:
+	pcie at 0xb0080000 {
+		compatible = "hisilicon,hip05-pcie", "snps,dw-pcie";
+		reg = <0 0xb0080000 0 0x10000>, <0 0xb0000000 0 0x10000>,
+		      <0x220 0x00000000 0 0x2000>;
+		reg-names = "rc_dbi", "subctrl", "config";
+		bus-range = <0  15>;
+		msi-parent = <&its_pcie>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		device_type = "pci";
+		dma-coherent;
+		ranges = <0x82000000 0 0x00000000 0x220 0x00000000 0 0x10000000>;
+		num-lanes = <8>;
+		port-id = <1>;
+		#interrupts-cells = <1>;
+		interrupts-map-mask = <0xf800 0 0 7>;
+		interrupts-map = <0x0 0 0 1 &mbigen_pcie 1 10
+				  0x0 0 0 2 &mbigen_pcie 2 11
+				  0x0 0 0 3 &mbigen_pcie 3 12
+				  0x0 0 0 4 &mbigen_pcie 4 13>;
+		status = "ok";
+	};
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 6/6] MAINTAINERS: Add pcie-hisi maintainer
  2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
                   ` (4 preceding siblings ...)
  2015-08-17 11:55 ` [PATCH v7 5/6] Documentation: DT: Add HiSilicon PCIe host binding Zhou Wang
@ 2015-08-17 11:55 ` Zhou Wang
  5 siblings, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-17 11:55 UTC (permalink / raw)
  To: linux-arm-kernel

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8133cef..7cd8e47 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7854,6 +7854,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/pci/xgene-pci-msi.txt
 F:	drivers/pci/host/pci-xgene-msi.c
 
+PCIE DRIVER FOR HISILICON
+M:	Zhou Wang <wangzhou1@hisilicon.com>
+L:	linux-pci at vger.kernel.org
+S:	Maintained
+F:	Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
+F:	drivers/pci/host/pcie-hisi.c
+
 PCMCIA SUBSYSTEM
 P:	Linux PCMCIA Team
 L:	linux-pcmcia at lists.infradead.org
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-17 11:55 ` [PATCH v7 3/6] PCI: designware: Add ARM64 support Zhou Wang
@ 2015-08-19 12:20   ` Zhou Wang
  2015-08-19 12:54   ` Lucas Stach
  1 sibling, 0 replies; 18+ messages in thread
From: Zhou Wang @ 2015-08-19 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Pratyush and Jingoo,

what is your opinion about this patch? It will be very appreciated if you
could take a look at it.

Thanks,
Zhou

On 2015/8/17 19:55, Zhou Wang wrote:
> This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
> function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
> move related operations to dw_pcie_host_init.
> 
> In past, we use:
> pci_common_init_dev
> 	-> pcibios_init_hw
> 		-> hw->scan (dw_pcie_scan_bus)
> to pass 0 to root_bus_nr in struct pcie_port. This patch set pp->root_bus_nr = 0
> in each PCIe host driver which is based on pcie-designware.
> 
> This patch also try to use of_pci_get_host_bridge_resources for ARM32 and ARM64
> according to the suggestion for Gabriele[1]
> 
> Finally this patch reverts commit f4c55c5a3f7f "PCI: designware: Program ATU
> with untranslated address" based on 1/6 in this series. we delete *_mod_base in
> pcie-designware. This was discussed in [2]
> 
> I have compiled the driver with multi_v7_defconfig. However, I don't have
> ARM32 PCIe related board to do test. It will be appreciated if someone could
> help to test it.
> 
> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> Tested-By: James Morse <james.morse@arm.com>
> 
> [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> [2] http://www.spinics.net/lists/arm-kernel/msg436779.html
> ---
>  drivers/pci/host/pci-dra7xx.c      |  15 +--
>  drivers/pci/host/pci-exynos.c      |   2 +-
>  drivers/pci/host/pci-imx6.c        |   2 +-
>  drivers/pci/host/pci-keystone-dw.c |   2 +-
>  drivers/pci/host/pci-keystone.c    |   2 +-
>  drivers/pci/host/pci-layerscape.c  |   2 +-
>  drivers/pci/host/pcie-designware.c | 229 ++++++++++++-------------------------
>  drivers/pci/host/pcie-designware.h |  14 +--
>  drivers/pci/host/pcie-spear13xx.c  |   2 +-
>  9 files changed, 95 insertions(+), 175 deletions(-)
> 
> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
> index 18ae7ff..1268c69 100644
> --- a/drivers/pci/host/pci-dra7xx.c
> +++ b/drivers/pci/host/pci-dra7xx.c
> @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
>  {
>  	dw_pcie_setup_rc(pp);
>  
> -	if (pp->io_mod_base)
> -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->io_base)
> +		pp->io_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->mem_mod_base)
> -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->mem_base)
> +		pp->mem_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->cfg0_mod_base) {
> -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->cfg0_base) {
> +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
>  	}
>  
>  	dra7xx_pcie_establish_link(pp);
> @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
>  
>  	pp = &dra7xx->pp;
>  	pp->dev = dev;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &dra7xx_pcie_host_ops;
>  
>  	pp->irq = platform_get_irq(pdev, 1);
> diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-exynos.c
> index f9f468d..9771bb0 100644
> --- a/drivers/pci/host/pci-exynos.c
> +++ b/drivers/pci/host/pci-exynos.c
> @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &exynos_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
> index 233a196..bec256c 100644
> --- a/drivers/pci/host/pci-imx6.c
> +++ b/drivers/pci/host/pci-imx6.c
> @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &imx6_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
> index f34892e..b1e4135 100644
> --- a/drivers/pci/host/pci-keystone-dw.c
> +++ b/drivers/pci/host/pci-keystone-dw.c
> @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
>  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
>  {
>  	struct pcie_port *pp = &ks_pcie->pp;
> -	u32 start = pp->mem.start, end = pp->mem.end;
> +	u32 start = pp->mem->start, end = pp->mem->end;
>  	int i, tr_size;
>  
>  	/* Disable BARs for inbound access */
> diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
> index 734da58..8113832 100644
> --- a/drivers/pci/host/pci-keystone.c
> +++ b/drivers/pci/host/pci-keystone.c
> @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
>  			return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &keystone_pcie_host_ops;
>  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
>  	if (ret) {
> diff --git a/drivers/pci/host/pci-layerscape.c b/drivers/pci/host/pci-layerscape.c
> index b2328ea1..79ff08c 100644
> --- a/drivers/pci/host/pci-layerscape.c
> +++ b/drivers/pci/host/pci-layerscape.c
> @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
>  	pp = &pcie->pp;
>  	pp->dev = pcie->dev;
>  	pp->dbi_base = pcie->dbi;
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &ls_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
> index c5d407c..e71a88e 100644
> --- a/drivers/pci/host/pcie-designware.c
> +++ b/drivers/pci/host/pcie-designware.c
> @@ -11,6 +11,7 @@
>   * published by the Free Software Foundation.
>   */
>  
> +#include <linux/hardirq.h>
>  #include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kernel.h>
> @@ -69,16 +70,7 @@
>  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
>  #define PCIE_ATU_UPPER_TARGET		0x91C
>  
> -static struct hw_pci dw_pci;
> -
> -static unsigned long global_io_offset;
> -
> -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> -{
> -	BUG_ON(!sys->private_data);
> -
> -	return sys->private_data;
> -}
> +static struct pci_ops dw_pcie_ops;
>  
>  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
>  {
> @@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
>  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
>  {
>  	int irq, pos0, i;
> -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> +	struct pcie_port *pp = desc->dev->bus->sysdata;
>  
>  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
>  				       order_base_2(no_irqs));
> @@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
>  {
>  	int irq, pos;
>  	struct msi_msg msg;
> -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> +	struct pcie_port *pp = pdev->bus->sysdata;
>  
>  	if (desc->msi_attrib.is_msix)
>  		return -EINVAL;
> @@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
>  {
>  	struct irq_data *data = irq_get_irq_data(irq);
>  	struct msi_desc *msi = irq_data_get_msi(data);
> -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> +	struct pcie_port *pp = msi->dev->bus->sysdata;
>  
>  	clear_irq_range(pp, irq, 1, data->hwirq);
>  }
> @@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  {
>  	struct device_node *np = pp->dev->of_node;
>  	struct platform_device *pdev = to_platform_device(pp->dev);
> -	struct of_pci_range range;
> -	struct of_pci_range_parser parser;
> +	struct pci_bus *bus;
>  	struct resource *cfg_res;
> -	u32 val, ns;
> -	const __be32 *addrp;
> -	int i, index, ret;
> -
> -	ns = of_n_size_cells(np);
> +	LIST_HEAD(res);
> +	u32 val;
> +	int i, ret;
> +	struct resource_entry *win;
>  
>  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
>  	if (cfg_res) {
> @@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  		pp->cfg1_size = resource_size(cfg_res)/2;
>  		pp->cfg0_base = cfg_res->start;
>  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> -
> -		/* Find the untranslated configuration space address */
> -		index = of_property_match_string(np, "reg-names", "config");
> -		addrp = of_get_address(np, index, NULL, NULL);
> -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
>  	} else {
>  		dev_err(pp->dev, "missing *config* reg space\n");
>  	}
>  
> -	if (of_pci_range_parser_init(&parser, np)) {
> -		dev_err(pp->dev, "missing ranges property\n");
> -		return -EINVAL;
> -	}
> +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
> +	if (ret)
> +		return ret;
>  
>  	/* Get the I/O and memory ranges from DT */
> -	for_each_of_pci_range(&parser, &range) {
> -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> -
> -		if (restype == IORESOURCE_IO) {
> -			of_pci_range_to_resource(&range, np, &pp->io);
> -			pp->io.name = "I/O";
> -			pp->io.start = max_t(resource_size_t,
> -					     PCIBIOS_MIN_IO,
> -					     range.pci_addr + global_io_offset);
> -			pp->io.end = min_t(resource_size_t,
> -					   IO_SPACE_LIMIT,
> -					   range.pci_addr + range.size
> -					   + global_io_offset - 1);
> -			pp->io_size = resource_size(&pp->io);
> -			pp->io_bus_addr = range.pci_addr;
> -			pp->io_base = range.cpu_addr;
> -
> -			/* Find the untranslated IO space address */
> -			pp->io_mod_base = range.cpu_addr;
> -		}
> -		if (restype == IORESOURCE_MEM) {
> -			of_pci_range_to_resource(&range, np, &pp->mem);
> -			pp->mem.name = "MEM";
> -			pp->mem_size = resource_size(&pp->mem);
> -			pp->mem_bus_addr = range.pci_addr;
> -
> -			/* Find the untranslated MEM space address */
> -			pp->mem_mod_base = range.cpu_addr;
> -		}
> -		if (restype == 0) {
> -			of_pci_range_to_resource(&range, np, &pp->cfg);
> -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> -			pp->cfg0_base = pp->cfg.start;
> -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> -
> -			/* Find the untranslated configuration space address */
> -			pp->cfg0_mod_base = range.cpu_addr;
> -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> -					    pp->cfg0_size;
> +	resource_list_for_each_entry(win, &res) {
> +		switch (resource_type(win->res)) {
> +		case IORESOURCE_IO:
> +			pp->io = win->res;
> +			pp->io->name = "I/O";
> +			pp->io_size = resource_size(pp->io);
> +			pp->io_bus_addr = pp->io->start - win->offset;
> +			ret = pci_remap_iospace(pp->io, pp->io_base);
> +			if (ret) {
> +				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
> +					 ret, pp->io);
> +				continue;
> +			}
> +			break;
> +		case IORESOURCE_MEM:
> +			pp->mem = win->res;
> +			pp->mem->name = "MEM";
> +			pp->mem_size = resource_size(pp->mem);
> +			pp->mem_bus_addr = pp->mem->start - win->offset;
> +			break;
> +		case 0:
> +			pp->cfg = win->res;
> +			pp->cfg0_size = resource_size(pp->cfg)/2;
> +			pp->cfg1_size = resource_size(pp->cfg)/2;
> +			pp->cfg0_base = pp->cfg->start;
> +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> +			break;
> +		case IORESOURCE_BUS:
> +			pp->busn = win->res;
> +			break;
> +		default:
> +			continue;
>  		}
>  	}
>  
> -	ret = of_pci_parse_bus_range(np, &pp->busn);
> -	if (ret < 0) {
> -		pp->busn.name = np->name;
> -		pp->busn.start = 0;
> -		pp->busn.end = 0xff;
> -		pp->busn.flags = IORESOURCE_BUS;
> -		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
> -			ret, &pp->busn);
> -	}
> -
>  	if (!pp->dbi_base) {
> -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> -					resource_size(&pp->cfg));
> +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> +					resource_size(pp->cfg));
>  		if (!pp->dbi_base) {
>  			dev_err(pp->dev, "error with ioremap\n");
>  			return -ENOMEM;
>  		}
>  	}
>  
> -	pp->mem_base = pp->mem.start;
> +	pp->mem_base = pp->mem->start;
>  
>  	if (!pp->va_cfg0_base) {
>  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> @@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  
>  	if (!pp->ops->rd_other_conf)
>  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
>  					  pp->mem_bus_addr, pp->mem_size);
>  
>  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> @@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  	val |= PORT_LOGIC_SPEED_CHANGE;
>  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
>  
> -#ifdef CONFIG_PCI_MSI
> +	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
> +				  pp, &res);
> +	if (!bus)
> +		return -ENOMEM;
> +
> +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> +	bus->msi = container_of(&pp->irq_domain, struct msi_controller, domain);
> +#else
>  	dw_pcie_msi_chip.dev = pp->dev;
> -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> +	bus->msi = &dw_pcie_msi_chip;
>  #endif
>  
> -	dw_pci.nr_controllers = 1;
> -	dw_pci.private_data = (void **)&pp;
> +	pci_scan_child_bus(bus);
> +	if (pp->ops->scan_bus)
> +		pp->ops->scan_bus(pp);
> +
> +#ifdef CONFIG_ARM
> +	/* support old dtbs that incorrectly describe IRQs */
> +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> +#endif
>  
> -	pci_common_init_dev(pp->dev, &dw_pci);
> +	pci_assign_unassigned_bus_resources(bus);
> +	pci_bus_add_devices(bus);
>  
>  	return 0;
>  }
> @@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
>  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  			int size, u32 *val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> @@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
>  			int where, int size, u32 val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> @@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
>  	.write = dw_pcie_wr_conf,
>  };
>  
> -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> -{
> -	struct pcie_port *pp;
> -
> -	pp = sys_to_pcie(sys);
> -
> -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> -		pci_ioremap_io(global_io_offset, pp->io_base);
> -		global_io_offset += SZ_64K;
> -		pci_add_resource_offset(&sys->resources, &pp->io,
> -					sys->io_offset);
> -	}
> -
> -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> -	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
> -	pci_add_resource(&sys->resources, &pp->busn);
> -
> -	return 1;
> -}
> -
> -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
> -{
> -	struct pci_bus *bus;
> -	struct pcie_port *pp = sys_to_pcie(sys);
> -
> -	pp->root_bus_nr = sys->busnr;
> -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> -				  &dw_pcie_ops, sys, &sys->resources);
> -	if (!bus)
> -		return NULL;
> -
> -	if (bus && pp->ops->scan_bus)
> -		pp->ops->scan_bus(pp);
> -
> -	return bus;
> -}
> -
> -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
> -{
> -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> -	int irq;
> -
> -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> -	if (!irq)
> -		irq = pp->irq;
> -
> -	return irq;
> -}
> -
> -static struct hw_pci dw_pci = {
> -	.setup		= dw_pcie_setup,
> -	.scan		= dw_pcie_scan_bus,
> -	.map_irq	= dw_pcie_map_irq,
> -};
> -
>  void dw_pcie_setup_rc(struct pcie_port *pp)
>  {
>  	u32 val;
> diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
> index d0bbd27..264c969 100644
> --- a/drivers/pci/host/pcie-designware.h
> +++ b/drivers/pci/host/pcie-designware.h
> @@ -27,25 +27,21 @@ struct pcie_port {
>  	u8			root_bus_nr;
>  	void __iomem		*dbi_base;
>  	u64			cfg0_base;
> -	u64			cfg0_mod_base;
>  	void __iomem		*va_cfg0_base;
>  	u32			cfg0_size;
>  	u64			cfg1_base;
> -	u64			cfg1_mod_base;
>  	void __iomem		*va_cfg1_base;
>  	u32			cfg1_size;
> -	u64			io_base;
> -	u64			io_mod_base;
> +	resource_size_t		io_base;
>  	phys_addr_t		io_bus_addr;
>  	u32			io_size;
>  	u64			mem_base;
> -	u64			mem_mod_base;
>  	phys_addr_t		mem_bus_addr;
>  	u32			mem_size;
> -	struct resource		cfg;
> -	struct resource		io;
> -	struct resource		mem;
> -	struct resource		busn;
> +	struct resource		*cfg;
> +	struct resource		*io;
> +	struct resource		*mem;
> +	struct resource		*busn;
>  	int			irq;
>  	u32			lanes;
>  	struct pcie_host_ops	*ops;
> diff --git a/drivers/pci/host/pcie-spear13xx.c b/drivers/pci/host/pcie-spear13xx.c
> index c49fbdc..03eb204 100644
> --- a/drivers/pci/host/pcie-spear13xx.c
> +++ b/drivers/pci/host/pcie-spear13xx.c
> @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct pcie_port *pp,
>  		return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &spear13xx_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> 

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-17 11:55 ` [PATCH v7 3/6] PCI: designware: Add ARM64 support Zhou Wang
  2015-08-19 12:20   ` Zhou Wang
@ 2015-08-19 12:54   ` Lucas Stach
  2015-08-19 15:16     ` Gabriele Paoloni
  1 sibling, 1 reply; 18+ messages in thread
From: Lucas Stach @ 2015-08-19 12:54 UTC (permalink / raw)
  To: linux-arm-kernel

Am Montag, den 17.08.2015, 19:55 +0800 schrieb Zhou Wang:
> This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
> function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
> move related operations to dw_pcie_host_init.
> 
> In past, we use:
> pci_common_init_dev
> 	-> pcibios_init_hw
> 		-> hw->scan (dw_pcie_scan_bus)
> to pass 0 to root_bus_nr in struct pcie_port. This patch set pp->root_bus_nr = 0
> in each PCIe host driver which is based on pcie-designware.
> 
This is incorrect at least if there are 2 instances of DW PCIe host in
the same SoC without using PCI domains. In that the case the bus range
determines the range of valid bus numbers per instance and the first
number is the root bus. Please look at the "bus-range" DT property in
the DW PCIe bindings.

Also we should finally remove this root-bus setup from the glue drivers
altogether. It's something that entirely belongs into the DW core code.

Regards,
Lucas

> This patch also try to use of_pci_get_host_bridge_resources for ARM32 and ARM64
> according to the suggestion for Gabriele[1]
> 
> Finally this patch reverts commit f4c55c5a3f7f "PCI: designware: Program ATU
> with untranslated address" based on 1/6 in this series. we delete *_mod_base in
> pcie-designware. This was discussed in [2]
> 
> I have compiled the driver with multi_v7_defconfig. However, I don't have
> ARM32 PCIe related board to do test. It will be appreciated if someone could
> help to test it.
> 
> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> Tested-By: James Morse <james.morse@arm.com>
> 
> [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> [2] http://www.spinics.net/lists/arm-kernel/msg436779.html
> ---
>  drivers/pci/host/pci-dra7xx.c      |  15 +--
>  drivers/pci/host/pci-exynos.c      |   2 +-
>  drivers/pci/host/pci-imx6.c        |   2 +-
>  drivers/pci/host/pci-keystone-dw.c |   2 +-
>  drivers/pci/host/pci-keystone.c    |   2 +-
>  drivers/pci/host/pci-layerscape.c  |   2 +-
>  drivers/pci/host/pcie-designware.c | 229 ++++++++++++-------------------------
>  drivers/pci/host/pcie-designware.h |  14 +--
>  drivers/pci/host/pcie-spear13xx.c  |   2 +-
>  9 files changed, 95 insertions(+), 175 deletions(-)
> 
> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
> index 18ae7ff..1268c69 100644
> --- a/drivers/pci/host/pci-dra7xx.c
> +++ b/drivers/pci/host/pci-dra7xx.c
> @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
>  {
>  	dw_pcie_setup_rc(pp);
>  
> -	if (pp->io_mod_base)
> -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->io_base)
> +		pp->io_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->mem_mod_base)
> -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->mem_base)
> +		pp->mem_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->cfg0_mod_base) {
> -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->cfg0_base) {
> +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
>  	}
>  
>  	dra7xx_pcie_establish_link(pp);
> @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
>  
>  	pp = &dra7xx->pp;
>  	pp->dev = dev;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &dra7xx_pcie_host_ops;
>  
>  	pp->irq = platform_get_irq(pdev, 1);
> diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-exynos.c
> index f9f468d..9771bb0 100644
> --- a/drivers/pci/host/pci-exynos.c
> +++ b/drivers/pci/host/pci-exynos.c
> @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &exynos_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
> index 233a196..bec256c 100644
> --- a/drivers/pci/host/pci-imx6.c
> +++ b/drivers/pci/host/pci-imx6.c
> @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &imx6_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
> index f34892e..b1e4135 100644
> --- a/drivers/pci/host/pci-keystone-dw.c
> +++ b/drivers/pci/host/pci-keystone-dw.c
> @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
>  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
>  {
>  	struct pcie_port *pp = &ks_pcie->pp;
> -	u32 start = pp->mem.start, end = pp->mem.end;
> +	u32 start = pp->mem->start, end = pp->mem->end;
>  	int i, tr_size;
>  
>  	/* Disable BARs for inbound access */
> diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
> index 734da58..8113832 100644
> --- a/drivers/pci/host/pci-keystone.c
> +++ b/drivers/pci/host/pci-keystone.c
> @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
>  			return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &keystone_pcie_host_ops;
>  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
>  	if (ret) {
> diff --git a/drivers/pci/host/pci-layerscape.c b/drivers/pci/host/pci-layerscape.c
> index b2328ea1..79ff08c 100644
> --- a/drivers/pci/host/pci-layerscape.c
> +++ b/drivers/pci/host/pci-layerscape.c
> @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
>  	pp = &pcie->pp;
>  	pp->dev = pcie->dev;
>  	pp->dbi_base = pcie->dbi;
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &ls_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
> index c5d407c..e71a88e 100644
> --- a/drivers/pci/host/pcie-designware.c
> +++ b/drivers/pci/host/pcie-designware.c
> @@ -11,6 +11,7 @@
>   * published by the Free Software Foundation.
>   */
>  
> +#include <linux/hardirq.h>
>  #include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kernel.h>
> @@ -69,16 +70,7 @@
>  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
>  #define PCIE_ATU_UPPER_TARGET		0x91C
>  
> -static struct hw_pci dw_pci;
> -
> -static unsigned long global_io_offset;
> -
> -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> -{
> -	BUG_ON(!sys->private_data);
> -
> -	return sys->private_data;
> -}
> +static struct pci_ops dw_pcie_ops;
>  
>  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
>  {
> @@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
>  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
>  {
>  	int irq, pos0, i;
> -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> +	struct pcie_port *pp = desc->dev->bus->sysdata;
>  
>  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
>  				       order_base_2(no_irqs));
> @@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
>  {
>  	int irq, pos;
>  	struct msi_msg msg;
> -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> +	struct pcie_port *pp = pdev->bus->sysdata;
>  
>  	if (desc->msi_attrib.is_msix)
>  		return -EINVAL;
> @@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
>  {
>  	struct irq_data *data = irq_get_irq_data(irq);
>  	struct msi_desc *msi = irq_data_get_msi(data);
> -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> +	struct pcie_port *pp = msi->dev->bus->sysdata;
>  
>  	clear_irq_range(pp, irq, 1, data->hwirq);
>  }
> @@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  {
>  	struct device_node *np = pp->dev->of_node;
>  	struct platform_device *pdev = to_platform_device(pp->dev);
> -	struct of_pci_range range;
> -	struct of_pci_range_parser parser;
> +	struct pci_bus *bus;
>  	struct resource *cfg_res;
> -	u32 val, ns;
> -	const __be32 *addrp;
> -	int i, index, ret;
> -
> -	ns = of_n_size_cells(np);
> +	LIST_HEAD(res);
> +	u32 val;
> +	int i, ret;
> +	struct resource_entry *win;
>  
>  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
>  	if (cfg_res) {
> @@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  		pp->cfg1_size = resource_size(cfg_res)/2;
>  		pp->cfg0_base = cfg_res->start;
>  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> -
> -		/* Find the untranslated configuration space address */
> -		index = of_property_match_string(np, "reg-names", "config");
> -		addrp = of_get_address(np, index, NULL, NULL);
> -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
>  	} else {
>  		dev_err(pp->dev, "missing *config* reg space\n");
>  	}
>  
> -	if (of_pci_range_parser_init(&parser, np)) {
> -		dev_err(pp->dev, "missing ranges property\n");
> -		return -EINVAL;
> -	}
> +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
> +	if (ret)
> +		return ret;
>  
>  	/* Get the I/O and memory ranges from DT */
> -	for_each_of_pci_range(&parser, &range) {
> -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> -
> -		if (restype == IORESOURCE_IO) {
> -			of_pci_range_to_resource(&range, np, &pp->io);
> -			pp->io.name = "I/O";
> -			pp->io.start = max_t(resource_size_t,
> -					     PCIBIOS_MIN_IO,
> -					     range.pci_addr + global_io_offset);
> -			pp->io.end = min_t(resource_size_t,
> -					   IO_SPACE_LIMIT,
> -					   range.pci_addr + range.size
> -					   + global_io_offset - 1);
> -			pp->io_size = resource_size(&pp->io);
> -			pp->io_bus_addr = range.pci_addr;
> -			pp->io_base = range.cpu_addr;
> -
> -			/* Find the untranslated IO space address */
> -			pp->io_mod_base = range.cpu_addr;
> -		}
> -		if (restype == IORESOURCE_MEM) {
> -			of_pci_range_to_resource(&range, np, &pp->mem);
> -			pp->mem.name = "MEM";
> -			pp->mem_size = resource_size(&pp->mem);
> -			pp->mem_bus_addr = range.pci_addr;
> -
> -			/* Find the untranslated MEM space address */
> -			pp->mem_mod_base = range.cpu_addr;
> -		}
> -		if (restype == 0) {
> -			of_pci_range_to_resource(&range, np, &pp->cfg);
> -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> -			pp->cfg0_base = pp->cfg.start;
> -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> -
> -			/* Find the untranslated configuration space address */
> -			pp->cfg0_mod_base = range.cpu_addr;
> -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> -					    pp->cfg0_size;
> +	resource_list_for_each_entry(win, &res) {
> +		switch (resource_type(win->res)) {
> +		case IORESOURCE_IO:
> +			pp->io = win->res;
> +			pp->io->name = "I/O";
> +			pp->io_size = resource_size(pp->io);
> +			pp->io_bus_addr = pp->io->start - win->offset;
> +			ret = pci_remap_iospace(pp->io, pp->io_base);
> +			if (ret) {
> +				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
> +					 ret, pp->io);
> +				continue;
> +			}
> +			break;
> +		case IORESOURCE_MEM:
> +			pp->mem = win->res;
> +			pp->mem->name = "MEM";
> +			pp->mem_size = resource_size(pp->mem);
> +			pp->mem_bus_addr = pp->mem->start - win->offset;
> +			break;
> +		case 0:
> +			pp->cfg = win->res;
> +			pp->cfg0_size = resource_size(pp->cfg)/2;
> +			pp->cfg1_size = resource_size(pp->cfg)/2;
> +			pp->cfg0_base = pp->cfg->start;
> +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> +			break;
> +		case IORESOURCE_BUS:
> +			pp->busn = win->res;
> +			break;
> +		default:
> +			continue;
>  		}
>  	}
>  
> -	ret = of_pci_parse_bus_range(np, &pp->busn);
> -	if (ret < 0) {
> -		pp->busn.name = np->name;
> -		pp->busn.start = 0;
> -		pp->busn.end = 0xff;
> -		pp->busn.flags = IORESOURCE_BUS;
> -		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
> -			ret, &pp->busn);
> -	}
> -
>  	if (!pp->dbi_base) {
> -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> -					resource_size(&pp->cfg));
> +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> +					resource_size(pp->cfg));
>  		if (!pp->dbi_base) {
>  			dev_err(pp->dev, "error with ioremap\n");
>  			return -ENOMEM;
>  		}
>  	}
>  
> -	pp->mem_base = pp->mem.start;
> +	pp->mem_base = pp->mem->start;
>  
>  	if (!pp->va_cfg0_base) {
>  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> @@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  
>  	if (!pp->ops->rd_other_conf)
>  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
>  					  pp->mem_bus_addr, pp->mem_size);
>  
>  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> @@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  	val |= PORT_LOGIC_SPEED_CHANGE;
>  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
>  
> -#ifdef CONFIG_PCI_MSI
> +	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
> +				  pp, &res);
> +	if (!bus)
> +		return -ENOMEM;
> +
> +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> +	bus->msi = container_of(&pp->irq_domain, struct msi_controller, domain);
> +#else
>  	dw_pcie_msi_chip.dev = pp->dev;
> -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> +	bus->msi = &dw_pcie_msi_chip;
>  #endif
>  
> -	dw_pci.nr_controllers = 1;
> -	dw_pci.private_data = (void **)&pp;
> +	pci_scan_child_bus(bus);
> +	if (pp->ops->scan_bus)
> +		pp->ops->scan_bus(pp);
> +
> +#ifdef CONFIG_ARM
> +	/* support old dtbs that incorrectly describe IRQs */
> +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> +#endif
>  
> -	pci_common_init_dev(pp->dev, &dw_pci);
> +	pci_assign_unassigned_bus_resources(bus);
> +	pci_bus_add_devices(bus);
>  
>  	return 0;
>  }
> @@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
>  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  			int size, u32 *val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> @@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
>  			int where, int size, u32 val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> @@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
>  	.write = dw_pcie_wr_conf,
>  };
>  
> -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> -{
> -	struct pcie_port *pp;
> -
> -	pp = sys_to_pcie(sys);
> -
> -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> -		pci_ioremap_io(global_io_offset, pp->io_base);
> -		global_io_offset += SZ_64K;
> -		pci_add_resource_offset(&sys->resources, &pp->io,
> -					sys->io_offset);
> -	}
> -
> -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> -	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
> -	pci_add_resource(&sys->resources, &pp->busn);
> -
> -	return 1;
> -}
> -
> -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
> -{
> -	struct pci_bus *bus;
> -	struct pcie_port *pp = sys_to_pcie(sys);
> -
> -	pp->root_bus_nr = sys->busnr;
> -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> -				  &dw_pcie_ops, sys, &sys->resources);
> -	if (!bus)
> -		return NULL;
> -
> -	if (bus && pp->ops->scan_bus)
> -		pp->ops->scan_bus(pp);
> -
> -	return bus;
> -}
> -
> -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
> -{
> -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> -	int irq;
> -
> -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> -	if (!irq)
> -		irq = pp->irq;
> -
> -	return irq;
> -}
> -
> -static struct hw_pci dw_pci = {
> -	.setup		= dw_pcie_setup,
> -	.scan		= dw_pcie_scan_bus,
> -	.map_irq	= dw_pcie_map_irq,
> -};
> -
>  void dw_pcie_setup_rc(struct pcie_port *pp)
>  {
>  	u32 val;
> diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
> index d0bbd27..264c969 100644
> --- a/drivers/pci/host/pcie-designware.h
> +++ b/drivers/pci/host/pcie-designware.h
> @@ -27,25 +27,21 @@ struct pcie_port {
>  	u8			root_bus_nr;
>  	void __iomem		*dbi_base;
>  	u64			cfg0_base;
> -	u64			cfg0_mod_base;
>  	void __iomem		*va_cfg0_base;
>  	u32			cfg0_size;
>  	u64			cfg1_base;
> -	u64			cfg1_mod_base;
>  	void __iomem		*va_cfg1_base;
>  	u32			cfg1_size;
> -	u64			io_base;
> -	u64			io_mod_base;
> +	resource_size_t		io_base;
>  	phys_addr_t		io_bus_addr;
>  	u32			io_size;
>  	u64			mem_base;
> -	u64			mem_mod_base;
>  	phys_addr_t		mem_bus_addr;
>  	u32			mem_size;
> -	struct resource		cfg;
> -	struct resource		io;
> -	struct resource		mem;
> -	struct resource		busn;
> +	struct resource		*cfg;
> +	struct resource		*io;
> +	struct resource		*mem;
> +	struct resource		*busn;
>  	int			irq;
>  	u32			lanes;
>  	struct pcie_host_ops	*ops;
> diff --git a/drivers/pci/host/pcie-spear13xx.c b/drivers/pci/host/pcie-spear13xx.c
> index c49fbdc..03eb204 100644
> --- a/drivers/pci/host/pcie-spear13xx.c
> +++ b/drivers/pci/host/pcie-spear13xx.c
> @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct pcie_port *pp,
>  		return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = 0;
>  	pp->ops = &spear13xx_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-19 12:54   ` Lucas Stach
@ 2015-08-19 15:16     ` Gabriele Paoloni
  2015-08-19 15:37       ` Lucas Stach
  0 siblings, 1 reply; 18+ messages in thread
From: Gabriele Paoloni @ 2015-08-19 15:16 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Lucas

I have rewritten the patch to take into account multiple controllers.

As you can see now there is a static var in dw_pcie_host_init() that tracks
the bus numbers used.
Drivers that do not specify the bus range in the DTB set pp->root_bus_nr = DW_ROOT_NR_UNDEFINED.
Designware will check if the flag is set and will use the automatic bus range
assignment.
Instead if the driver assigns pp->root_bus_nr according to the dtb, designwware
will use the value passed in by the driver.
Below the relevant section:


+	static int root_bus_nr = 0;
...
+	mutex_lock(&root_bus_nr_mux);
+
+	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
+		root_bus_nr = pp->root_bus_nr;
+
+	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
+			      pp, &res);
+	if (!bus) {
+		mutex_unlock(&root_bus_nr_mux);
+		return -ENOMEM;
+	}
+
+	root_bus_nr += bus->busn_res.end + 1;
+	mutex_unlock(&root_bus_nr_mux);

Please let me know what you think...

Many Thanks

Gab
----------

From: gabriele paoloni <gabriele.paoloni@huawei.com>

This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
move related operations to dw_pcie_host_init.
Also set pp->root_bus_nr = DW_ROOT_NR_UNDEFINED in all the drivers that
are based on designware to flag that the drivers do not read the bus
ranges from DT.
This patch also adds handling of multiple PCI domains in designware.
if the PCI host bridge driver does not specify a root bus number, in case
of multiple domains, designware will automatillay set the next domain root
bus number to the last bus number used in the last domain + 1.

This patch also try to use of_pci_get_host_bridge_resources for ARM32 and
ARM64 according to the suggestion for Gabriele[1]

This patch is based on Gabriele's patch about of_pci_range fix[2]

Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
Program ATU with untranslated address". This was discussed in [3]

I have compiled the driver with multi_v7_defconfig. However, I don't have
ARM32 PCIe related board to do test. It will be appreciated if someone could
help to test it.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>

[1] http://www.spinics.net/lists/linux-pci/msg42194.html
[2] https://patchwork.ozlabs.org/patch/495018/
[3] http://www.spinics.net/lists/arm-kernel/msg436779.html
---
 drivers/pci/host/pci-dra7xx.c      |  15 +--
 drivers/pci/host/pci-exynos.c      |   2 +-
 drivers/pci/host/pci-imx6.c        |   2 +-
 drivers/pci/host/pci-keystone-dw.c |   2 +-
 drivers/pci/host/pci-keystone.c    |   2 +-
 drivers/pci/host/pci-layerscape.c  |   2 +-
 drivers/pci/host/pcie-designware.c | 247 ++++++++++++++-----------------------
 drivers/pci/host/pcie-designware.h |  15 +--
 drivers/pci/host/pcie-spear13xx.c  |   2 +-
 9 files changed, 110 insertions(+), 179 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 5678b57..8d598fb 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
 	dw_pcie_setup_rc(pp);
 
-	if (pp->io_mod_base)
-		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->io_base)
+		pp->io_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->mem_mod_base)
-		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->mem_base)
+		pp->mem_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->cfg0_mod_base) {
-		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
-		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->cfg0_base) {
+		pp->cfg0_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_base &= CPU_TO_BUS_ADDR;
 	}
 
 	dra7xx_pcie_establish_link(pp);
@@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
 
 	pp = &dra7xx->pp;
 	pp->dev = dev;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &dra7xx_pcie_host_ops;
 
 	pp->irq = platform_get_irq(pdev, 1);
diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-exynos.c
index f9f468d..ed03a8f 100644
--- a/drivers/pci/host/pci-exynos.c
+++ b/drivers/pci/host/pci-exynos.c
@@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct pcie_port *pp,
 		}
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &exynos_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
index 233a196..0efac85 100644
--- a/drivers/pci/host/pci-imx6.c
+++ b/drivers/pci/host/pci-imx6.c
@@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
 		}
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &imx6_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
index f34892e..b1e4135 100644
--- a/drivers/pci/host/pci-keystone-dw.c
+++ b/drivers/pci/host/pci-keystone-dw.c
@@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
 void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
 {
 	struct pcie_port *pp = &ks_pcie->pp;
-	u32 start = pp->mem.start, end = pp->mem.end;
+	u32 start = pp->mem->start, end = pp->mem->end;
 	int i, tr_size;
 
 	/* Disable BARs for inbound access */
diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
index 734da58..b522956 100644
--- a/drivers/pci/host/pci-keystone.c
+++ b/drivers/pci/host/pci-keystone.c
@@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
 			return ret;
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &keystone_pcie_host_ops;
 	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
 	if (ret) {
diff --git a/drivers/pci/host/pci-layerscape.c b/drivers/pci/host/pci-layerscape.c
index b2328ea1..dd92ffa 100644
--- a/drivers/pci/host/pci-layerscape.c
+++ b/drivers/pci/host/pci-layerscape.c
@@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
 	pp = &pcie->pp;
 	pp->dev = pcie->dev;
 	pp->dbi_base = pcie->dbi;
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &ls_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 5307b35..bd2606b 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -11,6 +11,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/hardirq.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kernel.h>
@@ -69,16 +70,9 @@
 #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
 #define PCIE_ATU_UPPER_TARGET		0x91C
 
-static struct hw_pci dw_pci;
+static struct pci_ops dw_pcie_ops;
 
-static unsigned long global_io_offset;
-
-static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
-{
-	BUG_ON(!sys->private_data);
-
-	return sys->private_data;
-}
+DEFINE_MUTEX(root_bus_nr_mux);
 
 int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
 {
@@ -255,7 +249,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
 static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
 {
 	int irq, pos0, i;
-	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
+	struct pcie_port *pp = desc->dev->bus->sysdata;
 
 	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
 				       order_base_2(no_irqs));
@@ -298,7 +292,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
 {
 	int irq, pos;
 	struct msi_msg msg;
-	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
+	struct pcie_port *pp = pdev->bus->sysdata;
 
 	if (desc->msi_attrib.is_msix)
 		return -EINVAL;
@@ -327,7 +321,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
 {
 	struct irq_data *data = irq_get_irq_data(irq);
 	struct msi_desc *msi = irq_data_get_msi(data);
-	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
+	struct pcie_port *pp = msi->dev->bus->sysdata;
 
 	clear_irq_range(pp, irq, 1, data->hwirq);
 }
@@ -359,22 +353,17 @@ static const struct irq_domain_ops msi_domain_ops = {
 	.map = dw_pcie_msi_map,
 };
 
-int dw_pcie_host_init(struct pcie_port *pp)
+int __init dw_pcie_host_init(struct pcie_port *pp)
 {
 	struct device_node *np = pp->dev->of_node;
 	struct platform_device *pdev = to_platform_device(pp->dev);
-	struct of_pci_range range;
-	struct of_pci_range_parser parser;
+	struct pci_bus *bus;
 	struct resource *cfg_res;
-	u32 val, na, ns;
-	const __be32 *addrp;
-	int i, index, ret;
-
-	/* Find the address cell size and the number of cells in order to get
-	 * the untranslated address.
-	 */
-	of_property_read_u32(np, "#address-cells", &na);
-	ns = of_n_size_cells(np);
+	LIST_HEAD(res);
+	u32 val;
+	int i, ret;
+	struct resource_entry *win;
+	static int root_bus_nr = 0;
 
 	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
 	if (cfg_res) {
@@ -382,85 +371,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
 		pp->cfg1_size = resource_size(cfg_res)/2;
 		pp->cfg0_base = cfg_res->start;
 		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
-
-		/* Find the untranslated configuration space address */
-		index = of_property_match_string(np, "reg-names", "config");
-		addrp = of_get_address(np, index, NULL, NULL);
-		pp->cfg0_mod_base = of_read_number(addrp, ns);
-		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
 	} else {
 		dev_err(pp->dev, "missing *config* reg space\n");
 	}
 
-	if (of_pci_range_parser_init(&parser, np)) {
-		dev_err(pp->dev, "missing ranges property\n");
-		return -EINVAL;
-	}
+	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
+	if (ret)
+		return ret;
 
 	/* Get the I/O and memory ranges from DT */
-	for_each_of_pci_range(&parser, &range) {
-		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
-
-		if (restype == IORESOURCE_IO) {
-			of_pci_range_to_resource(&range, np, &pp->io);
-			pp->io.name = "I/O";
-			pp->io.start = max_t(resource_size_t,
-					     PCIBIOS_MIN_IO,
-					     range.pci_addr + global_io_offset);
-			pp->io.end = min_t(resource_size_t,
-					   IO_SPACE_LIMIT,
-					   range.pci_addr + range.size
-					   + global_io_offset - 1);
-			pp->io_size = resource_size(&pp->io);
-			pp->io_bus_addr = range.pci_addr;
-			pp->io_base = range.cpu_addr;
-
-			/* Find the untranslated IO space address */
-			pp->io_mod_base = range.cpu_addr;
-		}
-		if (restype == IORESOURCE_MEM) {
-			of_pci_range_to_resource(&range, np, &pp->mem);
-			pp->mem.name = "MEM";
-			pp->mem_size = resource_size(&pp->mem);
-			pp->mem_bus_addr = range.pci_addr;
-
-			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = range.cpu_addr;
-		}
-		if (restype == 0) {
-			of_pci_range_to_resource(&range, np, &pp->cfg);
-			pp->cfg0_size = resource_size(&pp->cfg)/2;
-			pp->cfg1_size = resource_size(&pp->cfg)/2;
-			pp->cfg0_base = pp->cfg.start;
-			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
-
-			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = range.cpu_addr;
-			pp->cfg1_mod_base = pp->cfg0_mod_base +
-					    pp->cfg0_size;
+	resource_list_for_each_entry(win, &res) {
+		switch (resource_type(win->res)) {
+		case IORESOURCE_IO:
+			pp->io = win->res;
+			pp->io->name = "I/O";
+			pp->io_size = resource_size(pp->io);
+			pp->io_bus_addr = pp->io->start - win->offset;
+			ret = pci_remap_iospace(pp->io, pp->io_base);
+			if (ret) {
+				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
+					 ret, pp->io);
+				continue;
+			}
+			break;
+		case IORESOURCE_MEM:
+			pp->mem = win->res;
+			pp->mem->name = "MEM";
+			pp->mem_size = resource_size(pp->mem);
+			pp->mem_bus_addr = pp->mem->start - win->offset;
+			break;
+		case 0:
+			pp->cfg = win->res;
+			pp->cfg0_size = resource_size(pp->cfg)/2;
+			pp->cfg1_size = resource_size(pp->cfg)/2;
+			pp->cfg0_base = pp->cfg->start;
+			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
+			break;
+		case IORESOURCE_BUS:
+			pp->busn = win->res;
+			break;
+		default:
+			continue;
 		}
 	}
 
-	ret = of_pci_parse_bus_range(np, &pp->busn);
-	if (ret < 0) {
-		pp->busn.name = np->name;
-		pp->busn.start = 0;
-		pp->busn.end = 0xff;
-		pp->busn.flags = IORESOURCE_BUS;
-		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
-			ret, &pp->busn);
-	}
-
 	if (!pp->dbi_base) {
-		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
-					resource_size(&pp->cfg));
+		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
+					resource_size(pp->cfg));
 		if (!pp->dbi_base) {
 			dev_err(pp->dev, "error with ioremap\n");
 			return -ENOMEM;
 		}
 	}
 
-	pp->mem_base = pp->mem.start;
+	pp->mem_base = pp->mem->start;
 
 	if (!pp->va_cfg0_base) {
 		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -509,7 +473,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 
 	if (!pp->ops->rd_other_conf)
 		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
-					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
+					  PCIE_ATU_TYPE_MEM, pp->mem_base,
 					  pp->mem_bus_addr, pp->mem_size);
 
 	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
@@ -521,15 +485,40 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	val |= PORT_LOGIC_SPEED_CHANGE;
 	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
 
-#ifdef CONFIG_PCI_MSI
 	dw_pcie_msi_chip.dev = pp->dev;
-	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
+
+	mutex_lock(&root_bus_nr_mux);
+
+	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
+		root_bus_nr = pp->root_bus_nr;
+
+	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
+			      pp, &res);
+	if (!bus) {
+		mutex_unlock(&root_bus_nr_mux);
+		return -ENOMEM;
+	}
+
+	root_bus_nr += bus->busn_res.end + 1;
+	mutex_unlock(&root_bus_nr_mux);
+
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+	bus->msi = container_of(&pp->irq_domain, struct msi_controller, domain);
+#else
+	bus->msi = &dw_pcie_msi_chip;
 #endif
 
-	dw_pci.nr_controllers = 1;
-	dw_pci.private_data = (void **)&pp;
+	pci_scan_child_bus(bus);
+	if (pp->ops->scan_bus)
+		pp->ops->scan_bus(pp);
+
+#ifdef CONFIG_ARM
+	/* support old dtbs that incorrectly describe IRQs */
+	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
+#endif
 
-	pci_common_init_dev(pp->dev, &dw_pci);
+	pci_assign_unassigned_bus_resources(bus);
+	pci_bus_add_devices(bus);
 
 	return 0;
 }
@@ -548,12 +537,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -563,7 +552,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -583,12 +572,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -598,7 +587,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -630,7 +619,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
 static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 			int size, u32 *val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
@@ -654,7 +643,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
 			int where, int size, u32 val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
@@ -678,62 +667,6 @@ static struct pci_ops dw_pcie_ops = {
 	.write = dw_pcie_wr_conf,
 };
 
-static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
-{
-	struct pcie_port *pp;
-
-	pp = sys_to_pcie(sys);
-
-	if (global_io_offset < SZ_1M && pp->io_size > 0) {
-		sys->io_offset = global_io_offset - pp->io_bus_addr;
-		pci_ioremap_io(global_io_offset, pp->io_base);
-		global_io_offset += SZ_64K;
-		pci_add_resource_offset(&sys->resources, &pp->io,
-					sys->io_offset);
-	}
-
-	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
-	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
-	pci_add_resource(&sys->resources, &pp->busn);
-
-	return 1;
-}
-
-static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
-{
-	struct pci_bus *bus;
-	struct pcie_port *pp = sys_to_pcie(sys);
-
-	pp->root_bus_nr = sys->busnr;
-	bus = pci_scan_root_bus(pp->dev, sys->busnr,
-				  &dw_pcie_ops, sys, &sys->resources);
-	if (!bus)
-		return NULL;
-
-	if (bus && pp->ops->scan_bus)
-		pp->ops->scan_bus(pp);
-
-	return bus;
-}
-
-static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
-{
-	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
-	int irq;
-
-	irq = of_irq_parse_and_map_pci(dev, slot, pin);
-	if (!irq)
-		irq = pp->irq;
-
-	return irq;
-}
-
-static struct hw_pci dw_pci = {
-	.setup		= dw_pcie_setup,
-	.scan		= dw_pcie_scan_bus,
-	.map_irq	= dw_pcie_map_irq,
-};
-
 void dw_pcie_setup_rc(struct pcie_port *pp)
 {
 	u32 val;
diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
index d0bbd27..0c2a7eb 100644
--- a/drivers/pci/host/pcie-designware.h
+++ b/drivers/pci/host/pcie-designware.h
@@ -21,31 +21,28 @@
  */
 #define MAX_MSI_IRQS			32
 #define MAX_MSI_CTRLS			(MAX_MSI_IRQS / 32)
+#define DW_ROOT_NR_UNDEFINED	-1
 
 struct pcie_port {
 	struct device		*dev;
 	u8			root_bus_nr;
 	void __iomem		*dbi_base;
 	u64			cfg0_base;
-	u64			cfg0_mod_base;
 	void __iomem		*va_cfg0_base;
 	u32			cfg0_size;
 	u64			cfg1_base;
-	u64			cfg1_mod_base;
 	void __iomem		*va_cfg1_base;
 	u32			cfg1_size;
-	u64			io_base;
-	u64			io_mod_base;
+	resource_size_t		io_base;
 	phys_addr_t		io_bus_addr;
 	u32			io_size;
 	u64			mem_base;
-	u64			mem_mod_base;
 	phys_addr_t		mem_bus_addr;
 	u32			mem_size;
-	struct resource		cfg;
-	struct resource		io;
-	struct resource		mem;
-	struct resource		busn;
+	struct resource		*cfg;
+	struct resource		*io;
+	struct resource		*mem;
+	struct resource		*busn;
 	int			irq;
 	u32			lanes;
 	struct pcie_host_ops	*ops;
diff --git a/drivers/pci/host/pcie-spear13xx.c b/drivers/pci/host/pcie-spear13xx.c
index c49fbdc..b2c59b9 100644
--- a/drivers/pci/host/pcie-spear13xx.c
+++ b/drivers/pci/host/pcie-spear13xx.c
@@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct pcie_port *pp,
 		return ret;
 	}
 
-	pp->root_bus_nr = -1;
+	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
 	pp->ops = &spear13xx_pcie_host_ops;
 
 	ret = dw_pcie_host_init(pp);
-- 
1.9.1


> -----Original Message-----
> From: Lucas Stach [mailto:l.stach at pengutronix.de]
> Sent: Wednesday, August 19, 2015 1:54 PM
> To: Wangzhou (B)
> Cc: Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand; Arnd Bergmann;
> linux at arm.linux.org.uk; thomas.petazzoni at free-electrons.com; Gabriele
> Paoloni; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> 
> Am Montag, den 17.08.2015, 19:55 +0800 schrieb Zhou Wang:
> > This patch tries to unify ARM32 and ARM64 PCIe in designware driver.
> Delete
> > function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct
> hw_pci,
> > move related operations to dw_pcie_host_init.
> >
> > In past, we use:
> > pci_common_init_dev
> > 	-> pcibios_init_hw
> > 		-> hw->scan (dw_pcie_scan_bus)
> > to pass 0 to root_bus_nr in struct pcie_port. This patch set pp-
> >root_bus_nr = 0
> > in each PCIe host driver which is based on pcie-designware.
> >
> This is incorrect at least if there are 2 instances of DW PCIe host in
> the same SoC without using PCI domains. In that the case the bus range
> determines the range of valid bus numbers per instance and the first
> number is the root bus. Please look at the "bus-range" DT property in
> the DW PCIe bindings.
> 
> Also we should finally remove this root-bus setup from the glue drivers
> altogether. It's something that entirely belongs into the DW core code.
> 
> Regards,
> Lucas
> 
> > This patch also try to use of_pci_get_host_bridge_resources for ARM32
> and ARM64
> > according to the suggestion for Gabriele[1]
> >
> > Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
> Program ATU
> > with untranslated address" based on 1/6 in this series. we delete
> *_mod_base in
> > pcie-designware. This was discussed in [2]
> >
> > I have compiled the driver with multi_v7_defconfig. However, I don't
> have
> > ARM32 PCIe related board to do test. It will be appreciated if
> someone could
> > help to test it.
> >
> > Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> > Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> > Tested-By: James Morse <james.morse@arm.com>
> >
> > [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> > [2] http://www.spinics.net/lists/arm-kernel/msg436779.html
> > ---
> >  drivers/pci/host/pci-dra7xx.c      |  15 +--
> >  drivers/pci/host/pci-exynos.c      |   2 +-
> >  drivers/pci/host/pci-imx6.c        |   2 +-
> >  drivers/pci/host/pci-keystone-dw.c |   2 +-
> >  drivers/pci/host/pci-keystone.c    |   2 +-
> >  drivers/pci/host/pci-layerscape.c  |   2 +-
> >  drivers/pci/host/pcie-designware.c | 229 ++++++++++++---------------
> ----------
> >  drivers/pci/host/pcie-designware.h |  14 +--
> >  drivers/pci/host/pcie-spear13xx.c  |   2 +-
> >  9 files changed, 95 insertions(+), 175 deletions(-)
> >
> > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> dra7xx.c
> > index 18ae7ff..1268c69 100644
> > --- a/drivers/pci/host/pci-dra7xx.c
> > +++ b/drivers/pci/host/pci-dra7xx.c
> > @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct
> pcie_port *pp)
> >  {
> >  	dw_pcie_setup_rc(pp);
> >
> > -	if (pp->io_mod_base)
> > -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->io_base)
> > +		pp->io_base &= CPU_TO_BUS_ADDR;
> >
> > -	if (pp->mem_mod_base)
> > -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->mem_base)
> > +		pp->mem_base &= CPU_TO_BUS_ADDR;
> >
> > -	if (pp->cfg0_mod_base) {
> > -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->cfg0_base) {
> > +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> > +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
> >  	}
> >
> >  	dra7xx_pcie_establish_link(pp);
> > @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct
> dra7xx_pcie *dra7xx,
> >
> >  	pp = &dra7xx->pp;
> >  	pp->dev = dev;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &dra7xx_pcie_host_ops;
> >
> >  	pp->irq = platform_get_irq(pdev, 1);
> > diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-
> exynos.c
> > index f9f468d..9771bb0 100644
> > --- a/drivers/pci/host/pci-exynos.c
> > +++ b/drivers/pci/host/pci-exynos.c
> > @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct
> pcie_port *pp,
> >  		}
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &exynos_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-
> imx6.c
> > index 233a196..bec256c 100644
> > --- a/drivers/pci/host/pci-imx6.c
> > +++ b/drivers/pci/host/pci-imx6.c
> > @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct
> pcie_port *pp,
> >  		}
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &imx6_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pci-keystone-dw.c
> b/drivers/pci/host/pci-keystone-dw.c
> > index f34892e..b1e4135 100644
> > --- a/drivers/pci/host/pci-keystone-dw.c
> > +++ b/drivers/pci/host/pci-keystone-dw.c
> > @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void
> __iomem *reg_virt)
> >  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
> >  {
> >  	struct pcie_port *pp = &ks_pcie->pp;
> > -	u32 start = pp->mem.start, end = pp->mem.end;
> > +	u32 start = pp->mem->start, end = pp->mem->end;
> >  	int i, tr_size;
> >
> >  	/* Disable BARs for inbound access */
> > diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-
> keystone.c
> > index 734da58..8113832 100644
> > --- a/drivers/pci/host/pci-keystone.c
> > +++ b/drivers/pci/host/pci-keystone.c
> > @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct
> keystone_pcie *ks_pcie,
> >  			return ret;
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &keystone_pcie_host_ops;
> >  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
> >  	if (ret) {
> > diff --git a/drivers/pci/host/pci-layerscape.c
> b/drivers/pci/host/pci-layerscape.c
> > index b2328ea1..79ff08c 100644
> > --- a/drivers/pci/host/pci-layerscape.c
> > +++ b/drivers/pci/host/pci-layerscape.c
> > @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
> >  	pp = &pcie->pp;
> >  	pp->dev = pcie->dev;
> >  	pp->dbi_base = pcie->dbi;
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &ls_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pcie-designware.c
> b/drivers/pci/host/pcie-designware.c
> > index c5d407c..e71a88e 100644
> > --- a/drivers/pci/host/pcie-designware.c
> > +++ b/drivers/pci/host/pcie-designware.c
> > @@ -11,6 +11,7 @@
> >   * published by the Free Software Foundation.
> >   */
> >
> > +#include <linux/hardirq.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/kernel.h>
> > @@ -69,16 +70,7 @@
> >  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
> >  #define PCIE_ATU_UPPER_TARGET		0x91C
> >
> > -static struct hw_pci dw_pci;
> > -
> > -static unsigned long global_io_offset;
> > -
> > -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> > -{
> > -	BUG_ON(!sys->private_data);
> > -
> > -	return sys->private_data;
> > -}
> > +static struct pci_ops dw_pcie_ops;
> >
> >  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32
> *val)
> >  {
> > @@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port
> *pp, int irq)
> >  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
> >  {
> >  	int irq, pos0, i;
> > -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> > +	struct pcie_port *pp = desc->dev->bus->sysdata;
> >
> >  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
> >  				       order_base_2(no_irqs));
> > @@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller
> *chip, struct pci_dev *pdev,
> >  {
> >  	int irq, pos;
> >  	struct msi_msg msg;
> > -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> > +	struct pcie_port *pp = pdev->bus->sysdata;
> >
> >  	if (desc->msi_attrib.is_msix)
> >  		return -EINVAL;
> > @@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct
> msi_controller *chip, unsigned int irq)
> >  {
> >  	struct irq_data *data = irq_get_irq_data(irq);
> >  	struct msi_desc *msi = irq_data_get_msi(data);
> > -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> > +	struct pcie_port *pp = msi->dev->bus->sysdata;
> >
> >  	clear_irq_range(pp, irq, 1, data->hwirq);
> >  }
> > @@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >  {
> >  	struct device_node *np = pp->dev->of_node;
> >  	struct platform_device *pdev = to_platform_device(pp->dev);
> > -	struct of_pci_range range;
> > -	struct of_pci_range_parser parser;
> > +	struct pci_bus *bus;
> >  	struct resource *cfg_res;
> > -	u32 val, ns;
> > -	const __be32 *addrp;
> > -	int i, index, ret;
> > -
> > -	ns = of_n_size_cells(np);
> > +	LIST_HEAD(res);
> > +	u32 val;
> > +	int i, ret;
> > +	struct resource_entry *win;
> >
> >  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> "config");
> >  	if (cfg_res) {
> > @@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >  		pp->cfg1_size = resource_size(cfg_res)/2;
> >  		pp->cfg0_base = cfg_res->start;
> >  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> > -
> > -		/* Find the untranslated configuration space address */
> > -		index = of_property_match_string(np, "reg-names", "config");
> > -		addrp = of_get_address(np, index, NULL, NULL);
> > -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> > -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
> >  	} else {
> >  		dev_err(pp->dev, "missing *config* reg space\n");
> >  	}
> >
> > -	if (of_pci_range_parser_init(&parser, np)) {
> > -		dev_err(pp->dev, "missing ranges property\n");
> > -		return -EINVAL;
> > -	}
> > +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp-
> >io_base);
> > +	if (ret)
> > +		return ret;
> >
> >  	/* Get the I/O and memory ranges from DT */
> > -	for_each_of_pci_range(&parser, &range) {
> > -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> > -
> > -		if (restype == IORESOURCE_IO) {
> > -			of_pci_range_to_resource(&range, np, &pp->io);
> > -			pp->io.name = "I/O";
> > -			pp->io.start = max_t(resource_size_t,
> > -					     PCIBIOS_MIN_IO,
> > -					     range.pci_addr + global_io_offset);
> > -			pp->io.end = min_t(resource_size_t,
> > -					   IO_SPACE_LIMIT,
> > -					   range.pci_addr + range.size
> > -					   + global_io_offset - 1);
> > -			pp->io_size = resource_size(&pp->io);
> > -			pp->io_bus_addr = range.pci_addr;
> > -			pp->io_base = range.cpu_addr;
> > -
> > -			/* Find the untranslated IO space address */
> > -			pp->io_mod_base = range.cpu_addr;
> > -		}
> > -		if (restype == IORESOURCE_MEM) {
> > -			of_pci_range_to_resource(&range, np, &pp->mem);
> > -			pp->mem.name = "MEM";
> > -			pp->mem_size = resource_size(&pp->mem);
> > -			pp->mem_bus_addr = range.pci_addr;
> > -
> > -			/* Find the untranslated MEM space address */
> > -			pp->mem_mod_base = range.cpu_addr;
> > -		}
> > -		if (restype == 0) {
> > -			of_pci_range_to_resource(&range, np, &pp->cfg);
> > -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> > -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> > -			pp->cfg0_base = pp->cfg.start;
> > -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > -
> > -			/* Find the untranslated configuration space address
> */
> > -			pp->cfg0_mod_base = range.cpu_addr;
> > -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> > -					    pp->cfg0_size;
> > +	resource_list_for_each_entry(win, &res) {
> > +		switch (resource_type(win->res)) {
> > +		case IORESOURCE_IO:
> > +			pp->io = win->res;
> > +			pp->io->name = "I/O";
> > +			pp->io_size = resource_size(pp->io);
> > +			pp->io_bus_addr = pp->io->start - win->offset;
> > +			ret = pci_remap_iospace(pp->io, pp->io_base);
> > +			if (ret) {
> > +				dev_warn(pp->dev, "error %d: failed to map
> resource %pR\n",
> > +					 ret, pp->io);
> > +				continue;
> > +			}
> > +			break;
> > +		case IORESOURCE_MEM:
> > +			pp->mem = win->res;
> > +			pp->mem->name = "MEM";
> > +			pp->mem_size = resource_size(pp->mem);
> > +			pp->mem_bus_addr = pp->mem->start - win->offset;
> > +			break;
> > +		case 0:
> > +			pp->cfg = win->res;
> > +			pp->cfg0_size = resource_size(pp->cfg)/2;
> > +			pp->cfg1_size = resource_size(pp->cfg)/2;
> > +			pp->cfg0_base = pp->cfg->start;
> > +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> > +			break;
> > +		case IORESOURCE_BUS:
> > +			pp->busn = win->res;
> > +			break;
> > +		default:
> > +			continue;
> >  		}
> >  	}
> >
> > -	ret = of_pci_parse_bus_range(np, &pp->busn);
> > -	if (ret < 0) {
> > -		pp->busn.name = np->name;
> > -		pp->busn.start = 0;
> > -		pp->busn.end = 0xff;
> > -		pp->busn.flags = IORESOURCE_BUS;
> > -		dev_dbg(pp->dev, "failed to parse bus-range property: %d,
> using default %pR\n",
> > -			ret, &pp->busn);
> > -	}
> > -
> >  	if (!pp->dbi_base) {
> > -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> > -					resource_size(&pp->cfg));
> > +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> > +					resource_size(pp->cfg));
> >  		if (!pp->dbi_base) {
> >  			dev_err(pp->dev, "error with ioremap\n");
> >  			return -ENOMEM;
> >  		}
> >  	}
> >
> > -	pp->mem_base = pp->mem.start;
> > +	pp->mem_base = pp->mem->start;
> >
> >  	if (!pp->va_cfg0_base) {
> >  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> > @@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >
> >  	if (!pp->ops->rd_other_conf)
> >  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> > -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> > +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
> >  					  pp->mem_bus_addr, pp->mem_size);
> >
> >  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> > @@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >  	val |= PORT_LOGIC_SPEED_CHANGE;
> >  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
> >
> > -#ifdef CONFIG_PCI_MSI
> > +	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
> > +				  pp, &res);
> > +	if (!bus)
> > +		return -ENOMEM;
> > +
> > +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> > +	bus->msi = container_of(&pp->irq_domain, struct msi_controller,
> domain);
> > +#else
> >  	dw_pcie_msi_chip.dev = pp->dev;
> > -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> > +	bus->msi = &dw_pcie_msi_chip;
> >  #endif
> >
> > -	dw_pci.nr_controllers = 1;
> > -	dw_pci.private_data = (void **)&pp;
> > +	pci_scan_child_bus(bus);
> > +	if (pp->ops->scan_bus)
> > +		pp->ops->scan_bus(pp);
> > +
> > +#ifdef CONFIG_ARM
> > +	/* support old dtbs that incorrectly describe IRQs */
> > +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> > +#endif
> >
> > -	pci_common_init_dev(pp->dev, &dw_pci);
> > +	pci_assign_unassigned_bus_resources(bus);
> > +	pci_bus_add_devices(bus);
> >
> >  	return 0;
> >  }
> > @@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct
> pcie_port *pp, struct pci_bus *bus,
> >
> >  	if (bus->parent->number == pp->root_bus_nr) {
> >  		type = PCIE_ATU_TYPE_CFG0;
> > -		cpu_addr = pp->cfg0_mod_base;
> > +		cpu_addr = pp->cfg0_base;
> >  		cfg_size = pp->cfg0_size;
> >  		va_cfg_base = pp->va_cfg0_base;
> >  	} else {
> >  		type = PCIE_ATU_TYPE_CFG1;
> > -		cpu_addr = pp->cfg1_mod_base;
> > +		cpu_addr = pp->cfg1_base;
> >  		cfg_size = pp->cfg1_size;
> >  		va_cfg_base = pp->va_cfg1_base;
> >  	}
> > @@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port
> *pp, struct pci_bus *bus,
> >  				  busdev, cfg_size);
> >  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
> >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> >  				  pp->io_bus_addr, pp->io_size);
> >
> >  	return ret;
> > @@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct
> pcie_port *pp, struct pci_bus *bus,
> >
> >  	if (bus->parent->number == pp->root_bus_nr) {
> >  		type = PCIE_ATU_TYPE_CFG0;
> > -		cpu_addr = pp->cfg0_mod_base;
> > +		cpu_addr = pp->cfg0_base;
> >  		cfg_size = pp->cfg0_size;
> >  		va_cfg_base = pp->va_cfg0_base;
> >  	} else {
> >  		type = PCIE_ATU_TYPE_CFG1;
> > -		cpu_addr = pp->cfg1_mod_base;
> > +		cpu_addr = pp->cfg1_base;
> >  		cfg_size = pp->cfg1_size;
> >  		va_cfg_base = pp->va_cfg1_base;
> >  	}
> > @@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port
> *pp, struct pci_bus *bus,
> >  				  busdev, cfg_size);
> >  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
> >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> >  				  pp->io_bus_addr, pp->io_size);
> >
> >  	return ret;
> > @@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct pcie_port
> *pp,
> >  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
> >  			int size, u32 *val)
> >  {
> > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > +	struct pcie_port *pp = bus->sysdata;
> >  	int ret;
> >
> >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> > @@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus,
> u32 devfn, int where,
> >  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
> >  			int where, int size, u32 val)
> >  {
> > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > +	struct pcie_port *pp = bus->sysdata;
> >  	int ret;
> >
> >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> > @@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
> >  	.write = dw_pcie_wr_conf,
> >  };
> >
> > -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> > -{
> > -	struct pcie_port *pp;
> > -
> > -	pp = sys_to_pcie(sys);
> > -
> > -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> > -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> > -		pci_ioremap_io(global_io_offset, pp->io_base);
> > -		global_io_offset += SZ_64K;
> > -		pci_add_resource_offset(&sys->resources, &pp->io,
> > -					sys->io_offset);
> > -	}
> > -
> > -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> > -	pci_add_resource_offset(&sys->resources, &pp->mem, sys-
> >mem_offset);
> > -	pci_add_resource(&sys->resources, &pp->busn);
> > -
> > -	return 1;
> > -}
> > -
> > -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data
> *sys)
> > -{
> > -	struct pci_bus *bus;
> > -	struct pcie_port *pp = sys_to_pcie(sys);
> > -
> > -	pp->root_bus_nr = sys->busnr;
> > -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> > -				  &dw_pcie_ops, sys, &sys->resources);
> > -	if (!bus)
> > -		return NULL;
> > -
> > -	if (bus && pp->ops->scan_bus)
> > -		pp->ops->scan_bus(pp);
> > -
> > -	return bus;
> > -}
> > -
> > -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8
> pin)
> > -{
> > -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> > -	int irq;
> > -
> > -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> > -	if (!irq)
> > -		irq = pp->irq;
> > -
> > -	return irq;
> > -}
> > -
> > -static struct hw_pci dw_pci = {
> > -	.setup		= dw_pcie_setup,
> > -	.scan		= dw_pcie_scan_bus,
> > -	.map_irq	= dw_pcie_map_irq,
> > -};
> > -
> >  void dw_pcie_setup_rc(struct pcie_port *pp)
> >  {
> >  	u32 val;
> > diff --git a/drivers/pci/host/pcie-designware.h
> b/drivers/pci/host/pcie-designware.h
> > index d0bbd27..264c969 100644
> > --- a/drivers/pci/host/pcie-designware.h
> > +++ b/drivers/pci/host/pcie-designware.h
> > @@ -27,25 +27,21 @@ struct pcie_port {
> >  	u8			root_bus_nr;
> >  	void __iomem		*dbi_base;
> >  	u64			cfg0_base;
> > -	u64			cfg0_mod_base;
> >  	void __iomem		*va_cfg0_base;
> >  	u32			cfg0_size;
> >  	u64			cfg1_base;
> > -	u64			cfg1_mod_base;
> >  	void __iomem		*va_cfg1_base;
> >  	u32			cfg1_size;
> > -	u64			io_base;
> > -	u64			io_mod_base;
> > +	resource_size_t		io_base;
> >  	phys_addr_t		io_bus_addr;
> >  	u32			io_size;
> >  	u64			mem_base;
> > -	u64			mem_mod_base;
> >  	phys_addr_t		mem_bus_addr;
> >  	u32			mem_size;
> > -	struct resource		cfg;
> > -	struct resource		io;
> > -	struct resource		mem;
> > -	struct resource		busn;
> > +	struct resource		*cfg;
> > +	struct resource		*io;
> > +	struct resource		*mem;
> > +	struct resource		*busn;
> >  	int			irq;
> >  	u32			lanes;
> >  	struct pcie_host_ops	*ops;
> > diff --git a/drivers/pci/host/pcie-spear13xx.c
> b/drivers/pci/host/pcie-spear13xx.c
> > index c49fbdc..03eb204 100644
> > --- a/drivers/pci/host/pcie-spear13xx.c
> > +++ b/drivers/pci/host/pcie-spear13xx.c
> > @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct
> pcie_port *pp,
> >  		return ret;
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = 0;
> >  	pp->ops = &spear13xx_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> 
> --
> Pengutronix e.K.             | Lucas Stach                 |
> Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-19 15:16     ` Gabriele Paoloni
@ 2015-08-19 15:37       ` Lucas Stach
  2015-08-19 16:34         ` Gabriele Paoloni
  0 siblings, 1 reply; 18+ messages in thread
From: Lucas Stach @ 2015-08-19 15:37 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Gab,

Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele Paoloni:
> Hi Lucas
> 
> I have rewritten the patch to take into account multiple controllers.
> 
> As you can see now there is a static var in dw_pcie_host_init() that tracks
> the bus numbers used.

This is wrong. The DT specifies the valid bus number range. You can not
just assign the next free bus number to the root bus.

It is perfectly valid to have a bus range of 0x00-0x10 assigned to one
instance and 0x50-0xff to the next instance. Additional with PCIe
hotplug you may not use the full range of the bus numbers on one
instance at the first scan, but only later populate more buses when more
bridges are added to the tree.

> Drivers that do not specify the bus range in the DTB set pp->root_bus_nr = DW_ROOT_NR_UNDEFINED.
> Designware will check if the flag is set and will use the automatic bus range
> assignment.

No, please lets get rid of this assignment altogether. The glue drivers
have no business in assigning the bus range. Please remove the
pp->root_bus_nr assignment from all the glue drivers.

"bus range" is a generic DW PCIe property, so just parse the root bus
number from the DT, it is handled the same way for all the DW based PCIe
drivers. The bindings specifies that if the bus range property is
missing the range is 0x00-0xff, so you can default to 0 as the root bus
number in that case.

Also I would think this conversion warrants a patch on its own and
should not be mixed in the ARM64 support patch.

Regards,
Lucas

> Instead if the driver assigns pp->root_bus_nr according to the dtb, designwware
> will use the value passed in by the driver.
> Below the relevant section:
> 
> 
> +	static int root_bus_nr = 0;
> ...
> +	mutex_lock(&root_bus_nr_mux);
> +
> +	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
> +		root_bus_nr = pp->root_bus_nr;
> +
> +	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
> +			      pp, &res);
> +	if (!bus) {
> +		mutex_unlock(&root_bus_nr_mux);
> +		return -ENOMEM;
> +	}
> +
> +	root_bus_nr += bus->busn_res.end + 1;
> +	mutex_unlock(&root_bus_nr_mux);
> 
> Please let me know what you think...
> 
> Many Thanks
> 
> Gab
> ----------
> 
> From: gabriele paoloni <gabriele.paoloni@huawei.com>
> 
> This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
> function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
> move related operations to dw_pcie_host_init.
> Also set pp->root_bus_nr = DW_ROOT_NR_UNDEFINED in all the drivers that
> are based on designware to flag that the drivers do not read the bus
> ranges from DT.
> This patch also adds handling of multiple PCI domains in designware.
> if the PCI host bridge driver does not specify a root bus number, in case
> of multiple domains, designware will automatillay set the next domain root
> bus number to the last bus number used in the last domain + 1.
> 
> This patch also try to use of_pci_get_host_bridge_resources for ARM32 and
> ARM64 according to the suggestion for Gabriele[1]
> 
> This patch is based on Gabriele's patch about of_pci_range fix[2]
> 
> Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
> Program ATU with untranslated address". This was discussed in [3]
> 
> I have compiled the driver with multi_v7_defconfig. However, I don't have
> ARM32 PCIe related board to do test. It will be appreciated if someone could
> help to test it.
> 
> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> 
> [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> [2] https://patchwork.ozlabs.org/patch/495018/
> [3] http://www.spinics.net/lists/arm-kernel/msg436779.html
> ---
>  drivers/pci/host/pci-dra7xx.c      |  15 +--
>  drivers/pci/host/pci-exynos.c      |   2 +-
>  drivers/pci/host/pci-imx6.c        |   2 +-
>  drivers/pci/host/pci-keystone-dw.c |   2 +-
>  drivers/pci/host/pci-keystone.c    |   2 +-
>  drivers/pci/host/pci-layerscape.c  |   2 +-
>  drivers/pci/host/pcie-designware.c | 247 ++++++++++++++-----------------------
>  drivers/pci/host/pcie-designware.h |  15 +--
>  drivers/pci/host/pcie-spear13xx.c  |   2 +-
>  9 files changed, 110 insertions(+), 179 deletions(-)
> 
> diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
> index 5678b57..8d598fb 100644
> --- a/drivers/pci/host/pci-dra7xx.c
> +++ b/drivers/pci/host/pci-dra7xx.c
> @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
>  {
>  	dw_pcie_setup_rc(pp);
>  
> -	if (pp->io_mod_base)
> -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->io_base)
> +		pp->io_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->mem_mod_base)
> -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->mem_base)
> +		pp->mem_base &= CPU_TO_BUS_ADDR;
>  
> -	if (pp->cfg0_mod_base) {
> -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> +	if (pp->cfg0_base) {
> +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
>  	}
>  
>  	dra7xx_pcie_establish_link(pp);
> @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct dra7xx_pcie *dra7xx,
>  
>  	pp = &dra7xx->pp;
>  	pp->dev = dev;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &dra7xx_pcie_host_ops;
>  
>  	pp->irq = platform_get_irq(pdev, 1);
> diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-exynos.c
> index f9f468d..ed03a8f 100644
> --- a/drivers/pci/host/pci-exynos.c
> +++ b/drivers/pci/host/pci-exynos.c
> @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &exynos_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
> index 233a196..0efac85 100644
> --- a/drivers/pci/host/pci-imx6.c
> +++ b/drivers/pci/host/pci-imx6.c
> @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct pcie_port *pp,
>  		}
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &imx6_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
> index f34892e..b1e4135 100644
> --- a/drivers/pci/host/pci-keystone-dw.c
> +++ b/drivers/pci/host/pci-keystone-dw.c
> @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
>  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
>  {
>  	struct pcie_port *pp = &ks_pcie->pp;
> -	u32 start = pp->mem.start, end = pp->mem.end;
> +	u32 start = pp->mem->start, end = pp->mem->end;
>  	int i, tr_size;
>  
>  	/* Disable BARs for inbound access */
> diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
> index 734da58..b522956 100644
> --- a/drivers/pci/host/pci-keystone.c
> +++ b/drivers/pci/host/pci-keystone.c
> @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
>  			return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &keystone_pcie_host_ops;
>  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
>  	if (ret) {
> diff --git a/drivers/pci/host/pci-layerscape.c b/drivers/pci/host/pci-layerscape.c
> index b2328ea1..dd92ffa 100644
> --- a/drivers/pci/host/pci-layerscape.c
> +++ b/drivers/pci/host/pci-layerscape.c
> @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
>  	pp = &pcie->pp;
>  	pp->dev = pcie->dev;
>  	pp->dbi_base = pcie->dbi;
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &ls_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
> index 5307b35..bd2606b 100644
> --- a/drivers/pci/host/pcie-designware.c
> +++ b/drivers/pci/host/pcie-designware.c
> @@ -11,6 +11,7 @@
>   * published by the Free Software Foundation.
>   */
>  
> +#include <linux/hardirq.h>
>  #include <linux/irq.h>
>  #include <linux/irqdomain.h>
>  #include <linux/kernel.h>
> @@ -69,16 +70,9 @@
>  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
>  #define PCIE_ATU_UPPER_TARGET		0x91C
>  
> -static struct hw_pci dw_pci;
> +static struct pci_ops dw_pcie_ops;
>  
> -static unsigned long global_io_offset;
> -
> -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> -{
> -	BUG_ON(!sys->private_data);
> -
> -	return sys->private_data;
> -}
> +DEFINE_MUTEX(root_bus_nr_mux);
>  
>  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
>  {
> @@ -255,7 +249,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
>  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
>  {
>  	int irq, pos0, i;
> -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> +	struct pcie_port *pp = desc->dev->bus->sysdata;
>  
>  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
>  				       order_base_2(no_irqs));
> @@ -298,7 +292,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
>  {
>  	int irq, pos;
>  	struct msi_msg msg;
> -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> +	struct pcie_port *pp = pdev->bus->sysdata;
>  
>  	if (desc->msi_attrib.is_msix)
>  		return -EINVAL;
> @@ -327,7 +321,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
>  {
>  	struct irq_data *data = irq_get_irq_data(irq);
>  	struct msi_desc *msi = irq_data_get_msi(data);
> -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> +	struct pcie_port *pp = msi->dev->bus->sysdata;
>  
>  	clear_irq_range(pp, irq, 1, data->hwirq);
>  }
> @@ -359,22 +353,17 @@ static const struct irq_domain_ops msi_domain_ops = {
>  	.map = dw_pcie_msi_map,
>  };
>  
> -int dw_pcie_host_init(struct pcie_port *pp)
> +int __init dw_pcie_host_init(struct pcie_port *pp)
>  {
>  	struct device_node *np = pp->dev->of_node;
>  	struct platform_device *pdev = to_platform_device(pp->dev);
> -	struct of_pci_range range;
> -	struct of_pci_range_parser parser;
> +	struct pci_bus *bus;
>  	struct resource *cfg_res;
> -	u32 val, na, ns;
> -	const __be32 *addrp;
> -	int i, index, ret;
> -
> -	/* Find the address cell size and the number of cells in order to get
> -	 * the untranslated address.
> -	 */
> -	of_property_read_u32(np, "#address-cells", &na);
> -	ns = of_n_size_cells(np);
> +	LIST_HEAD(res);
> +	u32 val;
> +	int i, ret;
> +	struct resource_entry *win;
> +	static int root_bus_nr = 0;
>  
>  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
>  	if (cfg_res) {
> @@ -382,85 +371,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  		pp->cfg1_size = resource_size(cfg_res)/2;
>  		pp->cfg0_base = cfg_res->start;
>  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> -
> -		/* Find the untranslated configuration space address */
> -		index = of_property_match_string(np, "reg-names", "config");
> -		addrp = of_get_address(np, index, NULL, NULL);
> -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
>  	} else {
>  		dev_err(pp->dev, "missing *config* reg space\n");
>  	}
>  
> -	if (of_pci_range_parser_init(&parser, np)) {
> -		dev_err(pp->dev, "missing ranges property\n");
> -		return -EINVAL;
> -	}
> +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
> +	if (ret)
> +		return ret;
>  
>  	/* Get the I/O and memory ranges from DT */
> -	for_each_of_pci_range(&parser, &range) {
> -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> -
> -		if (restype == IORESOURCE_IO) {
> -			of_pci_range_to_resource(&range, np, &pp->io);
> -			pp->io.name = "I/O";
> -			pp->io.start = max_t(resource_size_t,
> -					     PCIBIOS_MIN_IO,
> -					     range.pci_addr + global_io_offset);
> -			pp->io.end = min_t(resource_size_t,
> -					   IO_SPACE_LIMIT,
> -					   range.pci_addr + range.size
> -					   + global_io_offset - 1);
> -			pp->io_size = resource_size(&pp->io);
> -			pp->io_bus_addr = range.pci_addr;
> -			pp->io_base = range.cpu_addr;
> -
> -			/* Find the untranslated IO space address */
> -			pp->io_mod_base = range.cpu_addr;
> -		}
> -		if (restype == IORESOURCE_MEM) {
> -			of_pci_range_to_resource(&range, np, &pp->mem);
> -			pp->mem.name = "MEM";
> -			pp->mem_size = resource_size(&pp->mem);
> -			pp->mem_bus_addr = range.pci_addr;
> -
> -			/* Find the untranslated MEM space address */
> -			pp->mem_mod_base = range.cpu_addr;
> -		}
> -		if (restype == 0) {
> -			of_pci_range_to_resource(&range, np, &pp->cfg);
> -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> -			pp->cfg0_base = pp->cfg.start;
> -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> -
> -			/* Find the untranslated configuration space address */
> -			pp->cfg0_mod_base = range.cpu_addr;
> -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> -					    pp->cfg0_size;
> +	resource_list_for_each_entry(win, &res) {
> +		switch (resource_type(win->res)) {
> +		case IORESOURCE_IO:
> +			pp->io = win->res;
> +			pp->io->name = "I/O";
> +			pp->io_size = resource_size(pp->io);
> +			pp->io_bus_addr = pp->io->start - win->offset;
> +			ret = pci_remap_iospace(pp->io, pp->io_base);
> +			if (ret) {
> +				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
> +					 ret, pp->io);
> +				continue;
> +			}
> +			break;
> +		case IORESOURCE_MEM:
> +			pp->mem = win->res;
> +			pp->mem->name = "MEM";
> +			pp->mem_size = resource_size(pp->mem);
> +			pp->mem_bus_addr = pp->mem->start - win->offset;
> +			break;
> +		case 0:
> +			pp->cfg = win->res;
> +			pp->cfg0_size = resource_size(pp->cfg)/2;
> +			pp->cfg1_size = resource_size(pp->cfg)/2;
> +			pp->cfg0_base = pp->cfg->start;
> +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> +			break;
> +		case IORESOURCE_BUS:
> +			pp->busn = win->res;
> +			break;
> +		default:
> +			continue;
>  		}
>  	}
>  
> -	ret = of_pci_parse_bus_range(np, &pp->busn);
> -	if (ret < 0) {
> -		pp->busn.name = np->name;
> -		pp->busn.start = 0;
> -		pp->busn.end = 0xff;
> -		pp->busn.flags = IORESOURCE_BUS;
> -		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
> -			ret, &pp->busn);
> -	}
> -
>  	if (!pp->dbi_base) {
> -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> -					resource_size(&pp->cfg));
> +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> +					resource_size(pp->cfg));
>  		if (!pp->dbi_base) {
>  			dev_err(pp->dev, "error with ioremap\n");
>  			return -ENOMEM;
>  		}
>  	}
>  
> -	pp->mem_base = pp->mem.start;
> +	pp->mem_base = pp->mem->start;
>  
>  	if (!pp->va_cfg0_base) {
>  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> @@ -509,7 +473,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  
>  	if (!pp->ops->rd_other_conf)
>  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
>  					  pp->mem_bus_addr, pp->mem_size);
>  
>  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> @@ -521,15 +485,40 @@ int dw_pcie_host_init(struct pcie_port *pp)
>  	val |= PORT_LOGIC_SPEED_CHANGE;
>  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
>  
> -#ifdef CONFIG_PCI_MSI
>  	dw_pcie_msi_chip.dev = pp->dev;
> -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> +
> +	mutex_lock(&root_bus_nr_mux);
> +
> +	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
> +		root_bus_nr = pp->root_bus_nr;
> +
> +	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
> +			      pp, &res);
> +	if (!bus) {
> +		mutex_unlock(&root_bus_nr_mux);
> +		return -ENOMEM;
> +	}
> +
> +	root_bus_nr += bus->busn_res.end + 1;
> +	mutex_unlock(&root_bus_nr_mux);
> +
> +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> +	bus->msi = container_of(&pp->irq_domain, struct msi_controller, domain);
> +#else
> +	bus->msi = &dw_pcie_msi_chip;
>  #endif
>  
> -	dw_pci.nr_controllers = 1;
> -	dw_pci.private_data = (void **)&pp;
> +	pci_scan_child_bus(bus);
> +	if (pp->ops->scan_bus)
> +		pp->ops->scan_bus(pp);
> +
> +#ifdef CONFIG_ARM
> +	/* support old dtbs that incorrectly describe IRQs */
> +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> +#endif
>  
> -	pci_common_init_dev(pp->dev, &dw_pci);
> +	pci_assign_unassigned_bus_resources(bus);
> +	pci_bus_add_devices(bus);
>  
>  	return 0;
>  }
> @@ -548,12 +537,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -563,7 +552,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -583,12 +572,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  
>  	if (bus->parent->number == pp->root_bus_nr) {
>  		type = PCIE_ATU_TYPE_CFG0;
> -		cpu_addr = pp->cfg0_mod_base;
> +		cpu_addr = pp->cfg0_base;
>  		cfg_size = pp->cfg0_size;
>  		va_cfg_base = pp->va_cfg0_base;
>  	} else {
>  		type = PCIE_ATU_TYPE_CFG1;
> -		cpu_addr = pp->cfg1_mod_base;
> +		cpu_addr = pp->cfg1_base;
>  		cfg_size = pp->cfg1_size;
>  		va_cfg_base = pp->va_cfg1_base;
>  	}
> @@ -598,7 +587,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
>  				  busdev, cfg_size);
>  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
>  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> +				  PCIE_ATU_TYPE_IO, pp->io_base,
>  				  pp->io_bus_addr, pp->io_size);
>  
>  	return ret;
> @@ -630,7 +619,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
>  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  			int size, u32 *val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> @@ -654,7 +643,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
>  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
>  			int where, int size, u32 val)
>  {
> -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> +	struct pcie_port *pp = bus->sysdata;
>  	int ret;
>  
>  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> @@ -678,62 +667,6 @@ static struct pci_ops dw_pcie_ops = {
>  	.write = dw_pcie_wr_conf,
>  };
>  
> -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> -{
> -	struct pcie_port *pp;
> -
> -	pp = sys_to_pcie(sys);
> -
> -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> -		pci_ioremap_io(global_io_offset, pp->io_base);
> -		global_io_offset += SZ_64K;
> -		pci_add_resource_offset(&sys->resources, &pp->io,
> -					sys->io_offset);
> -	}
> -
> -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> -	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
> -	pci_add_resource(&sys->resources, &pp->busn);
> -
> -	return 1;
> -}
> -
> -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
> -{
> -	struct pci_bus *bus;
> -	struct pcie_port *pp = sys_to_pcie(sys);
> -
> -	pp->root_bus_nr = sys->busnr;
> -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> -				  &dw_pcie_ops, sys, &sys->resources);
> -	if (!bus)
> -		return NULL;
> -
> -	if (bus && pp->ops->scan_bus)
> -		pp->ops->scan_bus(pp);
> -
> -	return bus;
> -}
> -
> -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
> -{
> -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> -	int irq;
> -
> -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> -	if (!irq)
> -		irq = pp->irq;
> -
> -	return irq;
> -}
> -
> -static struct hw_pci dw_pci = {
> -	.setup		= dw_pcie_setup,
> -	.scan		= dw_pcie_scan_bus,
> -	.map_irq	= dw_pcie_map_irq,
> -};
> -
>  void dw_pcie_setup_rc(struct pcie_port *pp)
>  {
>  	u32 val;
> diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
> index d0bbd27..0c2a7eb 100644
> --- a/drivers/pci/host/pcie-designware.h
> +++ b/drivers/pci/host/pcie-designware.h
> @@ -21,31 +21,28 @@
>   */
>  #define MAX_MSI_IRQS			32
>  #define MAX_MSI_CTRLS			(MAX_MSI_IRQS / 32)
> +#define DW_ROOT_NR_UNDEFINED	-1
>  
>  struct pcie_port {
>  	struct device		*dev;
>  	u8			root_bus_nr;
>  	void __iomem		*dbi_base;
>  	u64			cfg0_base;
> -	u64			cfg0_mod_base;
>  	void __iomem		*va_cfg0_base;
>  	u32			cfg0_size;
>  	u64			cfg1_base;
> -	u64			cfg1_mod_base;
>  	void __iomem		*va_cfg1_base;
>  	u32			cfg1_size;
> -	u64			io_base;
> -	u64			io_mod_base;
> +	resource_size_t		io_base;
>  	phys_addr_t		io_bus_addr;
>  	u32			io_size;
>  	u64			mem_base;
> -	u64			mem_mod_base;
>  	phys_addr_t		mem_bus_addr;
>  	u32			mem_size;
> -	struct resource		cfg;
> -	struct resource		io;
> -	struct resource		mem;
> -	struct resource		busn;
> +	struct resource		*cfg;
> +	struct resource		*io;
> +	struct resource		*mem;
> +	struct resource		*busn;
>  	int			irq;
>  	u32			lanes;
>  	struct pcie_host_ops	*ops;
> diff --git a/drivers/pci/host/pcie-spear13xx.c b/drivers/pci/host/pcie-spear13xx.c
> index c49fbdc..b2c59b9 100644
> --- a/drivers/pci/host/pcie-spear13xx.c
> +++ b/drivers/pci/host/pcie-spear13xx.c
> @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct pcie_port *pp,
>  		return ret;
>  	}
>  
> -	pp->root_bus_nr = -1;
> +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
>  	pp->ops = &spear13xx_pcie_host_ops;
>  
>  	ret = dw_pcie_host_init(pp);
> -- 
> 1.9.1
> 
> 
> > -----Original Message-----
> > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > Sent: Wednesday, August 19, 2015 1:54 PM
> > To: Wangzhou (B)
> > Cc: Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand; Arnd Bergmann;
> > linux at arm.linux.org.uk; thomas.petazzoni at free-electrons.com; Gabriele
> > Paoloni; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > 
> > Am Montag, den 17.08.2015, 19:55 +0800 schrieb Zhou Wang:
> > > This patch tries to unify ARM32 and ARM64 PCIe in designware driver.
> > Delete
> > > function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct
> > hw_pci,
> > > move related operations to dw_pcie_host_init.
> > >
> > > In past, we use:
> > > pci_common_init_dev
> > > 	-> pcibios_init_hw
> > > 		-> hw->scan (dw_pcie_scan_bus)
> > > to pass 0 to root_bus_nr in struct pcie_port. This patch set pp-
> > >root_bus_nr = 0
> > > in each PCIe host driver which is based on pcie-designware.
> > >
> > This is incorrect at least if there are 2 instances of DW PCIe host in
> > the same SoC without using PCI domains. In that the case the bus range
> > determines the range of valid bus numbers per instance and the first
> > number is the root bus. Please look at the "bus-range" DT property in
> > the DW PCIe bindings.
> > 
> > Also we should finally remove this root-bus setup from the glue drivers
> > altogether. It's something that entirely belongs into the DW core code.
> > 
> > Regards,
> > Lucas
> > 
> > > This patch also try to use of_pci_get_host_bridge_resources for ARM32
> > and ARM64
> > > according to the suggestion for Gabriele[1]
> > >
> > > Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
> > Program ATU
> > > with untranslated address" based on 1/6 in this series. we delete
> > *_mod_base in
> > > pcie-designware. This was discussed in [2]
> > >
> > > I have compiled the driver with multi_v7_defconfig. However, I don't
> > have
> > > ARM32 PCIe related board to do test. It will be appreciated if
> > someone could
> > > help to test it.
> > >
> > > Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> > > Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> > > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> > > Tested-By: James Morse <james.morse@arm.com>
> > >
> > > [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> > > [2] http://www.spinics.net/lists/arm-kernel/msg436779.html
> > > ---
> > >  drivers/pci/host/pci-dra7xx.c      |  15 +--
> > >  drivers/pci/host/pci-exynos.c      |   2 +-
> > >  drivers/pci/host/pci-imx6.c        |   2 +-
> > >  drivers/pci/host/pci-keystone-dw.c |   2 +-
> > >  drivers/pci/host/pci-keystone.c    |   2 +-
> > >  drivers/pci/host/pci-layerscape.c  |   2 +-
> > >  drivers/pci/host/pcie-designware.c | 229 ++++++++++++---------------
> > ----------
> > >  drivers/pci/host/pcie-designware.h |  14 +--
> > >  drivers/pci/host/pcie-spear13xx.c  |   2 +-
> > >  9 files changed, 95 insertions(+), 175 deletions(-)
> > >
> > > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> > dra7xx.c
> > > index 18ae7ff..1268c69 100644
> > > --- a/drivers/pci/host/pci-dra7xx.c
> > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct
> > pcie_port *pp)
> > >  {
> > >  	dw_pcie_setup_rc(pp);
> > >
> > > -	if (pp->io_mod_base)
> > > -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > +	if (pp->io_base)
> > > +		pp->io_base &= CPU_TO_BUS_ADDR;
> > >
> > > -	if (pp->mem_mod_base)
> > > -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > +	if (pp->mem_base)
> > > +		pp->mem_base &= CPU_TO_BUS_ADDR;
> > >
> > > -	if (pp->cfg0_mod_base) {
> > > -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > +	if (pp->cfg0_base) {
> > > +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> > > +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
> > >  	}
> > >
> > >  	dra7xx_pcie_establish_link(pp);
> > > @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct
> > dra7xx_pcie *dra7xx,
> > >
> > >  	pp = &dra7xx->pp;
> > >  	pp->dev = dev;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &dra7xx_pcie_host_ops;
> > >
> > >  	pp->irq = platform_get_irq(pdev, 1);
> > > diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-
> > exynos.c
> > > index f9f468d..9771bb0 100644
> > > --- a/drivers/pci/host/pci-exynos.c
> > > +++ b/drivers/pci/host/pci-exynos.c
> > > @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct
> > pcie_port *pp,
> > >  		}
> > >  	}
> > >
> > > -	pp->root_bus_nr = -1;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &exynos_pcie_host_ops;
> > >
> > >  	ret = dw_pcie_host_init(pp);
> > > diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-
> > imx6.c
> > > index 233a196..bec256c 100644
> > > --- a/drivers/pci/host/pci-imx6.c
> > > +++ b/drivers/pci/host/pci-imx6.c
> > > @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct
> > pcie_port *pp,
> > >  		}
> > >  	}
> > >
> > > -	pp->root_bus_nr = -1;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &imx6_pcie_host_ops;
> > >
> > >  	ret = dw_pcie_host_init(pp);
> > > diff --git a/drivers/pci/host/pci-keystone-dw.c
> > b/drivers/pci/host/pci-keystone-dw.c
> > > index f34892e..b1e4135 100644
> > > --- a/drivers/pci/host/pci-keystone-dw.c
> > > +++ b/drivers/pci/host/pci-keystone-dw.c
> > > @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void
> > __iomem *reg_virt)
> > >  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
> > >  {
> > >  	struct pcie_port *pp = &ks_pcie->pp;
> > > -	u32 start = pp->mem.start, end = pp->mem.end;
> > > +	u32 start = pp->mem->start, end = pp->mem->end;
> > >  	int i, tr_size;
> > >
> > >  	/* Disable BARs for inbound access */
> > > diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-
> > keystone.c
> > > index 734da58..8113832 100644
> > > --- a/drivers/pci/host/pci-keystone.c
> > > +++ b/drivers/pci/host/pci-keystone.c
> > > @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct
> > keystone_pcie *ks_pcie,
> > >  			return ret;
> > >  	}
> > >
> > > -	pp->root_bus_nr = -1;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &keystone_pcie_host_ops;
> > >  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
> > >  	if (ret) {
> > > diff --git a/drivers/pci/host/pci-layerscape.c
> > b/drivers/pci/host/pci-layerscape.c
> > > index b2328ea1..79ff08c 100644
> > > --- a/drivers/pci/host/pci-layerscape.c
> > > +++ b/drivers/pci/host/pci-layerscape.c
> > > @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
> > >  	pp = &pcie->pp;
> > >  	pp->dev = pcie->dev;
> > >  	pp->dbi_base = pcie->dbi;
> > > -	pp->root_bus_nr = -1;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &ls_pcie_host_ops;
> > >
> > >  	ret = dw_pcie_host_init(pp);
> > > diff --git a/drivers/pci/host/pcie-designware.c
> > b/drivers/pci/host/pcie-designware.c
> > > index c5d407c..e71a88e 100644
> > > --- a/drivers/pci/host/pcie-designware.c
> > > +++ b/drivers/pci/host/pcie-designware.c
> > > @@ -11,6 +11,7 @@
> > >   * published by the Free Software Foundation.
> > >   */
> > >
> > > +#include <linux/hardirq.h>
> > >  #include <linux/irq.h>
> > >  #include <linux/irqdomain.h>
> > >  #include <linux/kernel.h>
> > > @@ -69,16 +70,7 @@
> > >  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
> > >  #define PCIE_ATU_UPPER_TARGET		0x91C
> > >
> > > -static struct hw_pci dw_pci;
> > > -
> > > -static unsigned long global_io_offset;
> > > -
> > > -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> > > -{
> > > -	BUG_ON(!sys->private_data);
> > > -
> > > -	return sys->private_data;
> > > -}
> > > +static struct pci_ops dw_pcie_ops;
> > >
> > >  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32
> > *val)
> > >  {
> > > @@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port
> > *pp, int irq)
> > >  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
> > >  {
> > >  	int irq, pos0, i;
> > > -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> > > +	struct pcie_port *pp = desc->dev->bus->sysdata;
> > >
> > >  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
> > >  				       order_base_2(no_irqs));
> > > @@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller
> > *chip, struct pci_dev *pdev,
> > >  {
> > >  	int irq, pos;
> > >  	struct msi_msg msg;
> > > -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> > > +	struct pcie_port *pp = pdev->bus->sysdata;
> > >
> > >  	if (desc->msi_attrib.is_msix)
> > >  		return -EINVAL;
> > > @@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct
> > msi_controller *chip, unsigned int irq)
> > >  {
> > >  	struct irq_data *data = irq_get_irq_data(irq);
> > >  	struct msi_desc *msi = irq_data_get_msi(data);
> > > -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> > > +	struct pcie_port *pp = msi->dev->bus->sysdata;
> > >
> > >  	clear_irq_range(pp, irq, 1, data->hwirq);
> > >  }
> > > @@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >  {
> > >  	struct device_node *np = pp->dev->of_node;
> > >  	struct platform_device *pdev = to_platform_device(pp->dev);
> > > -	struct of_pci_range range;
> > > -	struct of_pci_range_parser parser;
> > > +	struct pci_bus *bus;
> > >  	struct resource *cfg_res;
> > > -	u32 val, ns;
> > > -	const __be32 *addrp;
> > > -	int i, index, ret;
> > > -
> > > -	ns = of_n_size_cells(np);
> > > +	LIST_HEAD(res);
> > > +	u32 val;
> > > +	int i, ret;
> > > +	struct resource_entry *win;
> > >
> > >  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> > "config");
> > >  	if (cfg_res) {
> > > @@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >  		pp->cfg1_size = resource_size(cfg_res)/2;
> > >  		pp->cfg0_base = cfg_res->start;
> > >  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> > > -
> > > -		/* Find the untranslated configuration space address */
> > > -		index = of_property_match_string(np, "reg-names", "config");
> > > -		addrp = of_get_address(np, index, NULL, NULL);
> > > -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> > > -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
> > >  	} else {
> > >  		dev_err(pp->dev, "missing *config* reg space\n");
> > >  	}
> > >
> > > -	if (of_pci_range_parser_init(&parser, np)) {
> > > -		dev_err(pp->dev, "missing ranges property\n");
> > > -		return -EINVAL;
> > > -	}
> > > +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp-
> > >io_base);
> > > +	if (ret)
> > > +		return ret;
> > >
> > >  	/* Get the I/O and memory ranges from DT */
> > > -	for_each_of_pci_range(&parser, &range) {
> > > -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> > > -
> > > -		if (restype == IORESOURCE_IO) {
> > > -			of_pci_range_to_resource(&range, np, &pp->io);
> > > -			pp->io.name = "I/O";
> > > -			pp->io.start = max_t(resource_size_t,
> > > -					     PCIBIOS_MIN_IO,
> > > -					     range.pci_addr + global_io_offset);
> > > -			pp->io.end = min_t(resource_size_t,
> > > -					   IO_SPACE_LIMIT,
> > > -					   range.pci_addr + range.size
> > > -					   + global_io_offset - 1);
> > > -			pp->io_size = resource_size(&pp->io);
> > > -			pp->io_bus_addr = range.pci_addr;
> > > -			pp->io_base = range.cpu_addr;
> > > -
> > > -			/* Find the untranslated IO space address */
> > > -			pp->io_mod_base = range.cpu_addr;
> > > -		}
> > > -		if (restype == IORESOURCE_MEM) {
> > > -			of_pci_range_to_resource(&range, np, &pp->mem);
> > > -			pp->mem.name = "MEM";
> > > -			pp->mem_size = resource_size(&pp->mem);
> > > -			pp->mem_bus_addr = range.pci_addr;
> > > -
> > > -			/* Find the untranslated MEM space address */
> > > -			pp->mem_mod_base = range.cpu_addr;
> > > -		}
> > > -		if (restype == 0) {
> > > -			of_pci_range_to_resource(&range, np, &pp->cfg);
> > > -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> > > -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> > > -			pp->cfg0_base = pp->cfg.start;
> > > -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > > -
> > > -			/* Find the untranslated configuration space address
> > */
> > > -			pp->cfg0_mod_base = range.cpu_addr;
> > > -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> > > -					    pp->cfg0_size;
> > > +	resource_list_for_each_entry(win, &res) {
> > > +		switch (resource_type(win->res)) {
> > > +		case IORESOURCE_IO:
> > > +			pp->io = win->res;
> > > +			pp->io->name = "I/O";
> > > +			pp->io_size = resource_size(pp->io);
> > > +			pp->io_bus_addr = pp->io->start - win->offset;
> > > +			ret = pci_remap_iospace(pp->io, pp->io_base);
> > > +			if (ret) {
> > > +				dev_warn(pp->dev, "error %d: failed to map
> > resource %pR\n",
> > > +					 ret, pp->io);
> > > +				continue;
> > > +			}
> > > +			break;
> > > +		case IORESOURCE_MEM:
> > > +			pp->mem = win->res;
> > > +			pp->mem->name = "MEM";
> > > +			pp->mem_size = resource_size(pp->mem);
> > > +			pp->mem_bus_addr = pp->mem->start - win->offset;
> > > +			break;
> > > +		case 0:
> > > +			pp->cfg = win->res;
> > > +			pp->cfg0_size = resource_size(pp->cfg)/2;
> > > +			pp->cfg1_size = resource_size(pp->cfg)/2;
> > > +			pp->cfg0_base = pp->cfg->start;
> > > +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> > > +			break;
> > > +		case IORESOURCE_BUS:
> > > +			pp->busn = win->res;
> > > +			break;
> > > +		default:
> > > +			continue;
> > >  		}
> > >  	}
> > >
> > > -	ret = of_pci_parse_bus_range(np, &pp->busn);
> > > -	if (ret < 0) {
> > > -		pp->busn.name = np->name;
> > > -		pp->busn.start = 0;
> > > -		pp->busn.end = 0xff;
> > > -		pp->busn.flags = IORESOURCE_BUS;
> > > -		dev_dbg(pp->dev, "failed to parse bus-range property: %d,
> > using default %pR\n",
> > > -			ret, &pp->busn);
> > > -	}
> > > -
> > >  	if (!pp->dbi_base) {
> > > -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> > > -					resource_size(&pp->cfg));
> > > +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> > > +					resource_size(pp->cfg));
> > >  		if (!pp->dbi_base) {
> > >  			dev_err(pp->dev, "error with ioremap\n");
> > >  			return -ENOMEM;
> > >  		}
> > >  	}
> > >
> > > -	pp->mem_base = pp->mem.start;
> > > +	pp->mem_base = pp->mem->start;
> > >
> > >  	if (!pp->va_cfg0_base) {
> > >  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> > > @@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >
> > >  	if (!pp->ops->rd_other_conf)
> > >  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> > > -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> > > +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
> > >  					  pp->mem_bus_addr, pp->mem_size);
> > >
> > >  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> > > @@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > >  	val |= PORT_LOGIC_SPEED_CHANGE;
> > >  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
> > >
> > > -#ifdef CONFIG_PCI_MSI
> > > +	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
> > > +				  pp, &res);
> > > +	if (!bus)
> > > +		return -ENOMEM;
> > > +
> > > +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> > > +	bus->msi = container_of(&pp->irq_domain, struct msi_controller,
> > domain);
> > > +#else
> > >  	dw_pcie_msi_chip.dev = pp->dev;
> > > -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> > > +	bus->msi = &dw_pcie_msi_chip;
> > >  #endif
> > >
> > > -	dw_pci.nr_controllers = 1;
> > > -	dw_pci.private_data = (void **)&pp;
> > > +	pci_scan_child_bus(bus);
> > > +	if (pp->ops->scan_bus)
> > > +		pp->ops->scan_bus(pp);
> > > +
> > > +#ifdef CONFIG_ARM
> > > +	/* support old dtbs that incorrectly describe IRQs */
> > > +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> > > +#endif
> > >
> > > -	pci_common_init_dev(pp->dev, &dw_pci);
> > > +	pci_assign_unassigned_bus_resources(bus);
> > > +	pci_bus_add_devices(bus);
> > >
> > >  	return 0;
> > >  }
> > > @@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct
> > pcie_port *pp, struct pci_bus *bus,
> > >
> > >  	if (bus->parent->number == pp->root_bus_nr) {
> > >  		type = PCIE_ATU_TYPE_CFG0;
> > > -		cpu_addr = pp->cfg0_mod_base;
> > > +		cpu_addr = pp->cfg0_base;
> > >  		cfg_size = pp->cfg0_size;
> > >  		va_cfg_base = pp->va_cfg0_base;
> > >  	} else {
> > >  		type = PCIE_ATU_TYPE_CFG1;
> > > -		cpu_addr = pp->cfg1_mod_base;
> > > +		cpu_addr = pp->cfg1_base;
> > >  		cfg_size = pp->cfg1_size;
> > >  		va_cfg_base = pp->va_cfg1_base;
> > >  	}
> > > @@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port
> > *pp, struct pci_bus *bus,
> > >  				  busdev, cfg_size);
> > >  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
> > >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> > >  				  pp->io_bus_addr, pp->io_size);
> > >
> > >  	return ret;
> > > @@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct
> > pcie_port *pp, struct pci_bus *bus,
> > >
> > >  	if (bus->parent->number == pp->root_bus_nr) {
> > >  		type = PCIE_ATU_TYPE_CFG0;
> > > -		cpu_addr = pp->cfg0_mod_base;
> > > +		cpu_addr = pp->cfg0_base;
> > >  		cfg_size = pp->cfg0_size;
> > >  		va_cfg_base = pp->va_cfg0_base;
> > >  	} else {
> > >  		type = PCIE_ATU_TYPE_CFG1;
> > > -		cpu_addr = pp->cfg1_mod_base;
> > > +		cpu_addr = pp->cfg1_base;
> > >  		cfg_size = pp->cfg1_size;
> > >  		va_cfg_base = pp->va_cfg1_base;
> > >  	}
> > > @@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port
> > *pp, struct pci_bus *bus,
> > >  				  busdev, cfg_size);
> > >  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
> > >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> > >  				  pp->io_bus_addr, pp->io_size);
> > >
> > >  	return ret;
> > > @@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct pcie_port
> > *pp,
> > >  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
> > >  			int size, u32 *val)
> > >  {
> > > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > > +	struct pcie_port *pp = bus->sysdata;
> > >  	int ret;
> > >
> > >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> > > @@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus,
> > u32 devfn, int where,
> > >  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
> > >  			int where, int size, u32 val)
> > >  {
> > > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > > +	struct pcie_port *pp = bus->sysdata;
> > >  	int ret;
> > >
> > >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> > > @@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
> > >  	.write = dw_pcie_wr_conf,
> > >  };
> > >
> > > -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> > > -{
> > > -	struct pcie_port *pp;
> > > -
> > > -	pp = sys_to_pcie(sys);
> > > -
> > > -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> > > -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> > > -		pci_ioremap_io(global_io_offset, pp->io_base);
> > > -		global_io_offset += SZ_64K;
> > > -		pci_add_resource_offset(&sys->resources, &pp->io,
> > > -					sys->io_offset);
> > > -	}
> > > -
> > > -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> > > -	pci_add_resource_offset(&sys->resources, &pp->mem, sys-
> > >mem_offset);
> > > -	pci_add_resource(&sys->resources, &pp->busn);
> > > -
> > > -	return 1;
> > > -}
> > > -
> > > -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data
> > *sys)
> > > -{
> > > -	struct pci_bus *bus;
> > > -	struct pcie_port *pp = sys_to_pcie(sys);
> > > -
> > > -	pp->root_bus_nr = sys->busnr;
> > > -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> > > -				  &dw_pcie_ops, sys, &sys->resources);
> > > -	if (!bus)
> > > -		return NULL;
> > > -
> > > -	if (bus && pp->ops->scan_bus)
> > > -		pp->ops->scan_bus(pp);
> > > -
> > > -	return bus;
> > > -}
> > > -
> > > -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8
> > pin)
> > > -{
> > > -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> > > -	int irq;
> > > -
> > > -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> > > -	if (!irq)
> > > -		irq = pp->irq;
> > > -
> > > -	return irq;
> > > -}
> > > -
> > > -static struct hw_pci dw_pci = {
> > > -	.setup		= dw_pcie_setup,
> > > -	.scan		= dw_pcie_scan_bus,
> > > -	.map_irq	= dw_pcie_map_irq,
> > > -};
> > > -
> > >  void dw_pcie_setup_rc(struct pcie_port *pp)
> > >  {
> > >  	u32 val;
> > > diff --git a/drivers/pci/host/pcie-designware.h
> > b/drivers/pci/host/pcie-designware.h
> > > index d0bbd27..264c969 100644
> > > --- a/drivers/pci/host/pcie-designware.h
> > > +++ b/drivers/pci/host/pcie-designware.h
> > > @@ -27,25 +27,21 @@ struct pcie_port {
> > >  	u8			root_bus_nr;
> > >  	void __iomem		*dbi_base;
> > >  	u64			cfg0_base;
> > > -	u64			cfg0_mod_base;
> > >  	void __iomem		*va_cfg0_base;
> > >  	u32			cfg0_size;
> > >  	u64			cfg1_base;
> > > -	u64			cfg1_mod_base;
> > >  	void __iomem		*va_cfg1_base;
> > >  	u32			cfg1_size;
> > > -	u64			io_base;
> > > -	u64			io_mod_base;
> > > +	resource_size_t		io_base;
> > >  	phys_addr_t		io_bus_addr;
> > >  	u32			io_size;
> > >  	u64			mem_base;
> > > -	u64			mem_mod_base;
> > >  	phys_addr_t		mem_bus_addr;
> > >  	u32			mem_size;
> > > -	struct resource		cfg;
> > > -	struct resource		io;
> > > -	struct resource		mem;
> > > -	struct resource		busn;
> > > +	struct resource		*cfg;
> > > +	struct resource		*io;
> > > +	struct resource		*mem;
> > > +	struct resource		*busn;
> > >  	int			irq;
> > >  	u32			lanes;
> > >  	struct pcie_host_ops	*ops;
> > > diff --git a/drivers/pci/host/pcie-spear13xx.c
> > b/drivers/pci/host/pcie-spear13xx.c
> > > index c49fbdc..03eb204 100644
> > > --- a/drivers/pci/host/pcie-spear13xx.c
> > > +++ b/drivers/pci/host/pcie-spear13xx.c
> > > @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct
> > pcie_port *pp,
> > >  		return ret;
> > >  	}
> > >
> > > -	pp->root_bus_nr = -1;
> > > +	pp->root_bus_nr = 0;
> > >  	pp->ops = &spear13xx_pcie_host_ops;
> > >
> > >  	ret = dw_pcie_host_init(pp);
> > 
> > --
> > Pengutronix e.K.             | Lucas Stach                 |
> > Industrial Linux Solutions   | http://www.pengutronix.de/  |
> 

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-19 15:37       ` Lucas Stach
@ 2015-08-19 16:34         ` Gabriele Paoloni
  2015-08-20  8:27           ` Lucas Stach
  0 siblings, 1 reply; 18+ messages in thread
From: Gabriele Paoloni @ 2015-08-19 16:34 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Lucas

First of all many thanks for the quick reply, really appreciated

> -----Original Message-----
> From: Lucas Stach [mailto:l.stach at pengutronix.de]
> Sent: Wednesday, August 19, 2015 4:37 PM
> To: Gabriele Paoloni
> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand;
> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> 
> Hi Gab,
> 
> Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele Paoloni:
> > Hi Lucas
> >
> > I have rewritten the patch to take into account multiple controllers.
> >
> > As you can see now there is a static var in dw_pcie_host_init() that
> tracks
> > the bus numbers used.
> 
> This is wrong. The DT specifies the valid bus number range. You can not
> just assign the next free bus number to the root bus.

I think this is what is being done in
http://lxr.free-electrons.com/source/arch/arm/kernel/bios32.c#L495
and currently designware assigns the root bus number in
http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L730


In general I agree with you but if you look at all the current drivers 
based on designware none of them define the "bus-range" dtb property.
Therefore doing as you say would break the current driver when we have
multiple controllers...am I right?

If that is the case in order to fix this in the way you say I would need
to assign "bus-range" for all the PCIe drivers with multiple controllers:
in this case I would split the default range evenly (that is, if we have 
two controllers I would define "bus-range"  0-127 and 128-255)

If you think this solution is ok I can go for it. My only doubt was about
touching other vendors DTBs....


> 
> It is perfectly valid to have a bus range of 0x00-0x10 assigned to one
> instance and 0x50-0xff to the next instance. Additional with PCIe
> hotplug you may not use the full range of the bus numbers on one
> instance at the first scan, but only later populate more buses when
> more
> bridges are added to the tree.
> 
> > Drivers that do not specify the bus range in the DTB set pp-
> >root_bus_nr = DW_ROOT_NR_UNDEFINED.
> > Designware will check if the flag is set and will use the automatic
> bus range
> > assignment.
> 
> No, please lets get rid of this assignment altogether. The glue drivers
> have no business in assigning the bus range. Please remove the
> pp->root_bus_nr assignment from all the glue drivers.
> 
> "bus range" is a generic DW PCIe property, so just parse the root bus
> number from the DT, it is handled the same way for all the DW based
> PCIe
> drivers. The bindings specifies that if the bus range property is
> missing the range is 0x00-0xff, so you can default to 0 as the root bus
> number in that case.
> 
> Also I would think this conversion warrants a patch on its own and
> should not be mixed in the ARM64 support patch.
> 
> Regards,
> Lucas
> 
> > Instead if the driver assigns pp->root_bus_nr according to the dtb,
> designwware
> > will use the value passed in by the driver.
> > Below the relevant section:
> >
> >
> > +	static int root_bus_nr = 0;
> > ...
> > +	mutex_lock(&root_bus_nr_mux);
> > +
> > +	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
> > +		root_bus_nr = pp->root_bus_nr;
> > +
> > +	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
> > +			      pp, &res);
> > +	if (!bus) {
> > +		mutex_unlock(&root_bus_nr_mux);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	root_bus_nr += bus->busn_res.end + 1;
> > +	mutex_unlock(&root_bus_nr_mux);
> >
> > Please let me know what you think...
> >
> > Many Thanks
> >
> > Gab
> > ----------
> >
> > From: gabriele paoloni <gabriele.paoloni@huawei.com>
> >
> > This patch tries to unify ARM32 and ARM64 PCIe in designware driver.
> Delete
> > function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct
> hw_pci,
> > move related operations to dw_pcie_host_init.
> > Also set pp->root_bus_nr = DW_ROOT_NR_UNDEFINED in all the drivers
> that
> > are based on designware to flag that the drivers do not read the bus
> > ranges from DT.
> > This patch also adds handling of multiple PCI domains in designware.
> > if the PCI host bridge driver does not specify a root bus number, in
> case
> > of multiple domains, designware will automatillay set the next domain
> root
> > bus number to the last bus number used in the last domain + 1.
> >
> > This patch also try to use of_pci_get_host_bridge_resources for ARM32
> and
> > ARM64 according to the suggestion for Gabriele[1]
> >
> > This patch is based on Gabriele's patch about of_pci_range fix[2]
> >
> > Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
> > Program ATU with untranslated address". This was discussed in [3]
> >
> > I have compiled the driver with multi_v7_defconfig. However, I don't
> have
> > ARM32 PCIe related board to do test. It will be appreciated if
> someone could
> > help to test it.
> >
> > Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> > Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> >
> > [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> > [2] https://patchwork.ozlabs.org/patch/495018/
> > [3] http://www.spinics.net/lists/arm-kernel/msg436779.html
> > ---
> >  drivers/pci/host/pci-dra7xx.c      |  15 +--
> >  drivers/pci/host/pci-exynos.c      |   2 +-
> >  drivers/pci/host/pci-imx6.c        |   2 +-
> >  drivers/pci/host/pci-keystone-dw.c |   2 +-
> >  drivers/pci/host/pci-keystone.c    |   2 +-
> >  drivers/pci/host/pci-layerscape.c  |   2 +-
> >  drivers/pci/host/pcie-designware.c | 247 ++++++++++++++-------------
> ----------
> >  drivers/pci/host/pcie-designware.h |  15 +--
> >  drivers/pci/host/pcie-spear13xx.c  |   2 +-
> >  9 files changed, 110 insertions(+), 179 deletions(-)
> >
> > diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-
> dra7xx.c
> > index 5678b57..8d598fb 100644
> > --- a/drivers/pci/host/pci-dra7xx.c
> > +++ b/drivers/pci/host/pci-dra7xx.c
> > @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct
> pcie_port *pp)
> >  {
> >  	dw_pcie_setup_rc(pp);
> >
> > -	if (pp->io_mod_base)
> > -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->io_base)
> > +		pp->io_base &= CPU_TO_BUS_ADDR;
> >
> > -	if (pp->mem_mod_base)
> > -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->mem_base)
> > +		pp->mem_base &= CPU_TO_BUS_ADDR;
> >
> > -	if (pp->cfg0_mod_base) {
> > -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > +	if (pp->cfg0_base) {
> > +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> > +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
> >  	}
> >
> >  	dra7xx_pcie_establish_link(pp);
> > @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct
> dra7xx_pcie *dra7xx,
> >
> >  	pp = &dra7xx->pp;
> >  	pp->dev = dev;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &dra7xx_pcie_host_ops;
> >
> >  	pp->irq = platform_get_irq(pdev, 1);
> > diff --git a/drivers/pci/host/pci-exynos.c b/drivers/pci/host/pci-
> exynos.c
> > index f9f468d..ed03a8f 100644
> > --- a/drivers/pci/host/pci-exynos.c
> > +++ b/drivers/pci/host/pci-exynos.c
> > @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct
> pcie_port *pp,
> >  		}
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &exynos_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-
> imx6.c
> > index 233a196..0efac85 100644
> > --- a/drivers/pci/host/pci-imx6.c
> > +++ b/drivers/pci/host/pci-imx6.c
> > @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct
> pcie_port *pp,
> >  		}
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &imx6_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pci-keystone-dw.c
> b/drivers/pci/host/pci-keystone-dw.c
> > index f34892e..b1e4135 100644
> > --- a/drivers/pci/host/pci-keystone-dw.c
> > +++ b/drivers/pci/host/pci-keystone-dw.c
> > @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void
> __iomem *reg_virt)
> >  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
> >  {
> >  	struct pcie_port *pp = &ks_pcie->pp;
> > -	u32 start = pp->mem.start, end = pp->mem.end;
> > +	u32 start = pp->mem->start, end = pp->mem->end;
> >  	int i, tr_size;
> >
> >  	/* Disable BARs for inbound access */
> > diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-
> keystone.c
> > index 734da58..b522956 100644
> > --- a/drivers/pci/host/pci-keystone.c
> > +++ b/drivers/pci/host/pci-keystone.c
> > @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct
> keystone_pcie *ks_pcie,
> >  			return ret;
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &keystone_pcie_host_ops;
> >  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
> >  	if (ret) {
> > diff --git a/drivers/pci/host/pci-layerscape.c
> b/drivers/pci/host/pci-layerscape.c
> > index b2328ea1..dd92ffa 100644
> > --- a/drivers/pci/host/pci-layerscape.c
> > +++ b/drivers/pci/host/pci-layerscape.c
> > @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie *pcie)
> >  	pp = &pcie->pp;
> >  	pp->dev = pcie->dev;
> >  	pp->dbi_base = pcie->dbi;
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &ls_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > diff --git a/drivers/pci/host/pcie-designware.c
> b/drivers/pci/host/pcie-designware.c
> > index 5307b35..bd2606b 100644
> > --- a/drivers/pci/host/pcie-designware.c
> > +++ b/drivers/pci/host/pcie-designware.c
> > @@ -11,6 +11,7 @@
> >   * published by the Free Software Foundation.
> >   */
> >
> > +#include <linux/hardirq.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/kernel.h>
> > @@ -69,16 +70,9 @@
> >  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
> >  #define PCIE_ATU_UPPER_TARGET		0x91C
> >
> > -static struct hw_pci dw_pci;
> > +static struct pci_ops dw_pcie_ops;
> >
> > -static unsigned long global_io_offset;
> > -
> > -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
> > -{
> > -	BUG_ON(!sys->private_data);
> > -
> > -	return sys->private_data;
> > -}
> > +DEFINE_MUTEX(root_bus_nr_mux);
> >
> >  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32
> *val)
> >  {
> > @@ -255,7 +249,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port
> *pp, int irq)
> >  static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
> >  {
> >  	int irq, pos0, i;
> > -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> > +	struct pcie_port *pp = desc->dev->bus->sysdata;
> >
> >  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
> >  				       order_base_2(no_irqs));
> > @@ -298,7 +292,7 @@ static int dw_msi_setup_irq(struct msi_controller
> *chip, struct pci_dev *pdev,
> >  {
> >  	int irq, pos;
> >  	struct msi_msg msg;
> > -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> > +	struct pcie_port *pp = pdev->bus->sysdata;
> >
> >  	if (desc->msi_attrib.is_msix)
> >  		return -EINVAL;
> > @@ -327,7 +321,7 @@ static void dw_msi_teardown_irq(struct
> msi_controller *chip, unsigned int irq)
> >  {
> >  	struct irq_data *data = irq_get_irq_data(irq);
> >  	struct msi_desc *msi = irq_data_get_msi(data);
> > -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> > +	struct pcie_port *pp = msi->dev->bus->sysdata;
> >
> >  	clear_irq_range(pp, irq, 1, data->hwirq);
> >  }
> > @@ -359,22 +353,17 @@ static const struct irq_domain_ops
> msi_domain_ops = {
> >  	.map = dw_pcie_msi_map,
> >  };
> >
> > -int dw_pcie_host_init(struct pcie_port *pp)
> > +int __init dw_pcie_host_init(struct pcie_port *pp)
> >  {
> >  	struct device_node *np = pp->dev->of_node;
> >  	struct platform_device *pdev = to_platform_device(pp->dev);
> > -	struct of_pci_range range;
> > -	struct of_pci_range_parser parser;
> > +	struct pci_bus *bus;
> >  	struct resource *cfg_res;
> > -	u32 val, na, ns;
> > -	const __be32 *addrp;
> > -	int i, index, ret;
> > -
> > -	/* Find the address cell size and the number of cells in order to
> get
> > -	 * the untranslated address.
> > -	 */
> > -	of_property_read_u32(np, "#address-cells", &na);
> > -	ns = of_n_size_cells(np);
> > +	LIST_HEAD(res);
> > +	u32 val;
> > +	int i, ret;
> > +	struct resource_entry *win;
> > +	static int root_bus_nr = 0;
> >
> >  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> "config");
> >  	if (cfg_res) {
> > @@ -382,85 +371,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >  		pp->cfg1_size = resource_size(cfg_res)/2;
> >  		pp->cfg0_base = cfg_res->start;
> >  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> > -
> > -		/* Find the untranslated configuration space address */
> > -		index = of_property_match_string(np, "reg-names", "config");
> > -		addrp = of_get_address(np, index, NULL, NULL);
> > -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> > -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
> >  	} else {
> >  		dev_err(pp->dev, "missing *config* reg space\n");
> >  	}
> >
> > -	if (of_pci_range_parser_init(&parser, np)) {
> > -		dev_err(pp->dev, "missing ranges property\n");
> > -		return -EINVAL;
> > -	}
> > +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp-
> >io_base);
> > +	if (ret)
> > +		return ret;
> >
> >  	/* Get the I/O and memory ranges from DT */
> > -	for_each_of_pci_range(&parser, &range) {
> > -		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
> > -
> > -		if (restype == IORESOURCE_IO) {
> > -			of_pci_range_to_resource(&range, np, &pp->io);
> > -			pp->io.name = "I/O";
> > -			pp->io.start = max_t(resource_size_t,
> > -					     PCIBIOS_MIN_IO,
> > -					     range.pci_addr + global_io_offset);
> > -			pp->io.end = min_t(resource_size_t,
> > -					   IO_SPACE_LIMIT,
> > -					   range.pci_addr + range.size
> > -					   + global_io_offset - 1);
> > -			pp->io_size = resource_size(&pp->io);
> > -			pp->io_bus_addr = range.pci_addr;
> > -			pp->io_base = range.cpu_addr;
> > -
> > -			/* Find the untranslated IO space address */
> > -			pp->io_mod_base = range.cpu_addr;
> > -		}
> > -		if (restype == IORESOURCE_MEM) {
> > -			of_pci_range_to_resource(&range, np, &pp->mem);
> > -			pp->mem.name = "MEM";
> > -			pp->mem_size = resource_size(&pp->mem);
> > -			pp->mem_bus_addr = range.pci_addr;
> > -
> > -			/* Find the untranslated MEM space address */
> > -			pp->mem_mod_base = range.cpu_addr;
> > -		}
> > -		if (restype == 0) {
> > -			of_pci_range_to_resource(&range, np, &pp->cfg);
> > -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> > -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> > -			pp->cfg0_base = pp->cfg.start;
> > -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > -
> > -			/* Find the untranslated configuration space address
> */
> > -			pp->cfg0_mod_base = range.cpu_addr;
> > -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> > -					    pp->cfg0_size;
> > +	resource_list_for_each_entry(win, &res) {
> > +		switch (resource_type(win->res)) {
> > +		case IORESOURCE_IO:
> > +			pp->io = win->res;
> > +			pp->io->name = "I/O";
> > +			pp->io_size = resource_size(pp->io);
> > +			pp->io_bus_addr = pp->io->start - win->offset;
> > +			ret = pci_remap_iospace(pp->io, pp->io_base);
> > +			if (ret) {
> > +				dev_warn(pp->dev, "error %d: failed to map
> resource %pR\n",
> > +					 ret, pp->io);
> > +				continue;
> > +			}
> > +			break;
> > +		case IORESOURCE_MEM:
> > +			pp->mem = win->res;
> > +			pp->mem->name = "MEM";
> > +			pp->mem_size = resource_size(pp->mem);
> > +			pp->mem_bus_addr = pp->mem->start - win->offset;
> > +			break;
> > +		case 0:
> > +			pp->cfg = win->res;
> > +			pp->cfg0_size = resource_size(pp->cfg)/2;
> > +			pp->cfg1_size = resource_size(pp->cfg)/2;
> > +			pp->cfg0_base = pp->cfg->start;
> > +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> > +			break;
> > +		case IORESOURCE_BUS:
> > +			pp->busn = win->res;
> > +			break;
> > +		default:
> > +			continue;
> >  		}
> >  	}
> >
> > -	ret = of_pci_parse_bus_range(np, &pp->busn);
> > -	if (ret < 0) {
> > -		pp->busn.name = np->name;
> > -		pp->busn.start = 0;
> > -		pp->busn.end = 0xff;
> > -		pp->busn.flags = IORESOURCE_BUS;
> > -		dev_dbg(pp->dev, "failed to parse bus-range property: %d,
> using default %pR\n",
> > -			ret, &pp->busn);
> > -	}
> > -
> >  	if (!pp->dbi_base) {
> > -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> > -					resource_size(&pp->cfg));
> > +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> > +					resource_size(pp->cfg));
> >  		if (!pp->dbi_base) {
> >  			dev_err(pp->dev, "error with ioremap\n");
> >  			return -ENOMEM;
> >  		}
> >  	}
> >
> > -	pp->mem_base = pp->mem.start;
> > +	pp->mem_base = pp->mem->start;
> >
> >  	if (!pp->va_cfg0_base) {
> >  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
> > @@ -509,7 +473,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >
> >  	if (!pp->ops->rd_other_conf)
> >  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> > -					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
> > +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
> >  					  pp->mem_bus_addr, pp->mem_size);
> >
> >  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> > @@ -521,15 +485,40 @@ int dw_pcie_host_init(struct pcie_port *pp)
> >  	val |= PORT_LOGIC_SPEED_CHANGE;
> >  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
> >
> > -#ifdef CONFIG_PCI_MSI
> >  	dw_pcie_msi_chip.dev = pp->dev;
> > -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> > +
> > +	mutex_lock(&root_bus_nr_mux);
> > +
> > +	if (pp->root_bus_nr != DW_ROOT_NR_UNDEFINED)
> > +		root_bus_nr = pp->root_bus_nr;
> > +
> > +	bus = pci_create_root_bus(pp->dev, root_bus_nr, &dw_pcie_ops,
> > +			      pp, &res);
> > +	if (!bus) {
> > +		mutex_unlock(&root_bus_nr_mux);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	root_bus_nr += bus->busn_res.end + 1;
> > +	mutex_unlock(&root_bus_nr_mux);
> > +
> > +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> > +	bus->msi = container_of(&pp->irq_domain, struct msi_controller,
> domain);
> > +#else
> > +	bus->msi = &dw_pcie_msi_chip;
> >  #endif
> >
> > -	dw_pci.nr_controllers = 1;
> > -	dw_pci.private_data = (void **)&pp;
> > +	pci_scan_child_bus(bus);
> > +	if (pp->ops->scan_bus)
> > +		pp->ops->scan_bus(pp);
> > +
> > +#ifdef CONFIG_ARM
> > +	/* support old dtbs that incorrectly describe IRQs */
> > +	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
> > +#endif
> >
> > -	pci_common_init_dev(pp->dev, &dw_pci);
> > +	pci_assign_unassigned_bus_resources(bus);
> > +	pci_bus_add_devices(bus);
> >
> >  	return 0;
> >  }
> > @@ -548,12 +537,12 @@ static int dw_pcie_rd_other_conf(struct
> pcie_port *pp, struct pci_bus *bus,
> >
> >  	if (bus->parent->number == pp->root_bus_nr) {
> >  		type = PCIE_ATU_TYPE_CFG0;
> > -		cpu_addr = pp->cfg0_mod_base;
> > +		cpu_addr = pp->cfg0_base;
> >  		cfg_size = pp->cfg0_size;
> >  		va_cfg_base = pp->va_cfg0_base;
> >  	} else {
> >  		type = PCIE_ATU_TYPE_CFG1;
> > -		cpu_addr = pp->cfg1_mod_base;
> > +		cpu_addr = pp->cfg1_base;
> >  		cfg_size = pp->cfg1_size;
> >  		va_cfg_base = pp->va_cfg1_base;
> >  	}
> > @@ -563,7 +552,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port
> *pp, struct pci_bus *bus,
> >  				  busdev, cfg_size);
> >  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
> >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> >  				  pp->io_bus_addr, pp->io_size);
> >
> >  	return ret;
> > @@ -583,12 +572,12 @@ static int dw_pcie_wr_other_conf(struct
> pcie_port *pp, struct pci_bus *bus,
> >
> >  	if (bus->parent->number == pp->root_bus_nr) {
> >  		type = PCIE_ATU_TYPE_CFG0;
> > -		cpu_addr = pp->cfg0_mod_base;
> > +		cpu_addr = pp->cfg0_base;
> >  		cfg_size = pp->cfg0_size;
> >  		va_cfg_base = pp->va_cfg0_base;
> >  	} else {
> >  		type = PCIE_ATU_TYPE_CFG1;
> > -		cpu_addr = pp->cfg1_mod_base;
> > +		cpu_addr = pp->cfg1_base;
> >  		cfg_size = pp->cfg1_size;
> >  		va_cfg_base = pp->va_cfg1_base;
> >  	}
> > @@ -598,7 +587,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port
> *pp, struct pci_bus *bus,
> >  				  busdev, cfg_size);
> >  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
> >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> >  				  pp->io_bus_addr, pp->io_size);
> >
> >  	return ret;
> > @@ -630,7 +619,7 @@ static int dw_pcie_valid_config(struct pcie_port
> *pp,
> >  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
> >  			int size, u32 *val)
> >  {
> > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > +	struct pcie_port *pp = bus->sysdata;
> >  	int ret;
> >
> >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> > @@ -654,7 +643,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus,
> u32 devfn, int where,
> >  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
> >  			int where, int size, u32 val)
> >  {
> > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > +	struct pcie_port *pp = bus->sysdata;
> >  	int ret;
> >
> >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> > @@ -678,62 +667,6 @@ static struct pci_ops dw_pcie_ops = {
> >  	.write = dw_pcie_wr_conf,
> >  };
> >
> > -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> > -{
> > -	struct pcie_port *pp;
> > -
> > -	pp = sys_to_pcie(sys);
> > -
> > -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> > -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> > -		pci_ioremap_io(global_io_offset, pp->io_base);
> > -		global_io_offset += SZ_64K;
> > -		pci_add_resource_offset(&sys->resources, &pp->io,
> > -					sys->io_offset);
> > -	}
> > -
> > -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> > -	pci_add_resource_offset(&sys->resources, &pp->mem, sys-
> >mem_offset);
> > -	pci_add_resource(&sys->resources, &pp->busn);
> > -
> > -	return 1;
> > -}
> > -
> > -static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data
> *sys)
> > -{
> > -	struct pci_bus *bus;
> > -	struct pcie_port *pp = sys_to_pcie(sys);
> > -
> > -	pp->root_bus_nr = sys->busnr;
> > -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> > -				  &dw_pcie_ops, sys, &sys->resources);
> > -	if (!bus)
> > -		return NULL;
> > -
> > -	if (bus && pp->ops->scan_bus)
> > -		pp->ops->scan_bus(pp);
> > -
> > -	return bus;
> > -}
> > -
> > -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8
> pin)
> > -{
> > -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> > -	int irq;
> > -
> > -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> > -	if (!irq)
> > -		irq = pp->irq;
> > -
> > -	return irq;
> > -}
> > -
> > -static struct hw_pci dw_pci = {
> > -	.setup		= dw_pcie_setup,
> > -	.scan		= dw_pcie_scan_bus,
> > -	.map_irq	= dw_pcie_map_irq,
> > -};
> > -
> >  void dw_pcie_setup_rc(struct pcie_port *pp)
> >  {
> >  	u32 val;
> > diff --git a/drivers/pci/host/pcie-designware.h
> b/drivers/pci/host/pcie-designware.h
> > index d0bbd27..0c2a7eb 100644
> > --- a/drivers/pci/host/pcie-designware.h
> > +++ b/drivers/pci/host/pcie-designware.h
> > @@ -21,31 +21,28 @@
> >   */
> >  #define MAX_MSI_IRQS			32
> >  #define MAX_MSI_CTRLS			(MAX_MSI_IRQS / 32)
> > +#define DW_ROOT_NR_UNDEFINED	-1
> >
> >  struct pcie_port {
> >  	struct device		*dev;
> >  	u8			root_bus_nr;
> >  	void __iomem		*dbi_base;
> >  	u64			cfg0_base;
> > -	u64			cfg0_mod_base;
> >  	void __iomem		*va_cfg0_base;
> >  	u32			cfg0_size;
> >  	u64			cfg1_base;
> > -	u64			cfg1_mod_base;
> >  	void __iomem		*va_cfg1_base;
> >  	u32			cfg1_size;
> > -	u64			io_base;
> > -	u64			io_mod_base;
> > +	resource_size_t		io_base;
> >  	phys_addr_t		io_bus_addr;
> >  	u32			io_size;
> >  	u64			mem_base;
> > -	u64			mem_mod_base;
> >  	phys_addr_t		mem_bus_addr;
> >  	u32			mem_size;
> > -	struct resource		cfg;
> > -	struct resource		io;
> > -	struct resource		mem;
> > -	struct resource		busn;
> > +	struct resource		*cfg;
> > +	struct resource		*io;
> > +	struct resource		*mem;
> > +	struct resource		*busn;
> >  	int			irq;
> >  	u32			lanes;
> >  	struct pcie_host_ops	*ops;
> > diff --git a/drivers/pci/host/pcie-spear13xx.c
> b/drivers/pci/host/pcie-spear13xx.c
> > index c49fbdc..b2c59b9 100644
> > --- a/drivers/pci/host/pcie-spear13xx.c
> > +++ b/drivers/pci/host/pcie-spear13xx.c
> > @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct
> pcie_port *pp,
> >  		return ret;
> >  	}
> >
> > -	pp->root_bus_nr = -1;
> > +	pp->root_bus_nr = DW_ROOT_NR_UNDEFINED;
> >  	pp->ops = &spear13xx_pcie_host_ops;
> >
> >  	ret = dw_pcie_host_init(pp);
> > --
> > 1.9.1
> >
> >
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > > Sent: Wednesday, August 19, 2015 1:54 PM
> > > To: Wangzhou (B)
> > > Cc: Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand; Arnd
> Bergmann;
> > > linux at arm.linux.org.uk; thomas.petazzoni at free-electrons.com;
> Gabriele
> > > Paoloni; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > >
> > > Am Montag, den 17.08.2015, 19:55 +0800 schrieb Zhou Wang:
> > > > This patch tries to unify ARM32 and ARM64 PCIe in designware
> driver.
> > > Delete
> > > > function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and
> struct
> > > hw_pci,
> > > > move related operations to dw_pcie_host_init.
> > > >
> > > > In past, we use:
> > > > pci_common_init_dev
> > > > 	-> pcibios_init_hw
> > > > 		-> hw->scan (dw_pcie_scan_bus)
> > > > to pass 0 to root_bus_nr in struct pcie_port. This patch set pp-
> > > >root_bus_nr = 0
> > > > in each PCIe host driver which is based on pcie-designware.
> > > >
> > > This is incorrect at least if there are 2 instances of DW PCIe host
> in
> > > the same SoC without using PCI domains. In that the case the bus
> range
> > > determines the range of valid bus numbers per instance and the
> first
> > > number is the root bus. Please look at the "bus-range" DT property
> in
> > > the DW PCIe bindings.
> > >
> > > Also we should finally remove this root-bus setup from the glue
> drivers
> > > altogether. It's something that entirely belongs into the DW core
> code.
> > >
> > > Regards,
> > > Lucas
> > >
> > > > This patch also try to use of_pci_get_host_bridge_resources for
> ARM32
> > > and ARM64
> > > > according to the suggestion for Gabriele[1]
> > > >
> > > > Finally this patch reverts commit f4c55c5a3f7f "PCI: designware:
> > > Program ATU
> > > > with untranslated address" based on 1/6 in this series. we delete
> > > *_mod_base in
> > > > pcie-designware. This was discussed in [2]
> > > >
> > > > I have compiled the driver with multi_v7_defconfig. However, I
> don't
> > > have
> > > > ARM32 PCIe related board to do test. It will be appreciated if
> > > someone could
> > > > help to test it.
> > > >
> > > > Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> > > > Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> > > > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> > > > Tested-By: James Morse <james.morse@arm.com>
> > > >
> > > > [1] http://www.spinics.net/lists/linux-pci/msg42194.html
> > > > [2] http://www.spinics.net/lists/arm-kernel/msg436779.html
> > > > ---
> > > >  drivers/pci/host/pci-dra7xx.c      |  15 +--
> > > >  drivers/pci/host/pci-exynos.c      |   2 +-
> > > >  drivers/pci/host/pci-imx6.c        |   2 +-
> > > >  drivers/pci/host/pci-keystone-dw.c |   2 +-
> > > >  drivers/pci/host/pci-keystone.c    |   2 +-
> > > >  drivers/pci/host/pci-layerscape.c  |   2 +-
> > > >  drivers/pci/host/pcie-designware.c | 229 ++++++++++++-----------
> ----
> > > ----------
> > > >  drivers/pci/host/pcie-designware.h |  14 +--
> > > >  drivers/pci/host/pcie-spear13xx.c  |   2 +-
> > > >  9 files changed, 95 insertions(+), 175 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/host/pci-dra7xx.c
> b/drivers/pci/host/pci-
> > > dra7xx.c
> > > > index 18ae7ff..1268c69 100644
> > > > --- a/drivers/pci/host/pci-dra7xx.c
> > > > +++ b/drivers/pci/host/pci-dra7xx.c
> > > > @@ -141,15 +141,15 @@ static void dra7xx_pcie_host_init(struct
> > > pcie_port *pp)
> > > >  {
> > > >  	dw_pcie_setup_rc(pp);
> > > >
> > > > -	if (pp->io_mod_base)
> > > > -		pp->io_mod_base &= CPU_TO_BUS_ADDR;
> > > > +	if (pp->io_base)
> > > > +		pp->io_base &= CPU_TO_BUS_ADDR;
> > > >
> > > > -	if (pp->mem_mod_base)
> > > > -		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
> > > > +	if (pp->mem_base)
> > > > +		pp->mem_base &= CPU_TO_BUS_ADDR;
> > > >
> > > > -	if (pp->cfg0_mod_base) {
> > > > -		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
> > > > -		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
> > > > +	if (pp->cfg0_base) {
> > > > +		pp->cfg0_base &= CPU_TO_BUS_ADDR;
> > > > +		pp->cfg1_base &= CPU_TO_BUS_ADDR;
> > > >  	}
> > > >
> > > >  	dra7xx_pcie_establish_link(pp);
> > > > @@ -288,6 +288,7 @@ static int __init dra7xx_add_pcie_port(struct
> > > dra7xx_pcie *dra7xx,
> > > >
> > > >  	pp = &dra7xx->pp;
> > > >  	pp->dev = dev;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &dra7xx_pcie_host_ops;
> > > >
> > > >  	pp->irq = platform_get_irq(pdev, 1);
> > > > diff --git a/drivers/pci/host/pci-exynos.c
> b/drivers/pci/host/pci-
> > > exynos.c
> > > > index f9f468d..9771bb0 100644
> > > > --- a/drivers/pci/host/pci-exynos.c
> > > > +++ b/drivers/pci/host/pci-exynos.c
> > > > @@ -530,7 +530,7 @@ static int __init exynos_add_pcie_port(struct
> > > pcie_port *pp,
> > > >  		}
> > > >  	}
> > > >
> > > > -	pp->root_bus_nr = -1;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &exynos_pcie_host_ops;
> > > >
> > > >  	ret = dw_pcie_host_init(pp);
> > > > diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-
> > > imx6.c
> > > > index 233a196..bec256c 100644
> > > > --- a/drivers/pci/host/pci-imx6.c
> > > > +++ b/drivers/pci/host/pci-imx6.c
> > > > @@ -551,7 +551,7 @@ static int __init imx6_add_pcie_port(struct
> > > pcie_port *pp,
> > > >  		}
> > > >  	}
> > > >
> > > > -	pp->root_bus_nr = -1;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &imx6_pcie_host_ops;
> > > >
> > > >  	ret = dw_pcie_host_init(pp);
> > > > diff --git a/drivers/pci/host/pci-keystone-dw.c
> > > b/drivers/pci/host/pci-keystone-dw.c
> > > > index f34892e..b1e4135 100644
> > > > --- a/drivers/pci/host/pci-keystone-dw.c
> > > > +++ b/drivers/pci/host/pci-keystone-dw.c
> > > > @@ -327,7 +327,7 @@ static void ks_dw_pcie_clear_dbi_mode(void
> > > __iomem *reg_virt)
> > > >  void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
> > > >  {
> > > >  	struct pcie_port *pp = &ks_pcie->pp;
> > > > -	u32 start = pp->mem.start, end = pp->mem.end;
> > > > +	u32 start = pp->mem->start, end = pp->mem->end;
> > > >  	int i, tr_size;
> > > >
> > > >  	/* Disable BARs for inbound access */
> > > > diff --git a/drivers/pci/host/pci-keystone.c
> b/drivers/pci/host/pci-
> > > keystone.c
> > > > index 734da58..8113832 100644
> > > > --- a/drivers/pci/host/pci-keystone.c
> > > > +++ b/drivers/pci/host/pci-keystone.c
> > > > @@ -309,7 +309,7 @@ static int __init ks_add_pcie_port(struct
> > > keystone_pcie *ks_pcie,
> > > >  			return ret;
> > > >  	}
> > > >
> > > > -	pp->root_bus_nr = -1;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &keystone_pcie_host_ops;
> > > >  	ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
> > > >  	if (ret) {
> > > > diff --git a/drivers/pci/host/pci-layerscape.c
> > > b/drivers/pci/host/pci-layerscape.c
> > > > index b2328ea1..79ff08c 100644
> > > > --- a/drivers/pci/host/pci-layerscape.c
> > > > +++ b/drivers/pci/host/pci-layerscape.c
> > > > @@ -106,7 +106,7 @@ static int ls_add_pcie_port(struct ls_pcie
> *pcie)
> > > >  	pp = &pcie->pp;
> > > >  	pp->dev = pcie->dev;
> > > >  	pp->dbi_base = pcie->dbi;
> > > > -	pp->root_bus_nr = -1;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &ls_pcie_host_ops;
> > > >
> > > >  	ret = dw_pcie_host_init(pp);
> > > > diff --git a/drivers/pci/host/pcie-designware.c
> > > b/drivers/pci/host/pcie-designware.c
> > > > index c5d407c..e71a88e 100644
> > > > --- a/drivers/pci/host/pcie-designware.c
> > > > +++ b/drivers/pci/host/pcie-designware.c
> > > > @@ -11,6 +11,7 @@
> > > >   * published by the Free Software Foundation.
> > > >   */
> > > >
> > > > +#include <linux/hardirq.h>
> > > >  #include <linux/irq.h>
> > > >  #include <linux/irqdomain.h>
> > > >  #include <linux/kernel.h>
> > > > @@ -69,16 +70,7 @@
> > > >  #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
> > > >  #define PCIE_ATU_UPPER_TARGET		0x91C
> > > >
> > > > -static struct hw_pci dw_pci;
> > > > -
> > > > -static unsigned long global_io_offset;
> > > > -
> > > > -static inline struct pcie_port *sys_to_pcie(struct pci_sys_data
> *sys)
> > > > -{
> > > > -	BUG_ON(!sys->private_data);
> > > > -
> > > > -	return sys->private_data;
> > > > -}
> > > > +static struct pci_ops dw_pcie_ops;
> > > >
> > > >  int dw_pcie_cfg_read(void __iomem *addr, int where, int size,
> u32
> > > *val)
> > > >  {
> > > > @@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct
> pcie_port
> > > *pp, int irq)
> > > >  static int assign_irq(int no_irqs, struct msi_desc *desc, int
> *pos)
> > > >  {
> > > >  	int irq, pos0, i;
> > > > -	struct pcie_port *pp = sys_to_pcie(desc->dev->bus->sysdata);
> > > > +	struct pcie_port *pp = desc->dev->bus->sysdata;
> > > >
> > > >  	pos0 = bitmap_find_free_region(pp->msi_irq_in_use,
> MAX_MSI_IRQS,
> > > >  				       order_base_2(no_irqs));
> > > > @@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct
> msi_controller
> > > *chip, struct pci_dev *pdev,
> > > >  {
> > > >  	int irq, pos;
> > > >  	struct msi_msg msg;
> > > > -	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
> > > > +	struct pcie_port *pp = pdev->bus->sysdata;
> > > >
> > > >  	if (desc->msi_attrib.is_msix)
> > > >  		return -EINVAL;
> > > > @@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct
> > > msi_controller *chip, unsigned int irq)
> > > >  {
> > > >  	struct irq_data *data = irq_get_irq_data(irq);
> > > >  	struct msi_desc *msi = irq_data_get_msi(data);
> > > > -	struct pcie_port *pp = sys_to_pcie(msi->dev->bus->sysdata);
> > > > +	struct pcie_port *pp = msi->dev->bus->sysdata;
> > > >
> > > >  	clear_irq_range(pp, irq, 1, data->hwirq);
> > > >  }
> > > > @@ -363,14 +355,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >  {
> > > >  	struct device_node *np = pp->dev->of_node;
> > > >  	struct platform_device *pdev = to_platform_device(pp->dev);
> > > > -	struct of_pci_range range;
> > > > -	struct of_pci_range_parser parser;
> > > > +	struct pci_bus *bus;
> > > >  	struct resource *cfg_res;
> > > > -	u32 val, ns;
> > > > -	const __be32 *addrp;
> > > > -	int i, index, ret;
> > > > -
> > > > -	ns = of_n_size_cells(np);
> > > > +	LIST_HEAD(res);
> > > > +	u32 val;
> > > > +	int i, ret;
> > > > +	struct resource_entry *win;
> > > >
> > > >  	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> > > "config");
> > > >  	if (cfg_res) {
> > > > @@ -378,85 +368,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >  		pp->cfg1_size = resource_size(cfg_res)/2;
> > > >  		pp->cfg0_base = cfg_res->start;
> > > >  		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
> > > > -
> > > > -		/* Find the untranslated configuration space address
> */
> > > > -		index = of_property_match_string(np, "reg-names",
> "config");
> > > > -		addrp = of_get_address(np, index, NULL, NULL);
> > > > -		pp->cfg0_mod_base = of_read_number(addrp, ns);
> > > > -		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
> > > >  	} else {
> > > >  		dev_err(pp->dev, "missing *config* reg space\n");
> > > >  	}
> > > >
> > > > -	if (of_pci_range_parser_init(&parser, np)) {
> > > > -		dev_err(pp->dev, "missing ranges property\n");
> > > > -		return -EINVAL;
> > > > -	}
> > > > +	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res,
> &pp-
> > > >io_base);
> > > > +	if (ret)
> > > > +		return ret;
> > > >
> > > >  	/* Get the I/O and memory ranges from DT */
> > > > -	for_each_of_pci_range(&parser, &range) {
> > > > -		unsigned long restype = range.flags &
> IORESOURCE_TYPE_BITS;
> > > > -
> > > > -		if (restype == IORESOURCE_IO) {
> > > > -			of_pci_range_to_resource(&range, np, &pp->io);
> > > > -			pp->io.name = "I/O";
> > > > -			pp->io.start = max_t(resource_size_t,
> > > > -					     PCIBIOS_MIN_IO,
> > > > -					     range.pci_addr +
> global_io_offset);
> > > > -			pp->io.end = min_t(resource_size_t,
> > > > -					   IO_SPACE_LIMIT,
> > > > -					   range.pci_addr + range.size
> > > > -					   + global_io_offset - 1);
> > > > -			pp->io_size = resource_size(&pp->io);
> > > > -			pp->io_bus_addr = range.pci_addr;
> > > > -			pp->io_base = range.cpu_addr;
> > > > -
> > > > -			/* Find the untranslated IO space address */
> > > > -			pp->io_mod_base = range.cpu_addr;
> > > > -		}
> > > > -		if (restype == IORESOURCE_MEM) {
> > > > -			of_pci_range_to_resource(&range, np, &pp->mem);
> > > > -			pp->mem.name = "MEM";
> > > > -			pp->mem_size = resource_size(&pp->mem);
> > > > -			pp->mem_bus_addr = range.pci_addr;
> > > > -
> > > > -			/* Find the untranslated MEM space address */
> > > > -			pp->mem_mod_base = range.cpu_addr;
> > > > -		}
> > > > -		if (restype == 0) {
> > > > -			of_pci_range_to_resource(&range, np, &pp->cfg);
> > > > -			pp->cfg0_size = resource_size(&pp->cfg)/2;
> > > > -			pp->cfg1_size = resource_size(&pp->cfg)/2;
> > > > -			pp->cfg0_base = pp->cfg.start;
> > > > -			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
> > > > -
> > > > -			/* Find the untranslated configuration space
> address
> > > */
> > > > -			pp->cfg0_mod_base = range.cpu_addr;
> > > > -			pp->cfg1_mod_base = pp->cfg0_mod_base +
> > > > -					    pp->cfg0_size;
> > > > +	resource_list_for_each_entry(win, &res) {
> > > > +		switch (resource_type(win->res)) {
> > > > +		case IORESOURCE_IO:
> > > > +			pp->io = win->res;
> > > > +			pp->io->name = "I/O";
> > > > +			pp->io_size = resource_size(pp->io);
> > > > +			pp->io_bus_addr = pp->io->start - win->offset;
> > > > +			ret = pci_remap_iospace(pp->io, pp->io_base);
> > > > +			if (ret) {
> > > > +				dev_warn(pp->dev, "error %d: failed to
> map
> > > resource %pR\n",
> > > > +					 ret, pp->io);
> > > > +				continue;
> > > > +			}
> > > > +			break;
> > > > +		case IORESOURCE_MEM:
> > > > +			pp->mem = win->res;
> > > > +			pp->mem->name = "MEM";
> > > > +			pp->mem_size = resource_size(pp->mem);
> > > > +			pp->mem_bus_addr = pp->mem->start - win->offset;
> > > > +			break;
> > > > +		case 0:
> > > > +			pp->cfg = win->res;
> > > > +			pp->cfg0_size = resource_size(pp->cfg)/2;
> > > > +			pp->cfg1_size = resource_size(pp->cfg)/2;
> > > > +			pp->cfg0_base = pp->cfg->start;
> > > > +			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
> > > > +			break;
> > > > +		case IORESOURCE_BUS:
> > > > +			pp->busn = win->res;
> > > > +			break;
> > > > +		default:
> > > > +			continue;
> > > >  		}
> > > >  	}
> > > >
> > > > -	ret = of_pci_parse_bus_range(np, &pp->busn);
> > > > -	if (ret < 0) {
> > > > -		pp->busn.name = np->name;
> > > > -		pp->busn.start = 0;
> > > > -		pp->busn.end = 0xff;
> > > > -		pp->busn.flags = IORESOURCE_BUS;
> > > > -		dev_dbg(pp->dev, "failed to parse bus-range
> property: %d,
> > > using default %pR\n",
> > > > -			ret, &pp->busn);
> > > > -	}
> > > > -
> > > >  	if (!pp->dbi_base) {
> > > > -		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
> > > > -					resource_size(&pp->cfg));
> > > > +		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
> > > > +					resource_size(pp->cfg));
> > > >  		if (!pp->dbi_base) {
> > > >  			dev_err(pp->dev, "error with ioremap\n");
> > > >  			return -ENOMEM;
> > > >  		}
> > > >  	}
> > > >
> > > > -	pp->mem_base = pp->mem.start;
> > > > +	pp->mem_base = pp->mem->start;
> > > >
> > > >  	if (!pp->va_cfg0_base) {
> > > >  		pp->va_cfg0_base = devm_ioremap(pp->dev, pp-
> >cfg0_base,
> > > > @@ -505,7 +470,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >
> > > >  	if (!pp->ops->rd_other_conf)
> > > >  		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
> > > > -					  PCIE_ATU_TYPE_MEM, pp-
> >mem_mod_base,
> > > > +					  PCIE_ATU_TYPE_MEM, pp->mem_base,
> > > >  					  pp->mem_bus_addr, pp->mem_size);
> > > >
> > > >  	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
> > > > @@ -517,15 +482,29 @@ int dw_pcie_host_init(struct pcie_port *pp)
> > > >  	val |= PORT_LOGIC_SPEED_CHANGE;
> > > >  	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4,
> val);
> > > >
> > > > -#ifdef CONFIG_PCI_MSI
> > > > +	bus = pci_create_root_bus(pp->dev, pp->root_bus_nr,
> &dw_pcie_ops,
> > > > +				  pp, &res);
> > > > +	if (!bus)
> > > > +		return -ENOMEM;
> > > > +
> > > > +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> > > > +	bus->msi = container_of(&pp->irq_domain, struct
> msi_controller,
> > > domain);
> > > > +#else
> > > >  	dw_pcie_msi_chip.dev = pp->dev;
> > > > -	dw_pci.msi_ctrl = &dw_pcie_msi_chip;
> > > > +	bus->msi = &dw_pcie_msi_chip;
> > > >  #endif
> > > >
> > > > -	dw_pci.nr_controllers = 1;
> > > > -	dw_pci.private_data = (void **)&pp;
> > > > +	pci_scan_child_bus(bus);
> > > > +	if (pp->ops->scan_bus)
> > > > +		pp->ops->scan_bus(pp);
> > > > +
> > > > +#ifdef CONFIG_ARM
> > > > +	/* support old dtbs that incorrectly describe IRQs */
> > > > +	pci_fixup_irqs(pci_common_swizzle,
> of_irq_parse_and_map_pci);
> > > > +#endif
> > > >
> > > > -	pci_common_init_dev(pp->dev, &dw_pci);
> > > > +	pci_assign_unassigned_bus_resources(bus);
> > > > +	pci_bus_add_devices(bus);
> > > >
> > > >  	return 0;
> > > >  }
> > > > @@ -544,12 +523,12 @@ static int dw_pcie_rd_other_conf(struct
> > > pcie_port *pp, struct pci_bus *bus,
> > > >
> > > >  	if (bus->parent->number == pp->root_bus_nr) {
> > > >  		type = PCIE_ATU_TYPE_CFG0;
> > > > -		cpu_addr = pp->cfg0_mod_base;
> > > > +		cpu_addr = pp->cfg0_base;
> > > >  		cfg_size = pp->cfg0_size;
> > > >  		va_cfg_base = pp->va_cfg0_base;
> > > >  	} else {
> > > >  		type = PCIE_ATU_TYPE_CFG1;
> > > > -		cpu_addr = pp->cfg1_mod_base;
> > > > +		cpu_addr = pp->cfg1_base;
> > > >  		cfg_size = pp->cfg1_size;
> > > >  		va_cfg_base = pp->va_cfg1_base;
> > > >  	}
> > > > @@ -559,7 +538,7 @@ static int dw_pcie_rd_other_conf(struct
> pcie_port
> > > *pp, struct pci_bus *bus,
> > > >  				  busdev, cfg_size);
> > > >  	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size,
> val);
> > > >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > > > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > > > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> > > >  				  pp->io_bus_addr, pp->io_size);
> > > >
> > > >  	return ret;
> > > > @@ -579,12 +558,12 @@ static int dw_pcie_wr_other_conf(struct
> > > pcie_port *pp, struct pci_bus *bus,
> > > >
> > > >  	if (bus->parent->number == pp->root_bus_nr) {
> > > >  		type = PCIE_ATU_TYPE_CFG0;
> > > > -		cpu_addr = pp->cfg0_mod_base;
> > > > +		cpu_addr = pp->cfg0_base;
> > > >  		cfg_size = pp->cfg0_size;
> > > >  		va_cfg_base = pp->va_cfg0_base;
> > > >  	} else {
> > > >  		type = PCIE_ATU_TYPE_CFG1;
> > > > -		cpu_addr = pp->cfg1_mod_base;
> > > > +		cpu_addr = pp->cfg1_base;
> > > >  		cfg_size = pp->cfg1_size;
> > > >  		va_cfg_base = pp->va_cfg1_base;
> > > >  	}
> > > > @@ -594,7 +573,7 @@ static int dw_pcie_wr_other_conf(struct
> pcie_port
> > > *pp, struct pci_bus *bus,
> > > >  				  busdev, cfg_size);
> > > >  	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size,
> val);
> > > >  	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
> > > > -				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
> > > > +				  PCIE_ATU_TYPE_IO, pp->io_base,
> > > >  				  pp->io_bus_addr, pp->io_size);
> > > >
> > > >  	return ret;
> > > > @@ -626,7 +605,7 @@ static int dw_pcie_valid_config(struct
> pcie_port
> > > *pp,
> > > >  static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int
> where,
> > > >  			int size, u32 *val)
> > > >  {
> > > > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > > > +	struct pcie_port *pp = bus->sysdata;
> > > >  	int ret;
> > > >
> > > >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
> > > > @@ -650,7 +629,7 @@ static int dw_pcie_rd_conf(struct pci_bus
> *bus,
> > > u32 devfn, int where,
> > > >  static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
> > > >  			int where, int size, u32 val)
> > > >  {
> > > > -	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
> > > > +	struct pcie_port *pp = bus->sysdata;
> > > >  	int ret;
> > > >
> > > >  	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
> > > > @@ -674,62 +653,6 @@ static struct pci_ops dw_pcie_ops = {
> > > >  	.write = dw_pcie_wr_conf,
> > > >  };
> > > >
> > > > -static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
> > > > -{
> > > > -	struct pcie_port *pp;
> > > > -
> > > > -	pp = sys_to_pcie(sys);
> > > > -
> > > > -	if (global_io_offset < SZ_1M && pp->io_size > 0) {
> > > > -		sys->io_offset = global_io_offset - pp->io_bus_addr;
> > > > -		pci_ioremap_io(global_io_offset, pp->io_base);
> > > > -		global_io_offset += SZ_64K;
> > > > -		pci_add_resource_offset(&sys->resources, &pp->io,
> > > > -					sys->io_offset);
> > > > -	}
> > > > -
> > > > -	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
> > > > -	pci_add_resource_offset(&sys->resources, &pp->mem, sys-
> > > >mem_offset);
> > > > -	pci_add_resource(&sys->resources, &pp->busn);
> > > > -
> > > > -	return 1;
> > > > -}
> > > > -
> > > > -static struct pci_bus *dw_pcie_scan_bus(int nr, struct
> pci_sys_data
> > > *sys)
> > > > -{
> > > > -	struct pci_bus *bus;
> > > > -	struct pcie_port *pp = sys_to_pcie(sys);
> > > > -
> > > > -	pp->root_bus_nr = sys->busnr;
> > > > -	bus = pci_scan_root_bus(pp->dev, sys->busnr,
> > > > -				  &dw_pcie_ops, sys, &sys->resources);
> > > > -	if (!bus)
> > > > -		return NULL;
> > > > -
> > > > -	if (bus && pp->ops->scan_bus)
> > > > -		pp->ops->scan_bus(pp);
> > > > -
> > > > -	return bus;
> > > > -}
> > > > -
> > > > -static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot,
> u8
> > > pin)
> > > > -{
> > > > -	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
> > > > -	int irq;
> > > > -
> > > > -	irq = of_irq_parse_and_map_pci(dev, slot, pin);
> > > > -	if (!irq)
> > > > -		irq = pp->irq;
> > > > -
> > > > -	return irq;
> > > > -}
> > > > -
> > > > -static struct hw_pci dw_pci = {
> > > > -	.setup		= dw_pcie_setup,
> > > > -	.scan		= dw_pcie_scan_bus,
> > > > -	.map_irq	= dw_pcie_map_irq,
> > > > -};
> > > > -
> > > >  void dw_pcie_setup_rc(struct pcie_port *pp)
> > > >  {
> > > >  	u32 val;
> > > > diff --git a/drivers/pci/host/pcie-designware.h
> > > b/drivers/pci/host/pcie-designware.h
> > > > index d0bbd27..264c969 100644
> > > > --- a/drivers/pci/host/pcie-designware.h
> > > > +++ b/drivers/pci/host/pcie-designware.h
> > > > @@ -27,25 +27,21 @@ struct pcie_port {
> > > >  	u8			root_bus_nr;
> > > >  	void __iomem		*dbi_base;
> > > >  	u64			cfg0_base;
> > > > -	u64			cfg0_mod_base;
> > > >  	void __iomem		*va_cfg0_base;
> > > >  	u32			cfg0_size;
> > > >  	u64			cfg1_base;
> > > > -	u64			cfg1_mod_base;
> > > >  	void __iomem		*va_cfg1_base;
> > > >  	u32			cfg1_size;
> > > > -	u64			io_base;
> > > > -	u64			io_mod_base;
> > > > +	resource_size_t		io_base;
> > > >  	phys_addr_t		io_bus_addr;
> > > >  	u32			io_size;
> > > >  	u64			mem_base;
> > > > -	u64			mem_mod_base;
> > > >  	phys_addr_t		mem_bus_addr;
> > > >  	u32			mem_size;
> > > > -	struct resource		cfg;
> > > > -	struct resource		io;
> > > > -	struct resource		mem;
> > > > -	struct resource		busn;
> > > > +	struct resource		*cfg;
> > > > +	struct resource		*io;
> > > > +	struct resource		*mem;
> > > > +	struct resource		*busn;
> > > >  	int			irq;
> > > >  	u32			lanes;
> > > >  	struct pcie_host_ops	*ops;
> > > > diff --git a/drivers/pci/host/pcie-spear13xx.c
> > > b/drivers/pci/host/pcie-spear13xx.c
> > > > index c49fbdc..03eb204 100644
> > > > --- a/drivers/pci/host/pcie-spear13xx.c
> > > > +++ b/drivers/pci/host/pcie-spear13xx.c
> > > > @@ -286,7 +286,7 @@ static int spear13xx_add_pcie_port(struct
> > > pcie_port *pp,
> > > >  		return ret;
> > > >  	}
> > > >
> > > > -	pp->root_bus_nr = -1;
> > > > +	pp->root_bus_nr = 0;
> > > >  	pp->ops = &spear13xx_pcie_host_ops;
> > > >
> > > >  	ret = dw_pcie_host_init(pp);
> > >
> > > --
> > > Pengutronix e.K.             | Lucas Stach                 |
> > > Industrial Linux Solutions   | http://www.pengutronix.de/  |
> >
> 
> --
> Pengutronix e.K.             | Lucas Stach                 |
> Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-19 16:34         ` Gabriele Paoloni
@ 2015-08-20  8:27           ` Lucas Stach
  2015-08-20 11:10             ` Gabriele Paoloni
  0 siblings, 1 reply; 18+ messages in thread
From: Lucas Stach @ 2015-08-20  8:27 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Gab,

Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
> Hi Lucas
> 
> First of all many thanks for the quick reply, really appreciated
> 
> > -----Original Message-----
> > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > Sent: Wednesday, August 19, 2015 4:37 PM
> > To: Gabriele Paoloni
> > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand;
> > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > 
> > Hi Gab,
> > 
> > Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele Paoloni:
> > > Hi Lucas
> > >
> > > I have rewritten the patch to take into account multiple controllers.
> > >
> > > As you can see now there is a static var in dw_pcie_host_init() that
> > tracks
> > > the bus numbers used.
> > 
> > This is wrong. The DT specifies the valid bus number range. You can not
> > just assign the next free bus number to the root bus.
> 
> I think this is what is being done in
> http://lxr.free-electrons.com/source/arch/arm/kernel/bios32.c#L495
> and currently designware assigns the root bus number in
> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L730
> 
You found the right spot. It works a bit different than I remember, but
is less broken than your current code. :)

It actually assigns the next instance a root bus number behind the
current instances bus range. Please note the difference to your code
which assigns the next free bus number, which may still lay within the
current instances bus range.

> 
> In general I agree with you but if you look at all the current drivers 
> based on designware none of them define the "bus-range" dtb property.
> Therefore doing as you say would break the current driver when we have
> multiple controllers...am I right?
> 
The current _code_ works fine for multiple controllers. It's just that
you must define the bus range property in the DTB if you want to enable
multiple instances of one controller and support a kernel without PCI
domains support. As all current implementations have only a single
instance of the controller per SoC, or depend on PCI domain support,
it's totally fine for them to not define the bus ranges property, as
it's an optional property and it is well defined how things behave if it
is absent. We absolutely need to keep that specified behavior.

> If that is the case in order to fix this in the way you say I would need
> to assign "bus-range" for all the PCIe drivers with multiple controllers:
> in this case I would split the default range evenly (that is, if we have 
> two controllers I would define "bus-range"  0-127 and 128-255)
> 
> If you think this solution is ok I can go for it. My only doubt was about
> touching other vendors DTBs....
> 
You don't need to touch any of the current DTBs, as they are fine with
the default bus range behavior. You need to keep the specified behavior
of the bus range property with the new code.

Regards,
Lucas
> 
> > 
> > It is perfectly valid to have a bus range of 0x00-0x10 assigned to one
> > instance and 0x50-0xff to the next instance. Additional with PCIe
> > hotplug you may not use the full range of the bus numbers on one
> > instance at the first scan, but only later populate more buses when
> > more
> > bridges are added to the tree.
> > 
> > > Drivers that do not specify the bus range in the DTB set pp-
> > >root_bus_nr = DW_ROOT_NR_UNDEFINED.
> > > Designware will check if the flag is set and will use the automatic
> > bus range
> > > assignment.
> > 
> > No, please lets get rid of this assignment altogether. The glue drivers
> > have no business in assigning the bus range. Please remove the
> > pp->root_bus_nr assignment from all the glue drivers.
> > 
> > "bus range" is a generic DW PCIe property, so just parse the root bus
> > number from the DT, it is handled the same way for all the DW based
> > PCIe
> > drivers. The bindings specifies that if the bus range property is
> > missing the range is 0x00-0xff, so you can default to 0 as the root bus
> > number in that case.
> > 
> > Also I would think this conversion warrants a patch on its own and
> > should not be mixed in the ARM64 support patch.
> > 
> > Regards,
> > Lucas
> > 

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-20  8:27           ` Lucas Stach
@ 2015-08-20 11:10             ` Gabriele Paoloni
  2015-08-21 13:42               ` Liviu Dudau
  0 siblings, 1 reply; 18+ messages in thread
From: Gabriele Paoloni @ 2015-08-20 11:10 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Lucas

Again many thanks for explaining and for your time.

I got your point now and I have dug a bit better in the PCI_DOMAINS code.

However I have a question...see inline below

> -----Original Message-----
> From: Lucas Stach [mailto:l.stach at pengutronix.de]
> Sent: Thursday, August 20, 2015 9:27 AM
> To: Gabriele Paoloni
> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand;
> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> 
> Hi Gab,
> 
> Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
> > Hi Lucas
> >
> > First of all many thanks for the quick reply, really appreciated
> >
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > > Sent: Wednesday, August 19, 2015 4:37 PM
> > > To: Gabriele Paoloni
> > > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> Anand;
> > > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > >
> > > Hi Gab,
> > >
> > > Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele Paoloni:
> > > > Hi Lucas
> > > >
> > > > I have rewritten the patch to take into account multiple
> controllers.
> > > >
> > > > As you can see now there is a static var in dw_pcie_host_init()
> that
> > > tracks
> > > > the bus numbers used.
> > >
> > > This is wrong. The DT specifies the valid bus number range. You can
> not
> > > just assign the next free bus number to the root bus.
> >
> > I think this is what is being done in
> > http://lxr.free-electrons.com/source/arch/arm/kernel/bios32.c#L495
> > and currently designware assigns the root bus number in
> > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> designware.c#L730
> >
> You found the right spot. It works a bit different than I remember, but
> is less broken than your current code. :)
> 
> It actually assigns the next instance a root bus number behind the
> current instances bus range. Please note the difference to your code
> which assigns the next free bus number, which may still lay within the
> current instances bus range.

Mmmm here I have done a mistake: in the current designware the number of hw
controllers is always 1
http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L510
So the loop in bios32.c does not work on multiple PCIe ports...
However your comment about PCI_DOMAINS enlightened my mind and now I see
that when CONFIG_PCI_DOMAINS is defined we have the domains numbers
(tracked by __domain_nr).
So (correct me if I am wrong) if we have 2 PCIe ports that do not specify
the "bus-range" property, both root ports will enumerate starting from 
root_bus_nr = 0 and everything will still work ok.

So if my assumption is correct, I do not see why the orginal patch from Wang 
Zhou is wrong. 
The only point can be that assigning pp->root_bus_nr = 0 is not required 
as pp memory is kzalloc'ed (an therefore init to zero).


> 
> >
> > In general I agree with you but if you look at all the current
> drivers
> > based on designware none of them define the "bus-range" dtb property.
> > Therefore doing as you say would break the current driver when we
> have
> > multiple controllers...am I right?
> >
> The current _code_ works fine for multiple controllers. It's just that
> you must define the bus range property in the DTB if you want to enable
> multiple instances of one controller and support a kernel without PCI
> domains support. As all current implementations have only a single
> instance of the controller per SoC, or depend on PCI domain support,
> it's totally fine for them to not define the bus ranges property, as
> it's an optional property and it is well defined how things behave if
> it
> is absent. We absolutely need to keep that specified behavior.
> 
> > If that is the case in order to fix this in the way you say I would
> need
> > to assign "bus-range" for all the PCIe drivers with multiple
> controllers:
> > in this case I would split the default range evenly (that is, if we
> have
> > two controllers I would define "bus-range"  0-127 and 128-255)
> >
> > If you think this solution is ok I can go for it. My only doubt was
> about
> > touching other vendors DTBs....
> >
> You don't need to touch any of the current DTBs, as they are fine with
> the default bus range behavior. You need to keep the specified behavior
> of the bus range property with the new code.
> 
> Regards,
> Lucas
> >
> > >
> > > It is perfectly valid to have a bus range of 0x00-0x10 assigned to
> one
> > > instance and 0x50-0xff to the next instance. Additional with PCIe
> > > hotplug you may not use the full range of the bus numbers on one
> > > instance at the first scan, but only later populate more buses when
> > > more
> > > bridges are added to the tree.
> > >
> > > > Drivers that do not specify the bus range in the DTB set pp-
> > > >root_bus_nr = DW_ROOT_NR_UNDEFINED.
> > > > Designware will check if the flag is set and will use the
> automatic
> > > bus range
> > > > assignment.
> > >
> > > No, please lets get rid of this assignment altogether. The glue
> drivers
> > > have no business in assigning the bus range. Please remove the
> > > pp->root_bus_nr assignment from all the glue drivers.
> > >
> > > "bus range" is a generic DW PCIe property, so just parse the root
> bus
> > > number from the DT, it is handled the same way for all the DW based
> > > PCIe
> > > drivers. The bindings specifies that if the bus range property is
> > > missing the range is 0x00-0xff, so you can default to 0 as the root
> bus
> > > number in that case.
> > >
> > > Also I would think this conversion warrants a patch on its own and
> > > should not be mixed in the ARM64 support patch.
> > >
> > > Regards,
> > > Lucas
> > >
> 
> --
> Pengutronix e.K.             | Lucas Stach                 |
> Industrial Linux Solutions   | http://www.pengutronix.de/  |

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-20 11:10             ` Gabriele Paoloni
@ 2015-08-21 13:42               ` Liviu Dudau
  2015-08-21 14:19                 ` Gabriele Paoloni
  0 siblings, 1 reply; 18+ messages in thread
From: Liviu Dudau @ 2015-08-21 13:42 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Aug 20, 2015 at 12:10:24PM +0100, Gabriele Paoloni wrote:
> Hi Lucas
> 
> Again many thanks for explaining and for your time.
> 
> I got your point now and I have dug a bit better in the PCI_DOMAINS code.
> 
> However I have a question...see inline below
> 
> > -----Original Message-----
> > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > Sent: Thursday, August 20, 2015 9:27 AM
> > To: Gabriele Paoloni
> > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush Anand;
> > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > 
> > Hi Gab,
> > 
> > Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
> > > Hi Lucas
> > >
> > > First of all many thanks for the quick reply, really appreciated
> > >
> > > > -----Original Message-----
> > > > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > > > Sent: Wednesday, August 19, 2015 4:37 PM
> > > > To: Gabriele Paoloni
> > > > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> > Anand;
> > > > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > > > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > > > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > > > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > > >
> > > > Hi Gab,
> > > >
> > > > Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele Paoloni:
> > > > > Hi Lucas
> > > > >
> > > > > I have rewritten the patch to take into account multiple
> > controllers.
> > > > >
> > > > > As you can see now there is a static var in dw_pcie_host_init()
> > that
> > > > tracks
> > > > > the bus numbers used.
> > > >
> > > > This is wrong. The DT specifies the valid bus number range. You can
> > not
> > > > just assign the next free bus number to the root bus.
> > >
> > > I think this is what is being done in
> > > http://lxr.free-electrons.com/source/arch/arm/kernel/bios32.c#L495
> > > and currently designware assigns the root bus number in
> > > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> > designware.c#L730
> > >
> > You found the right spot. It works a bit different than I remember, but
> > is less broken than your current code. :)
> > 
> > It actually assigns the next instance a root bus number behind the
> > current instances bus range. Please note the difference to your code
> > which assigns the next free bus number, which may still lay within the
> > current instances bus range.

Hi Gabriele,

> 
> Mmmm here I have done a mistake: in the current designware the number of hw
> controllers is always 1
> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-designware.c#L510
> So the loop in bios32.c does not work on multiple PCIe ports...

Bios32.c is broken for multiple PCIe hosts for multiple reasons, this is just
one of them. Hence the effort to get rid of it for maintained drivers.

> However your comment about PCI_DOMAINS enlightened my mind and now I see
> that when CONFIG_PCI_DOMAINS is defined we have the domains numbers
> (tracked by __domain_nr).

CONFIG_PCI_DOMAINS in itself doesn't do much without adding code to your driver.
You could request a new domain for each controller with a special property to
disable that behaviour in the dts, or you could enable CONFIG_PCI_DOMAINS_GENERIC
which will give you a new domain automatically.

> So (correct me if I am wrong) if we have 2 PCIe ports that do not specify
> the "bus-range" property, both root ports will enumerate starting from 
> root_bus_nr = 0 and everything will still work ok.

Correct.

Best regards,
Liviu

> 
> So if my assumption is correct, I do not see why the orginal patch from Wang 
> Zhou is wrong. 
> The only point can be that assigning pp->root_bus_nr = 0 is not required 
> as pp memory is kzalloc'ed (an therefore init to zero).
> 
> 
> > 
> > >
> > > In general I agree with you but if you look at all the current
> > drivers
> > > based on designware none of them define the "bus-range" dtb property.
> > > Therefore doing as you say would break the current driver when we
> > have
> > > multiple controllers...am I right?
> > >
> > The current _code_ works fine for multiple controllers. It's just that
> > you must define the bus range property in the DTB if you want to enable
> > multiple instances of one controller and support a kernel without PCI
> > domains support. As all current implementations have only a single
> > instance of the controller per SoC, or depend on PCI domain support,
> > it's totally fine for them to not define the bus ranges property, as
> > it's an optional property and it is well defined how things behave if
> > it
> > is absent. We absolutely need to keep that specified behavior.
> > 
> > > If that is the case in order to fix this in the way you say I would
> > need
> > > to assign "bus-range" for all the PCIe drivers with multiple
> > controllers:
> > > in this case I would split the default range evenly (that is, if we
> > have
> > > two controllers I would define "bus-range"  0-127 and 128-255)
> > >
> > > If you think this solution is ok I can go for it. My only doubt was
> > about
> > > touching other vendors DTBs....
> > >
> > You don't need to touch any of the current DTBs, as they are fine with
> > the default bus range behavior. You need to keep the specified behavior
> > of the bus range property with the new code.
> > 
> > Regards,
> > Lucas
> > >
> > > >
> > > > It is perfectly valid to have a bus range of 0x00-0x10 assigned to
> > one
> > > > instance and 0x50-0xff to the next instance. Additional with PCIe
> > > > hotplug you may not use the full range of the bus numbers on one
> > > > instance at the first scan, but only later populate more buses when
> > > > more
> > > > bridges are added to the tree.
> > > >
> > > > > Drivers that do not specify the bus range in the DTB set pp-
> > > > >root_bus_nr = DW_ROOT_NR_UNDEFINED.
> > > > > Designware will check if the flag is set and will use the
> > automatic
> > > > bus range
> > > > > assignment.
> > > >
> > > > No, please lets get rid of this assignment altogether. The glue
> > drivers
> > > > have no business in assigning the bus range. Please remove the
> > > > pp->root_bus_nr assignment from all the glue drivers.
> > > >
> > > > "bus range" is a generic DW PCIe property, so just parse the root
> > bus
> > > > number from the DT, it is handled the same way for all the DW based
> > > > PCIe
> > > > drivers. The bindings specifies that if the bus range property is
> > > > missing the range is 0x00-0xff, so you can default to 0 as the root
> > bus
> > > > number in that case.
> > > >
> > > > Also I would think this conversion warrants a patch on its own and
> > > > should not be mixed in the ARM64 support patch.
> > > >
> > > > Regards,
> > > > Lucas
> > > >
> > 
> > --
> > Pengutronix e.K.             | Lucas Stach                 |
> > Industrial Linux Solutions   | http://www.pengutronix.de/  |
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ?\_(?)_/?

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-21 13:42               ` Liviu Dudau
@ 2015-08-21 14:19                 ` Gabriele Paoloni
  2015-08-24  6:26                   ` Zhou Wang
  0 siblings, 1 reply; 18+ messages in thread
From: Gabriele Paoloni @ 2015-08-21 14:19 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Liviu

Many Thanks for reviewing

> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> owner at vger.kernel.org] On Behalf Of Liviu Dudau
> Sent: Friday, August 21, 2015 2:43 PM
> To: Gabriele Paoloni
> Cc: Lucas Stach; Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com;
> Pratyush Anand; Arnd Bergmann; linux at arm.linux.org.uk;
> thomas.petazzoni at free-electrons.com; Lorenzo Pieralisi; James Morse;
> jason at lakedaemon.net; robh at kernel.org; linux-pci at vger.kernel.org;
> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth)
> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> 
> On Thu, Aug 20, 2015 at 12:10:24PM +0100, Gabriele Paoloni wrote:
> > Hi Lucas
> >
> > Again many thanks for explaining and for your time.
> >
> > I got your point now and I have dug a bit better in the PCI_DOMAINS
> code.
> >
> > However I have a question...see inline below
> >
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > > Sent: Thursday, August 20, 2015 9:27 AM
> > > To: Gabriele Paoloni
> > > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> Anand;
> > > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> > > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > >
> > > Hi Gab,
> > >
> > > Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
> > > > Hi Lucas
> > > >
> > > > First of all many thanks for the quick reply, really appreciated
> > > >
> > > > > -----Original Message-----
> > > > > From: Lucas Stach [mailto:l.stach at pengutronix.de]
> > > > > Sent: Wednesday, August 19, 2015 4:37 PM
> > > > > To: Gabriele Paoloni
> > > > > Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> > > Anand;
> > > > > Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> > > > > electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> > > > > Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org;
> linux-
> > > > > pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> > > > > devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> > > > > qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu
> (Kenneth)
> > > > > Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> > > > >
> > > > > Hi Gab,
> > > > >
> > > > > Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele
> Paoloni:
> > > > > > Hi Lucas
> > > > > >
> > > > > > I have rewritten the patch to take into account multiple
> > > controllers.
> > > > > >
> > > > > > As you can see now there is a static var in
> dw_pcie_host_init()
> > > that
> > > > > tracks
> > > > > > the bus numbers used.
> > > > >
> > > > > This is wrong. The DT specifies the valid bus number range. You
> can
> > > not
> > > > > just assign the next free bus number to the root bus.
> > > >
> > > > I think this is what is being done in
> > > > http://lxr.free-
> electrons.com/source/arch/arm/kernel/bios32.c#L495
> > > > and currently designware assigns the root bus number in
> > > > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> > > designware.c#L730
> > > >
> > > You found the right spot. It works a bit different than I remember,
> but
> > > is less broken than your current code. :)
> > >
> > > It actually assigns the next instance a root bus number behind the
> > > current instances bus range. Please note the difference to your
> code
> > > which assigns the next free bus number, which may still lay within
> the
> > > current instances bus range.
> 
> Hi Gabriele,
> 
> >
> > Mmmm here I have done a mistake: in the current designware the number
> of hw
> > controllers is always 1
> > http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> designware.c#L510
> > So the loop in bios32.c does not work on multiple PCIe ports...
> 
> Bios32.c is broken for multiple PCIe hosts for multiple reasons, this
> is just
> one of them. Hence the effort to get rid of it for maintained drivers.

As you can see we're pushing hard for this :)

> 
> > However your comment about PCI_DOMAINS enlightened my mind and now I
> see
> > that when CONFIG_PCI_DOMAINS is defined we have the domains numbers
> > (tracked by __domain_nr).
> 
> CONFIG_PCI_DOMAINS in itself doesn't do much without adding code to
> your driver.
> You could request a new domain for each controller with a special
> property to
> disable that behaviour in the dts, or you could enable
> CONFIG_PCI_DOMAINS_GENERIC
> which will give you a new domain automatically.

So far I see that for ARM and ARM64 CONFIG_PCI_DOMAINS_GENERIC defaults to 
the same value as CONFIG_PCI_DOMAINS (see arch/arm/Kconfig, arch/arm64/Kconfig). 
So I think this is how current designware based drivers get multiple PCIe 
controller to work (they just enable CONFIG_PCI_DOMAINS)...

 

> 
> > So (correct me if I am wrong) if we have 2 PCIe ports that do not
> specify
> > the "bus-range" property, both root ports will enumerate starting
> from
> > root_bus_nr = 0 and everything will still work ok.
> 
> Correct.
> 
> Best regards,
> Liviu
> 
> >
> > So if my assumption is correct, I do not see why the orginal patch
> from Wang
> > Zhou is wrong.
> > The only point can be that assigning pp->root_bus_nr = 0 is not
> required
> > as pp memory is kzalloc'ed (an therefore init to zero).
> >
> >
> > >
> > > >
> > > > In general I agree with you but if you look at all the current
> > > drivers
> > > > based on designware none of them define the "bus-range" dtb
> property.
> > > > Therefore doing as you say would break the current driver when we
> > > have
> > > > multiple controllers...am I right?
> > > >
> > > The current _code_ works fine for multiple controllers. It's just
> that
> > > you must define the bus range property in the DTB if you want to
> enable
> > > multiple instances of one controller and support a kernel without
> PCI
> > > domains support. As all current implementations have only a single
> > > instance of the controller per SoC, or depend on PCI domain support,
> > > it's totally fine for them to not define the bus ranges property,
> as
> > > it's an optional property and it is well defined how things behave
> if
> > > it
> > > is absent. We absolutely need to keep that specified behavior.
> > >
> > > > If that is the case in order to fix this in the way you say I
> would
> > > need
> > > > to assign "bus-range" for all the PCIe drivers with multiple
> > > controllers:
> > > > in this case I would split the default range evenly (that is, if
> we
> > > have
> > > > two controllers I would define "bus-range"  0-127 and 128-255)
> > > >
> > > > If you think this solution is ok I can go for it. My only doubt
> was
> > > about
> > > > touching other vendors DTBs....
> > > >
> > > You don't need to touch any of the current DTBs, as they are fine
> with
> > > the default bus range behavior. You need to keep the specified
> behavior
> > > of the bus range property with the new code.
> > >
> > > Regards,
> > > Lucas
> > > >
> > > > >
> > > > > It is perfectly valid to have a bus range of 0x00-0x10 assigned
> to
> > > one
> > > > > instance and 0x50-0xff to the next instance. Additional with
> PCIe
> > > > > hotplug you may not use the full range of the bus numbers on
> one
> > > > > instance at the first scan, but only later populate more buses
> when
> > > > > more
> > > > > bridges are added to the tree.
> > > > >
> > > > > > Drivers that do not specify the bus range in the DTB set pp-
> > > > > >root_bus_nr = DW_ROOT_NR_UNDEFINED.
> > > > > > Designware will check if the flag is set and will use the
> > > automatic
> > > > > bus range
> > > > > > assignment.
> > > > >
> > > > > No, please lets get rid of this assignment altogether. The glue
> > > drivers
> > > > > have no business in assigning the bus range. Please remove the
> > > > > pp->root_bus_nr assignment from all the glue drivers.
> > > > >
> > > > > "bus range" is a generic DW PCIe property, so just parse the
> root
> > > bus
> > > > > number from the DT, it is handled the same way for all the DW
> based
> > > > > PCIe
> > > > > drivers. The bindings specifies that if the bus range property
> is
> > > > > missing the range is 0x00-0xff, so you can default to 0 as the
> root
> > > bus
> > > > > number in that case.
> > > > >
> > > > > Also I would think this conversion warrants a patch on its own
> and
> > > > > should not be mixed in the ARM64 support patch.
> > > > >
> > > > > Regards,
> > > > > Lucas
> > > > >
> > >
> > > --
> > > Pengutronix e.K.             | Lucas Stach                 |
> > > Industrial Linux Solutions   | http://www.pengutronix.de/  |
> >
> 
> --
> ====================
> | I would like to |
> | fix the world,  |
> | but they're not |
> | giving me the   |
>  \ source code!  /
>   ---------------
>     ?\_(?)_/?
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-21 14:19                 ` Gabriele Paoloni
@ 2015-08-24  6:26                   ` Zhou Wang
  2015-08-24  8:28                     ` Gabriele Paoloni
  0 siblings, 1 reply; 18+ messages in thread
From: Zhou Wang @ 2015-08-24  6:26 UTC (permalink / raw)
  To: linux-arm-kernel

On 2015/8/21 22:19, Gabriele Paoloni wrote:
> Hi Liviu
> 
> Many Thanks for reviewing
> 
>> -----Original Message-----
>> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
>> owner at vger.kernel.org] On Behalf Of Liviu Dudau
>> Sent: Friday, August 21, 2015 2:43 PM
>> To: Gabriele Paoloni
>> Cc: Lucas Stach; Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com;
>> Pratyush Anand; Arnd Bergmann; linux at arm.linux.org.uk;
>> thomas.petazzoni at free-electrons.com; Lorenzo Pieralisi; James Morse;
>> jason at lakedaemon.net; robh at kernel.org; linux-pci at vger.kernel.org;
>> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
>> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; liudongdong (C);
>> qiujiang; xuwei (O); Liguozhu (Kenneth)
>> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
>>
>> On Thu, Aug 20, 2015 at 12:10:24PM +0100, Gabriele Paoloni wrote:
>>> Hi Lucas
>>>
>>> Again many thanks for explaining and for your time.
>>>
>>> I got your point now and I have dug a bit better in the PCI_DOMAINS
>> code.
>>>
>>> However I have a question...see inline below
>>>
>>>> -----Original Message-----
>>>> From: Lucas Stach [mailto:l.stach at pengutronix.de]
>>>> Sent: Thursday, August 20, 2015 9:27 AM
>>>> To: Gabriele Paoloni
>>>> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
>> Anand;
>>>> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
>>>> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
>>>> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
>>>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
>>>> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
>>>>
>>>> Hi Gab,
>>>>
>>>> Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
>>>>> Hi Lucas
>>>>>
>>>>> First of all many thanks for the quick reply, really appreciated
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Lucas Stach [mailto:l.stach at pengutronix.de]
>>>>>> Sent: Wednesday, August 19, 2015 4:37 PM
>>>>>> To: Gabriele Paoloni
>>>>>> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
>>>> Anand;
>>>>>> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
>>>>>> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
>>>>>> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org;
>> linux-
>>>>>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
>>>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
>>>>>> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu
>> (Kenneth)
>>>>>> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
>>>>>>
>>>>>> Hi Gab,
>>>>>>
>>>>>> Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele
>> Paoloni:
>>>>>>> Hi Lucas
>>>>>>>
>>>>>>> I have rewritten the patch to take into account multiple
>>>> controllers.
>>>>>>>
>>>>>>> As you can see now there is a static var in
>> dw_pcie_host_init()
>>>> that
>>>>>> tracks
>>>>>>> the bus numbers used.
>>>>>>
>>>>>> This is wrong. The DT specifies the valid bus number range. You
>> can
>>>> not
>>>>>> just assign the next free bus number to the root bus.
>>>>>
>>>>> I think this is what is being done in
>>>>> http://lxr.free-
>> electrons.com/source/arch/arm/kernel/bios32.c#L495
>>>>> and currently designware assigns the root bus number in
>>>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
>>>> designware.c#L730
>>>>>
>>>> You found the right spot. It works a bit different than I remember,
>> but
>>>> is less broken than your current code. :)
>>>>
>>>> It actually assigns the next instance a root bus number behind the
>>>> current instances bus range. Please note the difference to your
>> code
>>>> which assigns the next free bus number, which may still lay within
>> the
>>>> current instances bus range.
>>
>> Hi Gabriele,
>>
>>>
>>> Mmmm here I have done a mistake: in the current designware the number
>> of hw
>>> controllers is always 1
>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
>> designware.c#L510
>>> So the loop in bios32.c does not work on multiple PCIe ports...
>>
>> Bios32.c is broken for multiple PCIe hosts for multiple reasons, this
>> is just
>> one of them. Hence the effort to get rid of it for maintained drivers.
> 
> As you can see we're pushing hard for this :)
>

How about we pass pp->busn->start to pci_create_root_bus as root bus number?
We get pp->busn->start from resource list(res) whose elements come from
of_pci_get_host_bridge_resources.

If there is a bus-range item in dts, root bus number will get from it.
Oppositely, root bus number will be default value(0x0).

Thanks,
Zhou

>>
>>> However your comment about PCI_DOMAINS enlightened my mind and now I
>> see
>>> that when CONFIG_PCI_DOMAINS is defined we have the domains numbers
>>> (tracked by __domain_nr).
>>
>> CONFIG_PCI_DOMAINS in itself doesn't do much without adding code to
>> your driver.
>> You could request a new domain for each controller with a special
>> property to
>> disable that behaviour in the dts, or you could enable
>> CONFIG_PCI_DOMAINS_GENERIC
>> which will give you a new domain automatically.
> 
> So far I see that for ARM and ARM64 CONFIG_PCI_DOMAINS_GENERIC defaults to 
> the same value as CONFIG_PCI_DOMAINS (see arch/arm/Kconfig, arch/arm64/Kconfig). 
> So I think this is how current designware based drivers get multiple PCIe 
> controller to work (they just enable CONFIG_PCI_DOMAINS)...
> 
>  
> 
>>
>>> So (correct me if I am wrong) if we have 2 PCIe ports that do not
>> specify
>>> the "bus-range" property, both root ports will enumerate starting
>> from
>>> root_bus_nr = 0 and everything will still work ok.
>>
>> Correct.
>>
>> Best regards,
>> Liviu
>>
>>>
>>> So if my assumption is correct, I do not see why the orginal patch
>> from Wang
>>> Zhou is wrong.
>>> The only point can be that assigning pp->root_bus_nr = 0 is not
>> required
>>> as pp memory is kzalloc'ed (an therefore init to zero).
>>>
>>>
>>>>
>>>>>
>>>>> In general I agree with you but if you look at all the current
>>>> drivers
>>>>> based on designware none of them define the "bus-range" dtb
>> property.
>>>>> Therefore doing as you say would break the current driver when we
>>>> have
>>>>> multiple controllers...am I right?
>>>>>
>>>> The current _code_ works fine for multiple controllers. It's just
>> that
>>>> you must define the bus range property in the DTB if you want to
>> enable
>>>> multiple instances of one controller and support a kernel without
>> PCI
>>>> domains support. As all current implementations have only a single
>>>> instance of the controller per SoC, or depend on PCI domain support,
>>>> it's totally fine for them to not define the bus ranges property,
>> as
>>>> it's an optional property and it is well defined how things behave
>> if
>>>> it
>>>> is absent. We absolutely need to keep that specified behavior.
>>>>
>>>>> If that is the case in order to fix this in the way you say I
>> would
>>>> need
>>>>> to assign "bus-range" for all the PCIe drivers with multiple
>>>> controllers:
>>>>> in this case I would split the default range evenly (that is, if
>> we
>>>> have
>>>>> two controllers I would define "bus-range"  0-127 and 128-255)
>>>>>
>>>>> If you think this solution is ok I can go for it. My only doubt
>> was
>>>> about
>>>>> touching other vendors DTBs....
>>>>>
>>>> You don't need to touch any of the current DTBs, as they are fine
>> with
>>>> the default bus range behavior. You need to keep the specified
>> behavior
>>>> of the bus range property with the new code.
>>>>
>>>> Regards,
>>>> Lucas
>>>>>
>>>>>>
>>>>>> It is perfectly valid to have a bus range of 0x00-0x10 assigned
>> to
>>>> one
>>>>>> instance and 0x50-0xff to the next instance. Additional with
>> PCIe
>>>>>> hotplug you may not use the full range of the bus numbers on
>> one
>>>>>> instance at the first scan, but only later populate more buses
>> when
>>>>>> more
>>>>>> bridges are added to the tree.
>>>>>>
>>>>>>> Drivers that do not specify the bus range in the DTB set pp-
>>>>>>> root_bus_nr = DW_ROOT_NR_UNDEFINED.
>>>>>>> Designware will check if the flag is set and will use the
>>>> automatic
>>>>>> bus range
>>>>>>> assignment.
>>>>>>
>>>>>> No, please lets get rid of this assignment altogether. The glue
>>>> drivers
>>>>>> have no business in assigning the bus range. Please remove the
>>>>>> pp->root_bus_nr assignment from all the glue drivers.
>>>>>>
>>>>>> "bus range" is a generic DW PCIe property, so just parse the
>> root
>>>> bus
>>>>>> number from the DT, it is handled the same way for all the DW
>> based
>>>>>> PCIe
>>>>>> drivers. The bindings specifies that if the bus range property
>> is
>>>>>> missing the range is 0x00-0xff, so you can default to 0 as the
>> root
>>>> bus
>>>>>> number in that case.
>>>>>>
>>>>>> Also I would think this conversion warrants a patch on its own
>> and
>>>>>> should not be mixed in the ARM64 support patch.
>>>>>>
>>>>>> Regards,
>>>>>> Lucas
>>>>>>
>>>>
>>>> --
>>>> Pengutronix e.K.             | Lucas Stach                 |
>>>> Industrial Linux Solutions   | http://www.pengutronix.de/  |
>>>
>>
>> --
>> ====================
>> | I would like to |
>> | fix the world,  |
>> | but they're not |
>> | giving me the   |
>>  \ source code!  /
>>   ---------------
>>     ?\_(?)_/?
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH v7 3/6] PCI: designware: Add ARM64 support
  2015-08-24  6:26                   ` Zhou Wang
@ 2015-08-24  8:28                     ` Gabriele Paoloni
  0 siblings, 0 replies; 18+ messages in thread
From: Gabriele Paoloni @ 2015-08-24  8:28 UTC (permalink / raw)
  To: linux-arm-kernel



> -----Original Message-----
> From: Wangzhou (B)
> Sent: Monday, August 24, 2015 7:26 AM
> To: Gabriele Paoloni
> Cc: Liviu Dudau; Lucas Stach; Bjorn Helgaas; jingoohan1 at gmail.com;
> Pratyush Anand; Arnd Bergmann; linux at arm.linux.org.uk;
> thomas.petazzoni at free-electrons.com; Lorenzo Pieralisi; James Morse;
> jason at lakedaemon.net; robh at kernel.org; linux-pci at vger.kernel.org;
> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth)
> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> 
> On 2015/8/21 22:19, Gabriele Paoloni wrote:
> > Hi Liviu
> >
> > Many Thanks for reviewing
> >
> >> -----Original Message-----
> >> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-
> >> owner at vger.kernel.org] On Behalf Of Liviu Dudau
> >> Sent: Friday, August 21, 2015 2:43 PM
> >> To: Gabriele Paoloni
> >> Cc: Lucas Stach; Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com;
> >> Pratyush Anand; Arnd Bergmann; linux at arm.linux.org.uk;
> >> thomas.petazzoni at free-electrons.com; Lorenzo Pieralisi; James Morse;
> >> jason at lakedaemon.net; robh at kernel.org; linux-pci at vger.kernel.org;
> >> linux-arm-kernel at lists.infradead.org; devicetree at vger.kernel.org;
> >> Yuanzhichang; Zhudacai; zhangjukuo; qiuzhenfa; liudongdong (C);
> >> qiujiang; xuwei (O); Liguozhu (Kenneth)
> >> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> >>
> >> On Thu, Aug 20, 2015 at 12:10:24PM +0100, Gabriele Paoloni wrote:
> >>> Hi Lucas
> >>>
> >>> Again many thanks for explaining and for your time.
> >>>
> >>> I got your point now and I have dug a bit better in the PCI_DOMAINS
> >> code.
> >>>
> >>> However I have a question...see inline below
> >>>
> >>>> -----Original Message-----
> >>>> From: Lucas Stach [mailto:l.stach at pengutronix.de]
> >>>> Sent: Thursday, August 20, 2015 9:27 AM
> >>>> To: Gabriele Paoloni
> >>>> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> >> Anand;
> >>>> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> >>>> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> >>>> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org; linux-
> >>>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>>> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu (Kenneth)
> >>>> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> >>>>
> >>>> Hi Gab,
> >>>>
> >>>> Am Mittwoch, den 19.08.2015, 16:34 +0000 schrieb Gabriele Paoloni:
> >>>>> Hi Lucas
> >>>>>
> >>>>> First of all many thanks for the quick reply, really appreciated
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Lucas Stach [mailto:l.stach at pengutronix.de]
> >>>>>> Sent: Wednesday, August 19, 2015 4:37 PM
> >>>>>> To: Gabriele Paoloni
> >>>>>> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1 at gmail.com; Pratyush
> >>>> Anand;
> >>>>>> Arnd Bergmann; linux at arm.linux.org.uk; thomas.petazzoni at free-
> >>>>>> electrons.com; lorenzo.pieralisi at arm.com; james.morse at arm.com;
> >>>>>> Liviu.Dudau at arm.com; jason at lakedaemon.net; robh at kernel.org;
> >> linux-
> >>>>>> pci at vger.kernel.org; linux-arm-kernel at lists.infradead.org;
> >>>>>> devicetree at vger.kernel.org; Yuanzhichang; Zhudacai; zhangjukuo;
> >>>>>> qiuzhenfa; liudongdong (C); qiujiang; xuwei (O); Liguozhu
> >> (Kenneth)
> >>>>>> Subject: Re: [PATCH v7 3/6] PCI: designware: Add ARM64 support
> >>>>>>
> >>>>>> Hi Gab,
> >>>>>>
> >>>>>> Am Mittwoch, den 19.08.2015, 15:16 +0000 schrieb Gabriele
> >> Paoloni:
> >>>>>>> Hi Lucas
> >>>>>>>
> >>>>>>> I have rewritten the patch to take into account multiple
> >>>> controllers.
> >>>>>>>
> >>>>>>> As you can see now there is a static var in
> >> dw_pcie_host_init()
> >>>> that
> >>>>>> tracks
> >>>>>>> the bus numbers used.
> >>>>>>
> >>>>>> This is wrong. The DT specifies the valid bus number range. You
> >> can
> >>>> not
> >>>>>> just assign the next free bus number to the root bus.
> >>>>>
> >>>>> I think this is what is being done in
> >>>>> http://lxr.free-
> >> electrons.com/source/arch/arm/kernel/bios32.c#L495
> >>>>> and currently designware assigns the root bus number in
> >>>>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> >>>> designware.c#L730
> >>>>>
> >>>> You found the right spot. It works a bit different than I remember,
> >> but
> >>>> is less broken than your current code. :)
> >>>>
> >>>> It actually assigns the next instance a root bus number behind the
> >>>> current instances bus range. Please note the difference to your
> >> code
> >>>> which assigns the next free bus number, which may still lay within
> >> the
> >>>> current instances bus range.
> >>
> >> Hi Gabriele,
> >>
> >>>
> >>> Mmmm here I have done a mistake: in the current designware the
> number
> >> of hw
> >>> controllers is always 1
> >>> http://lxr.free-electrons.com/source/drivers/pci/host/pcie-
> >> designware.c#L510
> >>> So the loop in bios32.c does not work on multiple PCIe ports...
> >>
> >> Bios32.c is broken for multiple PCIe hosts for multiple reasons,
> this
> >> is just
> >> one of them. Hence the effort to get rid of it for maintained
> drivers.
> >
> > As you can see we're pushing hard for this :)
> >
> 
> How about we pass pp->busn->start to pci_create_root_bus as root bus
> number?
> We get pp->busn->start from resource list(res) whose elements come from
> of_pci_get_host_bridge_resources.
> 
> If there is a bus-range item in dts, root bus number will get from it.
> Oppositely, root bus number will be default value(0x0).
> 
> Thanks,
> Zhou

Sounds good to me. So we can remove root_bus_nr

> 
> >>
> >>> However your comment about PCI_DOMAINS enlightened my mind and now
> I
> >> see
> >>> that when CONFIG_PCI_DOMAINS is defined we have the domains numbers
> >>> (tracked by __domain_nr).
> >>
> >> CONFIG_PCI_DOMAINS in itself doesn't do much without adding code to
> >> your driver.
> >> You could request a new domain for each controller with a special
> >> property to
> >> disable that behaviour in the dts, or you could enable
> >> CONFIG_PCI_DOMAINS_GENERIC
> >> which will give you a new domain automatically.
> >
> > So far I see that for ARM and ARM64 CONFIG_PCI_DOMAINS_GENERIC
> defaults to
> > the same value as CONFIG_PCI_DOMAINS (see arch/arm/Kconfig,
> arch/arm64/Kconfig).
> > So I think this is how current designware based drivers get multiple
> PCIe
> > controller to work (they just enable CONFIG_PCI_DOMAINS)...
> >
> >
> >
> >>
> >>> So (correct me if I am wrong) if we have 2 PCIe ports that do not
> >> specify
> >>> the "bus-range" property, both root ports will enumerate starting
> >> from
> >>> root_bus_nr = 0 and everything will still work ok.
> >>
> >> Correct.
> >>
> >> Best regards,
> >> Liviu
> >>
> >>>
> >>> So if my assumption is correct, I do not see why the orginal patch
> >> from Wang
> >>> Zhou is wrong.
> >>> The only point can be that assigning pp->root_bus_nr = 0 is not
> >> required
> >>> as pp memory is kzalloc'ed (an therefore init to zero).
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> In general I agree with you but if you look at all the current
> >>>> drivers
> >>>>> based on designware none of them define the "bus-range" dtb
> >> property.
> >>>>> Therefore doing as you say would break the current driver when we
> >>>> have
> >>>>> multiple controllers...am I right?
> >>>>>
> >>>> The current _code_ works fine for multiple controllers. It's just
> >> that
> >>>> you must define the bus range property in the DTB if you want to
> >> enable
> >>>> multiple instances of one controller and support a kernel without
> >> PCI
> >>>> domains support. As all current implementations have only a single
> >>>> instance of the controller per SoC, or depend on PCI domain
> support,
> >>>> it's totally fine for them to not define the bus ranges property,
> >> as
> >>>> it's an optional property and it is well defined how things behave
> >> if
> >>>> it
> >>>> is absent. We absolutely need to keep that specified behavior.
> >>>>
> >>>>> If that is the case in order to fix this in the way you say I
> >> would
> >>>> need
> >>>>> to assign "bus-range" for all the PCIe drivers with multiple
> >>>> controllers:
> >>>>> in this case I would split the default range evenly (that is, if
> >> we
> >>>> have
> >>>>> two controllers I would define "bus-range"  0-127 and 128-255)
> >>>>>
> >>>>> If you think this solution is ok I can go for it. My only doubt
> >> was
> >>>> about
> >>>>> touching other vendors DTBs....
> >>>>>
> >>>> You don't need to touch any of the current DTBs, as they are fine
> >> with
> >>>> the default bus range behavior. You need to keep the specified
> >> behavior
> >>>> of the bus range property with the new code.
> >>>>
> >>>> Regards,
> >>>> Lucas
> >>>>>
> >>>>>>
> >>>>>> It is perfectly valid to have a bus range of 0x00-0x10 assigned
> >> to
> >>>> one
> >>>>>> instance and 0x50-0xff to the next instance. Additional with
> >> PCIe
> >>>>>> hotplug you may not use the full range of the bus numbers on
> >> one
> >>>>>> instance at the first scan, but only later populate more buses
> >> when
> >>>>>> more
> >>>>>> bridges are added to the tree.
> >>>>>>
> >>>>>>> Drivers that do not specify the bus range in the DTB set pp-
> >>>>>>> root_bus_nr = DW_ROOT_NR_UNDEFINED.
> >>>>>>> Designware will check if the flag is set and will use the
> >>>> automatic
> >>>>>> bus range
> >>>>>>> assignment.
> >>>>>>
> >>>>>> No, please lets get rid of this assignment altogether. The glue
> >>>> drivers
> >>>>>> have no business in assigning the bus range. Please remove the
> >>>>>> pp->root_bus_nr assignment from all the glue drivers.
> >>>>>>
> >>>>>> "bus range" is a generic DW PCIe property, so just parse the
> >> root
> >>>> bus
> >>>>>> number from the DT, it is handled the same way for all the DW
> >> based
> >>>>>> PCIe
> >>>>>> drivers. The bindings specifies that if the bus range property
> >> is
> >>>>>> missing the range is 0x00-0xff, so you can default to 0 as the
> >> root
> >>>> bus
> >>>>>> number in that case.
> >>>>>>
> >>>>>> Also I would think this conversion warrants a patch on its own
> >> and
> >>>>>> should not be mixed in the ARM64 support patch.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Lucas
> >>>>>>
> >>>>
> >>>> --
> >>>> Pengutronix e.K.             | Lucas Stach                 |
> >>>> Industrial Linux Solutions   | http://www.pengutronix.de/  |
> >>>
> >>
> >> --
> >> ====================
> >> | I would like to |
> >> | fix the world,  |
> >> | but they're not |
> >> | giving me the   |
> >>  \ source code!  /
> >>   ---------------
> >>     ?\_(?)_/?
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-pci"
> in
> >> the body of a message to majordomo at vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2015-08-24  8:28 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-08-17 11:55 [PATCH v7 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
2015-08-17 11:55 ` [PATCH v7 1/6] PCI: designware: move calculation of bus addresses to DRA7xx Zhou Wang
2015-08-17 11:55 ` [PATCH v7 2/6] ARM/PCI: remove align_resource in pci_sys_data Zhou Wang
2015-08-17 11:55 ` [PATCH v7 3/6] PCI: designware: Add ARM64 support Zhou Wang
2015-08-19 12:20   ` Zhou Wang
2015-08-19 12:54   ` Lucas Stach
2015-08-19 15:16     ` Gabriele Paoloni
2015-08-19 15:37       ` Lucas Stach
2015-08-19 16:34         ` Gabriele Paoloni
2015-08-20  8:27           ` Lucas Stach
2015-08-20 11:10             ` Gabriele Paoloni
2015-08-21 13:42               ` Liviu Dudau
2015-08-21 14:19                 ` Gabriele Paoloni
2015-08-24  6:26                   ` Zhou Wang
2015-08-24  8:28                     ` Gabriele Paoloni
2015-08-17 11:55 ` [PATCH v7 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05 Zhou Wang
2015-08-17 11:55 ` [PATCH v7 5/6] Documentation: DT: Add HiSilicon PCIe host binding Zhou Wang
2015-08-17 11:55 ` [PATCH v7 6/6] MAINTAINERS: Add pcie-hisi maintainer Zhou Wang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).