linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
@ 2020-11-29 23:07 Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
                   ` (6 more replies)
  0 siblings, 7 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Unify ECAM-related constants into a single set of standard constants
defining memory address shift values for the byte-level address that can
be used when accessing the PCI Express Configuration Space, and then
move native PCI Express controller drivers to use newly introduced
definitions retiring any driver-specific ones.

The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
PCI Express specification (see PCI Express Base Specification, Revision
5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
implement it the same way.

Most of the native PCI Express controller drivers define their ECAM-related
constants, many of these could be shared, or use open-coded values when
setting the ".bus_shift" field of the "struct pci_ecam_ops".

All of the newly added constants should remove ambiguity and reduce the
number of open-coded values, and also correlate more strongly with the
descriptions in the aforementioned specification (see Table 7-1
"Enhanced Configuration Address Mapping", p. 677).

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>

--- 
Changed in v6:
  Converted single patch into a series.
  Dropped changes related to PPC 4xx platform.
  Refactored pci_ecam_map_bus() so that bus, device function and offset
  are correctly masked, limiting offset to 4K as per the PCI Express
  Specification.  After the refactor this function will now use sensible
  defaults allowing for removal of the ".bus_shit" initialiser from all
  the users of the "ecam_ops" structure who do not use a non-standard
  ECAM bus shit values.

Changed in v5:
  Removed unused constant "PCIE_ECAM_DEV_SHIFT".
  Refactored ppc4xx_pciex_get_config_base() so that the "offset"
  parameter can be passed to so that the PCIE_ECAM_OFFSET() macro
  can be used.
  Used the ALIGN_DOWN() macro where 32 bit alignment is required
  instead using the 0xffc mask.
  Added CFG_ADDR_CFG_TYPE_1 macro to pci/controller/pcie-iproc.c to
  denote that this is a configuration type 1 address and access type.
  Refactored boundary check in pci/controller/vmd.c as used by the
  vmd_cfg_addr() function following addition of the PCIE_ECAM_OFFSET()
  macro.
  Changed the "bus->number" to simply pass the "bus" argument in the
  PCIE_ECAM_OFFSET() macro.

Changed in v4:
  Removed constants related to "CAM".
  Added more platforms and devices that can use new ECAM macros and
  constants.
  Removed unused ".bus_shift" initialisers from pci-xgene.c as
  xgene_pcie_map_bus() did not use these.

Changes in v3:
  Updated commit message wording.
  Updated regarding custom ECAM bus shift values and concerning PCI base
  configuration space access for Type 1 access.
  Refactored rockchip_pcie_rd_other_conf() and rockchip_pcie_wr_other_conf()
  and removed the "busdev" variable.
  Removed surplus "relbus" variable from nwl_pcie_map_bus() and
  xilinx_pcie_map_bus().
  Renamed the PCIE_ECAM_ADDR() macro to PCIE_ECAM_OFFSET().

Changes in v2:
  Use PCIE_ECAM_ADDR macro when computing ECAM address offset, but drop
  PCI_SLOT and PCI_FUNC macros from the PCIE_ECAM_ADDR macro in favour
  of using a single value for the device/function.

Krzysztof Wilczyński (5):
  PCI: Unify ECAM constants in native PCI Express drivers
  PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser
  PCI: iproc: Convert to use the new ECAM constants
  PCI: vmd: Update type of the __iomem pointers
  PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c

 drivers/pci/controller/dwc/pcie-al.c        | 12 ++------
 drivers/pci/controller/dwc/pcie-hisi.c      |  2 --
 drivers/pci/controller/pci-aardvark.c       | 13 ++-------
 drivers/pci/controller/pci-host-generic.c   |  1 -
 drivers/pci/controller/pci-thunder-ecam.c   |  1 -
 drivers/pci/controller/pci-thunder-pem.c    | 13 +++++++--
 drivers/pci/controller/pci-xgene.c          |  2 --
 drivers/pci/controller/pcie-brcmstb.c       | 16 ++---------
 drivers/pci/controller/pcie-iproc.c         | 31 ++++++---------------
 drivers/pci/controller/pcie-rockchip-host.c | 27 +++++++++---------
 drivers/pci/controller/pcie-rockchip.h      |  8 +-----
 drivers/pci/controller/pcie-tango.c         |  1 -
 drivers/pci/controller/pcie-xilinx-nwl.c    |  9 ++----
 drivers/pci/controller/pcie-xilinx.c        | 11 ++------
 drivers/pci/controller/vmd.c                | 19 ++++++-------
 drivers/pci/ecam.c                          | 23 ++++++++++-----
 include/linux/pci-ecam.h                    | 27 ++++++++++++++++++
 17 files changed, 96 insertions(+), 120 deletions(-)

-- 
2.29.2


^ permalink raw reply	[flat|nested] 32+ messages in thread

* [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
@ 2020-11-29 23:07 ` Krzysztof Wilczyński
  2020-11-30 11:08   ` Lorenzo Pieralisi
                     ` (2 more replies)
  2020-11-29 23:07 ` [PATCH v6 2/5] PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser Krzysztof Wilczyński
                   ` (5 subsequent siblings)
  6 siblings, 3 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Add ECAM-related constants to provide a set of standard constants
defining memory address shift values to the byte-level address that can
be used to access the PCI Express Configuration Space, and then move
native PCI Express controller drivers to use the newly introduced
definitions retiring driver-specific ones.

Refactor pci_ecam_map_bus() function to use newly added constants so
that limits to the bus, device function and offset (now limited to 4K as
per the specification) are in place to prevent the defective or
malicious caller from supplying incorrect configuration offset and thus
targeting the wrong device when accessing extended configuration space.
This refactor also allows for the ".bus_shit" initialisers to be dropped
when the user is not using a custom value as a default value will be
used as per the PCI Express Specification.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
---
 drivers/pci/controller/dwc/pcie-al.c        | 12 ++-------
 drivers/pci/controller/dwc/pcie-hisi.c      |  2 --
 drivers/pci/controller/pci-aardvark.c       | 13 +++-------
 drivers/pci/controller/pci-host-generic.c   |  1 -
 drivers/pci/controller/pci-thunder-ecam.c   |  1 -
 drivers/pci/controller/pcie-brcmstb.c       | 16 ++----------
 drivers/pci/controller/pcie-rockchip-host.c | 27 ++++++++++-----------
 drivers/pci/controller/pcie-rockchip.h      |  8 +-----
 drivers/pci/controller/pcie-tango.c         |  1 -
 drivers/pci/controller/pcie-xilinx-nwl.c    |  9 ++-----
 drivers/pci/controller/pcie-xilinx.c        | 11 ++-------
 drivers/pci/controller/vmd.c                | 11 ++++-----
 drivers/pci/ecam.c                          | 23 ++++++++++++------
 include/linux/pci-ecam.h                    | 27 +++++++++++++++++++++
 14 files changed, 73 insertions(+), 89 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-al.c b/drivers/pci/controller/dwc/pcie-al.c
index f973fbca90cf..af9e51ab1af8 100644
--- a/drivers/pci/controller/dwc/pcie-al.c
+++ b/drivers/pci/controller/dwc/pcie-al.c
@@ -76,7 +76,6 @@ static int al_pcie_init(struct pci_config_window *cfg)
 }
 
 const struct pci_ecam_ops al_pcie_ops = {
-	.bus_shift    = 20,
 	.init         =  al_pcie_init,
 	.pci_ops      = {
 		.map_bus    = al_pcie_map_bus,
@@ -138,8 +137,6 @@ struct al_pcie {
 	struct al_pcie_target_bus_cfg target_bus_cfg;
 };
 
-#define PCIE_ECAM_DEVFN(x)		(((x) & 0xff) << 12)
-
 #define to_al_pcie(x)		dev_get_drvdata((x)->dev)
 
 static inline u32 al_pcie_controller_readl(struct al_pcie *pcie, u32 offset)
@@ -226,11 +223,6 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
 	struct al_pcie_target_bus_cfg *target_bus_cfg = &pcie->target_bus_cfg;
 	unsigned int busnr_ecam = busnr & target_bus_cfg->ecam_mask;
 	unsigned int busnr_reg = busnr & target_bus_cfg->reg_mask;
-	void __iomem *pci_base_addr;
-
-	pci_base_addr = (void __iomem *)((uintptr_t)pp->va_cfg0_base +
-					 (busnr_ecam << 20) +
-					 PCIE_ECAM_DEVFN(devfn));
 
 	if (busnr_reg != target_bus_cfg->reg_val) {
 		dev_dbg(pcie->pci->dev, "Changing target bus busnum val from 0x%x to 0x%x\n",
@@ -241,7 +233,7 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
 				       target_bus_cfg->reg_mask);
 	}
 
-	return pci_base_addr + where;
+	return pp->va_cfg0_base + PCIE_ECAM_OFFSET(busnr_ecam, devfn, where);
 }
 
 static struct pci_ops al_child_pci_ops = {
@@ -264,7 +256,7 @@ static void al_pcie_config_prepare(struct al_pcie *pcie)
 
 	target_bus_cfg = &pcie->target_bus_cfg;
 
-	ecam_bus_mask = (pcie->ecam_size >> 20) - 1;
+	ecam_bus_mask = (pcie->ecam_size >> PCIE_ECAM_BUS_SHIFT) - 1;
 	if (ecam_bus_mask > 255) {
 		dev_warn(pcie->dev, "ECAM window size is larger than 256MB. Cutting off at 256\n");
 		ecam_bus_mask = 255;
diff --git a/drivers/pci/controller/dwc/pcie-hisi.c b/drivers/pci/controller/dwc/pcie-hisi.c
index 5ca86796d43a..8fc5960faf28 100644
--- a/drivers/pci/controller/dwc/pcie-hisi.c
+++ b/drivers/pci/controller/dwc/pcie-hisi.c
@@ -100,7 +100,6 @@ static int hisi_pcie_init(struct pci_config_window *cfg)
 }
 
 const struct pci_ecam_ops hisi_pcie_ops = {
-	.bus_shift    = 20,
 	.init         =  hisi_pcie_init,
 	.pci_ops      = {
 		.map_bus    = hisi_pcie_map_bus,
@@ -135,7 +134,6 @@ static int hisi_pcie_platform_init(struct pci_config_window *cfg)
 }
 
 static const struct pci_ecam_ops hisi_pcie_platform_ops = {
-	.bus_shift    = 20,
 	.init         =  hisi_pcie_platform_init,
 	.pci_ops      = {
 		.map_bus    = hisi_pcie_map_bus,
diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 0be485a25327..1043e54c73bd 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -16,6 +16,7 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/init.h>
 #include <linux/phy/phy.h>
 #include <linux/platform_device.h>
@@ -164,14 +165,6 @@
 #define PCIE_CONFIG_WR_TYPE0			0xa
 #define PCIE_CONFIG_WR_TYPE1			0xb
 
-#define PCIE_CONF_BUS(bus)			(((bus) & 0xff) << 20)
-#define PCIE_CONF_DEV(dev)			(((dev) & 0x1f) << 15)
-#define PCIE_CONF_FUNC(fun)			(((fun) & 0x7)	<< 12)
-#define PCIE_CONF_REG(reg)			((reg) & 0xffc)
-#define PCIE_CONF_ADDR(bus, devfn, where)	\
-	(PCIE_CONF_BUS(bus) | PCIE_CONF_DEV(PCI_SLOT(devfn))	| \
-	 PCIE_CONF_FUNC(PCI_FUNC(devfn)) | PCIE_CONF_REG(where))
-
 #define PIO_RETRY_CNT			500
 #define PIO_RETRY_DELAY			2 /* 2 us*/
 
@@ -687,7 +680,7 @@ static int advk_pcie_rd_conf(struct pci_bus *bus, u32 devfn,
 	advk_writel(pcie, reg, PIO_CTRL);
 
 	/* Program the address registers */
-	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
+	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
 	advk_writel(pcie, reg, PIO_ADDR_LS);
 	advk_writel(pcie, 0, PIO_ADDR_MS);
 
@@ -748,7 +741,7 @@ static int advk_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
 	advk_writel(pcie, reg, PIO_CTRL);
 
 	/* Program the address registers */
-	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
+	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
 	advk_writel(pcie, reg, PIO_ADDR_LS);
 	advk_writel(pcie, 0, PIO_ADDR_MS);
 
diff --git a/drivers/pci/controller/pci-host-generic.c b/drivers/pci/controller/pci-host-generic.c
index b51977abfdf1..63865aeb636b 100644
--- a/drivers/pci/controller/pci-host-generic.c
+++ b/drivers/pci/controller/pci-host-generic.c
@@ -49,7 +49,6 @@ static void __iomem *pci_dw_ecam_map_bus(struct pci_bus *bus,
 }
 
 static const struct pci_ecam_ops pci_dw_ecam_bus_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus	= pci_dw_ecam_map_bus,
 		.read		= pci_generic_config_read,
diff --git a/drivers/pci/controller/pci-thunder-ecam.c b/drivers/pci/controller/pci-thunder-ecam.c
index 7e8835fee5f7..f964fd26f7e0 100644
--- a/drivers/pci/controller/pci-thunder-ecam.c
+++ b/drivers/pci/controller/pci-thunder-ecam.c
@@ -346,7 +346,6 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
 }
 
 const struct pci_ecam_ops pci_thunder_ecam_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus        = pci_ecam_map_bus,
 		.read           = thunder_ecam_config_read,
diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index bea86899bd5d..7fc80fd6f13f 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -22,6 +22,7 @@
 #include <linux/of_pci.h>
 #include <linux/of_platform.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/printk.h>
 #include <linux/reset.h>
 #include <linux/sizes.h>
@@ -127,11 +128,7 @@
 #define  MSI_INT_MASK_CLR		0x14
 
 #define PCIE_EXT_CFG_DATA				0x8000
-
 #define PCIE_EXT_CFG_INDEX				0x9000
-#define  PCIE_EXT_BUSNUM_SHIFT				20
-#define  PCIE_EXT_SLOT_SHIFT				15
-#define  PCIE_EXT_FUNC_SHIFT				12
 
 #define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
 #define  PCIE_RGR1_SW_INIT_1_PERST_SHIFT		0x0
@@ -695,15 +692,6 @@ static bool brcm_pcie_link_up(struct brcm_pcie *pcie)
 	return dla && plu;
 }
 
-/* Configuration space read/write support */
-static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
-{
-	return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
-		| ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
-		| (busnr << PCIE_EXT_BUSNUM_SHIFT)
-		| (reg & ~3);
-}
-
 static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
 					int where)
 {
@@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
 		return PCI_SLOT(devfn) ? NULL : base + where;
 
 	/* For devices, write to the config space index register */
-	idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
+	idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
 	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
 	return base + PCIE_EXT_CFG_DATA + where;
 }
diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index 9705059523a6..f1d08a1b1591 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -157,12 +157,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 				       struct pci_bus *bus, u32 devfn,
 				       int where, int size, u32 *val)
 {
-	u32 busdev;
+	void __iomem *addr;
 
-	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
-				PCI_FUNC(devfn), where);
+	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 
-	if (!IS_ALIGNED(busdev, size)) {
+	if (!IS_ALIGNED((uintptr_t)addr, size)) {
 		*val = 0;
 		return PCIBIOS_BAD_REGISTER_NUMBER;
 	}
@@ -175,11 +174,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 						AXI_WRAPPER_TYPE1_CFG);
 
 	if (size == 4) {
-		*val = readl(rockchip->reg_base + busdev);
+		*val = readl(addr);
 	} else if (size == 2) {
-		*val = readw(rockchip->reg_base + busdev);
+		*val = readw(addr);
 	} else if (size == 1) {
-		*val = readb(rockchip->reg_base + busdev);
+		*val = readb(addr);
 	} else {
 		*val = 0;
 		return PCIBIOS_BAD_REGISTER_NUMBER;
@@ -191,11 +190,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
 				       struct pci_bus *bus, u32 devfn,
 				       int where, int size, u32 val)
 {
-	u32 busdev;
+	void __iomem *addr;
 
-	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
-				PCI_FUNC(devfn), where);
-	if (!IS_ALIGNED(busdev, size))
+	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
+
+	if (!IS_ALIGNED((uintptr_t)addr, size))
 		return PCIBIOS_BAD_REGISTER_NUMBER;
 
 	if (pci_is_root_bus(bus->parent))
@@ -206,11 +205,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
 						AXI_WRAPPER_TYPE1_CFG);
 
 	if (size == 4)
-		writel(val, rockchip->reg_base + busdev);
+		writel(val, addr);
 	else if (size == 2)
-		writew(val, rockchip->reg_base + busdev);
+		writew(val, addr);
 	else if (size == 1)
-		writeb(val, rockchip->reg_base + busdev);
+		writeb(val, addr);
 	else
 		return PCIBIOS_BAD_REGISTER_NUMBER;
 
diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
index c7d0178fc8c2..1650a5087450 100644
--- a/drivers/pci/controller/pcie-rockchip.h
+++ b/drivers/pci/controller/pcie-rockchip.h
@@ -13,6 +13,7 @@
 
 #include <linux/kernel.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 
 /*
  * The upper 16 bits of PCIE_CLIENT_CONFIG are a write mask for the lower 16
@@ -178,13 +179,6 @@
 #define MIN_AXI_ADDR_BITS_PASSED		8
 #define PCIE_RC_SEND_PME_OFF			0x11960
 #define ROCKCHIP_VENDOR_ID			0x1d87
-#define PCIE_ECAM_BUS(x)			(((x) & 0xff) << 20)
-#define PCIE_ECAM_DEV(x)			(((x) & 0x1f) << 15)
-#define PCIE_ECAM_FUNC(x)			(((x) & 0x7) << 12)
-#define PCIE_ECAM_REG(x)			(((x) & 0xfff) << 0)
-#define PCIE_ECAM_ADDR(bus, dev, func, reg) \
-	  (PCIE_ECAM_BUS(bus) | PCIE_ECAM_DEV(dev) | \
-	   PCIE_ECAM_FUNC(func) | PCIE_ECAM_REG(reg))
 #define PCIE_LINK_IS_L2(x) \
 	(((x) & PCIE_CLIENT_DEBUG_LTSSM_MASK) == PCIE_CLIENT_DEBUG_LTSSM_L2)
 #define PCIE_LINK_UP(x) \
diff --git a/drivers/pci/controller/pcie-tango.c b/drivers/pci/controller/pcie-tango.c
index d093a8ce4bb1..62a061f1d62e 100644
--- a/drivers/pci/controller/pcie-tango.c
+++ b/drivers/pci/controller/pcie-tango.c
@@ -208,7 +208,6 @@ static int smp8759_config_write(struct pci_bus *bus, unsigned int devfn,
 }
 
 static const struct pci_ecam_ops smp8759_ecam_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= smp8759_config_read,
diff --git a/drivers/pci/controller/pcie-xilinx-nwl.c b/drivers/pci/controller/pcie-xilinx-nwl.c
index f3cf7d61924f..7f29c2fdcd51 100644
--- a/drivers/pci/controller/pcie-xilinx-nwl.c
+++ b/drivers/pci/controller/pcie-xilinx-nwl.c
@@ -18,6 +18,7 @@
 #include <linux/of_platform.h>
 #include <linux/of_irq.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 #include <linux/irqchip/chained_irq.h>
 
@@ -124,8 +125,6 @@
 #define E_ECAM_CR_ENABLE		BIT(0)
 #define E_ECAM_SIZE_LOC			GENMASK(20, 16)
 #define E_ECAM_SIZE_SHIFT		16
-#define ECAM_BUS_LOC_SHIFT		20
-#define ECAM_DEV_LOC_SHIFT		12
 #define NWL_ECAM_VALUE_DEFAULT		12
 
 #define CFG_DMA_REG_BAR			GENMASK(2, 0)
@@ -240,15 +239,11 @@ static void __iomem *nwl_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
 				      int where)
 {
 	struct nwl_pcie *pcie = bus->sysdata;
-	int relbus;
 
 	if (!nwl_pcie_valid_device(bus, devfn))
 		return NULL;
 
-	relbus = (bus->number << ECAM_BUS_LOC_SHIFT) |
-			(devfn << ECAM_DEV_LOC_SHIFT);
-
-	return pcie->ecam_base + relbus + where;
+	return pcie->ecam_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 }
 
 /* PCIe operations */
diff --git a/drivers/pci/controller/pcie-xilinx.c b/drivers/pci/controller/pcie-xilinx.c
index 8523be61bba5..fa5baeb82653 100644
--- a/drivers/pci/controller/pcie-xilinx.c
+++ b/drivers/pci/controller/pcie-xilinx.c
@@ -21,6 +21,7 @@
 #include <linux/of_platform.h>
 #include <linux/of_irq.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
 #include "../pci.h"
@@ -86,10 +87,6 @@
 /* Phy Status/Control Register definitions */
 #define XILINX_PCIE_REG_PSCR_LNKUP	BIT(11)
 
-/* ECAM definitions */
-#define ECAM_BUS_NUM_SHIFT		20
-#define ECAM_DEV_NUM_SHIFT		12
-
 /* Number of MSI IRQs */
 #define XILINX_NUM_MSI_IRQS		128
 
@@ -183,15 +180,11 @@ static void __iomem *xilinx_pcie_map_bus(struct pci_bus *bus,
 					 unsigned int devfn, int where)
 {
 	struct xilinx_pcie_port *port = bus->sysdata;
-	int relbus;
 
 	if (!xilinx_pcie_valid_device(bus, devfn))
 		return NULL;
 
-	relbus = (bus->number << ECAM_BUS_NUM_SHIFT) |
-		 (devfn << ECAM_DEV_NUM_SHIFT);
-
-	return port->reg_base + relbus + where;
+	return port->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 }
 
 /* PCIe operations */
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index f375c21ceeb1..1361a79bd1e7 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/msi.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/srcu.h>
 #include <linux/rculist.h>
 #include <linux/rcupdate.h>
@@ -328,15 +329,13 @@ static void vmd_remove_irq_domain(struct vmd_dev *vmd)
 static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
 				  unsigned int devfn, int reg, int len)
 {
-	char __iomem *addr = vmd->cfgbar +
-			     ((bus->number - vmd->busn_start) << 20) +
-			     (devfn << 12) + reg;
+	unsigned int busnr_ecam = bus->number - vmd->busn_start;
+	u32 offset = PCIE_ECAM_OFFSET(busnr_ecam, devfn, reg);
 
-	if ((addr - vmd->cfgbar) + len >=
-	    resource_size(&vmd->dev->resource[VMD_CFGBAR]))
+	if (offset + len >= resource_size(&vmd->dev->resource[VMD_CFGBAR]))
 		return NULL;
 
-	return addr;
+	return vmd->cfgbar + offset;
 }
 
 /*
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index b54d32a31669..59f91d434859 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -131,25 +131,36 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 			       int where)
 {
 	struct pci_config_window *cfg = bus->sysdata;
+	unsigned int bus_shift = cfg->ops->bus_shift;
 	unsigned int devfn_shift = cfg->ops->bus_shift - 8;
 	unsigned int busn = bus->number;
 	void __iomem *base;
+	u32 bus_offset, devfn_offset;
 
 	if (busn < cfg->busr.start || busn > cfg->busr.end)
 		return NULL;
 
 	busn -= cfg->busr.start;
-	if (per_bus_mapping)
+	if (per_bus_mapping) {
 		base = cfg->winp[busn];
-	else
-		base = cfg->win + (busn << cfg->ops->bus_shift);
-	return base + (devfn << devfn_shift) + where;
+		busn = 0;
+	} else
+		base = cfg->win;
+
+	if (cfg->ops->bus_shift) {
+		bus_offset = (busn & PCIE_ECAM_BUS_MASK) << bus_shift;
+		devfn_offset = (devfn & PCIE_ECAM_DEVFN_MASK) << devfn_shift;
+		where &= PCIE_ECAM_REG_MASK;
+
+		return base + (bus_offset | devfn_offset | where);
+	}
+
+	return base + PCIE_ECAM_OFFSET(busn, devfn, where);
 }
 EXPORT_SYMBOL_GPL(pci_ecam_map_bus);
 
 /* ECAM ops */
 const struct pci_ecam_ops pci_generic_ecam_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= pci_generic_config_read,
@@ -161,7 +172,6 @@ EXPORT_SYMBOL_GPL(pci_generic_ecam_ops);
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 /* ECAM ops for 32-bit access only (non-compliant) */
 const struct pci_ecam_ops pci_32b_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= pci_generic_config_read32,
@@ -171,7 +181,6 @@ const struct pci_ecam_ops pci_32b_ops = {
 
 /* ECAM ops for 32-bit read only (non-compliant) */
 const struct pci_ecam_ops pci_32b_read_ops = {
-	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= pci_generic_config_read32,
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 033ce74f02e8..65d3d83015c3 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -9,6 +9,33 @@
 #include <linux/kernel.h>
 #include <linux/platform_device.h>
 
+/*
+ * Memory address shift values for the byte-level address that
+ * can be used when accessing the PCI Express Configuration Space.
+ */
+
+/*
+ * Enhanced Configuration Access Mechanism (ECAM)
+ *
+ * See PCI Express Base Specification, Revision 5.0, Version 1.0,
+ * Section 7.2.2, Table 7-1, p. 677.
+ */
+#define PCIE_ECAM_BUS_SHIFT	20 /* Bus number */
+#define PCIE_ECAM_DEVFN_SHIFT	12 /* Device and Function number */
+
+#define PCIE_ECAM_BUS_MASK	0xff
+#define PCIE_ECAM_DEVFN_MASK	0xff
+#define PCIE_ECAM_REG_MASK	0xfff /* Limit offset to a maximum of 4K */
+
+#define PCIE_ECAM_BUS(x)	(((x) & PCIE_ECAM_BUS_MASK) << PCIE_ECAM_BUS_SHIFT)
+#define PCIE_ECAM_DEVFN(x)	(((x) & PCIE_ECAM_DEVFN_MASK) << PCIE_ECAM_DEVFN_SHIFT)
+#define PCIE_ECAM_REG(x)	((x) & PCIE_ECAM_REG_MASK)
+
+#define PCIE_ECAM_OFFSET(bus, devfn, where) \
+	(PCIE_ECAM_BUS(bus) | \
+	 PCIE_ECAM_DEVFN(devfn) | \
+	 PCIE_ECAM_REG(where))
+
 /*
  * struct to hold pci ops and bus shift of the config window
  * for a PCI controller.
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH v6 2/5] PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
@ 2020-11-29 23:07 ` Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 3/5] PCI: iproc: Convert to use the new ECAM constants Krzysztof Wilczyński
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Add a custom constant for the ".bus_shit" initialiser to capture
a non-standard platform-specific ECAM bus shift value.

Standard values otherwise defined in the PCI Express Specification
are available in the include/linux/pci-ecam.h.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
---
 drivers/pci/controller/pci-thunder-pem.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-thunder-pem.c b/drivers/pci/controller/pci-thunder-pem.c
index 3f847969143e..1a3f70ac61fc 100644
--- a/drivers/pci/controller/pci-thunder-pem.c
+++ b/drivers/pci/controller/pci-thunder-pem.c
@@ -19,6 +19,15 @@
 #define PEM_CFG_WR 0x28
 #define PEM_CFG_RD 0x30
 
+/*
+ * Enhanced Configuration Access Mechanism (ECAM)
+ *
+ * N.B. This is a non-standard platform-specific ECAM bus shift value.  For
+ * standard values defined in the PCI Express Base Specification see
+ * include/linux/pci-ecam.h.
+ */
+#define THUNDER_PCIE_ECAM_BUS_SHIFT	24
+
 struct thunder_pem_pci {
 	u32		ea_entry[3];
 	void __iomem	*pem_reg_base;
@@ -404,7 +413,7 @@ static int thunder_pem_acpi_init(struct pci_config_window *cfg)
 }
 
 const struct pci_ecam_ops thunder_pem_ecam_ops = {
-	.bus_shift	= 24,
+	.bus_shift	= THUNDER_PCIE_ECAM_BUS_SHIFT,
 	.init		= thunder_pem_acpi_init,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
@@ -441,7 +450,7 @@ static int thunder_pem_platform_init(struct pci_config_window *cfg)
 }
 
 static const struct pci_ecam_ops pci_thunder_pem_ops = {
-	.bus_shift	= 24,
+	.bus_shift	= THUNDER_PCIE_ECAM_BUS_SHIFT,
 	.init		= thunder_pem_platform_init,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH v6 3/5] PCI: iproc: Convert to use the new ECAM constants
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 2/5] PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser Krzysztof Wilczyński
@ 2020-11-29 23:07 ` Krzysztof Wilczyński
  2020-11-29 23:07 ` [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers Krzysztof Wilczyński
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Change interface of the function iproc_pcie_map_ep_cfg_reg() so that use
of PCI_SLOT() and PCI_FUNC() macros and most of the local ECAM-specific
constants can be dropped, and the new PCIE_ECAM_OFFSET() macro can be
used instead.  Use the ALIGN_DOWN() macro to ensure that PCI Express
ECAM offset is always 32 bit aligned.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
---
 drivers/pci/controller/pcie-iproc.c | 31 ++++++++---------------------
 1 file changed, 8 insertions(+), 23 deletions(-)

diff --git a/drivers/pci/controller/pcie-iproc.c b/drivers/pci/controller/pcie-iproc.c
index 905e93808243..503662380ff8 100644
--- a/drivers/pci/controller/pcie-iproc.c
+++ b/drivers/pci/controller/pcie-iproc.c
@@ -6,6 +6,7 @@
 
 #include <linux/kernel.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/msi.h>
 #include <linux/clk.h>
 #include <linux/module.h>
@@ -39,16 +40,8 @@
 
 #define CFG_IND_ADDR_MASK		0x00001ffc
 
-#define CFG_ADDR_BUS_NUM_SHIFT		20
-#define CFG_ADDR_BUS_NUM_MASK		0x0ff00000
-#define CFG_ADDR_DEV_NUM_SHIFT		15
-#define CFG_ADDR_DEV_NUM_MASK		0x000f8000
-#define CFG_ADDR_FUNC_NUM_SHIFT		12
-#define CFG_ADDR_FUNC_NUM_MASK		0x00007000
-#define CFG_ADDR_REG_NUM_SHIFT		2
 #define CFG_ADDR_REG_NUM_MASK		0x00000ffc
-#define CFG_ADDR_CFG_TYPE_SHIFT		0
-#define CFG_ADDR_CFG_TYPE_MASK		0x00000003
+#define CFG_ADDR_CFG_TYPE_1		1
 
 #define SYS_RC_INTX_MASK		0xf
 
@@ -459,19 +452,15 @@ static inline void iproc_pcie_apb_err_disable(struct pci_bus *bus,
 
 static void __iomem *iproc_pcie_map_ep_cfg_reg(struct iproc_pcie *pcie,
 					       unsigned int busno,
-					       unsigned int slot,
-					       unsigned int fn,
+					       unsigned int devfn,
 					       int where)
 {
 	u16 offset;
 	u32 val;
 
 	/* EP device access */
-	val = (busno << CFG_ADDR_BUS_NUM_SHIFT) |
-		(slot << CFG_ADDR_DEV_NUM_SHIFT) |
-		(fn << CFG_ADDR_FUNC_NUM_SHIFT) |
-		(where & CFG_ADDR_REG_NUM_MASK) |
-		(1 & CFG_ADDR_CFG_TYPE_MASK);
+	val = ALIGN_DOWN(PCIE_ECAM_OFFSET(busno, devfn, where), 4) |
+		CFG_ADDR_CFG_TYPE_1;
 
 	iproc_pcie_write_reg(pcie, IPROC_PCIE_CFG_ADDR, val);
 	offset = iproc_pcie_reg_offset(pcie, IPROC_PCIE_CFG_DATA);
@@ -574,8 +563,6 @@ static int iproc_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
 				  int where, int size, u32 *val)
 {
 	struct iproc_pcie *pcie = iproc_data(bus);
-	unsigned int slot = PCI_SLOT(devfn);
-	unsigned int fn = PCI_FUNC(devfn);
 	unsigned int busno = bus->number;
 	void __iomem *cfg_data_p;
 	unsigned int data;
@@ -590,7 +577,7 @@ static int iproc_pcie_config_read(struct pci_bus *bus, unsigned int devfn,
 		return ret;
 	}
 
-	cfg_data_p = iproc_pcie_map_ep_cfg_reg(pcie, busno, slot, fn, where);
+	cfg_data_p = iproc_pcie_map_ep_cfg_reg(pcie, busno, devfn, where);
 
 	if (!cfg_data_p)
 		return PCIBIOS_DEVICE_NOT_FOUND;
@@ -631,13 +618,11 @@ static void __iomem *iproc_pcie_map_cfg_bus(struct iproc_pcie *pcie,
 					    int busno, unsigned int devfn,
 					    int where)
 {
-	unsigned slot = PCI_SLOT(devfn);
-	unsigned fn = PCI_FUNC(devfn);
 	u16 offset;
 
 	/* root complex access */
 	if (busno == 0) {
-		if (slot > 0 || fn > 0)
+		if (PCIE_ECAM_DEVFN(devfn) > 0)
 			return NULL;
 
 		iproc_pcie_write_reg(pcie, IPROC_PCIE_CFG_IND_ADDR,
@@ -649,7 +634,7 @@ static void __iomem *iproc_pcie_map_cfg_bus(struct iproc_pcie *pcie,
 			return (pcie->base + offset);
 	}
 
-	return iproc_pcie_map_ep_cfg_reg(pcie, busno, slot, fn, where);
+	return iproc_pcie_map_ep_cfg_reg(pcie, busno, devfn, where);
 }
 
 static void __iomem *iproc_pcie_bus_map_cfg_bus(struct pci_bus *bus,
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
                   ` (2 preceding siblings ...)
  2020-11-29 23:07 ` [PATCH v6 3/5] PCI: iproc: Convert to use the new ECAM constants Krzysztof Wilczyński
@ 2020-11-29 23:07 ` Krzysztof Wilczyński
  2020-11-30  9:06   ` David Laight
  2020-11-29 23:07 ` [PATCH v6 5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c Krzysztof Wilczyński
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Use "void __iomem" instead "char __iomem" pointer type when working with
the accessor functions (with names like readb() or writel(), etc.) to
better match a given accessor function signature where commonly the
address pointing to an I/O memory region would be a "void __iomem"
pointer.

Related:
  https://lwn.net/Articles/102232/

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
---
 drivers/pci/controller/vmd.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 1361a79bd1e7..59fa9a94860f 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -95,7 +95,7 @@ struct vmd_dev {
 	struct pci_dev		*dev;
 
 	spinlock_t		cfg_lock;
-	char __iomem		*cfgbar;
+	void __iomem		*cfgbar;
 
 	int msix_count;
 	struct vmd_irq_list	*irqs;
@@ -326,7 +326,7 @@ static void vmd_remove_irq_domain(struct vmd_dev *vmd)
 	}
 }
 
-static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
+static void __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
 				  unsigned int devfn, int reg, int len)
 {
 	unsigned int busnr_ecam = bus->number - vmd->busn_start;
@@ -346,7 +346,7 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
 			int len, u32 *value)
 {
 	struct vmd_dev *vmd = vmd_from_bus(bus);
-	char __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
+	void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
 	unsigned long flags;
 	int ret = 0;
 
@@ -381,7 +381,7 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
 			 int len, u32 value)
 {
 	struct vmd_dev *vmd = vmd_from_bus(bus);
-	char __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
+	void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
 	unsigned long flags;
 	int ret = 0;
 
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [PATCH v6 5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
                   ` (3 preceding siblings ...)
  2020-11-29 23:07 ` [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers Krzysztof Wilczyński
@ 2020-11-29 23:07 ` Krzysztof Wilczyński
  2020-12-01 15:34 ` [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Lorenzo Pieralisi
  2020-12-08 16:07 ` Qian Cai
  6 siblings, 0 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-29 23:07 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Removed unused ".bus_shift" initialisers from pci-xgene.c as
xgene_pcie_map_bus() did not use these.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
---
 drivers/pci/controller/pci-xgene.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/pci/controller/pci-xgene.c b/drivers/pci/controller/pci-xgene.c
index 8e0db84f089d..85e7c98265e8 100644
--- a/drivers/pci/controller/pci-xgene.c
+++ b/drivers/pci/controller/pci-xgene.c
@@ -257,7 +257,6 @@ static int xgene_v1_pcie_ecam_init(struct pci_config_window *cfg)
 }
 
 const struct pci_ecam_ops xgene_v1_pcie_ecam_ops = {
-	.bus_shift	= 16,
 	.init		= xgene_v1_pcie_ecam_init,
 	.pci_ops	= {
 		.map_bus	= xgene_pcie_map_bus,
@@ -272,7 +271,6 @@ static int xgene_v2_pcie_ecam_init(struct pci_config_window *cfg)
 }
 
 const struct pci_ecam_ops xgene_v2_pcie_ecam_ops = {
-	.bus_shift	= 16,
 	.init		= xgene_v2_pcie_ecam_init,
 	.pci_ops	= {
 		.map_bus	= xgene_pcie_map_bus,
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 32+ messages in thread

* RE: [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers
  2020-11-29 23:07 ` [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers Krzysztof Wilczyński
@ 2020-11-30  9:06   ` David Laight
  2020-11-30 17:20     ` Bjorn Helgaas
  0 siblings, 1 reply; 32+ messages in thread
From: David Laight @ 2020-11-30  9:06 UTC (permalink / raw)
  To: 'Krzysztof Wilczyński', Bjorn Helgaas
  Cc: Heiko Stuebner, Shawn Lin, Paul Mackerras, Thomas Petazzoni,
	Jonathan Chocron, Toan Le, Will Deacon, Rob Herring,
	Lorenzo Pieralisi, Michal Simek, linux-rockchip,
	bcm-kernel-feedback-list, Jonathan Derrick, linux-pci, Ray Jui,
	Florian Fainelli, linux-rpi-kernel, Jonathan Cameron,
	linux-arm-kernel, Scott Branden, Zhou Wang, Robert Richter,
	linuxppc-dev, Nicolas Saenz Julienne

From: Krzysztof Wilczynski
> Sent: 29 November 2020 23:08
> 
> Use "void __iomem" instead "char __iomem" pointer type when working with
> the accessor functions (with names like readb() or writel(), etc.) to
> better match a given accessor function signature where commonly the
> address pointing to an I/O memory region would be a "void __iomem"
> pointer.

ISTM that is heading in the wrong direction.

I think (form the variable names etc) that these are pointers
to specific registers.

So what you ought to have is a type for that register block.
Typically this is actually a structure - to give some type
checking that the offsets are being used with the correct
base address.

If the code is using numeric offsets (hardware engineers like
numeric offsets) then you can get some type protection by using
a structure that only contains a single field (char in this case).

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
@ 2020-11-30 11:08   ` Lorenzo Pieralisi
  2020-11-30 15:30     ` Krzysztof Wilczyński
  2020-11-30 18:23   ` Derrick, Jonathan
  2020-12-06 20:16   ` Krzysztof Wilczyński
  2 siblings, 1 reply; 32+ messages in thread
From: Lorenzo Pieralisi @ 2020-11-30 11:08 UTC (permalink / raw)
  To: Krzysztof Wilczyński
  Cc: Bjorn Helgaas, Rob Herring, Jonathan Cameron, Jonathan Chocron,
	Shawn Lin, Heiko Stuebner, Zhou Wang, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

On Sun, Nov 29, 2020 at 11:07:39PM +0000, Krzysztof Wilczyński wrote:
> Add ECAM-related constants to provide a set of standard constants
> defining memory address shift values to the byte-level address that can
> be used to access the PCI Express Configuration Space, and then move
> native PCI Express controller drivers to use the newly introduced
> definitions retiring driver-specific ones.
> 
> Refactor pci_ecam_map_bus() function to use newly added constants so
> that limits to the bus, device function and offset (now limited to 4K as
> per the specification) are in place to prevent the defective or
> malicious caller from supplying incorrect configuration offset and thus
> targeting the wrong device when accessing extended configuration space.
> This refactor also allows for the ".bus_shit" initialisers to be dropped
                                          ^^^^

Nice typo, I'd fix it while applying it though if you don't mind ;-),
no need to resend it.

Jokes aside, nice piece of work, thanks for that.

> when the user is not using a custom value as a default value will be
> used as per the PCI Express Specification.
> 
> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Krzysztof Wilczyński <kw@linux.com>

I think Bjorn's reviewed-by still stands so I will apply it.

Thanks !
Lorenzo

> ---
>  drivers/pci/controller/dwc/pcie-al.c        | 12 ++-------
>  drivers/pci/controller/dwc/pcie-hisi.c      |  2 --
>  drivers/pci/controller/pci-aardvark.c       | 13 +++-------
>  drivers/pci/controller/pci-host-generic.c   |  1 -
>  drivers/pci/controller/pci-thunder-ecam.c   |  1 -
>  drivers/pci/controller/pcie-brcmstb.c       | 16 ++----------
>  drivers/pci/controller/pcie-rockchip-host.c | 27 ++++++++++-----------
>  drivers/pci/controller/pcie-rockchip.h      |  8 +-----
>  drivers/pci/controller/pcie-tango.c         |  1 -
>  drivers/pci/controller/pcie-xilinx-nwl.c    |  9 ++-----
>  drivers/pci/controller/pcie-xilinx.c        | 11 ++-------
>  drivers/pci/controller/vmd.c                | 11 ++++-----
>  drivers/pci/ecam.c                          | 23 ++++++++++++------
>  include/linux/pci-ecam.h                    | 27 +++++++++++++++++++++
>  14 files changed, 73 insertions(+), 89 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-al.c b/drivers/pci/controller/dwc/pcie-al.c
> index f973fbca90cf..af9e51ab1af8 100644
> --- a/drivers/pci/controller/dwc/pcie-al.c
> +++ b/drivers/pci/controller/dwc/pcie-al.c
> @@ -76,7 +76,6 @@ static int al_pcie_init(struct pci_config_window *cfg)
>  }
>  
>  const struct pci_ecam_ops al_pcie_ops = {
> -	.bus_shift    = 20,
>  	.init         =  al_pcie_init,
>  	.pci_ops      = {
>  		.map_bus    = al_pcie_map_bus,
> @@ -138,8 +137,6 @@ struct al_pcie {
>  	struct al_pcie_target_bus_cfg target_bus_cfg;
>  };
>  
> -#define PCIE_ECAM_DEVFN(x)		(((x) & 0xff) << 12)
> -
>  #define to_al_pcie(x)		dev_get_drvdata((x)->dev)
>  
>  static inline u32 al_pcie_controller_readl(struct al_pcie *pcie, u32 offset)
> @@ -226,11 +223,6 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
>  	struct al_pcie_target_bus_cfg *target_bus_cfg = &pcie->target_bus_cfg;
>  	unsigned int busnr_ecam = busnr & target_bus_cfg->ecam_mask;
>  	unsigned int busnr_reg = busnr & target_bus_cfg->reg_mask;
> -	void __iomem *pci_base_addr;
> -
> -	pci_base_addr = (void __iomem *)((uintptr_t)pp->va_cfg0_base +
> -					 (busnr_ecam << 20) +
> -					 PCIE_ECAM_DEVFN(devfn));
>  
>  	if (busnr_reg != target_bus_cfg->reg_val) {
>  		dev_dbg(pcie->pci->dev, "Changing target bus busnum val from 0x%x to 0x%x\n",
> @@ -241,7 +233,7 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
>  				       target_bus_cfg->reg_mask);
>  	}
>  
> -	return pci_base_addr + where;
> +	return pp->va_cfg0_base + PCIE_ECAM_OFFSET(busnr_ecam, devfn, where);
>  }
>  
>  static struct pci_ops al_child_pci_ops = {
> @@ -264,7 +256,7 @@ static void al_pcie_config_prepare(struct al_pcie *pcie)
>  
>  	target_bus_cfg = &pcie->target_bus_cfg;
>  
> -	ecam_bus_mask = (pcie->ecam_size >> 20) - 1;
> +	ecam_bus_mask = (pcie->ecam_size >> PCIE_ECAM_BUS_SHIFT) - 1;
>  	if (ecam_bus_mask > 255) {
>  		dev_warn(pcie->dev, "ECAM window size is larger than 256MB. Cutting off at 256\n");
>  		ecam_bus_mask = 255;
> diff --git a/drivers/pci/controller/dwc/pcie-hisi.c b/drivers/pci/controller/dwc/pcie-hisi.c
> index 5ca86796d43a..8fc5960faf28 100644
> --- a/drivers/pci/controller/dwc/pcie-hisi.c
> +++ b/drivers/pci/controller/dwc/pcie-hisi.c
> @@ -100,7 +100,6 @@ static int hisi_pcie_init(struct pci_config_window *cfg)
>  }
>  
>  const struct pci_ecam_ops hisi_pcie_ops = {
> -	.bus_shift    = 20,
>  	.init         =  hisi_pcie_init,
>  	.pci_ops      = {
>  		.map_bus    = hisi_pcie_map_bus,
> @@ -135,7 +134,6 @@ static int hisi_pcie_platform_init(struct pci_config_window *cfg)
>  }
>  
>  static const struct pci_ecam_ops hisi_pcie_platform_ops = {
> -	.bus_shift    = 20,
>  	.init         =  hisi_pcie_platform_init,
>  	.pci_ops      = {
>  		.map_bus    = hisi_pcie_map_bus,
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index 0be485a25327..1043e54c73bd 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -16,6 +16,7 @@
>  #include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/init.h>
>  #include <linux/phy/phy.h>
>  #include <linux/platform_device.h>
> @@ -164,14 +165,6 @@
>  #define PCIE_CONFIG_WR_TYPE0			0xa
>  #define PCIE_CONFIG_WR_TYPE1			0xb
>  
> -#define PCIE_CONF_BUS(bus)			(((bus) & 0xff) << 20)
> -#define PCIE_CONF_DEV(dev)			(((dev) & 0x1f) << 15)
> -#define PCIE_CONF_FUNC(fun)			(((fun) & 0x7)	<< 12)
> -#define PCIE_CONF_REG(reg)			((reg) & 0xffc)
> -#define PCIE_CONF_ADDR(bus, devfn, where)	\
> -	(PCIE_CONF_BUS(bus) | PCIE_CONF_DEV(PCI_SLOT(devfn))	| \
> -	 PCIE_CONF_FUNC(PCI_FUNC(devfn)) | PCIE_CONF_REG(where))
> -
>  #define PIO_RETRY_CNT			500
>  #define PIO_RETRY_DELAY			2 /* 2 us*/
>  
> @@ -687,7 +680,7 @@ static int advk_pcie_rd_conf(struct pci_bus *bus, u32 devfn,
>  	advk_writel(pcie, reg, PIO_CTRL);
>  
>  	/* Program the address registers */
> -	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
> +	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
>  	advk_writel(pcie, reg, PIO_ADDR_LS);
>  	advk_writel(pcie, 0, PIO_ADDR_MS);
>  
> @@ -748,7 +741,7 @@ static int advk_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
>  	advk_writel(pcie, reg, PIO_CTRL);
>  
>  	/* Program the address registers */
> -	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
> +	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
>  	advk_writel(pcie, reg, PIO_ADDR_LS);
>  	advk_writel(pcie, 0, PIO_ADDR_MS);
>  
> diff --git a/drivers/pci/controller/pci-host-generic.c b/drivers/pci/controller/pci-host-generic.c
> index b51977abfdf1..63865aeb636b 100644
> --- a/drivers/pci/controller/pci-host-generic.c
> +++ b/drivers/pci/controller/pci-host-generic.c
> @@ -49,7 +49,6 @@ static void __iomem *pci_dw_ecam_map_bus(struct pci_bus *bus,
>  }
>  
>  static const struct pci_ecam_ops pci_dw_ecam_bus_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_dw_ecam_map_bus,
>  		.read		= pci_generic_config_read,
> diff --git a/drivers/pci/controller/pci-thunder-ecam.c b/drivers/pci/controller/pci-thunder-ecam.c
> index 7e8835fee5f7..f964fd26f7e0 100644
> --- a/drivers/pci/controller/pci-thunder-ecam.c
> +++ b/drivers/pci/controller/pci-thunder-ecam.c
> @@ -346,7 +346,6 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
>  }
>  
>  const struct pci_ecam_ops pci_thunder_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus        = pci_ecam_map_bus,
>  		.read           = thunder_ecam_config_read,
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index bea86899bd5d..7fc80fd6f13f 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -22,6 +22,7 @@
>  #include <linux/of_pci.h>
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/printk.h>
>  #include <linux/reset.h>
>  #include <linux/sizes.h>
> @@ -127,11 +128,7 @@
>  #define  MSI_INT_MASK_CLR		0x14
>  
>  #define PCIE_EXT_CFG_DATA				0x8000
> -
>  #define PCIE_EXT_CFG_INDEX				0x9000
> -#define  PCIE_EXT_BUSNUM_SHIFT				20
> -#define  PCIE_EXT_SLOT_SHIFT				15
> -#define  PCIE_EXT_FUNC_SHIFT				12
>  
>  #define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
>  #define  PCIE_RGR1_SW_INIT_1_PERST_SHIFT		0x0
> @@ -695,15 +692,6 @@ static bool brcm_pcie_link_up(struct brcm_pcie *pcie)
>  	return dla && plu;
>  }
>  
> -/* Configuration space read/write support */
> -static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
> -{
> -	return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
> -		| ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
> -		| (busnr << PCIE_EXT_BUSNUM_SHIFT)
> -		| (reg & ~3);
> -}
> -
>  static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  					int where)
>  {
> @@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  		return PCI_SLOT(devfn) ? NULL : base + where;
>  
>  	/* For devices, write to the config space index register */
> -	idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
> +	idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
>  	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
>  	return base + PCIE_EXT_CFG_DATA + where;
>  }
> diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
> index 9705059523a6..f1d08a1b1591 100644
> --- a/drivers/pci/controller/pcie-rockchip-host.c
> +++ b/drivers/pci/controller/pcie-rockchip-host.c
> @@ -157,12 +157,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  				       struct pci_bus *bus, u32 devfn,
>  				       int where, int size, u32 *val)
>  {
> -	u32 busdev;
> +	void __iomem *addr;
>  
> -	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
> -				PCI_FUNC(devfn), where);
> +	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  
> -	if (!IS_ALIGNED(busdev, size)) {
> +	if (!IS_ALIGNED((uintptr_t)addr, size)) {
>  		*val = 0;
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  	}
> @@ -175,11 +174,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  						AXI_WRAPPER_TYPE1_CFG);
>  
>  	if (size == 4) {
> -		*val = readl(rockchip->reg_base + busdev);
> +		*val = readl(addr);
>  	} else if (size == 2) {
> -		*val = readw(rockchip->reg_base + busdev);
> +		*val = readw(addr);
>  	} else if (size == 1) {
> -		*val = readb(rockchip->reg_base + busdev);
> +		*val = readb(addr);
>  	} else {
>  		*val = 0;
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
> @@ -191,11 +190,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
>  				       struct pci_bus *bus, u32 devfn,
>  				       int where, int size, u32 val)
>  {
> -	u32 busdev;
> +	void __iomem *addr;
>  
> -	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
> -				PCI_FUNC(devfn), where);
> -	if (!IS_ALIGNED(busdev, size))
> +	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
> +
> +	if (!IS_ALIGNED((uintptr_t)addr, size))
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  
>  	if (pci_is_root_bus(bus->parent))
> @@ -206,11 +205,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
>  						AXI_WRAPPER_TYPE1_CFG);
>  
>  	if (size == 4)
> -		writel(val, rockchip->reg_base + busdev);
> +		writel(val, addr);
>  	else if (size == 2)
> -		writew(val, rockchip->reg_base + busdev);
> +		writew(val, addr);
>  	else if (size == 1)
> -		writeb(val, rockchip->reg_base + busdev);
> +		writeb(val, addr);
>  	else
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  
> diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
> index c7d0178fc8c2..1650a5087450 100644
> --- a/drivers/pci/controller/pcie-rockchip.h
> +++ b/drivers/pci/controller/pcie-rockchip.h
> @@ -13,6 +13,7 @@
>  
>  #include <linux/kernel.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  
>  /*
>   * The upper 16 bits of PCIE_CLIENT_CONFIG are a write mask for the lower 16
> @@ -178,13 +179,6 @@
>  #define MIN_AXI_ADDR_BITS_PASSED		8
>  #define PCIE_RC_SEND_PME_OFF			0x11960
>  #define ROCKCHIP_VENDOR_ID			0x1d87
> -#define PCIE_ECAM_BUS(x)			(((x) & 0xff) << 20)
> -#define PCIE_ECAM_DEV(x)			(((x) & 0x1f) << 15)
> -#define PCIE_ECAM_FUNC(x)			(((x) & 0x7) << 12)
> -#define PCIE_ECAM_REG(x)			(((x) & 0xfff) << 0)
> -#define PCIE_ECAM_ADDR(bus, dev, func, reg) \
> -	  (PCIE_ECAM_BUS(bus) | PCIE_ECAM_DEV(dev) | \
> -	   PCIE_ECAM_FUNC(func) | PCIE_ECAM_REG(reg))
>  #define PCIE_LINK_IS_L2(x) \
>  	(((x) & PCIE_CLIENT_DEBUG_LTSSM_MASK) == PCIE_CLIENT_DEBUG_LTSSM_L2)
>  #define PCIE_LINK_UP(x) \
> diff --git a/drivers/pci/controller/pcie-tango.c b/drivers/pci/controller/pcie-tango.c
> index d093a8ce4bb1..62a061f1d62e 100644
> --- a/drivers/pci/controller/pcie-tango.c
> +++ b/drivers/pci/controller/pcie-tango.c
> @@ -208,7 +208,6 @@ static int smp8759_config_write(struct pci_bus *bus, unsigned int devfn,
>  }
>  
>  static const struct pci_ecam_ops smp8759_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= smp8759_config_read,
> diff --git a/drivers/pci/controller/pcie-xilinx-nwl.c b/drivers/pci/controller/pcie-xilinx-nwl.c
> index f3cf7d61924f..7f29c2fdcd51 100644
> --- a/drivers/pci/controller/pcie-xilinx-nwl.c
> +++ b/drivers/pci/controller/pcie-xilinx-nwl.c
> @@ -18,6 +18,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/of_irq.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  #include <linux/irqchip/chained_irq.h>
>  
> @@ -124,8 +125,6 @@
>  #define E_ECAM_CR_ENABLE		BIT(0)
>  #define E_ECAM_SIZE_LOC			GENMASK(20, 16)
>  #define E_ECAM_SIZE_SHIFT		16
> -#define ECAM_BUS_LOC_SHIFT		20
> -#define ECAM_DEV_LOC_SHIFT		12
>  #define NWL_ECAM_VALUE_DEFAULT		12
>  
>  #define CFG_DMA_REG_BAR			GENMASK(2, 0)
> @@ -240,15 +239,11 @@ static void __iomem *nwl_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
>  				      int where)
>  {
>  	struct nwl_pcie *pcie = bus->sysdata;
> -	int relbus;
>  
>  	if (!nwl_pcie_valid_device(bus, devfn))
>  		return NULL;
>  
> -	relbus = (bus->number << ECAM_BUS_LOC_SHIFT) |
> -			(devfn << ECAM_DEV_LOC_SHIFT);
> -
> -	return pcie->ecam_base + relbus + where;
> +	return pcie->ecam_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  }
>  
>  /* PCIe operations */
> diff --git a/drivers/pci/controller/pcie-xilinx.c b/drivers/pci/controller/pcie-xilinx.c
> index 8523be61bba5..fa5baeb82653 100644
> --- a/drivers/pci/controller/pcie-xilinx.c
> +++ b/drivers/pci/controller/pcie-xilinx.c
> @@ -21,6 +21,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/of_irq.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
>  #include "../pci.h"
> @@ -86,10 +87,6 @@
>  /* Phy Status/Control Register definitions */
>  #define XILINX_PCIE_REG_PSCR_LNKUP	BIT(11)
>  
> -/* ECAM definitions */
> -#define ECAM_BUS_NUM_SHIFT		20
> -#define ECAM_DEV_NUM_SHIFT		12
> -
>  /* Number of MSI IRQs */
>  #define XILINX_NUM_MSI_IRQS		128
>  
> @@ -183,15 +180,11 @@ static void __iomem *xilinx_pcie_map_bus(struct pci_bus *bus,
>  					 unsigned int devfn, int where)
>  {
>  	struct xilinx_pcie_port *port = bus->sysdata;
> -	int relbus;
>  
>  	if (!xilinx_pcie_valid_device(bus, devfn))
>  		return NULL;
>  
> -	relbus = (bus->number << ECAM_BUS_NUM_SHIFT) |
> -		 (devfn << ECAM_DEV_NUM_SHIFT);
> -
> -	return port->reg_base + relbus + where;
> +	return port->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  }
>  
>  /* PCIe operations */
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index f375c21ceeb1..1361a79bd1e7 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -11,6 +11,7 @@
>  #include <linux/module.h>
>  #include <linux/msi.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/srcu.h>
>  #include <linux/rculist.h>
>  #include <linux/rcupdate.h>
> @@ -328,15 +329,13 @@ static void vmd_remove_irq_domain(struct vmd_dev *vmd)
>  static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
>  				  unsigned int devfn, int reg, int len)
>  {
> -	char __iomem *addr = vmd->cfgbar +
> -			     ((bus->number - vmd->busn_start) << 20) +
> -			     (devfn << 12) + reg;
> +	unsigned int busnr_ecam = bus->number - vmd->busn_start;
> +	u32 offset = PCIE_ECAM_OFFSET(busnr_ecam, devfn, reg);
>  
> -	if ((addr - vmd->cfgbar) + len >=
> -	    resource_size(&vmd->dev->resource[VMD_CFGBAR]))
> +	if (offset + len >= resource_size(&vmd->dev->resource[VMD_CFGBAR]))
>  		return NULL;
>  
> -	return addr;
> +	return vmd->cfgbar + offset;
>  }
>  
>  /*
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index b54d32a31669..59f91d434859 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -131,25 +131,36 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>  			       int where)
>  {
>  	struct pci_config_window *cfg = bus->sysdata;
> +	unsigned int bus_shift = cfg->ops->bus_shift;
>  	unsigned int devfn_shift = cfg->ops->bus_shift - 8;
>  	unsigned int busn = bus->number;
>  	void __iomem *base;
> +	u32 bus_offset, devfn_offset;
>  
>  	if (busn < cfg->busr.start || busn > cfg->busr.end)
>  		return NULL;
>  
>  	busn -= cfg->busr.start;
> -	if (per_bus_mapping)
> +	if (per_bus_mapping) {
>  		base = cfg->winp[busn];
> -	else
> -		base = cfg->win + (busn << cfg->ops->bus_shift);
> -	return base + (devfn << devfn_shift) + where;
> +		busn = 0;
> +	} else
> +		base = cfg->win;
> +
> +	if (cfg->ops->bus_shift) {
> +		bus_offset = (busn & PCIE_ECAM_BUS_MASK) << bus_shift;
> +		devfn_offset = (devfn & PCIE_ECAM_DEVFN_MASK) << devfn_shift;
> +		where &= PCIE_ECAM_REG_MASK;
> +
> +		return base + (bus_offset | devfn_offset | where);
> +	}
> +
> +	return base + PCIE_ECAM_OFFSET(busn, devfn, where);
>  }
>  EXPORT_SYMBOL_GPL(pci_ecam_map_bus);
>  
>  /* ECAM ops */
>  const struct pci_ecam_ops pci_generic_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read,
> @@ -161,7 +172,6 @@ EXPORT_SYMBOL_GPL(pci_generic_ecam_ops);
>  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
>  /* ECAM ops for 32-bit access only (non-compliant) */
>  const struct pci_ecam_ops pci_32b_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read32,
> @@ -171,7 +181,6 @@ const struct pci_ecam_ops pci_32b_ops = {
>  
>  /* ECAM ops for 32-bit read only (non-compliant) */
>  const struct pci_ecam_ops pci_32b_read_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read32,
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> index 033ce74f02e8..65d3d83015c3 100644
> --- a/include/linux/pci-ecam.h
> +++ b/include/linux/pci-ecam.h
> @@ -9,6 +9,33 @@
>  #include <linux/kernel.h>
>  #include <linux/platform_device.h>
>  
> +/*
> + * Memory address shift values for the byte-level address that
> + * can be used when accessing the PCI Express Configuration Space.
> + */
> +
> +/*
> + * Enhanced Configuration Access Mechanism (ECAM)
> + *
> + * See PCI Express Base Specification, Revision 5.0, Version 1.0,
> + * Section 7.2.2, Table 7-1, p. 677.
> + */
> +#define PCIE_ECAM_BUS_SHIFT	20 /* Bus number */
> +#define PCIE_ECAM_DEVFN_SHIFT	12 /* Device and Function number */
> +
> +#define PCIE_ECAM_BUS_MASK	0xff
> +#define PCIE_ECAM_DEVFN_MASK	0xff
> +#define PCIE_ECAM_REG_MASK	0xfff /* Limit offset to a maximum of 4K */
> +
> +#define PCIE_ECAM_BUS(x)	(((x) & PCIE_ECAM_BUS_MASK) << PCIE_ECAM_BUS_SHIFT)
> +#define PCIE_ECAM_DEVFN(x)	(((x) & PCIE_ECAM_DEVFN_MASK) << PCIE_ECAM_DEVFN_SHIFT)
> +#define PCIE_ECAM_REG(x)	((x) & PCIE_ECAM_REG_MASK)
> +
> +#define PCIE_ECAM_OFFSET(bus, devfn, where) \
> +	(PCIE_ECAM_BUS(bus) | \
> +	 PCIE_ECAM_DEVFN(devfn) | \
> +	 PCIE_ECAM_REG(where))
> +
>  /*
>   * struct to hold pci ops and bus shift of the config window
>   * for a PCI controller.
> -- 
> 2.29.2
> 

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-30 11:08   ` Lorenzo Pieralisi
@ 2020-11-30 15:30     ` Krzysztof Wilczyński
  0 siblings, 0 replies; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-11-30 15:30 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Bjorn Helgaas, Rob Herring, Jonathan Cameron, Jonathan Chocron,
	Shawn Lin, Heiko Stuebner, Zhou Wang, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Hi Lorenzo!

On 20-11-30 11:08:58, Lorenzo Pieralisi wrote:
[...]
> > Refactor pci_ecam_map_bus() function to use newly added constants so
> > that limits to the bus, device function and offset (now limited to 4K as
> > per the specification) are in place to prevent the defective or
> > malicious caller from supplying incorrect configuration offset and thus
> > targeting the wrong device when accessing extended configuration space.
> > This refactor also allows for the ".bus_shit" initialisers to be dropped
>                                           ^^^^
> 
> Nice typo, I'd fix it while applying it though if you don't mind ;-),
> no need to resend it.

Oh doh!  Apologies. :)

> Jokes aside, nice piece of work, thanks for that.
> 
> > when the user is not using a custom value as a default value will be
> > used as per the PCI Express Specification.
> > 
> > Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> > Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
> 
> I think Bjorn's reviewed-by still stands so I will apply it.
[...]

Thank you!

Krzysztof

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers
  2020-11-30  9:06   ` David Laight
@ 2020-11-30 17:20     ` Bjorn Helgaas
  2020-11-30 18:19       ` Derrick, Jonathan
  0 siblings, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-11-30 17:20 UTC (permalink / raw)
  To: David Laight
  Cc: 'Krzysztof Wilczyński',
	Bjorn Helgaas, Heiko Stuebner, Shawn Lin, Paul Mackerras,
	Thomas Petazzoni, Jonathan Chocron, Toan Le, Will Deacon,
	Rob Herring, Lorenzo Pieralisi, Michal Simek, linux-rockchip,
	bcm-kernel-feedback-list, Jonathan Derrick, linux-pci, Ray Jui,
	Florian Fainelli, linux-rpi-kernel, Jonathan Cameron,
	linux-arm-kernel, Scott Branden, Zhou Wang, Robert Richter,
	linuxppc-dev, Nicolas Saenz Julienne

On Mon, Nov 30, 2020 at 09:06:56AM +0000, David Laight wrote:
> From: Krzysztof Wilczynski
> > Sent: 29 November 2020 23:08
> > 
> > Use "void __iomem" instead "char __iomem" pointer type when working with
> > the accessor functions (with names like readb() or writel(), etc.) to
> > better match a given accessor function signature where commonly the
> > address pointing to an I/O memory region would be a "void __iomem"
> > pointer.
> 
> ISTM that is heading in the wrong direction.
> 
> I think (form the variable names etc) that these are pointers
> to specific registers.
> 
> So what you ought to have is a type for that register block.
> Typically this is actually a structure - to give some type
> checking that the offsets are being used with the correct
> base address.

In this case, "cfgbar" is not really a pointer to a register; it's the
address of memory-mapped config space.  The VMD hardware turns
accesses to that space into PCI config transactions on its secondary
side.  xgene_pcie_get_cfg_base() and brcm_pcie_map_conf() are similar
situations and use "void *".

Bjorn

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers
  2020-11-30 17:20     ` Bjorn Helgaas
@ 2020-11-30 18:19       ` Derrick, Jonathan
  0 siblings, 0 replies; 32+ messages in thread
From: Derrick, Jonathan @ 2020-11-30 18:19 UTC (permalink / raw)
  To: helgaas, David.Laight
  Cc: heiko, f.fainelli, kw, robh, linuxppc-dev, thomas.petazzoni,
	toan, rjui, Jonathan.Cameron, jonnyc, bcm-kernel-feedback-list,
	sbranden, nsaenzjulienne, wangzhou1, rrichter, will, shawn.lin,
	linux-rpi-kernel, linux-arm-kernel, lorenzo.pieralisi, linux-pci,
	linux-rockchip, paulus, bhelgaas, michal.simek

On Mon, 2020-11-30 at 11:20 -0600, Bjorn Helgaas wrote:
> On Mon, Nov 30, 2020 at 09:06:56AM +0000, David Laight wrote:
> > From: Krzysztof Wilczynski
> > > Sent: 29 November 2020 23:08
> > > 
> > > Use "void __iomem" instead "char __iomem" pointer type when working with
> > > the accessor functions (with names like readb() or writel(), etc.) to
> > > better match a given accessor function signature where commonly the
> > > address pointing to an I/O memory region would be a "void __iomem"
> > > pointer.
> > 
> > ISTM that is heading in the wrong direction.
> > 
> > I think (form the variable names etc) that these are pointers
> > to specific registers.
> > 
> > So what you ought to have is a type for that register block.
> > Typically this is actually a structure - to give some type
> > checking that the offsets are being used with the correct
> > base address.
> 
> In this case, "cfgbar" is not really a pointer to a register; it's the
> address of memory-mapped config space.  The VMD hardware turns
> accesses to that space into PCI config transactions on its secondary
> side.  xgene_pcie_get_cfg_base() and brcm_pcie_map_conf() are similar
> situations and use "void *".
> 
> Bjorn

Yes it's just the passthrough window for PCI config bus ops.

Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
  2020-11-30 11:08   ` Lorenzo Pieralisi
@ 2020-11-30 18:23   ` Derrick, Jonathan
  2020-12-06 20:16   ` Krzysztof Wilczyński
  2 siblings, 0 replies; 32+ messages in thread
From: Derrick, Jonathan @ 2020-11-30 18:23 UTC (permalink / raw)
  To: kw, bhelgaas
  Cc: heiko, linux-rockchip, linuxppc-dev, robh, linux-rpi-kernel,
	toan, mpe, jonnyc, wangzhou1, Jonathan.Cameron, rjui,
	thomas.petazzoni, nsaenzjulienne, bcm-kernel-feedback-list,
	rrichter, will, shawn.lin, sbranden, linux-arm-kernel,
	lorenzo.pieralisi, linux-pci, michal.simek, benh, paulus,
	f.fainelli

On Sun, 2020-11-29 at 23:07 +0000, Krzysztof Wilczyński wrote:
> Add ECAM-related constants to provide a set of standard constants
> defining memory address shift values to the byte-level address that can
> be used to access the PCI Express Configuration Space, and then move
> native PCI Express controller drivers to use the newly introduced
> definitions retiring driver-specific ones.
> 
> Refactor pci_ecam_map_bus() function to use newly added constants so
> that limits to the bus, device function and offset (now limited to 4K as
> per the specification) are in place to prevent the defective or
> malicious caller from supplying incorrect configuration offset and thus
> targeting the wrong device when accessing extended configuration space.
> This refactor also allows for the ".bus_shit" initialisers to be dropped
> when the user is not using a custom value as a default value will be
> used as per the PCI Express Specification.
> 
> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
> ---
>  drivers/pci/controller/dwc/pcie-al.c        | 12 ++-------
>  drivers/pci/controller/dwc/pcie-hisi.c      |  2 --
>  drivers/pci/controller/pci-aardvark.c       | 13 +++-------
>  drivers/pci/controller/pci-host-generic.c   |  1 -
>  drivers/pci/controller/pci-thunder-ecam.c   |  1 -
>  drivers/pci/controller/pcie-brcmstb.c       | 16 ++----------
>  drivers/pci/controller/pcie-rockchip-host.c | 27 ++++++++++-----------
>  drivers/pci/controller/pcie-rockchip.h      |  8 +-----
>  drivers/pci/controller/pcie-tango.c         |  1 -
>  drivers/pci/controller/pcie-xilinx-nwl.c    |  9 ++-----
>  drivers/pci/controller/pcie-xilinx.c        | 11 ++-------
>  drivers/pci/controller/vmd.c                | 11 ++++-----
>  drivers/pci/ecam.c                          | 23 ++++++++++++------
>  include/linux/pci-ecam.h                    | 27 +++++++++++++++++++++
>  14 files changed, 73 insertions(+), 89 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-al.c b/drivers/pci/controller/dwc/pcie-al.c
> index f973fbca90cf..af9e51ab1af8 100644
> --- a/drivers/pci/controller/dwc/pcie-al.c
> +++ b/drivers/pci/controller/dwc/pcie-al.c
> @@ -76,7 +76,6 @@ static int al_pcie_init(struct pci_config_window *cfg)
>  }
>  
>  const struct pci_ecam_ops al_pcie_ops = {
> -	.bus_shift    = 20,
>  	.init         =  al_pcie_init,
>  	.pci_ops      = {
>  		.map_bus    = al_pcie_map_bus,
> @@ -138,8 +137,6 @@ struct al_pcie {
>  	struct al_pcie_target_bus_cfg target_bus_cfg;
>  };
>  
> -#define PCIE_ECAM_DEVFN(x)		(((x) & 0xff) << 12)
> -
>  #define to_al_pcie(x)		dev_get_drvdata((x)->dev)
>  
>  static inline u32 al_pcie_controller_readl(struct al_pcie *pcie, u32 offset)
> @@ -226,11 +223,6 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
>  	struct al_pcie_target_bus_cfg *target_bus_cfg = &pcie->target_bus_cfg;
>  	unsigned int busnr_ecam = busnr & target_bus_cfg->ecam_mask;
>  	unsigned int busnr_reg = busnr & target_bus_cfg->reg_mask;
> -	void __iomem *pci_base_addr;
> -
> -	pci_base_addr = (void __iomem *)((uintptr_t)pp->va_cfg0_base +
> -					 (busnr_ecam << 20) +
> -					 PCIE_ECAM_DEVFN(devfn));
>  
>  	if (busnr_reg != target_bus_cfg->reg_val) {
>  		dev_dbg(pcie->pci->dev, "Changing target bus busnum val from 0x%x to 0x%x\n",
> @@ -241,7 +233,7 @@ static void __iomem *al_pcie_conf_addr_map_bus(struct pci_bus *bus,
>  				       target_bus_cfg->reg_mask);
>  	}
>  
> -	return pci_base_addr + where;
> +	return pp->va_cfg0_base + PCIE_ECAM_OFFSET(busnr_ecam, devfn, where);
>  }
>  
>  static struct pci_ops al_child_pci_ops = {
> @@ -264,7 +256,7 @@ static void al_pcie_config_prepare(struct al_pcie *pcie)
>  
>  	target_bus_cfg = &pcie->target_bus_cfg;
>  
> -	ecam_bus_mask = (pcie->ecam_size >> 20) - 1;
> +	ecam_bus_mask = (pcie->ecam_size >> PCIE_ECAM_BUS_SHIFT) - 1;
>  	if (ecam_bus_mask > 255) {
>  		dev_warn(pcie->dev, "ECAM window size is larger than 256MB. Cutting off at 256\n");
>  		ecam_bus_mask = 255;
> diff --git a/drivers/pci/controller/dwc/pcie-hisi.c b/drivers/pci/controller/dwc/pcie-hisi.c
> index 5ca86796d43a..8fc5960faf28 100644
> --- a/drivers/pci/controller/dwc/pcie-hisi.c
> +++ b/drivers/pci/controller/dwc/pcie-hisi.c
> @@ -100,7 +100,6 @@ static int hisi_pcie_init(struct pci_config_window *cfg)
>  }
>  
>  const struct pci_ecam_ops hisi_pcie_ops = {
> -	.bus_shift    = 20,
>  	.init         =  hisi_pcie_init,
>  	.pci_ops      = {
>  		.map_bus    = hisi_pcie_map_bus,
> @@ -135,7 +134,6 @@ static int hisi_pcie_platform_init(struct pci_config_window *cfg)
>  }
>  
>  static const struct pci_ecam_ops hisi_pcie_platform_ops = {
> -	.bus_shift    = 20,
>  	.init         =  hisi_pcie_platform_init,
>  	.pci_ops      = {
>  		.map_bus    = hisi_pcie_map_bus,
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index 0be485a25327..1043e54c73bd 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -16,6 +16,7 @@
>  #include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/init.h>
>  #include <linux/phy/phy.h>
>  #include <linux/platform_device.h>
> @@ -164,14 +165,6 @@
>  #define PCIE_CONFIG_WR_TYPE0			0xa
>  #define PCIE_CONFIG_WR_TYPE1			0xb
>  
> -#define PCIE_CONF_BUS(bus)			(((bus) & 0xff) << 20)
> -#define PCIE_CONF_DEV(dev)			(((dev) & 0x1f) << 15)
> -#define PCIE_CONF_FUNC(fun)			(((fun) & 0x7)	<< 12)
> -#define PCIE_CONF_REG(reg)			((reg) & 0xffc)
> -#define PCIE_CONF_ADDR(bus, devfn, where)	\
> -	(PCIE_CONF_BUS(bus) | PCIE_CONF_DEV(PCI_SLOT(devfn))	| \
> -	 PCIE_CONF_FUNC(PCI_FUNC(devfn)) | PCIE_CONF_REG(where))
> -
>  #define PIO_RETRY_CNT			500
>  #define PIO_RETRY_DELAY			2 /* 2 us*/
>  
> @@ -687,7 +680,7 @@ static int advk_pcie_rd_conf(struct pci_bus *bus, u32 devfn,
>  	advk_writel(pcie, reg, PIO_CTRL);
>  
>  	/* Program the address registers */
> -	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
> +	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
>  	advk_writel(pcie, reg, PIO_ADDR_LS);
>  	advk_writel(pcie, 0, PIO_ADDR_MS);
>  
> @@ -748,7 +741,7 @@ static int advk_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
>  	advk_writel(pcie, reg, PIO_CTRL);
>  
>  	/* Program the address registers */
> -	reg = PCIE_CONF_ADDR(bus->number, devfn, where);
> +	reg = ALIGN_DOWN(PCIE_ECAM_OFFSET(bus->number, devfn, where), 4);
>  	advk_writel(pcie, reg, PIO_ADDR_LS);
>  	advk_writel(pcie, 0, PIO_ADDR_MS);
>  
> diff --git a/drivers/pci/controller/pci-host-generic.c b/drivers/pci/controller/pci-host-generic.c
> index b51977abfdf1..63865aeb636b 100644
> --- a/drivers/pci/controller/pci-host-generic.c
> +++ b/drivers/pci/controller/pci-host-generic.c
> @@ -49,7 +49,6 @@ static void __iomem *pci_dw_ecam_map_bus(struct pci_bus *bus,
>  }
>  
>  static const struct pci_ecam_ops pci_dw_ecam_bus_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_dw_ecam_map_bus,
>  		.read		= pci_generic_config_read,
> diff --git a/drivers/pci/controller/pci-thunder-ecam.c b/drivers/pci/controller/pci-thunder-ecam.c
> index 7e8835fee5f7..f964fd26f7e0 100644
> --- a/drivers/pci/controller/pci-thunder-ecam.c
> +++ b/drivers/pci/controller/pci-thunder-ecam.c
> @@ -346,7 +346,6 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
>  }
>  
>  const struct pci_ecam_ops pci_thunder_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus        = pci_ecam_map_bus,
>  		.read           = thunder_ecam_config_read,
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index bea86899bd5d..7fc80fd6f13f 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -22,6 +22,7 @@
>  #include <linux/of_pci.h>
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/printk.h>
>  #include <linux/reset.h>
>  #include <linux/sizes.h>
> @@ -127,11 +128,7 @@
>  #define  MSI_INT_MASK_CLR		0x14
>  
>  #define PCIE_EXT_CFG_DATA				0x8000
> -
>  #define PCIE_EXT_CFG_INDEX				0x9000
> -#define  PCIE_EXT_BUSNUM_SHIFT				20
> -#define  PCIE_EXT_SLOT_SHIFT				15
> -#define  PCIE_EXT_FUNC_SHIFT				12
>  
>  #define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
>  #define  PCIE_RGR1_SW_INIT_1_PERST_SHIFT		0x0
> @@ -695,15 +692,6 @@ static bool brcm_pcie_link_up(struct brcm_pcie *pcie)
>  	return dla && plu;
>  }
>  
> -/* Configuration space read/write support */
> -static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
> -{
> -	return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
> -		| ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
> -		| (busnr << PCIE_EXT_BUSNUM_SHIFT)
> -		| (reg & ~3);
> -}
> -
>  static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  					int where)
>  {
> @@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  		return PCI_SLOT(devfn) ? NULL : base + where;
>  
>  	/* For devices, write to the config space index register */
> -	idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
> +	idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
>  	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
>  	return base + PCIE_EXT_CFG_DATA + where;
>  }
> diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
> index 9705059523a6..f1d08a1b1591 100644
> --- a/drivers/pci/controller/pcie-rockchip-host.c
> +++ b/drivers/pci/controller/pcie-rockchip-host.c
> @@ -157,12 +157,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  				       struct pci_bus *bus, u32 devfn,
>  				       int where, int size, u32 *val)
>  {
> -	u32 busdev;
> +	void __iomem *addr;
>  
> -	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
> -				PCI_FUNC(devfn), where);
> +	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  
> -	if (!IS_ALIGNED(busdev, size)) {
> +	if (!IS_ALIGNED((uintptr_t)addr, size)) {
>  		*val = 0;
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  	}
> @@ -175,11 +174,11 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  						AXI_WRAPPER_TYPE1_CFG);
>  
>  	if (size == 4) {
> -		*val = readl(rockchip->reg_base + busdev);
> +		*val = readl(addr);
>  	} else if (size == 2) {
> -		*val = readw(rockchip->reg_base + busdev);
> +		*val = readw(addr);
>  	} else if (size == 1) {
> -		*val = readb(rockchip->reg_base + busdev);
> +		*val = readb(addr);
>  	} else {
>  		*val = 0;
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
> @@ -191,11 +190,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
>  				       struct pci_bus *bus, u32 devfn,
>  				       int where, int size, u32 val)
>  {
> -	u32 busdev;
> +	void __iomem *addr;
>  
> -	busdev = PCIE_ECAM_ADDR(bus->number, PCI_SLOT(devfn),
> -				PCI_FUNC(devfn), where);
> -	if (!IS_ALIGNED(busdev, size))
> +	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
> +
> +	if (!IS_ALIGNED((uintptr_t)addr, size))
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  
>  	if (pci_is_root_bus(bus->parent))
> @@ -206,11 +205,11 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
>  						AXI_WRAPPER_TYPE1_CFG);
>  
>  	if (size == 4)
> -		writel(val, rockchip->reg_base + busdev);
> +		writel(val, addr);
>  	else if (size == 2)
> -		writew(val, rockchip->reg_base + busdev);
> +		writew(val, addr);
>  	else if (size == 1)
> -		writeb(val, rockchip->reg_base + busdev);
> +		writeb(val, addr);
>  	else
>  		return PCIBIOS_BAD_REGISTER_NUMBER;
>  
> diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
> index c7d0178fc8c2..1650a5087450 100644
> --- a/drivers/pci/controller/pcie-rockchip.h
> +++ b/drivers/pci/controller/pcie-rockchip.h
> @@ -13,6 +13,7 @@
>  
>  #include <linux/kernel.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  
>  /*
>   * The upper 16 bits of PCIE_CLIENT_CONFIG are a write mask for the lower 16
> @@ -178,13 +179,6 @@
>  #define MIN_AXI_ADDR_BITS_PASSED		8
>  #define PCIE_RC_SEND_PME_OFF			0x11960
>  #define ROCKCHIP_VENDOR_ID			0x1d87
> -#define PCIE_ECAM_BUS(x)			(((x) & 0xff) << 20)
> -#define PCIE_ECAM_DEV(x)			(((x) & 0x1f) << 15)
> -#define PCIE_ECAM_FUNC(x)			(((x) & 0x7) << 12)
> -#define PCIE_ECAM_REG(x)			(((x) & 0xfff) << 0)
> -#define PCIE_ECAM_ADDR(bus, dev, func, reg) \
> -	  (PCIE_ECAM_BUS(bus) | PCIE_ECAM_DEV(dev) | \
> -	   PCIE_ECAM_FUNC(func) | PCIE_ECAM_REG(reg))
>  #define PCIE_LINK_IS_L2(x) \
>  	(((x) & PCIE_CLIENT_DEBUG_LTSSM_MASK) == PCIE_CLIENT_DEBUG_LTSSM_L2)
>  #define PCIE_LINK_UP(x) \
> diff --git a/drivers/pci/controller/pcie-tango.c b/drivers/pci/controller/pcie-tango.c
> index d093a8ce4bb1..62a061f1d62e 100644
> --- a/drivers/pci/controller/pcie-tango.c
> +++ b/drivers/pci/controller/pcie-tango.c
> @@ -208,7 +208,6 @@ static int smp8759_config_write(struct pci_bus *bus, unsigned int devfn,
>  }
>  
>  static const struct pci_ecam_ops smp8759_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= smp8759_config_read,
> diff --git a/drivers/pci/controller/pcie-xilinx-nwl.c b/drivers/pci/controller/pcie-xilinx-nwl.c
> index f3cf7d61924f..7f29c2fdcd51 100644
> --- a/drivers/pci/controller/pcie-xilinx-nwl.c
> +++ b/drivers/pci/controller/pcie-xilinx-nwl.c
> @@ -18,6 +18,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/of_irq.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  #include <linux/irqchip/chained_irq.h>
>  
> @@ -124,8 +125,6 @@
>  #define E_ECAM_CR_ENABLE		BIT(0)
>  #define E_ECAM_SIZE_LOC			GENMASK(20, 16)
>  #define E_ECAM_SIZE_SHIFT		16
> -#define ECAM_BUS_LOC_SHIFT		20
> -#define ECAM_DEV_LOC_SHIFT		12
>  #define NWL_ECAM_VALUE_DEFAULT		12
>  
>  #define CFG_DMA_REG_BAR			GENMASK(2, 0)
> @@ -240,15 +239,11 @@ static void __iomem *nwl_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
>  				      int where)
>  {
>  	struct nwl_pcie *pcie = bus->sysdata;
> -	int relbus;
>  
>  	if (!nwl_pcie_valid_device(bus, devfn))
>  		return NULL;
>  
> -	relbus = (bus->number << ECAM_BUS_LOC_SHIFT) |
> -			(devfn << ECAM_DEV_LOC_SHIFT);
> -
> -	return pcie->ecam_base + relbus + where;
> +	return pcie->ecam_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  }
>  
>  /* PCIe operations */
> diff --git a/drivers/pci/controller/pcie-xilinx.c b/drivers/pci/controller/pcie-xilinx.c
> index 8523be61bba5..fa5baeb82653 100644
> --- a/drivers/pci/controller/pcie-xilinx.c
> +++ b/drivers/pci/controller/pcie-xilinx.c
> @@ -21,6 +21,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/of_irq.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
>  #include "../pci.h"
> @@ -86,10 +87,6 @@
>  /* Phy Status/Control Register definitions */
>  #define XILINX_PCIE_REG_PSCR_LNKUP	BIT(11)
>  
> -/* ECAM definitions */
> -#define ECAM_BUS_NUM_SHIFT		20
> -#define ECAM_DEV_NUM_SHIFT		12
> -
>  /* Number of MSI IRQs */
>  #define XILINX_NUM_MSI_IRQS		128
>  
> @@ -183,15 +180,11 @@ static void __iomem *xilinx_pcie_map_bus(struct pci_bus *bus,
>  					 unsigned int devfn, int where)
>  {
>  	struct xilinx_pcie_port *port = bus->sysdata;
> -	int relbus;
>  
>  	if (!xilinx_pcie_valid_device(bus, devfn))
>  		return NULL;
>  
> -	relbus = (bus->number << ECAM_BUS_NUM_SHIFT) |
> -		 (devfn << ECAM_DEV_NUM_SHIFT);
> -
> -	return port->reg_base + relbus + where;
> +	return port->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  }
>  
>  /* PCIe operations */
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index f375c21ceeb1..1361a79bd1e7 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -11,6 +11,7 @@
>  #include <linux/module.h>
>  #include <linux/msi.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/srcu.h>
>  #include <linux/rculist.h>
>  #include <linux/rcupdate.h>
> @@ -328,15 +329,13 @@ static void vmd_remove_irq_domain(struct vmd_dev *vmd)
>  static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
>  				  unsigned int devfn, int reg, int len)
>  {
> -	char __iomem *addr = vmd->cfgbar +
> -			     ((bus->number - vmd->busn_start) << 20) +
> -			     (devfn << 12) + reg;
> +	unsigned int busnr_ecam = bus->number - vmd->busn_start;
> +	u32 offset = PCIE_ECAM_OFFSET(busnr_ecam, devfn, reg);
>  
> -	if ((addr - vmd->cfgbar) + len >=
> -	    resource_size(&vmd->dev->resource[VMD_CFGBAR]))
> +	if (offset + len >= resource_size(&vmd->dev->resource[VMD_CFGBAR]))
>  		return NULL;
>  
> -	return addr;
> +	return vmd->cfgbar + offset;
>  }
>  

For vmd.c:
Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>

Thanks


>  /*
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index b54d32a31669..59f91d434859 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -131,25 +131,36 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>  			       int where)
>  {
>  	struct pci_config_window *cfg = bus->sysdata;
> +	unsigned int bus_shift = cfg->ops->bus_shift;
>  	unsigned int devfn_shift = cfg->ops->bus_shift - 8;
>  	unsigned int busn = bus->number;
>  	void __iomem *base;
> +	u32 bus_offset, devfn_offset;
>  
>  	if (busn < cfg->busr.start || busn > cfg->busr.end)
>  		return NULL;
>  
>  	busn -= cfg->busr.start;
> -	if (per_bus_mapping)
> +	if (per_bus_mapping) {
>  		base = cfg->winp[busn];
> -	else
> -		base = cfg->win + (busn << cfg->ops->bus_shift);
> -	return base + (devfn << devfn_shift) + where;
> +		busn = 0;
> +	} else
> +		base = cfg->win;
> +
> +	if (cfg->ops->bus_shift) {
> +		bus_offset = (busn & PCIE_ECAM_BUS_MASK) << bus_shift;
> +		devfn_offset = (devfn & PCIE_ECAM_DEVFN_MASK) << devfn_shift;
> +		where &= PCIE_ECAM_REG_MASK;
> +
> +		return base + (bus_offset | devfn_offset | where);
> +	}
> +
> +	return base + PCIE_ECAM_OFFSET(busn, devfn, where);
>  }
>  EXPORT_SYMBOL_GPL(pci_ecam_map_bus);
>  
>  /* ECAM ops */
>  const struct pci_ecam_ops pci_generic_ecam_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read,
> @@ -161,7 +172,6 @@ EXPORT_SYMBOL_GPL(pci_generic_ecam_ops);
>  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
>  /* ECAM ops for 32-bit access only (non-compliant) */
>  const struct pci_ecam_ops pci_32b_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read32,
> @@ -171,7 +181,6 @@ const struct pci_ecam_ops pci_32b_ops = {
>  
>  /* ECAM ops for 32-bit read only (non-compliant) */
>  const struct pci_ecam_ops pci_32b_read_ops = {
> -	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus	= pci_ecam_map_bus,
>  		.read		= pci_generic_config_read32,
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> index 033ce74f02e8..65d3d83015c3 100644
> --- a/include/linux/pci-ecam.h
> +++ b/include/linux/pci-ecam.h
> @@ -9,6 +9,33 @@
>  #include <linux/kernel.h>
>  #include <linux/platform_device.h>
>  
> +/*
> + * Memory address shift values for the byte-level address that
> + * can be used when accessing the PCI Express Configuration Space.
> + */
> +
> +/*
> + * Enhanced Configuration Access Mechanism (ECAM)
> + *
> + * See PCI Express Base Specification, Revision 5.0, Version 1.0,
> + * Section 7.2.2, Table 7-1, p. 677.
> + */
> +#define PCIE_ECAM_BUS_SHIFT	20 /* Bus number */
> +#define PCIE_ECAM_DEVFN_SHIFT	12 /* Device and Function number */
> +
> +#define PCIE_ECAM_BUS_MASK	0xff
> +#define PCIE_ECAM_DEVFN_MASK	0xff
> +#define PCIE_ECAM_REG_MASK	0xfff /* Limit offset to a maximum of 4K */
> +
> +#define PCIE_ECAM_BUS(x)	(((x) & PCIE_ECAM_BUS_MASK) << PCIE_ECAM_BUS_SHIFT)
> +#define PCIE_ECAM_DEVFN(x)	(((x) & PCIE_ECAM_DEVFN_MASK) << PCIE_ECAM_DEVFN_SHIFT)
> +#define PCIE_ECAM_REG(x)	((x) & PCIE_ECAM_REG_MASK)
> +
> +#define PCIE_ECAM_OFFSET(bus, devfn, where) \
> +	(PCIE_ECAM_BUS(bus) | \
> +	 PCIE_ECAM_DEVFN(devfn) | \
> +	 PCIE_ECAM_REG(where))
> +
>  /*
>   * struct to hold pci ops and bus shift of the config window
>   * for a PCI controller.

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
                   ` (4 preceding siblings ...)
  2020-11-29 23:07 ` [PATCH v6 5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c Krzysztof Wilczyński
@ 2020-12-01 15:34 ` Lorenzo Pieralisi
  2020-12-08 15:41   ` Michael Walle
  2020-12-08 16:07 ` Qian Cai
  6 siblings, 1 reply; 32+ messages in thread
From: Lorenzo Pieralisi @ 2020-12-01 15:34 UTC (permalink / raw)
  To: Bjorn Helgaas, Krzysztof Wilczyński
  Cc: Lorenzo Pieralisi, Zhou Wang, Jonathan Cameron, Ray Jui,
	Jonathan Chocron, Michael Ellerman, bcm-kernel-feedback-list,
	Michal Simek, linux-rpi-kernel, Thomas Petazzoni, linux-rockchip,
	Nicolas Saenz Julienne, Will Deacon, linux-arm-kernel,
	linuxppc-dev, Shawn Lin, Jonathan Derrick, Scott Branden,
	Toan Le, Rob Herring, Heiko Stuebner, Paul Mackerras,
	Florian Fainelli, Robert Richter, Benjamin Herrenschmidt,
	linux-pci

On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
> Unify ECAM-related constants into a single set of standard constants
> defining memory address shift values for the byte-level address that can
> be used when accessing the PCI Express Configuration Space, and then
> move native PCI Express controller drivers to use newly introduced
> definitions retiring any driver-specific ones.
> 
> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> PCI Express specification (see PCI Express Base Specification, Revision
> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> implement it the same way.
> 
> [...]

Applied to pci/ecam, thanks!

[1/5] PCI: Unify ECAM constants in native PCI Express drivers
      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
[2/5] PCI: thunder-pem: Add constant for custom ".bus_shift" initialiser
      https://git.kernel.org/lpieralisi/pci/c/3c38579263
[3/5] PCI: iproc: Convert to use the new ECAM constants
      https://git.kernel.org/lpieralisi/pci/c/333ec9d3cc
[4/5] PCI: vmd: Update type of the __iomem pointers
      https://git.kernel.org/lpieralisi/pci/c/89094c12ea
[5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
      https://git.kernel.org/lpieralisi/pci/c/3dc62532a5

Thanks,
Lorenzo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
  2020-11-30 11:08   ` Lorenzo Pieralisi
  2020-11-30 18:23   ` Derrick, Jonathan
@ 2020-12-06 20:16   ` Krzysztof Wilczyński
  2020-12-07  3:25     ` Florian Fainelli
  2 siblings, 1 reply; 32+ messages in thread
From: Krzysztof Wilczyński @ 2020-12-06 20:16 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Florian Fainelli, Ray Jui, Scott Branden,
	Jonathan Derrick, linux-pci, linux-arm-kernel, linuxppc-dev,
	linux-rockchip, linux-rpi-kernel, bcm-kernel-feedback-list

Hello Nicolas, Florian and Florian,

[...]
> -/* Configuration space read/write support */
> -static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
> -{
> -	return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
> -		| ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
> -		| (busnr << PCIE_EXT_BUSNUM_SHIFT)
> -		| (reg & ~3);
> -}
> -
>  static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  					int where)
>  {
> @@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>  		return PCI_SLOT(devfn) ? NULL : base + where;
>  
>  	/* For devices, write to the config space index register */
> -	idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
> +	idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
>  	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
>  	return base + PCIE_EXT_CFG_DATA + where;
>  }
[...]

Passing the hard-coded 0 as the "reg" argument here never actually did
anything, thus the 32 bit alignment was never correctly enforced.

My question would be: should this be 32 bit aligned?  It seems like the
intention was to perhaps make the alignment?  I am sadly not intimately
familiar with his hardware, so I am not sure if there is something to
fix here or not.

Also, I wonder whether it would be safe to pass the offset (the "where"
variable) rather than hard-coded 0?

Thank you for help in advance!

Bjorn also asked the same question:
  https://lore.kernel.org/linux-pci/20201120203428.GA272511@bjorn-Precision-5520/

Krzysztof

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-06 20:16   ` Krzysztof Wilczyński
@ 2020-12-07  3:25     ` Florian Fainelli
  2020-12-07 20:29       ` Jim Quinlan
  0 siblings, 1 reply; 32+ messages in thread
From: Florian Fainelli @ 2020-12-07  3:25 UTC (permalink / raw)
  To: Krzysztof Wilczyński, Bjorn Helgaas, Jim Quinlan
  Cc: Rob Herring, Jonathan Cameron, Jonathan Chocron, Shawn Lin,
	Heiko Stuebner, Zhou Wang, Lorenzo Pieralisi, Will Deacon,
	Robert Richter, Michal Simek, Toan Le, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Thomas Petazzoni,
	Nicolas Saenz Julienne, Ray Jui, Scott Branden, Jonathan Derrick,
	linux-pci, linux-arm-kernel, linuxppc-dev, linux-rockchip,
	linux-rpi-kernel, bcm-kernel-feedback-list

+JimQ,

On 12/6/2020 12:16 PM, Krzysztof Wilczyński wrote:
> Hello Nicolas, Florian and Florian,
> 
> [...]
>> -/* Configuration space read/write support */
>> -static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
>> -{
>> -	return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
>> -		| ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
>> -		| (busnr << PCIE_EXT_BUSNUM_SHIFT)
>> -		| (reg & ~3);
>> -}
>> -
>>  static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>>  					int where)
>>  {
>> @@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
>>  		return PCI_SLOT(devfn) ? NULL : base + where;
>>  
>>  	/* For devices, write to the config space index register */
>> -	idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
>> +	idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
>>  	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
>>  	return base + PCIE_EXT_CFG_DATA + where;
>>  }
> [...]
> 
> Passing the hard-coded 0 as the "reg" argument here never actually did
> anything, thus the 32 bit alignment was never correctly enforced.
> 
> My question would be: should this be 32 bit aligned?  It seems like the
> intention was to perhaps make the alignment?  I am sadly not intimately
> familiar with his hardware, so I am not sure if there is something to
> fix here or not.
> 
> Also, I wonder whether it would be safe to pass the offset (the "where"
> variable) rather than hard-coded 0?
> 
> Thank you for help in advance!
> 
> Bjorn also asked the same question:
>   https://lore.kernel.org/linux-pci/20201120203428.GA272511@bjorn-Precision-5520/
> 
> Krzysztof
> 

-- 
Florian

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 1/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-07  3:25     ` Florian Fainelli
@ 2020-12-07 20:29       ` Jim Quinlan
  0 siblings, 0 replies; 32+ messages in thread
From: Jim Quinlan @ 2020-12-07 20:29 UTC (permalink / raw)
  To: Florian Fainelli
  Cc: Krzysztof Wilczyński, Bjorn Helgaas, Rob Herring,
	Jonathan Cameron, Jonathan Chocron, Shawn Lin, Heiko Stuebner,
	Zhou Wang, Lorenzo Pieralisi, Will Deacon, Robert Richter,
	Michal Simek, Toan Le, Michael Ellerman, Benjamin Herrenschmidt,
	Paul Mackerras, Thomas Petazzoni, Nicolas Saenz Julienne,
	Ray Jui, Scott Branden, Jonathan Derrick,
	open list:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS,
	moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE,
	linuxppc-dev, linux-rockchip,
	moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE,
	maintainer:BROADCOM BCM7XXX ARM ARCHITECTURE

[-- Attachment #1: Type: text/plain, Size: 3939 bytes --]

On Sun, Dec 6, 2020 at 10:25 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> +JimQ,
>
> On 12/6/2020 12:16 PM, Krzysztof Wilczyński wrote:
> > Hello Nicolas, Florian and Florian,
> >
> > [...]
> >> -/* Configuration space read/write support */
> >> -static inline int brcm_pcie_cfg_index(int busnr, int devfn, int reg)
> >> -{
> >> -    return ((PCI_SLOT(devfn) & 0x1f) << PCIE_EXT_SLOT_SHIFT)
> >> -            | ((PCI_FUNC(devfn) & 0x07) << PCIE_EXT_FUNC_SHIFT)
> >> -            | (busnr << PCIE_EXT_BUSNUM_SHIFT)
> >> -            | (reg & ~3);
> >> -}
> >> -
> >>  static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
> >>                                      int where)
> >>  {
> >> @@ -716,7 +704,7 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
> >>              return PCI_SLOT(devfn) ? NULL : base + where;
> >>
> >>      /* For devices, write to the config space index register */
> >> -    idx = brcm_pcie_cfg_index(bus->number, devfn, 0);
> >> +    idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
> >>      writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
> >>      return base + PCIE_EXT_CFG_DATA + where;
> >>  }
> > [...]
> >
> > Passing the hard-coded 0 as the "reg" argument here never actually did
> > anything, thus the 32 bit alignment was never correctly enforced.
> >
> > My question would be: should this be 32 bit aligned?  It seems like the
> > intention was to perhaps make the alignment?  I am sadly not intimately
> > familiar with his hardware, so I am not sure if there is something to
> > fix here or not.
Hello Krzystzof,

The value gets assigned to our config-space index register, which has
the lower two bits marked "unused".    We're making sure that we are
putting zeroes there but it is most likely not necessary.

> >
> > Also, I wonder whether it would be safe to pass the offset (the "where"
> > variable) rather than hard-coded 0?
The answer is "no" for this code but "maybe" in the future -- allow me
to explain.  We have two methods to access the config space:

(1) Set a designated index register to map to the base of a device's
config-space.  From then we can access a 4k register set.  This is the
method you see in the code and is why we set reg=0 for the index value
and then add "where" to the return address.

(2) Set our index register to the bus/slot/func/reg value, and then we
access a single data register.  In this case we do set the "reg" to
the register value to set the index and then only add "where & 0x3" to
the return address.

As it turns out, (1) is not compatible with some MIPs SOCs that we
still support as they do not have the 4k data register set.  So I may
be changing to (1) in a future pullreq, and if so, I will invoke
PCIE_ECAM_OFFSET(bus->number, devfn, where & ~3);

Regards,
Jim Quinlan
Broadcom STB


> >
> > Thank you for help in advance!
> >
> > Bjorn also asked the same question:
> >   https://lore.kernel.org/linux-pci/20201120203428.GA272511@bjorn-Precision-5520/
> >
> > Krzysztof
> >
>
> --
> Florian

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-01 15:34 ` [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Lorenzo Pieralisi
@ 2020-12-08 15:41   ` Michael Walle
  2020-12-08 21:06     ` Bjorn Helgaas
  2020-12-09 12:36     ` Bjorn Helgaas
  0 siblings, 2 replies; 32+ messages in thread
From: Michael Walle @ 2020-12-08 15:41 UTC (permalink / raw)
  To: lorenzo.pieralisi
  Cc: Jonathan.Cameron, bcm-kernel-feedback-list, benh, bhelgaas,
	f.fainelli, heiko, jonathan.derrick, jonnyc, kw,
	linux-arm-kernel, linux-pci, linux-rockchip, linux-rpi-kernel,
	linuxppc-dev, michal.simek, mpe, nsaenzjulienne, paulus, rjui,
	robh, rrichter, sbranden, shawn.lin, thomas.petazzoni, toan,
	wangzhou1, will, Michael Walle

Hi Lorenzo, Krzysztof,

>On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
>> Unify ECAM-related constants into a single set of standard constants
>> defining memory address shift values for the byte-level address that can
>> be used when accessing the PCI Express Configuration Space, and then
>> move native PCI Express controller drivers to use newly introduced
>> definitions retiring any driver-specific ones.
>> 
>> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
>> PCI Express specification (see PCI Express Base Specification, Revision
>> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
>> implement it the same way.
>> 
>> [...]
>
>Applied to pci/ecam, thanks!
>
>[1/5] PCI: Unify ECAM constants in native PCI Express drivers
>      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
>[2/5] PCI: thunder-pem: Add constant for custom ".bus_shift" initialiser
>      https://git.kernel.org/lpieralisi/pci/c/3c38579263
>[3/5] PCI: iproc: Convert to use the new ECAM constants
>      https://git.kernel.org/lpieralisi/pci/c/333ec9d3cc
>[4/5] PCI: vmd: Update type of the __iomem pointers
>      https://git.kernel.org/lpieralisi/pci/c/89094c12ea
>[5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
>      https://git.kernel.org/lpieralisi/pci/c/3dc62532a5

Patch 1/5 breaks LS1028A boards:

[..]
[    1.144426] pci-host-generic 1f0000000.pcie: host bridge /soc/pcie@1f0000000 ranges:
[    1.152276] pci-host-generic 1f0000000.pcie:      MEM 0x01f8000000..0x01f815ffff -> 0x0000000000
[    1.161161] pci-host-generic 1f0000000.pcie:      MEM 0x01f8160000..0x01f81cffff -> 0x0000000000
[    1.170043] pci-host-generic 1f0000000.pcie:      MEM 0x01f81d0000..0x01f81effff -> 0x0000000000
[    1.178924] pci-host-generic 1f0000000.pcie:      MEM 0x01f81f0000..0x01f820ffff -> 0x0000000000
[    1.187805] pci-host-generic 1f0000000.pcie:      MEM 0x01f8210000..0x01f822ffff -> 0x0000000000
[    1.196686] pci-host-generic 1f0000000.pcie:      MEM 0x01f8230000..0x01f824ffff -> 0x0000000000
[    1.205562] pci-host-generic 1f0000000.pcie:      MEM 0x01fc000000..0x01fc3fffff -> 0x0000000000
[    1.214465] pci-host-generic 1f0000000.pcie: ECAM at [mem 0x1f0000000-0x1f00fffff] for [bus 00]
[    1.223318] pci-host-generic 1f0000000.pcie: PCI host bridge to bus 0000:00
[    1.230350] pci_bus 0000:00: root bus resource [bus 00]
[    1.235625] pci_bus 0000:00: root bus resource [mem 0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
[    1.246077] pci_bus 0000:00: root bus resource [mem 0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
[    1.256969] pci_bus 0000:00: root bus resource [mem 0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
[    1.267427] pci_bus 0000:00: root bus resource [mem 0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
[    1.278326] pci_bus 0000:00: root bus resource [mem 0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
[    1.288779] pci_bus 0000:00: root bus resource [mem 0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
[    1.299669] pci_bus 0000:00: root bus resource [mem 0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
[    1.310138] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
[    1.316234] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff 64bit] (from Enhanced Allocation, properties 0x0)
[    1.326776] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff 64bit pref] (from Enhanced Allocation, properties 0x1)
[    1.337759] pci 0000:00:00.0: VF BAR 0: [mem 0x1f81d0000-0x1f81dffff 64bit] (from Enhanced Allocation, properties 0x4)
[    1.348563] pci 0000:00:00.0: VF BAR 2: [mem 0x1f81f0000-0x1f81fffff 64bit pref] (from Enhanced Allocation, properties 0x3)
[    1.359821] pci 0000:00:00.0: PME# supported from D0 D3hot
[    1.365368] pci 0000:00:00.0: VF(n) BAR0 space: [mem 0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
[    1.375381] pci 0000:00:00.0: VF(n) BAR2 space: [mem 0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
[    1.385983] Unable to handle kernel paging request at virtual address ffff800012132000
[    1.393972] Mem abort info:
[    1.396783]   ESR = 0x96000007
[    1.399859]   EC = 0x25: DABT (current EL), IL = 32 bits
[    1.405215]   SET = 0, FnV = 0
[    1.408290]   EA = 0, S1PTW = 0
[    1.411453] Data abort info:
[    1.414352]   ISV = 0, ISS = 0x00000007
[    1.418216]   CM = 0, WnR = 0
[    1.421205] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000000008369c000
[    1.427966] [ffff800012132000] pgd=00000020fffff003, p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffffa003, pte=0000000000000000
[    1.440618] Internal error: Oops: 96000007 [#1] PREEMPT SMP
[    1.446239] Modules linked in:
[    1.449320] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.10.0-rc3-00101-g2f378db5c89 #191
[    1.457484] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT)
[    1.465827] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
[    1.471892] pc : pci_generic_config_read+0x38/0xe0
[    1.476723] lr : pci_generic_config_read+0x24/0xe0
[    1.481553] sp : ffff80001211b920
[    1.484891] x29: ffff80001211b920 x28: 0000000000000000 
[    1.490252] x27: ffff8000116a04bc x26: 0000000000000000 
[    1.495612] x25: 0000000000000001 x24: ffff80001211ba54 
[    1.500972] x23: ffff0020009c3800 x22: 0000000000000000 
[    1.506332] x21: 0000000000000087 x20: ffff80001211b994 
[    1.511692] x19: 0000000000000004 x18: 0000000000000000 
[    1.517052] x17: 0000000000000000 x16: 00000000d5edfbc1 
[    1.522412] x15: ffffffffffffffff x14: ffff800011cf9948 
[    1.527772] x13: ffff002000305a1c x12: 0000000000000030 
[    1.533132] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f 
[    1.538491] x9 : 2c6b7173626d686f x8 : 000000000000ea60 
[    1.543851] x7 : ffff80001211ba54 x6 : 0000000000000000 
[    1.549211] x5 : 0000000000000000 x4 : ffff800012131000 
[    1.554570] x3 : 0000000000000000 x2 : 0000000000000000 
[    1.559930] x1 : 0000000000001000 x0 : ffff800012132000 
[    1.565290] Call trace:
[    1.567752]  pci_generic_config_read+0x38/0xe0
[    1.572233]  pci_bus_read_config_dword+0x84/0xd8
[    1.576890]  pci_bus_generic_read_dev_vendor_id+0x34/0x1b0
[    1.582423]  pci_bus_read_dev_vendor_id+0x4c/0x70
[    1.587167]  pci_scan_single_device+0x84/0xe0
[    1.591559]  pci_scan_slot+0x6c/0x120
[    1.595250]  pci_scan_child_bus_extend+0x54/0x298
[    1.599994]  pci_scan_root_bus_bridge+0xd4/0xf0
[    1.604562]  pci_host_probe+0x18/0xb0
[    1.608254]  pci_host_common_probe+0x13c/0x1a0
[    1.612735]  platform_drv_probe+0x54/0xa8
[    1.616777]  really_probe+0xe4/0x3b8
[    1.620380]  driver_probe_device+0x58/0xb8
[    1.624509]  device_driver_attach+0x74/0x80
[    1.628725]  __driver_attach+0x58/0xe0
[    1.632503]  bus_for_each_dev+0x74/0xc8
[    1.636369]  driver_attach+0x24/0x30
[    1.639972]  bus_add_driver+0x18c/0x1f0
[    1.643838]  driver_register+0x64/0x120
[    1.647704]  __platform_driver_register+0x48/0x58
[    1.652449]  gen_pci_driver_init+0x1c/0x28
[    1.656580]  do_one_initcall+0x4c/0x2c0
[    1.660447]  kernel_init_freeable+0x1e4/0x250
[    1.664840]  kernel_init+0x14/0x118
[    1.668355]  ret_from_fork+0x10/0x34
[    1.671961] Code: 7100067f 540001c0 71000a7f 54000300 (b9400001) 
[    1.678114] ---[ end trace 0aca1b048661e8b3 ]---
[    1.682770] note: swapper/0[1] exited with preempt_count 1
[    1.688305] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.696031] SMP: stopping secondary CPUs
[    1.699989] Kernel Offset: disabled
[    1.703503] CPU features: 0x0240022,61006008
[    1.707806] Memory Limit: none
[    1.710884] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---

There is a LS1028A eval board in kernelci here:
https://lavalab.nxp.com/scheduler/job/170566

I actually have this board which also have a LS1028A SoC:
https://lavalab.kontron.com/scheduler/job/1771

But in the latter you won't see much because earlycon isn't active. [I'm
about to fix that.]

By reverting patch 1/5, the board will work again.

-michael

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
                   ` (5 preceding siblings ...)
  2020-12-01 15:34 ` [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Lorenzo Pieralisi
@ 2020-12-08 16:07 ` Qian Cai
  6 siblings, 0 replies; 32+ messages in thread
From: Qian Cai @ 2020-12-08 16:07 UTC (permalink / raw)
  To: Krzysztof Wilczyński, Bjorn Helgaas
  Cc: Heiko Stuebner, Benjamin Herrenschmidt, Shawn Lin,
	Paul Mackerras, Thomas Petazzoni, Jonathan Chocron, Toan Le,
	Will Deacon, Rob Herring, Lorenzo Pieralisi, Michael Ellerman,
	Michal Simek, linux-rockchip, bcm-kernel-feedback-list,
	Jonathan Derrick, linux-pci, Ray Jui, Florian Fainelli,
	linux-rpi-kernel, Jonathan Cameron, linux-arm-kernel,
	Scott Branden, Zhou Wang, Robert Richter, linuxppc-dev,
	Nicolas Saenz Julienne, Stephen Rothwell,
	Linux Next Mailing List

On Sun, 2020-11-29 at 23:07 +0000, Krzysztof Wilczyński wrote:
> Unify ECAM-related constants into a single set of standard constants
> defining memory address shift values for the byte-level address that can
> be used when accessing the PCI Express Configuration Space, and then
> move native PCI Express controller drivers to use newly introduced
> definitions retiring any driver-specific ones.
> 
> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> PCI Express specification (see PCI Express Base Specification, Revision
> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> implement it the same way.
> 
> Most of the native PCI Express controller drivers define their ECAM-related
> constants, many of these could be shared, or use open-coded values when
> setting the ".bus_shift" field of the "struct pci_ecam_ops".
> 
> All of the newly added constants should remove ambiguity and reduce the
> number of open-coded values, and also correlate more strongly with the
> descriptions in the aforementioned specification (see Table 7-1
> "Enhanced Configuration Address Mapping", p. 677).
> 
> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Krzysztof Wilczyński <kw@linux.com>

Reverting this series on the top of today's linux-next fixed a boot crash on
arm64 Thunder X2 server.

.config: https://cailca.coding.net/public/linux/mm/git/files/master/arm64.config

[  186.285957][    T1] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
[  186.293127][    T1] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI HPX-Type3]
[  186.317072][    T1] acpi PNP0A08:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[  186.330336][    T1] acpi PNP0A08:00: ECAM area [mem 0x30000000-0x37ffffff] reserved by PNP0C02:00
[  186.339538][    T1] acpi PNP0A08:00: ECAM at [mem 0x30000000-0x37ffffff] for [bus 00-7f]
[  186.353258][    T1] PCI host bridge to bus 0000:00
[  186.358162][    T1] pci_bus 0000:00: root bus resource [mem 0x40000000-0x5fffffff window]
[  186.366509][    T1] pci_bus 0000:00: root bus resource [mem 0x10000000000-0x13fffffffff window]
[  186.375366][    T1] pci_bus 0000:00: root bus resource [bus 00-7f]
[  186.382652][    T1] pci 0000:00:00.0: [177d:af00] type 00 class 0x060000
[  186.395174][    T1] pci 0000:00:01.0: [177d:af84] type 01 class 0x060400
[  186.402433][    T1] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[  186.415652][    T1] Unable to handle kernel paging request at virtual address ffff800029370000
[  186.424398][    T1] Mem abort info:
[  186.427930][    T1]   ESR = 0x96000007
[  186.431725][    T1]   EC = 0x25: DABT (current EL), IL = 32 bits
[  186.437805][    T1]   SET = 0, FnV = 0
[  186.441599][    T1]   EA = 0, S1PTW = 0
[  186.445485][    T1] Data abort info:
[  186.449104][    T1]   ISV = 0, ISS = 0x00000007
[  186.453687][    T1]   CM = 0, WnR = 0
[  186.457396][    T1] swapper pgtable: 64k pages, 48-bit VAs, pgdp=00000000da920000
[  186.464979][    T1] [ffff800029370000] pgd=0000008ffcff0003, p4d=0000008ffcff0003, pud=0000008ffcff0003, pmd=00000000811d0003, pte=0000000000000000
[  186.478424][    T1] Internal error: Oops: 96000007 [#1] SMP
[  186.484059][    T1] Modules linked in:
[  186.487854][    T1] CPU: 38 PID: 1 Comm: swapper/0 Tainted: G        W    L    5.10.0-rc7-next-20201208 #3
[  186.497617][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.16 07/29/2020
[  186.508174][    T1] pstate: 20400089 (nzCv daIf +PAN -UAO -TCO BTYPE=--)
[  186.514954][    T1] pc : pci_generic_config_read+0x78/0x1d0
[  186.520587][    T1] lr : pci_generic_config_read+0x64/0x1d0
pci_generic_config_read at drivers/pci/access.c:83
[  186.526223][    T1] sp : ffff000005f0ef30
[  186.530278][    T1] x29: ffff000005f0ef30 x28: 0000000000000010 
[  186.536359][    T1] x27: 0000000000000000 x26: 0000000000000087 
[  186.542441][    T1] x25: 0000000000000000 x24: ffff00084a3a5000 
[  186.548517][    T1] x23: ffff000005f0f150 x22: 0000000000000004 
[  186.554593][    T1] x21: ffff800011404588 x20: ffff000005f0eff0 
[  186.560669][    T1] x19: ffff00084a3a5000 x18: 1fffe001cf0d53ed 
[  186.566750][    T1] x17: 0000000000000000 x16: 0000000000000003 
[  186.572831][    T1] x15: 0000000000000000 x14: 0000000000000003 
[  186.578908][    T1] x13: ffff600000be1ddf x12: 1fffe00000be1dde 
[  186.584983][    T1] x11: 1fffe00000be1dde x10: ffff600000be1dde 
[  186.591059][    T1] x9 : ffff800010c4f59c x8 : ffff000005f0eef3 
[  186.597139][    T1] x7 : 0000000000000001 x6 : 0000000000000001 
[  186.603222][    T1] x5 : 1fffe00109474a1c x4 : 1fffe010fd074cb2 
[  186.609298][    T1] x3 : 0000000000000000 x2 : 0000000000000000 
[  186.615374][    T1] x1 : 0000000000010000 x0 : ffff800029370000 
[  186.621451][    T1] Call trace:
[  186.624623][    T1]  pci_generic_config_read+0x78/0x1d0
__raw_readl at arch/arm64/include/asm/io.h:75
(inlined by) pci_generic_config_read at drivers/pci/access.c:93
[  186.629905][    T1]  pci_bus_read_config_dword+0xfc/0x198
pci_bus_read_config_dword at drivers/pci/access.c:65 (discriminator 2)
[  186.635362][    T1]  pci_bus_generic_read_dev_vendor_id+0x3c/0x310
pci_bus_generic_read_dev_vendor_id at drivers/pci/probe.c:2300
[  186.641611][    T1]  pci_bus_read_dev_vendor_id+0x7c/0xd0
pci_bus_read_dev_vendor_id at drivers/pci/probe.c:2329
[  186.647067][    T1]  pci_scan_single_device+0xe0/0x1f8
pci_scan_device at drivers/pci/probe.c:2342
(inlined by) pci_scan_single_device at drivers/pci/probe.c:2511
(inlined by) pci_scan_single_device at drivers/pci/probe.c:2501
[  186.652261][    T1]  pci_scan_slot+0x88/0x258
pci_scan_slot at drivers/pci/probe.c:2590
[  186.656663][    T1]  pci_scan_child_bus_extend+0x88/0x608
pci_scan_child_bus_extend at drivers/pci/probe.c:2807
[  186.662120][    T1]  pci_scan_child_bus+0x18/0x20
pci_scan_child_bus at drivers/pci/probe.c:2938
[  186.666875][    T1]  acpi_pci_root_create+0x518/0x7a8
acpi_pci_root_create at drivers/acpi/pci_root.c:938
[  186.671982][    T1]  pci_acpi_scan_root+0x2bc/0x4c0
pci_acpi_scan_root at arch/arm64/kernel/pci.c:189
[  186.676911][    T1]  acpi_pci_root_add+0x45c/0x920
acpi_pci_root_add at drivers/acpi/pci_root.c:609
[  186.681754][    T1]  acpi_bus_attach+0x270/0x6d8
acpi_scan_attach_handler at drivers/acpi/scan.c:1969
(inlined by) acpi_bus_attach at drivers/acpi/scan.c:2013
[  186.686419][    T1]  acpi_bus_attach+0x140/0x6d8
[  186.691085][    T1]  acpi_bus_attach+0x140/0x6d8
acpi_bus_attach at drivers/acpi/scan.c:2033 (discriminator 9)
[  186.695751][    T1]  acpi_bus_scan+0x98/0xf8
acpi_bus_scan at drivers/acpi/scan.c:2087
[  186.700067][    T1]  acpi_scan_init+0x220/0x524
[  186.704644][    T1]  acpi_init+0x460/0x4e8
[  186.708790][    T1]  do_one_initcall+0x170/0xb70
[  186.713461][    T1]  kernel_init_freeable+0x6a8/0x734
[  186.718574][    T1]  kernel_init+0x18/0x12c
[  186.722806][    T1]  ret_from_fork+0x10/0x1c
[  186.727123][    T1] Code: 710006df 540002e0 71000adf 540004e0 (b9400013) 
[  186.733988][    T1] ---[ end trace bad65ebbc8c09fe0 ]---
[  186.739359][    T1] Kernel panic - not syncing: Oops: Fatal exception
[  186.746115][    T1] SMP: stopping secondary CPUs
[  186.750968][    T1] ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

> 
> --- 
> Changed in v6:
>   Converted single patch into a series.
>   Dropped changes related to PPC 4xx platform.
>   Refactored pci_ecam_map_bus() so that bus, device function and offset
>   are correctly masked, limiting offset to 4K as per the PCI Express
>   Specification.  After the refactor this function will now use sensible
>   defaults allowing for removal of the ".bus_shit" initialiser from all
>   the users of the "ecam_ops" structure who do not use a non-standard
>   ECAM bus shit values.
> 
> Changed in v5:
>   Removed unused constant "PCIE_ECAM_DEV_SHIFT".
>   Refactored ppc4xx_pciex_get_config_base() so that the "offset"
>   parameter can be passed to so that the PCIE_ECAM_OFFSET() macro
>   can be used.
>   Used the ALIGN_DOWN() macro where 32 bit alignment is required
>   instead using the 0xffc mask.
>   Added CFG_ADDR_CFG_TYPE_1 macro to pci/controller/pcie-iproc.c to
>   denote that this is a configuration type 1 address and access type.
>   Refactored boundary check in pci/controller/vmd.c as used by the
>   vmd_cfg_addr() function following addition of the PCIE_ECAM_OFFSET()
>   macro.
>   Changed the "bus->number" to simply pass the "bus" argument in the
>   PCIE_ECAM_OFFSET() macro.
> 
> Changed in v4:
>   Removed constants related to "CAM".
>   Added more platforms and devices that can use new ECAM macros and
>   constants.
>   Removed unused ".bus_shift" initialisers from pci-xgene.c as
>   xgene_pcie_map_bus() did not use these.
> 
> Changes in v3:
>   Updated commit message wording.
>   Updated regarding custom ECAM bus shift values and concerning PCI base
>   configuration space access for Type 1 access.
>   Refactored rockchip_pcie_rd_other_conf() and rockchip_pcie_wr_other_conf()
>   and removed the "busdev" variable.
>   Removed surplus "relbus" variable from nwl_pcie_map_bus() and
>   xilinx_pcie_map_bus().
>   Renamed the PCIE_ECAM_ADDR() macro to PCIE_ECAM_OFFSET().
> 
> Changes in v2:
>   Use PCIE_ECAM_ADDR macro when computing ECAM address offset, but drop
>   PCI_SLOT and PCI_FUNC macros from the PCIE_ECAM_ADDR macro in favour
>   of using a single value for the device/function.
> 
> Krzysztof Wilczyński (5):
>   PCI: Unify ECAM constants in native PCI Express drivers
>   PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser
>   PCI: iproc: Convert to use the new ECAM constants
>   PCI: vmd: Update type of the __iomem pointers
>   PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
> 
>  drivers/pci/controller/dwc/pcie-al.c        | 12 ++------
>  drivers/pci/controller/dwc/pcie-hisi.c      |  2 --
>  drivers/pci/controller/pci-aardvark.c       | 13 ++-------
>  drivers/pci/controller/pci-host-generic.c   |  1 -
>  drivers/pci/controller/pci-thunder-ecam.c   |  1 -
>  drivers/pci/controller/pci-thunder-pem.c    | 13 +++++++--
>  drivers/pci/controller/pci-xgene.c          |  2 --
>  drivers/pci/controller/pcie-brcmstb.c       | 16 ++---------
>  drivers/pci/controller/pcie-iproc.c         | 31 ++++++---------------
>  drivers/pci/controller/pcie-rockchip-host.c | 27 +++++++++---------
>  drivers/pci/controller/pcie-rockchip.h      |  8 +-----
>  drivers/pci/controller/pcie-tango.c         |  1 -
>  drivers/pci/controller/pcie-xilinx-nwl.c    |  9 ++----
>  drivers/pci/controller/pcie-xilinx.c        | 11 ++------
>  drivers/pci/controller/vmd.c                | 19 ++++++-------
>  drivers/pci/ecam.c                          | 23 ++++++++++-----
>  include/linux/pci-ecam.h                    | 27 ++++++++++++++++++
>  17 files changed, 96 insertions(+), 120 deletions(-)
> 


^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-08 15:41   ` Michael Walle
@ 2020-12-08 21:06     ` Bjorn Helgaas
  2020-12-08 21:11       ` Michael Walle
  2020-12-09 12:36     ` Bjorn Helgaas
  1 sibling, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-08 21:06 UTC (permalink / raw)
  To: Michael Walle
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne, Qian Cai

[+cc Qian]

On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
> Hi Lorenzo, Krzysztof,
> 
> >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
> >> Unify ECAM-related constants into a single set of standard constants
> >> defining memory address shift values for the byte-level address that can
> >> be used when accessing the PCI Express Configuration Space, and then
> >> move native PCI Express controller drivers to use newly introduced
> >> definitions retiring any driver-specific ones.
> >> 
> >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> >> PCI Express specification (see PCI Express Base Specification, Revision
> >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> >> implement it the same way.
> >> 
> >> [...]
> >
> >Applied to pci/ecam, thanks!
> >
> >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
> >      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
> >[2/5] PCI: thunder-pem: Add constant for custom ".bus_shift" initialiser
> >      https://git.kernel.org/lpieralisi/pci/c/3c38579263
> >[3/5] PCI: iproc: Convert to use the new ECAM constants
> >      https://git.kernel.org/lpieralisi/pci/c/333ec9d3cc
> >[4/5] PCI: vmd: Update type of the __iomem pointers
> >      https://git.kernel.org/lpieralisi/pci/c/89094c12ea
> >[5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
> >      https://git.kernel.org/lpieralisi/pci/c/3dc62532a5
> 
> Patch 1/5 breaks LS1028A boards:

I temporarily dropped this series while we figure out what went wrong
here.

Bjorn

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-08 21:06     ` Bjorn Helgaas
@ 2020-12-08 21:11       ` Michael Walle
  0 siblings, 0 replies; 32+ messages in thread
From: Michael Walle @ 2020-12-08 21:11 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne, Qian Cai

Am 2020-12-08 22:06, schrieb Bjorn Helgaas:
> [+cc Qian]
> 
> On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
>> Hi Lorenzo, Krzysztof,
>> 
>> >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
>> >> Unify ECAM-related constants into a single set of standard constants
>> >> defining memory address shift values for the byte-level address that can
>> >> be used when accessing the PCI Express Configuration Space, and then
>> >> move native PCI Express controller drivers to use newly introduced
>> >> definitions retiring any driver-specific ones.
>> >>
>> >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
>> >> PCI Express specification (see PCI Express Base Specification, Revision
>> >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
>> >> implement it the same way.
>> >>
>> >> [...]
>> >
>> >Applied to pci/ecam, thanks!
>> >
>> >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
>> >      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
>> >[2/5] PCI: thunder-pem: Add constant for custom ".bus_shift" initialiser
>> >      https://git.kernel.org/lpieralisi/pci/c/3c38579263
>> >[3/5] PCI: iproc: Convert to use the new ECAM constants
>> >      https://git.kernel.org/lpieralisi/pci/c/333ec9d3cc
>> >[4/5] PCI: vmd: Update type of the __iomem pointers
>> >      https://git.kernel.org/lpieralisi/pci/c/89094c12ea
>> >[5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c
>> >      https://git.kernel.org/lpieralisi/pci/c/3dc62532a5
>> 
>> Patch 1/5 breaks LS1028A boards:
> 
> I temporarily dropped this series while we figure out what went wrong
> here.

Thanks, let me know if I can test something on the board.

-michael

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-08 15:41   ` Michael Walle
  2020-12-08 21:06     ` Bjorn Helgaas
@ 2020-12-09 12:36     ` Bjorn Helgaas
  2020-12-09 13:08       ` Michael Walle
  1 sibling, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-09 12:36 UTC (permalink / raw)
  To: Michael Walle
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne

On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
> >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
> >> Unify ECAM-related constants into a single set of standard constants
> >> defining memory address shift values for the byte-level address that can
> >> be used when accessing the PCI Express Configuration Space, and then
> >> move native PCI Express controller drivers to use newly introduced
> >> definitions retiring any driver-specific ones.
> >> 
> >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> >> PCI Express specification (see PCI Express Base Specification, Revision
> >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> >> implement it the same way.
> >> 
> >> [...]
> >
> >Applied to pci/ecam, thanks!
> >
> >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
> >      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692

> Patch 1/5 breaks LS1028A boards:
> 
> [..]
> [    1.144426] pci-host-generic 1f0000000.pcie: host bridge /soc/pcie@1f0000000 ranges:
> [    1.152276] pci-host-generic 1f0000000.pcie:      MEM 0x01f8000000..0x01f815ffff -> 0x0000000000
> [    1.161161] pci-host-generic 1f0000000.pcie:      MEM 0x01f8160000..0x01f81cffff -> 0x0000000000
> [    1.170043] pci-host-generic 1f0000000.pcie:      MEM 0x01f81d0000..0x01f81effff -> 0x0000000000
> [    1.178924] pci-host-generic 1f0000000.pcie:      MEM 0x01f81f0000..0x01f820ffff -> 0x0000000000
> [    1.187805] pci-host-generic 1f0000000.pcie:      MEM 0x01f8210000..0x01f822ffff -> 0x0000000000
> [    1.196686] pci-host-generic 1f0000000.pcie:      MEM 0x01f8230000..0x01f824ffff -> 0x0000000000
> [    1.205562] pci-host-generic 1f0000000.pcie:      MEM 0x01fc000000..0x01fc3fffff -> 0x0000000000

Can you attach your DT?  The fact that all these windows map to PCI
bus address 0 looks broken.  Prior to patch 1/5, do the devices below
this bridge actually work?

Looks like you're using the pci-host-generic driver; which of the
.compatible strings (pci-host-cam-generic, pci-host-ecam-generic,
marvell,armada8k-pcie-ecam, etc) are you using?  (I think that's in
the DT as well.)

> [    1.214465] pci-host-generic 1f0000000.pcie: ECAM at [mem 0x1f0000000-0x1f00fffff] for [bus 00]


> [    1.223318] pci-host-generic 1f0000000.pcie: PCI host bridge to bus 0000:00
> [    1.230350] pci_bus 0000:00: root bus resource [bus 00]
> [    1.235625] pci_bus 0000:00: root bus resource [mem 0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
> [    1.246077] pci_bus 0000:00: root bus resource [mem 0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
> [    1.256969] pci_bus 0000:00: root bus resource [mem 0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
> [    1.267427] pci_bus 0000:00: root bus resource [mem 0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
> [    1.278326] pci_bus 0000:00: root bus resource [mem 0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
> [    1.288779] pci_bus 0000:00: root bus resource [mem 0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
> [    1.299669] pci_bus 0000:00: root bus resource [mem 0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
> [    1.310138] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
> [    1.316234] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff 64bit] (from Enhanced Allocation, properties 0x0)
> [    1.326776] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff 64bit pref] (from Enhanced Allocation, properties 0x1)
> [    1.337759] pci 0000:00:00.0: VF BAR 0: [mem 0x1f81d0000-0x1f81dffff 64bit] (from Enhanced Allocation, properties 0x4)
> [    1.348563] pci 0000:00:00.0: VF BAR 2: [mem 0x1f81f0000-0x1f81fffff 64bit pref] (from Enhanced Allocation, properties 0x3)
> [    1.359821] pci 0000:00:00.0: PME# supported from D0 D3hot
> [    1.365368] pci 0000:00:00.0: VF(n) BAR0 space: [mem 0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
> [    1.375381] pci 0000:00:00.0: VF(n) BAR2 space: [mem 0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
> [    1.385983] Unable to handle kernel paging request at virtual address ffff800012132000

If ffff800012132000 were an actual ECAM address, we would expect the
low 20 bits to contain the device number, function number, and
config register offset, i.e.,

  dev   (0xffff800012132000 >> 15) & 0x01f = 0x6
  fn    (0xffff800012132000 >> 12) & 0x007 = 0x2
  reg   (0xffff800012132000)       & 0xfff = 0

but that's non-sensical since we probe for devices in order.  So maybe
this is a bad pointer somewhere else.  I looked at pci_ecam_map_bus()
but didn't see an obvious problem.  Maybe we could brute-force debug
this by adding some printks there.

> [    1.393972] Mem abort info:
> [    1.396783]   ESR = 0x96000007
> [    1.399859]   EC = 0x25: DABT (current EL), IL = 32 bits
> [    1.405215]   SET = 0, FnV = 0
> [    1.408290]   EA = 0, S1PTW = 0
> [    1.411453] Data abort info:
> [    1.414352]   ISV = 0, ISS = 0x00000007
> [    1.418216]   CM = 0, WnR = 0
> [    1.421205] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000000008369c000
> [    1.427966] [ffff800012132000] pgd=00000020fffff003, p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffffa003, pte=0000000000000000
> [    1.440618] Internal error: Oops: 96000007 [#1] PREEMPT SMP
> [    1.446239] Modules linked in:
> [    1.449320] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.10.0-rc3-00101-g2f378db5c89 #191
> [    1.457484] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT)
> [    1.465827] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
> [    1.471892] pc : pci_generic_config_read+0x38/0xe0
> [    1.476723] lr : pci_generic_config_read+0x24/0xe0
> [    1.481553] sp : ffff80001211b920
> [    1.484891] x29: ffff80001211b920 x28: 0000000000000000 
> [    1.490252] x27: ffff8000116a04bc x26: 0000000000000000 
> [    1.495612] x25: 0000000000000001 x24: ffff80001211ba54 
> [    1.500972] x23: ffff0020009c3800 x22: 0000000000000000 
> [    1.506332] x21: 0000000000000087 x20: ffff80001211b994 
> [    1.511692] x19: 0000000000000004 x18: 0000000000000000 
> [    1.517052] x17: 0000000000000000 x16: 00000000d5edfbc1 
> [    1.522412] x15: ffffffffffffffff x14: ffff800011cf9948 
> [    1.527772] x13: ffff002000305a1c x12: 0000000000000030 
> [    1.533132] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f 
> [    1.538491] x9 : 2c6b7173626d686f x8 : 000000000000ea60 
> [    1.543851] x7 : ffff80001211ba54 x6 : 0000000000000000 
> [    1.549211] x5 : 0000000000000000 x4 : ffff800012131000 
> [    1.554570] x3 : 0000000000000000 x2 : 0000000000000000 
> [    1.559930] x1 : 0000000000001000 x0 : ffff800012132000 
> [    1.565290] Call trace:
> [    1.567752]  pci_generic_config_read+0x38/0xe0
> [    1.572233]  pci_bus_read_config_dword+0x84/0xd8
> [    1.576890]  pci_bus_generic_read_dev_vendor_id+0x34/0x1b0
> [    1.582423]  pci_bus_read_dev_vendor_id+0x4c/0x70
> [    1.587167]  pci_scan_single_device+0x84/0xe0
> [    1.591559]  pci_scan_slot+0x6c/0x120
> [    1.595250]  pci_scan_child_bus_extend+0x54/0x298
> [    1.599994]  pci_scan_root_bus_bridge+0xd4/0xf0
> [    1.604562]  pci_host_probe+0x18/0xb0
> [    1.608254]  pci_host_common_probe+0x13c/0x1a0
> [    1.612735]  platform_drv_probe+0x54/0xa8
> [    1.616777]  really_probe+0xe4/0x3b8
> [    1.620380]  driver_probe_device+0x58/0xb8
> [    1.624509]  device_driver_attach+0x74/0x80
> [    1.628725]  __driver_attach+0x58/0xe0
> [    1.632503]  bus_for_each_dev+0x74/0xc8
> [    1.636369]  driver_attach+0x24/0x30
> [    1.639972]  bus_add_driver+0x18c/0x1f0
> [    1.643838]  driver_register+0x64/0x120
> [    1.647704]  __platform_driver_register+0x48/0x58
> [    1.652449]  gen_pci_driver_init+0x1c/0x28
> [    1.656580]  do_one_initcall+0x4c/0x2c0
> [    1.660447]  kernel_init_freeable+0x1e4/0x250
> [    1.664840]  kernel_init+0x14/0x118
> [    1.668355]  ret_from_fork+0x10/0x34
> [    1.671961] Code: 7100067f 540001c0 71000a7f 54000300 (b9400001) 
> [    1.678114] ---[ end trace 0aca1b048661e8b3 ]---
> [    1.682770] note: swapper/0[1] exited with preempt_count 1
> [    1.688305] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [    1.696031] SMP: stopping secondary CPUs
> [    1.699989] Kernel Offset: disabled
> [    1.703503] CPU features: 0x0240022,61006008
> [    1.707806] Memory Limit: none
> [    1.710884] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
> 
> There is a LS1028A eval board in kernelci here:
> https://lavalab.nxp.com/scheduler/job/170566
> 
> I actually have this board which also have a LS1028A SoC:
> https://lavalab.kontron.com/scheduler/job/1771
> 
> But in the latter you won't see much because earlycon isn't active. [I'm
> about to fix that.]
> 
> By reverting patch 1/5, the board will work again.
> 
> -michael
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 12:36     ` Bjorn Helgaas
@ 2020-12-09 13:08       ` Michael Walle
  2020-12-09 14:57         ` Bjorn Helgaas
  0 siblings, 1 reply; 32+ messages in thread
From: Michael Walle @ 2020-12-09 13:08 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Vladimir Oltean, Alexandru Marginean

[+ Vladimir and Alex]

Am 2020-12-09 13:36, schrieb Bjorn Helgaas:
> On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
>> >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
>> >> Unify ECAM-related constants into a single set of standard constants
>> >> defining memory address shift values for the byte-level address that can
>> >> be used when accessing the PCI Express Configuration Space, and then
>> >> move native PCI Express controller drivers to use newly introduced
>> >> definitions retiring any driver-specific ones.
>> >>
>> >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
>> >> PCI Express specification (see PCI Express Base Specification, Revision
>> >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
>> >> implement it the same way.
>> >>
>> >> [...]
>> >
>> >Applied to pci/ecam, thanks!
>> >
>> >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
>> >      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
> 
>> Patch 1/5 breaks LS1028A boards:
>> 
>> [..]
>> [    1.144426] pci-host-generic 1f0000000.pcie: host bridge 
>> /soc/pcie@1f0000000 ranges:
>> [    1.152276] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f8000000..0x01f815ffff -> 0x0000000000
>> [    1.161161] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f8160000..0x01f81cffff -> 0x0000000000
>> [    1.170043] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f81d0000..0x01f81effff -> 0x0000000000
>> [    1.178924] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f81f0000..0x01f820ffff -> 0x0000000000
>> [    1.187805] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f8210000..0x01f822ffff -> 0x0000000000
>> [    1.196686] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01f8230000..0x01f824ffff -> 0x0000000000
>> [    1.205562] pci-host-generic 1f0000000.pcie:      MEM 
>> 0x01fc000000..0x01fc3fffff -> 0x0000000000
> 
> Can you attach your DT?
That would be the following:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28-var3-ads2.dts

You'll find the PCI devices/bridge in:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi

> The fact that all these windows map to PCI
> bus address 0 looks broken.  Prior to patch 1/5, do the devices below
> this bridge actually work?

Yes, these should be the onboard network controller and ethernet
switch.

> Looks like you're using the pci-host-generic driver; which of the
> .compatible strings (pci-host-cam-generic, pci-host-ecam-generic,
> marvell,armada8k-pcie-ecam, etc) are you using?  (I think that's in
> the DT as well.)

compatible = "pci-host-ecam-generic";

>> [    1.214465] pci-host-generic 1f0000000.pcie: ECAM at [mem 
>> 0x1f0000000-0x1f00fffff] for [bus 00]
> 
> 
>> [    1.223318] pci-host-generic 1f0000000.pcie: PCI host bridge to bus 
>> 0000:00
>> [    1.230350] pci_bus 0000:00: root bus resource [bus 00]
>> [    1.235625] pci_bus 0000:00: root bus resource [mem 
>> 0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
>> [    1.246077] pci_bus 0000:00: root bus resource [mem 
>> 0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
>> [    1.256969] pci_bus 0000:00: root bus resource [mem 
>> 0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
>> [    1.267427] pci_bus 0000:00: root bus resource [mem 
>> 0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
>> [    1.278326] pci_bus 0000:00: root bus resource [mem 
>> 0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
>> [    1.288779] pci_bus 0000:00: root bus resource [mem 
>> 0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
>> [    1.299669] pci_bus 0000:00: root bus resource [mem 
>> 0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
>> [    1.310138] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
>> [    1.316234] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff 
>> 64bit] (from Enhanced Allocation, properties 0x0)
>> [    1.326776] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff 
>> 64bit pref] (from Enhanced Allocation, properties 0x1)
>> [    1.337759] pci 0000:00:00.0: VF BAR 0: [mem 
>> 0x1f81d0000-0x1f81dffff 64bit] (from Enhanced Allocation, properties 
>> 0x4)
>> [    1.348563] pci 0000:00:00.0: VF BAR 2: [mem 
>> 0x1f81f0000-0x1f81fffff 64bit pref] (from Enhanced Allocation, 
>> properties 0x3)
>> [    1.359821] pci 0000:00:00.0: PME# supported from D0 D3hot
>> [    1.365368] pci 0000:00:00.0: VF(n) BAR0 space: [mem 
>> 0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
>> [    1.375381] pci 0000:00:00.0: VF(n) BAR2 space: [mem 
>> 0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
>> [    1.385983] Unable to handle kernel paging request at virtual 
>> address ffff800012132000
> 
> If ffff800012132000 were an actual ECAM address, we would expect the
> low 20 bits to contain the device number, function number, and
> config register offset, i.e.,
> 
>   dev   (0xffff800012132000 >> 15) & 0x01f = 0x6
>   fn    (0xffff800012132000 >> 12) & 0x007 = 0x2
>   reg   (0xffff800012132000)       & 0xfff = 0
> 
> but that's non-sensical since we probe for devices in order.  So maybe
> this is a bad pointer somewhere else.  I looked at pci_ecam_map_bus()
> but didn't see an obvious problem.  Maybe we could brute-force debug
> this by adding some printks there.

I'm not that deep into PCI, but if you could tell me what to add where,
I'm happy to help ;)

>> [    1.393972] Mem abort info:
>> [    1.396783]   ESR = 0x96000007
>> [    1.399859]   EC = 0x25: DABT (current EL), IL = 32 bits
>> [    1.405215]   SET = 0, FnV = 0
>> [    1.408290]   EA = 0, S1PTW = 0
>> [    1.411453] Data abort info:
>> [    1.414352]   ISV = 0, ISS = 0x00000007
>> [    1.418216]   CM = 0, WnR = 0
>> [    1.421205] swapper pgtable: 4k pages, 48-bit VAs, 
>> pgdp=000000008369c000
>> [    1.427966] [ffff800012132000] pgd=00000020fffff003, 
>> p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffffa003, 
>> pte=0000000000000000
>> [    1.440618] Internal error: Oops: 96000007 [#1] PREEMPT SMP
>> [    1.446239] Modules linked in:
>> [    1.449320] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 
>> 5.10.0-rc3-00101-g2f378db5c89 #191
>> [    1.457484] Hardware name: Kontron SMARC-sAL28 (Single PHY) on 
>> SMARC Eval 2.0 carrier (DT)
>> [    1.465827] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
>> [    1.471892] pc : pci_generic_config_read+0x38/0xe0
>> [    1.476723] lr : pci_generic_config_read+0x24/0xe0
>> [    1.481553] sp : ffff80001211b920
>> [    1.484891] x29: ffff80001211b920 x28: 0000000000000000
>> [    1.490252] x27: ffff8000116a04bc x26: 0000000000000000
>> [    1.495612] x25: 0000000000000001 x24: ffff80001211ba54
>> [    1.500972] x23: ffff0020009c3800 x22: 0000000000000000
>> [    1.506332] x21: 0000000000000087 x20: ffff80001211b994
>> [    1.511692] x19: 0000000000000004 x18: 0000000000000000
>> [    1.517052] x17: 0000000000000000 x16: 00000000d5edfbc1
>> [    1.522412] x15: ffffffffffffffff x14: ffff800011cf9948
>> [    1.527772] x13: ffff002000305a1c x12: 0000000000000030
>> [    1.533132] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f
>> [    1.538491] x9 : 2c6b7173626d686f x8 : 000000000000ea60
>> [    1.543851] x7 : ffff80001211ba54 x6 : 0000000000000000
>> [    1.549211] x5 : 0000000000000000 x4 : ffff800012131000
>> [    1.554570] x3 : 0000000000000000 x2 : 0000000000000000
>> [    1.559930] x1 : 0000000000001000 x0 : ffff800012132000
>> [    1.565290] Call trace:
>> [    1.567752]  pci_generic_config_read+0x38/0xe0
>> [    1.572233]  pci_bus_read_config_dword+0x84/0xd8
>> [    1.576890]  pci_bus_generic_read_dev_vendor_id+0x34/0x1b0
>> [    1.582423]  pci_bus_read_dev_vendor_id+0x4c/0x70
>> [    1.587167]  pci_scan_single_device+0x84/0xe0
>> [    1.591559]  pci_scan_slot+0x6c/0x120
>> [    1.595250]  pci_scan_child_bus_extend+0x54/0x298
>> [    1.599994]  pci_scan_root_bus_bridge+0xd4/0xf0
>> [    1.604562]  pci_host_probe+0x18/0xb0
>> [    1.608254]  pci_host_common_probe+0x13c/0x1a0
>> [    1.612735]  platform_drv_probe+0x54/0xa8
>> [    1.616777]  really_probe+0xe4/0x3b8
>> [    1.620380]  driver_probe_device+0x58/0xb8
>> [    1.624509]  device_driver_attach+0x74/0x80
>> [    1.628725]  __driver_attach+0x58/0xe0
>> [    1.632503]  bus_for_each_dev+0x74/0xc8
>> [    1.636369]  driver_attach+0x24/0x30
>> [    1.639972]  bus_add_driver+0x18c/0x1f0
>> [    1.643838]  driver_register+0x64/0x120
>> [    1.647704]  __platform_driver_register+0x48/0x58
>> [    1.652449]  gen_pci_driver_init+0x1c/0x28
>> [    1.656580]  do_one_initcall+0x4c/0x2c0
>> [    1.660447]  kernel_init_freeable+0x1e4/0x250
>> [    1.664840]  kernel_init+0x14/0x118
>> [    1.668355]  ret_from_fork+0x10/0x34
>> [    1.671961] Code: 7100067f 540001c0 71000a7f 54000300 (b9400001)
>> [    1.678114] ---[ end trace 0aca1b048661e8b3 ]---
>> [    1.682770] note: swapper/0[1] exited with preempt_count 1
>> [    1.688305] Kernel panic - not syncing: Attempted to kill init! 
>> exitcode=0x0000000b
>> [    1.696031] SMP: stopping secondary CPUs
>> [    1.699989] Kernel Offset: disabled
>> [    1.703503] CPU features: 0x0240022,61006008
>> [    1.707806] Memory Limit: none
>> [    1.710884] ---[ end Kernel panic - not syncing: Attempted to kill 
>> init! exitcode=0x0000000b ]---
>> 
>> There is a LS1028A eval board in kernelci here:
>> https://lavalab.nxp.com/scheduler/job/170566
>> 
>> I actually have this board which also have a LS1028A SoC:
>> https://lavalab.kontron.com/scheduler/job/1771
>> 
>> But in the latter you won't see much because earlycon isn't active. 
>> [I'm
>> about to fix that.]
>> 
>> By reverting patch 1/5, the board will work again.
>> 
>> -michael
>> 
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

-- 
-michael

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 13:08       ` Michael Walle
@ 2020-12-09 14:57         ` Bjorn Helgaas
  2020-12-09 15:40           ` Michael Walle
  0 siblings, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-09 14:57 UTC (permalink / raw)
  To: Michael Walle
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Vladimir Oltean, Alexandru Marginean

On Wed, Dec 09, 2020 at 02:08:00PM +0100, Michael Walle wrote:
> [+ Vladimir and Alex]
> 
> Am 2020-12-09 13:36, schrieb Bjorn Helgaas:
> > On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
> > > >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
> > > >> Unify ECAM-related constants into a single set of standard constants
> > > >> defining memory address shift values for the byte-level address that can
> > > >> be used when accessing the PCI Express Configuration Space, and then
> > > >> move native PCI Express controller drivers to use newly introduced
> > > >> definitions retiring any driver-specific ones.
> > > >>
> > > >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> > > >> PCI Express specification (see PCI Express Base Specification, Revision
> > > >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> > > >> implement it the same way.
> > > >>
> > > >> [...]
> > > >
> > > >Applied to pci/ecam, thanks!
> > > >
> > > >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
> > > >      https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
> > 
> > > Patch 1/5 breaks LS1028A boards:
> > > 
> > > [..]
> > > [    1.144426] pci-host-generic 1f0000000.pcie: host bridge
> > > /soc/pcie@1f0000000 ranges:
> > > [    1.152276] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f8000000..0x01f815ffff -> 0x0000000000
> > > [    1.161161] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f8160000..0x01f81cffff -> 0x0000000000
> > > [    1.170043] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f81d0000..0x01f81effff -> 0x0000000000
> > > [    1.178924] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f81f0000..0x01f820ffff -> 0x0000000000
> > > [    1.187805] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f8210000..0x01f822ffff -> 0x0000000000
> > > [    1.196686] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01f8230000..0x01f824ffff -> 0x0000000000
> > > [    1.205562] pci-host-generic 1f0000000.pcie:      MEM
> > > 0x01fc000000..0x01fc3fffff -> 0x0000000000
> > 
> > Can you attach your DT?
> That would be the following:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28-var3-ads2.dts
> 
> You'll find the PCI devices/bridge in:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
> 
> > The fact that all these windows map to PCI
> > bus address 0 looks broken.  Prior to patch 1/5, do the devices below
> > this bridge actually work?
> 
> Yes, these should be the onboard network controller and ethernet
> switch.

Interesting.  I can't see how they could work with those address
mappings, but that's another question we can look at later.

> > Looks like you're using the pci-host-generic driver; which of the
> > .compatible strings (pci-host-cam-generic, pci-host-ecam-generic,
> > marvell,armada8k-pcie-ecam, etc) are you using?  (I think that's in
> > the DT as well.)
> 
> compatible = "pci-host-ecam-generic";
> -michael

Can you try the following just to get started?

diff --git a/drivers/pci/access.c b/drivers/pci/access.c
index 46935695cfb9..569a45727bc7 100644
--- a/drivers/pci/access.c
+++ b/drivers/pci/access.c
@@ -79,6 +79,7 @@ int pci_generic_config_read(struct pci_bus *bus, unsigned int devfn,
 {
 	void __iomem *addr;
 
+	pci_info(bus, "%s(%#x %#05x %d)\n", __func__, devfn, where, size);
 	addr = bus->ops->map_bus(bus, devfn, where);
 	if (!addr) {
 		*val = ~0;
@@ -101,6 +102,7 @@ int pci_generic_config_write(struct pci_bus *bus, unsigned int devfn,
 {
 	void __iomem *addr;
 
+	pci_info(bus, "%s(%#x %#05x %d)\n", __func__, devfn, where, size);
 	addr = bus->ops->map_bus(bus, devfn, where);
 	if (!addr)
 		return PCIBIOS_DEVICE_NOT_FOUND;
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 59f91d434859..78f776e590be 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -33,6 +33,8 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 	struct resource *conflict;
 	int i, err;
 
+	dev_info(dev, "%s cfg %pR bus %pR\n", __func__, cfgres, busr);
+
 	if (busr->start > busr->end)
 		return ERR_PTR(-EINVAL);
 
@@ -85,6 +87,9 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 			goto err_exit_iomap;
 	}
 
+	dev_info(dev, "%s per_bus_mapping %d win %px\n", __func__,
+		 per_bus_mapping, cfg->win);
+
 	if (ops->init) {
 		err = ops->init(cfg);
 		if (err)
@@ -140,6 +145,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 	if (busn < cfg->busr.start || busn > cfg->busr.end)
 		return NULL;
 
+	pci_info(bus, "%s(%#x %#05x): %pR\n", __func__, devfn, where,
+		 &cfg->busr);
 	busn -= cfg->busr.start;
 	if (per_bus_mapping) {
 		base = cfg->winp[busn];
@@ -147,6 +154,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 	} else
 		base = cfg->win;
 
+	pci_info(bus, "%s base %px bus_shift %d\n", __func__, base,
+		 cfg->ops->bus_shift);
 	if (cfg->ops->bus_shift) {
 		bus_offset = (busn & PCIE_ECAM_BUS_MASK) << bus_shift;
 		devfn_offset = (devfn & PCIE_ECAM_DEVFN_MASK) << devfn_shift;

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 14:57         ` Bjorn Helgaas
@ 2020-12-09 15:40           ` Michael Walle
  2020-12-09 20:29             ` Vladimir Oltean
  0 siblings, 1 reply; 32+ messages in thread
From: Michael Walle @ 2020-12-09 15:40 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: lorenzo.pieralisi, kw, heiko, benh, shawn.lin, paulus,
	thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli, mpe,
	michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Vladimir Oltean, Alexandru Marginean

Am 2020-12-09 15:57, schrieb Bjorn Helgaas:
[..]
> Can you try the following just to get started?
> 
> diff --git a/drivers/pci/access.c b/drivers/pci/access.c
> index 46935695cfb9..569a45727bc7 100644
> --- a/drivers/pci/access.c
> +++ b/drivers/pci/access.c
> @@ -79,6 +79,7 @@ int pci_generic_config_read(struct pci_bus *bus,
> unsigned int devfn,
>  {
>  	void __iomem *addr;
> 
> +	pci_info(bus, "%s(%#x %#05x %d)\n", __func__, devfn, where, size);
>  	addr = bus->ops->map_bus(bus, devfn, where);
>  	if (!addr) {
>  		*val = ~0;
> @@ -101,6 +102,7 @@ int pci_generic_config_write(struct pci_bus *bus,
> unsigned int devfn,
>  {
>  	void __iomem *addr;
> 
> +	pci_info(bus, "%s(%#x %#05x %d)\n", __func__, devfn, where, size);
>  	addr = bus->ops->map_bus(bus, devfn, where);
>  	if (!addr)
>  		return PCIBIOS_DEVICE_NOT_FOUND;
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index 59f91d434859..78f776e590be 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -33,6 +33,8 @@ struct pci_config_window *pci_ecam_create(struct 
> device *dev,
>  	struct resource *conflict;
>  	int i, err;
> 
> +	dev_info(dev, "%s cfg %pR bus %pR\n", __func__, cfgres, busr);
> +
>  	if (busr->start > busr->end)
>  		return ERR_PTR(-EINVAL);
> 
> @@ -85,6 +87,9 @@ struct pci_config_window *pci_ecam_create(struct 
> device *dev,
>  			goto err_exit_iomap;
>  	}
> 
> +	dev_info(dev, "%s per_bus_mapping %d win %px\n", __func__,
> +		 per_bus_mapping, cfg->win);
> +
>  	if (ops->init) {
>  		err = ops->init(cfg);
>  		if (err)
> @@ -140,6 +145,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus
> *bus, unsigned int devfn,
>  	if (busn < cfg->busr.start || busn > cfg->busr.end)
>  		return NULL;
> 
> +	pci_info(bus, "%s(%#x %#05x): %pR\n", __func__, devfn, where,
> +		 &cfg->busr);
>  	busn -= cfg->busr.start;
>  	if (per_bus_mapping) {
>  		base = cfg->winp[busn];
> @@ -147,6 +154,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus
> *bus, unsigned int devfn,
>  	} else
>  		base = cfg->win;
> 
> +	pci_info(bus, "%s base %px bus_shift %d\n", __func__, base,
> +		 cfg->ops->bus_shift);
>  	if (cfg->ops->bus_shift) {
>  		bus_offset = (busn & PCIE_ECAM_BUS_MASK) << bus_shift;
>  		devfn_offset = (devfn & PCIE_ECAM_DEVFN_MASK) << devfn_shift;

Hopefully my mail client won't mess up the output that much.

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.10.0-rc7-next-20201208-dirty 
(mwalle@mwalle01) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld 
(GNU Binutils for Debian) 2.31.1) #207 SMP PREEMPT Wed Dec 9 16:35:46 
CET 2020
[    0.000000] Machine model: Kontron SMARC-sAL28 (Single PHY) on SMARC 
Eval 2.0 carrier
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 32 MiB at 0x00000000fcc00000
[    0.000000] earlycon: ns16550a0 at MMIO 0x00000000021c0500 (options 
'115200n8')
[    0.000000] printk: bootconsole [ns16550a0] enabled
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 
0x0000000080000000-0x00000020ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x20ff7dcc00-0x20ff7defff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000020ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000002080000000-0x00000020ffffffff]
[    0.000000] Initmem setup node 0 [mem 
0x0000000080000000-0x00000020ffffffff]
[    0.000000] On node 0 totalpages: 1048576
[    0.000000]   DMA zone: 8192 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 524288 pages, LIFO batch:63
[    0.000000]   Normal zone: 8192 pages used for memmap
[    0.000000]   Normal zone: 524288 pages, LIFO batch:63
[    0.000000] percpu: Embedded 32 pages/cpu s90264 r8192 d32616 u131072
[    0.000000] pcpu-alloc: s90264 r8192 d32616 u131072 alloc=32*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Spectre-v3a
[    0.000000] CPU features: detected: Spectre-v2
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 
1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 
1032192
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: debug earlycon console=ttyS0,115200
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 
4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 
bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 
0x00000000f8c00000-0x00000000fcc00000] (64MB)
[    0.000000] Memory: 3978952K/4194304K available (15680K kernel code, 
3706K rwdata, 8048K rodata, 8256K init, 510K bss, 182584K reserved, 
32768K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, 
Nodes=1
[    0.000000] ftrace: allocating 55767 entries in 218 pages
[    0.000000] ftrace: allocated 218 pages with 5 groups
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to 
nr_cpu_ids=2.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Rude variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay 
is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, 
nr_cpu_ids=2
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 256 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 
0:0x0000000006040000
[    0.000000] ITS [mem 0x06020000-0x0603ffff]
[    0.000000] ITS@0x0000000006020000: allocated 65536 Devices 
@2080200000 (flat, esz 8, psz 64K, shr 0)
[    0.000000] ITS: using cache flushing for cmd queue
[    0.000000] GICv3: using LPI property table @0x00000020801c0000
[    0.000000] GIC: using cache flushing for LPI property table
[    0.000000] GICv3: CPU0: using allocated LPI pending table 
@0x00000020801d0000
[    0.000000] random: get_random_bytes called from 
start_kernel+0x3bc/0x578 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 25.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff 
max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns
[    0.000002] sched_clock: 56 bits at 25MHz, resolution 40ns, wraps 
every 4398046511100ns
[    0.008419] Console: colour dummy device 80x25
[    0.012951] Calibrating delay loop (skipped), value calculated using 
timer frequency.. 50.00 BogoMIPS (lpj=100000)
[    0.023407] pid_max: default: 32768 minimum: 301
[    0.028106] LSM: Security Framework initializing
[    0.032816] Mount-cache hash table entries: 8192 (order: 4, 65536 
bytes, linear)
[    0.040307] Mountpoint-cache hash table entries: 8192 (order: 4, 
65536 bytes, linear)
[    0.049064] rcu: Hierarchical SRCU implementation.
[    0.054069] Platform MSI: gic-its@6020000 domain created
[    0.059517] PCI/MSI: /interrupt-controller@6000000/gic-its@6020000 
domain created
[    0.067116] fsl-mc MSI: gic-its@6020000 domain created
[    0.072911] EFI services will not be available.
[    0.077578] smp: Bringing up secondary CPUs ...
[    0.082420] Detected PIPT I-cache on CPU1
[    0.082441] GICv3: CPU1: found redistributor 1 region 
0:0x0000000006060000
[    0.082448] GICv3: CPU1: using allocated LPI pending table 
@0x00000020801e0000
[    0.082475] CPU1: Booted secondary processor 0x0000000001 
[0x410fd083]
[    0.082541] smp: Brought up 1 node, 2 CPUs
[    0.111539] SMP: Total of 2 processors activated.
[    0.116288] CPU features: detected: 32-bit EL0 Support
[    0.121483] CPU features: detected: CRC32 instructions
[    0.126673] CPU features: detected: 32-bit EL1 Support
[    0.140754] CPU: All CPU(s) started at EL2
[    0.144903] alternatives: patching kernel code
[    0.150172] devtmpfs: initialized
[    0.155754] KASLR disabled due to lack of seed
[    0.160507] clocksource: jiffies: mask: 0xffffffff max_cycles: 
0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.170354] futex hash table entries: 512 (order: 3, 32768 bytes, 
linear)
[    0.177810] pinctrl core: initialized pinctrl subsystem
[    0.183687] DMI not present or invalid.
[    0.187857] NET: Registered protocol family 16
[    0.193117] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic 
allocations
[    0.200341] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for 
atomic allocations
[    0.208322] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for 
atomic allocations
[    0.216341] audit: initializing netlink subsys (disabled)
[    0.221897] audit: type=2000 audit(0.172:1): state=initialized 
audit_enabled=0 res=1
[    0.222415] thermal_sys: Registered thermal governor 'step_wise'
[    0.229723] thermal_sys: Registered thermal governor 
'power_allocator'
[    0.236033] cpuidle: using governor menu
[    0.246693] hw-breakpoint: found 6 breakpoint and 4 watchpoint 
registers.
[    0.253571] ASID allocator initialised with 65536 entries
[    0.259602] Serial: AMBA PL011 UART driver
[    0.274466] Machine: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 
2.0 carrier
[    0.281853] SoC family: QorIQ LS1028A
[    0.285542] SoC ID: svr:0x870b0110, Revision: 1.0
[    0.297947] HugeTLB registered 1.00 GiB page size, pre-allocated 0 
pages
[    0.304722] HugeTLB registered 32.0 MiB page size, pre-allocated 0 
pages
[    0.311489] HugeTLB registered 2.00 MiB page size, pre-allocated 0 
pages
[    0.318254] HugeTLB registered 64.0 KiB page size, pre-allocated 0 
pages
[    0.325775] cryptd: max_cpu_qlen set to 1000
[    0.331453] ACPI: Interpreter disabled.
[    0.336254] iommu: Default domain type: Translated
[    0.341365] vgaarb: loaded
[    0.344250] SCSI subsystem initialized
[    0.348129] libata version 3.00 loaded.
[    0.352170] usbcore: registered new interface driver usbfs
[    0.357733] usbcore: registered new interface driver hub
[    0.363112] usbcore: registered new device driver usb
[    0.368478] imx-i2c 2000000.i2c: can't get pinctrl, bus recovery not 
supported
[    0.376183] i2c i2c-0: IMX I2C adapter registered
[    0.381068] imx-i2c 2030000.i2c: can't get pinctrl, bus recovery not 
supported
[    0.388483] i2c i2c-1: IMX I2C adapter registered
[    0.393357] imx-i2c 2040000.i2c: can't get pinctrl, bus recovery not 
supported
[    0.400879] i2c i2c-2: IMX I2C adapter registered
[    0.405900] pps_core: LinuxPPS API ver. 1 registered
[    0.410913] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 
Rodolfo Giometti <giometti@linux.it>
[    0.420156] PTP clock support registered
[    0.424188] EDAC MC: Ver: 3.0.0
[    0.428104] FPGA manager framework
[    0.431612] Advanced Linux Sound Architecture Driver Initialized.
[    0.438221] clocksource: Switched to clocksource arch_sys_counter
[    0.908920] VFS: Disk quotas dquot_6.6.0
[    0.912921] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 
bytes)
[    0.919981] pnp: PnP ACPI: disabled
[    0.927129] NET: Registered protocol family 2
[    0.931782] tcp_listen_portaddr_hash hash table entries: 2048 (order: 
3, 32768 bytes, linear)
[    0.940413] TCP established hash table entries: 32768 (order: 6, 
262144 bytes, linear)
[    0.948491] TCP bind hash table entries: 32768 (order: 7, 524288 
bytes, linear)
[    0.956179] TCP: Hash tables configured (established 32768 bind 
32768)
[    0.962890] UDP hash table entries: 2048 (order: 4, 65536 bytes, 
linear)
[    0.969682] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, 
linear)
[    0.977005] NET: Registered protocol family 1
[    0.981700] RPC: Registered named UNIX socket transport module.
[    0.987696] RPC: Registered udp transport module.
[    0.992444] RPC: Registered tcp transport module.
[    0.997191] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.003697] PCI: CLS 0 bytes, default 64
[    1.008034] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 
7 counters available
[    1.016471] kvm [1]: IPA Size Limit: 44 bits
[    1.021451] kvm [1]: GICv3: no GICV resource entry
[    1.026291] kvm [1]: disabling GICv2 emulation
[    1.030786] kvm [1]: GIC system register CPU interface enabled
[    1.036703] kvm [1]: vgic interrupt IRQ9
[    1.040722] kvm [1]: Hyp mode initialized successfully
[    1.046971] Initialise system trusted keyrings
[    1.051545] workingset: timestamp_bits=42 max_order=20 bucket_order=0
[    1.061455] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    1.067780] NFS: Registering the id_resolver key type
[    1.072903] Key type id_resolver registered
[    1.077125] Key type id_legacy registered
[    1.081217] nfs4filelayout_init: NFSv4 File Layout Driver 
Registering...
[    1.088079] 9p: Installing v9fs 9p2000 file system support
[    1.122728] Key type asymmetric registered
[    1.126867] Asymmetric key parser 'x509' registered
[    1.131809] Block layer SCSI generic (bsg) driver version 0.4 loaded 
(major 245)
[    1.139278] io scheduler mq-deadline registered
[    1.143849] io scheduler kyber registered
[    1.153103] pci-host-generic 1f0000000.pcie: host bridge 
/soc/pcie@1f0000000 ranges:
[    1.160949] pci-host-generic 1f0000000.pcie:      MEM 
0x01f8000000..0x01f815ffff -> 0x0000000000
[    1.169837] pci-host-generic 1f0000000.pcie:      MEM 
0x01f8160000..0x01f81cffff -> 0x0000000000
[    1.178723] pci-host-generic 1f0000000.pcie:      MEM 
0x01f81d0000..0x01f81effff -> 0x0000000000
[    1.187605] pci-host-generic 1f0000000.pcie:      MEM 
0x01f81f0000..0x01f820ffff -> 0x0000000000
[    1.196487] pci-host-generic 1f0000000.pcie:      MEM 
0x01f8210000..0x01f822ffff -> 0x0000000000
[    1.205369] pci-host-generic 1f0000000.pcie:      MEM 
0x01f8230000..0x01f824ffff -> 0x0000000000
[    1.214247] pci-host-generic 1f0000000.pcie:      MEM 
0x01fc000000..0x01fc3fffff -> 0x0000000000
[    1.223141] pci-host-generic 1f0000000.pcie: pci_ecam_create cfg [mem 
0x1f0000000-0x1f00fffff] bus [bus 00]
[    1.232992] pci-host-generic 1f0000000.pcie: pci_ecam_create 
per_bus_mapping 0 win ffff8000123b1000
[    1.242128] pci-host-generic 1f0000000.pcie: ECAM at [mem 
0x1f0000000-0x1f00fffff] for [bus 00]
[    1.250984] pci-host-generic 1f0000000.pcie: PCI host bridge to bus 
0000:00
[    1.258015] pci_bus 0000:00: root bus resource [bus 00]
[    1.263292] pci_bus 0000:00: root bus resource [mem 
0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
[    1.273751] pci_bus 0000:00: root bus resource [mem 
0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
[    1.284647] pci_bus 0000:00: root bus resource [mem 
0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
[    1.295104] pci_bus 0000:00: root bus resource [mem 
0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
[    1.305995] pci_bus 0000:00: root bus resource [mem 
0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
[    1.316448] pci_bus 0000:00: root bus resource [mem 
0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
[    1.327340] pci_bus 0000:00: root bus resource [mem 
0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
[    1.337794] pci_bus 0000:00: pci_generic_config_read(0x0 0x000 4)
[    1.343942] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x000): [bus 00]
[    1.350177] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.357563] pci_bus 0000:00: pci_generic_config_read(0x0 0x00e 1)
[    1.363711] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x00e): [bus 00]
[    1.369946] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.377327] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    1.383475] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    1.389709] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.397090] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    1.403237] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    1.409471] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.416855] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    1.423002] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    1.429237] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.436618] pci_bus 0000:00: pci_generic_config_read(0x0 0x042 2)
[    1.442765] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x042): [bus 00]
[    1.448999] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.456380] pci_bus 0000:00: pci_generic_config_read(0x0 0x044 2)
[    1.462527] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x044): [bus 00]
[    1.468762] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.476148] pci_bus 0000:00: pci_generic_config_read(0x0 0x008 4)
[    1.482296] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x008): [bus 00]
[    1.488530] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.495912] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    1.502060] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    1.508294] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.515675] pci_bus 0000:00: pci_generic_config_read(0x0 0x000 4)
[    1.521822] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x000): [bus 00]
[    1.528056] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.535440] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    1.541587] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    1.547821] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.555202] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    1.561349] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    1.567583] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.574964] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    1.581111] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    1.587346] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.594728] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    1.600876] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    1.607110] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.614494] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
[    1.620558] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    1.626705] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.632940] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.640320] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.646555] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.652789] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.660171] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    1.666319] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.672553] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.679933] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.686168] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.692402] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.699782] pci_bus 0000:00: pci_generic_config_read(0x0 0x03d 1)
[    1.705929] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x03d): [bus 00]
[    1.712163] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.719546] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    1.725693] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.731928] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.739308] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.745543] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.751777] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.759156] pci_bus 0000:00: pci_generic_config_read(0x0 0x010 4)
[    1.765303] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x010): [bus 00]
[    1.771538] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.778920] pci_bus 0000:00: pci_generic_config_write(0x0 0x010 4)
[    1.785155] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x010): [bus 00]
[    1.791389] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.798770] pci_bus 0000:00: pci_generic_config_read(0x0 0x010 4)
[    1.804917] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x010): [bus 00]
[    1.811151] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.818531] pci_bus 0000:00: pci_generic_config_write(0x0 0x010 4)
[    1.824766] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x010): [bus 00]
[    1.831000] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.838382] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.844617] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.850852] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.858232] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    1.864379] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.870613] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.877993] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.884228] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.890462] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.897843] pci_bus 0000:00: pci_generic_config_read(0x0 0x014 4)
[    1.903990] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x014): [bus 00]
[    1.910224] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.917628] pci_bus 0000:00: pci_generic_config_write(0x0 0x014 4)
[    1.923864] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x014): [bus 00]
[    1.930098] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.937479] pci_bus 0000:00: pci_generic_config_read(0x0 0x014 4)
[    1.943626] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x014): [bus 00]
[    1.949861] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.957241] pci_bus 0000:00: pci_generic_config_write(0x0 0x014 4)
[    1.963476] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x014): [bus 00]
[    1.969709] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.977092] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    1.983327] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    1.989562] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    1.996943] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    2.003089] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.009324] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.016704] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.022938] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.029173] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.036555] pci_bus 0000:00: pci_generic_config_read(0x0 0x018 4)
[    2.042702] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x018): [bus 00]
[    2.048937] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.056317] pci_bus 0000:00: pci_generic_config_write(0x0 0x018 4)
[    2.062552] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x018): [bus 00]
[    2.068786] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.076166] pci_bus 0000:00: pci_generic_config_read(0x0 0x018 4)
[    2.082313] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x018): [bus 00]
[    2.088547] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.095930] pci_bus 0000:00: pci_generic_config_write(0x0 0x018 4)
[    2.102165] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x018): [bus 00]
[    2.108400] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.115780] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.122014] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.128248] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.135629] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    2.141776] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.148010] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.155392] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.161626] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.167861] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.175241] pci_bus 0000:00: pci_generic_config_read(0x0 0x01c 4)
[    2.181388] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x01c): [bus 00]
[    2.187622] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.195002] pci_bus 0000:00: pci_generic_config_write(0x0 0x01c 4)
[    2.201237] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x01c): [bus 00]
[    2.207471] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.214854] pci_bus 0000:00: pci_generic_config_read(0x0 0x01c 4)
[    2.221001] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x01c): [bus 00]
[    2.227235] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.234615] pci_bus 0000:00: pci_generic_config_write(0x0 0x01c 4)
[    2.240850] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x01c): [bus 00]
[    2.247084] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.254464] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.260698] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.266932] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.274314] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    2.280462] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.286696] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.294077] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.300312] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.306546] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.313925] pci_bus 0000:00: pci_generic_config_read(0x0 0x020 4)
[    2.320072] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x020): [bus 00]
[    2.326306] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.333686] pci_bus 0000:00: pci_generic_config_write(0x0 0x020 4)
[    2.339921] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x020): [bus 00]
[    2.346156] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.353538] pci_bus 0000:00: pci_generic_config_read(0x0 0x020 4)
[    2.359685] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x020): [bus 00]
[    2.365919] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.373300] pci_bus 0000:00: pci_generic_config_write(0x0 0x020 4)
[    2.379534] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x020): [bus 00]
[    2.385768] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.393148] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.399382] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.405617] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.413000] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    2.419148] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.425382] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.432762] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.438997] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.445231] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.452611] pci_bus 0000:00: pci_generic_config_read(0x0 0x024 4)
[    2.458758] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x024): [bus 00]
[    2.464993] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.472374] pci_bus 0000:00: pci_generic_config_write(0x0 0x024 4)
[    2.478609] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x024): [bus 00]
[    2.484843] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.492223] pci_bus 0000:00: pci_generic_config_read(0x0 0x024 4)
[    2.498370] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x024): [bus 00]
[    2.504604] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.511985] pci_bus 0000:00: pci_generic_config_write(0x0 0x024 4)
[    2.518219] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x024): [bus 00]
[    2.524454] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.531836] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.538071] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.544305] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.551686] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    2.557833] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.564067] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.571447] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.577681] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.583916] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.591298] pci_bus 0000:00: pci_generic_config_read(0x0 0x030 4)
[    2.597445] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x030): [bus 00]
[    2.603680] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.611060] pci_bus 0000:00: pci_generic_config_write(0x0 0x030 4)
[    2.617295] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x030): [bus 00]
[    2.623530] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.630910] pci_bus 0000:00: pci_generic_config_read(0x0 0x030 4)
[    2.637057] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x030): [bus 00]
[    2.643291] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.650673] pci_bus 0000:00: pci_generic_config_write(0x0 0x030 4)
[    2.656908] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x030): [bus 00]
[    2.663143] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.670523] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    2.676757] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    2.682992] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.690372] pci_bus 0000:00: pci_generic_config_read(0x0 0x02c 2)
[    2.696519] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x02c): [bus 00]
[    2.702754] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.710133] pci_bus 0000:00: pci_generic_config_read(0x0 0x02e 2)
[    2.716281] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x02e): [bus 00]
[    2.722515] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.729899] pci_bus 0000:00: pci_generic_config_read(0x0 0x048 2)
[    2.736046] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x048): [bus 00]
[    2.742280] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.749661] pci_bus 0000:00: pci_generic_config_write(0x0 0x048 2)
[    2.755895] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x048): [bus 00]
[    2.762130] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.769510] pci_bus 0000:00: pci_generic_config_read(0x0 0x044 4)
[    2.775657] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x044): [bus 00]
[    2.781891] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.789274] pci_bus 0000:00: pci_generic_config_read(0x0 0x048 2)
[    2.795421] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x048): [bus 00]
[    2.801655] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.809036] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    2.815183] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    2.821418] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.828798] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    2.834945] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    2.841180] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.848562] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    2.854709] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    2.860943] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.868324] pci_bus 0000:00: pci_generic_config_read(0x0 0x064 4)
[    2.874471] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x064): [bus 00]
[    2.880705] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.888085] pci_bus 0000:00: pci_generic_config_read(0x0 0x064 4)
[    2.894233] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x064): [bus 00]
[    2.900467] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.907864] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    2.914012] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    2.920246] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.927628] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    2.933776] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    2.940010] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.947390] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    2.953537] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    2.959771] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.967154] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    2.973301] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    2.979535] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    2.986916] pci_bus 0000:00: pci_generic_config_read(0x0 0x090 2)
[    2.993063] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x090): [bus 00]
[    2.999297] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.006677] pci_bus 0000:00: pci_generic_config_read(0x0 0x09c 2)
[    3.012824] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x09c): [bus 00]
[    3.019058] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.026441] pci_bus 0000:00: pci_generic_config_read(0x0 0x09e 1)
[    3.032587] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x09e): [bus 00]
[    3.038822] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.046202] pci_bus 0000:00: pci_generic_config_read(0x0 0x0a0 4)
[    3.052349] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0a0): [bus 00]
[    3.058584] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.065964] pci_bus 0000:00: pci_generic_config_read(0x0 0x0a4 4)
[    3.072111] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0a4): [bus 00]
[    3.078345] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.085725] pci_bus 0000:00: pci_generic_config_read(0x0 0x0a8 4)
[    3.091872] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0a8): [bus 00]
[    3.098106] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.105488] pci_bus 0000:00: pci_generic_config_read(0x0 0x0ac 4)
[    3.111635] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0ac): [bus 00]
[    3.117869] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.125251] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff 
64bit] (from Enhanced Allocation, properties 0x0)
[    3.135790] pci_bus 0000:00: pci_generic_config_read(0x0 0x0b0 4)
[    3.141937] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0b0): [bus 00]
[    3.148172] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.155552] pci_bus 0000:00: pci_generic_config_read(0x0 0x0b4 4)
[    3.161698] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0b4): [bus 00]
[    3.167933] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.175315] pci_bus 0000:00: pci_generic_config_read(0x0 0x0b8 4)
[    3.181462] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0b8): [bus 00]
[    3.187696] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.195076] pci_bus 0000:00: pci_generic_config_read(0x0 0x0bc 4)
[    3.201223] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0bc): [bus 00]
[    3.207457] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.214838] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff 
64bit pref] (from Enhanced Allocation, properties 0x1)
[    3.225816] pci_bus 0000:00: pci_generic_config_read(0x0 0x0c0 4)
[    3.231963] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0c0): [bus 00]
[    3.238197] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.245580] pci_bus 0000:00: pci_generic_config_read(0x0 0x0c4 4)
[    3.251727] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0c4): [bus 00]
[    3.257961] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.265342] pci_bus 0000:00: pci_generic_config_read(0x0 0x0c8 4)
[    3.271489] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0c8): [bus 00]
[    3.277723] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.285103] pci_bus 0000:00: pci_generic_config_read(0x0 0x0cc 4)
[    3.291250] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0cc): [bus 00]
[    3.297485] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.304868] pci 0000:00:00.0: VF BAR 0: [mem 0x1f81d0000-0x1f81dffff 
64bit] (from Enhanced Allocation, properties 0x4)
[    3.315671] pci_bus 0000:00: pci_generic_config_read(0x0 0x0d0 4)
[    3.321818] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0d0): [bus 00]
[    3.328053] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.335433] pci_bus 0000:00: pci_generic_config_read(0x0 0x0d4 4)
[    3.341579] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0d4): [bus 00]
[    3.347813] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.355193] pci_bus 0000:00: pci_generic_config_read(0x0 0x0d8 4)
[    3.361340] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0d8): [bus 00]
[    3.367575] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.374957] pci_bus 0000:00: pci_generic_config_read(0x0 0x0dc 4)
[    3.381103] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x0dc): [bus 00]
[    3.387337] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.394719] pci 0000:00:00.0: VF BAR 2: [mem 0x1f81f0000-0x1f81fffff 
64bit pref] (from Enhanced Allocation, properties 0x3)
[    3.405959] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    3.412106] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    3.418340] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.425722] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    3.431868] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    3.438103] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.445485] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    3.451632] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    3.457866] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.465247] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    3.471394] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    3.477628] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.485008] pci_bus 0000:00: pci_generic_config_read(0x0 0x090 2)
[    3.491155] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x090): [bus 00]
[    3.497389] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.504771] pci_bus 0000:00: pci_generic_config_read(0x0 0x09c 2)
[    3.510918] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x09c): [bus 00]
[    3.517153] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.524533] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    3.530680] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    3.536914] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.544294] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    3.550441] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    3.556675] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.564058] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    3.570205] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    3.576440] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.583820] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    3.589967] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    3.596202] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.603581] pci_bus 0000:00: pci_generic_config_read(0x0 0x082 2)
[    3.609729] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x082): [bus 00]
[    3.615964] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.623346] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    3.629493] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    3.635728] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.643108] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    3.649255] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    3.655489] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.662869] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    3.669016] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    3.675250] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.682634] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    3.688780] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    3.695015] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.702395] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    3.708542] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    3.714777] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.722157] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    3.728304] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    3.734538] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.741918] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    3.748065] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    3.754300] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.761682] pci_bus 0000:00: pci_generic_config_read(0x0 0x090 2)
[    3.767829] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x090): [bus 00]
[    3.774063] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.781443] pci_bus 0000:00: pci_generic_config_read(0x0 0x09c 2)
[    3.787590] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x09c): [bus 00]
[    3.793825] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.801205] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    3.807352] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    3.813586] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.820968] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    3.827115] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    3.833350] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.840730] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    3.846877] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    3.853111] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.860491] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    3.866637] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    3.872872] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.880254] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    3.886401] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    3.892636] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.900016] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    3.906163] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    3.912397] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.919778] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    3.925925] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    3.932159] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.939543] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    3.945690] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    3.951924] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.959305] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    3.965452] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    3.971686] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.979066] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    3.985213] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    3.991447] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    3.998829] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    4.004976] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    4.011211] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.018591] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    4.024738] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    4.030973] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.038353] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    4.044500] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    4.050734] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.058114] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    4.064261] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    4.070495] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.077877] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    4.084024] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    4.090259] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.097642] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    4.103789] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    4.110023] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.117403] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    4.123550] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    4.129784] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.137166] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    4.143314] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    4.149548] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.156929] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    4.163076] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    4.169310] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.176690] pci_bus 0000:00: pci_generic_config_read(0x0 0x090 2)
[    4.182837] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x090): [bus 00]
[    4.189071] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.196453] pci_bus 0000:00: pci_generic_config_read(0x0 0x092 2)
[    4.202600] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x092): [bus 00]
[    4.208835] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.216216] pci 0000:00:00.0: PME# supported from D0 D3hot
[    4.221754] pci_bus 0000:00: pci_generic_config_read(0x0 0x094 2)
[    4.227901] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x094): [bus 00]
[    4.234135] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.241516] pci_bus 0000:00: pci_generic_config_write(0x0 0x094 2)
[    4.247750] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x094): [bus 00]
[    4.253985] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.261368] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    4.267515] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    4.273749] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.281130] pci_bus 0000:00: pci_generic_config_read(0x0 0x006 2)
[    4.287277] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x006): [bus 00]
[    4.293511] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.300891] pci_bus 0000:00: pci_generic_config_read(0x0 0x034 1)
[    4.307038] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x034): [bus 00]
[    4.313273] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.320655] pci_bus 0000:00: pci_generic_config_read(0x0 0x040 2)
[    4.326802] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x040): [bus 00]
[    4.333036] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.340417] pci_bus 0000:00: pci_generic_config_read(0x0 0x080 2)
[    4.346563] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x080): [bus 00]
[    4.352798] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.360178] pci_bus 0000:00: pci_generic_config_read(0x0 0x090 2)
[    4.366325] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x090): [bus 00]
[    4.372559] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.379941] pci_bus 0000:00: pci_generic_config_read(0x0 0x09c 2)
[    4.386088] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x09c): [bus 00]
[    4.392322] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.399703] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    4.405850] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    4.412085] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.419465] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    4.425612] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    4.431846] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.439228] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    4.445376] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    4.451610] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.458992] pci_bus 0000:00: pci_generic_config_read(0x0 0x148 2)
[    4.465139] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x148): [bus 00]
[    4.471373] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.478754] pci_bus 0000:00: pci_generic_config_write(0x0 0x148 2)
[    4.484988] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x148): [bus 00]
[    4.491222] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.498604] pci_bus 0000:00: pci_generic_config_read(0x0 0x14e 2)
[    4.504752] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x14e): [bus 00]
[    4.510986] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.518367] pci_bus 0000:00: pci_generic_config_read(0x0 0x15c 4)
[    4.524514] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x15c): [bus 00]
[    4.530748] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.538128] pci_bus 0000:00: pci_generic_config_write(0x0 0x160 4)
[    4.544362] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x160): [bus 00]
[    4.550597] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.557979] pci 0000:00:00.0: VF(n) BAR0 space: [mem 
0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
[    4.567995] pci 0000:00:00.0: VF(n) BAR2 space: [mem 
0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
[    4.578446] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    4.584593] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.590828] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.598208] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    4.604443] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.610677] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.618057] pci_bus 0000:00: pci_generic_config_read(0x0 0x174 4)
[    4.624204] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x174): [bus 00]
[    4.630438] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.637821] pci_bus 0000:00: pci_generic_config_write(0x0 0x174 4)
[    4.644055] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x174): [bus 00]
[    4.650290] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.657670] pci_bus 0000:00: pci_generic_config_read(0x0 0x174 4)
[    4.663817] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x174): [bus 00]
[    4.670052] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.677431] pci_bus 0000:00: pci_generic_config_write(0x0 0x174 4)
[    4.683666] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x174): [bus 00]
[    4.689901] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.697283] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    4.703518] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.709752] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.717133] pci_bus 0000:00: pci_generic_config_read(0x0 0x004 2)
[    4.723279] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.729513] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.736893] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    4.743128] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.749363] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.756745] pci_bus 0000:00: pci_generic_config_read(0x0 0x178 4)
[    4.762892] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x178): [bus 00]
[    4.769127] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.776507] pci_bus 0000:00: pci_generic_config_write(0x0 0x178 4)
[    4.782741] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x178): [bus 00]
[    4.788976] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.796356] pci_bus 0000:00: pci_generic_config_read(0x0 0x178 4)
[    4.802503] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x178): [bus 00]
[    4.808738] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.816120] pci_bus 0000:00: pci_generic_config_write(0x0 0x178 4)
[    4.822355] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x178): [bus 00]
[    4.828589] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.835969] pci_bus 0000:00: pci_generic_config_write(0x0 0x004 2)
[    4.842204] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x004): [bus 00]
[    4.848438] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.855818] pci_bus 0000:00: pci_generic_config_read(0x0 0x15a 2)
[    4.861965] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x15a): [bus 00]
[    4.868199] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.875582] pci_bus 0000:00: pci_generic_config_read(0x0 0x144 4)
[    4.881729] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x144): [bus 00]
[    4.887963] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.895344] pci_bus 0000:00: pci_generic_config_read(0x0 0x152 1)
[    4.901491] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x152): [bus 00]
[    4.907725] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.915105] pci_bus 0000:00: pci_generic_config_write(0x0 0x150 2)
[    4.921340] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x150): [bus 00]
[    4.927574] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.934956] pci_bus 0000:00: pci_generic_config_read(0x0 0x154 2)
[    4.941104] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x154): [bus 00]
[    4.947338] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.954718] pci_bus 0000:00: pci_generic_config_read(0x0 0x156 2)
[    4.960865] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x156): [bus 00]
[    4.967099] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.974480] pci_bus 0000:00: pci_generic_config_write(0x0 0x150 2)
[    4.980714] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x150): [bus 00]
[    4.986948] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    4.994331] pci_bus 0000:00: pci_generic_config_read(0x0 0x154 2)
[    5.000478] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x154): [bus 00]
[    5.006712] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.014093] pci_bus 0000:00: pci_generic_config_read(0x0 0x156 2)
[    5.020239] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x156): [bus 00]
[    5.026474] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.033854] pci_bus 0000:00: pci_generic_config_write(0x0 0x150 2)
[    5.040089] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x150): [bus 00]
[    5.046323] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.053703] pci_bus 0000:00: pci_generic_config_read(0x0 0x154 2)
[    5.059850] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x154): [bus 00]
[    5.066084] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.073467] pci_bus 0000:00: pci_generic_config_read(0x0 0x156 2)
[    5.079614] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x156): [bus 00]
[    5.085848] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.093229] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    5.099376] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    5.105610] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.112990] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    5.119137] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    5.125372] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.132754] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    5.138901] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    5.145135] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.152515] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    5.158662] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    5.164897] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.172277] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    5.178423] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    5.184658] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.192039] pci_bus 0000:00: pci_generic_config_read(0x0 0x140 4)
[    5.198186] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x140): [bus 00]
[    5.204420] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.211801] pci_bus 0000:00: pci_generic_config_read(0x0 0x100 4)
[    5.217948] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x100): [bus 00]
[    5.224182] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.231562] pci_bus 0000:00: pci_generic_config_read(0x0 0x130 4)
[    5.237709] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x130): [bus 00]
[    5.243943] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.251327] pci_bus 0000:00: pci_generic_config_read(0x0 0x044 4)
[    5.257474] pci_bus 0000:00: pci_ecam_map_bus(0x0 0x044): [bus 00]
[    5.263708] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.271245] pci_bus 0000:00: pci_generic_config_read(0x1 0x000 4)
[    5.277392] pci_bus 0000:00: pci_ecam_map_bus(0x1 0x000): [bus 00]
[    5.283627] pci_bus 0000:00: pci_ecam_map_bus base ffff8000123b1000 
bus_shift 0
[    5.291007] Unable to handle kernel paging request at virtual address 
ffff8000123b2000
[    5.298996] Mem abort info:
[    5.301807]   ESR = 0x96000007
[    5.304883]   EC = 0x25: DABT (current EL), IL = 32 bits
[    5.310240]   SET = 0, FnV = 0
[    5.313315]   EA = 0, S1PTW = 0
[    5.316477] Data abort info:
[    5.319376]   ISV = 0, ISS = 0x00000007
[    5.323240]   CM = 0, WnR = 0
[    5.326228] swapper pgtable: 4k pages, 48-bit VAs, 
pgdp=0000000083741000
[    5.332989] [ffff8000123b2000] pgd=00000020fffff003, 
p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffff8003, 
pte=0000000000000000
[    5.345641] Internal error: Oops: 96000007 [#1] PREEMPT SMP
[    5.351262] Modules linked in:
[    5.354342] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 
5.10.0-rc7-next-20201208-dirty #207
[    5.362595] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC 
Eval 2.0 carrier (DT)
[    5.370937] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
[    5.377005] pc : pci_generic_config_read+0xdc/0x110
[    5.381925] lr : pci_generic_config_read+0x70/0x110
[    5.386842] sp : ffff80001239b8f0
[    5.390181] x29: ffff80001239b8f0 x28: 0000000000000000
[    5.395541] x27: ffff8000117504e4 x26: 0000000000000000
[    5.400902] x25: 0000000000000001 x24: ffff80001239ba44
[    5.406262] x23: 0000000000000000 x22: 0000000000000001
[    5.411622] x21: ffff002001490000 x20: 0000000000000004
[    5.416982] x19: ffff80001239b984 x18: 0000000000000010
[    5.422342] x17: 0000000000007fff x16: 00000000ffffffff
[    5.427702] x15: ffffffffffffffff x14: ffff800011f69948
[    5.433062] x13: ffff80009239b4f7 x12: ffff800012004188
[    5.438421] x11: 0000000000000003 x10: ffff800011fec148
[    5.443781] x9 : ffff800010101b10 x8 : 0000000000017fe8
[    5.449141] x7 : c0000000ffffefff x6 : 0000000000000001
[    5.454501] x5 : 0000000000000000 x4 : ffff8000123b2000
[    5.459860] x3 : 00000000fffffff8 x2 : 0000000001000000
[    5.465220] x1 : 0000000000000000 x0 : ffff8000123b2000
[    5.470580] Call trace:
[    5.473043]  pci_generic_config_read+0xdc/0x110
[    5.477614]  pci_bus_read_config_dword+0x8c/0xe0
[    5.482271]  pci_bus_generic_read_dev_vendor_id+0x3c/0x1b8
[    5.487804]  pci_bus_read_dev_vendor_id+0x54/0x78
[    5.492548]  pci_scan_single_device+0x8c/0xe8
[    5.496941]  pci_scan_slot+0x74/0x128
[    5.500632]  pci_scan_child_bus_extend+0x5c/0x2a0
[    5.505376]  pci_scan_root_bus_bridge+0xdc/0xf8
[    5.509944]  pci_host_probe+0x20/0xb8
[    5.513635]  pci_host_common_probe+0x144/0x1a8
[    5.518116]  platform_drv_probe+0x5c/0xb0
[    5.522158]  really_probe+0xec/0x3c0
[    5.525761]  driver_probe_device+0x60/0xc0
[    5.529891]  device_driver_attach+0x7c/0x88
[    5.534108]  __driver_attach+0x60/0xe8
[    5.537886]  bus_for_each_dev+0x7c/0xd0
[    5.541752]  driver_attach+0x2c/0x38
[    5.545355]  bus_add_driver+0x194/0x1f8
[    5.549221]  driver_register+0x6c/0x128
[    5.553087]  __platform_driver_register+0x50/0x60
[    5.557831]  gen_pci_driver_init+0x24/0x30
[    5.561962]  do_one_initcall+0x54/0x2d0
[    5.565830]  kernel_init_freeable+0x1ec/0x258
[    5.570223]  kernel_init+0x1c/0x120
[    5.573738]  ret_from_fork+0x10/0x30
[    5.577342] Code: b5000000 52800000 b9000261 14000008 (b9400001)
[    5.583496] ---[ end trace c0958e83ad22f359 ]---
[    5.588152] note: swapper/0[1] exited with preempt_count 1
[    5.593687] Kernel panic - not syncing: Attempted to kill init! 
exitcode=0x0000000b
[    5.601414] SMP: stopping secondary CPUs
[    5.605371] Kernel Offset: disabled
[    5.608885] CPU features: 0x00240022,61006008
[    5.613276] Memory Limit: none
[    5.616355] ---[ end Kernel panic - not syncing: Attempted to kill 
init! exitcode=0x0000000b ]---

-michael

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 15:40           ` Michael Walle
@ 2020-12-09 20:29             ` Vladimir Oltean
  2020-12-09 20:59               ` Bjorn Helgaas
  2020-12-10 17:38               ` Bjorn Helgaas
  0 siblings, 2 replies; 32+ messages in thread
From: Vladimir Oltean @ 2020-12-09 20:29 UTC (permalink / raw)
  To: Michael Walle
  Cc: Bjorn Helgaas, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 04:40:52PM +0100, Michael Walle wrote:
> Hopefully my mail client won't mess up the output that much.

I can reproduce on my LS1028A as well. The following fixes the bug for
me. I did not follow the discussion and see if it is helpful for others.
I don't understand how the bug came to be. There might be more to it
than what I'm seeing. If it's just what I'm seeing, then the patch was
pretty broken to begin with.

-----------------------------[cut here]-----------------------------
From b184da4088c9d39d25fee2486941cdf77688a409 Mon Sep 17 00:00:00 2001
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Date: Wed, 9 Dec 2020 22:17:32 +0200
Subject: [PATCH] PCI: fix invalid window size for the ECAM config space

The blamed commit forgot that pci_ecam_create() calculates the size of
the window for the ECAM's config space based on the spacing between two
buses. The drivers whose .bus_shift from struct pci_ecam_ops was changed
to zero in this commit are now using this invalid value for bus_shift
in calculating the window size.

Before (broken):
pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x1
After (fixed/restored):
pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x100000

Fixes: f3c07cf6924e ("PCI: Unify ECAM constants in native PCI Express drivers")
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/pci/ecam.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 59f91d434859..9fda0d49bc93 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -28,11 +28,19 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 		struct resource *cfgres, struct resource *busr,
 		const struct pci_ecam_ops *ops)
 {
+	unsigned int bus_shift = ops->bus_shift;
 	struct pci_config_window *cfg;
 	unsigned int bus_range, bus_range_max, bsz;
 	struct resource *conflict;
 	int i, err;
 
+	/*
+	 * struct pci_ecam_ops may omit specifying bus_shift
+	 * if it is as per spec
+	 */
+	if (!bus_shift)
+		bus_shift = PCIE_ECAM_BUS_SHIFT;
+
 	if (busr->start > busr->end)
 		return ERR_PTR(-EINVAL);
 
@@ -46,14 +54,14 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 	cfg->busr.end = busr->end;
 	cfg->busr.flags = IORESOURCE_BUS;
 	bus_range = resource_size(&cfg->busr);
-	bus_range_max = resource_size(cfgres) >> ops->bus_shift;
+	bus_range_max = resource_size(cfgres) >> bus_shift;
 	if (bus_range > bus_range_max) {
 		bus_range = bus_range_max;
 		cfg->busr.end = busr->start + bus_range - 1;
 		dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
 			 cfgres, &cfg->busr, busr);
 	}
-	bsz = 1 << ops->bus_shift;
+	bsz = 1 << bus_shift;
 
 	cfg->res.start = cfgres->start;
 	cfg->res.end = cfgres->end;
-----------------------------[cut here]-----------------------------

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 20:29             ` Vladimir Oltean
@ 2020-12-09 20:59               ` Bjorn Helgaas
  2020-12-09 21:20                 ` Vladimir Oltean
  2020-12-10 17:38               ` Bjorn Helgaas
  1 sibling, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-09 20:59 UTC (permalink / raw)
  To: Vladimir Oltean
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 10:29:04PM +0200, Vladimir Oltean wrote:
> On Wed, Dec 09, 2020 at 04:40:52PM +0100, Michael Walle wrote:
> > Hopefully my mail client won't mess up the output that much.
> 
> I can reproduce on my LS1028A as well. The following fixes the bug for
> me. I did not follow the discussion and see if it is helpful for others.
> I don't understand how the bug came to be. There might be more to it
> than what I'm seeing. If it's just what I'm seeing, then the patch was
> pretty broken to begin with.
> 
> -----------------------------[cut here]-----------------------------
> From b184da4088c9d39d25fee2486941cdf77688a409 Mon Sep 17 00:00:00 2001
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> Date: Wed, 9 Dec 2020 22:17:32 +0200
> Subject: [PATCH] PCI: fix invalid window size for the ECAM config space
> 
> The blamed commit forgot that pci_ecam_create() calculates the size of
> the window for the ECAM's config space based on the spacing between two
> buses. The drivers whose .bus_shift from struct pci_ecam_ops was changed
> to zero in this commit are now using this invalid value for bus_shift
> in calculating the window size.
> 
> Before (broken):
> pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x1
> After (fixed/restored):
> pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x100000
> 
> Fixes: f3c07cf6924e ("PCI: Unify ECAM constants in native PCI Express drivers")
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  drivers/pci/ecam.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index 59f91d434859..9fda0d49bc93 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -28,11 +28,19 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  		struct resource *cfgres, struct resource *busr,
>  		const struct pci_ecam_ops *ops)
>  {
> +	unsigned int bus_shift = ops->bus_shift;
>  	struct pci_config_window *cfg;
>  	unsigned int bus_range, bus_range_max, bsz;
>  	struct resource *conflict;
>  	int i, err;
>  
> +	/*
> +	 * struct pci_ecam_ops may omit specifying bus_shift
> +	 * if it is as per spec
> +	 */
> +	if (!bus_shift)
> +		bus_shift = PCIE_ECAM_BUS_SHIFT;

Yep, that's the theory.  Thanks for testing it!

>  	if (busr->start > busr->end)
>  		return ERR_PTR(-EINVAL);
>  
> @@ -46,14 +54,14 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  	cfg->busr.end = busr->end;
>  	cfg->busr.flags = IORESOURCE_BUS;
>  	bus_range = resource_size(&cfg->busr);
> -	bus_range_max = resource_size(cfgres) >> ops->bus_shift;
> +	bus_range_max = resource_size(cfgres) >> bus_shift;
>  	if (bus_range > bus_range_max) {
>  		bus_range = bus_range_max;
>  		cfg->busr.end = busr->start + bus_range - 1;
>  		dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
>  			 cfgres, &cfg->busr, busr);
>  	}
> -	bsz = 1 << ops->bus_shift;
> +	bsz = 1 << bus_shift;
>  
>  	cfg->res.start = cfgres->start;
>  	cfg->res.end = cfgres->end;
> -----------------------------[cut here]-----------------------------

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 20:59               ` Bjorn Helgaas
@ 2020-12-09 21:20                 ` Vladimir Oltean
  2020-12-09 21:34                   ` Bjorn Helgaas
  0 siblings, 1 reply; 32+ messages in thread
From: Vladimir Oltean @ 2020-12-09 21:20 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 02:59:13PM -0600, Bjorn Helgaas wrote:
> Yep, that's the theory.  Thanks for testing it!

Testing what? I'm not following.

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 21:20                 ` Vladimir Oltean
@ 2020-12-09 21:34                   ` Bjorn Helgaas
  2020-12-09 21:43                     ` Vladimir Oltean
  0 siblings, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-09 21:34 UTC (permalink / raw)
  To: Vladimir Oltean
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 11:20:17PM +0200, Vladimir Oltean wrote:
> On Wed, Dec 09, 2020 at 02:59:13PM -0600, Bjorn Helgaas wrote:
> > Yep, that's the theory.  Thanks for testing it!
> 
> Testing what? I'm not following.

You posted a patch that you said fixed the bug for you.  The fix is
exactly the theory we've been discussing, so you have already verified
that the theory is correct.

I'm sure Krzysztof will update his patch, and we'll get this tidied up
in -next again.

Bjorn

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 21:34                   ` Bjorn Helgaas
@ 2020-12-09 21:43                     ` Vladimir Oltean
  2020-12-09 22:05                       ` Bjorn Helgaas
  0 siblings, 1 reply; 32+ messages in thread
From: Vladimir Oltean @ 2020-12-09 21:43 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 03:34:49PM -0600, Bjorn Helgaas wrote:
> On Wed, Dec 09, 2020 at 11:20:17PM +0200, Vladimir Oltean wrote:
> > On Wed, Dec 09, 2020 at 02:59:13PM -0600, Bjorn Helgaas wrote:
> > > Yep, that's the theory.  Thanks for testing it!
> >
> > Testing what? I'm not following.
>
> You posted a patch that you said fixed the bug for you.  The fix is
> exactly the theory we've been discussing, so you have already verified
> that the theory is correct.
>
> I'm sure Krzysztof will update his patch, and we'll get this tidied up
> in -next again.

If you were discussing this already, I missed it. I was copied to this
thread out of the blue two emails ago. I also looked at the full thread
on patchwork, I don't see anything being said about the culprit being
the size of the config space mapping.

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 21:43                     ` Vladimir Oltean
@ 2020-12-09 22:05                       ` Bjorn Helgaas
  0 siblings, 0 replies; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-09 22:05 UTC (permalink / raw)
  To: Vladimir Oltean
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 11:43:59PM +0200, Vladimir Oltean wrote:
> On Wed, Dec 09, 2020 at 03:34:49PM -0600, Bjorn Helgaas wrote:
> > On Wed, Dec 09, 2020 at 11:20:17PM +0200, Vladimir Oltean wrote:
> > > On Wed, Dec 09, 2020 at 02:59:13PM -0600, Bjorn Helgaas wrote:
> > > > Yep, that's the theory.  Thanks for testing it!
> > >
> > > Testing what? I'm not following.
> >
> > You posted a patch that you said fixed the bug for you.  The fix is
> > exactly the theory we've been discussing, so you have already verified
> > that the theory is correct.
> >
> > I'm sure Krzysztof will update his patch, and we'll get this tidied up
> > in -next again.
> 
> If you were discussing this already, I missed it. I was copied to this
> thread out of the blue two emails ago. I also looked at the full thread
> on patchwork, I don't see anything being said about the culprit being
> the size of the config space mapping.

Oh, sorry, this was an IRC discussion on #linux-pci (OFTC):

  10:51 AM <bjorn_> so the fault is on the first read for 00:00.1.  forget my noise about extracting the device/func from the *virtual* address.  the *physical* address is supposed to be aligned so you can do that, but not the virtual address
  10:55 AM <bjorn_> kwilczynski: oh, i think i see it: pci_ecam_create() does "bsz = 1 << ops->bus_shift", but we removed .bus_shift for this case
  10:55 AM <bjorn_> needs to default to 20 if it's not specified
  10:56 AM <bjorn_> result is that we only map one page of the ECAM space, so we fault when we access the second page (which is where 00:00.1 starts)

Anyway, thanks very much again for fixing this and confirming the fix!

Bjorn

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-09 20:29             ` Vladimir Oltean
  2020-12-09 20:59               ` Bjorn Helgaas
@ 2020-12-10 17:38               ` Bjorn Helgaas
  2020-12-10 19:58                 ` Michael Walle
  1 sibling, 1 reply; 32+ messages in thread
From: Bjorn Helgaas @ 2020-12-10 17:38 UTC (permalink / raw)
  To: Vladimir Oltean
  Cc: Michael Walle, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

On Wed, Dec 09, 2020 at 10:29:04PM +0200, Vladimir Oltean wrote:
> On Wed, Dec 09, 2020 at 04:40:52PM +0100, Michael Walle wrote:
> > Hopefully my mail client won't mess up the output that much.
> 
> I can reproduce on my LS1028A as well. The following fixes the bug for
> me. I did not follow the discussion and see if it is helpful for others.
> I don't understand how the bug came to be. There might be more to it
> than what I'm seeing. If it's just what I'm seeing, then the patch was
> pretty broken to begin with.

I squashed the fix below into a pci/ecam branch for v5.11, thanks!

> -----------------------------[cut here]-----------------------------
> From b184da4088c9d39d25fee2486941cdf77688a409 Mon Sep 17 00:00:00 2001
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> Date: Wed, 9 Dec 2020 22:17:32 +0200
> Subject: [PATCH] PCI: fix invalid window size for the ECAM config space
> 
> The blamed commit forgot that pci_ecam_create() calculates the size of
> the window for the ECAM's config space based on the spacing between two
> buses. The drivers whose .bus_shift from struct pci_ecam_ops was changed
> to zero in this commit are now using this invalid value for bus_shift
> in calculating the window size.
> 
> Before (broken):
> pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x1
> After (fixed/restored):
> pci_ecam_create: remapping config space from addr 0x1f0000000, bus_range 0x1, bsz 0x100000
> 
> Fixes: f3c07cf6924e ("PCI: Unify ECAM constants in native PCI Express drivers")
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  drivers/pci/ecam.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index 59f91d434859..9fda0d49bc93 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -28,11 +28,19 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  		struct resource *cfgres, struct resource *busr,
>  		const struct pci_ecam_ops *ops)
>  {
> +	unsigned int bus_shift = ops->bus_shift;
>  	struct pci_config_window *cfg;
>  	unsigned int bus_range, bus_range_max, bsz;
>  	struct resource *conflict;
>  	int i, err;
>  
> +	/*
> +	 * struct pci_ecam_ops may omit specifying bus_shift
> +	 * if it is as per spec
> +	 */
> +	if (!bus_shift)
> +		bus_shift = PCIE_ECAM_BUS_SHIFT;
> +
>  	if (busr->start > busr->end)
>  		return ERR_PTR(-EINVAL);
>  
> @@ -46,14 +54,14 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  	cfg->busr.end = busr->end;
>  	cfg->busr.flags = IORESOURCE_BUS;
>  	bus_range = resource_size(&cfg->busr);
> -	bus_range_max = resource_size(cfgres) >> ops->bus_shift;
> +	bus_range_max = resource_size(cfgres) >> bus_shift;
>  	if (bus_range > bus_range_max) {
>  		bus_range = bus_range_max;
>  		cfg->busr.end = busr->start + bus_range - 1;
>  		dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
>  			 cfgres, &cfg->busr, busr);
>  	}
> -	bsz = 1 << ops->bus_shift;
> +	bsz = 1 << bus_shift;
>  
>  	cfg->res.start = cfgres->start;
>  	cfg->res.end = cfgres->end;
> -----------------------------[cut here]-----------------------------

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
  2020-12-10 17:38               ` Bjorn Helgaas
@ 2020-12-10 19:58                 ` Michael Walle
  0 siblings, 0 replies; 32+ messages in thread
From: Michael Walle @ 2020-12-10 19:58 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Vladimir Oltean, lorenzo.pieralisi, kw, heiko, benh, shawn.lin,
	paulus, thomas.petazzoni, jonnyc, toan, will, robh, f.fainelli,
	mpe, michal.simek, linux-rockchip, bcm-kernel-feedback-list,
	jonathan.derrick, linux-pci, rjui, linux-rpi-kernel,
	Jonathan.Cameron, bhelgaas, linux-arm-kernel, sbranden,
	wangzhou1, rrichter, linuxppc-dev, nsaenzjulienne,
	Alexandru Marginean

Am 2020-12-10 18:38, schrieb Bjorn Helgaas:
> On Wed, Dec 09, 2020 at 10:29:04PM +0200, Vladimir Oltean wrote:
>> On Wed, Dec 09, 2020 at 04:40:52PM +0100, Michael Walle wrote:
>> > Hopefully my mail client won't mess up the output that much.
>> 
>> I can reproduce on my LS1028A as well. The following fixes the bug for
>> me. I did not follow the discussion and see if it is helpful for 
>> others.
>> I don't understand how the bug came to be. There might be more to it
>> than what I'm seeing. If it's just what I'm seeing, then the patch was
>> pretty broken to begin with.
> 
> I squashed the fix below into a pci/ecam branch for v5.11, thanks!

FWIW
Tested-by: Michael Walle <michael@walle.cc>

-michael

^ permalink raw reply	[flat|nested] 32+ messages in thread

end of thread, other threads:[~2020-12-10 19:59 UTC | newest]

Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-29 23:07 [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Krzysztof Wilczyński
2020-11-29 23:07 ` [PATCH v6 1/5] " Krzysztof Wilczyński
2020-11-30 11:08   ` Lorenzo Pieralisi
2020-11-30 15:30     ` Krzysztof Wilczyński
2020-11-30 18:23   ` Derrick, Jonathan
2020-12-06 20:16   ` Krzysztof Wilczyński
2020-12-07  3:25     ` Florian Fainelli
2020-12-07 20:29       ` Jim Quinlan
2020-11-29 23:07 ` [PATCH v6 2/5] PCI: thunder-pem: Add constant for custom ".bus_shit" initialiser Krzysztof Wilczyński
2020-11-29 23:07 ` [PATCH v6 3/5] PCI: iproc: Convert to use the new ECAM constants Krzysztof Wilczyński
2020-11-29 23:07 ` [PATCH v6 4/5] PCI: vmd: Update type of the __iomem pointers Krzysztof Wilczyński
2020-11-30  9:06   ` David Laight
2020-11-30 17:20     ` Bjorn Helgaas
2020-11-30 18:19       ` Derrick, Jonathan
2020-11-29 23:07 ` [PATCH v6 5/5] PCI: xgene: Removed unused ".bus_shift" initialisers from pci-xgene.c Krzysztof Wilczyński
2020-12-01 15:34 ` [PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers Lorenzo Pieralisi
2020-12-08 15:41   ` Michael Walle
2020-12-08 21:06     ` Bjorn Helgaas
2020-12-08 21:11       ` Michael Walle
2020-12-09 12:36     ` Bjorn Helgaas
2020-12-09 13:08       ` Michael Walle
2020-12-09 14:57         ` Bjorn Helgaas
2020-12-09 15:40           ` Michael Walle
2020-12-09 20:29             ` Vladimir Oltean
2020-12-09 20:59               ` Bjorn Helgaas
2020-12-09 21:20                 ` Vladimir Oltean
2020-12-09 21:34                   ` Bjorn Helgaas
2020-12-09 21:43                     ` Vladimir Oltean
2020-12-09 22:05                       ` Bjorn Helgaas
2020-12-10 17:38               ` Bjorn Helgaas
2020-12-10 19:58                 ` Michael Walle
2020-12-08 16:07 ` Qian Cai

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).