[PATCH v1 00/10] USB host support for Raspberry Pi 4 board

[PATCH v1 00/10] USB host support for Raspberry Pi 4 board

[Sylwester Nawrocki]

Hi all,

This patch series adds USB host support for Raspberry Pi 4 board. 
It includes the Broadcom STB PCIe driver ported from Linux kernel,
a memory mapping update for the xHCI controller behind a PCIe bridge
for 32-bit and 64-bit system builds and some related fixes and updates
in the usb/xhci and the pci core code.

Any changes since previous (RFC) version are listed in individual 
patches.  The most significant change was rewriting the PCIe driver 
to align with current mainline Linux kernel version.  As suggested, 
readq/writeq have been removed from xhci_{readq/writeq} helpers 
instead of adding a Kconfig option.

This patch series is based on v2020.04 tag.

Regards,
Sylwester

Marek Szyprowski (4):
  rpi4: shorten a mapping for the DRAM
  rpi4: add a mapping for the PCIe XHCI controller MMIO registers (ARM
    64bit)
  rpi4: add a mapping for the PCIe XHCI controller MMIO registers (ARM
    32bit)
  config: Enable support for the XHCI controller on RPI4 board

Nicolas Saenz Julienne (1):
  linux/bitfield.h: Add primitives for manipulating bitfields both in
    host- and fixed-endian.

Sylwester Nawrocki (5):
  usb: xhci: Add missing cache flush in the scratchpad array
    initialization
  usb: xhci: Use only 32-bit accesses in xhci_writeq/xhci_readq
  pci: Move some PCIe register offset definitions to a common header
  pci: Add some PCI Express capability register offset definitions
  pci: Add driver for Broadcom STB PCIe controller

 arch/arm/mach-bcm283x/Kconfig             |   1 +
 arch/arm/mach-bcm283x/include/mach/base.h |   7 +
 arch/arm/mach-bcm283x/init.c              |  72 +++-
 configs/rpi_4_32b_defconfig               |   9 +
 configs/rpi_4_defconfig                   |  10 +
 configs/rpi_arm64_defconfig               |   9 +-
 drivers/pci/Kconfig                       |   6 +
 drivers/pci/Makefile                      |   1 +
 drivers/pci/pci-rcar-gen3.c               |   8 -
 drivers/pci/pcie_brcmstb.c                | 593 ++++++++++++++++++++++++++++++
 drivers/pci/pcie_intel_fpga.c             |   3 -
 drivers/usb/host/xhci-mem.c               |   3 +
 include/linux/bitfield.h                  |  46 +++
 include/pci.h                             |  19 +-
 include/usb/xhci.h                        |   8 -
 15 files changed, 769 insertions(+), 26 deletions(-)
 create mode 100644 drivers/pci/pcie_brcmstb.c

-- 
2.7.4

[PATCH v1 01/10] usb: xhci: Add missing cache flush in the scratchpad array initialization

[Sylwester Nawrocki]

In current code there is no cache flush after initializing the scratchpad
buffer array with the scratchpad buffer pointers. This leads to a failure
of the "slot enable" command on the rpi4 board (Broadcom STB PCIe
controller + VL805 USB hub) - the very first TRB transfer on the command
ring fails and there is a timeout while waiting for the command completion
event. After adding the missing cache flush everything seems to be working
as expected.

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - none.
---
 drivers/usb/host/xhci-mem.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c
index 93450ee..729bdc3 100644
--- a/drivers/usb/host/xhci-mem.c
+++ b/drivers/usb/host/xhci-mem.c
@@ -393,6 +393,9 @@ static int xhci_scratchpad_alloc(struct xhci_ctrl *ctrl)
 		scratchpad->sp_array[i] = cpu_to_le64(ptr);
 	}
 
+	xhci_flush_cache((uintptr_t)scratchpad->sp_array,
+			 sizeof(u64) * num_sp);
+
 	return 0;
 
 fail_sp3:
-- 
2.7.4

[PATCH v1 02/10] usb: xhci: Use only 32-bit accesses in xhci_writeq/xhci_readq

[Sylwester Nawrocki]

There might be hardware configurations where 64-bit data accesses
to XHCI registers are not supported properly.  This patch removes
the readq/writeq so always two 32-bit accesses are used to read/write
64-bit XHCI registers, similarly as it is done in Linux kernel.

This patch fixes operation of the XHCI controller on RPI4 Broadcom
BCM2711 SoC based board, where the VL805 USB XHCI controller is
connected to the PCIe Root Complex, which is attached to the system
through the SCB bridge.

Even though the architecture is 64-bit the PCIe BAR is 32-bit and likely
the 64-bit wide register accesses initiated by the CPU are not properly
translated to a sequence of 32-bit PCIe accesses.
xhci_readq(), for example, always returns same value in upper and lower
32-bits, e.g. 0xabcd1234abcd1234 instead of 0x00000000abcd1234.

Cc: Sergey Temerkhanov <s.temerkhanov@gmail.com>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - dropped Kconfig option, switched to not using readq/writeq
   unconditionally.
---
 include/usb/xhci.h | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/include/usb/xhci.h b/include/usb/xhci.h
index 6017504..c16106a 100644
--- a/include/usb/xhci.h
+++ b/include/usb/xhci.h
@@ -1111,28 +1111,20 @@ static inline void xhci_writel(uint32_t volatile *regs, const unsigned int val)
  */
 static inline u64 xhci_readq(__le64 volatile *regs)
 {
-#if BITS_PER_LONG == 64
-	return readq(regs);
-#else
 	__u32 *ptr = (__u32 *)regs;
 	u64 val_lo = readl(ptr);
 	u64 val_hi = readl(ptr + 1);
 	return val_lo + (val_hi << 32);
-#endif
 }
 
 static inline void xhci_writeq(__le64 volatile *regs, const u64 val)
 {
-#if BITS_PER_LONG == 64
-	writeq(val, regs);
-#else
 	__u32 *ptr = (__u32 *)regs;
 	u32 val_lo = lower_32_bits(val);
 	/* FIXME */
 	u32 val_hi = upper_32_bits(val);
 	writel(val_lo, ptr);
 	writel(val_hi, ptr + 1);
-#endif
 }
 
 int xhci_hcd_init(int index, struct xhci_hccr **ret_hccr,
-- 
2.7.4

[PATCH v1 03/10] pci: Move some PCIe register offset definitions to a common header

[Sylwester Nawrocki]

Some PCI Express register offsets are currently defined in multiple
drivers, move them to a common header to avoid re-definitions and
as a pre-requisite for adding new PCIe driver.
While at it replace some spaces with tabs.

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - whitespace clean up.
---
 drivers/pci/pci-rcar-gen3.c   |  8 --------
 drivers/pci/pcie_intel_fpga.c |  3 ---
 include/pci.h                 | 13 +++++++++++--
 3 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/pci-rcar-gen3.c b/drivers/pci/pci-rcar-gen3.c
index 30eff67..393f1c9 100644
--- a/drivers/pci/pci-rcar-gen3.c
+++ b/drivers/pci/pci-rcar-gen3.c
@@ -117,14 +117,6 @@
 #define RCAR_PCI_MAX_RESOURCES	4
 #define MAX_NR_INBOUND_MAPS	6
 
-#define PCI_EXP_FLAGS		2		/* Capabilities register */
-#define PCI_EXP_FLAGS_TYPE	0x00f0		/* Device/Port type */
-#define PCI_EXP_TYPE_ROOT_PORT	0x4		/* Root Port */
-#define PCI_EXP_LNKCAP		12		/* Link Capabilities */
-#define PCI_EXP_LNKCAP_DLLLARC	0x00100000	/* Data Link Layer Link Active Reporting Capable */
-#define PCI_EXP_SLTCAP		20		/* Slot Capabilities */
-#define PCI_EXP_SLTCAP_PSN	0xfff80000	/* Physical Slot Number */
-
 enum {
 	RCAR_PCI_ACCESS_READ,
 	RCAR_PCI_ACCESS_WRITE,
diff --git a/drivers/pci/pcie_intel_fpga.c b/drivers/pci/pcie_intel_fpga.c
index 6a9f29c..69363a0 100644
--- a/drivers/pci/pcie_intel_fpga.c
+++ b/drivers/pci/pcie_intel_fpga.c
@@ -65,9 +65,6 @@
 #define IS_ROOT_PORT(pcie, bdf)				\
 		((PCI_BUS(bdf) == pcie->first_busno) ? true : false)
 
-#define PCI_EXP_LNKSTA		18	/* Link Status */
-#define PCI_EXP_LNKSTA_DLLLA	0x2000	/* Data Link Layer Link Active */
-
 /**
  * struct intel_fpga_pcie - Intel FPGA PCIe controller state
  * @bus: Pointer to the PCI bus
diff --git a/include/pci.h b/include/pci.h
index 174ddd4..5bf91a4 100644
--- a/include/pci.h
+++ b/include/pci.h
@@ -471,10 +471,19 @@
 #define  PCI_EA_FIELD_MASK	0xfffffffc	/* For Base & Max Offset */
 
 /* PCI Express capabilities */
+#define PCI_EXP_FLAGS		2	/* Capabilities register */
+#define  PCI_EXP_FLAGS_TYPE	0x00f0	/* Device/Port type */
+#define  PCI_EXP_TYPE_ROOT_PORT 0x4	/* Root Port */
 #define PCI_EXP_DEVCAP		4	/* Device capabilities */
-#define  PCI_EXP_DEVCAP_FLR     0x10000000 /* Function Level Reset */
+#define  PCI_EXP_DEVCAP_FLR	0x10000000 /* Function Level Reset */
 #define PCI_EXP_DEVCTL		8	/* Device Control */
-#define  PCI_EXP_DEVCTL_BCR_FLR 0x8000  /* Bridge Configuration Retry / FLR */
+#define  PCI_EXP_DEVCTL_BCR_FLR	0x8000  /* Bridge Configuration Retry / FLR */
+#define PCI_EXP_LNKCAP		12	/* Link Capabilities */
+#define  PCI_EXP_LNKCAP_DLLLARC	0x00100000 /* Data Link Layer Link Active Reporting Capable */
+#define PCI_EXP_LNKSTA		18	/* Link Status */
+#define  PCI_EXP_LNKSTA_DLLLA	0x2000	/* Data Link Layer Link Active */
+#define PCI_EXP_SLTCAP		20	/* Slot Capabilities */
+#define  PCI_EXP_SLTCAP_PSN	0xfff80000 /* Physical Slot Number */
 
 /* Include the ID list */
 
-- 
2.7.4

[PATCH v1 04/10] rpi4: shorten a mapping for the DRAM

[Sylwester Nawrocki]

From: Marek Szyprowski <m.szyprowski@samsung.com>

Remove the overlap between DRAM and device's IO area.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
Changes since RFC:
 - none.
---
 arch/arm/mach-bcm283x/init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/mach-bcm283x/init.c b/arch/arm/mach-bcm283x/init.c
index 9966d6c..4295356 100644
--- a/arch/arm/mach-bcm283x/init.c
+++ b/arch/arm/mach-bcm283x/init.c
@@ -38,7 +38,7 @@ static struct mm_region bcm2711_mem_map[] = {
 	{
 		.virt = 0x00000000UL,
 		.phys = 0x00000000UL,
-		.size = 0xfe000000UL,
+		.size = 0xfc000000UL,
 		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
 			 PTE_BLOCK_INNER_SHARE
 	}, {
-- 
2.7.4

[PATCH v1 05/10] rpi4: add a mapping for the PCIe XHCI controller MMIO registers (ARM 64bit)

[Sylwester Nawrocki]

From: Marek Szyprowski <m.szyprowski@samsung.com>

Create a non-cacheable mapping for the 0x600000000 physical memory region,
where MMIO registers for the PCIe XHCI controller are instantiated by the
PCIe bridge.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
Changes since RFC:
 - none.
---
 arch/arm/mach-bcm283x/init.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/arch/arm/mach-bcm283x/init.c b/arch/arm/mach-bcm283x/init.c
index 4295356..6a748da 100644
--- a/arch/arm/mach-bcm283x/init.c
+++ b/arch/arm/mach-bcm283x/init.c
@@ -11,10 +11,15 @@
 #include <dm/device.h>
 #include <fdt_support.h>
 
+#define BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS	0x600000000UL
+#define BCM2711_RPI4_PCIE_XHCI_MMIO_SIZE	0x800000UL
+
 #ifdef CONFIG_ARM64
 #include <asm/armv8/mmu.h>
 
-static struct mm_region bcm283x_mem_map[] = {
+#define MAX_MAP_MAX_ENTRIES (4)
+
+static struct mm_region bcm283x_mem_map[MAX_MAP_MAX_ENTRIES] = {
 	{
 		.virt = 0x00000000UL,
 		.phys = 0x00000000UL,
@@ -34,7 +39,7 @@ static struct mm_region bcm283x_mem_map[] = {
 	}
 };
 
-static struct mm_region bcm2711_mem_map[] = {
+static struct mm_region bcm2711_mem_map[MAX_MAP_MAX_ENTRIES] = {
 	{
 		.virt = 0x00000000UL,
 		.phys = 0x00000000UL,
@@ -49,6 +54,13 @@ static struct mm_region bcm2711_mem_map[] = {
 			 PTE_BLOCK_NON_SHARE |
 			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
 	}, {
+		.virt = BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS,
+		.phys = BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS,
+		.size = BCM2711_RPI4_PCIE_XHCI_MMIO_SIZE,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
 		/* List terminator */
 		0,
 	}
@@ -71,7 +83,7 @@ static void _rpi_update_mem_map(struct mm_region *pd)
 {
 	int i;
 
-	for (i = 0; i < 2; i++) {
+	for (i = 0; i < MAX_MAP_MAX_ENTRIES; i++) {
 		mem_map[i].virt = pd[i].virt;
 		mem_map[i].phys = pd[i].phys;
 		mem_map[i].size = pd[i].size;
-- 
2.7.4

[PATCH v1 06/10] rpi4: add a mapping for the PCIe XHCI controller MMIO registers (ARM 32bit)

[Sylwester Nawrocki]

From: Marek Szyprowski <m.szyprowski@samsung.com>

Create a non-cacheable mapping for the 0x600000000 physical memory region,
where MMIO registers for the PCIe XHCI controller are instantiated by the
PCIe bridge. Due to 32bit limit in the CPU virtual address space in ARM
32bit mode, this region is mapped at 0xff800000 CPU virtual address.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
Changes since RFC:
 - none.
---
 arch/arm/mach-bcm283x/Kconfig             |  1 +
 arch/arm/mach-bcm283x/include/mach/base.h |  7 +++++
 arch/arm/mach-bcm283x/init.c              | 52 +++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+)

diff --git a/arch/arm/mach-bcm283x/Kconfig b/arch/arm/mach-bcm283x/Kconfig
index 00419bf..bcb7f1d 100644
--- a/arch/arm/mach-bcm283x/Kconfig
+++ b/arch/arm/mach-bcm283x/Kconfig
@@ -36,6 +36,7 @@ config BCM2711_32B
 	select BCM2711
 	select ARMV7_LPAE
 	select CPU_V7A
+	select PHYS_64BIT
 
 config BCM2711_64B
 	bool "Broadcom BCM2711 SoC 64-bit support"
diff --git a/arch/arm/mach-bcm283x/include/mach/base.h b/arch/arm/mach-bcm283x/include/mach/base.h
index c4ae398..1d10dc9 100644
--- a/arch/arm/mach-bcm283x/include/mach/base.h
+++ b/arch/arm/mach-bcm283x/include/mach/base.h
@@ -6,6 +6,13 @@
 #ifndef _BCM283x_BASE_H_
 #define _BCM283x_BASE_H_
 
+#include <linux/types.h>
+
 extern unsigned long rpi_bcm283x_base;
 
+#ifdef CONFIG_ARMV7_LPAE
+extern void *rpi4_phys_to_virt(phys_addr_t paddr);
+#define phys_to_virt(x) rpi4_phys_to_virt(x)
+#endif
+
 #endif
diff --git a/arch/arm/mach-bcm283x/init.c b/arch/arm/mach-bcm283x/init.c
index 6a748da..5d0d160 100644
--- a/arch/arm/mach-bcm283x/init.c
+++ b/arch/arm/mach-bcm283x/init.c
@@ -145,6 +145,58 @@ int mach_cpu_init(void)
 }
 
 #ifdef CONFIG_ARMV7_LPAE
+
+#define BCM2711_RPI4_PCIE_XHCI_MMIO_VIRT	0xff800000UL
+
+void *rpi4_phys_to_virt(phys_addr_t paddr)
+{
+	if (paddr >= BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS)
+		paddr = paddr - BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS +
+			BCM2711_RPI4_PCIE_XHCI_MMIO_VIRT;
+	return (void *)(unsigned long)paddr;
+}
+
+static void set_section_phys(unsigned int section, phys_addr_t phys,
+			     enum dcache_option option)
+{
+	u64 *page_table = (u64 *)gd->arch.tlb_addr;
+	/* Need to set the access flag to not fault */
+	u64 value = TTB_SECT_AP | TTB_SECT_AF;
+
+	/* Add the page offset */
+	value |= (phys);
+
+	/* Add caching bits */
+	value |= option;
+
+	/* Set PTE */
+	page_table[section] = value;
+}
+
+static void rpi4_create_pcie_xhci_mapping(void)
+{
+	unsigned sect = BCM2711_RPI4_PCIE_XHCI_MMIO_VIRT >> MMU_SECTION_SHIFT;
+	phys_addr_t phys_addr = BCM2711_RPI4_PCIE_XHCI_MMIO_PHYS;
+	unsigned int size = BCM2711_RPI4_PCIE_XHCI_MMIO_SIZE;
+
+	while (size) {
+		set_section_phys(sect, phys_addr, DCACHE_OFF);
+		sect++;
+		phys_addr += MMU_SECTION_SIZE;
+		size -= MMU_SECTION_SIZE;
+	}
+}
+
+void arm_init_domains(void)
+{
+	/*
+	 * Hijack this function to prepare a mappings for the PCIe MMIO
+	 * region for the XHCI controller on RPi4 board.
+	 * This code is called before enabling the MMU in ARM 32bit mode.
+	 */
+	rpi4_create_pcie_xhci_mapping();
+}
+
 void enable_caches(void)
 {
 	dcache_enable();
-- 
2.7.4

[PATCH v1 07/10] linux/bitfield.h: Add primitives for manipulating bitfields both in host- and fixed-endian.

[Sylwester Nawrocki]

From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Imports Al Viro's original Linux commit 00b0c9b82663a, which contains
an in depth explanation and two fixes from Johannes Berg:
 e7d4a95da86e0 "bitfield: fix *_encode_bits()",
 37a3862e12382 "bitfield: add u8 helpers".

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - new patch.
---
 include/linux/bitfield.h | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/include/linux/bitfield.h b/include/linux/bitfield.h
index 8b9d6ff..4964213 100644
--- a/include/linux/bitfield.h
+++ b/include/linux/bitfield.h
@@ -103,4 +103,50 @@
 		(typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask));	\
 	})
 
+extern void __compiletime_error("value doesn't fit into mask")
+__field_overflow(void);
+extern void __compiletime_error("bad bitfield mask")
+__bad_mask(void);
+static __always_inline u64 field_multiplier(u64 field)
+{
+	if ((field | (field - 1)) & ((field | (field - 1)) + 1))
+		__bad_mask();
+	return field & -field;
+}
+static __always_inline u64 field_mask(u64 field)
+{
+	return field / field_multiplier(field);
+}
+#define ____MAKE_OP(type,base,to,from)					\
+static __always_inline __##type type##_encode_bits(base v, base field)	\
+{									\
+	if (__builtin_constant_p(v) && (v & ~field_mask(field)))	\
+		__field_overflow();					\
+	return to((v & field_mask(field)) * field_multiplier(field));	\
+}									\
+static __always_inline __##type type##_replace_bits(__##type old,	\
+					base val, base field)		\
+{									\
+	return (old & ~to(field)) | type##_encode_bits(val, field);	\
+}									\
+static __always_inline void type##p_replace_bits(__##type *p,		\
+					base val, base field)		\
+{									\
+	*p = (*p & ~to(field)) | type##_encode_bits(val, field);	\
+}									\
+static __always_inline base type##_get_bits(__##type v, base field)	\
+{									\
+	return (from(v) & field)/field_multiplier(field);		\
+}
+#define __MAKE_OP(size)							\
+	____MAKE_OP(le##size,u##size,cpu_to_le##size,le##size##_to_cpu)	\
+	____MAKE_OP(be##size,u##size,cpu_to_be##size,be##size##_to_cpu)	\
+	____MAKE_OP(u##size,u##size,,)
+____MAKE_OP(u8,u8,,)
+__MAKE_OP(16)
+__MAKE_OP(32)
+__MAKE_OP(64)
+#undef __MAKE_OP
+#undef ____MAKE_OP
+
 #endif
-- 
2.7.4

[PATCH v1 08/10] pci: Add some PCI Express capability register offset definitions

[Sylwester Nawrocki]

Add PCI Express capability definitions required by the Broadcom
STB PCIe controller driver.

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - ensure the entries are added in order, sorted by ascending
   address values.
---
 include/pci.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/pci.h b/include/pci.h
index 5bf91a4..5307478 100644
--- a/include/pci.h
+++ b/include/pci.h
@@ -479,11 +479,17 @@
 #define PCI_EXP_DEVCTL		8	/* Device Control */
 #define  PCI_EXP_DEVCTL_BCR_FLR	0x8000  /* Bridge Configuration Retry / FLR */
 #define PCI_EXP_LNKCAP		12	/* Link Capabilities */
+#define  PCI_EXP_LNKCAP_SLS	0x0000000f /* Supported Link Speeds */
+#define  PCI_EXP_LNKCAP_MLW	0x000003f0 /* Maximum Link Width */
 #define  PCI_EXP_LNKCAP_DLLLARC	0x00100000 /* Data Link Layer Link Active Reporting Capable */
 #define PCI_EXP_LNKSTA		18	/* Link Status */
+#define  PCI_EXP_LNKSTA_CLS	0x000f	/* Current Link Speed */
+#define  PCI_EXP_LNKSTA_NLW	0x03f0	/* Negotiated Link Width */
+#define  PCI_EXP_LNKSTA_NLW_SHIFT 4	/* start of NLW mask in link status */
 #define  PCI_EXP_LNKSTA_DLLLA	0x2000	/* Data Link Layer Link Active */
 #define PCI_EXP_SLTCAP		20	/* Slot Capabilities */
 #define  PCI_EXP_SLTCAP_PSN	0xfff80000 /* Physical Slot Number */
+#define PCI_EXP_LNKCTL2		48	/* Link Control 2 */
 
 /* Include the ID list */
 
-- 
2.7.4

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Sylwester Nawrocki]

This patch adds basic driver for the Broadcom STB PCIe host controller.
The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
handling removed. The inbound access memory region is not currently
parsed from dma-ranges DT property and a fixed 4GB region is used.

The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
USB Host Controller.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
---
Changes since RFC:
 - reworked to align with current Linux mainline version and u-boot driver
   by Nicolas Saenz Julienne
---
 drivers/pci/Kconfig        |   6 +
 drivers/pci/Makefile       |   1 +
 drivers/pci/pcie_brcmstb.c | 593 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 600 insertions(+)
 create mode 100644 drivers/pci/pcie_brcmstb.c

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 437cd9a..056a021 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -197,4 +197,10 @@ config PCIE_MEDIATEK
 	  Say Y here if you want to enable Gen2 PCIe controller,
 	  which could be found on MT7623 SoC family.
 
+config PCI_BRCMSTB
+	bool "Broadcom STB PCIe controller"
+	depends on DM_PCI
+	depends on ARCH_BCM283X
+	help
+	  Say Y here if you want to enable Broadcom STB PCIe controller support.
 endif
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index c051ecc..3e53b1f 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -43,3 +43,4 @@ obj-$(CONFIG_PCI_PHYTIUM) += pcie_phytium.o
 obj-$(CONFIG_PCIE_INTEL_FPGA) += pcie_intel_fpga.o
 obj-$(CONFIG_PCI_KEYSTONE) += pcie_dw_ti.o
 obj-$(CONFIG_PCIE_MEDIATEK) += pcie_mediatek.o
+obj-$(CONFIG_PCI_BRCMSTB) += pcie_brcmstb.o
diff --git a/drivers/pci/pcie_brcmstb.c b/drivers/pci/pcie_brcmstb.c
new file mode 100644
index 0000000..dfe9833
--- /dev/null
+++ b/drivers/pci/pcie_brcmstb.c
@@ -0,0 +1,593 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Broadcom STB PCIe controller driver
+ *
+ * Copyright (C) 2020 Samsung Electronics Co., Ltd.
+ *
+ * Based on upstream Linux kernel driver:
+ * drivers/pci/controller/pcie-brcmstb.c
+ * Copyright (C) 2009 - 2017 Broadcom
+ *
+ * Based driver by Nicolas Saenz Julienne
+ * Copyright (C) 2020 Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
+ */
+
+#include <asm/io.h>
+#include <common.h>
+#include <dm.h>
+#include <dm/ofnode.h>
+#include <errno.h>
+#include <linux/bitfield.h>
+#include <linux/log2.h>
+#include <pci.h>
+
+/* BRCM_PCIE_CAP_REGS - Offset for the mandatory capability config regs */
+#define BRCM_PCIE_CAP_REGS				0x00ac
+
+/* Broadcom STB PCIe Register Offsets */
+#define PCIE_RC_CFG_VENDOR_VENDOR_SPECIFIC_REG1				0x0188
+#define  PCIE_RC_CFG_VENDOR_VENDOR_SPECIFIC_REG1_ENDIAN_MODE_BAR2_MASK	0xc
+#define  PCIE_RC_CFG_VENDOR_SPCIFIC_REG1_LITTLE_ENDIAN			0x0
+
+#define PCIE_RC_CFG_PRIV1_ID_VAL3			0x043c
+#define  PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK	0xffffff
+
+#define PCIE_RC_DL_MDIO_ADDR				0x1100
+#define PCIE_RC_DL_MDIO_WR_DATA				0x1104
+#define PCIE_RC_DL_MDIO_RD_DATA				0x1108
+
+#define PCIE_MISC_MISC_CTRL				0x4008
+#define  PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK		0x1000
+#define  PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK	0x2000
+#define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
+#define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
+#define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
+
+#define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO		0x400c
+#define PCIE_MEM_WIN0_LO(win)	\
+		PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO + ((win) * 4)
+
+#define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_HI		0x4010
+#define PCIE_MEM_WIN0_HI(win)	\
+		PCIE_MISC_CPU_2_PCIE_MEM_WIN0_HI + ((win) * 4)
+
+#define PCIE_MISC_RC_BAR1_CONFIG_LO			0x402c
+#define  PCIE_MISC_RC_BAR1_CONFIG_LO_SIZE_MASK		0x1f
+
+#define PCIE_MISC_RC_BAR2_CONFIG_LO			0x4034
+#define  PCIE_MISC_RC_BAR2_CONFIG_LO_SIZE_MASK		0x1f
+#define PCIE_MISC_RC_BAR2_CONFIG_HI			0x4038
+
+#define PCIE_MISC_RC_BAR3_CONFIG_LO			0x403c
+#define  PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK		0x1f
+
+#define PCIE_MISC_PCIE_STATUS				0x4068
+#define  PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK		0x80
+#define  PCIE_MISC_PCIE_STATUS_PCIE_DL_ACTIVE_MASK	0x20
+#define  PCIE_MISC_PCIE_STATUS_PCIE_PHYLINKUP_MASK	0x10
+#define  PCIE_MISC_PCIE_STATUS_PCIE_LINK_IN_L23_MASK	0x40
+
+#define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT		 0x4070
+#define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_LIMIT_MASK	 0xfff00000
+#define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_MASK	 0xfff0
+#define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_HI_SHIFT	 12
+#define PCIE_MEM_WIN0_BASE_LIMIT(win)	\
+		PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT + ((win) * 4)
+
+#define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_HI			 0x4080
+#define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_HI_BASE_MASK	 0xff
+#define PCIE_MEM_WIN0_BASE_HI(win)	\
+		PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_HI + ((win) * 8)
+
+#define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LIMIT_HI			 0x4084
+#define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LIMIT_HI_LIMIT_MASK	 0xff
+#define PCIE_MEM_WIN0_LIMIT_HI(win)	\
+		PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LIMIT_HI + ((win) * 8)
+
+#define PCIE_MISC_HARD_PCIE_HARD_DEBUG				 0x4204
+#define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK 0x2
+#define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK	 0x08000000
+
+#define PCIE_MSI_INTR2_CLR				0x4508
+#define PCIE_MSI_INTR2_MASK_SET				0x4510
+
+#define PCIE_EXT_CFG_DATA				0x8000
+
+#define PCIE_EXT_CFG_INDEX				0x9000
+#define  PCIE_EXT_BUSNUM_SHIFT				20
+#define  PCIE_EXT_SLOT_SHIFT				15
+#define  PCIE_EXT_FUNC_SHIFT				12
+
+#define PCIE_RGR1_SW_INIT_1				0x9210
+#define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
+#define  PCIE_RGR1_SW_INIT_1_INIT_MASK			0x2
+
+/* PCIe parameters */
+#define BRCM_NUM_PCIE_OUT_WINS		0x4
+
+/* MDIO registers */
+#define MDIO_PORT0			0x0
+#define MDIO_DATA_MASK			0x7fffffff
+#define MDIO_PORT_MASK			0xf0000
+#define MDIO_REGAD_MASK			0xffff
+#define MDIO_CMD_MASK			0xfff00000
+#define MDIO_CMD_READ			0x1
+#define MDIO_CMD_WRITE			0x0
+#define MDIO_DATA_DONE_MASK		0x80000000
+#define MDIO_RD_DONE(x)			(((x) & MDIO_DATA_DONE_MASK) ? 1 : 0)
+#define MDIO_WT_DONE(x)			(((x) & MDIO_DATA_DONE_MASK) ? 0 : 1)
+#define SSC_REGS_ADDR			0x1100
+#define SET_ADDR_OFFSET			0x1f
+#define SSC_CNTL_OFFSET			0x2
+#define SSC_CNTL_OVRD_EN_MASK		0x8000
+#define SSC_CNTL_OVRD_VAL_MASK		0x4000
+#define SSC_STATUS_OFFSET		0x1
+#define SSC_STATUS_SSC_MASK		0x400
+#define SSC_STATUS_PLL_LOCK_MASK	0x800
+
+struct brcm_pcie {
+	void __iomem		*base;
+
+	int			gen;
+	bool			ssc;
+};
+
+#define msleep(a) udelay((a) * 1000)
+
+/*
+ * This is to convert the size of the inbound "BAR" region to the
+ * non-linear values of PCIE_X_MISC_RC_BAR[123]_CONFIG_LO.SIZE
+ */
+static int brcm_pcie_encode_ibar_size(u64 size)
+{
+	int log2_in = ilog2(size);
+
+	if (log2_in >= 12 && log2_in <= 15)
+		/* Covers 4KB to 32KB (inclusive) */
+		return (log2_in - 12) + 0x1c;
+	else if (log2_in >= 16 && log2_in <= 37)
+		/* Covers 64KB to 32GB, (inclusive) */
+		return log2_in - 15;
+	/* Something is awry so disable */
+	return 0;
+}
+
+/* Configuration space read/write support */
+static inline int brcm_pcie_cfg_index(pci_dev_t bdf, int reg)
+{
+	return (PCI_DEV(bdf) << PCIE_EXT_SLOT_SHIFT)
+		| (PCI_FUNC(bdf) << PCIE_EXT_FUNC_SHIFT)
+		| (PCI_BUS(bdf) << PCIE_EXT_BUSNUM_SHIFT)
+		| (reg & ~3);
+}
+
+/* The controller is capable of serving in both RC and EP roles */
+static bool brcm_pcie_rc_mode(struct brcm_pcie *pcie)
+{
+	u32 val = readl(pcie->base + PCIE_MISC_PCIE_STATUS);
+
+	return !!FIELD_GET(PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK, val);
+}
+
+static bool brcm_pcie_link_up(struct brcm_pcie *pcie)
+{
+	u32 val = readl(pcie->base + PCIE_MISC_PCIE_STATUS);
+	u32 dla = FIELD_GET(PCIE_MISC_PCIE_STATUS_PCIE_DL_ACTIVE_MASK, val);
+	u32 plu = FIELD_GET(PCIE_MISC_PCIE_STATUS_PCIE_PHYLINKUP_MASK, val);
+
+	return dla && plu;
+}
+
+static int brcm_pcie_config_address(const struct udevice *udev, pci_dev_t bdf,
+				    uint offset, void **paddress)
+{
+	struct brcm_pcie *pcie = dev_get_priv(udev);
+	unsigned int bus = PCI_BUS(bdf);
+	unsigned int dev = PCI_DEV(bdf);
+	int idx;
+
+	/*
+	 * Busses 0 (host PCIe bridge) and 1 (its immediate child)
+	 * are limited to a single device each
+	 */
+	if ((bus == (udev->seq + 1)) && dev > 0)
+		return -ENODEV;
+
+	/* Accesses to the RC go right to the RC registers if PCI device == 0 */
+	if (bus == udev->seq) {
+		if (PCI_DEV(bdf))
+			return -ENODEV;
+
+		*paddress = pcie->base + offset;
+		return 0;
+	}
+
+	/* For devices, write to the config space index register */
+	idx = brcm_pcie_cfg_index(bdf, 0);
+
+	writel(idx, pcie->base + PCIE_EXT_CFG_INDEX);
+	*paddress = pcie->base + PCIE_EXT_CFG_DATA + offset;
+
+	return 0;
+}
+
+static int brcm_pcie_read_config(const struct udevice *bus, pci_dev_t bdf,
+				 uint offset, ulong *valuep,
+				 enum pci_size_t size)
+{
+	return pci_generic_mmap_read_config(bus, brcm_pcie_config_address,
+					    bdf, offset, valuep, size);
+}
+
+static int brcm_pcie_write_config(struct udevice *bus, pci_dev_t bdf,
+				  uint offset, ulong value,
+				  enum pci_size_t size)
+{
+	return pci_generic_mmap_write_config(bus, brcm_pcie_config_address,
+					     bdf, offset, value, size);
+}
+
+static const char *link_speed_to_str(unsigned int s)
+{
+	static const char * const speed_str[] = { "??", "2.5", "5.0", "8.0" };
+
+	if (s >= ARRAY_SIZE(speed_str))
+		s = 0;
+
+	return speed_str[s];
+}
+
+static inline void brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val)
+{
+	u32 tmp;
+
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
+	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_INIT_MASK);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+}
+
+static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
+{
+	u32 tmp;
+
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
+	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+}
+
+static u32 brcm_pcie_mdio_form_pkt(int port, int regad, int cmd)
+{
+	u32 pkt = 0;
+
+	pkt |= FIELD_PREP(MDIO_PORT_MASK, port);
+	pkt |= FIELD_PREP(MDIO_REGAD_MASK, regad);
+	pkt |= FIELD_PREP(MDIO_CMD_MASK, cmd);
+
+	return pkt;
+}
+
+/* Negative return value indicates error */
+static int brcm_pcie_mdio_read(void __iomem *base, u8 port, u8 regad, u32 *val)
+{
+	int tries;
+	u32 data;
+
+	writel(brcm_pcie_mdio_form_pkt(port, regad, MDIO_CMD_READ),
+	       base + PCIE_RC_DL_MDIO_ADDR);
+	readl(base + PCIE_RC_DL_MDIO_ADDR);
+
+	data = readl(base + PCIE_RC_DL_MDIO_RD_DATA);
+	for (tries = 0; !MDIO_RD_DONE(data) && tries < 10; tries++) {
+		udelay(10);
+		data = readl(base + PCIE_RC_DL_MDIO_RD_DATA);
+	}
+
+	*val = FIELD_GET(MDIO_DATA_MASK, data);
+	return MDIO_RD_DONE(data) ? 0 : -EIO;
+}
+
+/* Negative return value indicates error */
+static int brcm_pcie_mdio_write(void __iomem *base, u8 port,
+				u8 regad, u16 wrdata)
+{
+	int tries;
+	u32 data;
+
+	writel(brcm_pcie_mdio_form_pkt(port, regad, MDIO_CMD_WRITE),
+	       base + PCIE_RC_DL_MDIO_ADDR);
+	readl(base + PCIE_RC_DL_MDIO_ADDR);
+	writel(MDIO_DATA_DONE_MASK | wrdata, base + PCIE_RC_DL_MDIO_WR_DATA);
+
+	data = readl(base + PCIE_RC_DL_MDIO_WR_DATA);
+	for (tries = 0; !MDIO_WT_DONE(data) && tries < 10; tries++) {
+		udelay(10);
+		data = readl(base + PCIE_RC_DL_MDIO_WR_DATA);
+	}
+
+	return MDIO_WT_DONE(data) ? 0 : -EIO;
+}
+
+/*
+ * Configures device for Spread Spectrum Clocking (SSC) mode; negative
+ * return value indicates error.
+ */
+static int brcm_pcie_set_ssc(struct brcm_pcie *pcie)
+{
+	void __iomem *base = pcie->base;
+	int pll, ssc;
+	int ret;
+	u32 tmp;
+
+	ret = brcm_pcie_mdio_write(base, MDIO_PORT0, SET_ADDR_OFFSET,
+				   SSC_REGS_ADDR);
+	if (ret < 0)
+		return ret;
+
+	ret = brcm_pcie_mdio_read(base, MDIO_PORT0, SSC_CNTL_OFFSET, &tmp);
+	if (ret < 0)
+		return ret;
+
+	u32p_replace_bits(&tmp, 1, SSC_CNTL_OVRD_EN_MASK);
+	u32p_replace_bits(&tmp, 1, SSC_CNTL_OVRD_VAL_MASK);
+	ret = brcm_pcie_mdio_write(base, MDIO_PORT0, SSC_CNTL_OFFSET, tmp);
+	if (ret < 0)
+		return ret;
+
+	udelay(1000);
+	ret = brcm_pcie_mdio_read(base, MDIO_PORT0, SSC_STATUS_OFFSET, &tmp);
+	if (ret < 0)
+		return ret;
+
+	ssc = FIELD_GET(SSC_STATUS_SSC_MASK, tmp);
+	pll = FIELD_GET(SSC_STATUS_PLL_LOCK_MASK, tmp);
+
+	return ssc && pll ? 0 : -EIO;
+}
+
+/* Limits operation to a specific generation (1, 2, or 3) */
+static void brcm_pcie_set_gen(struct brcm_pcie *pcie, int gen)
+{
+	void __iomem *base = pcie->base;
+
+	u16 lnkctl2 = readw(base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCTL2);
+	u32 lnkcap = readl(base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCAP);
+
+	lnkcap = (lnkcap & ~PCI_EXP_LNKCAP_SLS) | gen;
+	writel(lnkcap, base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCAP);
+
+	lnkctl2 = (lnkctl2 & ~0xf) | gen;
+	writew(lnkctl2, base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKCTL2);
+}
+
+static void brcm_pcie_set_outbound_win(struct brcm_pcie *pcie,
+				       unsigned int win, u64 phys_addr,
+				       u64 pcie_addr, u64 size)
+{
+	void __iomem *base = pcie->base;
+	u32 phys_addr_mb_high, limit_addr_mb_high;
+	phys_addr_t phys_addr_mb, limit_addr_mb;
+	int high_addr_shift;
+	u32 tmp;
+
+	/* Set the base of the pcie_addr window */
+	writel(lower_32_bits(pcie_addr), base + PCIE_MEM_WIN0_LO(win));
+	writel(upper_32_bits(pcie_addr), base + PCIE_MEM_WIN0_HI(win));
+
+	/* Write the addr base & limit lower bits (in MBs) */
+	phys_addr_mb = phys_addr / SZ_1M;
+	limit_addr_mb = (phys_addr + size - 1) / SZ_1M;
+
+	tmp = readl(base + PCIE_MEM_WIN0_BASE_LIMIT(win));
+	u32p_replace_bits(&tmp, phys_addr_mb,
+			  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_MASK);
+	u32p_replace_bits(&tmp, limit_addr_mb,
+			  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_LIMIT_MASK);
+	writel(tmp, base + PCIE_MEM_WIN0_BASE_LIMIT(win));
+
+	/* Write the cpu & limit addr upper bits */
+	high_addr_shift = PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_HI_SHIFT;
+	phys_addr_mb_high = phys_addr_mb >> high_addr_shift;
+	tmp = readl(base + PCIE_MEM_WIN0_BASE_HI(win));
+	u32p_replace_bits(&tmp, phys_addr_mb_high,
+			  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_HI_BASE_MASK);
+	writel(tmp, base + PCIE_MEM_WIN0_BASE_HI(win));
+
+	limit_addr_mb_high = limit_addr_mb >> high_addr_shift;
+	tmp = readl(base + PCIE_MEM_WIN0_LIMIT_HI(win));
+	u32p_replace_bits(&tmp, limit_addr_mb_high,
+			  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LIMIT_HI_LIMIT_MASK);
+	writel(tmp, base + PCIE_MEM_WIN0_LIMIT_HI(win));
+}
+
+static int brcm_pcie_probe(struct udevice *dev)
+{
+	struct udevice *ctlr = pci_get_controller(dev);
+	struct pci_controller *hose = dev_get_uclass_priv(ctlr);
+	struct brcm_pcie *pcie = dev_get_priv(dev);
+	void __iomem *base = pcie->base;
+	bool ssc_good = false;
+	int num_out_wins = 0;
+	u64 rc_bar2_offset, rc_bar2_size;
+	unsigned int scb_size_val;
+	int i, ret;
+	u16 nlw, cls, lnksta;
+	u32 tmp;
+
+	/* Reset the bridge */
+	brcm_pcie_bridge_sw_init_set(pcie, 1);
+
+	udelay(150);
+
+	/* Take the bridge out of reset */
+	brcm_pcie_bridge_sw_init_set(pcie, 0);
+
+	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+	tmp &= ~PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK;
+	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+	/* Wait for SerDes to be stable */
+	udelay(150);
+
+	/* Set SCB_MAX_BURST_SIZE, CFG_READ_UR_MODE, SCB_ACCESS_EN */
+	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK);
+	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK);
+	u32p_replace_bits(&tmp, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128,
+			  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
+	writel(tmp, base + PCIE_MISC_MISC_CTRL);
+
+	/*
+	 * TODO: Use the base address and size(s) provided in the dma-ranges
+	 * property.
+	 */
+	rc_bar2_offset = 0;
+	rc_bar2_size = 1ULL << 32;
+
+	tmp = lower_32_bits(rc_bar2_offset);
+	u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(rc_bar2_size),
+			  PCIE_MISC_RC_BAR2_CONFIG_LO_SIZE_MASK);
+	writel(tmp, base + PCIE_MISC_RC_BAR2_CONFIG_LO);
+	writel(upper_32_bits(rc_bar2_offset),
+	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
+
+	scb_size_val = rc_bar2_size ?
+		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
+	tmp = readl(base + PCIE_MISC_MISC_CTRL);
+	u32p_replace_bits(&tmp, scb_size_val,
+			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
+	writel(tmp, base + PCIE_MISC_MISC_CTRL);
+
+	/* Disable the PCIe->GISB memory window (RC_BAR1) */
+	tmp = readl(base + PCIE_MISC_RC_BAR1_CONFIG_LO);
+	tmp &= ~PCIE_MISC_RC_BAR1_CONFIG_LO_SIZE_MASK;
+	writel(tmp, base + PCIE_MISC_RC_BAR1_CONFIG_LO);
+
+	/* Disable the PCIe->SCB memory window (RC_BAR3) */
+	tmp = readl(base + PCIE_MISC_RC_BAR3_CONFIG_LO);
+	tmp &= ~PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK;
+	writel(tmp, base + PCIE_MISC_RC_BAR3_CONFIG_LO);
+
+	/* Mask all interrupts since we are not handling any yet */
+	writel(0xffffffff, base + PCIE_MSI_INTR2_MASK_SET);
+
+	/* Clear any interrupts we find on boot */
+	writel(0xffffffff, base + PCIE_MSI_INTR2_CLR);
+
+	if (pcie->gen)
+		brcm_pcie_set_gen(pcie, pcie->gen);
+
+	/* Unassert the fundamental reset */
+	brcm_pcie_perst_set(pcie, 0);
+
+	/* Give the RC/EP time to wake up, before trying to configure RC.
+	 * Intermittently check status for link-up, up to a total of 100ms.
+	 */
+	for (i = 0; i < 100 && !brcm_pcie_link_up(pcie); i += 5)
+		msleep(5);
+
+	if (!brcm_pcie_link_up(pcie)) {
+		printf("PCIe BRCM: link down\n");
+		return -ENODEV;
+	}
+
+	if (!brcm_pcie_rc_mode(pcie)) {
+		printf("PCIe misconfigured; is in EP mode\n");
+		return -EINVAL;
+	}
+
+	for (i = 0; i < hose->region_count; i++) {
+		struct pci_region *reg = &hose->regions[i];
+
+		if (reg->flags != PCI_REGION_MEM)
+			continue;
+
+		if (num_out_wins >= BRCM_NUM_PCIE_OUT_WINS)
+			return -EINVAL;
+
+		brcm_pcie_set_outbound_win(pcie, num_out_wins, reg->phys_start,
+					   reg->bus_start, reg->size);
+
+		num_out_wins++;
+	}
+
+	/*
+	 * For config space accesses on the RC, show the right class for
+	 * a PCIe-PCIe bridge (the default setting is to be EP mode).
+	 */
+	tmp = readl(base + PCIE_RC_CFG_PRIV1_ID_VAL3);
+	u32p_replace_bits(&tmp, 0x060400,
+			  PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK);
+	writel(tmp, base + PCIE_RC_CFG_PRIV1_ID_VAL3);
+
+	if (pcie->ssc) {
+		ret = brcm_pcie_set_ssc(base);
+		if (ret == 0)
+			ssc_good = true;
+		else
+			printf("PCIe BRCM: failed attempt to enter SSC mode\n");
+	}
+
+	lnksta = readw(base + BRCM_PCIE_CAP_REGS + PCI_EXP_LNKSTA);
+	cls = FIELD_GET(PCI_EXP_LNKSTA_CLS, lnksta);
+	nlw = FIELD_GET(PCI_EXP_LNKSTA_NLW, lnksta);
+
+	printf("PCIe BRCM: link up, %s Gbps x%u %s\n", link_speed_to_str(cls),
+	       nlw, ssc_good ? "(SSC)" : "(!SSC)");
+
+	/* PCIe->SCB endian mode for BAR */
+	tmp = readl(base + PCIE_RC_CFG_VENDOR_VENDOR_SPECIFIC_REG1);
+	u32p_replace_bits(&tmp, PCIE_RC_CFG_VENDOR_SPCIFIC_REG1_LITTLE_ENDIAN,
+		PCIE_RC_CFG_VENDOR_VENDOR_SPECIFIC_REG1_ENDIAN_MODE_BAR2_MASK);
+	writel(tmp, base + PCIE_RC_CFG_VENDOR_VENDOR_SPECIFIC_REG1);
+
+	/*
+	 * Refclk from RC should be gated with CLKREQ# input when ASPM L0s,L1
+	 * is enabled => setting the CLKREQ_DEBUG_ENABLE field to 1.
+	 */
+	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+	tmp |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK;
+	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+
+	return 0;
+}
+
+static int brcm_pcie_ofdata_to_platdata(struct udevice *dev)
+{
+	struct brcm_pcie *pcie = dev_get_priv(dev);
+	ofnode dn = dev_ofnode(dev);
+	u32 max_link_speed;
+	int ret;
+
+	/* Get the controller base address */
+	pcie->base = dev_read_addr_ptr(dev);
+	if (!pcie->base)
+		return -EINVAL;
+
+	pcie->ssc = ofnode_read_bool(dn, "brcm,enable-ssc");
+
+	ret = ofnode_read_u32(dn, "max-link-speed", &max_link_speed);
+	if (ret < 0 || max_link_speed > 4)
+		pcie->gen = 0;
+	else
+		pcie->gen = max_link_speed;
+
+	return 0;
+}
+
+static const struct dm_pci_ops brcm_pcie_ops = {
+	.read_config	= brcm_pcie_read_config,
+	.write_config	= brcm_pcie_write_config,
+};
+
+static const struct udevice_id brcm_pcie_ids[] = {
+	{ .compatible = "brcm,bcm2711-pcie" },
+	{ }
+};
+
+U_BOOT_DRIVER(pcie_brcm_base) = {
+	.name			= "pcie_brcm",
+	.id			= UCLASS_PCI,
+	.ops			= &brcm_pcie_ops,
+	.of_match		= brcm_pcie_ids,
+	.probe			= brcm_pcie_probe,
+	.ofdata_to_platdata	= brcm_pcie_ofdata_to_platdata,
+	.priv_auto_alloc_size	= sizeof(struct brcm_pcie),
+};
-- 
2.7.4

[PATCH v1 10/10] config: Enable support for the XHCI controller on RPI4 board

[Sylwester Nawrocki]

From: Marek Szyprowski <m.szyprowski@samsung.com>

This requires enabling BRCMSTB PCIe and XHCI_PCI drivers as well as PCI
and USB commands. To get it working one has to call the following commands:
"pci enum; usb start;", thus such commands have been added to the default
"preboot" environment variable. One has to update their environment if it
is already configured to get this feature working out of the box.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
Changes since RFC:
 - none.
---
 configs/rpi_4_32b_defconfig |  9 +++++++++
 configs/rpi_4_defconfig     | 10 ++++++++++
 configs/rpi_arm64_defconfig |  9 ++++++++-
 3 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/configs/rpi_4_32b_defconfig b/configs/rpi_4_32b_defconfig
index 72cda5d..0dd763f 100644
--- a/configs/rpi_4_32b_defconfig
+++ b/configs/rpi_4_32b_defconfig
@@ -8,6 +8,8 @@ CONFIG_NR_DRAM_BANKS=2
 CONFIG_DISTRO_DEFAULTS=y
 CONFIG_OF_BOARD_SETUP=y
 # CONFIG_ARCH_FIXUP_FDT_MEMORY is not set
+CONFIG_USE_PREBOOT=y
+CONFIG_PREBOOT="pci enum; usb start;"
 CONFIG_MISC_INIT_R=y
 # CONFIG_DISPLAY_CPUINFO is not set
 # CONFIG_DISPLAY_BOARDINFO is not set
@@ -15,6 +17,8 @@ CONFIG_SYS_PROMPT="U-Boot> "
 CONFIG_CMD_DFU=y
 CONFIG_CMD_GPIO=y
 CONFIG_CMD_MMC=y
+CONFIG_CMD_PCI=y
+CONFIG_CMD_USB=y
 CONFIG_CMD_FS_UUID=y
 CONFIG_OF_BOARD=y
 CONFIG_ENV_FAT_INTERFACE="mmc"
@@ -28,12 +32,17 @@ CONFIG_MMC_SDHCI=y
 CONFIG_MMC_SDHCI_BCM2835=y
 CONFIG_DM_ETH=y
 CONFIG_BCMGENET=y
+CONFIG_PCI=y
+CONFIG_DM_PCI=y
+CONFIG_PCI_BRCMSTB=y
 CONFIG_PINCTRL=y
 # CONFIG_PINCTRL_GENERIC is not set
 # CONFIG_REQUIRE_SERIAL_CONSOLE is not set
 CONFIG_USB=y
 CONFIG_DM_USB=y
 CONFIG_DM_USB_GADGET=y
+CONFIG_USB_XHCI_HCD=y
+CONFIG_USB_XHCI_PCI=y
 CONFIG_USB_GADGET=y
 CONFIG_USB_GADGET_MANUFACTURER="FSL"
 CONFIG_USB_GADGET_VENDOR_NUM=0x0525
diff --git a/configs/rpi_4_defconfig b/configs/rpi_4_defconfig
index 6d148da..f80e5da 100644
--- a/configs/rpi_4_defconfig
+++ b/configs/rpi_4_defconfig
@@ -8,6 +8,8 @@ CONFIG_NR_DRAM_BANKS=2
 CONFIG_DISTRO_DEFAULTS=y
 CONFIG_OF_BOARD_SETUP=y
 # CONFIG_ARCH_FIXUP_FDT_MEMORY is not set
+CONFIG_USE_PREBOOT=y
+CONFIG_PREBOOT="pci enum; usb start;"
 CONFIG_MISC_INIT_R=y
 # CONFIG_DISPLAY_CPUINFO is not set
 # CONFIG_DISPLAY_BOARDINFO is not set
@@ -15,6 +17,8 @@ CONFIG_SYS_PROMPT="U-Boot> "
 CONFIG_CMD_DFU=y
 CONFIG_CMD_GPIO=y
 CONFIG_CMD_MMC=y
+CONFIG_CMD_PCI=y
+CONFIG_CMD_USB=y
 CONFIG_CMD_FS_UUID=y
 CONFIG_OF_BOARD=y
 CONFIG_ENV_FAT_INTERFACE="mmc"
@@ -28,12 +32,18 @@ CONFIG_MMC_SDHCI=y
 CONFIG_MMC_SDHCI_BCM2835=y
 CONFIG_DM_ETH=y
 CONFIG_BCMGENET=y
+CONFIG_PCI=y
+CONFIG_DM_PCI=y
+CONFIG_PCI_BRCMSTB=y
 CONFIG_PINCTRL=y
 # CONFIG_PINCTRL_GENERIC is not set
 # CONFIG_REQUIRE_SERIAL_CONSOLE is not set
 CONFIG_USB=y
 CONFIG_DM_USB=y
 CONFIG_DM_USB_GADGET=y
+CONFIG_USB_XHCI_HCD=y
+CONFIG_XHCI_64BIT_DWORD_ACCESS_ONLY=y
+CONFIG_USB_XHCI_PCI=y
 CONFIG_USB_GADGET=y
 CONFIG_USB_GADGET_MANUFACTURER="FSL"
 CONFIG_USB_GADGET_VENDOR_NUM=0x0525
diff --git a/configs/rpi_arm64_defconfig b/configs/rpi_arm64_defconfig
index fea86be..926dfc3 100644
--- a/configs/rpi_arm64_defconfig
+++ b/configs/rpi_arm64_defconfig
@@ -7,13 +7,14 @@ CONFIG_NR_DRAM_BANKS=2
 CONFIG_DISTRO_DEFAULTS=y
 CONFIG_OF_BOARD_SETUP=y
 CONFIG_USE_PREBOOT=y
-CONFIG_PREBOOT="usb start"
+CONFIG_PREBOOT="pci enum; usb start;"
 CONFIG_MISC_INIT_R=y
 # CONFIG_DISPLAY_CPUINFO is not set
 # CONFIG_DISPLAY_BOARDINFO is not set
 CONFIG_SYS_PROMPT="U-Boot> "
 CONFIG_CMD_GPIO=y
 CONFIG_CMD_MMC=y
+CONFIG_CMD_PCI=y
 CONFIG_CMD_USB=y
 CONFIG_CMD_FS_UUID=y
 CONFIG_OF_BOARD=y
@@ -26,11 +27,17 @@ CONFIG_MMC_SDHCI=y
 CONFIG_MMC_SDHCI_BCM2835=y
 CONFIG_DM_ETH=y
 CONFIG_BCMGENET=y
+CONFIG_PCI=y
+CONFIG_DM_PCI=y
+CONFIG_PCI_BRCMSTB=y
 CONFIG_PINCTRL=y
 # CONFIG_PINCTRL_GENERIC is not set
 # CONFIG_REQUIRE_SERIAL_CONSOLE is not set
 CONFIG_USB=y
 CONFIG_DM_USB=y
+CONFIG_USB_XHCI_HCD=y
+CONFIG_XHCI_64BIT_DWORD_ACCESS_ONLY=y
+CONFIG_USB_XHCI_PCI=y
 CONFIG_USB_DWC2=y
 CONFIG_USB_KEYBOARD=y
 CONFIG_USB_HOST_ETHER=y
-- 
2.7.4

[PATCH v1 02/10] usb: xhci: Use only 32-bit accesses in xhci_writeq/xhci_readq

[Bin Meng]

On Sat, Apr 25, 2020 at 12:51 AM Sylwester Nawrocki
<s.nawrocki@samsung.com> wrote:
>
> There might be hardware configurations where 64-bit data accesses
> to XHCI registers are not supported properly.  This patch removes
> the readq/writeq so always two 32-bit accesses are used to read/write
> 64-bit XHCI registers, similarly as it is done in Linux kernel.
>
> This patch fixes operation of the XHCI controller on RPI4 Broadcom
> BCM2711 SoC based board, where the VL805 USB XHCI controller is
> connected to the PCIe Root Complex, which is attached to the system
> through the SCB bridge.
>
> Even though the architecture is 64-bit the PCIe BAR is 32-bit and likely
> the 64-bit wide register accesses initiated by the CPU are not properly
> translated to a sequence of 32-bit PCIe accesses.
> xhci_readq(), for example, always returns same value in upper and lower
> 32-bits, e.g. 0xabcd1234abcd1234 instead of 0x00000000abcd1234.
>
> Cc: Sergey Temerkhanov <s.temerkhanov@gmail.com>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> ---
> Changes since RFC:
>  - dropped Kconfig option, switched to not using readq/writeq
>    unconditionally.
> ---
>  include/usb/xhci.h | 8 --------
>  1 file changed, 8 deletions(-)
>

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

[PATCH v1 07/10] linux/bitfield.h: Add primitives for manipulating bitfields both in host- and fixed-endian.

[Bin Meng]

On Sat, Apr 25, 2020 at 12:52 AM Sylwester Nawrocki
<s.nawrocki@samsung.com> wrote:
>
> From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
>
> Imports Al Viro's original Linux commit 00b0c9b82663a, which contains
> an in depth explanation and two fixes from Johannes Berg:
>  e7d4a95da86e0 "bitfield: fix *_encode_bits()",
>  37a3862e12382 "bitfield: add u8 helpers".
>
> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>

nits: remove the ending period in the commit title

> ---
> Changes since RFC:
>  - new patch.
> ---
>  include/linux/bitfield.h | 46 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
>
> diff --git a/include/linux/bitfield.h b/include/linux/bitfield.h
> index 8b9d6ff..4964213 100644
> --- a/include/linux/bitfield.h
> +++ b/include/linux/bitfield.h
> @@ -103,4 +103,50 @@
>                 (typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)); \
>         })
>
> +extern void __compiletime_error("value doesn't fit into mask")
> +__field_overflow(void);
> +extern void __compiletime_error("bad bitfield mask")
> +__bad_mask(void);
> +static __always_inline u64 field_multiplier(u64 field)
> +{
> +       if ((field | (field - 1)) & ((field | (field - 1)) + 1))
> +               __bad_mask();
> +       return field & -field;
> +}
> +static __always_inline u64 field_mask(u64 field)
> +{
> +       return field / field_multiplier(field);
> +}
> +#define ____MAKE_OP(type,base,to,from)                                 \
> +static __always_inline __##type type##_encode_bits(base v, base field) \
> +{                                                                      \
> +       if (__builtin_constant_p(v) && (v & ~field_mask(field)))        \
> +               __field_overflow();                                     \
> +       return to((v & field_mask(field)) * field_multiplier(field));   \
> +}                                                                      \
> +static __always_inline __##type type##_replace_bits(__##type old,      \
> +                                       base val, base field)           \
> +{                                                                      \
> +       return (old & ~to(field)) | type##_encode_bits(val, field);     \
> +}                                                                      \
> +static __always_inline void type##p_replace_bits(__##type *p,          \
> +                                       base val, base field)           \
> +{                                                                      \
> +       *p = (*p & ~to(field)) | type##_encode_bits(val, field);        \
> +}                                                                      \
> +static __always_inline base type##_get_bits(__##type v, base field)    \
> +{                                                                      \
> +       return (from(v) & field)/field_multiplier(field);               \
> +}
> +#define __MAKE_OP(size)                                                        \
> +       ____MAKE_OP(le##size,u##size,cpu_to_le##size,le##size##_to_cpu) \
> +       ____MAKE_OP(be##size,u##size,cpu_to_be##size,be##size##_to_cpu) \
> +       ____MAKE_OP(u##size,u##size,,)
> +____MAKE_OP(u8,u8,,)
> +__MAKE_OP(16)
> +__MAKE_OP(32)
> +__MAKE_OP(64)
> +#undef __MAKE_OP
> +#undef ____MAKE_OP
> +
>  #endif

Could we add blank line between macros and functions to improve some readablity?

Regards,
Bin

[PATCH v1 07/10] linux/bitfield.h: Add primitives for manipulating bitfields both in host- and fixed-endian.

[Sylwester Nawrocki]

Hi Bin,

On 25.04.2020 03:43, Bin Meng wrote:
> Could we add blank line between macros and functions to improve some 
> readablity?

Thanks for your review. It might be indeed a bit easier to read with 
some extra empty lines, given there is no any comments to those macro 
definitions. I have made a change as follows for the next iteration.

---------------------8<---------------------
diff --git a/include/linux/bitfield.h b/include/linux/bitfield.h
index 4964213..7acba4c 100644
--- a/include/linux/bitfield.h
+++ b/include/linux/bitfield.h
@@ -117,6 +117,7 @@ static __always_inline u64 field_mask(u64 field)
  {
         return field / field_multiplier(field);
  }
+
  #define ____MAKE_OP(type,base,to,from)                                 \
  static __always_inline __##type type##_encode_bits(base v, base field) \
  { \
@@ -138,14 +139,17 @@ static __always_inline base 
type##_get_bits(__##type v, base field)       \
  { \
         return (from(v) & field)/field_multiplier(field);               \
  }
+
  #define __MAKE_OP(size) \
____MAKE_OP(le##size,u##size,cpu_to_le##size,le##size##_to_cpu) \
____MAKE_OP(be##size,u##size,cpu_to_be##size,be##size##_to_cpu) \
         ____MAKE_OP(u##size,u##size,,)
+
  ____MAKE_OP(u8,u8,,)
  __MAKE_OP(16)
  __MAKE_OP(32)
  __MAKE_OP(64)
+
  #undef __MAKE_OP
  #undef ____MAKE_OP
--------------------->8---------------------

--
Regards,
Sylwester

[PATCH v1 10/10] config: Enable support for the XHCI controller on RPI4 board

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> +CONFIG_XHCI_64BIT_DWORD_ACCESS_ONLY=y

This one slipped trough :)

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/381a8e64/attachment.sig>

[PATCH v1 10/10] config: Enable support for the XHCI controller on RPI4 board

[Sylwester Nawrocki]

On 27.04.2020 11:11, Nicolas Saenz Julienne wrote:
> On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
>> +CONFIG_XHCI_64BIT_DWORD_ACCESS_ONLY=y
> This one slipped trough :)

Indeed, I have already removed it for next iteration.

-- 
Regards,
Sylwester

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> This patch adds basic driver for the Broadcom STB PCIe host controller.
> The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
> handling removed. The inbound access memory region is not currently
> parsed from dma-ranges DT property and a fixed 4GB region is used.
> 
> The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
> USB Host Controller.
> 
> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> ---
> Changes since RFC:
>  - reworked to align with current Linux mainline version and u-boot driver
>    by Nicolas Saenz Julienne

[...]

> +
> +	/*
> +	 * TODO: Use the base address and size(s) provided in the dma-ranges
> +	 * property.
> +	 */
> +	rc_bar2_offset = 0;
> +	rc_bar2_size = 1ULL << 32;

From experience this works fine, although it highly depends on how DMA memory is
handled in u-boot.

For example, in arm64 Linux, DMA memory was allocated from ZONE_DMA32, which
would return memory smaller than 4GB. This was not good enough for bcm2711b0 --
revision b0 of rpi4's SoC, so far the most common out there -- which has an
internal wiring bug that prevents PCIe from accessing memory higher than 3GB
(RPi4 might have up to 4GB).  So we had to introduce a ZONE_DMA, which covers
the lower GB of memory, in order to allocate suitable DMA memory for PCI and
other DMA limited devices.

While playing around with u-boot's xHCI I saw that all memory is allocated is
located in the lower 1GB. But never got to look into why. I'm curious to know
if someone knows how's that handled in u-boot. Ultimately, depending on how it
works, we might be able to just disregard dma-ranges altogether.

For some extra context xhci_malloc() gets its memory from memalign(). And it's
not clear to me how that function decides which memory to use.

> +
> +	tmp = lower_32_bits(rc_bar2_offset);
> +	u32p_replace_bits(&tmp, brcm_pcie_encode_ibar_size(rc_bar2_size),
> +			  PCIE_MISC_RC_BAR2_CONFIG_LO_SIZE_MASK);
> +	writel(tmp, base + PCIE_MISC_RC_BAR2_CONFIG_LO);
> +	writel(upper_32_bits(rc_bar2_offset),
> +	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
> +
> +	scb_size_val = rc_bar2_size ?
> +		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
> +	tmp = readl(base + PCIE_MISC_MISC_CTRL);
> +	u32p_replace_bits(&tmp, scb_size_val,
> +			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> +	writel(tmp, base + PCIE_MISC_MISC_CTRL);
> +

[...]

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/70b780ca/attachment.sig>

[PATCH v1 01/10] usb: xhci: Add missing cache flush in the scratchpad array initialization

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> In current code there is no cache flush after initializing the scratchpad
> buffer array with the scratchpad buffer pointers. This leads to a failure
> of the "slot enable" command on the rpi4 board (Broadcom STB PCIe
> controller + VL805 USB hub) - the very first TRB transfer on the command
> ring fails and there is a timeout while waiting for the command completion
> event. After adding the missing cache flush everything seems to be working
> as expected.
> 
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/9f7302dd/attachment.sig>

[PATCH v1 02/10] usb: xhci: Use only 32-bit accesses in xhci_writeq/xhci_readq

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> There might be hardware configurations where 64-bit data accesses
> to XHCI registers are not supported properly.  This patch removes
> the readq/writeq so always two 32-bit accesses are used to read/write
> 64-bit XHCI registers, similarly as it is done in Linux kernel.
> 
> This patch fixes operation of the XHCI controller on RPI4 Broadcom
> BCM2711 SoC based board, where the VL805 USB XHCI controller is
> connected to the PCIe Root Complex, which is attached to the system
> through the SCB bridge.
> 
> Even though the architecture is 64-bit the PCIe BAR is 32-bit and likely
> the 64-bit wide register accesses initiated by the CPU are not properly
> translated to a sequence of 32-bit PCIe accesses.
> xhci_readq(), for example, always returns same value in upper and lower
> 32-bits, e.g. 0xabcd1234abcd1234 instead of 0x00000000abcd1234.
> 
> Cc: Sergey Temerkhanov <s.temerkhanov@gmail.com>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> ---
> Changes since RFC:
>  - dropped Kconfig option, switched to not using readq/writeq
>    unconditionally.

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/28bcedf3/attachment.sig>

[PATCH v1 03/10] pci: Move some PCIe register offset definitions to a common header

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> Some PCI Express register offsets are currently defined in multiple
> drivers, move them to a common header to avoid re-definitions and
> as a pre-requisite for adding new PCIe driver.
> While at it replace some spaces with tabs.
> 
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/021e84b1/attachment.sig>

[PATCH v1 04/10] rpi4: shorten a mapping for the DRAM

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> From: Marek Szyprowski <m.szyprowski@samsung.com>
> 
> Remove the overlap between DRAM and device's IO area.
> 
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> ---
> Changes since RFC:
>  - none.

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/d158184f/attachment.sig>

[PATCH v1 05/10] rpi4: add a mapping for the PCIe XHCI controller MMIO registers (ARM 64bit)

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> From: Marek Szyprowski <m.szyprowski@samsung.com>
> 
> Create a non-cacheable mapping for the 0x600000000 physical memory region,
> where MMIO registers for the PCIe XHCI controller are instantiated by the
> PCIe bridge.
> 
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> ---
> Changes since RFC:
>  - none.

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/e1e6e7b8/attachment.sig>

[PATCH v1 08/10] pci: Add some PCI Express capability register offset definitions

[Nicolas Saenz Julienne]

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> Add PCI Express capability definitions required by the Broadcom
> STB PCIe controller driver.
> 
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> ---
> Changes since RFC:
>  - ensure the entries are added in order, sorted by ascending
>    address values.

Reviewed-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200427/e4ec48b3/attachment.sig>

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Marek Szyprowski]

Hi Nicolas,

On 27.04.2020 12:15, Nicolas Saenz Julienne wrote:
> On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
>> This patch adds basic driver for the Broadcom STB PCIe host controller.
>> The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
>> handling removed. The inbound access memory region is not currently
>> parsed from dma-ranges DT property and a fixed 4GB region is used.
>>
>> The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
>> USB Host Controller.
>>
>> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
>> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
>> ---
>> Changes since RFC:
>>   - reworked to align with current Linux mainline version and u-boot driver
>>     by Nicolas Saenz Julienne
> [...]
>
>> +
>> +	/*
>> +	 * TODO: Use the base address and size(s) provided in the dma-ranges
>> +	 * property.
>> +	 */
>> +	rc_bar2_offset = 0;
>> +	rc_bar2_size = 1ULL << 32;
>  From experience this works fine, although it highly depends on how DMA memory is
> handled in u-boot.
>
> For example, in arm64 Linux, DMA memory was allocated from ZONE_DMA32, which
> would return memory smaller than 4GB. This was not good enough for bcm2711b0 --
> revision b0 of rpi4's SoC, so far the most common out there -- which has an
> internal wiring bug that prevents PCIe from accessing memory higher than 3GB
> (RPi4 might have up to 4GB).  So we had to introduce a ZONE_DMA, which covers
> the lower GB of memory, in order to allocate suitable DMA memory for PCI and
> other DMA limited devices.
>
> While playing around with u-boot's xHCI I saw that all memory is allocated is
> located in the lower 1GB. But never got to look into why. I'm curious to know
> if someone knows how's that handled in u-boot. Ultimately, depending on how it
> works, we might be able to just disregard dma-ranges altogether.
>
> For some extra context xhci_malloc() gets its memory from memalign(). And it's
> not clear to me how that function decides which memory to use.

I think that memalign() allocates memory from the uboot's defined SDRAM 
(from its end). Assuming that CONFIG_SYS_SDRAM_SIZE is set to 128M in 
include/configs/rpi.h it should be always safe, but I will check that 
tomorrow to be 100% sure.


Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Marek Szyprowski]

Hi All,

On 27.04.2020 15:54, Marek Szyprowski wrote:
> On 27.04.2020 12:15, Nicolas Saenz Julienne wrote:
>> On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
>>> This patch adds basic driver for the Broadcom STB PCIe host controller.
>>> The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
>>> handling removed. The inbound access memory region is not currently
>>> parsed from dma-ranges DT property and a fixed 4GB region is used.
>>>
>>> The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
>>> USB Host Controller.
>>>
>>> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
>>> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
>>> ---
>>> Changes since RFC:
>>> ? - reworked to align with current Linux mainline version and u-boot 
>>> driver
>>> ??? by Nicolas Saenz Julienne
>> [...]
>>
>>> +
>>> +??? /*
>>> +???? * TODO: Use the base address and size(s) provided in the 
>>> dma-ranges
>>> +???? * property.
>>> +???? */
>>> +??? rc_bar2_offset = 0;
>>> +??? rc_bar2_size = 1ULL << 32;
>> ?From experience this works fine, although it highly depends on how 
>> DMA memory is
>> handled in u-boot.
>>
>> For example, in arm64 Linux, DMA memory was allocated from 
>> ZONE_DMA32, which
>> would return memory smaller than 4GB. This was not good enough for 
>> bcm2711b0 --
>> revision b0 of rpi4's SoC, so far the most common out there -- which 
>> has an
>> internal wiring bug that prevents PCIe from accessing memory higher 
>> than 3GB
>> (RPi4 might have up to 4GB).? So we had to introduce a ZONE_DMA, 
>> which covers
>> the lower GB of memory, in order to allocate suitable DMA memory for 
>> PCI and
>> other DMA limited devices.
>>
>> While playing around with u-boot's xHCI I saw that all memory is 
>> allocated is
>> located in the lower 1GB. But never got to look into why. I'm curious 
>> to know
>> if someone knows how's that handled in u-boot. Ultimately, depending 
>> on how it
>> works, we might be able to just disregard dma-ranges altogether.
>>
>> For some extra context xhci_malloc() gets its memory from memalign(). 
>> And it's
>> not clear to me how that function decides which memory to use.
>
> I think that memalign() allocates memory from the uboot's defined 
> SDRAM (from its end). Assuming that CONFIG_SYS_SDRAM_SIZE is set to 
> 128M in include/configs/rpi.h it should be always safe, but I will 
> check that tomorrow to be 100% sure.

Okay, I've checked and memalign always get memory from the malloc pool, 
which is set almost at the end of the first memory bank (in runtime), so 
it is always below the first 1GiB. So this should be safe.

It is however not safe for explicit reads (and possible other 
transactions) above 3rd GiB, see the log below:

Initialising SDRAM 'Samsung' 16Gb x2 total-size: 32 Gbit
Loading recovery.elf hnd: 0x00000000
Failed to read recovery.elf error: 3
Loading start4x.elf hnd: 0x000011de
Loading fixup4x.dat hnd: 0x00000259
MEM GPU: 76 ARM: 948 TOTAL: 1024
FIXUP src: 128 256 dst: 948 1024
Starting start4x.elf @ 0xfec00200



U-Boot 2020.04-00011-gaefcd8d-dirty (Apr 28 2020 - 10:13:31 +0200)

DRAM:? 3.9 GiB
RPI 4 Model B (0xc03112)
MMC:?? emmc2 at 7e340000: 0, mmcnr at 7e300000: 1
Loading Environment from FAT... OK
In:??? serial
Out:?? serial
Err:?? serial
Net:?? eth0: genet at 7d580000
PCIe BRCM: link up, 5.0 Gbps x1 (!SSC)
starting USB...
Bus Register 5000420 NbrPorts 5
Starting the controller
USB XHCI 1.00
scanning bus xhci_pci for devices...
3 USB Device(s) found
 ?????? scanning usb for storage devices... 1 Storage Device(s) found
Hit any key to stop autoboot:? 0
U-Boot> md 00001000
00001000: 04000000 54020000 02000000 03000000??? .......T........
00001010: 08000000 65020000 61666564 00746c75??? .......edefault.
00001020: 03000000 04000000 54010000 0f000000??? ...........T....
00001030: 01000000 5f697064 6f697067 00000030??? ....dpi_gpio0...
00001040: 03000000 70000000 73020000 00000000??? .......p...s....
00001050: 01000000 02000000 03000000 04000000??? ................
00001060: 05000000 06000000 07000000 08000000??? ................
00001070: 09000000 0a000000 0b000000 0c000000??? ................
00001080: 0d000000 0e000000 0f000000 10000000??? ................
00001090: 11000000 12000000 13000000 14000000??? ................
000010a0: 15000000 16000000 17000000 18000000??? ................
000010b0: 19000000 1a000000 1b000000 03000000??? ................
000010c0: 04000000 7d020000 06000000 03000000??? .......}........
000010d0: 04000000 54010000 39000000 02000000??? .......T...9....
000010e0: 01000000 636d6d65 6970675f 0032326f??? ....emmc_gpio22.
000010f0: 03000000 18000000 73020000 16000000??? ...........s....
U-Boot> usb read 00001000 0 10

usb read: device 0 block # 0, count 16 ... 16 blocks read: OK
U-Boot> md 00001000
00001000: 1000b8fa 00bcd08e 0000b8b0 c08ed88e??? ................
00001010: 7c00befb b90600bf a4f30200 000621ea??? ...|.........!..
00001020: 07bebe00 0b750438 8110c683 7507fefe??? ....8.u........u
00001030: b416ebf3 bb01b002 80b27c00 8b01748a??? .........|...t..
00001040: 13cd024c 007c00ea 00feeb00 00000000??? L.....|.........
00001050: 00000000 00000000 00000000 00000000??? ................
00001060: 00000000 00000000 00000000 00000000??? ................
00001070: 00000000 00000000 00000000 00000000??? ................
00001080: 00000000 00000000 00000000 00000000??? ................
00001090: 00000000 00000000 00000000 00000000??? ................
000010a0: 00000000 00000000 00000000 00000000??? ................
000010b0: 00000000 00000000 00000000 00000000??? ................
000010c0: 00000000 00000000 00000000 00000000??? ................
000010d0: 00000000 00000000 00000000 00000000??? ................
000010e0: 00000000 00000000 00000000 00000000??? ................
000010f0: 00000000 00000000 00000000 00000000??? ................
U-Boot> md 40001000
40001000: ffffffff ffffffff ffffffff ffffffff??? ................
40001010: ffffffff ffffffff ffffffff ffffffff??? ................
40001020: ffdfffff ffffffff ffffffff fffffffd??? ................
40001030: ffffefff ffffffff ffffffff ffffffff??? ................
40001040: ffffffff fe7fffff ffffff7f ffffffff??? ................
40001050: ff7fdfff fdffffff ffffffff ffffffff??? ................
40001060: ffffffff 77ffbfdf ffffffff ffffdfff??? .......w........
40001070: fffdffdf fffff7ff ffffffff fffffdff??? ................
40001080: ffffffff ffffffff fffffffd ffffffff??? ................
40001090: fffffffb ffffffff ffffffff ffffffff??? ................
400010a0: ffffffff fffff7ff fffffdff fff7ffff??? ................
400010b0: ffffffff ffffffff fffeffff f7ffffff??? ................
400010c0: ffffffff ffffffff ffffffff fffffffd??? ................
400010d0: ffffffff ffdfffff fffdffff ffffffff??? ................
400010e0: 7fff7ffd ffffffff 7fffffff ffffffff??? ................
400010f0: ffffff7f ffbfffbf ffffdfff fffffffe??? ................
U-Boot> usb read 40001000 0 10

usb read: device 0 block # 0, count 16 ... 16 blocks read: OK
U-Boot> md 40001000
40001000: 1000b8fa 00bcd08e 0000b8b0 c08ed88e??? ................
40001010: 7c00befb b90600bf a4f30200 000621ea??? ...|.........!..
40001020: 07bebe00 0b750438 8110c683 7507fefe??? ....8.u........u
40001030: b416ebf3 bb01b002 80b27c00 8b01748a??? .........|...t..
40001040: 13cd024c 007c00ea 00feeb00 00000000??? L.....|.........
40001050: 00000000 00000000 00000000 00000000??? ................
40001060: 00000000 00000000 00000000 00000000??? ................
40001070: 00000000 00000000 00000000 00000000??? ................
40001080: 00000000 00000000 00000000 00000000??? ................
40001090: 00000000 00000000 00000000 00000000??? ................
400010a0: 00000000 00000000 00000000 00000000??? ................
400010b0: 00000000 00000000 00000000 00000000??? ................
400010c0: 00000000 00000000 00000000 00000000??? ................
400010d0: 00000000 00000000 00000000 00000000??? ................
400010e0: 00000000 00000000 00000000 00000000??? ................
400010f0: 00000000 00000000 00000000 00000000??? ................
U-Boot> md 80001000
80001000: 6d254fbc 252fa38e e985672f 03b6d513??? .O%m../%/g......
80001010: 3e681fbe d65269ad b503e985 be48b50b ..h>.iR.......H.
80001020: 6ddacf4f 25d0a371 e97a67d9 0341d5ec??? O..mq..%.gz...A.
80001030: 3e971fc1 d6ad695e b5f4e96e beb7b5f4 ...>^i..n.......
80001040: dd117c69 5c71e187 42f6a0c7 1c71f2e6??? i|....q\...B..q.
80001050: b6c2f1cb 63a6939a 945719b1 21cf29ad??? .......c..W..).!
80001060: ddeefc9e 5c8ce97e 4209a82e 1c86520d??? ....~..\...B.R..
80001070: b6b53164 63599365 c4a8194e a030a952??? d1..e.YcN...R.0.
80001080: d9d1919f be400dbd 98d16e64 22e6f443??? ...... at .dn..C.."
80001090: 567291df 12f2eae6 8c555bba bf08666e??? ..rV.....[U.nf..
800010a0: d82e9164 febf8dc6 d82feebf 2211f4ae??? d........./...."
800010b0: 568d1160 1205ea3b 8ca31b45 3ff7a681??? `..V;...E......?
800010c0: 11bb547b 46761efb 87165bb6 fc41e2ce??? {T....vF.[....A.
800010d0: 951b2ba4 4737b38b 3f286527 073e22ee .+....7G'e(?.">.
800010e0: 1144ddaf 46891e87 87e35b49 fcbca221??? ..D....FI[..!...
800010f0: 95e5ab5f 45c9b364 3fd565c8 07c12a17??? _...d..E.e.?.*..
U-Boot> usb read 80001000 0 10

usb read: device 0 block # 0, count 16 ... 16 blocks read: OK
U-Boot> md 80001000
80001000: 1000b8fa 00bcd08e 0000b8b0 c08ed88e??? ................
80001010: 7c00befb b90600bf a4f30200 000621ea??? ...|.........!..
80001020: 07bebe00 0b750438 8110c683 7507fefe??? ....8.u........u
80001030: b416ebf3 bb01b002 80b27c00 8b01748a??? .........|...t..
80001040: 13cd024c 007c00ea 00feeb00 00000000??? L.....|.........
80001050: 00000000 00000000 00000000 00000000??? ................
80001060: 00000000 00000000 00000000 00000000??? ................
80001070: 00000000 00000000 00000000 00000000??? ................
80001080: 00000000 00000000 00000000 00000000??? ................
80001090: 00000000 00000000 00000000 00000000??? ................
800010a0: 00000000 00000000 00000000 00000000??? ................
800010b0: 00000000 00000000 00000000 00000000??? ................
800010c0: 00000000 00000000 00000000 00000000??? ................
800010d0: 00000000 00000000 00000000 00000000??? ................
800010e0: 00000000 00000000 00000000 00000000??? ................
800010f0: 00000000 00000000 00000000 00000000??? ................
U-Boot> md c0001000
c0001000: ff6fffff fffff7ff fffffff7 ffffffff??? ..o.............
c0001010: 7ffffdff ffeff7ff ff7fffff ffffdfff??? ................
c0001020: feffffff ffffbdff ffffffff fffff77f??? ................
c0001030: ffffffff ffdfffff fffffff7 efffffff??? ................
c0001040: fdffffff ffffdfff 7ff5fdff ffffffef??? ................
c0001050: ffbfffff fffffdf7 fff7dfff ffffffff??? ................
c0001060: ff7fffff ffffffff ffffffff ffffffff??? ................
c0001070: dff7ffff ffffffff ffffffff dfffffff??? ................
c0001080: ffffffff fbffffff ffffffff ffffff7f??? ................
c0001090: ffffffdf ff7ffeff ffffffff ffffdfff??? ................
c00010a0: ffffffff ffffffff 7ffffeff ffffdfff??? ................
c00010b0: ffffffff ffffffff ffffffff ffffffff??? ................
c00010c0: feffffff ffffffff ffffbfff 7fffdfff??? ................
c00010d0: ffffffff dfffffff 7fffffff ffffffff??? ................
c00010e0: ffff7fdf ffffffff dfbfffff ffffffff??? ................
c00010f0: ffffffff ffffffff ff7ff5ff ffffffff??? ................
U-Boot> usb read c0001000 0 10

usb read: device 0 block # 0, count 16 ... 16 blocks read: OK
U-Boot> md c0001000
c0001000: ff6fffff fffff7ff fffffff7 ffffffff??? ..o.............
c0001010: 7ffffdff ffeff7ff ff7fffff ffffdfff??? ................
c0001020: feffffff ffffbdff ffffffff fffff77f??? ................
c0001030: ffffffff ffdfffff fffffff7 efffffff??? ................
c0001040: fdffffff ffffdfff 7ff5fdff ffffffef??? ................
c0001050: ffbfffff fffffdf7 fff7dfff ffffffff??? ................
c0001060: ff7fffff ffffffff ffffffff ffffffff??? ................
c0001070: dff7ffff ffffffff ffffffff dfffffff??? ................
c0001080: ffffffff fbffffff ffffffff ffffff7f??? ................
c0001090: ffffffdf ff7ffeff ffffffff ffffdfff??? ................
c00010a0: ffffffff ffffffff 7ffffeff ffffdfff??? ................
c00010b0: ffffffff ffffffff ffffffff ffffffff??? ................
c00010c0: feffffff ffffffff ffffbfff 7fffdfff??? ................
c00010d0: ffffffff dfffffff 7fffffff ffffffff??? ................
c00010e0: ffff7fdf ffffffff dfbfffff ffffffff??? ................
c00010f0: ffffffff ffffffff ff7ff5ff ffffffff??? ................
U-Boot>

I think that there cannot be done much about it. u-boot doesn't have any 
true DMA-mapping layer or a way to express the current limitations. IMHO 
it is enough that it works for malloc'ed memory and everything else 
should be considered as not really supported.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Nicolas Saenz Julienne]

Hi Maerk,

On Tue, 2020-04-28 at 11:49 +0200, Marek Szyprowski wrote:
> Hi All,
> 
> On 27.04.2020 15:54, Marek Szyprowski wrote:
> > On 27.04.2020 12:15, Nicolas Saenz Julienne wrote:
> > > On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> > > > This patch adds basic driver for the Broadcom STB PCIe host controller.
> > > > The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
> > > > handling removed. The inbound access memory region is not currently
> > > > parsed from dma-ranges DT property and a fixed 4GB region is used.
> > > > 
> > > > The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
> > > > USB Host Controller.
> > > > 
> > > > Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> > > > Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> > > > ---
> > > > Changes since RFC:
> > > >   - reworked to align with current Linux mainline version and u-boot 
> > > > driver
> > > >     by Nicolas Saenz Julienne
> > > [...]
> > > 
> > > > +
> > > > +    /*
> > > > +     * TODO: Use the base address and size(s) provided in the 
> > > > dma-ranges
> > > > +     * property.
> > > > +     */
> > > > +    rc_bar2_offset = 0;
> > > > +    rc_bar2_size = 1ULL << 32;
> > >  From experience this works fine, although it highly depends on how 
> > > DMA memory is
> > > handled in u-boot.
> > > 
> > > For example, in arm64 Linux, DMA memory was allocated from 
> > > ZONE_DMA32, which
> > > would return memory smaller than 4GB. This was not good enough for 
> > > bcm2711b0 --
> > > revision b0 of rpi4's SoC, so far the most common out there -- which 
> > > has an
> > > internal wiring bug that prevents PCIe from accessing memory higher 
> > > than 3GB
> > > (RPi4 might have up to 4GB).  So we had to introduce a ZONE_DMA, 
> > > which covers
> > > the lower GB of memory, in order to allocate suitable DMA memory for 
> > > PCI and
> > > other DMA limited devices.
> > > 
> > > While playing around with u-boot's xHCI I saw that all memory is 
> > > allocated is
> > > located in the lower 1GB. But never got to look into why. I'm curious 
> > > to know
> > > if someone knows how's that handled in u-boot. Ultimately, depending 
> > > on how it
> > > works, we might be able to just disregard dma-ranges altogether.
> > > 
> > > For some extra context xhci_malloc() gets its memory from memalign(). 
> > > And it's
> > > not clear to me how that function decides which memory to use.
> > 
> > I think that memalign() allocates memory from the uboot's defined 
> > SDRAM (from its end). Assuming that CONFIG_SYS_SDRAM_SIZE is set to 
> > 128M in include/configs/rpi.h it should be always safe, but I will 
> > check that tomorrow to be 100% sure.

Thanks for having a look at this!

> Okay, I've checked and memalign always get memory from the malloc pool, 
> which is set almost at the end of the first memory bank (in runtime), so 
> it is always below the first 1GiB. So this should be safe.

I see.

> It is however not safe for explicit reads (and possible other 
> transactions) above 3rd GiB, see the log below:

[...]

> I think that there cannot be done much about it. u-boot doesn't have any 
> true DMA-mapping layer or a way to express the current limitations. IMHO 
> it is enough that it works for malloc'ed memory and everything else 
> should be considered as not really supported.

I agree. And on the good side, it's very unlikeyly we'll ever have to parse
the dma-rages.

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200428/36ef2d8d/attachment.sig>

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Nicolas Saenz Julienne]

Hi Sylwester,

On Fri, 2020-04-24 at 18:50 +0200, Sylwester Nawrocki wrote:
> This patch adds basic driver for the Broadcom STB PCIe host controller.
> The code is based on Linux upstream driver (pcie-brcmstb.c) with MSI
> handling removed. The inbound access memory region is not currently
> parsed from dma-ranges DT property and a fixed 4GB region is used.
> 
> The patch has been tested on RPI4 board, i.e. on BCM2711 SoC with VL805
> USB Host Controller.
> 
> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> ---
> Changes since RFC:
>  - reworked to align with current Linux mainline version and u-boot driver
>    by Nicolas Saenz Julienne

[...]

> +
> +	/*
> +	 * For config space accesses on the RC, show the right class for
> +	 * a PCIe-PCIe bridge (the default setting is to be EP mode).
> +	 */
> +	tmp = readl(base + PCIE_RC_CFG_PRIV1_ID_VAL3);
> +	u32p_replace_bits(&tmp, 0x060400,
> +			  PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK);
> +	writel(tmp, base + PCIE_RC_CFG_PRIV1_ID_VAL3);
> +
> +	if (pcie->ssc) {
> +		ret = brcm_pcie_set_ssc(base);

This call should be like this:

	ret = brcm_pcie_set_ssc(pcie);

It builds alright since base is a void pointer. But it obvioulsy causes issues
further down the line.

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200428/46d2b639/attachment.sig>

[PATCH v1 09/10] pci: Add driver for Broadcom STB PCIe controller

[Sylwester Nawrocki]

Hi Nicolas,

On 4/28/20 15:59, Nicolas Saenz Julienne wrote:
>> +	/*
>> +	 * For config space accesses on the RC, show the right class for
>> +	 * a PCIe-PCIe bridge (the default setting is to be EP mode).
>> +	 */
>> +	tmp = readl(base + PCIE_RC_CFG_PRIV1_ID_VAL3);
>> +	u32p_replace_bits(&tmp, 0x060400,
>> +			  PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK);
>> +	writel(tmp, base + PCIE_RC_CFG_PRIV1_ID_VAL3);
>> +
>> +	if (pcie->ssc) {
>> +		ret = brcm_pcie_set_ssc(base);

> This call should be like this:
> 
> 	ret = brcm_pcie_set_ssc(pcie);
> 
> It builds alright since base is a void pointer. But it obvioulsy causes issues
> further down the line.

Thanks for pointing this out, I have fixed it and re-tested again with
SSC enabled and disabled.

--
Regards,
Sylwester