IOMMU Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support
@ 2019-08-20 14:58 Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 01/11] asm-generic: add dma_zone_size Nicolas Saenz Julienne
                   ` (10 more replies)
  0 siblings, 11 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv
  Cc: phill, linux-s390, f.fainelli, frowand.list, linuxppc-dev,
	linux-kernel, eric, mbrugger, linux-rpi-kernel, akpm, will,
	nsaenzjulienne

Hi all,
this series attempts to address some issues we found while bringing up
the new Raspberry Pi 4 in arm64 and it's intended to serve as a follow
up of these discussions:
v1: https://lkml.org/lkml/2019/7/31/922
RFC: https://lkml.org/lkml/2019/7/17/476

The new Raspberry Pi 4 has up to 4GB of memory but most peripherals can
only address the first GB: their DMA address range is
0xc0000000-0xfc000000 which is aliased to the first GB of physical
memory 0x00000000-0x3c000000. Note that only some peripherals have these
limitations: the PCIe, V3D, GENET, and 40-bit DMA channels have a wider
view of the address space by virtue of being hooked up trough a second
interconnect.

Part of this is solved in arm32 by setting up the machine specific
'.dma_zone_size = SZ_1G', which takes care of reserving the coherent
memory area at the right spot. That said no buffer bouncing (needed for
dma streaming) is available at the moment, but that's a story for
another series.

Unfortunately there is no such thing as 'dma_zone_size' in arm64. Only
ZONE_DMA32 is created which is interpreted by dma-direct and the arm64
arch code as if all peripherals where be able to address the first 4GB
of memory.

In the light of this, the series implements the following changes:

- Create generic 'dma_zone_size' in order for hardware description code
  to set it up when needed.

- Add a function in early_init_dt_scan() to setup 'dma_zone_size' for
  the RPi4.

- Create both DMA zones in arm64, ZONE_DMA will contain the area
  addressable by all peripherals and ZONE_DMA32 the rest of the 32 bit
  addressable memory. ZONE_DMA32 might be left empty.

- Reserve the CMA area in a place suitable for all peripherals.

- Inform dma-direct of the new runtime calculated min_mask.

This series has been tested on multiple devices both by checking the
zones setup matches the expectations and by double-checking physical
addresses on pages allocated on the three relevant areas GFP_DMA,
GFP_DMA32, GFP_KERNEL:

- On an RPi4 with variations on the ram memory size. But also forcing
  the situation where all three memory zones are nonempty by setting a 3G
  ZONE_DMA32 ceiling on a 4G setup. Both with and without NUMA support.

- On a Synquacer box[1] with 32G of memory.

- On an ACPI based Huawei TaiShan server[2] with 256G of memory.

- On a QEMU virtual machine running arm64's OpenSUSE Tumbleweed.

That's all.

Regards,
Nicolas

[1] https://www.96boards.org/product/developerbox/
[2] https://e.huawei.com/en/products/cloud-computing-dc/servers/taishan-server/taishan-2280-v2

---

Changes in v2:
- More in depth testing.
- Create new global 'dma_zone_size'.
- New approach to getting the dma_zone_size, instead of parsing the dts
  we hardcode it conditionally to the machine compatible name.
- Fix ZONE_DMA and ZONE_DMA32 split, now ZONE_DMA32 remains empty if
  ZONE_DMA fits the whole 32 bit addressable space.
- Take into account devices with DMA offset.
- Rename new dma-direct variable to zone_dma_bits.
- Try new approach by merging both ZONE_DMA and ZONE_DMA32 comments
  in mmzone.h, add new up to date examples.

Nicolas Saenz Julienne (11):
  asm-generic: add dma_zone_size
  arm: use generic dma_zone_size
  of/fdt: add of_fdt_machine_is_compatible function
  of/fdt: add early_init_dt_get_dma_zone_size()
  arm64: mm: use arm64_dma_phys_limit instead of calling
    max_zone_dma_phys()
  arm64: rename variables used to calculate ZONE_DMA32's size
  arm64: re-introduce max_zone_dma_phys()
  arm64: use both ZONE_DMA and ZONE_DMA32
  dma-direct: turn ARCH_ZONE_DMA_BITS into a variable
  arm64: edit zone_dma_bits to fine tune dma-direct min mask
  mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type'

 arch/arm/include/asm/dma.h      |  8 ++--
 arch/arm/mm/init.c              | 12 ++----
 arch/arm64/Kconfig              |  4 ++
 arch/arm64/mm/init.c            | 73 +++++++++++++++++++++++++--------
 arch/powerpc/include/asm/page.h |  9 ----
 arch/powerpc/mm/mem.c           | 16 +++++---
 arch/s390/include/asm/page.h    |  2 -
 arch/s390/mm/init.c             |  1 +
 drivers/of/fdt.c                | 15 +++++++
 include/asm-generic/dma.h       |  8 +++-
 include/linux/dma-direct.h      |  2 +
 include/linux/mmzone.h          | 46 ++++++++++++---------
 kernel/dma/direct.c             | 13 +++---
 mm/page_alloc.c                 |  3 ++
 14 files changed, 140 insertions(+), 72 deletions(-)

-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-26  7:09   ` Christoph Hellwig
  2019-08-20 14:58 ` [PATCH v2 02/11] arm: use generic dma_zone_size Nicolas Saenz Julienne
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Arnd Bergmann
  Cc: phill, f.fainelli, frowand.list, linux-kernel, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

Some architectures have platform specific DMA addressing limitations.
This will allow for hardware description code to provide the constraints
in a generic manner, so as for arch code to properly setup it's memory
zones and DMA mask.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2: None

 include/asm-generic/dma.h | 8 +++++++-
 mm/page_alloc.c           | 3 +++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/include/asm-generic/dma.h b/include/asm-generic/dma.h
index 43d0c8af8058..c2f39cdb64f6 100644
--- a/include/asm-generic/dma.h
+++ b/include/asm-generic/dma.h
@@ -8,7 +8,13 @@
  *
  * Some code relies on seeing MAX_DMA_ADDRESS though.
  */
-#define MAX_DMA_ADDRESS PAGE_OFFSET
+#define MAX_DMA_ADDRESS	 (PAGE_OFFSET + dma_zone_size)
+
+/*
+ * Some architectures may have platform specific DMA addressing constraints.
+ * Firmware can use this to fine tune the device's DMA memory zone.
+ */
+extern u64 dma_zone_size __ro_after_init;
 
 extern int request_dma(unsigned int dmanr, const char *device_id);
 extern void free_dma(unsigned int dmanr);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 272c6de1bf4e..b514afee5451 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -133,6 +133,9 @@ EXPORT_SYMBOL(_totalram_pages);
 unsigned long totalreserve_pages __read_mostly;
 unsigned long totalcma_pages __read_mostly;
 
+u64 dma_zone_size __ro_after_init;
+EXPORT_SYMBOL(dma_zone_size);
+
 int percpu_pagelist_fraction;
 gfp_t gfp_allowed_mask __read_mostly = GFP_BOOT_MASK;
 #ifdef CONFIG_INIT_ON_ALLOC_DEFAULT_ON
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 02/11] arm: use generic dma_zone_size
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 01/11] asm-generic: add dma_zone_size Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function Nicolas Saenz Julienne
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, linux-kernel
  Cc: phill, f.fainelli, frowand.list, Russell King, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

'dma_zone_size' was created as a generic replacement to
'arm_dma_zone_size'. Use it accordingly.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2: None

 arch/arm/include/asm/dma.h |  8 +++++---
 arch/arm/mm/init.c         | 12 ++++--------
 2 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/arch/arm/include/asm/dma.h b/arch/arm/include/asm/dma.h
index a81dda65c576..52d19ffd92b4 100644
--- a/arch/arm/include/asm/dma.h
+++ b/arch/arm/include/asm/dma.h
@@ -2,16 +2,18 @@
 #ifndef __ASM_ARM_DMA_H
 #define __ASM_ARM_DMA_H
 
+#include <asm-generic/dma.h>
+
 /*
  * This is the maximum virtual address which can be DMA'd from.
  */
+#undef MAX_DMA_ADDRESS
 #ifndef CONFIG_ZONE_DMA
 #define MAX_DMA_ADDRESS	0xffffffffUL
 #else
 #define MAX_DMA_ADDRESS	({ \
-	extern phys_addr_t arm_dma_zone_size; \
-	arm_dma_zone_size && arm_dma_zone_size < (0x10000000 - PAGE_OFFSET) ? \
-		(PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; })
+	dma_zone_size && dma_zone_size < (0x10000000 - PAGE_OFFSET) ? \
+		(PAGE_OFFSET + dma_zone_size) : 0xffffffffUL; })
 #endif
 
 #ifdef CONFIG_ISA_DMA_API
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 16d373d587c4..95680bad245a 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -79,10 +79,6 @@ static void __init find_limits(unsigned long *min, unsigned long *max_low,
 }
 
 #ifdef CONFIG_ZONE_DMA
-
-phys_addr_t arm_dma_zone_size __read_mostly;
-EXPORT_SYMBOL(arm_dma_zone_size);
-
 /*
  * The DMA mask corresponding to the maximum bus address allocatable
  * using GFP_DMA.  The default here places no restriction on DMA
@@ -109,8 +105,8 @@ void __init setup_dma_zone(const struct machine_desc *mdesc)
 {
 #ifdef CONFIG_ZONE_DMA
 	if (mdesc->dma_zone_size) {
-		arm_dma_zone_size = mdesc->dma_zone_size;
-		arm_dma_limit = PHYS_OFFSET + arm_dma_zone_size - 1;
+		dma_zone_size = mdesc->dma_zone_size;
+		arm_dma_limit = PHYS_OFFSET + dma_zone_size - 1;
 	} else
 		arm_dma_limit = 0xffffffff;
 	arm_dma_pfn_limit = arm_dma_limit >> PAGE_SHIFT;
@@ -164,9 +160,9 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max_low,
 	 * Adjust the sizes according to any special requirements for
 	 * this machine type.
 	 */
-	if (arm_dma_zone_size)
+	if (dma_zone_size)
 		arm_adjust_dma_zone(zone_size, zhole_size,
-			arm_dma_zone_size >> PAGE_SHIFT);
+			dma_zone_size >> PAGE_SHIFT);
 #endif
 
 	free_area_init_node(0, zone_size, min, zhole_size);
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 01/11] asm-generic: add dma_zone_size Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 02/11] arm: use generic dma_zone_size Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 17:16   ` Rob Herring
  2019-08-20 14:58 ` [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size() Nicolas Saenz Julienne
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Frank Rowand
  Cc: phill, f.fainelli, linux-kernel, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

Provides the same functionality as of_machine_is_compatible.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2: None

 drivers/of/fdt.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index 9cdf14b9aaab..06ffbd39d9af 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -802,6 +802,13 @@ const char * __init of_flat_dt_get_machine_name(void)
 	return name;
 }
 
+static const int __init of_fdt_machine_is_compatible(char *name)
+{
+	unsigned long dt_root = of_get_flat_dt_root();
+
+	return of_flat_dt_is_compatible(dt_root, name);
+}
+
 /**
  * of_flat_dt_match_machine - Iterate match tables to find matching machine.
  *
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size()
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (2 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 17:14   ` Rob Herring
  2019-08-20 14:58 ` [PATCH v2 05/11] arm64: mm: use arm64_dma_phys_limit instead of calling max_zone_dma_phys() Nicolas Saenz Julienne
                   ` (6 subsequent siblings)
  10 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Frank Rowand
  Cc: phill, f.fainelli, linux-kernel, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

Some devices might have weird DMA addressing limitations that only apply
to a subset of the available peripherals. For example the Raspberry Pi 4
has two interconnects, one able to address the whole lower 4G memory
area and another one limited to the lower 1G.

Being an uncommon situation we simply hardcode the device wide DMA
addressable memory size conditionally to the machine compatible name and
set 'dma_zone_size' accordingly.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

---

Changes in v2:
- New approach to getting dma_zone_size, instead of parsing the dts we
  hardcode it conditionally to the machine compatible name.

 drivers/of/fdt.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index 06ffbd39d9af..f756e8c05a77 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -27,6 +27,7 @@
 
 #include <asm/setup.h>  /* for COMMAND_LINE_SIZE */
 #include <asm/page.h>
+#include <asm/dma.h>	/* for dma_zone_size */
 
 #include "of_private.h"
 
@@ -1195,6 +1196,12 @@ void __init early_init_dt_scan_nodes(void)
 	of_scan_flat_dt(early_init_dt_scan_memory, NULL);
 }
 
+void __init early_init_dt_get_dma_zone_size(void)
+{
+	if (of_fdt_machine_is_compatible("brcm,bcm2711"))
+		dma_zone_size = 0x3c000000;
+}
+
 bool __init early_init_dt_scan(void *params)
 {
 	bool status;
@@ -1204,6 +1211,7 @@ bool __init early_init_dt_scan(void *params)
 		return false;
 
 	early_init_dt_scan_nodes();
+	early_init_dt_get_dma_zone_size();
 	return true;
 }
 
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 05/11] arm64: mm: use arm64_dma_phys_limit instead of calling max_zone_dma_phys()
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (3 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size() Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 06/11] arm64: rename variables used to calculate ZONE_DMA32's size Nicolas Saenz Julienne
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, linux-kernel
  Cc: phill, f.fainelli, frowand.list, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

By the time we call zones_sizes_init() arm64_dma_phys_limit already
contains the result of max_zone_dma_phys(). We use the variable instead
of calling the function directly to save some precious cpu time.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2: None

 arch/arm64/mm/init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index f3c795278def..6112d6c90fa8 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -181,7 +181,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 	unsigned long max_zone_pfns[MAX_NR_ZONES]  = {0};
 
 #ifdef CONFIG_ZONE_DMA32
-	max_zone_pfns[ZONE_DMA32] = PFN_DOWN(max_zone_dma_phys());
+	max_zone_pfns[ZONE_DMA32] = PFN_DOWN(arm64_dma_phys_limit);
 #endif
 	max_zone_pfns[ZONE_NORMAL] = max;
 
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 06/11] arm64: rename variables used to calculate ZONE_DMA32's size
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (4 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 05/11] arm64: mm: use arm64_dma_phys_limit instead of calling max_zone_dma_phys() Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 07/11] arm64: re-introduce max_zone_dma_phys() Nicolas Saenz Julienne
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, linux-kernel
  Cc: phill, f.fainelli, frowand.list, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

Let the name indicate that they are used to calculate ZONE_DMA32's size
as opposed to ZONE_DMA.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2: None

 arch/arm64/mm/init.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 6112d6c90fa8..8956c22634dd 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -50,7 +50,7 @@
 s64 memstart_addr __ro_after_init = -1;
 EXPORT_SYMBOL(memstart_addr);
 
-phys_addr_t arm64_dma_phys_limit __ro_after_init;
+phys_addr_t arm64_dma32_phys_limit __ro_after_init;
 
 #ifdef CONFIG_KEXEC_CORE
 /*
@@ -168,7 +168,7 @@ static void __init reserve_elfcorehdr(void)
  * currently assumes that for memory starting above 4G, 32-bit devices will
  * use a DMA offset.
  */
-static phys_addr_t __init max_zone_dma_phys(void)
+static phys_addr_t __init max_zone_dma32_phys(void)
 {
 	phys_addr_t offset = memblock_start_of_DRAM() & GENMASK_ULL(63, 32);
 	return min(offset + (1ULL << 32), memblock_end_of_DRAM());
@@ -181,7 +181,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 	unsigned long max_zone_pfns[MAX_NR_ZONES]  = {0};
 
 #ifdef CONFIG_ZONE_DMA32
-	max_zone_pfns[ZONE_DMA32] = PFN_DOWN(arm64_dma_phys_limit);
+	max_zone_pfns[ZONE_DMA32] = PFN_DOWN(arm64_dma32_phys_limit);
 #endif
 	max_zone_pfns[ZONE_NORMAL] = max;
 
@@ -194,16 +194,16 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 {
 	struct memblock_region *reg;
 	unsigned long zone_size[MAX_NR_ZONES], zhole_size[MAX_NR_ZONES];
-	unsigned long max_dma = min;
+	unsigned long max_dma32 = min;
 
 	memset(zone_size, 0, sizeof(zone_size));
 
 	/* 4GB maximum for 32-bit only capable devices */
 #ifdef CONFIG_ZONE_DMA32
-	max_dma = PFN_DOWN(arm64_dma_phys_limit);
-	zone_size[ZONE_DMA32] = max_dma - min;
+	max_dma32 = PFN_DOWN(arm64_dma32_phys_limit);
+	zone_size[ZONE_DMA32] = max_dma32 - min;
 #endif
-	zone_size[ZONE_NORMAL] = max - max_dma;
+	zone_size[ZONE_NORMAL] = max - max_dma32;
 
 	memcpy(zhole_size, zone_size, sizeof(zhole_size));
 
@@ -215,14 +215,14 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 			continue;
 
 #ifdef CONFIG_ZONE_DMA32
-		if (start < max_dma) {
-			unsigned long dma_end = min(end, max_dma);
+		if (start < max_dma32) {
+			unsigned long dma_end = min(end, max_dma32);
 			zhole_size[ZONE_DMA32] -= dma_end - start;
 		}
 #endif
-		if (end > max_dma) {
+		if (end > max_dma32) {
 			unsigned long normal_end = min(end, max);
-			unsigned long normal_start = max(start, max_dma);
+			unsigned long normal_start = max(start, max_dma32);
 			zhole_size[ZONE_NORMAL] -= normal_end - normal_start;
 		}
 	}
@@ -407,9 +407,9 @@ void __init arm64_memblock_init(void)
 
 	/* 4GB maximum for 32-bit only capable devices */
 	if (IS_ENABLED(CONFIG_ZONE_DMA32))
-		arm64_dma_phys_limit = max_zone_dma_phys();
+		arm64_dma32_phys_limit = max_zone_dma32_phys();
 	else
-		arm64_dma_phys_limit = PHYS_MASK + 1;
+		arm64_dma32_phys_limit = PHYS_MASK + 1;
 
 	reserve_crashkernel();
 
@@ -417,7 +417,7 @@ void __init arm64_memblock_init(void)
 
 	high_memory = __va(memblock_end_of_DRAM() - 1) + 1;
 
-	dma_contiguous_reserve(arm64_dma_phys_limit);
+	dma_contiguous_reserve(arm64_dma32_phys_limit);
 }
 
 void __init bootmem_init(void)
@@ -521,7 +521,7 @@ static void __init free_unused_memmap(void)
 void __init mem_init(void)
 {
 	if (swiotlb_force == SWIOTLB_FORCE ||
-	    max_pfn > (arm64_dma_phys_limit >> PAGE_SHIFT))
+	    max_pfn > (arm64_dma32_phys_limit >> PAGE_SHIFT))
 		swiotlb_init(1);
 	else
 		swiotlb_force = SWIOTLB_NO_FORCE;
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 07/11] arm64: re-introduce max_zone_dma_phys()
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (5 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 06/11] arm64: rename variables used to calculate ZONE_DMA32's size Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 08/11] arm64: use both ZONE_DMA and ZONE_DMA32 Nicolas Saenz Julienne
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, linux-kernel
  Cc: phill, f.fainelli, frowand.list, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

Some devices might have multiple interconnects with different DMA
addressing limitations. This function provides the higher physical
address accessible by all peripherals on the SoC. If such limitation
doesn't exist it'll return the maximum physical address of the 32 bit
addressable area.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2:
- Update function's behavior to fit new dma zones split
- Use dma_zone_size
- Take into account devices with a hardcoded DMA offset

 arch/arm64/mm/init.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 8956c22634dd..bc7999020c71 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -174,6 +174,17 @@ static phys_addr_t __init max_zone_dma32_phys(void)
 	return min(offset + (1ULL << 32), memblock_end_of_DRAM());
 }
 
+static phys_addr_t __init max_zone_dma_phys(void)
+
+{
+	phys_addr_t offset = memblock_start_of_DRAM() & GENMASK_ULL(63, 32);
+
+	if (dma_zone_size)
+		return min(offset + dma_zone_size, memblock_end_of_DRAM());
+	else
+		return max_zone_dma32_phys();
+}
+
 #ifdef CONFIG_NUMA
 
 static void __init zone_sizes_init(unsigned long min, unsigned long max)
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 08/11] arm64: use both ZONE_DMA and ZONE_DMA32
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (6 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 07/11] arm64: re-introduce max_zone_dma_phys() Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable Nicolas Saenz Julienne
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Will Deacon
  Cc: phill, f.fainelli, linux-kernel, eric, mbrugger,
	linux-rpi-kernel, akpm, frowand.list, nsaenzjulienne

So far all arm64 devices have supported 32 bit DMA masks for their
peripherals. This is not true anymore for the Raspberry Pi 4 as most of
it's peripherals can only address the first GB or memory of a total of
up to 4 GB.

This goes against ZONE_DMA32's original intent, and breaks other
subsystems as it's expected for ZONE_DMA32 to be addressable with a 32
bit mask. So it was decided to use ZONE_DMA for this specific case.

ZONE_DMA will contain the memory addressable by all the SoC's
peripherals and ZONE_DMA32 the rest of the 32 bit addressable memory. If
all peripherals where able to address the whole 32 bit addressable space
ZONE_DMA32 will be left empty.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2:
- ZONE_DMA will never be left empty
- Update comment to reflect new zones split

 arch/arm64/Kconfig   |  4 ++++
 arch/arm64/mm/init.c | 39 +++++++++++++++++++++++++++++++--------
 2 files changed, 35 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 3adcec05b1f6..a9fd71d3bc8e 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -266,6 +266,10 @@ config GENERIC_CSUM
 config GENERIC_CALIBRATE_DELAY
 	def_bool y
 
+config ZONE_DMA
+	bool "Support DMA zone" if EXPERT
+	default y
+
 config ZONE_DMA32
 	bool "Support DMA32 zone" if EXPERT
 	default y
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index bc7999020c71..c51ce79b692b 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -50,6 +50,14 @@
 s64 memstart_addr __ro_after_init = -1;
 EXPORT_SYMBOL(memstart_addr);
 
+/*
+ * We create both ZONE_DMA and ZONE_DMA32. ZONE_DMA's size is decided based on
+ * whether all the device's peripherals are able to address the first naturally
+ * aligned 4G of memory. If not, ZONE_DMA covers the area common to all them
+ * and ZONE_DMA32 the rest. If ZONE_DMA fits the whole 4G area, ZONE_DMA32 is
+ * left empty.
+ */
+phys_addr_t arm64_dma_phys_limit __ro_after_init;
 phys_addr_t arm64_dma32_phys_limit __ro_after_init;
 
 #ifdef CONFIG_KEXEC_CORE
@@ -191,6 +199,9 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 {
 	unsigned long max_zone_pfns[MAX_NR_ZONES]  = {0};
 
+#ifdef CONFIG_ZONE_DMA
+	max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit);
+#endif
 #ifdef CONFIG_ZONE_DMA32
 	max_zone_pfns[ZONE_DMA32] = PFN_DOWN(arm64_dma32_phys_limit);
 #endif
@@ -206,13 +217,17 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 	struct memblock_region *reg;
 	unsigned long zone_size[MAX_NR_ZONES], zhole_size[MAX_NR_ZONES];
 	unsigned long max_dma32 = min;
+	unsigned long max_dma = min;
 
 	memset(zone_size, 0, sizeof(zone_size));
 
-	/* 4GB maximum for 32-bit only capable devices */
+#ifdef CONFIG_ZONE_DMA
+	max_dma = PFN_DOWN(arm64_dma_phys_limit);
+	zone_size[ZONE_DMA] = max_dma - min;
+#endif
 #ifdef CONFIG_ZONE_DMA32
 	max_dma32 = PFN_DOWN(arm64_dma32_phys_limit);
-	zone_size[ZONE_DMA32] = max_dma32 - min;
+	zone_size[ZONE_DMA32] = max_dma32 - max_dma;
 #endif
 	zone_size[ZONE_NORMAL] = max - max_dma32;
 
@@ -224,11 +239,17 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 
 		if (start >= max)
 			continue;
-
+#ifdef CONFIG_ZONE_DMA
+		if (start < max_dma) {
+			unsigned long dma_end = min_not_zero(end, max_dma);
+			zhole_size[ZONE_DMA] -= dma_end - start;
+		}
+#endif
 #ifdef CONFIG_ZONE_DMA32
 		if (start < max_dma32) {
-			unsigned long dma_end = min(end, max_dma32);
-			zhole_size[ZONE_DMA32] -= dma_end - start;
+			unsigned long dma32_end = min(end, max_dma32);
+			unsigned long dma32_start = max(start, max_dma);
+			zhole_size[ZONE_DMA32] -= dma32_end - dma32_start;
 		}
 #endif
 		if (end > max_dma32) {
@@ -416,7 +437,9 @@ void __init arm64_memblock_init(void)
 
 	early_init_fdt_scan_reserved_mem();
 
-	/* 4GB maximum for 32-bit only capable devices */
+	if (IS_ENABLED(CONFIG_ZONE_DMA))
+		arm64_dma_phys_limit = max_zone_dma_phys();
+
 	if (IS_ENABLED(CONFIG_ZONE_DMA32))
 		arm64_dma32_phys_limit = max_zone_dma32_phys();
 	else
@@ -428,7 +451,7 @@ void __init arm64_memblock_init(void)
 
 	high_memory = __va(memblock_end_of_DRAM() - 1) + 1;
 
-	dma_contiguous_reserve(arm64_dma32_phys_limit);
+	dma_contiguous_reserve(arm64_dma_phys_limit ? : arm64_dma32_phys_limit);
 }
 
 void __init bootmem_init(void)
@@ -531,7 +554,7 @@ static void __init free_unused_memmap(void)
  */
 void __init mem_init(void)
 {
-	if (swiotlb_force == SWIOTLB_FORCE ||
+	if (swiotlb_force == SWIOTLB_FORCE || arm64_dma_phys_limit ||
 	    max_pfn > (arm64_dma32_phys_limit >> PAGE_SHIFT))
 		swiotlb_init(1);
 	else
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (7 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 08/11] arm64: use both ZONE_DMA and ZONE_DMA32 Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-26  7:05   ` Christoph Hellwig
  2019-08-26 11:33   ` Michael Ellerman
  2019-08-20 14:58 ` [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask Nicolas Saenz Julienne
  2019-08-20 14:58 ` [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type' Nicolas Saenz Julienne
  10 siblings, 2 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Marek Szyprowski
  Cc: phill, linux-s390, f.fainelli, Vasily Gorbik,
	Christian Borntraeger, Michael Ellerman, frowand.list,
	linuxppc-dev, Heiko Carstens, linux-kernel, eric, mbrugger,
	Paul Mackerras, linux-rpi-kernel, Benjamin Herrenschmidt, akpm,
	will, nsaenzjulienne

Some architectures, notably arm64, are interested in tweaking this
depending on their runtime dma addressing limitations.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2:
- Rename new variable to zone_dma_bits
- Update comment with Christoph's suggestion
- Remove old powerpc comment

 arch/powerpc/include/asm/page.h |  9 ---------
 arch/powerpc/mm/mem.c           | 16 +++++++++++-----
 arch/s390/include/asm/page.h    |  2 --
 arch/s390/mm/init.c             |  1 +
 include/linux/dma-direct.h      |  2 ++
 kernel/dma/direct.c             | 13 ++++++-------
 6 files changed, 20 insertions(+), 23 deletions(-)

diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h
index 0d52f57fca04..73668a21ae78 100644
--- a/arch/powerpc/include/asm/page.h
+++ b/arch/powerpc/include/asm/page.h
@@ -319,13 +319,4 @@ struct vm_area_struct;
 #endif /* __ASSEMBLY__ */
 #include <asm/slice.h>
 
-/*
- * Allow 30-bit DMA for very limited Broadcom wifi chips on many powerbooks.
- */
-#ifdef CONFIG_PPC32
-#define ARCH_ZONE_DMA_BITS 30
-#else
-#define ARCH_ZONE_DMA_BITS 31
-#endif
-
 #endif /* _ASM_POWERPC_PAGE_H */
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 9191a66b3bc5..2a69f87585df 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -31,6 +31,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/memremap.h>
+#include <linux/dma-direct.h>
 
 #include <asm/pgalloc.h>
 #include <asm/prom.h>
@@ -201,10 +202,10 @@ static int __init mark_nonram_nosave(void)
  * everything else. GFP_DMA32 page allocations automatically fall back to
  * ZONE_DMA.
  *
- * By using 31-bit unconditionally, we can exploit ARCH_ZONE_DMA_BITS to
- * inform the generic DMA mapping code.  32-bit only devices (if not handled
- * by an IOMMU anyway) will take a first dip into ZONE_NORMAL and get
- * otherwise served by ZONE_DMA.
+ * By using 31-bit unconditionally, we can exploit zone_dma_bits to inform the
+ * generic DMA mapping code.  32-bit only devices (if not handled by an IOMMU
+ * anyway) will take a first dip into ZONE_NORMAL and get otherwise served by
+ * ZONE_DMA.
  */
 static unsigned long max_zone_pfns[MAX_NR_ZONES];
 
@@ -237,9 +238,14 @@ void __init paging_init(void)
 	printk(KERN_DEBUG "Memory hole size: %ldMB\n",
 	       (long int)((top_of_ram - total_ram) >> 20));
 
+	if (IS_ENABLED(CONFIG_PPC32))
+		zone_dma_bits = 30;
+	else
+		zone_dma_bits = 31;
+
 #ifdef CONFIG_ZONE_DMA
 	max_zone_pfns[ZONE_DMA]	= min(max_low_pfn,
-				      1UL << (ARCH_ZONE_DMA_BITS - PAGE_SHIFT));
+				      1UL << (zone_dma_bits - PAGE_SHIFT));
 #endif
 	max_zone_pfns[ZONE_NORMAL] = max_low_pfn;
 #ifdef CONFIG_HIGHMEM
diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h
index 823578c6b9e2..a4d38092530a 100644
--- a/arch/s390/include/asm/page.h
+++ b/arch/s390/include/asm/page.h
@@ -177,8 +177,6 @@ static inline int devmem_is_allowed(unsigned long pfn)
 #define VM_DATA_DEFAULT_FLAGS	(VM_READ | VM_WRITE | \
 				 VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
 
-#define ARCH_ZONE_DMA_BITS	31
-
 #include <asm-generic/memory_model.h>
 #include <asm-generic/getorder.h>
 
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 20340a03ad90..bd98465b8b9f 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -118,6 +118,7 @@ void __init paging_init(void)
 
 	sparse_memory_present_with_active_regions(MAX_NUMNODES);
 	sparse_init();
+	zone_dma_bits = 31;
 	memset(max_zone_pfns, 0, sizeof(max_zone_pfns));
 	max_zone_pfns[ZONE_DMA] = PFN_DOWN(MAX_DMA_ADDRESS);
 	max_zone_pfns[ZONE_NORMAL] = max_low_pfn;
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index adf993a3bd58..d03af3605460 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -5,6 +5,8 @@
 #include <linux/dma-mapping.h>
 #include <linux/mem_encrypt.h>
 
+extern unsigned int zone_dma_bits;
+
 #ifdef CONFIG_ARCH_HAS_PHYS_TO_DMA
 #include <asm/dma-direct.h>
 #else
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 795c9b095d75..b23cd65f26e0 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -16,12 +16,11 @@
 #include <linux/swiotlb.h>
 
 /*
- * Most architectures use ZONE_DMA for the first 16 Megabytes, but
- * some use it for entirely different regions:
+ * Most architectures use ZONE_DMA for the first 16 Megabytes, but some use it
+ * it for entirely different regions. In that case the arch code needs to
+ * override the variable below for dma-direct to work properly.
  */
-#ifndef ARCH_ZONE_DMA_BITS
-#define ARCH_ZONE_DMA_BITS 24
-#endif
+unsigned int zone_dma_bits __ro_after_init = 24;
 
 static void report_addr(struct device *dev, dma_addr_t dma_addr, size_t size)
 {
@@ -69,7 +68,7 @@ static gfp_t __dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask,
 	 * Note that GFP_DMA32 and GFP_DMA are no ops without the corresponding
 	 * zones.
 	 */
-	if (*phys_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))
+	if (*phys_mask <= DMA_BIT_MASK(zone_dma_bits))
 		return GFP_DMA;
 	if (*phys_mask <= DMA_BIT_MASK(32))
 		return GFP_DMA32;
@@ -387,7 +386,7 @@ int dma_direct_supported(struct device *dev, u64 mask)
 	u64 min_mask;
 
 	if (IS_ENABLED(CONFIG_ZONE_DMA))
-		min_mask = DMA_BIT_MASK(ARCH_ZONE_DMA_BITS);
+		min_mask = DMA_BIT_MASK(zone_dma_bits);
 	else
 		min_mask = DMA_BIT_MASK(32);
 
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (8 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-26  7:06   ` Christoph Hellwig
  2019-08-20 14:58 ` [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type' Nicolas Saenz Julienne
  10 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, linux-kernel
  Cc: phill, f.fainelli, frowand.list, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

With the introduction of ZONE_DMA in arm64 devices are not forced to
support 32 bit DMA masks. We have to inform dma-direct of this
limitation whenever it happens.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
---

Changes in v2:
- Make sure to filter the higher part of arm64_dma_phys_limit
- Rename variable to zone_dma_bits

 arch/arm64/mm/init.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index c51ce79b692b..c5e619f21ad8 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -22,6 +22,7 @@
 #include <linux/of_fdt.h>
 #include <linux/dma-mapping.h>
 #include <linux/dma-contiguous.h>
+#include <linux/dma-direct.h>
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
 #include <linux/vmalloc.h>
@@ -437,8 +438,10 @@ void __init arm64_memblock_init(void)
 
 	early_init_fdt_scan_reserved_mem();
 
-	if (IS_ENABLED(CONFIG_ZONE_DMA))
+	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
 		arm64_dma_phys_limit = max_zone_dma_phys();
+		zone_dma_bits = ilog2((arm64_dma_phys_limit - 1) & GENMASK_ULL(31, 0)) + 1;
+	}
 
 	if (IS_ENABLED(CONFIG_ZONE_DMA32))
 		arm64_dma32_phys_limit = max_zone_dma32_phys();
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type'
  2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
                   ` (9 preceding siblings ...)
  2019-08-20 14:58 ` [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask Nicolas Saenz Julienne
@ 2019-08-20 14:58 ` Nicolas Saenz Julienne
  2019-08-26  7:07   ` Christoph Hellwig
  10 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 14:58 UTC (permalink / raw)
  To: catalin.marinas, hch, wahrenst, marc.zyngier, robh+dt,
	Robin Murphy, linux-arm-kernel, devicetree, linux-arch, iommu,
	linux-mm, linux-riscv, Paul Walmsley, Palmer Dabbelt, Albert Ou
  Cc: phill, f.fainelli, frowand.list, linux-kernel, eric, mbrugger,
	linux-rpi-kernel, akpm, will, nsaenzjulienne

These zones usage has evolved with time and the comments were outdated.
This joins both ZONE_DMA and ZONE_DMA32 explanation and gives up to date
examples on how they are used on different architectures.

Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>

---

Changes in v2:
- Try another approach merging both zones explanations into one
- Address Christoph's comments
- If this approach doesn't get much traction I'll just drop the patch
  from the series as it's not really essential

 include/linux/mmzone.h | 46 +++++++++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index d77d717c620c..9c150223d41f 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -356,33 +356,41 @@ struct per_cpu_nodestat {
 #endif /* !__GENERATING_BOUNDS.H */
 
 enum zone_type {
-#ifdef CONFIG_ZONE_DMA
 	/*
-	 * ZONE_DMA is used when there are devices that are not able
-	 * to do DMA to all of addressable memory (ZONE_NORMAL). Then we
-	 * carve out the portion of memory that is needed for these devices.
-	 * The range is arch specific.
+	 * ZONE_DMA and ZONE_DMA32 are used when there are peripherals not able
+	 * to DMA to all of the addressable memory (ZONE_NORMAL).
+	 * On architectures where this area covers the whole 32 bit address
+	 * space ZONE_DMA32 is used. ZONE_DMA is left for the ones with smaller
+	 * DMA addressing constraints. This distinction is important as a 32bit
+	 * DMA mask is assumed when ZONE_DMA32 is defined. Some 64-bit
+	 * platforms may need both zones as they support peripherals with
+	 * different DMA addressing limitations.
+	 *
+	 * Some examples:
+	 *
+	 *  - i386 and x86_64 have a fixed 16M ZONE_DMA and ZONE_DMA32 for the
+	 *    rest of the lower 4G.
+	 *
+	 *  - arm only uses ZONE_DMA, the size, up to 4G, may vary depending on
+	 *    the specific device.
+	 *
+	 *  - powerpc only uses ZONE_DMA, the size, up to 2G, may vary
+	 *    depending on the specific device.
 	 *
-	 * Some examples
+	 *  - s390 uses ZONE_DMA fixed to the lower 2G.
 	 *
-	 * Architecture		Limit
-	 * ---------------------------
-	 * parisc, ia64, sparc	<4G
-	 * s390, powerpc	<2G
-	 * arm			Various
-	 * alpha		Unlimited or 0-16MB.
+	 *  - arm64 uses ZONE_DMA to mark the area addresable by all
+	 *    peripherals on the device, and ZONE_DMA32 for the rest of the
+	 *    lower 4G. ZONE_DMA32 might be left empty.
 	 *
-	 * i386, x86_64 and multiple other arches
-	 * 			<16M.
+	 *  - ia64 and riscv only use ZONE_DMA32.
+	 *
+	 *  - parisc uses neither.
 	 */
+#ifdef CONFIG_ZONE_DMA
 	ZONE_DMA,
 #endif
 #ifdef CONFIG_ZONE_DMA32
-	/*
-	 * x86_64 needs two ZONE_DMAs because it supports devices that are
-	 * only able to do DMA to the lower 16M but also 32 bit devices that
-	 * can only do DMA areas below 4G.
-	 */
 	ZONE_DMA32,
 #endif
 	/*
-- 
2.22.0

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size()
  2019-08-20 14:58 ` [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size() Nicolas Saenz Julienne
@ 2019-08-20 17:14   ` Rob Herring
  0 siblings, 0 replies; 27+ messages in thread
From: Rob Herring @ 2019-08-20 17:14 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: open list:GENERIC INCLUDE/ASM HEADER FILES, devicetree,
	moderated list:BROADCOM BCM2835 ARM ARCHITECTURE,
	Florian Fainelli, Andrew Morton, Frank Rowand, Eric Anholt,
	Marc Zyngier, Catalin Marinas, Will Deacon, linux-kernel,
	linux-mm, Linux IOMMU, Matthias Brugger, Stefan Wahren,
	linux-riscv, Robin Murphy, Christoph Hellwig,
	moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE, phill

On Tue, Aug 20, 2019 at 9:58 AM Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Some devices might have weird DMA addressing limitations that only apply
> to a subset of the available peripherals. For example the Raspberry Pi 4
> has two interconnects, one able to address the whole lower 4G memory
> area and another one limited to the lower 1G.
>
> Being an uncommon situation we simply hardcode the device wide DMA
> addressable memory size conditionally to the machine compatible name and
> set 'dma_zone_size' accordingly.
>
> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
>
> ---
>
> Changes in v2:
> - New approach to getting dma_zone_size, instead of parsing the dts we
>   hardcode it conditionally to the machine compatible name.
>
>  drivers/of/fdt.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
> index 06ffbd39d9af..f756e8c05a77 100644
> --- a/drivers/of/fdt.c
> +++ b/drivers/of/fdt.c
> @@ -27,6 +27,7 @@
>
>  #include <asm/setup.h>  /* for COMMAND_LINE_SIZE */
>  #include <asm/page.h>
> +#include <asm/dma.h>   /* for dma_zone_size */
>
>  #include "of_private.h"
>
> @@ -1195,6 +1196,12 @@ void __init early_init_dt_scan_nodes(void)
>         of_scan_flat_dt(early_init_dt_scan_memory, NULL);
>  }
>
> +void __init early_init_dt_get_dma_zone_size(void)

static

With that,

Reviewed-by: Rob Herring <robh@kernel.org>

> +{
> +       if (of_fdt_machine_is_compatible("brcm,bcm2711"))
> +               dma_zone_size = 0x3c000000;
> +}
> +
>  bool __init early_init_dt_scan(void *params)
>  {
>         bool status;
> @@ -1204,6 +1211,7 @@ bool __init early_init_dt_scan(void *params)
>                 return false;
>
>         early_init_dt_scan_nodes();
> +       early_init_dt_get_dma_zone_size();
>         return true;
>  }
>
> --
> 2.22.0
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function
  2019-08-20 14:58 ` [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function Nicolas Saenz Julienne
@ 2019-08-20 17:16   ` Rob Herring
  2019-08-20 17:27     ` Nicolas Saenz Julienne
  0 siblings, 1 reply; 27+ messages in thread
From: Rob Herring @ 2019-08-20 17:16 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: open list:GENERIC INCLUDE/ASM HEADER FILES, devicetree,
	moderated list:BROADCOM BCM2835 ARM ARCHITECTURE,
	Florian Fainelli, Andrew Morton, Frank Rowand, Eric Anholt,
	Marc Zyngier, Catalin Marinas, Will Deacon, linux-kernel,
	linux-mm, Linux IOMMU, Matthias Brugger, Stefan Wahren,
	linux-riscv, Robin Murphy, Christoph Hellwig,
	moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE, phill

On Tue, Aug 20, 2019 at 9:58 AM Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Provides the same functionality as of_machine_is_compatible.
>
> Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> ---
>
> Changes in v2: None
>
>  drivers/of/fdt.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
> index 9cdf14b9aaab..06ffbd39d9af 100644
> --- a/drivers/of/fdt.c
> +++ b/drivers/of/fdt.c
> @@ -802,6 +802,13 @@ const char * __init of_flat_dt_get_machine_name(void)
>         return name;
>  }
>
> +static const int __init of_fdt_machine_is_compatible(char *name)

No point in const return (though name could possibly be const), and
the return could be bool instead.

With that,

Reviewed-by: Rob Herring <robh@kernel.org>

> +{
> +       unsigned long dt_root = of_get_flat_dt_root();
> +
> +       return of_flat_dt_is_compatible(dt_root, name);
> +}
> +
>  /**
>   * of_flat_dt_match_machine - Iterate match tables to find matching machine.
>   *
> --
> 2.22.0
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function
  2019-08-20 17:16   ` Rob Herring
@ 2019-08-20 17:27     ` Nicolas Saenz Julienne
  0 siblings, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-20 17:27 UTC (permalink / raw)
  To: Rob Herring
  Cc: open list:GENERIC INCLUDE/ASM HEADER FILES, devicetree,
	Florian Fainelli, phill, Will Deacon, linux-mm, Marc Zyngier,
	Catalin Marinas, linux-kernel, Christoph Hellwig, Eric Anholt,
	Linux IOMMU, Matthias Brugger,
	moderated list:BROADCOM BCM2835 ARM ARCHITECTURE, Robin Murphy,
	Andrew Morton, Frank Rowand, linux-riscv,
	moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE,
	Stefan Wahren

[-- Attachment #1.1: Type: text/plain, Size: 1094 bytes --]

Hi Rob,
thanks for the rewiew.

On Tue, 2019-08-20 at 12:16 -0500, Rob Herring wrote:
> On Tue, Aug 20, 2019 at 9:58 AM Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > Provides the same functionality as of_machine_is_compatible.
> > 
> > Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> > ---
> > 
> > Changes in v2: None
> > 
> >  drivers/of/fdt.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
> > index 9cdf14b9aaab..06ffbd39d9af 100644
> > --- a/drivers/of/fdt.c
> > +++ b/drivers/of/fdt.c
> > @@ -802,6 +802,13 @@ const char * __init of_flat_dt_get_machine_name(void)
> >         return name;
> >  }
> > 
> > +static const int __init of_fdt_machine_is_compatible(char *name)
> 
> No point in const return (though name could possibly be const), and
> the return could be bool instead.

Sorry, I completely missed that const, shouldn't have been there to begin with.

I'll add a const to the name argument and return a bool on the next version.

Regards,
Nicolas



[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable
  2019-08-20 14:58 ` [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable Nicolas Saenz Julienne
@ 2019-08-26  7:05   ` Christoph Hellwig
  2019-08-26 11:33   ` Michael Ellerman
  1 sibling, 0 replies; 27+ messages in thread
From: Christoph Hellwig @ 2019-08-26  7:05 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: catalin.marinas, Heiko Carstens, eric, Paul Mackerras,
	linux-riscv, will, hch, linux-arch, linux-s390, f.fainelli,
	frowand.list, Christian Borntraeger, Benjamin Herrenschmidt,
	devicetree, Vasily Gorbik, marc.zyngier, robh+dt,
	linux-rpi-kernel, linux-arm-kernel, phill, mbrugger, linux-mm,
	linuxppc-dev, linux-kernel, iommu, wahrenst, Michael Ellerman,
	akpm, Robin Murphy

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask
  2019-08-20 14:58 ` [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask Nicolas Saenz Julienne
@ 2019-08-26  7:06   ` Christoph Hellwig
  2019-08-26 11:08     ` Nicolas Saenz Julienne
  0 siblings, 1 reply; 27+ messages in thread
From: Christoph Hellwig @ 2019-08-26  7:06 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: catalin.marinas, eric, linux-riscv, will, hch, linux-arch,
	f.fainelli, frowand.list, devicetree, marc.zyngier, robh+dt,
	linux-rpi-kernel, linux-arm-kernel, phill, mbrugger, linux-mm,
	linux-kernel, iommu, wahrenst, akpm, Robin Murphy

On Tue, Aug 20, 2019 at 04:58:18PM +0200, Nicolas Saenz Julienne wrote:
> -	if (IS_ENABLED(CONFIG_ZONE_DMA))
> +	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
>  		arm64_dma_phys_limit = max_zone_dma_phys();
> +		zone_dma_bits = ilog2((arm64_dma_phys_limit - 1) & GENMASK_ULL(31, 0)) + 1;

This adds a way too long line.  I also find the use of GENMASK_ULL
horribly obsfucating, but I know that opinion is't shared by everyone.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type'
  2019-08-20 14:58 ` [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type' Nicolas Saenz Julienne
@ 2019-08-26  7:07   ` Christoph Hellwig
  0 siblings, 0 replies; 27+ messages in thread
From: Christoph Hellwig @ 2019-08-26  7:07 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: catalin.marinas, Palmer Dabbelt, eric, linux-riscv, will, hch,
	linux-arch, f.fainelli, frowand.list, devicetree, Albert Ou,
	marc.zyngier, robh+dt, linux-rpi-kernel, Paul Walmsley,
	linux-arm-kernel, phill, mbrugger, linux-mm, linux-kernel, iommu,
	wahrenst, akpm, Robin Murphy

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-20 14:58 ` [PATCH v2 01/11] asm-generic: add dma_zone_size Nicolas Saenz Julienne
@ 2019-08-26  7:09   ` Christoph Hellwig
  2019-08-26 13:46     ` Nicolas Saenz Julienne
  0 siblings, 1 reply; 27+ messages in thread
From: Christoph Hellwig @ 2019-08-26  7:09 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: catalin.marinas, eric, linux-riscv, frowand.list, hch,
	linux-arch, f.fainelli, will, devicetree, Arnd Bergmann,
	marc.zyngier, robh+dt, linux-rpi-kernel, linux-arm-kernel, phill,
	mbrugger, linux-mm, linux-kernel, iommu, wahrenst, akpm,
	Robin Murphy

On Tue, Aug 20, 2019 at 04:58:09PM +0200, Nicolas Saenz Julienne wrote:
> Some architectures have platform specific DMA addressing limitations.
> This will allow for hardware description code to provide the constraints
> in a generic manner, so as for arch code to properly setup it's memory
> zones and DMA mask.

I know this just spreads the arm code, but I still kinda hate it.

MAX_DMA_ADDRESS is such an oddly defined concepts.  We have the mm
code that uses it to start allocating after the dma zones, but
I think that would better be done using a function returning
1 << max(zone_dma_bits, 32) or so.  Then we have about a handful
of drivers using it that all seem rather bogus, and one of which
I think are usable on arm64.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask
  2019-08-26  7:06   ` Christoph Hellwig
@ 2019-08-26 11:08     ` Nicolas Saenz Julienne
  2019-08-27  7:03       ` Petr Tesarik
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-26 11:08 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: linux-arch, devicetree, linux-rpi-kernel, f.fainelli, will, eric,
	marc.zyngier, catalin.marinas, frowand.list, linux-kernel,
	linux-mm, iommu, robh+dt, wahrenst, mbrugger, linux-riscv,
	Robin Murphy, akpm, linux-arm-kernel, phill

[-- Attachment #1.1: Type: text/plain, Size: 821 bytes --]

On Mon, 2019-08-26 at 09:06 +0200, Christoph Hellwig wrote:
> On Tue, Aug 20, 2019 at 04:58:18PM +0200, Nicolas Saenz Julienne wrote:
> > -	if (IS_ENABLED(CONFIG_ZONE_DMA))
> > +	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
> >  		arm64_dma_phys_limit = max_zone_dma_phys();
> > +		zone_dma_bits = ilog2((arm64_dma_phys_limit - 1) &
> > GENMASK_ULL(31, 0)) + 1;
>
Hi Christoph,
thanks for the rewiews.

> This adds a way too long line.

I know, I couldn't find a way to split the operation without making it even
harder to read. I'll find a solution.

> I also find the use of GENMASK_ULL
> horribly obsfucating, but I know that opinion is't shared by everyone.

Don't have any preference so I'll happily change it. Any suggestions? Using the
explicit 0xffffffffULL seems hard to read, how about SZ_4GB - 1?


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable
  2019-08-20 14:58 ` [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable Nicolas Saenz Julienne
  2019-08-26  7:05   ` Christoph Hellwig
@ 2019-08-26 11:33   ` Michael Ellerman
  1 sibling, 0 replies; 27+ messages in thread
From: Michael Ellerman @ 2019-08-26 11:33 UTC (permalink / raw)
  To: Nicolas Saenz Julienne, catalin.marinas, hch, wahrenst,
	marc.zyngier, robh+dt, Robin Murphy, linux-arm-kernel,
	devicetree, linux-arch, iommu, linux-mm, linux-riscv,
	Marek Szyprowski
  Cc: linux-s390, f.fainelli, Vasily Gorbik, Christian Borntraeger,
	frowand.list, linuxppc-dev, Heiko Carstens, linux-kernel, eric,
	mbrugger, Paul Mackerras, linux-rpi-kernel,
	Benjamin Herrenschmidt, akpm, will, nsaenzjulienne

Nicolas Saenz Julienne <nsaenzjulienne@suse.de> writes:
> diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h
> index 0d52f57fca04..73668a21ae78 100644
> --- a/arch/powerpc/include/asm/page.h
> +++ b/arch/powerpc/include/asm/page.h
> @@ -319,13 +319,4 @@ struct vm_area_struct;
>  #endif /* __ASSEMBLY__ */
>  #include <asm/slice.h>
>  
> -/*
> - * Allow 30-bit DMA for very limited Broadcom wifi chips on many powerbooks.

This comment got lost.

> diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> index 9191a66b3bc5..2a69f87585df 100644
> --- a/arch/powerpc/mm/mem.c
> +++ b/arch/powerpc/mm/mem.c
> @@ -237,9 +238,14 @@ void __init paging_init(void)
>  	printk(KERN_DEBUG "Memory hole size: %ldMB\n",
>  	       (long int)((top_of_ram - total_ram) >> 20));
>  
> +	if (IS_ENABLED(CONFIG_PPC32))

Can you please propagate it here?

> +		zone_dma_bits = 30;
> +	else
> +		zone_dma_bits = 31;
> +

cheers
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-26  7:09   ` Christoph Hellwig
@ 2019-08-26 13:46     ` Nicolas Saenz Julienne
  2019-08-28  9:44       ` Nicolas Saenz Julienne
  2019-08-30 14:45       ` Catalin Marinas
  0 siblings, 2 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-26 13:46 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: catalin.marinas, linux-mm, linux-riscv, will, linux-arch,
	f.fainelli, frowand.list, devicetree, Arnd Bergmann,
	marc.zyngier, robh+dt, linux-rpi-kernel, linux-arm-kernel, phill,
	mbrugger, eric, linux-kernel, iommu, wahrenst, akpm,
	Robin Murphy

[-- Attachment #1.1: Type: text/plain, Size: 1724 bytes --]

On Mon, 2019-08-26 at 09:09 +0200, Christoph Hellwig wrote:
> On Tue, Aug 20, 2019 at 04:58:09PM +0200, Nicolas Saenz Julienne wrote:
> > Some architectures have platform specific DMA addressing limitations.
> > This will allow for hardware description code to provide the constraints
> > in a generic manner, so as for arch code to properly setup it's memory
> > zones and DMA mask.
> 
> I know this just spreads the arm code, but I still kinda hate it.

Rob's main concern was finding a way to pass the constraint from HW definition
to arch without widening fdt's architecture specific function surface. I'd say
it's fair to argue that having a generic mechanism makes sense as it'll now
traverse multiple archs and subsystems.

I get adding globals like this is not very appealing, yet I went with it as it
was the easier to integrate with arm's code. Any alternative suggestions?

> MAX_DMA_ADDRESS is such an oddly defined concepts.  We have the mm
> code that uses it to start allocating after the dma zones, but
> I think that would better be done using a function returning
> 1 << max(zone_dma_bits, 32) or so.  Then we have about a handful
> of drivers using it that all seem rather bogus, and one of which
> I think are usable on arm64.

Is it safe to assume DMA limitations will always be a power of 2? I ask as RPi4
kinda isn't: ZONE_DMA is 0x3c000000 bytes big, I'm approximating the zone mask
to 30 as [0x3c000000 0x3fffffff] isn't defined as memory so it's unlikely that
we´ll encounter buffers there. But I don't know how it could affect mm
initialization code.

This also rules out 'zone_dma_bits' as a mechanism to pass ZONE_DMA's size from
HW definition code to arch's.


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask
  2019-08-26 11:08     ` Nicolas Saenz Julienne
@ 2019-08-27  7:03       ` Petr Tesarik
  0 siblings, 0 replies; 27+ messages in thread
From: Petr Tesarik @ 2019-08-27  7:03 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: catalin.marinas, linux-mm, linux-riscv, frowand.list,
	Christoph Hellwig, linux-arch, f.fainelli, will, devicetree,
	marc.zyngier, robh+dt, linux-rpi-kernel, linux-arm-kernel, phill,
	mbrugger, eric, linux-kernel, iommu, wahrenst, akpm,
	Robin Murphy

[-- Attachment #1.1: Type: text/plain, Size: 820 bytes --]

On Mon, 26 Aug 2019 13:08:50 +0200
Nicolas Saenz Julienne <nsaenzjulienne@suse.de> wrote:

> On Mon, 2019-08-26 at 09:06 +0200, Christoph Hellwig wrote:
> > On Tue, Aug 20, 2019 at 04:58:18PM +0200, Nicolas Saenz Julienne wrote:  
> > > -	if (IS_ENABLED(CONFIG_ZONE_DMA))
> > > +	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
> > >  		arm64_dma_phys_limit = max_zone_dma_phys();
> > > +		zone_dma_bits = ilog2((arm64_dma_phys_limit - 1) &
> > > GENMASK_ULL(31, 0)) + 1;  
> >  
> Hi Christoph,
> thanks for the rewiews.
> 
> > This adds a way too long line.  
> 
> I know, I couldn't find a way to split the operation without making it even
> harder to read. I'll find a solution.

If all else fails, move the code to an inline function and call it e.g.
phys_limit_to_dma_bits().

Just my two cents,
Petr T

[-- Attachment #1.2: Digitální podpis OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-26 13:46     ` Nicolas Saenz Julienne
@ 2019-08-28  9:44       ` Nicolas Saenz Julienne
  2019-08-30 14:45       ` Catalin Marinas
  1 sibling, 0 replies; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-28  9:44 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: catalin.marinas, eric, linux-riscv, frowand.list, linux-arch,
	f.fainelli, will, devicetree, Arnd Bergmann, marc.zyngier,
	robh+dt, linux-rpi-kernel, linux-arm-kernel, phill, mbrugger,
	linux-mm, linux-kernel, iommu, wahrenst, akpm, Robin Murphy

[-- Attachment #1.1: Type: text/plain, Size: 2255 bytes --]

On Mon, 2019-08-26 at 15:46 +0200, Nicolas Saenz Julienne wrote:
> On Mon, 2019-08-26 at 09:09 +0200, Christoph Hellwig wrote:
> > On Tue, Aug 20, 2019 at 04:58:09PM +0200, Nicolas Saenz Julienne wrote:
> > > Some architectures have platform specific DMA addressing limitations.
> > > This will allow for hardware description code to provide the constraints
> > > in a generic manner, so as for arch code to properly setup it's memory
> > > zones and DMA mask.
> > 
> > I know this just spreads the arm code, but I still kinda hate it.
> 
> Rob's main concern was finding a way to pass the constraint from HW definition
> to arch without widening fdt's architecture specific function surface. I'd say
> it's fair to argue that having a generic mechanism makes sense as it'll now
> traverse multiple archs and subsystems.
> 
> I get adding globals like this is not very appealing, yet I went with it as it
> was the easier to integrate with arm's code. Any alternative suggestions?
> 
> > MAX_DMA_ADDRESS is such an oddly defined concepts.  We have the mm
> > code that uses it to start allocating after the dma zones, but
> > I think that would better be done using a function returning
> > 1 << max(zone_dma_bits, 32) or so.  Then we have about a handful
> > of drivers using it that all seem rather bogus, and one of which
> > I think are usable on arm64.
> 
> Is it safe to assume DMA limitations will always be a power of 2? I ask as
> RPi4
> kinda isn't: ZONE_DMA is 0x3c000000 bytes big, I'm approximating the zone mask
> to 30 as [0x3c000000 0x3fffffff] isn't defined as memory so it's unlikely that
> we´ll encounter buffers there. But I don't know how it could affect mm
> initialization code.
> 
> This also rules out 'zone_dma_bits' as a mechanism to pass ZONE_DMA's size
> from
> HW definition code to arch's.

Hi Christoph,
I gave it a thought and think this whole MAX_DMA_ADDRESS topic falls out of the
scope of the series. I agree it's something that we should get rid of, but
fixing it isn't going to affect the overall enhancement intended here.  I'd
rather focus on how are we going to pass the DMA zone data into the arch code
and fix MAX_DMA_ADDRESS on another series.

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-26 13:46     ` Nicolas Saenz Julienne
  2019-08-28  9:44       ` Nicolas Saenz Julienne
@ 2019-08-30 14:45       ` Catalin Marinas
  2019-08-30 17:24         ` Nicolas Saenz Julienne
  1 sibling, 1 reply; 27+ messages in thread
From: Catalin Marinas @ 2019-08-30 14:45 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: linux-mm, linux-riscv, will, Christoph Hellwig, linux-arch,
	f.fainelli, frowand.list, devicetree, Arnd Bergmann,
	marc.zyngier, robh+dt, linux-rpi-kernel, linux-arm-kernel, phill,
	mbrugger, eric, linux-kernel, iommu, wahrenst, akpm,
	Robin Murphy

On Mon, Aug 26, 2019 at 03:46:52PM +0200, Nicolas Saenz Julienne wrote:
> On Mon, 2019-08-26 at 09:09 +0200, Christoph Hellwig wrote:
> > On Tue, Aug 20, 2019 at 04:58:09PM +0200, Nicolas Saenz Julienne wrote:
> > > Some architectures have platform specific DMA addressing limitations.
> > > This will allow for hardware description code to provide the constraints
> > > in a generic manner, so as for arch code to properly setup it's memory
> > > zones and DMA mask.
> > 
> > I know this just spreads the arm code, but I still kinda hate it.
> 
> Rob's main concern was finding a way to pass the constraint from HW definition
> to arch without widening fdt's architecture specific function surface. I'd say
> it's fair to argue that having a generic mechanism makes sense as it'll now
> traverse multiple archs and subsystems.
> 
> I get adding globals like this is not very appealing, yet I went with it as it
> was the easier to integrate with arm's code. Any alternative suggestions?

In some discussion with Robin, since it's just RPi4 that we are aware of
having such requirement on arm64, he suggested that we have a permanent
ZONE_DMA on arm64 with a default size of 1GB. It should cover all arm64
SoCs we know of without breaking the single Image binary. The arch/arm
can use its current mach-* support.

I may like this more than the proposed early_init_dt_get_dma_zone_size()
here which checks for specific SoCs (my preferred way was to build the
mask from all buses described in DT but I hadn't realised the
complications).

-- 
Catalin
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-30 14:45       ` Catalin Marinas
@ 2019-08-30 17:24         ` Nicolas Saenz Julienne
  2019-09-02 13:01           ` Christoph Hellwig
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Saenz Julienne @ 2019-08-30 17:24 UTC (permalink / raw)
  To: Catalin Marinas, Christoph Hellwig
  Cc: linux-arch, devicetree, f.fainelli, phill, Arnd Bergmann,
	mbrugger, marc.zyngier, frowand.list, wahrenst, eric,
	linux-kernel, linux-mm, iommu, robh+dt, linux-rpi-kernel,
	Robin Murphy, linux-riscv, will, akpm, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 2088 bytes --]

On Fri, 2019-08-30 at 15:45 +0100, Catalin Marinas wrote:
> On Mon, Aug 26, 2019 at 03:46:52PM +0200, Nicolas Saenz Julienne wrote:
> > On Mon, 2019-08-26 at 09:09 +0200, Christoph Hellwig wrote:
> > > On Tue, Aug 20, 2019 at 04:58:09PM +0200, Nicolas Saenz Julienne wrote:
> > > > Some architectures have platform specific DMA addressing limitations.
> > > > This will allow for hardware description code to provide the constraints
> > > > in a generic manner, so as for arch code to properly setup it's memory
> > > > zones and DMA mask.
> > > 
> > > I know this just spreads the arm code, but I still kinda hate it.
> > 
> > Rob's main concern was finding a way to pass the constraint from HW
> > definition
> > to arch without widening fdt's architecture specific function surface. I'd
> > say
> > it's fair to argue that having a generic mechanism makes sense as it'll now
> > traverse multiple archs and subsystems.
> > 
> > I get adding globals like this is not very appealing, yet I went with it as
> > it
> > was the easier to integrate with arm's code. Any alternative suggestions?
> 
> In some discussion with Robin, since it's just RPi4 that we are aware of
> having such requirement on arm64, he suggested that we have a permanent
> ZONE_DMA on arm64 with a default size of 1GB. It should cover all arm64
> SoCs we know of without breaking the single Image binary. The arch/arm
> can use its current mach-* support.
> 
> I may like this more than the proposed early_init_dt_get_dma_zone_size()
> here which checks for specific SoCs (my preferred way was to build the
> mask from all buses described in DT but I hadn't realised the
> complications).

Hi Catalin, thanks for giving it a thought.

I'll be happy to implement it that way. I agree it's a good compromise.

@Christoph, do you still want the patch where I create 'zone_dma_bits'? With a
hardcoded ZONE_DMA it's not absolutely necessary. Though I remember you said it
was a first step towards being able to initialize dma-direct's min_mask in
meminit.

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v2 01/11] asm-generic: add dma_zone_size
  2019-08-30 17:24         ` Nicolas Saenz Julienne
@ 2019-09-02 13:01           ` Christoph Hellwig
  0 siblings, 0 replies; 27+ messages in thread
From: Christoph Hellwig @ 2019-09-02 13:01 UTC (permalink / raw)
  To: Nicolas Saenz Julienne
  Cc: Catalin Marinas, eric, linux-riscv, frowand.list,
	Christoph Hellwig, linux-arch, f.fainelli, will, devicetree,
	Arnd Bergmann, marc.zyngier, robh+dt, linux-rpi-kernel,
	linux-arm-kernel, phill, mbrugger, linux-mm, linux-kernel, iommu,
	wahrenst, akpm, Robin Murphy

On Fri, Aug 30, 2019 at 07:24:25PM +0200, Nicolas Saenz Julienne wrote:
> I'll be happy to implement it that way. I agree it's a good compromise.
> 
> @Christoph, do you still want the patch where I create 'zone_dma_bits'? With a
> hardcoded ZONE_DMA it's not absolutely necessary. Though I remember you said it
> was a first step towards being able to initialize dma-direct's min_mask in
> meminit.

I do like the variable better than the current #define.  I wonder if
really want a mask or a max_zone_dma_address like variable.  So for this
series feel free to drop the patch.   I'll see if I'll pick it up
later or if we can find some way to automatically propagate that
information from the zone initialization.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, back to index

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-20 14:58 [PATCH v2 00/11] Raspberry Pi 4 DMA addressing support Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 01/11] asm-generic: add dma_zone_size Nicolas Saenz Julienne
2019-08-26  7:09   ` Christoph Hellwig
2019-08-26 13:46     ` Nicolas Saenz Julienne
2019-08-28  9:44       ` Nicolas Saenz Julienne
2019-08-30 14:45       ` Catalin Marinas
2019-08-30 17:24         ` Nicolas Saenz Julienne
2019-09-02 13:01           ` Christoph Hellwig
2019-08-20 14:58 ` [PATCH v2 02/11] arm: use generic dma_zone_size Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 03/11] of/fdt: add of_fdt_machine_is_compatible function Nicolas Saenz Julienne
2019-08-20 17:16   ` Rob Herring
2019-08-20 17:27     ` Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 04/11] of/fdt: add early_init_dt_get_dma_zone_size() Nicolas Saenz Julienne
2019-08-20 17:14   ` Rob Herring
2019-08-20 14:58 ` [PATCH v2 05/11] arm64: mm: use arm64_dma_phys_limit instead of calling max_zone_dma_phys() Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 06/11] arm64: rename variables used to calculate ZONE_DMA32's size Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 07/11] arm64: re-introduce max_zone_dma_phys() Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 08/11] arm64: use both ZONE_DMA and ZONE_DMA32 Nicolas Saenz Julienne
2019-08-20 14:58 ` [PATCH v2 09/11] dma-direct: turn ARCH_ZONE_DMA_BITS into a variable Nicolas Saenz Julienne
2019-08-26  7:05   ` Christoph Hellwig
2019-08-26 11:33   ` Michael Ellerman
2019-08-20 14:58 ` [PATCH v2 10/11] arm64: edit zone_dma_bits to fine tune dma-direct min mask Nicolas Saenz Julienne
2019-08-26  7:06   ` Christoph Hellwig
2019-08-26 11:08     ` Nicolas Saenz Julienne
2019-08-27  7:03       ` Petr Tesarik
2019-08-20 14:58 ` [PATCH v2 11/11] mm: refresh ZONE_DMA and ZONE_DMA32 comments in 'enum zone_type' Nicolas Saenz Julienne
2019-08-26  7:07   ` Christoph Hellwig

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org
	public-inbox-index linux-iommu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git