* clean up the DMA mapping headers
@ 2020-09-30 8:55 Christoph Hellwig
2020-09-30 8:55 ` [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h> Christoph Hellwig
` (9 more replies)
0 siblings, 10 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Hi all,
this series cleans up the dma-mapping headers by moving everything not
required by normal drivers out of <linux/dma-mapping.h> into a new
<linux/dma-map-ops.h> and then folding most other DMA mapping related
headers either into the new dma-map-ops.h one, or by moving them to
kernel/dma/ and thus out of the global scope. A bunch of cleanups
for the DMA CMA code are thrown in as well, as they help keeping the
exposed bits in the header small.
Diffstat:
arch/arm/include/asm/dma-contiguous.h | 15
b/Documentation/admin-guide/kernel-parameters.txt | 2
b/MAINTAINERS | 2
b/arch/alpha/kernel/pci_iommu.c | 2
b/arch/arc/mm/dma.c | 2
b/arch/arm/common/dmabounce.c | 1
b/arch/arm/include/asm/dma-iommu.h | 1
b/arch/arm/include/asm/dma-mapping.h | 1
b/arch/arm/mach-davinci/devices-da8xx.c | 18 -
b/arch/arm/mach-highbank/highbank.c | 2
b/arch/arm/mach-imx/mach-imx27_visstrim_m10.c | 2
b/arch/arm/mach-imx/mach-mx31moboard.c | 2
b/arch/arm/mach-mvebu/coherency.c | 2
b/arch/arm/mach-shmobile/setup-rcar-gen2.c | 2
b/arch/arm/mm/dma-mapping-nommu.c | 1
b/arch/arm/mm/dma-mapping.c | 5
b/arch/arm/mm/init.c | 2
b/arch/arm/xen/mm.c | 2
b/arch/arm64/mm/dma-mapping.c | 2
b/arch/arm64/mm/init.c | 3
b/arch/c6x/mm/dma-coherent.c | 2
b/arch/csky/kernel/setup.c | 2
b/arch/csky/mm/dma-mapping.c | 4
b/arch/hexagon/kernel/dma.c | 2
b/arch/ia64/hp/common/sba_iommu.c | 2
b/arch/ia64/kernel/dma-mapping.c | 2
b/arch/ia64/mm/init.c | 2
b/arch/m68k/kernel/dma.c | 2
b/arch/microblaze/kernel/dma.c | 3
b/arch/microblaze/mm/consistent.c | 2
b/arch/microblaze/mm/init.c | 2
b/arch/mips/jazz/jazzdma.c | 2
b/arch/mips/kernel/setup.c | 2
b/arch/mips/mm/dma-noncoherent.c | 3
b/arch/nds32/kernel/dma.c | 2
b/arch/openrisc/kernel/dma.c | 2
b/arch/parisc/kernel/drivers.c | 1
b/arch/parisc/kernel/pci-dma.c | 2
b/arch/powerpc/include/asm/iommu.h | 2
b/arch/powerpc/include/asm/pci.h | 2
b/arch/powerpc/mm/dma-noncoherent.c | 2
b/arch/powerpc/platforms/ps3/system-bus.c | 2
b/arch/powerpc/platforms/pseries/ibmebus.c | 2
b/arch/powerpc/platforms/pseries/vio.c | 2
b/arch/s390/kernel/setup.c | 2
b/arch/s390/pci/pci_dma.c | 2
b/arch/sh/boards/mach-ap325rxa/setup.c | 1
b/arch/sh/boards/mach-ecovec24/setup.c | 1
b/arch/sh/boards/mach-kfr2r09/setup.c | 2
b/arch/sh/boards/mach-migor/setup.c | 2
b/arch/sh/boards/mach-se/7724/setup.c | 1
b/arch/sh/drivers/pci/fixups-dreamcast.c | 2
b/arch/sh/drivers/pci/pci.c | 1
b/arch/sh/kernel/dma-coherent.c | 2
b/arch/sparc/kernel/iommu.c | 2
b/arch/sparc/kernel/ioport.c | 2
b/arch/sparc/kernel/pci_sun4v.c | 1
b/arch/sparc/mm/io-unit.c | 2
b/arch/sparc/mm/iommu.c | 2
b/arch/x86/include/asm/dma-mapping.h | 2
b/arch/x86/kernel/amd_gart_64.c | 1
b/arch/x86/kernel/pci-dma.c | 2
b/arch/x86/kernel/setup.c | 2
b/arch/x86/xen/pci-swiotlb-xen.c | 2
b/arch/xtensa/kernel/pci-dma.c | 3
b/arch/xtensa/mm/init.c | 2
b/drivers/acpi/arm64/iort.c | 2
b/drivers/acpi/scan.c | 2
b/drivers/base/dd.c | 2
b/drivers/dma-buf/heaps/cma_heap.c | 2
b/drivers/gpu/drm/exynos/exynos_drm_dma.c | 2
b/drivers/gpu/drm/msm/msm_gem.c | 1
b/drivers/iommu/amd/iommu.c | 3
b/drivers/iommu/dma-iommu.c | 3
b/drivers/iommu/intel/iommu.c | 4
b/drivers/media/platform/exynos4-is/fimc-is.c | 1
b/drivers/misc/mic/bus/mic_bus.c | 1
b/drivers/misc/mic/bus/scif_bus.c | 2
b/drivers/misc/mic/bus/scif_bus.h | 2
b/drivers/misc/mic/bus/vop_bus.c | 2
b/drivers/misc/mic/host/mic_boot.c | 1
b/drivers/of/device.c | 1
b/drivers/parisc/ccio-dma.c | 1
b/drivers/parisc/sba_iommu.c | 1
b/drivers/pci/pci-driver.c | 1
b/drivers/pci/xen-pcifront.c | 1
b/drivers/remoteproc/remoteproc_core.c | 1
b/drivers/remoteproc/remoteproc_virtio.c | 2
b/drivers/vdpa/vdpa_sim/vdpa_sim.c | 2
b/drivers/xen/swiotlb-xen.c | 2
b/include/asm-generic/Kbuild | 1
b/include/linux/dma-direct.h | 108 -------
b/include/linux/dma-map-ops.h | 339 ++++++++++++++++++++++
b/include/linux/dma-mapping.h | 172 -----------
b/kernel/dma/Kconfig | 2
b/kernel/dma/coherent.c | 1
b/kernel/dma/contiguous.c | 42 ++
b/kernel/dma/debug.c | 5
b/kernel/dma/debug.h | 44 --
b/kernel/dma/direct.c | 4
b/kernel/dma/direct.h | 119 +++++++
b/kernel/dma/dummy.c | 2
b/kernel/dma/mapping.c | 5
b/kernel/dma/ops_helpers.c | 3
b/kernel/dma/pool.c | 3
b/kernel/dma/swiotlb.c | 2
b/kernel/dma/virt.c | 2
b/mm/memory.c | 1
include/asm-generic/dma-contiguous.h | 10
include/linux/dma-contiguous.h | 182 -----------
include/linux/dma-noncoherent.h | 109 -------
111 files changed, 620 insertions(+), 737 deletions(-)
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h>
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-10-11 14:33 ` Guenter Roeck
2020-09-30 8:55 ` [PATCH 2/9] dma-contiguous: remove dma_declare_contiguous Christoph Hellwig
` (8 subsequent siblings)
9 siblings, 1 reply; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Split out all the bits that are purely for dma_map_ops implementations
and related code into a new <linux/dma-map-ops.h> header so that they
don't get pulled into all the drivers. That also means the architecture
specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h>
any more, which leads to a missing includes that were pulled in by the
x86 or arm versions in a few not overly portable drivers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
MAINTAINERS | 1 +
arch/alpha/kernel/pci_iommu.c | 2 +-
arch/arc/mm/dma.c | 1 +
arch/arm/common/dmabounce.c | 1 +
arch/arm/mach-highbank/highbank.c | 2 +-
arch/arm/mach-imx/mach-imx27_visstrim_m10.c | 2 +-
arch/arm/mach-imx/mach-mx31moboard.c | 2 +-
arch/arm/mach-mvebu/coherency.c | 2 +-
arch/arm/mm/dma-mapping-nommu.c | 1 +
arch/arm/mm/dma-mapping.c | 2 +-
arch/arm64/mm/dma-mapping.c | 1 +
arch/ia64/hp/common/sba_iommu.c | 2 +-
arch/ia64/kernel/dma-mapping.c | 2 +-
arch/mips/jazz/jazzdma.c | 1 +
arch/mips/mm/dma-noncoherent.c | 1 +
arch/parisc/kernel/drivers.c | 1 +
arch/powerpc/include/asm/iommu.h | 2 +-
arch/powerpc/include/asm/pci.h | 2 +-
arch/powerpc/platforms/ps3/system-bus.c | 2 +-
arch/powerpc/platforms/pseries/ibmebus.c | 2 +-
arch/powerpc/platforms/pseries/vio.c | 2 +-
arch/s390/pci/pci_dma.c | 2 +-
arch/sh/boards/mach-ap325rxa/setup.c | 1 +
arch/sh/boards/mach-ecovec24/setup.c | 1 +
arch/sh/boards/mach-kfr2r09/setup.c | 2 +-
arch/sh/boards/mach-migor/setup.c | 2 +-
arch/sh/boards/mach-se/7724/setup.c | 1 +
arch/sh/drivers/pci/fixups-dreamcast.c | 2 +-
arch/sparc/kernel/iommu.c | 2 +-
arch/sparc/kernel/pci_sun4v.c | 1 +
arch/sparc/mm/io-unit.c | 2 +-
arch/sparc/mm/iommu.c | 2 +-
arch/x86/kernel/amd_gart_64.c | 1 +
arch/x86/kernel/pci-dma.c | 1 +
arch/x86/kernel/setup.c | 2 +
arch/x86/xen/pci-swiotlb-xen.c | 2 +-
drivers/acpi/arm64/iort.c | 2 +-
drivers/acpi/scan.c | 2 +-
drivers/base/dd.c | 2 +-
drivers/gpu/drm/exynos/exynos_drm_dma.c | 2 +-
drivers/gpu/drm/msm/msm_gem.c | 1 +
drivers/iommu/dma-iommu.c | 1 +
drivers/iommu/intel/iommu.c | 2 +-
drivers/misc/mic/bus/mic_bus.c | 1 +
drivers/misc/mic/bus/scif_bus.c | 2 +-
drivers/misc/mic/bus/scif_bus.h | 2 +-
drivers/misc/mic/bus/vop_bus.c | 2 +-
drivers/misc/mic/host/mic_boot.c | 1 +
drivers/of/device.c | 1 +
drivers/parisc/ccio-dma.c | 1 +
drivers/parisc/sba_iommu.c | 1 +
drivers/pci/xen-pcifront.c | 1 +
drivers/remoteproc/remoteproc_core.c | 1 +
drivers/remoteproc/remoteproc_virtio.c | 2 +-
drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +-
drivers/xen/swiotlb-xen.c | 1 +
include/linux/dma-map-ops.h | 158 ++++++++++++++++++
include/linux/dma-mapping.h | 168 +-------------------
kernel/dma/coherent.c | 1 +
kernel/dma/direct.c | 1 +
kernel/dma/dummy.c | 2 +-
kernel/dma/mapping.c | 2 +-
kernel/dma/ops_helpers.c | 1 +
kernel/dma/virt.c | 2 +-
64 files changed, 223 insertions(+), 200 deletions(-)
create mode 100644 include/linux/dma-map-ops.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 190c7fa2ea010a..b13fc17943079a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5202,6 +5202,7 @@ T: git git://git.infradead.org/users/hch/dma-mapping.git
F: include/asm-generic/dma-mapping.h
F: include/linux/dma-direct.h
F: include/linux/dma-mapping.h
+F: include/linux/dma-map-ops.h
F: include/linux/dma-noncoherent.h
F: kernel/dma/
diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index 447e0fd0ed3895..d84b19aa8e9da7 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -11,7 +11,7 @@
#include <linux/export.h>
#include <linux/scatterlist.h>
#include <linux/log2.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/iommu-helper.h>
#include <asm/io.h>
diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c
index e947572a521ec0..a8c453e98d753b 100644
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -3,6 +3,7 @@
* Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
*/
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <asm/cache.h>
#include <asm/cacheflush.h>
diff --git a/arch/arm/common/dmabounce.c b/arch/arm/common/dmabounce.c
index d3e00ea9208834..7996c04393d501 100644
--- a/arch/arm/common/dmabounce.c
+++ b/arch/arm/common/dmabounce.c
@@ -25,6 +25,7 @@
#include <linux/page-flags.h>
#include <linux/device.h>
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
#include <linux/dmapool.h>
#include <linux/list.h>
#include <linux/scatterlist.h>
diff --git a/arch/arm/mach-highbank/highbank.c b/arch/arm/mach-highbank/highbank.c
index 56bf29523c657c..db607955a7e451 100644
--- a/arch/arm/mach-highbank/highbank.c
+++ b/arch/arm/mach-highbank/highbank.c
@@ -5,7 +5,7 @@
#include <linux/clk.h>
#include <linux/clkdev.h>
#include <linux/clocksource.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/input.h>
#include <linux/io.h>
#include <linux/irqchip.h>
diff --git a/arch/arm/mach-imx/mach-imx27_visstrim_m10.c b/arch/arm/mach-imx/mach-imx27_visstrim_m10.c
index 3da4c0920198b6..a329e50928b649 100644
--- a/arch/arm/mach-imx/mach-imx27_visstrim_m10.c
+++ b/arch/arm/mach-imx/mach-imx27_visstrim_m10.c
@@ -16,7 +16,7 @@
#include <linux/input.h>
#include <linux/gpio.h>
#include <linux/delay.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/leds.h>
#include <linux/platform_data/asoc-mx27vis.h>
#include <sound/tlv320aic32x4.h>
diff --git a/arch/arm/mach-imx/mach-mx31moboard.c b/arch/arm/mach-imx/mach-mx31moboard.c
index 96845a4eaf57ea..7f780ad2d45925 100644
--- a/arch/arm/mach-imx/mach-mx31moboard.c
+++ b/arch/arm/mach-imx/mach-mx31moboard.c
@@ -4,7 +4,7 @@
*/
#include <linux/delay.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/gfp.h>
#include <linux/gpio.h>
#include <linux/init.h>
diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index 8f8748a0c84f44..49e3c8d20c2fa1 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -25,7 +25,7 @@
#include <linux/of_address.h>
#include <linux/io.h>
#include <linux/smp.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/mbus.h>
diff --git a/arch/arm/mm/dma-mapping-nommu.c b/arch/arm/mm/dma-mapping-nommu.c
index 43c6d66b6e733a..6bfd2b8845050b 100644
--- a/arch/arm/mm/dma-mapping-nommu.c
+++ b/arch/arm/mm/dma-mapping-nommu.c
@@ -8,6 +8,7 @@
#include <linux/export.h>
#include <linux/mm.h>
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <asm/cachetype.h>
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 7738b4d23f692f..8bf0bc6bc31172 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -15,7 +15,7 @@
#include <linux/init.h>
#include <linux/device.h>
#include <linux/dma-direct.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <linux/dma-contiguous.h>
#include <linux/highmem.h>
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 6c45350e33aa5a..3afd3bd659d8de 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -6,6 +6,7 @@
#include <linux/gfp.h>
#include <linux/cache.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <linux/dma-iommu.h>
#include <xen/xen.h>
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index cafbb848a34e4d..9148ddbf02e501 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -33,7 +33,7 @@
#include <linux/bitops.h> /* hweight64() */
#include <linux/crash_dump.h>
#include <linux/iommu-helper.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/prefetch.h>
#include <linux/swiotlb.h>
diff --git a/arch/ia64/kernel/dma-mapping.c b/arch/ia64/kernel/dma-mapping.c
index f640ed6fe1d576..cd0c166bfbc23e 100644
--- a/arch/ia64/kernel/dma-mapping.c
+++ b/arch/ia64/kernel/dma-mapping.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/export.h>
/* Set this to 1 if there is a HW IOMMU in the system */
diff --git a/arch/mips/jazz/jazzdma.c b/arch/mips/jazz/jazzdma.c
index f53bc043334c01..b8fb42e56da0a8 100644
--- a/arch/mips/jazz/jazzdma.c
+++ b/arch/mips/jazz/jazzdma.c
@@ -16,6 +16,7 @@
#include <linux/memblock.h>
#include <linux/spinlock.h>
#include <linux/gfp.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <asm/mipsregs.h>
#include <asm/jazz.h>
diff --git a/arch/mips/mm/dma-noncoherent.c b/arch/mips/mm/dma-noncoherent.c
index f34ad1f09799f1..f4e8404ee0490d 100644
--- a/arch/mips/mm/dma-noncoherent.c
+++ b/arch/mips/mm/dma-noncoherent.c
@@ -5,6 +5,7 @@
* swiped from i386, and cloned for MIPS by Geert, polished by Ralf.
*/
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <linux/dma-contiguous.h>
#include <linux/highmem.h>
diff --git a/arch/parisc/kernel/drivers.c b/arch/parisc/kernel/drivers.c
index a5f3e50fe97619..80fa0650736ba7 100644
--- a/arch/parisc/kernel/drivers.c
+++ b/arch/parisc/kernel/drivers.c
@@ -30,6 +30,7 @@
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/export.h>
+#include <linux/dma-map-ops.h>
#include <asm/hardware.h>
#include <asm/io.h>
#include <asm/pdc.h>
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 5032f1593299ef..deef7c94d7b61d 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -12,7 +12,7 @@
#include <linux/compiler.h>
#include <linux/spinlock.h>
#include <linux/device.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/bitops.h>
#include <asm/machdep.h>
#include <asm/types.h>
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 63ed7e3b0ba3e4..6436f0b41539e3 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -9,7 +9,7 @@
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/string.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <asm/machdep.h>
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 7bc5f9be3e12d8..c62aaa29a9d5ee 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -9,7 +9,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/export.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/err.h>
#include <linux/slab.h>
diff --git a/arch/powerpc/platforms/pseries/ibmebus.c b/arch/powerpc/platforms/pseries/ibmebus.c
index a6f101c958e84b..8c6e509f696756 100644
--- a/arch/powerpc/platforms/pseries/ibmebus.c
+++ b/arch/powerpc/platforms/pseries/ibmebus.c
@@ -40,7 +40,7 @@
#include <linux/export.h>
#include <linux/console.h>
#include <linux/kobject.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/interrupt.h>
#include <linux/of.h>
#include <linux/slab.h>
diff --git a/arch/powerpc/platforms/pseries/vio.c b/arch/powerpc/platforms/pseries/vio.c
index 98ed7b09b3fe50..b2797cfe4e2b08 100644
--- a/arch/powerpc/platforms/pseries/vio.c
+++ b/arch/powerpc/platforms/pseries/vio.c
@@ -20,7 +20,7 @@
#include <linux/console.h>
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/kobject.h>
#include <asm/iommu.h>
diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c
index 9291023e9469c2..ebc9a49523aa31 100644
--- a/arch/s390/pci/pci_dma.c
+++ b/arch/s390/pci/pci_dma.c
@@ -10,7 +10,7 @@
#include <linux/slab.h>
#include <linux/export.h>
#include <linux/iommu-helper.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/vmalloc.h>
#include <linux/pci.h>
#include <asm/pci_dma.h>
diff --git a/arch/sh/boards/mach-ap325rxa/setup.c b/arch/sh/boards/mach-ap325rxa/setup.c
index 665cad452798bb..bac8a058ebd7cd 100644
--- a/arch/sh/boards/mach-ap325rxa/setup.c
+++ b/arch/sh/boards/mach-ap325rxa/setup.c
@@ -13,6 +13,7 @@
#include <cpu/sh7723.h>
+#include <linux/dma-map-ops.h>
#include <linux/clkdev.h>
#include <linux/delay.h>
#include <linux/device.h>
diff --git a/arch/sh/boards/mach-ecovec24/setup.c b/arch/sh/boards/mach-ecovec24/setup.c
index dd427bac5cdeb7..bab91a99124e1f 100644
--- a/arch/sh/boards/mach-ecovec24/setup.c
+++ b/arch/sh/boards/mach-ecovec24/setup.c
@@ -36,6 +36,7 @@
#include <linux/usb/r8a66597.h>
#include <linux/usb/renesas_usbhs.h>
#include <linux/videodev2.h>
+#include <linux/dma-map-ops.h>
#include <media/drv-intf/renesas-ceu.h>
#include <media/i2c/mt9t112.h>
diff --git a/arch/sh/boards/mach-kfr2r09/setup.c b/arch/sh/boards/mach-kfr2r09/setup.c
index 96538ba3aa323f..eeb5ce341efdd7 100644
--- a/arch/sh/boards/mach-kfr2r09/setup.c
+++ b/arch/sh/boards/mach-kfr2r09/setup.c
@@ -14,7 +14,6 @@
#include <linux/clkdev.h>
#include <linux/delay.h>
-#include <linux/dma-mapping.h>
#include <linux/gpio.h>
#include <linux/gpio/machine.h>
#include <linux/i2c.h>
@@ -33,6 +32,7 @@
#include <linux/sh_intc.h>
#include <linux/usb/r8a66597.h>
#include <linux/videodev2.h>
+#include <linux/dma-map-ops.h>
#include <mach/kfr2r09.h>
diff --git a/arch/sh/boards/mach-migor/setup.c b/arch/sh/boards/mach-migor/setup.c
index 9ed369dad62df7..6703a2122c0d6b 100644
--- a/arch/sh/boards/mach-migor/setup.c
+++ b/arch/sh/boards/mach-migor/setup.c
@@ -5,7 +5,7 @@
* Copyright (C) 2008 Magnus Damm
*/
#include <linux/clkdev.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
diff --git a/arch/sh/boards/mach-se/7724/setup.c b/arch/sh/boards/mach-se/7724/setup.c
index 32f5dd9448894a..8d6541ba01865b 100644
--- a/arch/sh/boards/mach-se/7724/setup.c
+++ b/arch/sh/boards/mach-se/7724/setup.c
@@ -32,6 +32,7 @@
#include <linux/smc91x.h>
#include <linux/usb/r8a66597.h>
#include <linux/videodev2.h>
+#include <linux/dma-map-ops.h>
#include <mach-se/mach/se7724.h>
#include <media/drv-intf/renesas-ceu.h>
diff --git a/arch/sh/drivers/pci/fixups-dreamcast.c b/arch/sh/drivers/pci/fixups-dreamcast.c
index 7be8694c0d1310..41e4daee8f043f 100644
--- a/arch/sh/drivers/pci/fixups-dreamcast.c
+++ b/arch/sh/drivers/pci/fixups-dreamcast.c
@@ -19,7 +19,7 @@
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/pci.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <asm/io.h>
#include <asm/irq.h>
diff --git a/arch/sparc/kernel/iommu.c b/arch/sparc/kernel/iommu.c
index c3e4e2df26a8b8..a034f571d8695e 100644
--- a/arch/sparc/kernel/iommu.c
+++ b/arch/sparc/kernel/iommu.c
@@ -10,7 +10,7 @@
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/device.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/errno.h>
#include <linux/iommu-helper.h>
#include <linux/bitmap.h>
diff --git a/arch/sparc/kernel/pci_sun4v.c b/arch/sparc/kernel/pci_sun4v.c
index 6b92dd51c0026f..9de57e88f7a177 100644
--- a/arch/sparc/kernel/pci_sun4v.c
+++ b/arch/sparc/kernel/pci_sun4v.c
@@ -16,6 +16,7 @@
#include <linux/export.h>
#include <linux/log2.h>
#include <linux/of_device.h>
+#include <linux/dma-map-ops.h>
#include <asm/iommu-common.h>
#include <asm/iommu.h>
diff --git a/arch/sparc/mm/io-unit.c b/arch/sparc/mm/io-unit.c
index 430a47a1b6aeff..bf3e6d2fe5d949 100644
--- a/arch/sparc/mm/io-unit.c
+++ b/arch/sparc/mm/io-unit.c
@@ -11,7 +11,7 @@
#include <linux/spinlock.h>
#include <linux/mm.h>
#include <linux/bitops.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/of.h>
#include <linux/of_device.h>
diff --git a/arch/sparc/mm/iommu.c b/arch/sparc/mm/iommu.c
index 3a388b1c5d4bc9..0c0342e5b10d52 100644
--- a/arch/sparc/mm/iommu.c
+++ b/arch/sparc/mm/iommu.c
@@ -12,7 +12,7 @@
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/slab.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/of.h>
#include <linux/of_device.h>
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index c96dcaa572ebd3..9ac696487b134b 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -32,6 +32,7 @@
#include <linux/gfp.h>
#include <linux/atomic.h>
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
#include <asm/mtrr.h>
#include <asm/proto.h>
#include <asm/iommu.h>
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 5dcedad21dffa2..4892dd043d414c 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -1,4 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
+#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/dma-debug.h>
#include <linux/iommu.h>
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 3511736fbc747e..9286fa9d575e6d 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -7,6 +7,7 @@
*/
#include <linux/console.h>
#include <linux/crash_dump.h>
+#include <linux/dma-contiguous.h>
#include <linux/dmi.h>
#include <linux/efi.h>
#include <linux/init_ohci1394_dma.h>
@@ -19,6 +20,7 @@
#include <linux/hugetlb.h>
#include <linux/tboot.h>
#include <linux/usb/xhci-dbgp.h>
+#include <linux/swiotlb.h>
#include <uapi/linux/mount.h>
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
index 33293ce01d8db5..19ae3e4fe4e98e 100644
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -2,7 +2,7 @@
/* Glue code to lib/swiotlb-xen.c */
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/pci.h>
#include <xen/swiotlb-xen.h>
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index de18c07ca02cc4..6446b2572f075f 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -18,7 +18,7 @@
#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#define IORT_TYPE_MASK(type) (1 << (type))
#define IORT_MSI_TYPE (1 << ACPI_IORT_NODE_ITS_GROUP)
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 2142f155476176..e0b7d7a605b5a2 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -13,7 +13,7 @@
#include <linux/kthread.h>
#include <linux/dmi.h>
#include <linux/nls.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/platform_data/x86/apple.h>
#include <linux/pgtable.h>
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 857b0a928e8d0d..b3d43ace5c2b94 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -19,7 +19,7 @@
#include <linux/debugfs.h>
#include <linux/device.h>
#include <linux/delay.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kthread.h>
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dma.c b/drivers/gpu/drm/exynos/exynos_drm_dma.c
index 58b89ec11b0eb3..78b8f3403c3039 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dma.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dma.c
@@ -5,7 +5,7 @@
// Author: Andrzej Hajda <a.hajda@samsung.com>
#include <linux/dma-iommu.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/iommu.h>
#include <linux/platform_device.h>
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index b2f49152b4d4c2..6787ab0d0e431d 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -4,6 +4,7 @@
* Author: Rob Clark <robdclark@gmail.com>
*/
+#include <linux/dma-map-ops.h>
#include <linux/spinlock.h>
#include <linux/shmem_fs.h>
#include <linux/dma-buf.h>
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index c12c1dc43d312e..d2e3f26228151f 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -10,6 +10,7 @@
#include <linux/acpi_iort.h>
#include <linux/device.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-contiguous.h>
#include <linux/dma-iommu.h>
#include <linux/dma-noncoherent.h>
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 2c426dbdf17f3b..bd3470142b0678 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -23,7 +23,7 @@
#include <linux/spinlock.h>
#include <linux/pci.h>
#include <linux/dmar.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/mempool.h>
#include <linux/memory.h>
#include <linux/cpu.h>
diff --git a/drivers/misc/mic/bus/mic_bus.c b/drivers/misc/mic/bus/mic_bus.c
index ed9a8351c3bfd1..a08cb29692a8fe 100644
--- a/drivers/misc/mic/bus/mic_bus.c
+++ b/drivers/misc/mic/bus/mic_bus.c
@@ -9,6 +9,7 @@
* This implementation is very similar to the the virtio bus driver
* implementation @ drivers/virtio/virtio.c
*/
+#include <linux/dma-map-ops.h>
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/idr.h>
diff --git a/drivers/misc/mic/bus/scif_bus.c b/drivers/misc/mic/bus/scif_bus.c
index ae84109649d0f3..ad7c3604f1510f 100644
--- a/drivers/misc/mic/bus/scif_bus.c
+++ b/drivers/misc/mic/bus/scif_bus.c
@@ -9,7 +9,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/idr.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include "scif_bus.h"
diff --git a/drivers/misc/mic/bus/scif_bus.h b/drivers/misc/mic/bus/scif_bus.h
index 642cd43bcabc6d..4981eb56f87979 100644
--- a/drivers/misc/mic/bus/scif_bus.h
+++ b/drivers/misc/mic/bus/scif_bus.h
@@ -12,7 +12,7 @@
* Everything a scif driver needs to work with any particular scif
* hardware abstraction layer.
*/
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/mic_common.h>
#include "../common/mic_dev.h"
diff --git a/drivers/misc/mic/bus/vop_bus.c b/drivers/misc/mic/bus/vop_bus.c
index 3c865534868a0e..6935ddca1bd5b1 100644
--- a/drivers/misc/mic/bus/vop_bus.c
+++ b/drivers/misc/mic/bus/vop_bus.c
@@ -9,7 +9,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/idr.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include "vop_bus.h"
diff --git a/drivers/misc/mic/host/mic_boot.c b/drivers/misc/mic/host/mic_boot.c
index fb5b3989753daf..8cb85b8b3e199b 100644
--- a/drivers/misc/mic/host/mic_boot.c
+++ b/drivers/misc/mic/host/mic_boot.c
@@ -10,6 +10,7 @@
#include <linux/firmware.h>
#include <linux/pci.h>
#include <linux/kmod.h>
+#include <linux/dma-map-ops.h>
#include <linux/mic_common.h>
#include <linux/mic_bus.h>
#include "../bus/scif_bus.h"
diff --git a/drivers/of/device.c b/drivers/of/device.c
index 6e3ae7ebc33eeb..655dee422563f0 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -6,6 +6,7 @@
#include <linux/of_address.h>
#include <linux/of_iommu.h>
#include <linux/dma-direct.h> /* for bus_dma_region */
+#include <linux/dma-map-ops.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
diff --git a/drivers/parisc/ccio-dma.c b/drivers/parisc/ccio-dma.c
index 8cf0b9c8bdf795..b5f9ee81a46c1e 100644
--- a/drivers/parisc/ccio-dma.c
+++ b/drivers/parisc/ccio-dma.c
@@ -39,6 +39,7 @@
#include <linux/reboot.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <linux/iommu-helper.h>
#include <linux/export.h>
diff --git a/drivers/parisc/sba_iommu.c b/drivers/parisc/sba_iommu.c
index 6fcde7980358ae..dce4cdf786cdb1 100644
--- a/drivers/parisc/sba_iommu.c
+++ b/drivers/parisc/sba_iommu.c
@@ -25,6 +25,7 @@
#include <linux/mm.h>
#include <linux/string.h>
#include <linux/pci.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <linux/iommu-helper.h>
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index c0e85be598c112..c6fe0cfec0f681 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -22,6 +22,7 @@
#include <linux/bitops.h>
#include <linux/time.h>
#include <linux/ktime.h>
+#include <linux/swiotlb.h>
#include <xen/platform_pci.h>
#include <asm/xen/swiotlb-xen.h>
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 8157dd491d2819..dab2c0f5caf0ec 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -22,6 +22,7 @@
#include <linux/device.h>
#include <linux/slab.h>
#include <linux/mutex.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-mapping.h>
#include <linux/dma-direct.h> /* XXX: pokes into bus_dma_range */
#include <linux/firmware.h>
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index dfd3808c34fdbd..0cc617f760682c 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -9,7 +9,7 @@
* Brian Swetland <swetland@google.com>
*/
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/export.h>
#include <linux/of_reserved_mem.h>
#include <linux/remoteproc.h>
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 62d6403271450e..2629911c29bbf4 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -18,7 +18,7 @@
#include <linux/wait.h>
#include <linux/uuid.h>
#include <linux/iommu.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/sysfs.h>
#include <linux/file.h>
#include <linux/etherdevice.h>
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 030a225624b060..71ff4bf380738c 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -27,6 +27,7 @@
#define pr_fmt(fmt) "xen:" KBUILD_MODNAME ": " fmt
#include <linux/memblock.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/dma-noncoherent.h>
#include <linux/export.h>
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
new file mode 100644
index 00000000000000..4b4ba5bdcf6a8d
--- /dev/null
+++ b/include/linux/dma-map-ops.h
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This header is for implementations of dma_map_ops and related code.
+ * It should not be included in drivers just using the DMA API.
+ */
+#ifndef _LINUX_DMA_MAP_OPS_H
+#define _LINUX_DMA_MAP_OPS_H
+
+#include <linux/dma-mapping.h>
+
+struct dma_map_ops {
+ void *(*alloc)(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp,
+ unsigned long attrs);
+ void (*free)(struct device *dev, size_t size, void *vaddr,
+ dma_addr_t dma_handle, unsigned long attrs);
+ struct page *(*alloc_pages)(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, enum dma_data_direction dir,
+ gfp_t gfp);
+ void (*free_pages)(struct device *dev, size_t size, struct page *vaddr,
+ dma_addr_t dma_handle, enum dma_data_direction dir);
+ void *(*alloc_noncoherent)(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, enum dma_data_direction dir,
+ gfp_t gfp);
+ void (*free_noncoherent)(struct device *dev, size_t size, void *vaddr,
+ dma_addr_t dma_handle, enum dma_data_direction dir);
+ int (*mmap)(struct device *, struct vm_area_struct *,
+ void *, dma_addr_t, size_t, unsigned long attrs);
+
+ int (*get_sgtable)(struct device *dev, struct sg_table *sgt,
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs);
+
+ dma_addr_t (*map_page)(struct device *dev, struct page *page,
+ unsigned long offset, size_t size,
+ enum dma_data_direction dir, unsigned long attrs);
+ void (*unmap_page)(struct device *dev, dma_addr_t dma_handle,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
+ /*
+ * map_sg returns 0 on error and a value > 0 on success.
+ * It should never return a value < 0.
+ */
+ int (*map_sg)(struct device *dev, struct scatterlist *sg, int nents,
+ enum dma_data_direction dir, unsigned long attrs);
+ void (*unmap_sg)(struct device *dev, struct scatterlist *sg, int nents,
+ enum dma_data_direction dir, unsigned long attrs);
+ dma_addr_t (*map_resource)(struct device *dev, phys_addr_t phys_addr,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
+ void (*unmap_resource)(struct device *dev, dma_addr_t dma_handle,
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
+ void (*sync_single_for_cpu)(struct device *dev, dma_addr_t dma_handle,
+ size_t size, enum dma_data_direction dir);
+ void (*sync_single_for_device)(struct device *dev,
+ dma_addr_t dma_handle, size_t size,
+ enum dma_data_direction dir);
+ void (*sync_sg_for_cpu)(struct device *dev, struct scatterlist *sg,
+ int nents, enum dma_data_direction dir);
+ void (*sync_sg_for_device)(struct device *dev, struct scatterlist *sg,
+ int nents, enum dma_data_direction dir);
+ void (*cache_sync)(struct device *dev, void *vaddr, size_t size,
+ enum dma_data_direction direction);
+ int (*dma_supported)(struct device *dev, u64 mask);
+ u64 (*get_required_mask)(struct device *dev);
+ size_t (*max_mapping_size)(struct device *dev);
+ unsigned long (*get_merge_boundary)(struct device *dev);
+};
+
+#ifdef CONFIG_DMA_OPS
+#include <asm/dma-mapping.h>
+
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+{
+ if (dev->dma_ops)
+ return dev->dma_ops;
+ return get_arch_dma_ops(dev->bus);
+}
+
+static inline void set_dma_ops(struct device *dev,
+ const struct dma_map_ops *dma_ops)
+{
+ dev->dma_ops = dma_ops;
+}
+#else /* CONFIG_DMA_OPS */
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+{
+ return NULL;
+}
+static inline void set_dma_ops(struct device *dev,
+ const struct dma_map_ops *dma_ops)
+{
+}
+#endif /* CONFIG_DMA_OPS */
+
+#ifdef CONFIG_DMA_DECLARE_COHERENT
+int dma_declare_coherent_memory(struct device *dev, phys_addr_t phys_addr,
+ dma_addr_t device_addr, size_t size);
+int dma_alloc_from_dev_coherent(struct device *dev, ssize_t size,
+ dma_addr_t *dma_handle, void **ret);
+int dma_release_from_dev_coherent(struct device *dev, int order, void *vaddr);
+int dma_mmap_from_dev_coherent(struct device *dev, struct vm_area_struct *vma,
+ void *cpu_addr, size_t size, int *ret);
+
+void *dma_alloc_from_global_coherent(struct device *dev, ssize_t size,
+ dma_addr_t *dma_handle);
+int dma_release_from_global_coherent(int order, void *vaddr);
+int dma_mmap_from_global_coherent(struct vm_area_struct *vma, void *cpu_addr,
+ size_t size, int *ret);
+
+#else
+static inline int dma_declare_coherent_memory(struct device *dev,
+ phys_addr_t phys_addr, dma_addr_t device_addr, size_t size)
+{
+ return -ENOSYS;
+}
+#define dma_alloc_from_dev_coherent(dev, size, handle, ret) (0)
+#define dma_release_from_dev_coherent(dev, order, vaddr) (0)
+#define dma_mmap_from_dev_coherent(dev, vma, vaddr, order, ret) (0)
+
+static inline void *dma_alloc_from_global_coherent(struct device *dev,
+ ssize_t size, dma_addr_t *dma_handle)
+{
+ return NULL;
+}
+static inline int dma_release_from_global_coherent(int order, void *vaddr)
+{
+ return 0;
+}
+static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
+ void *cpu_addr, size_t size, int *ret)
+{
+ return 0;
+}
+#endif /* CONFIG_DMA_DECLARE_COHERENT */
+
+#ifdef CONFIG_ARCH_HAS_SETUP_DMA_OPS
+void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
+ const struct iommu_ops *iommu, bool coherent);
+#else
+static inline void arch_setup_dma_ops(struct device *dev, u64 dma_base,
+ u64 size, const struct iommu_ops *iommu, bool coherent)
+{
+}
+#endif /* CONFIG_ARCH_HAS_SETUP_DMA_OPS */
+
+#ifdef CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS
+void arch_teardown_dma_ops(struct device *dev);
+#else
+static inline void arch_teardown_dma_ops(struct device *dev)
+{
+}
+#endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
+
+extern const struct dma_map_ops dma_dummy_ops;
+
+#endif /* _LINUX_DMA_MAP_OPS_H */
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 7c77cd6f3604a7..9591cd482d7c2d 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -62,72 +62,6 @@
*/
#define DMA_ATTR_PRIVILEGED (1UL << 9)
-struct dma_map_ops {
- void* (*alloc)(struct device *dev, size_t size,
- dma_addr_t *dma_handle, gfp_t gfp,
- unsigned long attrs);
- void (*free)(struct device *dev, size_t size,
- void *vaddr, dma_addr_t dma_handle,
- unsigned long attrs);
- struct page *(*alloc_pages)(struct device *dev, size_t size,
- dma_addr_t *dma_handle, enum dma_data_direction dir,
- gfp_t gfp);
- void (*free_pages)(struct device *dev, size_t size, struct page *vaddr,
- dma_addr_t dma_handle, enum dma_data_direction dir);
- void* (*alloc_noncoherent)(struct device *dev, size_t size,
- dma_addr_t *dma_handle, enum dma_data_direction dir,
- gfp_t gfp);
- void (*free_noncoherent)(struct device *dev, size_t size, void *vaddr,
- dma_addr_t dma_handle, enum dma_data_direction dir);
- int (*mmap)(struct device *, struct vm_area_struct *,
- void *, dma_addr_t, size_t,
- unsigned long attrs);
-
- int (*get_sgtable)(struct device *dev, struct sg_table *sgt, void *,
- dma_addr_t, size_t, unsigned long attrs);
-
- dma_addr_t (*map_page)(struct device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction dir,
- unsigned long attrs);
- void (*unmap_page)(struct device *dev, dma_addr_t dma_handle,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs);
- /*
- * map_sg returns 0 on error and a value > 0 on success.
- * It should never return a value < 0.
- */
- int (*map_sg)(struct device *dev, struct scatterlist *sg,
- int nents, enum dma_data_direction dir,
- unsigned long attrs);
- void (*unmap_sg)(struct device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction dir,
- unsigned long attrs);
- dma_addr_t (*map_resource)(struct device *dev, phys_addr_t phys_addr,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs);
- void (*unmap_resource)(struct device *dev, dma_addr_t dma_handle,
- size_t size, enum dma_data_direction dir,
- unsigned long attrs);
- void (*sync_single_for_cpu)(struct device *dev,
- dma_addr_t dma_handle, size_t size,
- enum dma_data_direction dir);
- void (*sync_single_for_device)(struct device *dev,
- dma_addr_t dma_handle, size_t size,
- enum dma_data_direction dir);
- void (*sync_sg_for_cpu)(struct device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction dir);
- void (*sync_sg_for_device)(struct device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction dir);
- int (*dma_supported)(struct device *dev, u64 mask);
- u64 (*get_required_mask)(struct device *dev);
- size_t (*max_mapping_size)(struct device *dev);
- unsigned long (*get_merge_boundary)(struct device *dev);
-};
-
/*
* A dma_addr_t can hold any valid DMA or bus address for the platform. It can
* be given to a device to use as a DMA source or target. It is specific to a
@@ -140,79 +74,9 @@ struct dma_map_ops {
*/
#define DMA_MAPPING_ERROR (~(dma_addr_t)0)
-extern const struct dma_map_ops dma_virt_ops;
-extern const struct dma_map_ops dma_dummy_ops;
-
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
-#ifdef CONFIG_DMA_DECLARE_COHERENT
-/*
- * These three functions are only for dma allocator.
- * Don't use them in device drivers.
- */
-int dma_alloc_from_dev_coherent(struct device *dev, ssize_t size,
- dma_addr_t *dma_handle, void **ret);
-int dma_release_from_dev_coherent(struct device *dev, int order, void *vaddr);
-
-int dma_mmap_from_dev_coherent(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, size_t size, int *ret);
-
-void *dma_alloc_from_global_coherent(struct device *dev, ssize_t size, dma_addr_t *dma_handle);
-int dma_release_from_global_coherent(int order, void *vaddr);
-int dma_mmap_from_global_coherent(struct vm_area_struct *vma, void *cpu_addr,
- size_t size, int *ret);
-
-#else
-#define dma_alloc_from_dev_coherent(dev, size, handle, ret) (0)
-#define dma_release_from_dev_coherent(dev, order, vaddr) (0)
-#define dma_mmap_from_dev_coherent(dev, vma, vaddr, order, ret) (0)
-
-static inline void *dma_alloc_from_global_coherent(struct device *dev, ssize_t size,
- dma_addr_t *dma_handle)
-{
- return NULL;
-}
-
-static inline int dma_release_from_global_coherent(int order, void *vaddr)
-{
- return 0;
-}
-
-static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
- void *cpu_addr, size_t size,
- int *ret)
-{
- return 0;
-}
-#endif /* CONFIG_DMA_DECLARE_COHERENT */
-
#ifdef CONFIG_HAS_DMA
-#include <asm/dma-mapping.h>
-
-#ifdef CONFIG_DMA_OPS
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
-{
- if (dev->dma_ops)
- return dev->dma_ops;
- return get_arch_dma_ops(dev->bus);
-}
-
-static inline void set_dma_ops(struct device *dev,
- const struct dma_map_ops *dma_ops)
-{
- dev->dma_ops = dma_ops;
-}
-#else /* CONFIG_DMA_OPS */
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
-{
- return NULL;
-}
-static inline void set_dma_ops(struct device *dev,
- const struct dma_map_ops *dma_ops)
-{
-}
-#endif /* CONFIG_DMA_OPS */
-
static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
{
debug_dma_mapping_error(dev, dma_addr);
@@ -595,24 +459,6 @@ static inline bool dma_addressing_limited(struct device *dev)
dma_get_required_mask(dev);
}
-#ifdef CONFIG_ARCH_HAS_SETUP_DMA_OPS
-void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- const struct iommu_ops *iommu, bool coherent);
-#else
-static inline void arch_setup_dma_ops(struct device *dev, u64 dma_base,
- u64 size, const struct iommu_ops *iommu, bool coherent)
-{
-}
-#endif /* CONFIG_ARCH_HAS_SETUP_DMA_OPS */
-
-#ifdef CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS
-void arch_teardown_dma_ops(struct device *dev);
-#else
-static inline void arch_teardown_dma_ops(struct device *dev)
-{
-}
-#endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
-
static inline unsigned int dma_get_max_seg_size(struct device *dev)
{
if (dev->dma_parms && dev->dma_parms->max_segment_size)
@@ -672,18 +518,6 @@ static inline int dma_get_cache_alignment(void)
return 1;
}
-#ifdef CONFIG_DMA_DECLARE_COHERENT
-int dma_declare_coherent_memory(struct device *dev, phys_addr_t phys_addr,
- dma_addr_t device_addr, size_t size);
-#else
-static inline int
-dma_declare_coherent_memory(struct device *dev, phys_addr_t phys_addr,
- dma_addr_t device_addr, size_t size)
-{
- return -ENOSYS;
-}
-#endif /* CONFIG_DMA_DECLARE_COHERENT */
-
static inline void *dmam_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t gfp)
{
@@ -741,4 +575,6 @@ static inline int dma_mmap_wc(struct device *dev,
int dma_direct_set_offset(struct device *dev, phys_addr_t cpu_start,
dma_addr_t dma_start, u64 size);
+extern const struct dma_map_ops dma_virt_ops;
+
#endif /* _LINUX_DMA_MAPPING_H */
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index c0685196fb6d42..5b5b6c7ec7f28e 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -8,6 +8,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
struct dma_coherent_mem {
void *virt_base;
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 121a9c1969dd3a..8cf5689a8c4044 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -8,6 +8,7 @@
#include <linux/export.h>
#include <linux/mm.h>
#include <linux/dma-direct.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <linux/dma-contiguous.h>
#include <linux/pfn.h>
diff --git a/kernel/dma/dummy.c b/kernel/dma/dummy.c
index 6974b1bd7d0b88..eacd4c5b10bf6d 100644
--- a/kernel/dma/dummy.c
+++ b/kernel/dma/dummy.c
@@ -2,7 +2,7 @@
/*
* Dummy DMA ops that always fail.
*/
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
static int dma_dummy_mmap(struct device *dev, struct vm_area_struct *vma,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 9669550656a0b4..2e13e6d3903fa0 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -8,7 +8,7 @@
#include <linux/memblock.h> /* for max_pfn */
#include <linux/acpi.h>
#include <linux/dma-direct.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/export.h>
#include <linux/gfp.h>
#include <linux/of_device.h>
diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c
index 5828e5e01b7913..60d7b6cdfd8d15 100644
--- a/kernel/dma/ops_helpers.c
+++ b/kernel/dma/ops_helpers.c
@@ -4,6 +4,7 @@
* the allocated memory contains normal pages in the direct kernel mapping.
*/
#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
/*
diff --git a/kernel/dma/virt.c b/kernel/dma/virt.c
index 6986bf1fd6689c..59d32317dd574a 100644
--- a/kernel/dma/virt.c
+++ b/kernel/dma/virt.c
@@ -4,7 +4,7 @@
*/
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
static void *dma_virt_alloc(struct device *dev, size_t size,
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/9] dma-contiguous: remove dma_declare_contiguous
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
2020-09-30 8:55 ` [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h> Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 3/9] dma-contiguous: remove dev_set_cma_area Christoph Hellwig
` (7 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
dma_declare_contiguous is a trivial wrapper around
dma_contiguous_reserve_area and just has a single caller.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mach-davinci/devices-da8xx.c | 16 +++++++++-----
include/linux/dma-contiguous.h | 32 ---------------------------
2 files changed, 10 insertions(+), 38 deletions(-)
diff --git a/arch/arm/mach-davinci/devices-da8xx.c b/arch/arm/mach-davinci/devices-da8xx.c
index feb206bdf6e172..2e2853582b459e 100644
--- a/arch/arm/mach-davinci/devices-da8xx.c
+++ b/arch/arm/mach-davinci/devices-da8xx.c
@@ -884,6 +884,7 @@ early_param("rproc_mem", early_rproc_mem);
void __init da8xx_rproc_reserve_cma(void)
{
+ struct cma *cma;
int ret;
if (!rproc_base || !rproc_size) {
@@ -897,13 +898,16 @@ void __init da8xx_rproc_reserve_cma(void)
pr_info("%s: reserving 0x%lx @ 0x%lx...\n",
__func__, rproc_size, (unsigned long)rproc_base);
- ret = dma_declare_contiguous(&da8xx_dsp.dev, rproc_size, rproc_base, 0);
- if (ret)
- pr_err("%s: dma_declare_contiguous failed %d\n", __func__, ret);
- else
- rproc_mem_inited = true;
+ ret = dma_contiguous_reserve_area(rproc_size, rproc_base, 0, &cma,
+ true);
+ if (ret) {
+ pr_err("%s: dma_contiguous_reserve_area failed %d\n",
+ __func__, ret);
+ return;
+ }
+ dev_set_cma_area(&da8xx_dsp.dev, cma);
+ rproc_mem_inited = true;
}
-
#else
void __init da8xx_rproc_reserve_cma(void)
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index fe55e004f1f433..62fd55d0723546 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -83,31 +83,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
phys_addr_t limit, struct cma **res_cma,
bool fixed);
-/**
- * dma_declare_contiguous() - reserve area for contiguous memory handling
- * for particular device
- * @dev: Pointer to device structure.
- * @size: Size of the reserved memory.
- * @base: Start address of the reserved memory (optional, 0 for any).
- * @limit: End address of the reserved memory (optional, 0 for any).
- *
- * This function reserves memory for specified device. It should be
- * called by board specific code when early allocator (memblock or bootmem)
- * is still activate.
- */
-
-static inline int dma_declare_contiguous(struct device *dev, phys_addr_t size,
- phys_addr_t base, phys_addr_t limit)
-{
- struct cma *cma;
- int ret;
- ret = dma_contiguous_reserve_area(size, base, limit, &cma, true);
- if (ret == 0)
- dev_set_cma_area(dev, cma);
-
- return ret;
-}
-
struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
unsigned int order, bool no_warn);
bool dma_release_from_contiguous(struct device *dev, struct page *pages,
@@ -135,13 +110,6 @@ static inline int dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base
return -ENOSYS;
}
-static inline
-int dma_declare_contiguous(struct device *dev, phys_addr_t size,
- phys_addr_t base, phys_addr_t limit)
-{
- return -ENOSYS;
-}
-
static inline
struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
unsigned int order, bool no_warn)
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 3/9] dma-contiguous: remove dev_set_cma_area
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
2020-09-30 8:55 ` [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h> Christoph Hellwig
2020-09-30 8:55 ` [PATCH 2/9] dma-contiguous: remove dma_declare_contiguous Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 4/9] dma-contiguous: remove dma_contiguous_set_default Christoph Hellwig
` (6 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
dev_set_cma_area contains a trivial assignment. It has just three
callers that all have a non-NULL device and depend on CONFIG_DMA_CMA,
so remove the wrapper.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/mach-davinci/devices-da8xx.c | 2 +-
include/linux/dma-contiguous.h | 8 --------
kernel/dma/contiguous.c | 4 ++--
3 files changed, 3 insertions(+), 11 deletions(-)
diff --git a/arch/arm/mach-davinci/devices-da8xx.c b/arch/arm/mach-davinci/devices-da8xx.c
index 2e2853582b459e..1207eabe8d1cc4 100644
--- a/arch/arm/mach-davinci/devices-da8xx.c
+++ b/arch/arm/mach-davinci/devices-da8xx.c
@@ -905,7 +905,7 @@ void __init da8xx_rproc_reserve_cma(void)
__func__, ret);
return;
}
- dev_set_cma_area(&da8xx_dsp.dev, cma);
+ da8xx_dsp.dev.cma_area = cma;
rproc_mem_inited = true;
}
#else
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index 62fd55d0723546..41ec08d81bc317 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -66,12 +66,6 @@ static inline struct cma *dev_get_cma_area(struct device *dev)
return dma_contiguous_default_area;
}
-static inline void dev_set_cma_area(struct device *dev, struct cma *cma)
-{
- if (dev)
- dev->cma_area = cma;
-}
-
static inline void dma_contiguous_set_default(struct cma *cma)
{
dma_contiguous_default_area = cma;
@@ -97,8 +91,6 @@ static inline struct cma *dev_get_cma_area(struct device *dev)
return NULL;
}
-static inline void dev_set_cma_area(struct device *dev, struct cma *cma) { }
-
static inline void dma_contiguous_set_default(struct cma *cma) { }
static inline void dma_contiguous_reserve(phys_addr_t limit) { }
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index f4c150810fd25c..95adcee972e85c 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -359,14 +359,14 @@ void dma_free_contiguous(struct device *dev, struct page *page, size_t size)
static int rmem_cma_device_init(struct reserved_mem *rmem, struct device *dev)
{
- dev_set_cma_area(dev, rmem->priv);
+ dev->cma_area = rmem->priv;
return 0;
}
static void rmem_cma_device_release(struct reserved_mem *rmem,
struct device *dev)
{
- dev_set_cma_area(dev, NULL);
+ dev->cma_area = NULL;
}
static const struct reserved_mem_ops rmem_cma_ops = {
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 4/9] dma-contiguous: remove dma_contiguous_set_default
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (2 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 3/9] dma-contiguous: remove dev_set_cma_area Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 5/9] dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h> Christoph Hellwig
` (5 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
dma_contiguous_set_default contains a trivial assignment, and has a
single caller that is compiled if CONFIG_CMA_DMA is enabled.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/dma-contiguous.h | 7 -------
kernel/dma/contiguous.c | 2 +-
2 files changed, 1 insertion(+), 8 deletions(-)
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index 41ec08d81bc317..f9ce1ee58d4180 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -66,11 +66,6 @@ static inline struct cma *dev_get_cma_area(struct device *dev)
return dma_contiguous_default_area;
}
-static inline void dma_contiguous_set_default(struct cma *cma)
-{
- dma_contiguous_default_area = cma;
-}
-
void dma_contiguous_reserve(phys_addr_t addr_limit);
int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
@@ -91,8 +86,6 @@ static inline struct cma *dev_get_cma_area(struct device *dev)
return NULL;
}
-static inline void dma_contiguous_set_default(struct cma *cma) { }
-
static inline void dma_contiguous_reserve(phys_addr_t limit) { }
static inline int dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index 95adcee972e85c..bf05ec2256e149 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -407,7 +407,7 @@ static int __init rmem_cma_setup(struct reserved_mem *rmem)
dma_contiguous_early_fixup(rmem->base, rmem->size);
if (default_cma)
- dma_contiguous_set_default(cma);
+ dma_contiguous_default_area = cma;
rmem->ops = &rmem_cma_ops;
rmem->priv = cma;
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 5/9] dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h>
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (3 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 4/9] dma-contiguous: remove dma_contiguous_set_default Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 6/9] dma-mapping: remove <asm/dma-contiguous.h> Christoph Hellwig
` (4 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Merge dma-contiguous.h into dma-map-ops.h, after removing the comment
describing the contiguous allocator into kernel/dma/contigous.c.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
.../admin-guide/kernel-parameters.txt | 2 +-
arch/arm/mach-davinci/devices-da8xx.c | 2 +-
arch/arm/mach-shmobile/setup-rcar-gen2.c | 2 +-
arch/arm/mm/dma-mapping.c | 1 -
arch/arm/mm/init.c | 2 +-
arch/arm64/mm/init.c | 3 +-
arch/csky/kernel/setup.c | 2 +-
arch/csky/mm/dma-mapping.c | 3 +-
arch/microblaze/mm/init.c | 2 +-
arch/mips/kernel/setup.c | 2 +-
arch/mips/mm/dma-noncoherent.c | 1 -
arch/s390/kernel/setup.c | 2 +-
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/kernel/setup.c | 2 +-
arch/xtensa/kernel/pci-dma.c | 2 +-
arch/xtensa/mm/init.c | 2 +-
drivers/dma-buf/heaps/cma_heap.c | 2 +-
drivers/iommu/amd/iommu.c | 3 +-
drivers/iommu/dma-iommu.c | 1 -
drivers/iommu/intel/iommu.c | 2 +-
drivers/media/platform/exynos4-is/fimc-is.c | 1 -
include/linux/dma-contiguous.h | 135 ------------------
include/linux/dma-map-ops.h | 65 +++++++++
kernel/dma/Kconfig | 2 +-
kernel/dma/contiguous.c | 30 +++-
kernel/dma/direct.c | 1 -
kernel/dma/ops_helpers.c | 1 -
kernel/dma/pool.c | 2 +-
28 files changed, 112 insertions(+), 164 deletions(-)
delete mode 100644 include/linux/dma-contiguous.h
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index e464cf0b502591..7657e00e83ca38 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -597,7 +597,7 @@
placement constraint by the physical address range of
memory allocations. A value of 0 disables CMA
altogether. For more information, see
- include/linux/dma-contiguous.h
+ kernel/dma/contiguous.c
cma_pernuma=nn[MG]
[ARM64,KNL]
diff --git a/arch/arm/mach-davinci/devices-da8xx.c b/arch/arm/mach-davinci/devices-da8xx.c
index 1207eabe8d1cc4..bb368938fc4921 100644
--- a/arch/arm/mach-davinci/devices-da8xx.c
+++ b/arch/arm/mach-davinci/devices-da8xx.c
@@ -10,7 +10,7 @@
#include <linux/clk-provider.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
#include <linux/io.h>
diff --git a/arch/arm/mach-shmobile/setup-rcar-gen2.c b/arch/arm/mach-shmobile/setup-rcar-gen2.c
index c42ff8c314c82d..e00f5b3b929362 100644
--- a/arch/arm/mach-shmobile/setup-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/setup-rcar-gen2.c
@@ -9,7 +9,7 @@
#include <linux/clocksource.h>
#include <linux/device.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/memblock.h>
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 8bf0bc6bc31172..154c24cec94c74 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -17,7 +17,6 @@
#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
-#include <linux/dma-contiguous.h>
#include <linux/highmem.h>
#include <linux/memblock.h>
#include <linux/slab.h>
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 000c1b48e9734f..ab1d1931a3525e 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -18,7 +18,7 @@
#include <linux/highmem.h>
#include <linux/gfp.h>
#include <linux/memblock.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/sizes.h>
#include <linux/stop_machine.h>
#include <linux/swiotlb.h>
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index f1c75957ff3c24..1b591ddb28b01b 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -21,8 +21,7 @@
#include <linux/of.h>
#include <linux/of_fdt.h>
#include <linux/dma-direct.h>
-#include <linux/dma-mapping.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/efi.h>
#include <linux/swiotlb.h>
#include <linux/vmalloc.h>
diff --git a/arch/csky/kernel/setup.c b/arch/csky/kernel/setup.c
index 0481f4e34538ea..e4cab16056d606 100644
--- a/arch/csky/kernel/setup.c
+++ b/arch/csky/kernel/setup.c
@@ -7,7 +7,7 @@
#include <linux/of.h>
#include <linux/of_fdt.h>
#include <linux/start_kernel.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/screen_info.h>
#include <asm/sections.h>
#include <asm/mmu_context.h>
diff --git a/arch/csky/mm/dma-mapping.c b/arch/csky/mm/dma-mapping.c
index 8f6571ae27c867..3d9ff26c4bb4d8 100644
--- a/arch/csky/mm/dma-mapping.c
+++ b/arch/csky/mm/dma-mapping.c
@@ -2,8 +2,7 @@
// Copyright (C) 2018 Hangzhou C-SKY Microsystems co.,ltd.
#include <linux/cache.h>
-#include <linux/dma-mapping.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <linux/genalloc.h>
#include <linux/highmem.h>
diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c
index 3344d4a1fe890c..7659a8b86580fd 100644
--- a/arch/microblaze/mm/init.c
+++ b/arch/microblaze/mm/init.c
@@ -7,7 +7,7 @@
* for more details.
*/
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/memblock.h>
#include <linux/init.h>
#include <linux/kernel.h>
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index bf5f5acab0a82f..464bfd3957ae96 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -24,7 +24,7 @@
#include <linux/kexec.h>
#include <linux/sizes.h>
#include <linux/device.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/decompress/generic.h>
#include <linux/of_fdt.h>
#include <linux/of_reserved_mem.h>
diff --git a/arch/mips/mm/dma-noncoherent.c b/arch/mips/mm/dma-noncoherent.c
index f4e8404ee0490d..22d99ccc70c81e 100644
--- a/arch/mips/mm/dma-noncoherent.c
+++ b/arch/mips/mm/dma-noncoherent.c
@@ -7,7 +7,6 @@
#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
-#include <linux/dma-contiguous.h>
#include <linux/highmem.h>
#include <asm/cache.h>
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index c2c1b4e723eafb..151092565a2704 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -37,7 +37,7 @@
#include <linux/root_dev.h>
#include <linux/console.h>
#include <linux/kernel_stat.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/device.h>
#include <linux/notifier.h>
#include <linux/pfn.h>
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index fed67eafcaccc2..e0c380b3ec1407 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -11,7 +11,6 @@
#include <linux/dma-debug.h>
#include <asm/io.h>
#include <asm/swiotlb.h>
-#include <linux/dma-contiguous.h>
extern int iommu_merge;
extern int panic_on_overflow;
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 9286fa9d575e6d..e8155e85bd8f34 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -7,7 +7,7 @@
*/
#include <linux/console.h>
#include <linux/crash_dump.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dmi.h>
#include <linux/efi.h>
#include <linux/init_ohci1394_dma.h>
diff --git a/arch/xtensa/kernel/pci-dma.c b/arch/xtensa/kernel/pci-dma.c
index 17c4384f849558..790dbe022a0796 100644
--- a/arch/xtensa/kernel/pci-dma.c
+++ b/arch/xtensa/kernel/pci-dma.c
@@ -11,7 +11,7 @@
* Joe Taylor <joe@tensilica.com, joetylr@yahoo.com>
*/
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
#include <linux/dma-direct.h>
#include <linux/gfp.h>
diff --git a/arch/xtensa/mm/init.c b/arch/xtensa/mm/init.c
index a05b306cf371bf..8079007842ac8e 100644
--- a/arch/xtensa/mm/init.c
+++ b/arch/xtensa/mm/init.c
@@ -26,7 +26,7 @@
#include <linux/nodemask.h>
#include <linux/mm.h>
#include <linux/of_fdt.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <asm/bootparam.h>
#include <asm/page.h>
diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c
index 626cf7fd033afd..e55384dc115ba2 100644
--- a/drivers/dma-buf/heaps/cma_heap.c
+++ b/drivers/dma-buf/heaps/cma_heap.c
@@ -10,7 +10,7 @@
#include <linux/device.h>
#include <linux/dma-buf.h>
#include <linux/dma-heap.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/err.h>
#include <linux/errno.h>
#include <linux/highmem.h>
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 10e4200d355204..5396eb8d730bca 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -18,7 +18,7 @@
#include <linux/slab.h>
#include <linux/debugfs.h>
#include <linux/scatterlist.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/dma-iommu.h>
#include <linux/iommu-helper.h>
@@ -28,7 +28,6 @@
#include <linux/export.h>
#include <linux/irq.h>
#include <linux/msi.h>
-#include <linux/dma-contiguous.h>
#include <linux/irqdomain.h>
#include <linux/percpu.h>
#include <linux/iova.h>
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index d2e3f26228151f..22c221bba13e2f 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -11,7 +11,6 @@
#include <linux/acpi_iort.h>
#include <linux/device.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-contiguous.h>
#include <linux/dma-iommu.h>
#include <linux/dma-noncoherent.h>
#include <linux/gfp.h>
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index bd3470142b0678..0c5b4500ae83d3 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -37,7 +37,7 @@
#include <linux/dmi.h>
#include <linux/pci-ats.h>
#include <linux/memblock.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/crash_dump.h>
#include <linux/numa.h>
diff --git a/drivers/media/platform/exynos4-is/fimc-is.c b/drivers/media/platform/exynos4-is/fimc-is.c
index a474014f0a0fa5..32f5982afa1ab2 100644
--- a/drivers/media/platform/exynos4-is/fimc-is.c
+++ b/drivers/media/platform/exynos4-is/fimc-is.c
@@ -12,7 +12,6 @@
#include <linux/device.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
-#include <linux/dma-contiguous.h>
#include <linux/errno.h>
#include <linux/firmware.h>
#include <linux/interrupt.h>
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
deleted file mode 100644
index f9ce1ee58d4180..00000000000000
--- a/include/linux/dma-contiguous.h
+++ /dev/null
@@ -1,135 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-#ifndef __LINUX_CMA_H
-#define __LINUX_CMA_H
-
-/*
- * Contiguous Memory Allocator for DMA mapping framework
- * Copyright (c) 2010-2011 by Samsung Electronics.
- * Written by:
- * Marek Szyprowski <m.szyprowski@samsung.com>
- * Michal Nazarewicz <mina86@mina86.com>
- */
-
-/*
- * Contiguous Memory Allocator
- *
- * The Contiguous Memory Allocator (CMA) makes it possible to
- * allocate big contiguous chunks of memory after the system has
- * booted.
- *
- * Why is it needed?
- *
- * Various devices on embedded systems have no scatter-getter and/or
- * IO map support and require contiguous blocks of memory to
- * operate. They include devices such as cameras, hardware video
- * coders, etc.
- *
- * Such devices often require big memory buffers (a full HD frame
- * is, for instance, more then 2 mega pixels large, i.e. more than 6
- * MB of memory), which makes mechanisms such as kmalloc() or
- * alloc_page() ineffective.
- *
- * At the same time, a solution where a big memory region is
- * reserved for a device is suboptimal since often more memory is
- * reserved then strictly required and, moreover, the memory is
- * inaccessible to page system even if device drivers don't use it.
- *
- * CMA tries to solve this issue by operating on memory regions
- * where only movable pages can be allocated from. This way, kernel
- * can use the memory for pagecache and when device driver requests
- * it, allocated pages can be migrated.
- *
- * Driver usage
- *
- * CMA should not be used by the device drivers directly. It is
- * only a helper framework for dma-mapping subsystem.
- *
- * For more information, see kernel-docs in kernel/dma/contiguous.c
- */
-
-#ifdef __KERNEL__
-
-#include <linux/device.h>
-#include <linux/mm.h>
-
-struct cma;
-struct page;
-
-#ifdef CONFIG_DMA_CMA
-
-extern struct cma *dma_contiguous_default_area;
-
-static inline struct cma *dev_get_cma_area(struct device *dev)
-{
- if (dev && dev->cma_area)
- return dev->cma_area;
- return dma_contiguous_default_area;
-}
-
-void dma_contiguous_reserve(phys_addr_t addr_limit);
-
-int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
- phys_addr_t limit, struct cma **res_cma,
- bool fixed);
-
-struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
- unsigned int order, bool no_warn);
-bool dma_release_from_contiguous(struct device *dev, struct page *pages,
- int count);
-struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp);
-void dma_free_contiguous(struct device *dev, struct page *page, size_t size);
-
-#else
-
-static inline struct cma *dev_get_cma_area(struct device *dev)
-{
- return NULL;
-}
-
-static inline void dma_contiguous_reserve(phys_addr_t limit) { }
-
-static inline int dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
- phys_addr_t limit, struct cma **res_cma,
- bool fixed)
-{
- return -ENOSYS;
-}
-
-static inline
-struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
- unsigned int order, bool no_warn)
-{
- return NULL;
-}
-
-static inline
-bool dma_release_from_contiguous(struct device *dev, struct page *pages,
- int count)
-{
- return false;
-}
-
-/* Use fallback alloc() and free() when CONFIG_DMA_CMA=n */
-static inline struct page *dma_alloc_contiguous(struct device *dev, size_t size,
- gfp_t gfp)
-{
- return NULL;
-}
-
-static inline void dma_free_contiguous(struct device *dev, struct page *page,
- size_t size)
-{
- __free_pages(page, get_order(size));
-}
-
-#endif
-
-#ifdef CONFIG_DMA_PERNUMA_CMA
-void dma_pernuma_cma_reserve(void);
-#else
-static inline void dma_pernuma_cma_reserve(void) { }
-#endif
-
-#endif
-
-#endif
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index 4b4ba5bdcf6a8d..474fc81bd4921c 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -8,6 +8,8 @@
#include <linux/dma-mapping.h>
+struct cma;
+
struct dma_map_ops {
void *(*alloc)(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t gfp,
@@ -94,6 +96,69 @@ static inline void set_dma_ops(struct device *dev,
}
#endif /* CONFIG_DMA_OPS */
+#ifdef CONFIG_DMA_CMA
+extern struct cma *dma_contiguous_default_area;
+
+static inline struct cma *dev_get_cma_area(struct device *dev)
+{
+ if (dev && dev->cma_area)
+ return dev->cma_area;
+ return dma_contiguous_default_area;
+}
+
+void dma_contiguous_reserve(phys_addr_t addr_limit);
+int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
+ phys_addr_t limit, struct cma **res_cma, bool fixed);
+
+struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
+ unsigned int order, bool no_warn);
+bool dma_release_from_contiguous(struct device *dev, struct page *pages,
+ int count);
+struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp);
+void dma_free_contiguous(struct device *dev, struct page *page, size_t size);
+#else /* CONFIG_DMA_CMA */
+static inline struct cma *dev_get_cma_area(struct device *dev)
+{
+ return NULL;
+}
+static inline void dma_contiguous_reserve(phys_addr_t limit)
+{
+}
+static inline int dma_contiguous_reserve_area(phys_addr_t size,
+ phys_addr_t base, phys_addr_t limit, struct cma **res_cma,
+ bool fixed)
+{
+ return -ENOSYS;
+}
+static inline struct page *dma_alloc_from_contiguous(struct device *dev,
+ size_t count, unsigned int order, bool no_warn)
+{
+ return NULL;
+}
+static inline bool dma_release_from_contiguous(struct device *dev,
+ struct page *pages, int count)
+{
+ return false;
+}
+/* Use fallback alloc() and free() when CONFIG_DMA_CMA=n */
+static inline struct page *dma_alloc_contiguous(struct device *dev, size_t size,
+ gfp_t gfp)
+{
+ return NULL;
+}
+static inline void dma_free_contiguous(struct device *dev, struct page *page,
+ size_t size)
+{
+ __free_pages(page, get_order(size));
+}
+#endif /* CONFIG_DMA_CMA*/
+
+#ifdef CONFIG_DMA_PERNUMA_CMA
+void dma_pernuma_cma_reserve(void);
+#else
+static inline void dma_pernuma_cma_reserve(void) { }
+#endif /* CONFIG_DMA_PERNUMA_CMA */
+
#ifdef CONFIG_DMA_DECLARE_COHERENT
int dma_declare_coherent_memory(struct device *dev, phys_addr_t phys_addr,
dma_addr_t device_addr, size_t size);
diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
index c5f717021f5654..598e772baebb10 100644
--- a/kernel/dma/Kconfig
+++ b/kernel/dma/Kconfig
@@ -118,7 +118,7 @@ config DMA_CMA
You can disable CMA by specifying "cma=0" on the kernel's command
line.
- For more information see <include/linux/dma-contiguous.h>.
+ For more information see <kernel/dma/contiguous.c>.
If unsure, say "n".
if DMA_CMA
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index bf05ec2256e149..6bfb763fff6fca 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -5,6 +5,34 @@
* Written by:
* Marek Szyprowski <m.szyprowski@samsung.com>
* Michal Nazarewicz <mina86@mina86.com>
+ *
+ * Contiguous Memory Allocator
+ *
+ * The Contiguous Memory Allocator (CMA) makes it possible to
+ * allocate big contiguous chunks of memory after the system has
+ * booted.
+ *
+ * Why is it needed?
+ *
+ * Various devices on embedded systems have no scatter-getter and/or
+ * IO map support and require contiguous blocks of memory to
+ * operate. They include devices such as cameras, hardware video
+ * coders, etc.
+ *
+ * Such devices often require big memory buffers (a full HD frame
+ * is, for instance, more then 2 mega pixels large, i.e. more than 6
+ * MB of memory), which makes mechanisms such as kmalloc() or
+ * alloc_page() ineffective.
+ *
+ * At the same time, a solution where a big memory region is
+ * reserved for a device is suboptimal since often more memory is
+ * reserved then strictly required and, moreover, the memory is
+ * inaccessible to page system even if device drivers don't use it.
+ *
+ * CMA tries to solve this issue by operating on memory regions
+ * where only movable pages can be allocated from. This way, kernel
+ * can use the memory for pagecache and when device driver requests
+ * it, allocated pages can be migrated.
*/
#define pr_fmt(fmt) "cma: " fmt
@@ -21,7 +49,7 @@
#include <linux/memblock.h>
#include <linux/err.h>
#include <linux/sizes.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/cma.h>
#ifdef CONFIG_CMA_SIZE_MBYTES
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 8cf5689a8c4044..87697c86f0b82a 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -10,7 +10,6 @@
#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
-#include <linux/dma-contiguous.h>
#include <linux/pfn.h>
#include <linux/vmalloc.h>
#include <linux/set_memory.h>
diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c
index 60d7b6cdfd8d15..bc80fd2648bcf2 100644
--- a/kernel/dma/ops_helpers.c
+++ b/kernel/dma/ops_helpers.c
@@ -3,7 +3,6 @@
* Helpers for DMA ops implementations. These generally rely on the fact that
* the allocated memory contains normal pages in the direct kernel mapping.
*/
-#include <linux/dma-contiguous.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-noncoherent.h>
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index fe11643ff9cc7b..c9fb5c3d8bd002 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -5,7 +5,7 @@
*/
#include <linux/cma.h>
#include <linux/debugfs.h>
-#include <linux/dma-contiguous.h>
+#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
#include <linux/dma-noncoherent.h>
#include <linux/init.h>
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 6/9] dma-mapping: remove <asm/dma-contiguous.h>
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (4 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 5/9] dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h> Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 7/9] dma-mapping: move dma-debug.h to kernel/dma/ Christoph Hellwig
` (3 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Just provide a weak default definition of dma_contiguous_early_fixup and
let arm override it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/include/asm/dma-contiguous.h | 15 ---------------
arch/arm/mm/dma-mapping.c | 1 -
include/asm-generic/Kbuild | 1 -
include/asm-generic/dma-contiguous.h | 10 ----------
include/linux/dma-map-ops.h | 2 ++
kernel/dma/contiguous.c | 6 +++++-
6 files changed, 7 insertions(+), 28 deletions(-)
delete mode 100644 arch/arm/include/asm/dma-contiguous.h
delete mode 100644 include/asm-generic/dma-contiguous.h
diff --git a/arch/arm/include/asm/dma-contiguous.h b/arch/arm/include/asm/dma-contiguous.h
deleted file mode 100644
index d785187a6f8ac1..00000000000000
--- a/arch/arm/include/asm/dma-contiguous.h
+++ /dev/null
@@ -1,15 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef ASMARM_DMA_CONTIGUOUS_H
-#define ASMARM_DMA_CONTIGUOUS_H
-
-#ifdef __KERNEL__
-#ifdef CONFIG_DMA_CMA
-
-#include <linux/types.h>
-
-void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);
-
-#endif
-#endif
-
-#endif
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 154c24cec94c74..911fc6ea26071e 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -34,7 +34,6 @@
#include <asm/dma-iommu.h>
#include <asm/mach/map.h>
#include <asm/system_info.h>
-#include <asm/dma-contiguous.h>
#include <xen/swiotlb-xen.h>
#include "dma.h"
diff --git a/include/asm-generic/Kbuild b/include/asm-generic/Kbuild
index 74b0612601dd1b..62ebdc731ee239 100644
--- a/include/asm-generic/Kbuild
+++ b/include/asm-generic/Kbuild
@@ -16,7 +16,6 @@ mandatory-y += current.h
mandatory-y += delay.h
mandatory-y += device.h
mandatory-y += div64.h
-mandatory-y += dma-contiguous.h
mandatory-y += dma-mapping.h
mandatory-y += dma.h
mandatory-y += emergency-restart.h
diff --git a/include/asm-generic/dma-contiguous.h b/include/asm-generic/dma-contiguous.h
deleted file mode 100644
index f24b0f9a4f05b6..00000000000000
--- a/include/asm-generic/dma-contiguous.h
+++ /dev/null
@@ -1,10 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_GENERIC_DMA_CONTIGUOUS_H
-#define _ASM_GENERIC_DMA_CONTIGUOUS_H
-
-#include <linux/types.h>
-
-static inline void
-dma_contiguous_early_fixup(phys_addr_t base, unsigned long size) { }
-
-#endif
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index 474fc81bd4921c..7912f5d00ed950 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -116,6 +116,8 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
int count);
struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp);
void dma_free_contiguous(struct device *dev, struct page *page, size_t size);
+
+void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);
#else /* CONFIG_DMA_CMA */
static inline struct cma *dev_get_cma_area(struct device *dev)
{
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index 6bfb763fff6fca..a2ee330a3749ec 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -44,7 +44,6 @@
#endif
#include <asm/page.h>
-#include <asm/dma-contiguous.h>
#include <linux/memblock.h>
#include <linux/err.h>
@@ -212,6 +211,11 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
}
}
+void __weak
+dma_contiguous_early_fixup(phys_addr_t base, unsigned long size)
+{
+}
+
/**
* dma_contiguous_reserve_area() - reserve custom contiguous area
* @size: Size of the reserved area (in bytes),
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 7/9] dma-mapping: move dma-debug.h to kernel/dma/
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (5 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 6/9] dma-mapping: remove <asm/dma-contiguous.h> Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma Christoph Hellwig
` (2 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Most of dma-debug.h is not required by anything outside of kernel/dma.
Move the four declarations needed by dma-mappin.h or dma-ops providers
into dma-mapping.h and dma-map-ops.h, and move the remainder of the
file to kernel/dma/debug.h.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
arch/arm/include/asm/dma-iommu.h | 1 -
arch/arm/include/asm/dma-mapping.h | 1 -
arch/microblaze/kernel/dma.c | 1 -
arch/sh/drivers/pci/pci.c | 1 -
arch/x86/include/asm/dma-mapping.h | 1 -
arch/x86/kernel/pci-dma.c | 1 -
drivers/pci/pci-driver.c | 1 +
include/linux/dma-map-ops.h | 12 +++++
include/linux/dma-mapping.h | 16 ++++++-
kernel/dma/debug.c | 5 +--
.../linux/dma-debug.h => kernel/dma/debug.h | 44 ++-----------------
kernel/dma/mapping.c | 1 +
mm/memory.c | 1 -
13 files changed, 34 insertions(+), 52 deletions(-)
rename include/linux/dma-debug.h => kernel/dma/debug.h (81%)
diff --git a/arch/arm/include/asm/dma-iommu.h b/arch/arm/include/asm/dma-iommu.h
index 86405cc81385c8..fe9ef6f79e9cfe 100644
--- a/arch/arm/include/asm/dma-iommu.h
+++ b/arch/arm/include/asm/dma-iommu.h
@@ -6,7 +6,6 @@
#include <linux/mm_types.h>
#include <linux/scatterlist.h>
-#include <linux/dma-debug.h>
#include <linux/kref.h>
struct dma_iommu_mapping {
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 0a1a536368c3a4..77082246a5e121 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -6,7 +6,6 @@
#include <linux/mm_types.h>
#include <linux/scatterlist.h>
-#include <linux/dma-debug.h>
#include <xen/xen.h>
#include <asm/xen/hypervisor.h>
diff --git a/arch/microblaze/kernel/dma.c b/arch/microblaze/kernel/dma.c
index d7bebd04247b72..a564863db06e98 100644
--- a/arch/microblaze/kernel/dma.c
+++ b/arch/microblaze/kernel/dma.c
@@ -10,7 +10,6 @@
#include <linux/device.h>
#include <linux/dma-noncoherent.h>
#include <linux/gfp.h>
-#include <linux/dma-debug.h>
#include <linux/export.h>
#include <linux/bug.h>
#include <asm/cacheflush.h>
diff --git a/arch/sh/drivers/pci/pci.c b/arch/sh/drivers/pci/pci.c
index 6ab0b7377f6634..a3903304f33faa 100644
--- a/arch/sh/drivers/pci/pci.c
+++ b/arch/sh/drivers/pci/pci.c
@@ -13,7 +13,6 @@
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/types.h>
-#include <linux/dma-debug.h>
#include <linux/io.h>
#include <linux/mutex.h>
#include <linux/spinlock.h>
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index e0c380b3ec1407..bb1654fe0ce74c 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -8,7 +8,6 @@
*/
#include <linux/scatterlist.h>
-#include <linux/dma-debug.h>
#include <asm/io.h>
#include <asm/swiotlb.h>
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 4892dd043d414c..de234e7a8962eb 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -1,7 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
-#include <linux/dma-debug.h>
#include <linux/iommu.h>
#include <linux/dmar.h>
#include <linux/export.h>
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index 449466f71040d2..d1b7169c068403 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -19,6 +19,7 @@
#include <linux/kexec.h>
#include <linux/of_device.h>
#include <linux/acpi.h>
+#include <linux/dma-map-ops.h>
#include "pci.h"
#include "pcie/portdrv.h"
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index 7912f5d00ed950..9891def42da71f 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -220,6 +220,18 @@ static inline void arch_teardown_dma_ops(struct device *dev)
}
#endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
+#ifdef CONFIG_DMA_API_DEBUG
+void dma_debug_add_bus(struct bus_type *bus);
+void debug_dma_dump_mappings(struct device *dev);
+#else
+static inline void dma_debug_add_bus(struct bus_type *bus)
+{
+}
+static inline void debug_dma_dump_mappings(struct device *dev)
+{
+}
+#endif /* CONFIG_DMA_API_DEBUG */
+
extern const struct dma_map_ops dma_dummy_ops;
#endif /* _LINUX_DMA_MAP_OPS_H */
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 9591cd482d7c2d..3f029afdc9dc6a 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -6,7 +6,6 @@
#include <linux/string.h>
#include <linux/device.h>
#include <linux/err.h>
-#include <linux/dma-debug.h>
#include <linux/dma-direction.h>
#include <linux/scatterlist.h>
#include <linux/bug.h>
@@ -76,6 +75,21 @@
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#ifdef CONFIG_DMA_API_DEBUG
+void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
+void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len);
+#else
+static inline void debug_dma_mapping_error(struct device *dev,
+ dma_addr_t dma_addr)
+{
+}
+static inline void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len)
+{
+}
+#endif /* CONFIG_DMA_API_DEBUG */
+
#ifdef CONFIG_HAS_DMA
static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
{
diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c
index 4211800d9f3e4d..14de1271463fd0 100644
--- a/kernel/dma/debug.c
+++ b/kernel/dma/debug.c
@@ -9,10 +9,9 @@
#include <linux/sched/task_stack.h>
#include <linux/scatterlist.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
#include <linux/sched/task.h>
#include <linux/stacktrace.h>
-#include <linux/dma-debug.h>
#include <linux/spinlock.h>
#include <linux/vmalloc.h>
#include <linux/debugfs.h>
@@ -24,8 +23,8 @@
#include <linux/ctype.h>
#include <linux/list.h>
#include <linux/slab.h>
-
#include <asm/sections.h>
+#include "debug.h"
#define HASH_SIZE 16384ULL
#define HASH_FN_SHIFT 13
diff --git a/include/linux/dma-debug.h b/kernel/dma/debug.h
similarity index 81%
rename from include/linux/dma-debug.h
rename to kernel/dma/debug.h
index 7b3b04ba60f32a..83643b3010b2fc 100644
--- a/include/linux/dma-debug.h
+++ b/kernel/dma/debug.h
@@ -5,28 +5,14 @@
* Author: Joerg Roedel <joerg.roedel@amd.com>
*/
-#ifndef __DMA_DEBUG_H
-#define __DMA_DEBUG_H
-
-#include <linux/types.h>
-
-struct device;
-struct scatterlist;
-struct bus_type;
+#ifndef _KERNEL_DMA_DEBUG_H
+#define _KERNEL_DMA_DEBUG_H
#ifdef CONFIG_DMA_API_DEBUG
-
-extern void dma_debug_add_bus(struct bus_type *bus);
-
-extern void debug_dma_map_single(struct device *dev, const void *addr,
- unsigned long len);
-
extern void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr);
-extern void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
-
extern void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
size_t size, int direction);
@@ -64,31 +50,13 @@ extern void debug_dma_sync_sg_for_cpu(struct device *dev,
extern void debug_dma_sync_sg_for_device(struct device *dev,
struct scatterlist *sg,
int nelems, int direction);
-
-extern void debug_dma_dump_mappings(struct device *dev);
-
#else /* CONFIG_DMA_API_DEBUG */
-
-static inline void dma_debug_add_bus(struct bus_type *bus)
-{
-}
-
-static inline void debug_dma_map_single(struct device *dev, const void *addr,
- unsigned long len)
-{
-}
-
static inline void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr)
{
}
-static inline void debug_dma_mapping_error(struct device *dev,
- dma_addr_t dma_addr)
-{
-}
-
static inline void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
size_t size, int direction)
{
@@ -150,11 +118,5 @@ static inline void debug_dma_sync_sg_for_device(struct device *dev,
int nelems, int direction)
{
}
-
-static inline void debug_dma_dump_mappings(struct device *dev)
-{
-}
-
#endif /* CONFIG_DMA_API_DEBUG */
-
-#endif /* __DMA_DEBUG_H */
+#endif /* _KERNEL_DMA_DEBUG_H */
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 2e13e6d3903fa0..335ba183e0956a 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -14,6 +14,7 @@
#include <linux/of_device.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>
+#include "debug.h"
/*
* Managed DMA API
diff --git a/mm/memory.c b/mm/memory.c
index f3eb5597590232..7e4e8b81a73823 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -65,7 +65,6 @@
#include <linux/gfp.h>
#include <linux/migrate.h>
#include <linux/string.h>
-#include <linux/dma-debug.h>
#include <linux/debugfs.h>
#include <linux/userfaultfd_k.h>
#include <linux/dax.h>
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (6 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 7/9] dma-mapping: move dma-debug.h to kernel/dma/ Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-10-19 2:25 ` Alexey Kardashevskiy
2020-09-30 8:55 ` [PATCH 9/9] dma-mapping: merge <linux/dma-noncoherent.h> into <linux/dma-map-ops.h> Christoph Hellwig
2020-10-05 17:05 ` clean up the DMA mapping headers Christoph Hellwig
9 siblings, 1 reply; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Most of the dma_direct symbols should only be used by direct.c and
mapping.c, so move them to kernel/dma. In fact more of dma-direct.h
should eventually move, but that will require more coordination with
other subsystems.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/dma-direct.h | 106 ---------------------------------
kernel/dma/direct.c | 2 +-
kernel/dma/direct.h | 119 +++++++++++++++++++++++++++++++++++++
kernel/dma/mapping.c | 2 +-
4 files changed, 121 insertions(+), 108 deletions(-)
create mode 100644 kernel/dma/direct.h
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 38ed3b55034d50..a2d6640c42c04e 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -120,114 +120,8 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
void dma_direct_free_pages(struct device *dev, size_t size,
struct page *page, dma_addr_t dma_addr,
enum dma_data_direction dir);
-int dma_direct_get_sgtable(struct device *dev, struct sg_table *sgt,
- void *cpu_addr, dma_addr_t dma_addr, size_t size,
- unsigned long attrs);
-bool dma_direct_can_mmap(struct device *dev);
-int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size,
- unsigned long attrs);
int dma_direct_supported(struct device *dev, u64 mask);
-bool dma_direct_need_sync(struct device *dev, dma_addr_t dma_addr);
-int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
- enum dma_data_direction dir, unsigned long attrs);
dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir, unsigned long attrs);
-size_t dma_direct_max_mapping_size(struct device *dev);
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
- defined(CONFIG_SWIOTLB)
-void dma_direct_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction dir);
-#else
-static inline void dma_direct_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
-}
-#endif
-
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
- defined(CONFIG_SWIOTLB)
-void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction dir, unsigned long attrs);
-void dma_direct_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir);
-#else
-static inline void dma_direct_unmap_sg(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir,
- unsigned long attrs)
-{
-}
-static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
-}
-#endif
-
-static inline void dma_direct_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- phys_addr_t paddr = dma_to_phys(dev, addr);
-
- if (unlikely(is_swiotlb_buffer(paddr)))
- swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
-
- if (!dev_is_dma_coherent(dev))
- arch_sync_dma_for_device(paddr, size, dir);
-}
-
-static inline void dma_direct_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- phys_addr_t paddr = dma_to_phys(dev, addr);
-
- if (!dev_is_dma_coherent(dev)) {
- arch_sync_dma_for_cpu(paddr, size, dir);
- arch_sync_dma_for_cpu_all();
- }
-
- if (unlikely(is_swiotlb_buffer(paddr)))
- swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
-
- if (dir == DMA_FROM_DEVICE)
- arch_dma_mark_clean(paddr, size);
-}
-
-static inline dma_addr_t dma_direct_map_page(struct device *dev,
- struct page *page, unsigned long offset, size_t size,
- enum dma_data_direction dir, unsigned long attrs)
-{
- phys_addr_t phys = page_to_phys(page) + offset;
- dma_addr_t dma_addr = phys_to_dma(dev, phys);
-
- if (unlikely(swiotlb_force == SWIOTLB_FORCE))
- return swiotlb_map(dev, phys, size, dir, attrs);
-
- if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
- if (swiotlb_force != SWIOTLB_NO_FORCE)
- return swiotlb_map(dev, phys, size, dir, attrs);
-
- dev_WARN_ONCE(dev, 1,
- "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
- &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
- return DMA_MAPPING_ERROR;
- }
-
- if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- arch_sync_dma_for_device(phys, size, dir);
- return dma_addr;
-}
-
-static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
- phys_addr_t phys = dma_to_phys(dev, addr);
-
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_direct_sync_single_for_cpu(dev, addr, size, dir);
-
- if (unlikely(is_swiotlb_buffer(phys)))
- swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
-}
#endif /* _LINUX_DMA_DIRECT_H */
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 87697c86f0b82a..bf9f77623022bb 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -7,13 +7,13 @@
#include <linux/memblock.h> /* for max_pfn */
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
#include <linux/scatterlist.h>
#include <linux/pfn.h>
#include <linux/vmalloc.h>
#include <linux/set_memory.h>
#include <linux/slab.h>
+#include "direct.h"
/*
* Most architectures use ZONE_DMA for the first 16 Megabytes, but some use it
diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
new file mode 100644
index 00000000000000..b9861557873768
--- /dev/null
+++ b/kernel/dma/direct.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 Christoph Hellwig.
+ *
+ * DMA operations that map physical memory directly without using an IOMMU.
+ */
+#ifndef _KERNEL_DMA_DIRECT_H
+#define _KERNEL_DMA_DIRECT_H
+
+#include <linux/dma-direct.h>
+
+int dma_direct_get_sgtable(struct device *dev, struct sg_table *sgt,
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs);
+bool dma_direct_can_mmap(struct device *dev);
+int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs);
+bool dma_direct_need_sync(struct device *dev, dma_addr_t dma_addr);
+int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+ enum dma_data_direction dir, unsigned long attrs);
+size_t dma_direct_max_mapping_size(struct device *dev);
+
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+ defined(CONFIG_SWIOTLB)
+void dma_direct_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction dir);
+#else
+static inline void dma_direct_sync_sg_for_device(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+}
+#endif
+
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
+ defined(CONFIG_SWIOTLB)
+void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction dir, unsigned long attrs);
+void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir);
+#else
+static inline void dma_direct_unmap_sg(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir,
+ unsigned long attrs)
+{
+}
+static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+}
+#endif
+
+static inline void dma_direct_sync_single_for_device(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+ phys_addr_t paddr = dma_to_phys(dev, addr);
+
+ if (unlikely(is_swiotlb_buffer(paddr)))
+ swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
+
+ if (!dev_is_dma_coherent(dev))
+ arch_sync_dma_for_device(paddr, size, dir);
+}
+
+static inline void dma_direct_sync_single_for_cpu(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+ phys_addr_t paddr = dma_to_phys(dev, addr);
+
+ if (!dev_is_dma_coherent(dev)) {
+ arch_sync_dma_for_cpu(paddr, size, dir);
+ arch_sync_dma_for_cpu_all();
+ }
+
+ if (unlikely(is_swiotlb_buffer(paddr)))
+ swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
+
+ if (dir == DMA_FROM_DEVICE)
+ arch_dma_mark_clean(paddr, size);
+}
+
+static inline dma_addr_t dma_direct_map_page(struct device *dev,
+ struct page *page, unsigned long offset, size_t size,
+ enum dma_data_direction dir, unsigned long attrs)
+{
+ phys_addr_t phys = page_to_phys(page) + offset;
+ dma_addr_t dma_addr = phys_to_dma(dev, phys);
+
+ if (unlikely(swiotlb_force == SWIOTLB_FORCE))
+ return swiotlb_map(dev, phys, size, dir, attrs);
+
+ if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
+ if (swiotlb_force != SWIOTLB_NO_FORCE)
+ return swiotlb_map(dev, phys, size, dir, attrs);
+
+ dev_WARN_ONCE(dev, 1,
+ "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
+ &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
+ return DMA_MAPPING_ERROR;
+ }
+
+ if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ arch_sync_dma_for_device(phys, size, dir);
+ return dma_addr;
+}
+
+static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ phys_addr_t phys = dma_to_phys(dev, addr);
+
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_single_for_cpu(dev, addr, size, dir);
+
+ if (unlikely(is_swiotlb_buffer(phys)))
+ swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
+}
+#endif /* _KERNEL_DMA_DIRECT_H */
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 335ba183e0956a..51bb8fa8eb8948 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -7,7 +7,6 @@
*/
#include <linux/memblock.h> /* for max_pfn */
#include <linux/acpi.h>
-#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
#include <linux/export.h>
#include <linux/gfp.h>
@@ -15,6 +14,7 @@
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "debug.h"
+#include "direct.h"
/*
* Managed DMA API
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 9/9] dma-mapping: merge <linux/dma-noncoherent.h> into <linux/dma-map-ops.h>
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (7 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma Christoph Hellwig
@ 2020-09-30 8:55 ` Christoph Hellwig
2020-10-05 17:05 ` clean up the DMA mapping headers Christoph Hellwig
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-09-30 8:55 UTC (permalink / raw)
To: iommu
Cc: Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
Move more nitty gritty DMA implementation details into the common
internal header.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
MAINTAINERS | 1 -
arch/arc/mm/dma.c | 1 -
arch/arm/mm/dma-mapping.c | 1 -
arch/arm/xen/mm.c | 2 +-
arch/arm64/mm/dma-mapping.c | 1 -
arch/c6x/mm/dma-coherent.c | 2 +-
arch/csky/mm/dma-mapping.c | 1 -
arch/hexagon/kernel/dma.c | 2 +-
arch/ia64/mm/init.c | 2 +-
arch/m68k/kernel/dma.c | 2 +-
arch/microblaze/kernel/dma.c | 2 +-
arch/microblaze/mm/consistent.c | 2 +-
arch/mips/jazz/jazzdma.c | 1 -
arch/mips/mm/dma-noncoherent.c | 1 -
arch/nds32/kernel/dma.c | 2 +-
arch/openrisc/kernel/dma.c | 2 +-
arch/parisc/kernel/pci-dma.c | 2 +-
arch/powerpc/mm/dma-noncoherent.c | 2 +-
arch/sh/kernel/dma-coherent.c | 2 +-
arch/sparc/kernel/ioport.c | 2 +-
arch/xtensa/kernel/pci-dma.c | 1 -
drivers/iommu/dma-iommu.c | 1 -
drivers/xen/swiotlb-xen.c | 3 +-
include/linux/dma-direct.h | 2 +-
include/linux/dma-map-ops.h | 102 ++++++++++++++++++++++++++++
include/linux/dma-noncoherent.h | 109 ------------------------------
kernel/dma/ops_helpers.c | 1 -
kernel/dma/pool.c | 1 -
kernel/dma/swiotlb.c | 2 +-
29 files changed, 118 insertions(+), 137 deletions(-)
delete mode 100644 include/linux/dma-noncoherent.h
diff --git a/MAINTAINERS b/MAINTAINERS
index b13fc17943079a..c0dbe6e9de6549 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5203,7 +5203,6 @@ F: include/asm-generic/dma-mapping.h
F: include/linux/dma-direct.h
F: include/linux/dma-mapping.h
F: include/linux/dma-map-ops.h
-F: include/linux/dma-noncoherent.h
F: kernel/dma/
DMA-BUF HEAPS FRAMEWORK
diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c
index a8c453e98d753b..517988e60cfc4d 100644
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -4,7 +4,6 @@
*/
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <asm/cache.h>
#include <asm/cacheflush.h>
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 911fc6ea26071e..c4b8df2ad32850 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -16,7 +16,6 @@
#include <linux/device.h>
#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <linux/highmem.h>
#include <linux/memblock.h>
#include <linux/slab.h>
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index 396797ffe2b1df..5c80088db13b59 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
#include <linux/cpu.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/gfp.h>
#include <linux/highmem.h>
#include <linux/export.h>
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 3afd3bd659d8de..93e87b2875567e 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -7,7 +7,6 @@
#include <linux/gfp.h>
#include <linux/cache.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <linux/dma-iommu.h>
#include <xen/xen.h>
#include <xen/swiotlb-xen.h>
diff --git a/arch/c6x/mm/dma-coherent.c b/arch/c6x/mm/dma-coherent.c
index a5909091cb1424..03df07a831fc99 100644
--- a/arch/c6x/mm/dma-coherent.c
+++ b/arch/c6x/mm/dma-coherent.c
@@ -15,7 +15,7 @@
#include <linux/bitops.h>
#include <linux/module.h>
#include <linux/interrupt.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/memblock.h>
#include <asm/cacheflush.h>
diff --git a/arch/csky/mm/dma-mapping.c b/arch/csky/mm/dma-mapping.c
index 3d9ff26c4bb4d8..c3a775a7e8f9d8 100644
--- a/arch/csky/mm/dma-mapping.c
+++ b/arch/csky/mm/dma-mapping.c
@@ -3,7 +3,6 @@
#include <linux/cache.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <linux/genalloc.h>
#include <linux/highmem.h>
#include <linux/io.h>
diff --git a/arch/hexagon/kernel/dma.c b/arch/hexagon/kernel/dma.c
index 25f388d9cfcc36..00b9a81075dd54 100644
--- a/arch/hexagon/kernel/dma.c
+++ b/arch/hexagon/kernel/dma.c
@@ -5,7 +5,7 @@
* Copyright (c) 2010-2012, The Linux Foundation. All rights reserved.
*/
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/memblock.h>
#include <linux/genalloc.h>
#include <linux/module.h>
diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
index 02e5aa08294ee0..ccba04d1267182 100644
--- a/arch/ia64/mm/init.c
+++ b/arch/ia64/mm/init.c
@@ -8,7 +8,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/dmar.h>
#include <linux/efi.h>
#include <linux/elf.h>
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index b1ca3522eccc2f..1c1b875fadc180 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -6,7 +6,7 @@
#undef DEBUG
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
diff --git a/arch/microblaze/kernel/dma.c b/arch/microblaze/kernel/dma.c
index a564863db06e98..04d091ade4172b 100644
--- a/arch/microblaze/kernel/dma.c
+++ b/arch/microblaze/kernel/dma.c
@@ -8,7 +8,7 @@
*/
#include <linux/device.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/gfp.h>
#include <linux/export.h>
#include <linux/bug.h>
diff --git a/arch/microblaze/mm/consistent.c b/arch/microblaze/mm/consistent.c
index e09b66e43cb63f..81dffe43b18c1e 100644
--- a/arch/microblaze/mm/consistent.c
+++ b/arch/microblaze/mm/consistent.c
@@ -11,7 +11,7 @@
#include <linux/types.h>
#include <linux/mm.h>
#include <linux/init.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <asm/cpuinfo.h>
#include <asm/cacheflush.h>
diff --git a/arch/mips/jazz/jazzdma.c b/arch/mips/jazz/jazzdma.c
index b8fb42e56da0a8..461457b289828a 100644
--- a/arch/mips/jazz/jazzdma.c
+++ b/arch/mips/jazz/jazzdma.c
@@ -17,7 +17,6 @@
#include <linux/spinlock.h>
#include <linux/gfp.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <asm/mipsregs.h>
#include <asm/jazz.h>
#include <asm/io.h>
diff --git a/arch/mips/mm/dma-noncoherent.c b/arch/mips/mm/dma-noncoherent.c
index 22d99ccc70c81e..38d3d9143b47fb 100644
--- a/arch/mips/mm/dma-noncoherent.c
+++ b/arch/mips/mm/dma-noncoherent.c
@@ -6,7 +6,6 @@
*/
#include <linux/dma-direct.h>
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <linux/highmem.h>
#include <asm/cache.h>
diff --git a/arch/nds32/kernel/dma.c b/arch/nds32/kernel/dma.c
index 69d762182d49bf..2ac8e6c82a61a5 100644
--- a/arch/nds32/kernel/dma.c
+++ b/arch/nds32/kernel/dma.c
@@ -3,7 +3,7 @@
#include <linux/types.h>
#include <linux/mm.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/cache.h>
#include <linux/highmem.h>
#include <asm/cacheflush.h>
diff --git a/arch/openrisc/kernel/dma.c b/arch/openrisc/kernel/dma.c
index 345727638d52da..1b16d97e7da7f9 100644
--- a/arch/openrisc/kernel/dma.c
+++ b/arch/openrisc/kernel/dma.c
@@ -13,7 +13,7 @@
* DMA mapping callbacks...
*/
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/pagewalk.h>
#include <asm/cpuinfo.h>
diff --git a/arch/parisc/kernel/pci-dma.c b/arch/parisc/kernel/pci-dma.c
index ce38c0b9158125..36610a5c029fc2 100644
--- a/arch/parisc/kernel/pci-dma.c
+++ b/arch/parisc/kernel/pci-dma.c
@@ -26,7 +26,7 @@
#include <linux/string.h>
#include <linux/types.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <asm/cacheflush.h>
#include <asm/dma.h> /* for DMA_CHUNK_SIZE */
diff --git a/arch/powerpc/mm/dma-noncoherent.c b/arch/powerpc/mm/dma-noncoherent.c
index 5ab4f868e919b8..30260b5d146da3 100644
--- a/arch/powerpc/mm/dma-noncoherent.c
+++ b/arch/powerpc/mm/dma-noncoherent.c
@@ -11,7 +11,7 @@
#include <linux/types.h>
#include <linux/highmem.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <asm/tlbflush.h>
#include <asm/dma.h>
diff --git a/arch/sh/kernel/dma-coherent.c b/arch/sh/kernel/dma-coherent.c
index cd46a9825e3c59..6a44c0e7ba40ef 100644
--- a/arch/sh/kernel/dma-coherent.c
+++ b/arch/sh/kernel/dma-coherent.c
@@ -3,7 +3,7 @@
* Copyright (C) 2004 - 2007 Paul Mundt
*/
#include <linux/mm.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <asm/cacheflush.h>
#include <asm/addrspace.h>
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index d6874c9b639f8d..8e1d72a1675949 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -38,7 +38,7 @@
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/scatterlist.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/of_device.h>
#include <asm/io.h>
diff --git a/arch/xtensa/kernel/pci-dma.c b/arch/xtensa/kernel/pci-dma.c
index 790dbe022a0796..94955caa4488c8 100644
--- a/arch/xtensa/kernel/pci-dma.c
+++ b/arch/xtensa/kernel/pci-dma.c
@@ -12,7 +12,6 @@
*/
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
#include <linux/dma-direct.h>
#include <linux/gfp.h>
#include <linux/highmem.h>
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 22c221bba13e2f..3a00fb64477b0c 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -12,7 +12,6 @@
#include <linux/device.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-iommu.h>
-#include <linux/dma-noncoherent.h>
#include <linux/gfp.h>
#include <linux/huge_mm.h>
#include <linux/iommu.h>
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 71ff4bf380738c..71ce1b7a23d1cc 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -27,9 +27,8 @@
#define pr_fmt(fmt) "xen:" KBUILD_MODNAME ": " fmt
#include <linux/memblock.h>
-#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/export.h>
#include <xen/swiotlb-xen.h>
#include <xen/page.h>
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index a2d6640c42c04e..18aade195884d7 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -7,7 +7,7 @@
#define _LINUX_DMA_DIRECT_H 1
#include <linux/dma-mapping.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/memblock.h> /* for min_low_pfn */
#include <linux/mem_encrypt.h>
#include <linux/swiotlb.h>
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index 9891def42da71f..33c6e24707a94b 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -7,6 +7,7 @@
#define _LINUX_DMA_MAP_OPS_H
#include <linux/dma-mapping.h>
+#include <linux/pgtable.h>
struct cma;
@@ -202,6 +203,107 @@ static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
}
#endif /* CONFIG_DMA_DECLARE_COHERENT */
+#ifdef CONFIG_ARCH_HAS_DMA_COHERENCE_H
+#include <asm/dma-coherence.h>
+#elif defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+static inline bool dev_is_dma_coherent(struct device *dev)
+{
+ return dev->dma_coherent;
+}
+#else
+static inline bool dev_is_dma_coherent(struct device *dev)
+{
+ return true;
+}
+#endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */
+
+/*
+ * Check if an allocation needs to be marked uncached to be coherent.
+ */
+static __always_inline bool dma_alloc_need_uncached(struct device *dev,
+ unsigned long attrs)
+{
+ if (dev_is_dma_coherent(dev))
+ return false;
+ if (attrs & DMA_ATTR_NO_KERNEL_MAPPING)
+ return false;
+ return true;
+}
+
+void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
+ gfp_t gfp, unsigned long attrs);
+void arch_dma_free(struct device *dev, size_t size, void *cpu_addr,
+ dma_addr_t dma_addr, unsigned long attrs);
+
+#ifdef CONFIG_MMU
+/*
+ * Page protection so that devices that can't snoop CPU caches can use the
+ * memory coherently. We default to pgprot_noncached which is usually used
+ * for ioremap as a safe bet, but architectures can override this with less
+ * strict semantics if possible.
+ */
+#ifndef pgprot_dmacoherent
+#define pgprot_dmacoherent(prot) pgprot_noncached(prot)
+#endif
+
+pgprot_t dma_pgprot(struct device *dev, pgprot_t prot, unsigned long attrs);
+#else
+static inline pgprot_t dma_pgprot(struct device *dev, pgprot_t prot,
+ unsigned long attrs)
+{
+ return prot; /* no protection bits supported without page tables */
+}
+#endif /* CONFIG_MMU */
+
+#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE
+void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir);
+#else
+static inline void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+}
+#endif /* ARCH_HAS_SYNC_DMA_FOR_DEVICE */
+
+#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU
+void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir);
+#else
+static inline void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+}
+#endif /* ARCH_HAS_SYNC_DMA_FOR_CPU */
+
+#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL
+void arch_sync_dma_for_cpu_all(void);
+#else
+static inline void arch_sync_dma_for_cpu_all(void)
+{
+}
+#endif /* CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL */
+
+#ifdef CONFIG_ARCH_HAS_DMA_PREP_COHERENT
+void arch_dma_prep_coherent(struct page *page, size_t size);
+#else
+static inline void arch_dma_prep_coherent(struct page *page, size_t size)
+{
+}
+#endif /* CONFIG_ARCH_HAS_DMA_PREP_COHERENT */
+
+#ifdef CONFIG_ARCH_HAS_DMA_MARK_CLEAN
+void arch_dma_mark_clean(phys_addr_t paddr, size_t size);
+#else
+static inline void arch_dma_mark_clean(phys_addr_t paddr, size_t size)
+{
+}
+#endif /* ARCH_HAS_DMA_MARK_CLEAN */
+
+void *arch_dma_set_uncached(void *addr, size_t size);
+void arch_dma_clear_uncached(void *addr, size_t size);
+
#ifdef CONFIG_ARCH_HAS_SETUP_DMA_OPS
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent);
diff --git a/include/linux/dma-noncoherent.h b/include/linux/dma-noncoherent.h
deleted file mode 100644
index e61283e06576a8..00000000000000
--- a/include/linux/dma-noncoherent.h
+++ /dev/null
@@ -1,109 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_DMA_NONCOHERENT_H
-#define _LINUX_DMA_NONCOHERENT_H 1
-
-#include <linux/dma-mapping.h>
-#include <linux/pgtable.h>
-
-#ifdef CONFIG_ARCH_HAS_DMA_COHERENCE_H
-#include <asm/dma-coherence.h>
-#elif defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
-static inline bool dev_is_dma_coherent(struct device *dev)
-{
- return dev->dma_coherent;
-}
-#else
-static inline bool dev_is_dma_coherent(struct device *dev)
-{
- return true;
-}
-#endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */
-
-/*
- * Check if an allocation needs to be marked uncached to be coherent.
- */
-static __always_inline bool dma_alloc_need_uncached(struct device *dev,
- unsigned long attrs)
-{
- if (dev_is_dma_coherent(dev))
- return false;
- if (attrs & DMA_ATTR_NO_KERNEL_MAPPING)
- return false;
- return true;
-}
-
-void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
- gfp_t gfp, unsigned long attrs);
-void arch_dma_free(struct device *dev, size_t size, void *cpu_addr,
- dma_addr_t dma_addr, unsigned long attrs);
-
-#ifdef CONFIG_MMU
-/*
- * Page protection so that devices that can't snoop CPU caches can use the
- * memory coherently. We default to pgprot_noncached which is usually used
- * for ioremap as a safe bet, but architectures can override this with less
- * strict semantics if possible.
- */
-#ifndef pgprot_dmacoherent
-#define pgprot_dmacoherent(prot) pgprot_noncached(prot)
-#endif
-
-pgprot_t dma_pgprot(struct device *dev, pgprot_t prot, unsigned long attrs);
-#else
-static inline pgprot_t dma_pgprot(struct device *dev, pgprot_t prot,
- unsigned long attrs)
-{
- return prot; /* no protection bits supported without page tables */
-}
-#endif /* CONFIG_MMU */
-
-#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE
-void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
- enum dma_data_direction dir);
-#else
-static inline void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
- enum dma_data_direction dir)
-{
-}
-#endif /* ARCH_HAS_SYNC_DMA_FOR_DEVICE */
-
-#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU
-void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
- enum dma_data_direction dir);
-#else
-static inline void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
- enum dma_data_direction dir)
-{
-}
-#endif /* ARCH_HAS_SYNC_DMA_FOR_CPU */
-
-#ifdef CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL
-void arch_sync_dma_for_cpu_all(void);
-#else
-static inline void arch_sync_dma_for_cpu_all(void)
-{
-}
-#endif /* CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL */
-
-#ifdef CONFIG_ARCH_HAS_DMA_PREP_COHERENT
-void arch_dma_prep_coherent(struct page *page, size_t size);
-#else
-static inline void arch_dma_prep_coherent(struct page *page, size_t size)
-{
-}
-#endif /* CONFIG_ARCH_HAS_DMA_PREP_COHERENT */
-
-#ifdef CONFIG_ARCH_HAS_DMA_MARK_CLEAN
-void arch_dma_mark_clean(phys_addr_t paddr, size_t size);
-#else
-static inline void arch_dma_mark_clean(phys_addr_t paddr, size_t size)
-{
-}
-#endif /* ARCH_HAS_DMA_MARK_CLEAN */
-
-void *arch_dma_set_uncached(void *addr, size_t size);
-void arch_dma_clear_uncached(void *addr, size_t size);
-
-#endif /* _LINUX_DMA_NONCOHERENT_H */
diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c
index bc80fd2648bcf2..910ae69cae7774 100644
--- a/kernel/dma/ops_helpers.c
+++ b/kernel/dma/ops_helpers.c
@@ -4,7 +4,6 @@
* the allocated memory contains normal pages in the direct kernel mapping.
*/
#include <linux/dma-map-ops.h>
-#include <linux/dma-noncoherent.h>
/*
* Create scatter-list for the already allocated DMA buffer.
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index c9fb5c3d8bd002..d4637f72239b40 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -7,7 +7,6 @@
#include <linux/debugfs.h>
#include <linux/dma-map-ops.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
#include <linux/init.h>
#include <linux/genalloc.h>
#include <linux/set_memory.h>
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 4ea72d145cd27d..2be1e8b34ae3b9 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -22,7 +22,7 @@
#include <linux/cache.h>
#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
+#include <linux/dma-map-ops.h>
#include <linux/mm.h>
#include <linux/export.h>
#include <linux/spinlock.h>
--
2.28.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: clean up the DMA mapping headers
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
` (8 preceding siblings ...)
2020-09-30 8:55 ` [PATCH 9/9] dma-mapping: merge <linux/dma-noncoherent.h> into <linux/dma-map-ops.h> Christoph Hellwig
@ 2020-10-05 17:05 ` Christoph Hellwig
9 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-10-05 17:05 UTC (permalink / raw)
To: iommu
Cc: linux-arch, Sekhar Nori, Russell King, linux-kernel,
Robin Murphy, linux-arm-kernel
I've pulled this into the dma-mapping for-next tree now.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h>
2020-09-30 8:55 ` [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h> Christoph Hellwig
@ 2020-10-11 14:33 ` Guenter Roeck
2020-10-13 11:29 ` Christoph Hellwig
0 siblings, 1 reply; 14+ messages in thread
From: Guenter Roeck @ 2020-10-11 14:33 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, Russell King, Sekhar Nori, Marek Szyprowski, Robin Murphy,
linux-kernel, linux-arm-kernel, linux-arch
On Wed, Sep 30, 2020 at 10:55:40AM +0200, Christoph Hellwig wrote:
> Split out all the bits that are purely for dma_map_ops implementations
> and related code into a new <linux/dma-map-ops.h> header so that they
> don't get pulled into all the drivers. That also means the architecture
> specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h>
> any more, which leads to a missing includes that were pulled in by the
> x86 or arm versions in a few not overly portable drivers.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
This patch results in the following build error.
Building arm:ixp4xx_defconfig ... failed
--------------
Error log:
arch/arm/mach-ixp4xx/common.c: In function 'ixp4xx_platform_notify_remove':
arch/arm/mach-ixp4xx/common.c:291:3: error: implicit declaration of function 'dmabounce_unregister_dev' [-Werror=implicit-function-declaration]
291 | dmabounce_unregister_dev(dev);
| ^~~~~~~~~~~~~~~~~~~~~~~~
arch/arm/mach-ixp4xx/common.c: In function 'ixp4xx_platform_notify':
arch/arm/mach-ixp4xx/common.c:307:3: error: implicit declaration of function 'dmabounce_register_dev' [-Werror=implicit-function-declaration]
307 | dmabounce_register_dev(dev, 2048, 4096, ixp4xx_needs_bounce);
Bisect log attached.
Guenter
---
# bad: [d67bc7812221606e1886620a357b13f906814af7] Add linux-next specific files for 20201009
# good: [549738f15da0e5a00275977623be199fbbf7df50] Linux 5.9-rc8
git bisect start 'HEAD' 'v5.9-rc8'
# bad: [b71be15b496cc71a3434a198fc1a1b9e08af6c57] Merge remote-tracking branch 'bpf-next/master' into master
git bisect bad b71be15b496cc71a3434a198fc1a1b9e08af6c57
# bad: [6be11f939f380ef14bc94242cb0262197ce2a054] Merge remote-tracking branch 'i2c/i2c/for-next' into master
git bisect bad 6be11f939f380ef14bc94242cb0262197ce2a054
# bad: [c03a115d8ad8a87b6d275c3c91c13bc111217bf6] Merge remote-tracking branch 'samsung-krzk/for-next' into master
git bisect bad c03a115d8ad8a87b6d275c3c91c13bc111217bf6
# good: [4f7233567480421eab9c622f395854bfd925f9f2] Merge branch 'arm/dt' into for-next
git bisect good 4f7233567480421eab9c622f395854bfd925f9f2
# bad: [0fa3d3941f0f9fc9b39621af3309b5fdf0c1a6cb] Merge remote-tracking branch 'arm/for-next' into master
git bisect bad 0fa3d3941f0f9fc9b39621af3309b5fdf0c1a6cb
# good: [1357d119df6c359a2903a46522c2511d990db9bb] Merge remote-tracking branch 'sound-current/for-linus' into master
git bisect good 1357d119df6c359a2903a46522c2511d990db9bb
# good: [bd02811392faa248d95f8559498c03933eb7d78b] Merge remote-tracking branch 'kspp-gustavo/for-next/kspp' into master
git bisect good bd02811392faa248d95f8559498c03933eb7d78b
# bad: [0b1abd1fb7efafc25231c54a67c6fbb3d3127efd] dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h>
git bisect bad 0b1abd1fb7efafc25231c54a67c6fbb3d3127efd
# good: [b9bb694b9f62f4b31652223ed3ca38cf98bbb370] iommu/io-pgtable-arm: Clean up faulty sanity check
git bisect good b9bb694b9f62f4b31652223ed3ca38cf98bbb370
# good: [ed4bc1890b4984d0af447ad3cc1f93541623f8f3] hal2: convert to dma_alloc_noncoherent
git bisect good ed4bc1890b4984d0af447ad3cc1f93541623f8f3
# good: [e8d39a903cc680c8c1ab5e528428f182f42b8a35] dma-iommu: implement ->alloc_noncoherent
git bisect good e8d39a903cc680c8c1ab5e528428f182f42b8a35
# bad: [0a0f0d8be76dcd4390ff538e7060fda34db79717] dma-mapping: split <linux/dma-mapping.h>
git bisect bad 0a0f0d8be76dcd4390ff538e7060fda34db79717
# good: [0de327969b61a245e3a47b60009eae73fe513cef] cma: decrease CMA_ALIGNMENT lower limit to 2
git bisect good 0de327969b61a245e3a47b60009eae73fe513cef
# first bad commit: [0a0f0d8be76dcd4390ff538e7060fda34db79717] dma-mapping: split <linux/dma-mapping.h>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h>
2020-10-11 14:33 ` Guenter Roeck
@ 2020-10-13 11:29 ` Christoph Hellwig
0 siblings, 0 replies; 14+ messages in thread
From: Christoph Hellwig @ 2020-10-13 11:29 UTC (permalink / raw)
To: Guenter Roeck
Cc: Christoph Hellwig, iommu, Russell King, Sekhar Nori,
Marek Szyprowski, Robin Murphy, linux-kernel, linux-arm-kernel,
linux-arch
Thanks for the report, I've commited the obvious fix.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma
2020-09-30 8:55 ` [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma Christoph Hellwig
@ 2020-10-19 2:25 ` Alexey Kardashevskiy
0 siblings, 0 replies; 14+ messages in thread
From: Alexey Kardashevskiy @ 2020-10-19 2:25 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, Sekhar Nori, Russell King, linux-kernel,
Robin Murphy, linux-arm-kernel, Michael Ellerman
On 30/09/2020 18:55, Christoph Hellwig wrote:
> Most of the dma_direct symbols should only be used by direct.c and
> mapping.c, so move them to kernel/dma. In fact more of dma-direct.h
> should eventually move, but that will require more coordination with
> other subsystems.
Because of this change,
http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20200713062348.100552-1-aik@ozlabs.ru/
does not work anymore.
Should I send a patch moving
dma_direct_map_sg/dma_direct_map_page/+unmap back to include/ or there
is a better idea? thanks,
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> include/linux/dma-direct.h | 106 ---------------------------------
> kernel/dma/direct.c | 2 +-
> kernel/dma/direct.h | 119 +++++++++++++++++++++++++++++++++++++
> kernel/dma/mapping.c | 2 +-
> 4 files changed, 121 insertions(+), 108 deletions(-)
> create mode 100644 kernel/dma/direct.h
>
> diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
> index 38ed3b55034d50..a2d6640c42c04e 100644
> --- a/include/linux/dma-direct.h
> +++ b/include/linux/dma-direct.h
> @@ -120,114 +120,8 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
> void dma_direct_free_pages(struct device *dev, size_t size,
> struct page *page, dma_addr_t dma_addr,
> enum dma_data_direction dir);
> -int dma_direct_get_sgtable(struct device *dev, struct sg_table *sgt,
> - void *cpu_addr, dma_addr_t dma_addr, size_t size,
> - unsigned long attrs);
> -bool dma_direct_can_mmap(struct device *dev);
> -int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
> - void *cpu_addr, dma_addr_t dma_addr, size_t size,
> - unsigned long attrs);
> int dma_direct_supported(struct device *dev, u64 mask);
> -bool dma_direct_need_sync(struct device *dev, dma_addr_t dma_addr);
> -int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
> - enum dma_data_direction dir, unsigned long attrs);
> dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t paddr,
> size_t size, enum dma_data_direction dir, unsigned long attrs);
> -size_t dma_direct_max_mapping_size(struct device *dev);
>
> -#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
> - defined(CONFIG_SWIOTLB)
> -void dma_direct_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
> - int nents, enum dma_data_direction dir);
> -#else
> -static inline void dma_direct_sync_sg_for_device(struct device *dev,
> - struct scatterlist *sgl, int nents, enum dma_data_direction dir)
> -{
> -}
> -#endif
> -
> -#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
> - defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
> - defined(CONFIG_SWIOTLB)
> -void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
> - int nents, enum dma_data_direction dir, unsigned long attrs);
> -void dma_direct_sync_sg_for_cpu(struct device *dev,
> - struct scatterlist *sgl, int nents, enum dma_data_direction dir);
> -#else
> -static inline void dma_direct_unmap_sg(struct device *dev,
> - struct scatterlist *sgl, int nents, enum dma_data_direction dir,
> - unsigned long attrs)
> -{
> -}
> -static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
> - struct scatterlist *sgl, int nents, enum dma_data_direction dir)
> -{
> -}
> -#endif
> -
> -static inline void dma_direct_sync_single_for_device(struct device *dev,
> - dma_addr_t addr, size_t size, enum dma_data_direction dir)
> -{
> - phys_addr_t paddr = dma_to_phys(dev, addr);
> -
> - if (unlikely(is_swiotlb_buffer(paddr)))
> - swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
> -
> - if (!dev_is_dma_coherent(dev))
> - arch_sync_dma_for_device(paddr, size, dir);
> -}
> -
> -static inline void dma_direct_sync_single_for_cpu(struct device *dev,
> - dma_addr_t addr, size_t size, enum dma_data_direction dir)
> -{
> - phys_addr_t paddr = dma_to_phys(dev, addr);
> -
> - if (!dev_is_dma_coherent(dev)) {
> - arch_sync_dma_for_cpu(paddr, size, dir);
> - arch_sync_dma_for_cpu_all();
> - }
> -
> - if (unlikely(is_swiotlb_buffer(paddr)))
> - swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
> -
> - if (dir == DMA_FROM_DEVICE)
> - arch_dma_mark_clean(paddr, size);
> -}
> -
> -static inline dma_addr_t dma_direct_map_page(struct device *dev,
> - struct page *page, unsigned long offset, size_t size,
> - enum dma_data_direction dir, unsigned long attrs)
> -{
> - phys_addr_t phys = page_to_phys(page) + offset;
> - dma_addr_t dma_addr = phys_to_dma(dev, phys);
> -
> - if (unlikely(swiotlb_force == SWIOTLB_FORCE))
> - return swiotlb_map(dev, phys, size, dir, attrs);
> -
> - if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
> - if (swiotlb_force != SWIOTLB_NO_FORCE)
> - return swiotlb_map(dev, phys, size, dir, attrs);
> -
> - dev_WARN_ONCE(dev, 1,
> - "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
> - &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
> - return DMA_MAPPING_ERROR;
> - }
> -
> - if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> - arch_sync_dma_for_device(phys, size, dir);
> - return dma_addr;
> -}
> -
> -static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
> - size_t size, enum dma_data_direction dir, unsigned long attrs)
> -{
> - phys_addr_t phys = dma_to_phys(dev, addr);
> -
> - if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> - dma_direct_sync_single_for_cpu(dev, addr, size, dir);
> -
> - if (unlikely(is_swiotlb_buffer(phys)))
> - swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
> -}
> #endif /* _LINUX_DMA_DIRECT_H */
> diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
> index 87697c86f0b82a..bf9f77623022bb 100644
> --- a/kernel/dma/direct.c
> +++ b/kernel/dma/direct.c
> @@ -7,13 +7,13 @@
> #include <linux/memblock.h> /* for max_pfn */
> #include <linux/export.h>
> #include <linux/mm.h>
> -#include <linux/dma-direct.h>
> #include <linux/dma-map-ops.h>
> #include <linux/scatterlist.h>
> #include <linux/pfn.h>
> #include <linux/vmalloc.h>
> #include <linux/set_memory.h>
> #include <linux/slab.h>
> +#include "direct.h"
>
> /*
> * Most architectures use ZONE_DMA for the first 16 Megabytes, but some use it
> diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h
> new file mode 100644
> index 00000000000000..b9861557873768
> --- /dev/null
> +++ b/kernel/dma/direct.h
> @@ -0,0 +1,119 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2018 Christoph Hellwig.
> + *
> + * DMA operations that map physical memory directly without using an IOMMU.
> + */
> +#ifndef _KERNEL_DMA_DIRECT_H
> +#define _KERNEL_DMA_DIRECT_H
> +
> +#include <linux/dma-direct.h>
> +
> +int dma_direct_get_sgtable(struct device *dev, struct sg_table *sgt,
> + void *cpu_addr, dma_addr_t dma_addr, size_t size,
> + unsigned long attrs);
> +bool dma_direct_can_mmap(struct device *dev);
> +int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,
> + void *cpu_addr, dma_addr_t dma_addr, size_t size,
> + unsigned long attrs);
> +bool dma_direct_need_sync(struct device *dev, dma_addr_t dma_addr);
> +int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
> + enum dma_data_direction dir, unsigned long attrs);
> +size_t dma_direct_max_mapping_size(struct device *dev);
> +
> +#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
> + defined(CONFIG_SWIOTLB)
> +void dma_direct_sync_sg_for_device(struct device *dev, struct scatterlist *sgl,
> + int nents, enum dma_data_direction dir);
> +#else
> +static inline void dma_direct_sync_sg_for_device(struct device *dev,
> + struct scatterlist *sgl, int nents, enum dma_data_direction dir)
> +{
> +}
> +#endif
> +
> +#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
> + defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL) || \
> + defined(CONFIG_SWIOTLB)
> +void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
> + int nents, enum dma_data_direction dir, unsigned long attrs);
> +void dma_direct_sync_sg_for_cpu(struct device *dev,
> + struct scatterlist *sgl, int nents, enum dma_data_direction dir);
> +#else
> +static inline void dma_direct_unmap_sg(struct device *dev,
> + struct scatterlist *sgl, int nents, enum dma_data_direction dir,
> + unsigned long attrs)
> +{
> +}
> +static inline void dma_direct_sync_sg_for_cpu(struct device *dev,
> + struct scatterlist *sgl, int nents, enum dma_data_direction dir)
> +{
> +}
> +#endif
> +
> +static inline void dma_direct_sync_single_for_device(struct device *dev,
> + dma_addr_t addr, size_t size, enum dma_data_direction dir)
> +{
> + phys_addr_t paddr = dma_to_phys(dev, addr);
> +
> + if (unlikely(is_swiotlb_buffer(paddr)))
> + swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_DEVICE);
> +
> + if (!dev_is_dma_coherent(dev))
> + arch_sync_dma_for_device(paddr, size, dir);
> +}
> +
> +static inline void dma_direct_sync_single_for_cpu(struct device *dev,
> + dma_addr_t addr, size_t size, enum dma_data_direction dir)
> +{
> + phys_addr_t paddr = dma_to_phys(dev, addr);
> +
> + if (!dev_is_dma_coherent(dev)) {
> + arch_sync_dma_for_cpu(paddr, size, dir);
> + arch_sync_dma_for_cpu_all();
> + }
> +
> + if (unlikely(is_swiotlb_buffer(paddr)))
> + swiotlb_tbl_sync_single(dev, paddr, size, dir, SYNC_FOR_CPU);
> +
> + if (dir == DMA_FROM_DEVICE)
> + arch_dma_mark_clean(paddr, size);
> +}
> +
> +static inline dma_addr_t dma_direct_map_page(struct device *dev,
> + struct page *page, unsigned long offset, size_t size,
> + enum dma_data_direction dir, unsigned long attrs)
> +{
> + phys_addr_t phys = page_to_phys(page) + offset;
> + dma_addr_t dma_addr = phys_to_dma(dev, phys);
> +
> + if (unlikely(swiotlb_force == SWIOTLB_FORCE))
> + return swiotlb_map(dev, phys, size, dir, attrs);
> +
> + if (unlikely(!dma_capable(dev, dma_addr, size, true))) {
> + if (swiotlb_force != SWIOTLB_NO_FORCE)
> + return swiotlb_map(dev, phys, size, dir, attrs);
> +
> + dev_WARN_ONCE(dev, 1,
> + "DMA addr %pad+%zu overflow (mask %llx, bus limit %llx).\n",
> + &dma_addr, size, *dev->dma_mask, dev->bus_dma_limit);
> + return DMA_MAPPING_ERROR;
> + }
> +
> + if (!dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + arch_sync_dma_for_device(phys, size, dir);
> + return dma_addr;
> +}
> +
> +static inline void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
> + size_t size, enum dma_data_direction dir, unsigned long attrs)
> +{
> + phys_addr_t phys = dma_to_phys(dev, addr);
> +
> + if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + dma_direct_sync_single_for_cpu(dev, addr, size, dir);
> +
> + if (unlikely(is_swiotlb_buffer(phys)))
> + swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs);
> +}
> +#endif /* _KERNEL_DMA_DIRECT_H */
> diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
> index 335ba183e0956a..51bb8fa8eb8948 100644
> --- a/kernel/dma/mapping.c
> +++ b/kernel/dma/mapping.c
> @@ -7,7 +7,6 @@
> */
> #include <linux/memblock.h> /* for max_pfn */
> #include <linux/acpi.h>
> -#include <linux/dma-direct.h>
> #include <linux/dma-map-ops.h>
> #include <linux/export.h>
> #include <linux/gfp.h>
> @@ -15,6 +14,7 @@
> #include <linux/slab.h>
> #include <linux/vmalloc.h>
> #include "debug.h"
> +#include "direct.h"
>
> /*
> * Managed DMA API
>
--
Alexey
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-10-19 2:25 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-30 8:55 clean up the DMA mapping headers Christoph Hellwig
2020-09-30 8:55 ` [PATCH 1/9] dma-mapping: split <linux/dma-mapping.h> Christoph Hellwig
2020-10-11 14:33 ` Guenter Roeck
2020-10-13 11:29 ` Christoph Hellwig
2020-09-30 8:55 ` [PATCH 2/9] dma-contiguous: remove dma_declare_contiguous Christoph Hellwig
2020-09-30 8:55 ` [PATCH 3/9] dma-contiguous: remove dev_set_cma_area Christoph Hellwig
2020-09-30 8:55 ` [PATCH 4/9] dma-contiguous: remove dma_contiguous_set_default Christoph Hellwig
2020-09-30 8:55 ` [PATCH 5/9] dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h> Christoph Hellwig
2020-09-30 8:55 ` [PATCH 6/9] dma-mapping: remove <asm/dma-contiguous.h> Christoph Hellwig
2020-09-30 8:55 ` [PATCH 7/9] dma-mapping: move dma-debug.h to kernel/dma/ Christoph Hellwig
2020-09-30 8:55 ` [PATCH 8/9] dma-mapping: move large parts of <linux/dma-direct.h> to kernel/dma Christoph Hellwig
2020-10-19 2:25 ` Alexey Kardashevskiy
2020-09-30 8:55 ` [PATCH 9/9] dma-mapping: merge <linux/dma-noncoherent.h> into <linux/dma-map-ops.h> Christoph Hellwig
2020-10-05 17:05 ` clean up the DMA mapping headers Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).