IOMMU Archive on lore.kernel.org
 help / color / Atom feed
From: Alexander Graf via iommu <iommu@lists.linux-foundation.org>
To: <iommu@lists.linux-foundation.org>
Cc: Mark Rutland <mark.rutland@arm.com>,
	benh@amazon.com, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	linux-doc@vger.kernel.org, Jan Kiszka <jan.kiszka@siemens.com>,
	x86@kernel.org, linux-kernel@vger.kernel.org, aggh@amazon.com,
	alcioa@amazon.com, aagch@amazon.com, dhr@amazon.com,
	dwmw@amazon.com, Robin Murphy <robin.murphy@arm.com>,
	Christoph Hellwig <hch@lst.de>
Subject: [PATCH] swiotlb: Allow swiotlb to live at pre-defined address
Date: Thu, 26 Mar 2020 17:29:22 +0100
Message-ID: <20200326162922.27085-1-graf@amazon.com> (raw)

The swiotlb is a very convenient fallback mechanism for bounce buffering of
DMAable data. It is usually used for the compatibility case where devices
can only DMA to a "low region".

However, in some scenarios this "low region" may be bound even more
heavily. For example, there are embedded system where only an SRAM region
is shared between device and CPU. There are also heterogeneous computing
scenarios where only a subset of RAM is cache coherent between the
components of the system. There are partitioning hypervisors, where
a "control VM" that implements device emulation has limited view into a
partition's memory for DMA capabilities due to safety concerns.

This patch adds a command line driven mechanism to move all DMA memory into
a predefined shared memory region which may or may not be part of the
physical address layout of the Operating System.

Ideally, the typical path to set this configuration would be through Device
Tree or ACPI, but neither of the two mechanisms is standardized yet. Also,
in the x86 MicroVM use case, we have neither ACPI nor Device Tree, but
instead configure the system purely through kernel command line options.

I'm sure other people will find the functionality useful going forward
though and extend it to be triggered by DT/ACPI in the future.

Signed-off-by: Alexander Graf <graf@amazon.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  3 +-
 Documentation/x86/x86_64/boot-options.rst       |  4 ++-
 kernel/dma/swiotlb.c                            | 46 +++++++++++++++++++++++--
 3 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index c07815d230bc..d085d55c3cbe 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4785,11 +4785,12 @@
 			it if 0 is given (See Documentation/admin-guide/cgroup-v1/memory.rst)
 
 	swiotlb=	[ARM,IA-64,PPC,MIPS,X86]
-			Format: { <int> | force | noforce }
+			Format: { <int> | force | noforce | addr=<phys addr> }
 			<int> -- Number of I/O TLB slabs
 			force -- force using of bounce buffers even if they
 			         wouldn't be automatically used by the kernel
 			noforce -- Never use bounce buffers (for debugging)
+			addr=<phys addr> -- Try to allocate SWIOTLB at defined address
 
 	switches=	[HW,M68k]
 
diff --git a/Documentation/x86/x86_64/boot-options.rst b/Documentation/x86/x86_64/boot-options.rst
index 2b98efb5ba7f..ca46c57b68c9 100644
--- a/Documentation/x86/x86_64/boot-options.rst
+++ b/Documentation/x86/x86_64/boot-options.rst
@@ -297,11 +297,13 @@ iommu options only relevant to the AMD GART hardware IOMMU:
 iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
 implementation:
 
-    swiotlb=<pages>[,force]
+    swiotlb=<pages>[,force][,addr=<phys addr>]
       <pages>
         Prereserve that many 128K pages for the software IO bounce buffering.
       force
         Force all IO through the software TLB.
+      addr=<phys addr>
+        Try to allocate SWIOTLB at defined address
 
 Settings for the IBM Calgary hardware IOMMU currently found in IBM
 pSeries and xSeries machines
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index c19379fabd20..83da0caa2f93 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -46,6 +46,7 @@
 #include <linux/init.h>
 #include <linux/memblock.h>
 #include <linux/iommu-helper.h>
+#include <linux/io.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/swiotlb.h>
@@ -102,6 +103,12 @@ unsigned int max_segment;
 #define INVALID_PHYS_ADDR (~(phys_addr_t)0)
 static phys_addr_t *io_tlb_orig_addr;
 
+/*
+ * The TLB phys addr may be defined on the command line. Store it here if it is.
+ */
+static phys_addr_t io_tlb_addr = INVALID_PHYS_ADDR;
+
+
 /*
  * Protect the above data structures in the map and unmap calls
  */
@@ -119,11 +126,23 @@ setup_io_tlb_npages(char *str)
 	}
 	if (*str == ',')
 		++str;
-	if (!strcmp(str, "force")) {
+	if (!strncmp(str, "force", 5)) {
 		swiotlb_force = SWIOTLB_FORCE;
-	} else if (!strcmp(str, "noforce")) {
+		str += 5;
+	} else if (!strncmp(str, "noforce", 7)) {
 		swiotlb_force = SWIOTLB_NO_FORCE;
 		io_tlb_nslabs = 1;
+		str += 7;
+	}
+
+	if (*str == ',')
+		++str;
+	if (!strncmp(str, "addr=", 5)) {
+		char *addrstr = str + 5;
+
+		io_tlb_addr = kstrtoul(addrstr, 0, &str);
+		if (addrstr == str)
+			io_tlb_addr = INVALID_PHYS_ADDR;
 	}
 
 	return 0;
@@ -239,6 +258,25 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	return 0;
 }
 
+static int __init swiotlb_init_io(int verbose, unsigned long bytes)
+{
+	unsigned __iomem char *vstart;
+
+	if (io_tlb_addr == INVALID_PHYS_ADDR)
+		return -EINVAL;
+
+	vstart = memremap(io_tlb_addr, bytes, MEMREMAP_WB);
+	if (!vstart)
+		return -EINVAL;
+
+	if (swiotlb_init_with_tbl(vstart, io_tlb_nslabs, verbose)) {
+		memunmap(vstart);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 /*
  * Statically reserve bounce buffer space and initialize bounce buffer data
  * structures for the software IO TLB used to implement the DMA API.
@@ -257,6 +295,10 @@ swiotlb_init(int verbose)
 
 	bytes = io_tlb_nslabs << IO_TLB_SHIFT;
 
+	/* Map IO TLB from device memory */
+	if (!swiotlb_init_io(verbose, bytes))
+		return;
+
 	/* Get IO TLB memory from the low pages */
 	vstart = memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);
 	if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, verbose))
-- 
2.16.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

             reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-26 16:29 Alexander Graf via iommu [this message]
2020-03-26 17:05 ` Christoph Hellwig
2020-03-26 17:11   ` Alexander Graf via iommu
2020-03-26 17:16     ` David Woodhouse
2020-03-30 13:24     ` Mark Rutland
2020-03-27  9:58 ` Jan Kiszka
2020-03-28 11:57 ` Dave Young
2020-03-30  6:06   ` Kairui Song
2020-03-30 13:40     ` Konrad Rzeszutek Wilk
2020-03-30 20:42       ` Alexander Graf via iommu
2020-03-30 23:37         ` Anthony Yznaga
2020-03-31  1:59         ` Dave Young
2020-03-31  2:16         ` Baoquan He
2020-03-31  1:46       ` Dave Young

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200326162922.27085-1-graf@amazon.com \
    --to=iommu@lists.linux-foundation.org \
    --cc=aagch@amazon.com \
    --cc=aggh@amazon.com \
    --cc=alcioa@amazon.com \
    --cc=benh@amazon.com \
    --cc=dhr@amazon.com \
    --cc=dwmw@amazon.com \
    --cc=graf@amazon.com \
    --cc=hch@lst.de \
    --cc=jan.kiszka@siemens.com \
    --cc=konrad.wilk@oracle.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=robin.murphy@arm.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org
	public-inbox-index linux-iommu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git