From: Sven Peter <sven@svenpeter.dev>
To: iommu@lists.linux-foundation.org, Robin Murphy <robin.murphy@arm.com>
Cc: Sven Peter <sven@svenpeter.dev>, Joerg Roedel <joro@8bytes.org>,
Will Deacon <will@kernel.org>, Arnd Bergmann <arnd@kernel.org>,
Marc Zyngier <maz@kernel.org>,
Mohamed Mediouni <mohamed.mediouni@caramail.com>,
Alexander Graf <graf@amazon.com>,
Hector Martin <marcan@marcan.st>,
Alyssa Rosenzweig <alyssa@rosenzweig.io>,
linux-kernel@vger.kernel.org
Subject: [PATCH v3 2/6] iommu/dma: Support granule > PAGE_SIZE in dma_map_sg
Date: Tue, 19 Oct 2021 18:37:33 +0200 [thread overview]
Message-ID: <20211019163737.46269-3-sven@svenpeter.dev> (raw)
In-Reply-To: <20211019163737.46269-1-sven@svenpeter.dev>
Add support to iommu_dma_map_sg's impedance matching to also align
sg_lists correctly when the IOMMU granule is larger than PAGE_SIZE.
Co-developed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Sven Peter <sven@svenpeter.dev>
---
drivers/iommu/dma-iommu.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 17f25632a0d6..ea799e70fc98 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
#include <linux/irq.h>
#include <linux/mm.h>
#include <linux/mutex.h>
+#include <linux/pfn.h>
#include <linux/pci.h>
#include <linux/swiotlb.h>
#include <linux/scatterlist.h>
@@ -878,8 +879,9 @@ static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents,
unsigned int s_length = sg_dma_len(s);
unsigned int s_iova_len = s->length;
- s->offset += s_iova_off;
- s->length = s_length;
+ sg_set_page(s,
+ pfn_to_page(PHYS_PFN(sg_phys(s) + s_iova_off)),
+ s_length, s_iova_off & ~PAGE_MASK);
sg_dma_address(s) = DMA_MAPPING_ERROR;
sg_dma_len(s) = 0;
@@ -920,13 +922,17 @@ static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents,
static void __invalidate_sg(struct scatterlist *sg, int nents)
{
struct scatterlist *s;
+ phys_addr_t orig_paddr;
int i;
for_each_sg(sg, s, nents, i) {
- if (sg_dma_address(s) != DMA_MAPPING_ERROR)
- s->offset += sg_dma_address(s);
- if (sg_dma_len(s))
- s->length = sg_dma_len(s);
+ if (sg_dma_len(s)) {
+ orig_paddr = sg_phys(s) + sg_dma_address(s);
+ sg_set_page(s,
+ pfn_to_page(PHYS_PFN(orig_paddr)),
+ sg_dma_len(s),
+ sg_dma_address(s) & ~PAGE_MASK);
+ }
sg_dma_address(s) = DMA_MAPPING_ERROR;
sg_dma_len(s) = 0;
}
@@ -1003,15 +1009,16 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg,
* stashing the unaligned parts in the as-yet-unused DMA fields.
*/
for_each_sg(sg, s, nents, i) {
- size_t s_iova_off = iova_offset(iovad, s->offset);
+ phys_addr_t s_phys = sg_phys(s);
+ size_t s_iova_off = iova_offset(iovad, s_phys);
size_t s_length = s->length;
size_t pad_len = (mask - iova_len + 1) & mask;
sg_dma_address(s) = s_iova_off;
sg_dma_len(s) = s_length;
- s->offset -= s_iova_off;
s_length = iova_align(iovad, s_length + s_iova_off);
- s->length = s_length;
+ sg_set_page(s, pfn_to_page(PHYS_PFN(s_phys - s_iova_off)),
+ s_length, s->offset & ~s_iova_off);
/*
* Due to the alignment of our single IOVA allocation, we can
--
2.25.1
next prev parent reply other threads:[~2021-10-19 16:38 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-19 16:37 [PATCH v3 0/6] Support IOMMU page sizes larger than the CPU page size Sven Peter
2021-10-19 16:37 ` [PATCH v3 1/6] iommu/dma: Disable get_sgtable for granule > PAGE_SIZE Sven Peter
2021-10-19 16:37 ` Sven Peter [this message]
2021-10-19 16:37 ` [PATCH v3 3/6] iommu/dma: Support granule > PAGE_SIZE allocations Sven Peter
2021-10-19 16:37 ` [PATCH v3 4/6] iommu: Move IOMMU pagesize check to attach_device Sven Peter
2021-10-20 5:21 ` Lu Baolu
2021-10-20 14:22 ` Marc Zyngier
2021-10-21 2:22 ` Lu Baolu
2021-10-21 8:10 ` Marc Zyngier
2021-10-22 2:52 ` Lu Baolu
2021-10-22 8:06 ` Marc Zyngier
2021-10-22 13:39 ` Robin Murphy
2021-10-21 8:31 ` Sven Peter
2021-10-19 16:37 ` [PATCH v3 5/6] iommu: Introduce __IOMMU_DOMAIN_LP Sven Peter
2021-10-19 16:37 ` [PATCH v3 6/6] iommu/dart: Remove force_bypass logic Sven Peter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211019163737.46269-3-sven@svenpeter.dev \
--to=sven@svenpeter.dev \
--cc=alyssa@rosenzweig.io \
--cc=arnd@kernel.org \
--cc=graf@amazon.com \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=maz@kernel.org \
--cc=mohamed.mediouni@caramail.com \
--cc=robin.murphy@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).