From: David Stevens <stevensd@chromium.org> To: Robin Murphy <robin.murphy@arm.com>, Christoph Hellwig <hch@lst.de> Cc: Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>, Lu Baolu <baolu.lu@linux.intel.com>, Tom Murphy <murphyt7@tcd.ie>, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, David Stevens <stevensd@chromium.org> Subject: [PATCH v5 7/7] dma-iommu: account for min_align_mask Date: Mon, 16 Aug 2021 11:57:55 +0900 [thread overview] Message-ID: <20210816025755.2906695-8-stevensd@google.com> (raw) In-Reply-To: <20210816025755.2906695-1-stevensd@google.com> From: David Stevens <stevensd@chromium.org> For devices which set min_align_mask, swiotlb preserves the offset of the original physical address within that mask. Since __iommu_dma_map accounts for non-aligned addresses, passing a non-aligned swiotlb address with the swiotlb aligned size results in the offset being accounted for twice in the size passed to iommu_map_atomic. The extra page exposed to DMA is also not cleaned up by __iommu_dma_unmap, since that function unmaps with the correct size. This causes mapping failures if the iova gets reused, due to collisions in the iommu page tables. To fix this, pass the original size to __iommu_dma_map, since that function already handles alignment. Additionally, when swiotlb returns non-aligned addresses, there is padding at the start of the bounce buffer that needs to be cleared. Fixes: 1f221a0d0dbf ("swiotlb: respect min_align_mask") Signed-off-by: David Stevens <stevensd@chromium.org> --- drivers/iommu/dma-iommu.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index b1b0327cc2f6..b44b367839c2 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -788,7 +788,6 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, struct iommu_domain *domain = iommu_get_dma_domain(dev); struct iommu_dma_cookie *cookie = domain->iova_cookie; struct iova_domain *iovad = &cookie->iovad; - size_t aligned_size = size; dma_addr_t iova, dma_mask = dma_get_mask(dev); /* @@ -797,8 +796,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, */ if (IS_ENABLED(CONFIG_SWIOTLB) && dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { - void *padding_start; - size_t padding_size; + void *tlb_start; + size_t aligned_size, iova_off, mapping_end_off; aligned_size = iova_align(iovad, size); phys = swiotlb_tbl_map_single(dev, phys, size, aligned_size, @@ -807,24 +806,27 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, if (phys == DMA_MAPPING_ERROR) return DMA_MAPPING_ERROR; - /* Cleanup the padding area. */ - padding_start = phys_to_virt(phys); - padding_size = aligned_size; + iova_off = iova_offset(iovad, phys); + tlb_start = phys_to_virt(phys - iova_off); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) && (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)) { - padding_start += size; - padding_size -= size; + /* Cleanup the padding area. */ + mapping_end_off = iova_off + size; + memset(tlb_start, 0, iova_off); + memset(tlb_start + mapping_end_off, 0, + aligned_size - mapping_end_off); + } else { + /* Nothing was sync'ed, so clear the whole buffer. */ + memset(tlb_start, 0, aligned_size); } - - memset(padding_start, 0, padding_size); } if (!coherent && !(attrs & DMA_ATTR_SKIP_CPU_SYNC)) arch_sync_dma_for_device(phys, size, dir); - iova = __iommu_dma_map(dev, phys, aligned_size, prot, dma_mask); + iova = __iommu_dma_map(dev, phys, size, prot, dma_mask); if (iova == DMA_MAPPING_ERROR && is_swiotlb_buffer(phys)) swiotlb_tbl_unmap_single(dev, phys, size, dir, attrs); return iova; -- 2.33.0.rc1.237.g0d66db33f3-goog
WARNING: multiple messages have this Message-ID (diff)
From: David Stevens <stevensd@chromium.org> To: Robin Murphy <robin.murphy@arm.com>, Christoph Hellwig <hch@lst.de> Cc: linux-kernel@vger.kernel.org, Tom Murphy <murphyt7@tcd.ie>, iommu@lists.linux-foundation.org, David Stevens <stevensd@chromium.org>, Will Deacon <will@kernel.org> Subject: [PATCH v5 7/7] dma-iommu: account for min_align_mask Date: Mon, 16 Aug 2021 11:57:55 +0900 [thread overview] Message-ID: <20210816025755.2906695-8-stevensd@google.com> (raw) In-Reply-To: <20210816025755.2906695-1-stevensd@google.com> From: David Stevens <stevensd@chromium.org> For devices which set min_align_mask, swiotlb preserves the offset of the original physical address within that mask. Since __iommu_dma_map accounts for non-aligned addresses, passing a non-aligned swiotlb address with the swiotlb aligned size results in the offset being accounted for twice in the size passed to iommu_map_atomic. The extra page exposed to DMA is also not cleaned up by __iommu_dma_unmap, since that function unmaps with the correct size. This causes mapping failures if the iova gets reused, due to collisions in the iommu page tables. To fix this, pass the original size to __iommu_dma_map, since that function already handles alignment. Additionally, when swiotlb returns non-aligned addresses, there is padding at the start of the bounce buffer that needs to be cleared. Fixes: 1f221a0d0dbf ("swiotlb: respect min_align_mask") Signed-off-by: David Stevens <stevensd@chromium.org> --- drivers/iommu/dma-iommu.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index b1b0327cc2f6..b44b367839c2 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -788,7 +788,6 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, struct iommu_domain *domain = iommu_get_dma_domain(dev); struct iommu_dma_cookie *cookie = domain->iova_cookie; struct iova_domain *iovad = &cookie->iovad; - size_t aligned_size = size; dma_addr_t iova, dma_mask = dma_get_mask(dev); /* @@ -797,8 +796,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, */ if (IS_ENABLED(CONFIG_SWIOTLB) && dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { - void *padding_start; - size_t padding_size; + void *tlb_start; + size_t aligned_size, iova_off, mapping_end_off; aligned_size = iova_align(iovad, size); phys = swiotlb_tbl_map_single(dev, phys, size, aligned_size, @@ -807,24 +806,27 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, if (phys == DMA_MAPPING_ERROR) return DMA_MAPPING_ERROR; - /* Cleanup the padding area. */ - padding_start = phys_to_virt(phys); - padding_size = aligned_size; + iova_off = iova_offset(iovad, phys); + tlb_start = phys_to_virt(phys - iova_off); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) && (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)) { - padding_start += size; - padding_size -= size; + /* Cleanup the padding area. */ + mapping_end_off = iova_off + size; + memset(tlb_start, 0, iova_off); + memset(tlb_start + mapping_end_off, 0, + aligned_size - mapping_end_off); + } else { + /* Nothing was sync'ed, so clear the whole buffer. */ + memset(tlb_start, 0, aligned_size); } - - memset(padding_start, 0, padding_size); } if (!coherent && !(attrs & DMA_ATTR_SKIP_CPU_SYNC)) arch_sync_dma_for_device(phys, size, dir); - iova = __iommu_dma_map(dev, phys, aligned_size, prot, dma_mask); + iova = __iommu_dma_map(dev, phys, size, prot, dma_mask); if (iova == DMA_MAPPING_ERROR && is_swiotlb_buffer(phys)) swiotlb_tbl_unmap_single(dev, phys, size, dir, attrs); return iova; -- 2.33.0.rc1.237.g0d66db33f3-goog _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-08-16 2:59 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-16 2:57 [PATCH v5 0/7] Fixes for dma-iommu swiotlb bounce buffers David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 2:57 ` [PATCH v5 1/7] dma-iommu: fix sync_sg with swiotlb David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 2:57 ` [PATCH v5 2/7] dma-iommu: fix arch_sync_dma for map David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 12:05 ` Christoph Hellwig 2021-08-16 12:05 ` Christoph Hellwig 2021-08-16 2:57 ` [PATCH v5 3/7] dma-iommu: skip extra sync during unmap w/swiotlb David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 12:05 ` Christoph Hellwig 2021-08-16 12:05 ` Christoph Hellwig 2021-08-16 2:57 ` [PATCH v5 4/7] dma-iommu: fold _swiotlb helpers into callers David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 12:07 ` Christoph Hellwig 2021-08-16 12:07 ` Christoph Hellwig 2021-08-16 2:57 ` [PATCH v5 5/7] dma-iommu: Check CONFIG_SWIOTLB more broadly David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 12:08 ` Christoph Hellwig 2021-08-16 12:08 ` Christoph Hellwig 2021-08-16 2:57 ` [PATCH v5 6/7] swiotlb: support aligned swiotlb buffers David Stevens 2021-08-16 2:57 ` David Stevens 2021-08-16 2:57 ` David Stevens [this message] 2021-08-16 2:57 ` [PATCH v5 7/7] dma-iommu: account for min_align_mask David Stevens
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210816025755.2906695-8-stevensd@google.com \ --to=stevensd@chromium.org \ --cc=baolu.lu@linux.intel.com \ --cc=hch@lst.de \ --cc=iommu@lists.linux-foundation.org \ --cc=joro@8bytes.org \ --cc=linux-kernel@vger.kernel.org \ --cc=murphyt7@tcd.ie \ --cc=robin.murphy@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.