From: Robin Murphy <robin.murphy@arm.com>
To: Arnd Bergmann <arnd@kernel.org>, linux-kernel@vger.kernel.org
Cc: Arnd Bergmann <arnd@arndb.de>, Vineet Gupta <vgupta@kernel.org>,
Russell King <linux@armlinux.org.uk>,
Neil Armstrong <neil.armstrong@linaro.org>,
Linus Walleij <linus.walleij@linaro.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>, Guo Ren <guoren@kernel.org>,
Brian Cain <bcain@quicinc.com>,
Geert Uytterhoeven <geert@linux-m68k.org>,
Michal Simek <monstr@monstr.eu>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Dinh Nguyen <dinguyen@kernel.org>,
Stafford Horne <shorne@gmail.com>, Helge Deller <deller@gmx.de>,
Michael Ellerman <mpe@ellerman.id.au>,
Christophe Leroy <christophe.leroy@csgroup.eu>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Rich Felker <dalias@libc.org>,
John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>,
"David S. Miller" <davem@davemloft.net>,
Max Filippov <jcmvbkbc@gmail.com>, Christoph Hellwig <hch@lst.de>,
Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>,
Conor Dooley <conor.dooley@microchip.com>,
linux-snps-arc@lists.infradead.org,
linux-arm-kernel@lists.infradead.org, linux-oxnas@groups.io,
linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org,
linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org,
linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org,
linux-sh@vger.kernel.org, sparclinux@vger.kernel.org,
linux-xtensa@linux-xtensa.org
Subject: Re: [PATCH 20/21] ARM: dma-mapping: split out arch_dma_mark_clean() helper
Date: Mon, 27 Mar 2023 13:48:35 +0100 [thread overview]
Message-ID: <cb9367fb-0897-244d-15b6-fdfafde2a1c0@arm.com> (raw)
In-Reply-To: <20230327121317.4081816-21-arnd@kernel.org>
On 2023-03-27 13:13, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
>
> The arm version of the arch_sync_dma_for_cpu() function annotates pages as
> PG_dcache_clean after a DMA, but no other architecture does this here. On
> ia64, the same thing is done in arch_sync_dma_for_cpu(), so it makes sense
> to use the same hook in order to have identical arch_sync_dma_for_cpu()
> semantics as all other architectures.
>
> Splitting this out has multiple effects:
>
> - for dma-direct, this now gets called after arch_sync_dma_for_cpu()
> for DMA_FROM_DEVICE mappings, but not for DMA_BIDIRECTIONAL. While
> it would not be harmful to keep doing it for bidirectional mappings,
> those are apparently not used in any callers that care about the flag.
>
> - Since arm has its own dma-iommu abstraction, this now also needs to
> call the same function, so the calls are added there to mirror the
> dma-direct version.
>
> - Like dma-direct, the dma-iommu version now marks the dcache clean
> for both coherent and noncoherent devices after a DMA, but it only
> does this for DMA_FROM_DEVICE, not DMA_BIDIRECTIONAL.
>
> [ HELP NEEDED: can anyone confirm that it is a correct assumption
> on arm that a cache-coherent device writing to a page always results
> in it being in a PG_dcache_clean state like on ia64, or can a device
> write directly into the dcache?]
In AMBA at least, if a snooping write hits in a cache then the data is
most likely going to get routed directly into that cache. If it has
write-back write-allocate attributes it could also land in any cache
along its normal path to RAM; it wouldn't have to go all the way.
Hence all the fun we have where treating a coherent device as
non-coherent can still be almost as broken as the other way round :)
Cheers,
Robin.
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> arch/arm/Kconfig | 1 +
> arch/arm/mm/dma-mapping.c | 71 +++++++++++++++++++++++----------------
> 2 files changed, 43 insertions(+), 29 deletions(-)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index e24a9820e12f..125d58c54ab1 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -7,6 +7,7 @@ config ARM
> select ARCH_HAS_BINFMT_FLAT
> select ARCH_HAS_CURRENT_STACK_POINTER
> select ARCH_HAS_DEBUG_VIRTUAL if MMU
> + select ARCH_HAS_DMA_MARK_CLEAN if MMU
> select ARCH_HAS_DMA_WRITE_COMBINE if !ARM_DMA_MEM_BUFFERABLE
> select ARCH_HAS_ELF_RANDOMIZE
> select ARCH_HAS_FORTIFY_SOURCE
> diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
> index cc702cb27ae7..b703cb83d27e 100644
> --- a/arch/arm/mm/dma-mapping.c
> +++ b/arch/arm/mm/dma-mapping.c
> @@ -665,6 +665,28 @@ static void dma_cache_maint(phys_addr_t paddr,
> } while (left);
> }
>
> +/*
> + * Mark the D-cache clean for these pages to avoid extra flushing.
> + */
> +void arch_dma_mark_clean(phys_addr_t paddr, size_t size)
> +{
> + unsigned long pfn = PFN_UP(paddr);
> + unsigned long off = paddr & (PAGE_SIZE - 1);
> + size_t left = size;
> +
> + if (size < PAGE_SIZE)
> + return;
> +
> + if (off)
> + left -= PAGE_SIZE - off;
> +
> + while (left >= PAGE_SIZE) {
> + struct page *page = pfn_to_page(pfn++);
> + set_bit(PG_dcache_clean, &page->flags);
> + left -= PAGE_SIZE;
> + }
> +}
> +
> static bool arch_sync_dma_cpu_needs_post_dma_flush(void)
> {
> if (IS_ENABLED(CONFIG_CPU_V6) ||
> @@ -715,24 +737,6 @@ void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
> outer_inv_range(paddr, paddr + size);
> dma_cache_maint(paddr, size, dmac_inv_range);
> }
> -
> - /*
> - * Mark the D-cache clean for these pages to avoid extra flushing.
> - */
> - if (dir != DMA_TO_DEVICE && size >= PAGE_SIZE) {
> - unsigned long pfn = PFN_UP(paddr);
> - unsigned long off = paddr & (PAGE_SIZE - 1);
> - size_t left = size;
> -
> - if (off)
> - left -= PAGE_SIZE - off;
> -
> - while (left >= PAGE_SIZE) {
> - struct page *page = pfn_to_page(pfn++);
> - set_bit(PG_dcache_clean, &page->flags);
> - left -= PAGE_SIZE;
> - }
> - }
> }
>
> #ifdef CONFIG_ARM_DMA_USE_IOMMU
> @@ -1294,6 +1298,17 @@ static int arm_iommu_map_sg(struct device *dev, struct scatterlist *sg,
> return -EINVAL;
> }
>
> +static void arm_iommu_sync_dma_for_cpu(phys_addr_t phys, size_t len,
> + enum dma_data_direction dir,
> + bool dma_coherent)
> +{
> + if (!dma_coherent)
> + arch_sync_dma_for_cpu(phys, s->length, dir);
> +
> + if (dir == DMA_FROM_DEVICE)
> + arch_dma_mark_clean(phys, s->length);
> +}
> +
> /**
> * arm_iommu_unmap_sg - unmap a set of SG buffers mapped by dma_map_sg
> * @dev: valid struct device pointer
> @@ -1316,8 +1331,9 @@ static void arm_iommu_unmap_sg(struct device *dev,
> if (sg_dma_len(s))
> __iommu_remove_mapping(dev, sg_dma_address(s),
> sg_dma_len(s));
> - if (!dev->dma_coherent && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> - arch_sync_dma_for_cpu(sg_phys(s), s->length, dir);
> + if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> + arm_iommu_sync_dma_for_cpu(sg_phys(s), s->length, dir,
> + dev->dma_coherent);
> }
> }
>
> @@ -1335,12 +1351,9 @@ static void arm_iommu_sync_sg_for_cpu(struct device *dev,
> struct scatterlist *s;
> int i;
>
> - if (dev->dma_coherent)
> - return;
> -
> for_each_sg(sg, s, nents, i)
> - arch_sync_dma_for_cpu(sg_phys(s), s->length, dir);
> -
> + arm_iommu_sync_dma_for_cpu(sg_phys(s), s->length, dir,
> + dev->dma_coherent);
> }
>
> /**
> @@ -1425,9 +1438,9 @@ static void arm_iommu_unmap_page(struct device *dev, dma_addr_t handle,
> if (!iova)
> return;
>
> - if (!dev->dma_coherent && !(attrs & DMA_ATTR_SKIP_CPU_SYNC)) {
> + if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
> phys = iommu_iova_to_phys(mapping->domain, handle);
> - arch_sync_dma_for_cpu(phys, size, dir);
> + arm_iommu_sync_dma_for_cpu(phys, size, dir, dev->dma_coherent);
> }
>
> iommu_unmap(mapping->domain, iova, len);
> @@ -1497,11 +1510,11 @@ static void arm_iommu_sync_single_for_cpu(struct device *dev,
> struct dma_iommu_mapping *mapping = to_dma_iommu_mapping(dev);
> phys_addr_t phys;
>
> - if (dev->dma_coherent || !(handle & PAGE_MASK))
> + if (!(handle & PAGE_MASK))
> return;
>
> phys = iommu_iova_to_phys(mapping->domain, handle);
> - arch_sync_dma_for_cpu(phys, size, dir);
> + arm_iommu_sync_dma_for_cpu(phys, size, dir, dev->dma_coherent);
> }
>
> static void arm_iommu_sync_single_for_device(struct device *dev,
next prev parent reply other threads:[~2023-03-27 12:48 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-27 12:12 [PATCH 00/21] dma-mapping: unify support for cache flushes Arnd Bergmann
2023-03-27 12:12 ` [PATCH 01/21] openrisc: dma-mapping: flush bidirectional mappings Arnd Bergmann
2023-03-27 12:12 ` [PATCH 02/21] xtensa: dma-mapping: use normal cache invalidation rules Arnd Bergmann
2023-03-27 15:42 ` Max Filippov
2023-03-27 12:12 ` [PATCH 03/21] sparc32: flush caches in dma_sync_*for_device Arnd Bergmann
2023-03-27 12:13 ` [PATCH 04/21] microblaze: dma-mapping: skip extra DMA flushes Arnd Bergmann
2023-03-27 12:13 ` [PATCH 05/21] powerpc: dma-mapping: split out cache operation logic Arnd Bergmann
2023-03-27 12:13 ` [PATCH 06/21] powerpc: dma-mapping: minimize for_cpu flushing Arnd Bergmann
2023-03-27 12:56 ` Christophe Leroy
2023-03-27 13:02 ` Arnd Bergmann
2023-03-27 12:13 ` [PATCH 07/21] powerpc: dma-mapping: always clean cache in _for_device() op Arnd Bergmann
2023-03-27 12:13 ` [PATCH 08/21] riscv: dma-mapping: only invalidate after DMA, not flush Arnd Bergmann
2023-03-29 20:48 ` Conor Dooley
2023-03-30 7:10 ` Arnd Bergmann
2023-03-29 21:51 ` Jessica Clarke
2023-03-30 12:59 ` Lad, Prabhakar
2023-04-19 14:22 ` Palmer Dabbelt
2023-03-27 12:13 ` [PATCH 09/21] riscv: dma-mapping: skip invalidation before bidirectional DMA Arnd Bergmann
2023-03-29 20:16 ` Conor Dooley
2023-03-30 13:26 ` Lad, Prabhakar
2023-04-19 14:22 ` Palmer Dabbelt
2023-05-05 5:47 ` Guo Ren
2023-05-05 13:18 ` Arnd Bergmann
2023-05-06 7:25 ` Guo Ren
2023-05-06 7:53 ` Arnd Bergmann
2023-03-27 12:13 ` [PATCH 10/21] csky: dma-mapping: skip invalidating before DMA from device Arnd Bergmann
2023-03-27 13:37 ` Guo Ren
2023-03-27 12:13 ` [PATCH 11/21] mips: dma-mapping: skip invalidating before bidirectional DMA Arnd Bergmann
2023-03-27 12:13 ` [PATCH 12/21] mips: dma-mapping: split out cache operation logic Arnd Bergmann
2023-03-27 12:13 ` [PATCH 13/21] arc: dma-mapping: skip invalidating before bidirectional DMA Arnd Bergmann
2023-04-02 6:52 ` Vineet Gupta
2023-04-04 8:27 ` Shahab Vahedi
2023-04-06 9:01 ` Shahab Vahedi
2023-03-27 12:13 ` [PATCH 14/21] parisc: dma-mapping: use regular flush/invalidate ops Arnd Bergmann
2023-03-27 12:13 ` [PATCH 15/21] ARM: dma-mapping: always invalidate WT caches before DMA Arnd Bergmann
2023-03-31 9:01 ` Linus Walleij
2023-03-31 9:07 ` Russell King (Oracle)
2023-03-31 9:35 ` Russell King (Oracle)
2023-03-31 10:38 ` Arnd Bergmann
2023-03-31 11:08 ` Russell King (Oracle)
2023-03-31 12:32 ` Arnd Bergmann
2023-03-27 12:13 ` [PATCH 16/21] ARM: dma-mapping: bring back dmac_{clean,inv}_range Arnd Bergmann
2023-03-27 13:10 ` Russell King (Oracle)
2023-03-27 12:13 ` [PATCH 17/21] ARM: dma-mapping: use arch_sync_dma_for_{device,cpu}() internally Arnd Bergmann
2023-03-31 9:10 ` Linus Walleij
2023-03-31 12:48 ` Arnd Bergmann
2023-03-27 12:13 ` [PATCH 18/21] ARM: drop SMP support for ARM11MPCore Arnd Bergmann
2023-03-30 7:48 ` Neil Armstrong
2023-03-30 10:03 ` Arnd Bergmann
2023-03-30 16:40 ` Neil Armstrong
2023-03-30 8:12 ` Linus Walleij
2023-03-30 11:51 ` Ard Biesheuvel
2023-03-31 17:09 ` Catalin Marinas
2023-03-27 12:13 ` [PATCH 19/21] ARM: dma-mapping: use generic form of arch_sync_dma_* helpers Arnd Bergmann
2023-03-27 12:13 ` [PATCH 20/21] ARM: dma-mapping: split out arch_dma_mark_clean() helper Arnd Bergmann
2023-03-27 12:48 ` Robin Murphy [this message]
2023-03-31 14:00 ` Arnd Bergmann
2023-03-31 15:12 ` Robin Murphy
2023-03-31 17:20 ` Arnd Bergmann
2023-03-27 15:01 ` Russell King (Oracle)
2023-03-31 14:06 ` Arnd Bergmann
2023-03-31 15:54 ` Russell King (Oracle)
2023-07-03 7:54 ` Geert Uytterhoeven
2023-07-06 14:11 ` Christoph Hellwig
2023-03-27 12:13 ` [PATCH 21/21] dma-mapping: replace custom code with generic implementation Arnd Bergmann
2023-03-27 22:25 ` Christoph Hellwig
2023-03-31 13:04 ` Arnd Bergmann
2023-03-30 14:06 ` Lad, Prabhakar
2023-04-13 12:13 ` Biju Das
2023-04-13 12:51 ` Arnd Bergmann
2023-06-27 16:52 ` Geert Uytterhoeven
2023-03-31 16:53 ` [PATCH 00/21] dma-mapping: unify support for cache flushes Catalin Marinas
2023-03-31 20:27 ` Arnd Bergmann
2023-05-25 7:46 ` Lad, Prabhakar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cb9367fb-0897-244d-15b6-fdfafde2a1c0@arm.com \
--to=robin.murphy@arm.com \
--cc=arnd@arndb.de \
--cc=arnd@kernel.org \
--cc=bcain@quicinc.com \
--cc=catalin.marinas@arm.com \
--cc=christophe.leroy@csgroup.eu \
--cc=conor.dooley@microchip.com \
--cc=dalias@libc.org \
--cc=davem@davemloft.net \
--cc=deller@gmx.de \
--cc=dinguyen@kernel.org \
--cc=geert@linux-m68k.org \
--cc=glaubitz@physik.fu-berlin.de \
--cc=guoren@kernel.org \
--cc=hch@lst.de \
--cc=jcmvbkbc@gmail.com \
--cc=linus.walleij@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-csky@vger.kernel.org \
--cc=linux-hexagon@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-m68k@lists.linux-m68k.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-openrisc@vger.kernel.org \
--cc=linux-oxnas@groups.io \
--cc=linux-parisc@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-sh@vger.kernel.org \
--cc=linux-snps-arc@lists.infradead.org \
--cc=linux-xtensa@linux-xtensa.org \
--cc=linux@armlinux.org.uk \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=monstr@monstr.eu \
--cc=mpe@ellerman.id.au \
--cc=neil.armstrong@linaro.org \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=prabhakar.mahadev-lad.rj@bp.renesas.com \
--cc=shorne@gmail.com \
--cc=sparclinux@vger.kernel.org \
--cc=tsbogend@alpha.franken.de \
--cc=vgupta@kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).