* [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64
@ 2021-12-21 3:55 Xiongfeng Wang
2021-12-21 9:17 ` Arnd Bergmann
2021-12-22 11:26 ` Catalin Marinas
0 siblings, 2 replies; 4+ messages in thread
From: Xiongfeng Wang @ 2021-12-21 3:55 UTC (permalink / raw)
To: will, catalin.marinas, mark.rutland, peterz, corbet
Cc: linux-arm-kernel, linux-doc, linux-kernel, moyufeng,
wangxiongfeng2, linux-arch
For memory accesses with write-combining attributes (e.g. those returned
by ioremap_wc()), the CPU may wait for prior accesses to be merged with
subsequent ones. But in some situation, such wait is bad for the
performance.
We introduce io_stop_wc() to prevent the merging of write-combining
memory accesses before this macro with those after it.
We add implementation for ARM64 using DGH instruction and provide NOP
implementation for other architectures.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Will Deacon <will@kernel.org>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
---
v1->v2: change 'Normal-Non Cacheable' to 'write-combining'
---
Documentation/memory-barriers.txt | 8 ++++++++
arch/arm64/include/asm/barrier.h | 9 +++++++++
include/asm-generic/barrier.h | 11 +++++++++++
3 files changed, 28 insertions(+)
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index 7367ada13208..b12df9137e1c 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1950,6 +1950,14 @@ There are some more advanced barrier functions:
For load from persistent memory, existing read memory barriers are sufficient
to ensure read ordering.
+ (*) io_stop_wc();
+
+ For memory accesses with write-combining attributes (e.g. those returned
+ by ioremap_wc(), the CPU may wait for prior accesses to be merged with
+ subsequent ones. io_stop_wc() can be used to prevent the merging of
+ write-combining memory accesses before this macro with those after it when
+ such wait has performance implications.
+
===============================
IMPLICIT KERNEL MEMORY BARRIERS
===============================
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index 1c5a00598458..62217be36217 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -26,6 +26,14 @@
#define __tsb_csync() asm volatile("hint #18" : : : "memory")
#define csdb() asm volatile("hint #20" : : : "memory")
+/*
+ * Data Gathering Hint:
+ * This instruction prevents merging memory accesses with Normal-NC or
+ * Device-GRE attributes before the hint instruction with any memory accesses
+ * appearing after the hint instruction.
+ */
+#define dgh() asm volatile("hint #6" : : : "memory")
+
#ifdef CONFIG_ARM64_PSEUDO_NMI
#define pmr_sync() \
do { \
@@ -46,6 +54,7 @@
#define dma_rmb() dmb(oshld)
#define dma_wmb() dmb(oshst)
+#define io_stop_wc() dgh()
#define tsb_csync() \
do { \
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index 640f09479bdf..4c2c1b830344 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -251,5 +251,16 @@ do { \
#define pmem_wmb() wmb()
#endif
+/*
+ * ioremap_wc() maps I/O memory as memory with write-combining attributes. For
+ * this kind of memory accesses, the CPU may wait for prior accesses to be
+ * merged with subsequent ones. In some situation, such wait is bad for the
+ * performance. io_stop_wc() can be used to prevent the merging of
+ * write-combining memory accesses before this macro with those after it.
+ */
+#ifndef io_stop_wc
+#define io_stop_wc do { } while (0)
+#endif
+
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */
--
2.20.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64
2021-12-21 3:55 [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64 Xiongfeng Wang
@ 2021-12-21 9:17 ` Arnd Bergmann
2021-12-21 11:32 ` Catalin Marinas
2021-12-22 11:26 ` Catalin Marinas
1 sibling, 1 reply; 4+ messages in thread
From: Arnd Bergmann @ 2021-12-21 9:17 UTC (permalink / raw)
To: Xiongfeng Wang
Cc: Will Deacon, Catalin Marinas, Mark Rutland, Peter Zijlstra,
Jonathan Corbet, Linux ARM, open list:DOCUMENTATION,
Linux Kernel Mailing List, Yufeng Mo, linux-arch
On Tue, Dec 21, 2021 at 4:55 AM Xiongfeng Wang
<wangxiongfeng2@huawei.com> wrote:
>
> For memory accesses with write-combining attributes (e.g. those returned
> by ioremap_wc()), the CPU may wait for prior accesses to be merged with
> subsequent ones. But in some situation, such wait is bad for the
> performance.
>
> We introduce io_stop_wc() to prevent the merging of write-combining
> memory accesses before this macro with those after it.
>
> We add implementation for ARM64 using DGH instruction and provide NOP
> implementation for other architectures.
>
> Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
> Suggested-by: Will Deacon <will@kernel.org>
> Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> ---
> v1->v2: change 'Normal-Non Cacheable' to 'write-combining'
For asm-generic:
Acked-by: Arnd Bergmann <arnd@arndb.de>
Will, Catalin: if you are happy with this version, please merge it through the
arm64 tree.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64
2021-12-21 9:17 ` Arnd Bergmann
@ 2021-12-21 11:32 ` Catalin Marinas
0 siblings, 0 replies; 4+ messages in thread
From: Catalin Marinas @ 2021-12-21 11:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Xiongfeng Wang, Will Deacon, Mark Rutland, Peter Zijlstra,
Jonathan Corbet, Linux ARM, open list:DOCUMENTATION,
Linux Kernel Mailing List, Yufeng Mo, linux-arch
On Tue, Dec 21, 2021 at 10:17:27AM +0100, Arnd Bergmann wrote:
> On Tue, Dec 21, 2021 at 4:55 AM Xiongfeng Wang
> <wangxiongfeng2@huawei.com> wrote:
> >
> > For memory accesses with write-combining attributes (e.g. those returned
> > by ioremap_wc()), the CPU may wait for prior accesses to be merged with
> > subsequent ones. But in some situation, such wait is bad for the
> > performance.
> >
> > We introduce io_stop_wc() to prevent the merging of write-combining
> > memory accesses before this macro with those after it.
> >
> > We add implementation for ARM64 using DGH instruction and provide NOP
> > implementation for other architectures.
> >
> > Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
> > Suggested-by: Will Deacon <will@kernel.org>
> > Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> > ---
> > v1->v2: change 'Normal-Non Cacheable' to 'write-combining'
>
> For asm-generic:
>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
>
> Will, Catalin: if you are happy with this version, please merge it through the
> arm64 tree.
Thanks for the ack Arnd. I'll queue this through the arm64 tree.
--
Catalin
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64
2021-12-21 3:55 [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64 Xiongfeng Wang
2021-12-21 9:17 ` Arnd Bergmann
@ 2021-12-22 11:26 ` Catalin Marinas
1 sibling, 0 replies; 4+ messages in thread
From: Catalin Marinas @ 2021-12-22 11:26 UTC (permalink / raw)
To: will, Xiongfeng Wang, corbet, mark.rutland, peterz
Cc: moyufeng, linux-doc, linux-arm-kernel, linux-arch, linux-kernel
On Tue, 21 Dec 2021 11:55:56 +0800, Xiongfeng Wang wrote:
> For memory accesses with write-combining attributes (e.g. those returned
> by ioremap_wc()), the CPU may wait for prior accesses to be merged with
> subsequent ones. But in some situation, such wait is bad for the
> performance.
>
> We introduce io_stop_wc() to prevent the merging of write-combining
> memory accesses before this macro with those after it.
>
> [...]
Applied to arm64 (for-next/misc), thanks!
[1/1] asm-generic: introduce io_stop_wc() and add implementation for ARM64
https://git.kernel.org/arm64/c/d5624bb29f49
--
Catalin
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-12-22 11:26 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-21 3:55 [PATCH v2] asm-generic: introduce io_stop_wc() and add implementation for ARM64 Xiongfeng Wang
2021-12-21 9:17 ` Arnd Bergmann
2021-12-21 11:32 ` Catalin Marinas
2021-12-22 11:26 ` Catalin Marinas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).