From: Christophe Leroy <christophe.leroy@c-s.fr>
To: Alastair D'Silva <alastair@d-silva.org>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH] powerpc: Convert ____flush_dcache_icache_phys() to C
Date: Fri, 16 Aug 2019 15:52:52 +0000 (UTC) [thread overview]
Message-ID: <de7a813c71c4823797bb351bea8be15acae83be2.1565970465.git.christophe.leroy@c-s.fr> (raw)
Resulting code (8xx with 16 bytes per cacheline and 16k pages)
0000016c <__flush_dcache_icache_phys>:
16c: 54 63 00 22 rlwinm r3,r3,0,0,17
170: 7d 20 00 a6 mfmsr r9
174: 39 40 04 00 li r10,1024
178: 55 28 07 34 rlwinm r8,r9,0,28,26
17c: 7c 67 1b 78 mr r7,r3
180: 7d 49 03 a6 mtctr r10
184: 7d 00 01 24 mtmsr r8
188: 4c 00 01 2c isync
18c: 7c 00 18 6c dcbst 0,r3
190: 38 63 00 10 addi r3,r3,16
194: 42 00 ff f8 bdnz 18c <__flush_dcache_icache_phys+0x20>
198: 7c 00 04 ac hwsync
19c: 7d 49 03 a6 mtctr r10
1a0: 7c 00 3f ac icbi 0,r7
1a4: 38 e7 00 10 addi r7,r7,16
1a8: 42 00 ff f8 bdnz 1a0 <__flush_dcache_icache_phys+0x34>
1ac: 7c 00 04 ac hwsync
1b0: 7d 20 01 24 mtmsr r9
1b4: 4c 00 01 2c isync
1b8: 4e 80 00 20 blr
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
This patch is on top of Alastair's series "powerpc: convert cache asm to C"
Patch 3 of that series should touch __flush_dcache_icache_phys and this
patch could come just after patch 3.
arch/powerpc/include/asm/cacheflush.h | 8 +++++
arch/powerpc/mm/mem.c | 55 ++++++++++++++++++++++++++++-------
2 files changed, 53 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/cacheflush.h b/arch/powerpc/include/asm/cacheflush.h
index 1826bf2cc137..bf4f2dc4eb76 100644
--- a/arch/powerpc/include/asm/cacheflush.h
+++ b/arch/powerpc/include/asm/cacheflush.h
@@ -47,6 +47,14 @@ void flush_icache_user_range(struct vm_area_struct *vma,
struct page *page, unsigned long addr,
int len);
void flush_dcache_icache_page(struct page *page);
+#if defined(CONFIG_PPC32) && !defined(CONFIG_BOOKE)
+void __flush_dcache_icache_phys(unsigned long physaddr);
+#else
+static inline void __flush_dcache_icache_phys(unsigned long physaddr)
+{
+ BUG();
+}
+#endif
/**
* flush_dcache_range(): Write any modified data cache blocks out to memory and invalidate them.
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 43be99de7c9a..43009f9227c4 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -402,6 +402,50 @@ void flush_dcache_page(struct page *page)
}
EXPORT_SYMBOL(flush_dcache_page);
+#if defined(CONFIG_PPC32) && !defined(CONFIG_BOOKE)
+void __flush_dcache_icache_phys(unsigned long physaddr)
+{
+ unsigned long bytes = l1_dcache_bytes();
+ unsigned long nb = PAGE_SIZE / bytes;
+ unsigned long addr = physaddr & PAGE_MASK;
+ unsigned long msr, msr0;
+ unsigned long loop1 = addr, loop2 = addr;
+
+ if (cpu_has_feature(CPU_FTR_COHERENT_ICACHE)) {
+ /* For a snooping icache, we still need a dummy icbi to purge all the
+ * prefetched instructions from the ifetch buffers. We also need a sync
+ * before the icbi to order the the actual stores to memory that might
+ * have modified instructions with the icbi.
+ */
+ mb(); /* sync */
+ icbi((void *)addr);
+ mb(); /* sync */
+ isync();
+ return;
+ }
+ msr0 = mfmsr();
+ msr = msr0 & ~MSR_DR;
+ asm volatile(
+ " mtctr %2;"
+ " mtmsr %3;"
+ " isync;"
+ "0: dcbst 0, %0;"
+ " addi %0, %0, %4;"
+ " bdnz 0b;"
+ " sync;"
+ " mtctr %2;"
+ "1: icbi 0, %1;"
+ " addi %1, %1, %4;"
+ " bdnz 1b;"
+ " sync;"
+ " mtmsr %5;"
+ " isync;"
+ : "+r" (loop1), "+r" (loop2)
+ : "r" (nb), "r" (msr), "i" (bytes), "r" (msr0)
+ : "ctr", "memory");
+}
+#endif
+
void flush_dcache_icache_page(struct page *page)
{
#ifdef CONFIG_HUGETLB_PAGE
@@ -419,16 +463,7 @@ void flush_dcache_icache_page(struct page *page)
__flush_dcache_icache(start);
kunmap_atomic(start);
} else {
- unsigned long msr = mfmsr();
-
- /* Clear the DR bit so that we operate on physical
- * rather than virtual addresses
- */
- mtmsr(msr & ~(MSR_DR));
-
- __flush_dcache_icache((void *)physaddr);
-
- mtmsr(msr);
+ __flush_dcache_icache_phys(page_to_pfn(page) << PAGE_SHIFT);
}
#endif
}
--
2.13.3
next reply other threads:[~2019-08-16 15:54 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-16 15:52 Christophe Leroy [this message]
2019-08-20 4:36 ` [RFC PATCH] powerpc: Convert ____flush_dcache_icache_phys() to C Alastair D'Silva
2019-08-21 20:27 ` Christophe Leroy
2019-08-22 0:27 ` Alastair D'Silva
2019-08-22 5:06 ` Christophe Leroy
2019-08-22 5:47 ` Alastair D'Silva
2019-09-02 1:48 ` Michael Ellerman
2019-09-02 11:11 ` Segher Boessenkool
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=de7a813c71c4823797bb351bea8be15acae83be2.1565970465.git.christophe.leroy@c-s.fr \
--to=christophe.leroy@c-s.fr \
--cc=alastair@d-silva.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).