* [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB
@ 2021-02-03 4:58 Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state Aneesh Kumar K.V
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Aneesh Kumar K.V @ 2021-02-03 4:58 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
THP config results in compound pages. Make sure the kernel enables
the PageCompound() check with CONFIG_HUGETLB_PAGE disabled and
CONFIG_TRANSPARENT_HUGEPAGE enabled.
This makes sure we correctly flush the icache with THP pages.
flush_dcache_icache_page only matter for platforms that don't support
COHERENT_ICACHE.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/hugetlb.h | 2 --
arch/powerpc/mm/hugetlbpage.c | 18 ------------------
arch/powerpc/mm/mem.c | 28 ++++++++++++++++++++++------
3 files changed, 22 insertions(+), 26 deletions(-)
diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h
index 013165e62618..f18c543bc01d 100644
--- a/arch/powerpc/include/asm/hugetlb.h
+++ b/arch/powerpc/include/asm/hugetlb.h
@@ -17,8 +17,6 @@ extern bool hugetlb_disabled;
void hugetlbpage_init_default(void);
-void flush_dcache_icache_hugepage(struct page *page);
-
int slice_is_hugepage_only_range(struct mm_struct *mm, unsigned long addr,
unsigned long len);
diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
index 8b3cc4d688e8..7bdcb93eebae 100644
--- a/arch/powerpc/mm/hugetlbpage.c
+++ b/arch/powerpc/mm/hugetlbpage.c
@@ -663,24 +663,6 @@ static int __init hugetlbpage_init(void)
arch_initcall(hugetlbpage_init);
-void flush_dcache_icache_hugepage(struct page *page)
-{
- int i;
- void *start;
-
- BUG_ON(!PageCompound(page));
-
- for (i = 0; i < compound_nr(page); i++) {
- if (!PageHighMem(page)) {
- __flush_dcache_icache(page_address(page+i));
- } else {
- start = kmap_atomic(page+i);
- __flush_dcache_icache(start);
- kunmap_atomic(start);
- }
- }
-}
-
void __init gigantic_hugetlb_cma_reserve(void)
{
unsigned long order = 0;
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index afab328d0887..ed64ca80d5fd 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -494,14 +494,30 @@ void flush_dcache_page(struct page *page)
}
EXPORT_SYMBOL(flush_dcache_page);
-void flush_dcache_icache_page(struct page *page)
+static void flush_dcache_icache_hugepage(struct page *page)
{
-#ifdef CONFIG_HUGETLB_PAGE
- if (PageCompound(page)) {
- flush_dcache_icache_hugepage(page);
- return;
+ int i;
+ void *start;
+
+ BUG_ON(!PageCompound(page));
+
+ for (i = 0; i < compound_nr(page); i++) {
+ if (!PageHighMem(page)) {
+ __flush_dcache_icache(page_address(page+i));
+ } else {
+ start = kmap_atomic(page+i);
+ __flush_dcache_icache(start);
+ kunmap_atomic(start);
+ }
}
-#endif
+}
+
+void flush_dcache_icache_page(struct page *page)
+{
+
+ if (PageCompound(page))
+ return flush_dcache_icache_hugepage(page);
+
#if defined(CONFIG_PPC_8xx) || defined(CONFIG_PPC64)
/* On 8xx there is no need to kmap since highmem is not supported */
__flush_dcache_icache(page_address(page));
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state
2021-02-03 4:58 [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Aneesh Kumar K.V
@ 2021-02-03 4:58 ` Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 3/3] powerpc/mm: Remove dcache flush from memory remove Aneesh Kumar K.V
2021-02-12 0:19 ` [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Aneesh Kumar K.V @ 2021-02-03 4:58 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
This just add a better name for PG_arch_1. No functional change in this patch.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/cacheflush.h | 6 ++++++
arch/powerpc/include/asm/kvm_ppc.h | 4 ++--
arch/powerpc/mm/book3s64/hash_utils.c | 4 ++--
arch/powerpc/mm/mem.c | 4 ++--
arch/powerpc/mm/pgtable.c | 14 +++++++-------
5 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/arch/powerpc/include/asm/cacheflush.h b/arch/powerpc/include/asm/cacheflush.h
index 138e46d8c04e..f63495109f63 100644
--- a/arch/powerpc/include/asm/cacheflush.h
+++ b/arch/powerpc/include/asm/cacheflush.h
@@ -8,6 +8,12 @@
#include <asm/cputable.h>
#include <asm/cpu_has_feature.h>
+/*
+ * This flag is used to indicate that the page pointed to by a pte is clean
+ * and does not require cleaning before returning it to the user.
+ */
+#define PG_dcache_clean PG_arch_1
+
#ifdef CONFIG_PPC_BOOK3S_64
/*
* Book3s has no ptesync after setting a pte, so without this ptesync it's
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 0a056c64c317..c90d4b128b66 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -881,9 +881,9 @@ static inline void kvmppc_mmu_flush_icache(kvm_pfn_t pfn)
/* Clear i-cache for new pages */
page = pfn_to_page(pfn);
- if (!test_bit(PG_arch_1, &page->flags)) {
+ if (!test_bit(PG_dcache_clean, &page->flags)) {
flush_dcache_icache_page(page);
- set_bit(PG_arch_1, &page->flags);
+ set_bit(PG_dcache_clean, &page->flags);
}
}
diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s64/hash_utils.c
index 73b06adb6eeb..5358c397f6c7 100644
--- a/arch/powerpc/mm/book3s64/hash_utils.c
+++ b/arch/powerpc/mm/book3s64/hash_utils.c
@@ -1143,10 +1143,10 @@ unsigned int hash_page_do_lazy_icache(unsigned int pp, pte_t pte, int trap)
page = pte_page(pte);
/* page is dirty */
- if (!test_bit(PG_arch_1, &page->flags) && !PageReserved(page)) {
+ if (!test_bit(PG_dcache_clean, &page->flags) && !PageReserved(page)) {
if (trap == 0x400) {
flush_dcache_icache_page(page);
- set_bit(PG_arch_1, &page->flags);
+ set_bit(PG_dcache_clean, &page->flags);
} else
pp |= HPTE_R_N;
}
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index ed64ca80d5fd..883e67d37bbc 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -489,8 +489,8 @@ void flush_dcache_page(struct page *page)
if (cpu_has_feature(CPU_FTR_COHERENT_ICACHE))
return;
/* avoid an atomic op if possible */
- if (test_bit(PG_arch_1, &page->flags))
- clear_bit(PG_arch_1, &page->flags);
+ if (test_bit(PG_dcache_clean, &page->flags))
+ clear_bit(PG_dcache_clean, &page->flags);
}
EXPORT_SYMBOL(flush_dcache_page);
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index 15555c95cebc..7d997caccfa5 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -81,9 +81,9 @@ static pte_t set_pte_filter_hash(pte_t pte)
struct page *pg = maybe_pte_to_page(pte);
if (!pg)
return pte;
- if (!test_bit(PG_arch_1, &pg->flags)) {
+ if (!test_bit(PG_dcache_clean, &pg->flags)) {
flush_dcache_icache_page(pg);
- set_bit(PG_arch_1, &pg->flags);
+ set_bit(PG_dcache_clean, &pg->flags);
}
}
return pte;
@@ -116,13 +116,13 @@ static inline pte_t set_pte_filter(pte_t pte)
return pte;
/* If the page clean, we move on */
- if (test_bit(PG_arch_1, &pg->flags))
+ if (test_bit(PG_dcache_clean, &pg->flags))
return pte;
/* If it's an exec fault, we flush the cache and make it clean */
if (is_exec_fault()) {
flush_dcache_icache_page(pg);
- set_bit(PG_arch_1, &pg->flags);
+ set_bit(PG_dcache_clean, &pg->flags);
return pte;
}
@@ -161,12 +161,12 @@ static pte_t set_access_flags_filter(pte_t pte, struct vm_area_struct *vma,
goto bail;
/* If the page is already clean, we move on */
- if (test_bit(PG_arch_1, &pg->flags))
+ if (test_bit(PG_dcache_clean, &pg->flags))
goto bail;
- /* Clean the page and set PG_arch_1 */
+ /* Clean the page and set PG_dcache_clean */
flush_dcache_icache_page(pg);
- set_bit(PG_arch_1, &pg->flags);
+ set_bit(PG_dcache_clean, &pg->flags);
bail:
return pte_mkexec(pte);
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 3/3] powerpc/mm: Remove dcache flush from memory remove.
2021-02-03 4:58 [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state Aneesh Kumar K.V
@ 2021-02-03 4:58 ` Aneesh Kumar K.V
2021-02-12 0:19 ` [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Aneesh Kumar K.V @ 2021-02-03 4:58 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
We added dcache flush on memory add/remove in
commit fb5924fddf9e ("powerpc/mm: Flush cache on memory hot(un)plug")
to handle crashes on GPU hotplug. Instead of adding dcache flush in
generic memory add/remove routine which is used even for regular
memory, we should handle these devices specific flush in the device
driver code.
memtrace did handle this in the driver and that was removed by
commit 7fd6641de28f ("powerpc/powernv/memtrace: Let the arch
hotunplug code flush cache"). This patch reverts that commit.
The dcache flush in memory add was removed by
Fixes: ea458effa88e ("powerpc: Don't flush caches when adding memory")
which I don't think is correct. The reason why we require dcache flush
in memtrace is to make sure we don't have a dirty cache when we remap
a pfn to cache inhibited. We should do that when the memtrace module
removes the memory and make the pfn available for HTM traces to map it
as cache inhibited.
The other device mentioned in
commit fb5924fddf9e ("powerpc/mm: Flush cache on memory hot(un)plug") is
nvlink device with coherent memory. The support for that was removed in
commit 3182215dd0b2 ("powerpc/powernv/npu: Remove NPU DMA ops")
and commit 25b2995a35b6 ("mm: remove MEMORY_DEVICE_PUBLIC support")
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/mm/mem.c | 22 -----------------
arch/powerpc/platforms/powernv/memtrace.c | 29 +++++++++++++++++++++++
2 files changed, 29 insertions(+), 22 deletions(-)
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 883e67d37bbc..4e8ce6d85232 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -91,27 +91,6 @@ int __weak remove_section_mapping(unsigned long start, unsigned long end)
return -ENODEV;
}
-#define FLUSH_CHUNK_SIZE SZ_1G
-/**
- * flush_dcache_range_chunked(): Write any modified data cache blocks out to
- * memory and invalidate them, in chunks of up to FLUSH_CHUNK_SIZE
- * Does not invalidate the corresponding instruction cache blocks.
- *
- * @start: the start address
- * @stop: the stop address (exclusive)
- * @chunk: the max size of the chunks
- */
-static void flush_dcache_range_chunked(unsigned long start, unsigned long stop,
- unsigned long chunk)
-{
- unsigned long i;
-
- for (i = start; i < stop; i += chunk) {
- flush_dcache_range(i, min(stop, i + chunk));
- cond_resched();
- }
-}
-
int __ref arch_create_linear_mapping(int nid, u64 start, u64 size,
struct mhp_params *params)
{
@@ -136,7 +115,6 @@ void __ref arch_remove_linear_mapping(u64 start, u64 size)
/* Remove htab bolted mappings for this section of memory */
start = (unsigned long)__va(start);
- flush_dcache_range_chunked(start, start + size, FLUSH_CHUNK_SIZE);
mutex_lock(&linear_mapping_mutex);
ret = remove_section_mapping(start, start + size);
diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c
index 5fc9408bb0b3..019669eb21d2 100644
--- a/arch/powerpc/platforms/powernv/memtrace.c
+++ b/arch/powerpc/platforms/powernv/memtrace.c
@@ -19,6 +19,7 @@
#include <linux/numa.h>
#include <asm/machdep.h>
#include <asm/debugfs.h>
+#include <asm/cacheflush.h>
/* This enables us to keep track of the memory removed from each node. */
struct memtrace_entry {
@@ -51,6 +52,27 @@ static const struct file_operations memtrace_fops = {
.open = simple_open,
};
+#define FLUSH_CHUNK_SIZE SZ_1G
+/**
+ * flush_dcache_range_chunked(): Write any modified data cache blocks out to
+ * memory and invalidate them, in chunks of up to FLUSH_CHUNK_SIZE
+ * Does not invalidate the corresponding instruction cache blocks.
+ *
+ * @start: the start address
+ * @stop: the stop address (exclusive)
+ * @chunk: the max size of the chunks
+ */
+static void flush_dcache_range_chunked(unsigned long start, unsigned long stop,
+ unsigned long chunk)
+{
+ unsigned long i;
+
+ for (i = start; i < stop; i += chunk) {
+ flush_dcache_range(i, min(stop, i + chunk));
+ cond_resched();
+ }
+}
+
static void memtrace_clear_range(unsigned long start_pfn,
unsigned long nr_pages)
{
@@ -62,6 +84,13 @@ static void memtrace_clear_range(unsigned long start_pfn,
cond_resched();
clear_page(__va(PFN_PHYS(pfn)));
}
+ /*
+ * Before we go ahead and use this range as cache inhibited range
+ * flush the cache.
+ */
+ flush_dcache_range_chunked(PFN_PHYS(start_pfn),
+ PFN_PHYS(start_pfn + nr_pages),
+ FLUSH_CHUNK_SIZE);
}
static u64 memtrace_alloc_node(u32 nid, u64 size)
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB
2021-02-03 4:58 [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 3/3] powerpc/mm: Remove dcache flush from memory remove Aneesh Kumar K.V
@ 2021-02-12 0:19 ` Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Michael Ellerman @ 2021-02-12 0:19 UTC (permalink / raw)
To: Aneesh Kumar K.V, mpe, linuxppc-dev
On Wed, 3 Feb 2021 10:28:10 +0530, Aneesh Kumar K.V wrote:
> THP config results in compound pages. Make sure the kernel enables
> the PageCompound() check with CONFIG_HUGETLB_PAGE disabled and
> CONFIG_TRANSPARENT_HUGEPAGE enabled.
>
> This makes sure we correctly flush the icache with THP pages.
> flush_dcache_icache_page only matter for platforms that don't support
> COHERENT_ICACHE.
Applied to powerpc/next.
[1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB
https://git.kernel.org/powerpc/c/c7ba2d636342093cfb842f47640e5b62192adfed
[2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state
https://git.kernel.org/powerpc/c/ec94b9b23d620d40ab2ced094a30c22bb8d69b9f
[3/3] powerpc/mm: Remove dcache flush from memory remove.
https://git.kernel.org/powerpc/c/2ac02e5ecec0cc2484d60a73b1bc6394aa2fad28
cheers
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-02-12 0:25 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-03 4:58 [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 2/3] powerpc/mm: Add PG_dcache_clean to indicate dcache clean state Aneesh Kumar K.V
2021-02-03 4:58 ` [PATCH 3/3] powerpc/mm: Remove dcache flush from memory remove Aneesh Kumar K.V
2021-02-12 0:19 ` [PATCH 1/3] powerpc/mm: Enable compound page check for both THP and HugeTLB Michael Ellerman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.