* [PATCH v10 00/12] Support page table check PowerPC
@ 2024-03-13 4:21 Rohan McLure
2024-03-13 4:21 ` [PATCH v10 01/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_set" Rohan McLure
` (11 more replies)
0 siblings, 12 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
Support page table check on all PowerPC platforms. This works by
serialising assignments, reassignments and clears of page table
entries at each level in order to ensure that anonymous mappings
have at most one writable consumer, and likewise that file-backed
mappings are not simultaneously also anonymous mappings.
In order to support this infrastructure, a number of stubs must be
defined for all powerpc platforms. Additionally, seperate set_pte_at()
and set_pte_at_unchecked(), to allow for internal, uninstrumented mappings.
v10:
* Revert patches that removed address and mm parameters from page table
check routines, including consuming code from arm64, x86_64 and
riscv.
* Implement *_user_accessible_page() routines in terms of pte_user()
where available (64-bit, book3s) but otherwise by checking the
address (on platforms where the pte does not imply whether the
mapping is for user or kernel)
* Internal set_pte_at() calls replaced with set_pte_at_unchecked(), which
is identical, but prevents double instrumentation.
v9:
* Adapt to using the set_ptes() API, using __set_pte_at() where we need
must avoid instrumentation.
* Use the logic of *_access_permitted() for implementing
*_user_accessible_page(), which are required routines for page table
check.
* Even though we no longer need p{m,u,4}d_leaf(), still default
implement these to assist in refactoring out extant
p{m,u,4}_is_leaf().
* Add p{m,u}_pte() stubs where asm-generic does not provide them, as
page table check wants all *user_accessible_page() variants, and we
would like to default implement the variants in terms of
pte_user_accessible_page().
* Avoid the ugly pmdp_collapse_flush() macro nonsense! Just instrument
its constituent calls instead for radix and hash.
Link: https://lore.kernel.org/linuxppc-dev/20231130025404.37179-2-rmclure@linux.ibm.com/
v8:
* Fix linux/page_table_check.h include in asm/pgtable.h breaking
32-bit.
Link: https://lore.kernel.org/linuxppc-dev/20230215231153.2147454-1-rmclure@linux.ibm.com/
v7:
* Remove use of extern in set_pte prototypes
* Clean up pmdp_collapse_flush macro
* Replace set_pte_at with static inline function
* Fix commit message for patch 7
Link: https://lore.kernel.org/linuxppc-dev/20230215020155.1969194-1-rmclure@linux.ibm.com/
v6:
* Support huge pages and p{m,u}d accounting.
* Remove instrumentation from set_pte from kernel internal pages.
* 64s: Implement pmdp_collapse_flush in terms of __pmdp_collapse_flush
as access to the mm_struct * is required.
Link: https://lore.kernel.org/linuxppc-dev/20230214015939.1853438-1-rmclure@linux.ibm.com/
v5:
Link: https://lore.kernel.org/linuxppc-dev/20221118002146.25979-1-rmclure@linux.ibm.com/
Rohan McLure (12):
Revert "mm/page_table_check: remove unused parameter in
[__]page_table_check_pud_set"
Revert "mm/page_table_check: remove unused parameter in
[__]page_table_check_pmd_set"
mm: Provide addr parameter to page_table_check_pte_set()
Revert "mm/page_table_check: remove unused parameter in
[__]page_table_check_pud_clear"
Revert "mm/page_table_check: remove unused parameter in
[__]page_table_check_pmd_clear"
Revert "mm/page_table_check: remove unused parameter in
[__]page_table_check_pte_clear"
mm: Provide address parameter to p{te,md,ud}_user_accessible_page()
powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf
powerpc: mm: Add common pud_pfn stub for all platforms
poweprc: mm: Implement *_user_accessible_page() for ptes
powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal
usages
powerpc: mm: Support page table check
arch/arm64/include/asm/pgtable.h | 18 ++---
arch/powerpc/Kconfig | 1 +
arch/powerpc/include/asm/book3s/32/pgtable.h | 7 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 74 +++++++++++++++-----
arch/powerpc/include/asm/pgtable.h | 66 ++++++++++-------
arch/powerpc/kvm/book3s_64_mmu_radix.c | 12 ++--
arch/powerpc/mm/book3s64/hash_pgtable.c | 6 +-
arch/powerpc/mm/book3s64/pgtable.c | 17 +++--
arch/powerpc/mm/book3s64/radix_pgtable.c | 25 ++++---
arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +-
arch/powerpc/mm/pgtable.c | 17 ++++-
arch/powerpc/mm/pgtable_32.c | 2 +-
arch/powerpc/mm/pgtable_64.c | 6 +-
arch/powerpc/xmon/xmon.c | 6 +-
arch/riscv/include/asm/pgtable.h | 18 ++---
arch/x86/include/asm/pgtable.h | 20 +++---
include/linux/page_table_check.h | 67 +++++++++++-------
include/linux/pgtable.h | 8 +--
mm/page_table_check.c | 39 ++++++-----
19 files changed, 261 insertions(+), 150 deletions(-)
--
2.44.0
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v10 01/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_set"
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 02/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_set" Rohan McLure
` (10 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
This reverts commit 6d144436d954311f2dbacb5bf7b084042448d83e.
Reinstate previously unused parameters for the purpose of supporting
powerpc platforms, as many do not encode user/kernel ownership of the
page in the pte, but instead in the address of the access.
riscv: Respect change to delete mm, addr parameters from __set_pte_at()
This commit also changed calls to __set_pte_at() to use fewer parameters
on riscv. Keep that change rather than reverting it, as the signature of
__set_pte_at() is changed in a different commit.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 2 +-
arch/riscv/include/asm/pgtable.h | 2 +-
arch/x86/include/asm/pgtable.h | 2 +-
include/linux/page_table_check.h | 11 +++++++----
mm/page_table_check.c | 3 ++-
5 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 79ce70fbb751..a965b59401b3 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -548,7 +548,7 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
static inline void set_pud_at(struct mm_struct *mm, unsigned long addr,
pud_t *pudp, pud_t pud)
{
- page_table_check_pud_set(mm, pudp, pud);
+ page_table_check_pud_set(mm, addr, pudp, pud);
return __set_pte_at(mm, addr, (pte_t *)pudp, pud_pte(pud),
PUD_SIZE >> PAGE_SHIFT);
}
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 6066822e7396..4fc99dd3bb42 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -717,7 +717,7 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
static inline void set_pud_at(struct mm_struct *mm, unsigned long addr,
pud_t *pudp, pud_t pud)
{
- page_table_check_pud_set(mm, pudp, pud);
+ page_table_check_pud_set(mm, addr, pudp, pud);
return __set_pte_at((pte_t *)pudp, pud_pte(pud));
}
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 9d077bca6a10..a9b1e8e6d4b9 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1252,7 +1252,7 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
static inline void set_pud_at(struct mm_struct *mm, unsigned long addr,
pud_t *pudp, pud_t pud)
{
- page_table_check_pud_set(mm, pudp, pud);
+ page_table_check_pud_set(mm, addr, pudp, pud);
native_set_pud(pudp, pud);
}
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 6722941c7cb8..d188428512f5 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -20,7 +20,8 @@ void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte,
unsigned int nr);
void __page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t pmd);
-void __page_table_check_pud_set(struct mm_struct *mm, pud_t *pudp, pud_t pud);
+void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
+ pud_t *pudp, pud_t pud);
void __page_table_check_pte_clear_range(struct mm_struct *mm,
unsigned long addr,
pmd_t pmd);
@@ -83,13 +84,14 @@ static inline void page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp,
__page_table_check_pmd_set(mm, pmdp, pmd);
}
-static inline void page_table_check_pud_set(struct mm_struct *mm, pud_t *pudp,
+static inline void page_table_check_pud_set(struct mm_struct *mm,
+ unsigned long addr, pud_t *pudp,
pud_t pud)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_pud_set(mm, pudp, pud);
+ __page_table_check_pud_set(mm, addr, pudp, pud);
}
static inline void page_table_check_pte_clear_range(struct mm_struct *mm,
@@ -134,7 +136,8 @@ static inline void page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp,
{
}
-static inline void page_table_check_pud_set(struct mm_struct *mm, pud_t *pudp,
+static inline void page_table_check_pud_set(struct mm_struct *mm,
+ unsigned long addr, pud_t *pudp,
pud_t pud)
{
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index af69c3c8f7c2..75167537ebd7 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -210,7 +210,8 @@ void __page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t pmd)
}
EXPORT_SYMBOL(__page_table_check_pmd_set);
-void __page_table_check_pud_set(struct mm_struct *mm, pud_t *pudp, pud_t pud)
+void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
+ pud_t *pudp, pud_t pud)
{
if (&init_mm == mm)
return;
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 02/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_set"
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
2024-03-13 4:21 ` [PATCH v10 01/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_set" Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 03/12] mm: Provide addr parameter to page_table_check_pte_set() Rohan McLure
` (9 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
This reverts commit a3b837130b5865521fa8662aceaa6ebc8d29389a.
Reinstate previously unused parameters for the purpose of supporting
powerpc platforms, as many do not encode user/kernel ownership of the
page in the pte, but instead in the address of the access.
riscv: Respect change to delete mm, addr parameters from __set_pte_at()
This commit also changed calls to __set_pte_at() to use fewer parameters
on riscv. Keep that change rather than reverting it, as the signature of
__set_pte_at() is changed in a different commit.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 4 ++--
arch/riscv/include/asm/pgtable.h | 4 ++--
arch/x86/include/asm/pgtable.h | 4 ++--
include/linux/page_table_check.h | 11 +++++++----
mm/page_table_check.c | 3 ++-
5 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index a965b59401b3..e97c1b7e3ee1 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -540,7 +540,7 @@ static inline void __set_pte_at(struct mm_struct *mm,
static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(mm, pmdp, pmd);
+ page_table_check_pmd_set(mm, addr, pmdp, pmd);
return __set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd),
PMD_SIZE >> PAGE_SHIFT);
}
@@ -1001,7 +1001,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm,
static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(vma->vm_mm, pmdp, pmd);
+ page_table_check_pmd_set(vma->vm_mm, address, pmdp, pmd);
return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd)));
}
#endif
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 4fc99dd3bb42..4e1ef3a77879 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -710,7 +710,7 @@ static inline pmd_t pmd_mkdirty(pmd_t pmd)
static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(mm, pmdp, pmd);
+ page_table_check_pmd_set(mm, addr, pmdp, pmd);
return __set_pte_at((pte_t *)pmdp, pmd_pte(pmd));
}
@@ -781,7 +781,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm,
static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(vma->vm_mm, pmdp, pmd);
+ page_table_check_pmd_set(vma->vm_mm, address, pmdp, pmd);
return __pmd(atomic_long_xchg((atomic_long_t *)pmdp, pmd_val(pmd)));
}
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index a9b1e8e6d4b9..6a7dc2524344 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1245,7 +1245,7 @@ static inline pud_t native_local_pudp_get_and_clear(pud_t *pudp)
static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(mm, pmdp, pmd);
+ page_table_check_pmd_set(mm, addr, pmdp, pmd);
set_pmd(pmdp, pmd);
}
@@ -1390,7 +1390,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm,
static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp, pmd_t pmd)
{
- page_table_check_pmd_set(vma->vm_mm, pmdp, pmd);
+ page_table_check_pmd_set(vma->vm_mm, address, pmdp, pmd);
if (IS_ENABLED(CONFIG_SMP)) {
return xchg(pmdp, pmd);
} else {
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index d188428512f5..5855d690c48a 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -19,7 +19,8 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd);
void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte,
unsigned int nr);
-void __page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t pmd);
+void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd);
void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
pud_t *pudp, pud_t pud);
void __page_table_check_pte_clear_range(struct mm_struct *mm,
@@ -75,13 +76,14 @@ static inline void page_table_check_ptes_set(struct mm_struct *mm,
__page_table_check_ptes_set(mm, ptep, pte, nr);
}
-static inline void page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp,
+static inline void page_table_check_pmd_set(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp,
pmd_t pmd)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_pmd_set(mm, pmdp, pmd);
+ __page_table_check_pmd_set(mm, addr, pmdp, pmd);
}
static inline void page_table_check_pud_set(struct mm_struct *mm,
@@ -131,7 +133,8 @@ static inline void page_table_check_ptes_set(struct mm_struct *mm,
{
}
-static inline void page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp,
+static inline void page_table_check_pmd_set(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp,
pmd_t pmd)
{
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 75167537ebd7..7b9d7b45505d 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -197,7 +197,8 @@ void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte,
}
EXPORT_SYMBOL(__page_table_check_ptes_set);
-void __page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t pmd)
+void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
{
if (&init_mm == mm)
return;
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 03/12] mm: Provide addr parameter to page_table_check_pte_set()
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
2024-03-13 4:21 ` [PATCH v10 01/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_set" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 02/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_set" Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 04/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_clear" Rohan McLure
` (8 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
To provide support for powerpc platforms, provide an address parameter
to the page_table_check_pte_set() routine. This parameter is needed on
some powerpc platforms which do not encode whether a mapping is for
user or kernel in the pte. On such platforms, this can be inferred
from the addr parameter.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 2 +-
arch/riscv/include/asm/pgtable.h | 2 +-
include/linux/page_table_check.h | 12 +++++++-----
include/linux/pgtable.h | 2 +-
mm/page_table_check.c | 4 ++--
5 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index e97c1b7e3ee1..965e35adb206 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -345,7 +345,7 @@ static inline void set_ptes(struct mm_struct *mm,
unsigned long __always_unused addr,
pte_t *ptep, pte_t pte, unsigned int nr)
{
- page_table_check_ptes_set(mm, ptep, pte, nr);
+ page_table_check_ptes_set(mm, addr, ptep, pte, nr);
__sync_cache_and_tags(pte, nr);
for (;;) {
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 4e1ef3a77879..a4b5da7f0704 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -530,7 +530,7 @@ static inline void __set_pte_at(pte_t *ptep, pte_t pteval)
static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pteval, unsigned int nr)
{
- page_table_check_ptes_set(mm, ptep, pteval, nr);
+ page_table_check_ptes_set(mm, addr, ptep, pteval, nr);
for (;;) {
__set_pte_at(ptep, pteval);
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 5855d690c48a..9243c920ed02 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -17,8 +17,8 @@ void __page_table_check_zero(struct page *page, unsigned int order);
void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte);
void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd);
void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud);
-void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte,
- unsigned int nr);
+void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte, unsigned int nr);
void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd);
void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
@@ -68,12 +68,13 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, unsigned int nr)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_ptes_set(mm, ptep, pte, nr);
+ __page_table_check_ptes_set(mm, addr, ptep, pte, nr);
}
static inline void page_table_check_pmd_set(struct mm_struct *mm,
@@ -129,7 +130,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, unsigned int nr)
{
}
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index f6d0e3513948..5da04d056bc3 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -238,7 +238,7 @@ static inline pte_t pte_next_pfn(pte_t pte)
static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte, unsigned int nr)
{
- page_table_check_ptes_set(mm, ptep, pte, nr);
+ page_table_check_ptes_set(mm, addr, ptep, pte, nr);
arch_enter_lazy_mmu_mode();
for (;;) {
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 7b9d7b45505d..3a338fee6d00 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -182,8 +182,8 @@ void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
}
EXPORT_SYMBOL(__page_table_check_pud_clear);
-void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte,
- unsigned int nr)
+void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte, unsigned int nr)
{
unsigned int i;
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 04/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_clear"
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (2 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 03/12] mm: Provide addr parameter to page_table_check_pte_set() Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 05/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_clear" Rohan McLure
` (7 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
This reverts commit 931c38e16499a057e30a3033f4d6a9c242f0f156.
Reinstate previously unused parameters for the purpose of supporting
powerpc platforms, as many do not encode user/kernel ownership of the
page in the pte, but instead in the address of the access.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/x86/include/asm/pgtable.h | 2 +-
include/linux/page_table_check.h | 11 +++++++----
include/linux/pgtable.h | 2 +-
mm/page_table_check.c | 5 +++--
4 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 6a7dc2524344..3e7003b01c9d 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1363,7 +1363,7 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
{
pud_t pud = native_pudp_get_and_clear(pudp);
- page_table_check_pud_clear(mm, pud);
+ page_table_check_pud_clear(mm, addr, pud);
return pud;
}
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 9243c920ed02..d01a00ffc1f9 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -16,7 +16,8 @@ extern struct page_ext_operations page_table_check_ops;
void __page_table_check_zero(struct page *page, unsigned int order);
void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte);
void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd);
-void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud);
+void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
+ pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte, unsigned int nr);
void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
@@ -59,12 +60,13 @@ static inline void page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
__page_table_check_pmd_clear(mm, pmd);
}
-static inline void page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
+static inline void page_table_check_pud_clear(struct mm_struct *mm,
+ unsigned long addr, pud_t pud)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_pud_clear(mm, pud);
+ __page_table_check_pud_clear(mm, addr, pud);
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
@@ -125,7 +127,8 @@ static inline void page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
{
}
-static inline void page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
+static inline void page_table_check_pud_clear(struct mm_struct *mm,
+ unsigned long addr, pud_t pud)
{
}
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 5da04d056bc3..77877ae8abef 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -544,7 +544,7 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
pud_t pud = *pudp;
pud_clear(pudp);
- page_table_check_pud_clear(mm, pud);
+ page_table_check_pud_clear(mm, address, pud);
return pud;
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 3a338fee6d00..a8c8fd7f06f8 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -171,7 +171,8 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
}
EXPORT_SYMBOL(__page_table_check_pmd_clear);
-void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud)
+void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
+ pud_t pud)
{
if (&init_mm == mm)
return;
@@ -217,7 +218,7 @@ void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
if (&init_mm == mm)
return;
- __page_table_check_pud_clear(mm, *pudp);
+ __page_table_check_pud_clear(mm, addr, *pudp);
if (pud_user_accessible_page(pud)) {
page_table_check_set(pud_pfn(pud), PUD_SIZE >> PAGE_SHIFT,
pud_write(pud));
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 05/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_clear"
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (3 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 04/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_clear" Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 06/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pte_clear" Rohan McLure
` (6 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
This reverts commit 1831414cd729a34af937d56ad684a66599de6344.
Reinstate previously unused parameters for the purpose of supporting
powerpc platforms, as many do not encode user/kernel ownership of the
page in the pte, but instead in the address of the access.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 2 +-
arch/riscv/include/asm/pgtable.h | 2 +-
arch/x86/include/asm/pgtable.h | 2 +-
include/linux/page_table_check.h | 11 +++++++----
include/linux/pgtable.h | 2 +-
mm/page_table_check.c | 5 +++--
6 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 965e35adb206..4210d3b071ec 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -965,7 +965,7 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
{
pmd_t pmd = __pmd(xchg_relaxed(&pmd_val(*pmdp), 0));
- page_table_check_pmd_clear(mm, pmd);
+ page_table_check_pmd_clear(mm, address, pmd);
return pmd;
}
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index a4b5da7f0704..cf8e18f27649 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -765,7 +765,7 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
{
pmd_t pmd = __pmd(atomic_long_xchg((atomic_long_t *)pmdp, 0));
- page_table_check_pmd_clear(mm, pmd);
+ page_table_check_pmd_clear(mm, address, pmd);
return pmd;
}
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 3e7003b01c9d..26722f553c43 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1352,7 +1352,7 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long
{
pmd_t pmd = native_pmdp_get_and_clear(pmdp);
- page_table_check_pmd_clear(mm, pmd);
+ page_table_check_pmd_clear(mm, addr, pmd);
return pmd;
}
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index d01a00ffc1f9..0a6ebfa46a31 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -15,7 +15,8 @@ extern struct page_ext_operations page_table_check_ops;
void __page_table_check_zero(struct page *page, unsigned int order);
void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte);
-void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd);
+void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
+ pmd_t pmd);
void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
@@ -52,12 +53,13 @@ static inline void page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
__page_table_check_pte_clear(mm, pte);
}
-static inline void page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
+static inline void page_table_check_pmd_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t pmd)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_pmd_clear(mm, pmd);
+ __page_table_check_pmd_clear(mm, addr, pmd);
}
static inline void page_table_check_pud_clear(struct mm_struct *mm,
@@ -123,7 +125,8 @@ static inline void page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
{
}
-static inline void page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
+static inline void page_table_check_pmd_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t pmd)
{
}
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 77877ae8abef..d0d1a0bbf905 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -531,7 +531,7 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
pmd_t pmd = *pmdp;
pmd_clear(pmdp);
- page_table_check_pmd_clear(mm, pmd);
+ page_table_check_pmd_clear(mm, address, pmd);
return pmd;
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index a8c8fd7f06f8..7afaad9c6e6f 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -160,7 +160,8 @@ void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
}
EXPORT_SYMBOL(__page_table_check_pte_clear);
-void __page_table_check_pmd_clear(struct mm_struct *mm, pmd_t pmd)
+void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
+ pmd_t pmd)
{
if (&init_mm == mm)
return;
@@ -204,7 +205,7 @@ void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
if (&init_mm == mm)
return;
- __page_table_check_pmd_clear(mm, *pmdp);
+ __page_table_check_pmd_clear(mm, addr, *pmdp);
if (pmd_user_accessible_page(pmd)) {
page_table_check_set(pmd_pfn(pmd), PMD_SIZE >> PAGE_SHIFT,
pmd_write(pmd));
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 06/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pte_clear"
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (4 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 05/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_clear" Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 07/12] mm: Provide address parameter to p{te,md,ud}_user_accessible_page() Rohan McLure
` (5 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
This reverts commit aa232204c4689427cefa55fe975692b57291523a.
Reinstate previously unused parameters for the purpose of supporting
powerpc platforms, as many do not encode user/kernel ownership of the
page in the pte, but instead in the address of the access.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 2 +-
arch/riscv/include/asm/pgtable.h | 2 +-
arch/x86/include/asm/pgtable.h | 4 ++--
include/linux/page_table_check.h | 11 +++++++----
include/linux/pgtable.h | 2 +-
mm/page_table_check.c | 7 ++++---
6 files changed, 16 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 4210d3b071ec..74bb81744df2 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -953,7 +953,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
{
pte_t pte = __pte(xchg_relaxed(&pte_val(*ptep), 0));
- page_table_check_pte_clear(mm, pte);
+ page_table_check_pte_clear(mm, address, pte);
return pte;
}
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index cf8e18f27649..2fa6625f591a 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -561,7 +561,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
{
pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0));
- page_table_check_pte_clear(mm, pte);
+ page_table_check_pte_clear(mm, address, pte);
return pte;
}
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 26722f553c43..e8fd625de280 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1283,7 +1283,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep)
{
pte_t pte = native_ptep_get_and_clear(ptep);
- page_table_check_pte_clear(mm, pte);
+ page_table_check_pte_clear(mm, addr, pte);
return pte;
}
@@ -1299,7 +1299,7 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
* care about updates and native needs no locking
*/
pte = native_local_ptep_get_and_clear(ptep);
- page_table_check_pte_clear(mm, pte);
+ page_table_check_pte_clear(mm, addr, pte);
} else {
pte = ptep_get_and_clear(mm, addr, ptep);
}
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 0a6ebfa46a31..48721a4a2b84 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -14,7 +14,8 @@ extern struct static_key_true page_table_check_disabled;
extern struct page_ext_operations page_table_check_ops;
void __page_table_check_zero(struct page *page, unsigned int order);
-void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte);
+void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr,
+ pte_t pte);
void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
pmd_t pmd);
void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
@@ -45,12 +46,13 @@ static inline void page_table_check_free(struct page *page, unsigned int order)
__page_table_check_zero(page, order);
}
-static inline void page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
+static inline void page_table_check_pte_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t pte)
{
if (static_branch_likely(&page_table_check_disabled))
return;
- __page_table_check_pte_clear(mm, pte);
+ __page_table_check_pte_clear(mm, addr, pte);
}
static inline void page_table_check_pmd_clear(struct mm_struct *mm,
@@ -121,7 +123,8 @@ static inline void page_table_check_free(struct page *page, unsigned int order)
{
}
-static inline void page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
+static inline void page_table_check_pte_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t pte)
{
}
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index d0d1a0bbf905..89af325129f2 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -428,7 +428,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
{
pte_t pte = ptep_get(ptep);
pte_clear(mm, address, ptep);
- page_table_check_pte_clear(mm, pte);
+ page_table_check_pte_clear(mm, address, pte);
return pte;
}
#endif
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 7afaad9c6e6f..98cccee74b02 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -149,7 +149,8 @@ void __page_table_check_zero(struct page *page, unsigned int order)
page_ext_put(page_ext);
}
-void __page_table_check_pte_clear(struct mm_struct *mm, pte_t pte)
+void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr,
+ pte_t pte)
{
if (&init_mm == mm)
return;
@@ -193,7 +194,7 @@ void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
return;
for (i = 0; i < nr; i++)
- __page_table_check_pte_clear(mm, ptep_get(ptep + i));
+ __page_table_check_pte_clear(mm, addr, ptep_get(ptep + i));
if (pte_user_accessible_page(pte))
page_table_check_set(pte_pfn(pte), nr, pte_write(pte));
}
@@ -241,7 +242,7 @@ void __page_table_check_pte_clear_range(struct mm_struct *mm,
if (WARN_ON(!ptep))
return;
for (i = 0; i < PTRS_PER_PTE; i++) {
- __page_table_check_pte_clear(mm, ptep_get(ptep));
+ __page_table_check_pte_clear(mm, addr, ptep_get(ptep));
addr += PAGE_SIZE;
ptep++;
}
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 07/12] mm: Provide address parameter to p{te,md,ud}_user_accessible_page()
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (5 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 06/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pte_clear" Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 4:21 ` [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf Rohan McLure
` (4 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
On several powerpc platforms, a page table entry may not imply whether
the relevant mapping is for userspace or kernelspace. Instead, such
platforms infer this by the address which is being accessed.
Add an additional address argument to each of these routines in order to
provide support for page table check on powerpc.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/arm64/include/asm/pgtable.h | 6 +++---
arch/riscv/include/asm/pgtable.h | 6 +++---
arch/x86/include/asm/pgtable.h | 6 +++---
mm/page_table_check.c | 12 ++++++------
4 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 74bb81744df2..8ea22deff9a3 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -874,17 +874,17 @@ static inline int pgd_devmap(pgd_t pgd)
#endif
#ifdef CONFIG_PAGE_TABLE_CHECK
-static inline bool pte_user_accessible_page(pte_t pte)
+static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
{
return pte_present(pte) && (pte_user(pte) || pte_user_exec(pte));
}
-static inline bool pmd_user_accessible_page(pmd_t pmd)
+static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
{
return pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
}
-static inline bool pud_user_accessible_page(pud_t pud)
+static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
{
return pud_leaf(pud) && (pud_user(pud) || pud_user_exec(pud));
}
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 2fa6625f591a..f5c937007590 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -722,17 +722,17 @@ static inline void set_pud_at(struct mm_struct *mm, unsigned long addr,
}
#ifdef CONFIG_PAGE_TABLE_CHECK
-static inline bool pte_user_accessible_page(pte_t pte)
+static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
{
return pte_present(pte) && pte_user(pte);
}
-static inline bool pmd_user_accessible_page(pmd_t pmd)
+static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
{
return pmd_leaf(pmd) && pmd_user(pmd);
}
-static inline bool pud_user_accessible_page(pud_t pud)
+static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
{
return pud_leaf(pud) && pud_user(pud);
}
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index e8fd625de280..514374a27124 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1695,17 +1695,17 @@ static inline bool arch_has_hw_nonleaf_pmd_young(void)
#endif
#ifdef CONFIG_PAGE_TABLE_CHECK
-static inline bool pte_user_accessible_page(pte_t pte)
+static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
{
return (pte_val(pte) & _PAGE_PRESENT) && (pte_val(pte) & _PAGE_USER);
}
-static inline bool pmd_user_accessible_page(pmd_t pmd)
+static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
{
return pmd_leaf(pmd) && (pmd_val(pmd) & _PAGE_PRESENT) && (pmd_val(pmd) & _PAGE_USER);
}
-static inline bool pud_user_accessible_page(pud_t pud)
+static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
{
return pud_leaf(pud) && (pud_val(pud) & _PAGE_PRESENT) && (pud_val(pud) & _PAGE_USER);
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 98cccee74b02..aa5e16c8328e 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -155,7 +155,7 @@ void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr,
if (&init_mm == mm)
return;
- if (pte_user_accessible_page(pte)) {
+ if (pte_user_accessible_page(pte, addr)) {
page_table_check_clear(pte_pfn(pte), PAGE_SIZE >> PAGE_SHIFT);
}
}
@@ -167,7 +167,7 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
if (&init_mm == mm)
return;
- if (pmd_user_accessible_page(pmd)) {
+ if (pmd_user_accessible_page(pmd, addr)) {
page_table_check_clear(pmd_pfn(pmd), PMD_SIZE >> PAGE_SHIFT);
}
}
@@ -179,7 +179,7 @@ void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
if (&init_mm == mm)
return;
- if (pud_user_accessible_page(pud)) {
+ if (pud_user_accessible_page(pud, addr)) {
page_table_check_clear(pud_pfn(pud), PUD_SIZE >> PAGE_SHIFT);
}
}
@@ -195,7 +195,7 @@ void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
for (i = 0; i < nr; i++)
__page_table_check_pte_clear(mm, addr, ptep_get(ptep + i));
- if (pte_user_accessible_page(pte))
+ if (pte_user_accessible_page(pte, addr))
page_table_check_set(pte_pfn(pte), nr, pte_write(pte));
}
EXPORT_SYMBOL(__page_table_check_ptes_set);
@@ -207,7 +207,7 @@ void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr,
return;
__page_table_check_pmd_clear(mm, addr, *pmdp);
- if (pmd_user_accessible_page(pmd)) {
+ if (pmd_user_accessible_page(pmd, addr)) {
page_table_check_set(pmd_pfn(pmd), PMD_SIZE >> PAGE_SHIFT,
pmd_write(pmd));
}
@@ -221,7 +221,7 @@ void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr,
return;
__page_table_check_pud_clear(mm, addr, *pudp);
- if (pud_user_accessible_page(pud)) {
+ if (pud_user_accessible_page(pud, addr)) {
page_table_check_set(pud_pfn(pud), PUD_SIZE >> PAGE_SHIFT,
pud_write(pud));
}
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (6 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 07/12] mm: Provide address parameter to p{te,md,ud}_user_accessible_page() Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 10:33 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms Rohan McLure
` (3 subsequent siblings)
11 siblings, 1 reply; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
Replace occurrences of p{u,m,4}d_is_leaf with p{u,m,4}_leaf, as the
latter is the name given to checking that a higher-level entry in
multi-level paging contains a page translation entry (pte) throughout
all other archs.
Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
v9: No longer required in order to implement page table check, just a
refactor.
v10: Fix more occurances, and just delete p{u,m,4}_is_leaf() stubs as
equivalent p{u,m,4}_leaf() stubs already exist.
---
arch/powerpc/include/asm/book3s/64/pgtable.h | 10 ++++----
arch/powerpc/include/asm/pgtable.h | 24 --------------------
arch/powerpc/kvm/book3s_64_mmu_radix.c | 12 +++++-----
arch/powerpc/mm/book3s64/radix_pgtable.c | 14 ++++++------
arch/powerpc/mm/pgtable.c | 6 ++---
arch/powerpc/mm/pgtable_64.c | 6 ++---
arch/powerpc/xmon/xmon.c | 6 ++---
7 files changed, 26 insertions(+), 52 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index 62c43d3d80ec..382724c5e872 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -1443,16 +1443,14 @@ static inline bool is_pte_rw_upgrade(unsigned long old_val, unsigned long new_va
/*
* Like pmd_huge() and pmd_large(), but works regardless of config options
*/
-#define pmd_is_leaf pmd_is_leaf
-#define pmd_leaf pmd_is_leaf
-static inline bool pmd_is_leaf(pmd_t pmd)
+#define pmd_leaf pmd_leaf
+static inline bool pmd_leaf(pmd_t pmd)
{
return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE));
}
-#define pud_is_leaf pud_is_leaf
-#define pud_leaf pud_is_leaf
-static inline bool pud_is_leaf(pud_t pud)
+#define pud_leaf pud_leaf
+static inline bool pud_leaf(pud_t pud)
{
return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE));
}
diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
index 9224f23065ff..0c0ffbe7a3b5 100644
--- a/arch/powerpc/include/asm/pgtable.h
+++ b/arch/powerpc/include/asm/pgtable.h
@@ -180,30 +180,6 @@ static inline void pte_frag_set(mm_context_t *ctx, void *p)
}
#endif
-#ifndef pmd_is_leaf
-#define pmd_is_leaf pmd_is_leaf
-static inline bool pmd_is_leaf(pmd_t pmd)
-{
- return false;
-}
-#endif
-
-#ifndef pud_is_leaf
-#define pud_is_leaf pud_is_leaf
-static inline bool pud_is_leaf(pud_t pud)
-{
- return false;
-}
-#endif
-
-#ifndef p4d_is_leaf
-#define p4d_is_leaf p4d_is_leaf
-static inline bool p4d_is_leaf(p4d_t p4d)
-{
- return false;
-}
-#endif
-
#define pmd_pgtable pmd_pgtable
static inline pgtable_t pmd_pgtable(pmd_t pmd)
{
diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
index 4a1abb9f7c05..408d98f8a514 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -503,7 +503,7 @@ static void kvmppc_unmap_free_pmd(struct kvm *kvm, pmd_t *pmd, bool full,
for (im = 0; im < PTRS_PER_PMD; ++im, ++p) {
if (!pmd_present(*p))
continue;
- if (pmd_is_leaf(*p)) {
+ if (pmd_leaf(*p)) {
if (full) {
pmd_clear(p);
} else {
@@ -532,7 +532,7 @@ static void kvmppc_unmap_free_pud(struct kvm *kvm, pud_t *pud,
for (iu = 0; iu < PTRS_PER_PUD; ++iu, ++p) {
if (!pud_present(*p))
continue;
- if (pud_is_leaf(*p)) {
+ if (pud_leaf(*p)) {
pud_clear(p);
} else {
pmd_t *pmd;
@@ -635,12 +635,12 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
new_pud = pud_alloc_one(kvm->mm, gpa);
pmd = NULL;
- if (pud && pud_present(*pud) && !pud_is_leaf(*pud))
+ if (pud && pud_present(*pud) && !pud_leaf(*pud))
pmd = pmd_offset(pud, gpa);
else if (level <= 1)
new_pmd = kvmppc_pmd_alloc();
- if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_is_leaf(*pmd)))
+ if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_leaf(*pmd)))
new_ptep = kvmppc_pte_alloc();
/* Check if we might have been invalidated; let the guest retry if so */
@@ -658,7 +658,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
new_pud = NULL;
}
pud = pud_offset(p4d, gpa);
- if (pud_is_leaf(*pud)) {
+ if (pud_leaf(*pud)) {
unsigned long hgpa = gpa & PUD_MASK;
/* Check if we raced and someone else has set the same thing */
@@ -709,7 +709,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
new_pmd = NULL;
}
pmd = pmd_offset(pud, gpa);
- if (pmd_is_leaf(*pmd)) {
+ if (pmd_leaf(*pmd)) {
unsigned long lgpa = gpa & PMD_MASK;
/* Check if we raced and someone else has set the same thing */
diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index e16e2fd104c5..46fa46ce6526 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -204,14 +204,14 @@ static void radix__change_memory_range(unsigned long start, unsigned long end,
pudp = pud_alloc(&init_mm, p4dp, idx);
if (!pudp)
continue;
- if (pud_is_leaf(*pudp)) {
+ if (pud_leaf(*pudp)) {
ptep = (pte_t *)pudp;
goto update_the_pte;
}
pmdp = pmd_alloc(&init_mm, pudp, idx);
if (!pmdp)
continue;
- if (pmd_is_leaf(*pmdp)) {
+ if (pmd_leaf(*pmdp)) {
ptep = pmdp_ptep(pmdp);
goto update_the_pte;
}
@@ -767,7 +767,7 @@ static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr,
if (!pmd_present(*pmd))
continue;
- if (pmd_is_leaf(*pmd)) {
+ if (pmd_leaf(*pmd)) {
if (IS_ALIGNED(addr, PMD_SIZE) &&
IS_ALIGNED(next, PMD_SIZE)) {
if (!direct)
@@ -807,7 +807,7 @@ static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr,
if (!pud_present(*pud))
continue;
- if (pud_is_leaf(*pud)) {
+ if (pud_leaf(*pud)) {
if (!IS_ALIGNED(addr, PUD_SIZE) ||
!IS_ALIGNED(next, PUD_SIZE)) {
WARN_ONCE(1, "%s: unaligned range\n", __func__);
@@ -845,7 +845,7 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct,
if (!p4d_present(*p4d))
continue;
- if (p4d_is_leaf(*p4d)) {
+ if (p4d_leaf(*p4d)) {
if (!IS_ALIGNED(addr, P4D_SIZE) ||
!IS_ALIGNED(next, P4D_SIZE)) {
WARN_ONCE(1, "%s: unaligned range\n", __func__);
@@ -1540,7 +1540,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
int pud_clear_huge(pud_t *pud)
{
- if (pud_is_leaf(*pud)) {
+ if (pud_leaf(*pud)) {
pud_clear(pud);
return 1;
}
@@ -1587,7 +1587,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
int pmd_clear_huge(pmd_t *pmd)
{
- if (pmd_is_leaf(*pmd)) {
+ if (pmd_leaf(*pmd)) {
pmd_clear(pmd);
return 1;
}
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index a04ae4449a02..e8e0289d7ab0 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -413,7 +413,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
if (p4d_none(p4d))
return NULL;
- if (p4d_is_leaf(p4d)) {
+ if (p4d_leaf(p4d)) {
ret_pte = (pte_t *)p4dp;
goto out;
}
@@ -435,7 +435,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
if (pud_none(pud))
return NULL;
- if (pud_is_leaf(pud)) {
+ if (pud_leaf(pud)) {
ret_pte = (pte_t *)pudp;
goto out;
}
@@ -474,7 +474,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
goto out;
}
- if (pmd_is_leaf(pmd)) {
+ if (pmd_leaf(pmd)) {
ret_pte = (pte_t *)pmdp;
goto out;
}
diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
index 5ac1fd30341b..0604c80dae66 100644
--- a/arch/powerpc/mm/pgtable_64.c
+++ b/arch/powerpc/mm/pgtable_64.c
@@ -100,7 +100,7 @@ EXPORT_SYMBOL(__pte_frag_size_shift);
/* 4 level page table */
struct page *p4d_page(p4d_t p4d)
{
- if (p4d_is_leaf(p4d)) {
+ if (p4d_leaf(p4d)) {
if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
VM_WARN_ON(!p4d_huge(p4d));
return pte_page(p4d_pte(p4d));
@@ -111,7 +111,7 @@ struct page *p4d_page(p4d_t p4d)
struct page *pud_page(pud_t pud)
{
- if (pud_is_leaf(pud)) {
+ if (pud_leaf(pud)) {
if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
VM_WARN_ON(!pud_huge(pud));
return pte_page(pud_pte(pud));
@@ -125,7 +125,7 @@ struct page *pud_page(pud_t pud)
*/
struct page *pmd_page(pmd_t pmd)
{
- if (pmd_is_leaf(pmd)) {
+ if (pmd_leaf(pmd)) {
/*
* vmalloc_to_page may be called on any vmap address (not only
* vmalloc), and it uses pmd_page() etc., when huge vmap is
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index c85fa3f0dd3b..d79d6633f333 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -3340,7 +3340,7 @@ static void show_pte(unsigned long addr)
return;
}
- if (p4d_is_leaf(*p4dp)) {
+ if (p4d_leaf(*p4dp)) {
format_pte(p4dp, p4d_val(*p4dp));
return;
}
@@ -3354,7 +3354,7 @@ static void show_pte(unsigned long addr)
return;
}
- if (pud_is_leaf(*pudp)) {
+ if (pud_leaf(*pudp)) {
format_pte(pudp, pud_val(*pudp));
return;
}
@@ -3368,7 +3368,7 @@ static void show_pte(unsigned long addr)
return;
}
- if (pmd_is_leaf(*pmdp)) {
+ if (pmd_leaf(*pmdp)) {
format_pte(pmdp, pmd_val(*pmdp));
return;
}
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (7 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 11:08 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes Rohan McLure
` (2 subsequent siblings)
11 siblings, 1 reply; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
Prior to this commit, pud_pfn was implemented with BUILD_BUG as the inline
function for 64-bit Book3S systems but is never included, as its
invocations in generic code are guarded by calls to pud_devmap which return
zero on such systems. A future patch will provide support for page table
checks, the generic code for which depends on a pud_pfn stub being
implemented, even while the patch will not interact with puds directly.
Remove the 64-bit Book3S stub and define pud_pfn to warn on all
platforms. pud_pfn may be defined properly on a per-platform basis
should it grow real usages in future.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
arch/powerpc/include/asm/pgtable.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
index 0c0ffbe7a3b5..13f661831333 100644
--- a/arch/powerpc/include/asm/pgtable.h
+++ b/arch/powerpc/include/asm/pgtable.h
@@ -213,6 +213,20 @@ static inline bool arch_supports_memmap_on_memory(unsigned long vmemmap_size)
#endif /* CONFIG_PPC64 */
+/*
+ * Currently only consumed by page_table_check_pud_{set,clear}. Since clears
+ * and sets to page table entries at any level are done through
+ * page_table_check_pte_{set,clear}, provide stub implementation.
+ */
+#ifndef pud_pfn
+#define pud_pfn pud_pfn
+static inline int pud_pfn(pud_t pud)
+{
+ WARN_ONCE(1, "pud: platform does not use pud entries directly");
+ return 0;
+}
+#endif
+
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_PGTABLE_H */
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (8 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 11:19 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages Rohan McLure
2024-03-13 4:21 ` [PATCH v10 12/12] powerpc: mm: Support page table check Rohan McLure
11 siblings, 1 reply; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
Page table checking depends on architectures providing an
implementation of p{te,md,ud}_user_accessible_page. With
refactorisations made on powerpc/mm, the pte_access_permitted() and
similar methods verify whether a userland page is accessible with the
required permissions.
Since page table checking is the only user of
p{te,md,ud}_user_accessible_page(), implement these for all platforms,
using some of the same preliminay checks taken by pte_access_permitted()
on that platform.
Since Commit 8e9bd41e4ce1 ("powerpc/nohash: Replace pte_user() by pte_read()")
pte_user() is no longer required to be present on all platforms as it
may be equivalent to or implied by pte_read(). Hence implementations are
specialised.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
v9: New implementation
v10: Let book3s/64 use pte_user(), but otherwise default other platforms
to using the address provided with the call to infer whether it is a
user page or not. pmd/pud variants will warn on all other platforms, as
they should not be used for user page mappings
---
arch/powerpc/include/asm/book3s/64/pgtable.h | 19 ++++++++++++++
arch/powerpc/include/asm/pgtable.h | 26 ++++++++++++++++++++
2 files changed, 45 insertions(+)
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index 382724c5e872..ca765331e21d 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -538,6 +538,12 @@ static inline bool pte_access_permitted(pte_t pte, bool write)
return arch_pte_access_permitted(pte_val(pte), write, 0);
}
+#define pte_user_accessible_page pte_user_accessible_page
+static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
+{
+ return pte_present(pte) && pte_user(pte);
+}
+
/*
* Conversion functions: convert a page and protection to a page entry,
* and a page entry and page directory to the page they refer to.
@@ -881,6 +887,7 @@ static inline int pud_present(pud_t pud)
extern struct page *pud_page(pud_t pud);
extern struct page *pmd_page(pmd_t pmd);
+
static inline pte_t pud_pte(pud_t pud)
{
return __pte_raw(pud_raw(pud));
@@ -926,6 +933,12 @@ static inline bool pud_access_permitted(pud_t pud, bool write)
return pte_access_permitted(pud_pte(pud), write);
}
+#define pud_user_accessible_page pud_user_accessible_page
+static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
+{
+ return pte_user_accessible_page(pud_pte(pud), addr);
+}
+
#define __p4d_raw(x) ((p4d_t) { __pgd_raw(x) })
static inline __be64 p4d_raw(p4d_t x)
{
@@ -1091,6 +1104,12 @@ static inline bool pmd_access_permitted(pmd_t pmd, bool write)
return pte_access_permitted(pmd_pte(pmd), write);
}
+#define pmd_user_accessible_page pmd_user_accessible_page
+static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
+{
+ return pte_user_accessible_page(pmd_pte(pmd), addr);
+}
+
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot);
extern pud_t pfn_pud(unsigned long pfn, pgprot_t pgprot);
diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
index 13f661831333..3741a63fb82e 100644
--- a/arch/powerpc/include/asm/pgtable.h
+++ b/arch/powerpc/include/asm/pgtable.h
@@ -227,6 +227,32 @@ static inline int pud_pfn(pud_t pud)
}
#endif
+#ifndef pte_user_accessible_page
+#define pte_user_accessible_page pte_user_accessible_page
+static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
+{
+ return pte_present(pte) && !is_kernel_addr(addr);
+}
+#endif
+
+#ifndef pmd_user_accessible_page
+#define pmd_user_accessible_page pmd_user_accessible_page
+static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
+{
+ WARN_ONCE(1, "pmd: platform does not use pmd entries directly");
+ return false;
+}
+#endif
+
+#ifndef pud_user_accessible_page
+#define pud_user_accessible_page pud_user_accessible_page
+static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
+{
+ WARN_ONCE(1, "pud: platform does not use pud entries directly");
+ return false;
+}
+#endif
+
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_PGTABLE_H */
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (9 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
2024-03-13 11:30 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 12/12] powerpc: mm: Support page table check Rohan McLure
11 siblings, 1 reply; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
In the new set_ptes() API, set_pte_at() (a special case of set_ptes())
is intended to be instrumented by the page table check facility. There
are however several other routines that constitute the API for setting
page table entries, including set_pmd_at() among others. Such routines
are themselves implemented in terms of set_ptes_at().
A future patch providing support for page table checking on powerpc
must take care to avoid duplicate calls to
page_table_check_p{te,md,ud}_set(). Allow for assignment of pte entries
without instrumentation through the set_pte_at_unchecked() routine
introduced in this patch.
Cause API-facing routines that call set_pte_at() to instead call
set_pte_at_unchecked(), which will remain uninstrumented by page
table check. set_ptes() is itself implemented by calls to
__set_pte_at(), so this eliminates redundant code.
Also prefer set_pte_at_unchecked() in early-boot usages which should not be
instrumented.
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
v9: New patch
v10: don't reuse __set_pte_at(), as that will not apply filters. Instead
use new set_pte_at_unchecked().
---
arch/powerpc/include/asm/pgtable.h | 2 ++
arch/powerpc/mm/book3s64/hash_pgtable.c | 2 +-
arch/powerpc/mm/book3s64/pgtable.c | 6 +++---
arch/powerpc/mm/book3s64/radix_pgtable.c | 8 ++++----
arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +-
arch/powerpc/mm/pgtable.c | 7 +++++++
arch/powerpc/mm/pgtable_32.c | 2 +-
7 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
index 3741a63fb82e..6ff1d8cfa216 100644
--- a/arch/powerpc/include/asm/pgtable.h
+++ b/arch/powerpc/include/asm/pgtable.h
@@ -44,6 +44,8 @@ struct mm_struct;
void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
pte_t pte, unsigned int nr);
#define set_ptes set_ptes
+void set_pte_at_unchecked(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte);
#define update_mmu_cache(vma, addr, ptep) \
update_mmu_cache_range(NULL, vma, addr, ptep, 1)
diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c
index 988948d69bc1..871472f99a01 100644
--- a/arch/powerpc/mm/book3s64/hash_pgtable.c
+++ b/arch/powerpc/mm/book3s64/hash_pgtable.c
@@ -165,7 +165,7 @@ int hash__map_kernel_page(unsigned long ea, unsigned long pa, pgprot_t prot)
ptep = pte_alloc_kernel(pmdp, ea);
if (!ptep)
return -ENOMEM;
- set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
+ set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
} else {
/*
* If the mm subsystem is not fully up, we cannot create a
diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c
index 3438ab72c346..25082ab6018b 100644
--- a/arch/powerpc/mm/book3s64/pgtable.c
+++ b/arch/powerpc/mm/book3s64/pgtable.c
@@ -116,7 +116,7 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
WARN_ON(!(pmd_large(pmd)));
#endif
trace_hugepage_set_pmd(addr, pmd_val(pmd));
- return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
+ return set_pte_at_unchecked(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
}
void set_pud_at(struct mm_struct *mm, unsigned long addr,
@@ -133,7 +133,7 @@ void set_pud_at(struct mm_struct *mm, unsigned long addr,
WARN_ON(!(pud_large(pud)));
#endif
trace_hugepage_set_pud(addr, pud_val(pud));
- return set_pte_at(mm, addr, pudp_ptep(pudp), pud_pte(pud));
+ return set_pte_at_unchecked(mm, addr, pudp_ptep(pudp), pud_pte(pud));
}
static void do_serialize(void *arg)
@@ -539,7 +539,7 @@ void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr,
if (radix_enabled())
return radix__ptep_modify_prot_commit(vma, addr,
ptep, old_pte, pte);
- set_pte_at(vma->vm_mm, addr, ptep, pte);
+ set_pte_at_unchecked(vma->vm_mm, addr, ptep, pte);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index 46fa46ce6526..c661e42bb2f1 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -109,7 +109,7 @@ static int early_map_kernel_page(unsigned long ea, unsigned long pa,
ptep = pte_offset_kernel(pmdp, ea);
set_the_pte:
- set_pte_at(&init_mm, ea, ptep, pfn_pte(pfn, flags));
+ set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pfn, flags));
asm volatile("ptesync": : :"memory");
return 0;
}
@@ -1522,7 +1522,7 @@ void radix__ptep_modify_prot_commit(struct vm_area_struct *vma,
(atomic_read(&mm->context.copros) > 0))
radix__flush_tlb_page(vma, addr);
- set_pte_at(mm, addr, ptep, pte);
+ set_pte_at_unchecked(mm, addr, ptep, pte);
}
int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
@@ -1533,7 +1533,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
if (!radix_enabled())
return 0;
- set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pud);
+ set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep, new_pud);
return 1;
}
@@ -1580,7 +1580,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
if (!radix_enabled())
return 0;
- set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pmd);
+ set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep, new_pmd);
return 1;
}
diff --git a/arch/powerpc/mm/nohash/book3e_pgtable.c b/arch/powerpc/mm/nohash/book3e_pgtable.c
index 1c5e4ecbebeb..10d487b2b991 100644
--- a/arch/powerpc/mm/nohash/book3e_pgtable.c
+++ b/arch/powerpc/mm/nohash/book3e_pgtable.c
@@ -111,7 +111,7 @@ int __ref map_kernel_page(unsigned long ea, phys_addr_t pa, pgprot_t prot)
}
ptep = pte_offset_kernel(pmdp, ea);
}
- set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
+ set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
smp_wmb();
return 0;
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index e8e0289d7ab0..352679cf2684 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -227,6 +227,13 @@ void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
}
}
+void set_pte_at_unchecked(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte)
+{
+ pte = set_pte_filter(pte, addr);
+ __set_pte_at(mm, addr, ptep, pte, 0);
+}
+
void unmap_kernel_page(unsigned long va)
{
pmd_t *pmdp = pmd_off_k(va);
diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
index 4be97b4a44f9..a5a26faf91ec 100644
--- a/arch/powerpc/mm/pgtable_32.c
+++ b/arch/powerpc/mm/pgtable_32.c
@@ -89,7 +89,7 @@ int __ref map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot)
* hash table
*/
BUG_ON((pte_present(*pg) | pte_hashpte(*pg)) && pgprot_val(prot));
- set_pte_at(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT, prot));
+ set_pte_at_unchecked(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT, prot));
}
smp_wmb();
return err;
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v10 12/12] powerpc: mm: Support page table check
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
` (10 preceding siblings ...)
2024-03-13 4:21 ` [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages Rohan McLure
@ 2024-03-13 4:21 ` Rohan McLure
11 siblings, 0 replies; 19+ messages in thread
From: Rohan McLure @ 2024-03-13 4:21 UTC (permalink / raw)
To: linuxppc-dev; +Cc: x86, linux-mm, Rohan McLure, linux-riscv, linux-arm-kernel
On creation and clearing of a page table mapping, instrument such calls
by invoking page_table_check_pte_set and page_table_check_pte_clear
respectively. These calls serve as a sanity check against illegal
mappings.
Enable ARCH_SUPPORTS_PAGE_TABLE_CHECK for all platforms.
See also:
riscv support in commit 3fee229a8eb9 ("riscv/mm: enable
ARCH_SUPPORTS_PAGE_TABLE_CHECK")
arm64 in commit 42b2547137f5 ("arm64/mm: enable
ARCH_SUPPORTS_PAGE_TABLE_CHECK")
x86_64 in commit d283d422c6c4 ("x86: mm: add x86_64 support for page table
check")
Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
---
v9: Updated for new API. Instrument pmdp_collapse_flush's two
constituent calls to avoid header hell
v10: Cause p{u,m}dp_huge_get_and_clear() to resemble one another
---
arch/powerpc/Kconfig | 1 +
arch/powerpc/include/asm/book3s/32/pgtable.h | 7 ++-
arch/powerpc/include/asm/book3s/64/pgtable.h | 45 +++++++++++++++-----
arch/powerpc/mm/book3s64/hash_pgtable.c | 4 ++
arch/powerpc/mm/book3s64/pgtable.c | 11 +++--
arch/powerpc/mm/book3s64/radix_pgtable.c | 3 ++
arch/powerpc/mm/pgtable.c | 4 ++
7 files changed, 61 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index b9fc064d38d2..2dfa5ccb25cc 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -166,6 +166,7 @@ config PPC
select ARCH_STACKWALK
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_DEBUG_PAGEALLOC if PPC_BOOK3S || PPC_8xx || 40x
+ select ARCH_SUPPORTS_PAGE_TABLE_CHECK
select ARCH_USE_BUILTIN_BSWAP
select ARCH_USE_CMPXCHG_LOCKREF if PPC64
select ARCH_USE_MEMTEST
diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h b/arch/powerpc/include/asm/book3s/32/pgtable.h
index 52971ee30717..a97edbc09984 100644
--- a/arch/powerpc/include/asm/book3s/32/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/32/pgtable.h
@@ -201,6 +201,7 @@ void unmap_kernel_page(unsigned long va);
#ifndef __ASSEMBLY__
#include <linux/sched.h>
#include <linux/threads.h>
+#include <linux/page_table_check.h>
/* Bits to mask out from a PGD to get to the PUD page */
#define PGD_MASKED_BITS 0
@@ -314,7 +315,11 @@ static inline int __ptep_test_and_clear_young(struct mm_struct *mm,
static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep)
{
- return __pte(pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, 0, 0));
+ pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, 0, 0));
+
+ page_table_check_pte_clear(mm, addr, old_pte);
+
+ return old_pte;
}
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index ca765331e21d..4ad88d4ede88 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -145,6 +145,8 @@
#define PAGE_KERNEL_ROX __pgprot(_PAGE_BASE | _PAGE_KERNEL_ROX)
#ifndef __ASSEMBLY__
+#include <linux/page_table_check.h>
+
/*
* page table defines
*/
@@ -415,8 +417,11 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long addr, pte_t *ptep)
{
- unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0);
- return __pte(old);
+ pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~0UL, 0, 0));
+
+ page_table_check_pte_clear(mm, addr, old_pte);
+
+ return old_pte;
}
#define __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL
@@ -425,11 +430,16 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
pte_t *ptep, int full)
{
if (full && radix_enabled()) {
+ pte_t old_pte;
+
/*
* We know that this is a full mm pte clear and
* hence can be sure there is no parallel set_pte.
*/
- return radix__ptep_get_and_clear_full(mm, addr, ptep, full);
+ old_pte = radix__ptep_get_and_clear_full(mm, addr, ptep, full);
+ page_table_check_pte_clear(mm, addr, old_pte);
+
+ return old_pte;
}
return ptep_get_and_clear(mm, addr, ptep);
}
@@ -1334,19 +1344,34 @@ extern int pudp_test_and_clear_young(struct vm_area_struct *vma,
static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
unsigned long addr, pmd_t *pmdp)
{
- if (radix_enabled())
- return radix__pmdp_huge_get_and_clear(mm, addr, pmdp);
- return hash__pmdp_huge_get_and_clear(mm, addr, pmdp);
+ pmd_t old_pmd;
+
+ if (radix_enabled()) {
+ old_pmd = radix__pmdp_huge_get_and_clear(mm, addr, pmdp);
+ } else {
+ old_pmd = hash__pmdp_huge_get_and_clear(mm, addr, pmdp);
+ }
+
+ page_table_check_pmd_clear(mm, addr, old_pmd);
+
+ return old_pmd;
}
#define __HAVE_ARCH_PUDP_HUGE_GET_AND_CLEAR
static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
unsigned long addr, pud_t *pudp)
{
- if (radix_enabled())
- return radix__pudp_huge_get_and_clear(mm, addr, pudp);
- BUG();
- return *pudp;
+ pud_t old_pud;
+
+ if (radix_enabled()) {
+ old_pud = radix__pudp_huge_get_and_clear(mm, addr, pudp);
+ } else {
+ BUG();
+ }
+
+ page_table_check_pud_clear(mm, addr, old_pud);
+
+ return old_pud;
}
static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,
diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c
index 871472f99a01..f200d55c35d8 100644
--- a/arch/powerpc/mm/book3s64/hash_pgtable.c
+++ b/arch/powerpc/mm/book3s64/hash_pgtable.c
@@ -8,6 +8,7 @@
#include <linux/sched.h>
#include <linux/mm_types.h>
#include <linux/mm.h>
+#include <linux/page_table_check.h>
#include <linux/stop_machine.h>
#include <asm/sections.h>
@@ -231,6 +232,9 @@ pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addres
pmd = *pmdp;
pmd_clear(pmdp);
+
+ page_table_check_pmd_clear(vma->vm_mm, address, pmd);
+
/*
* Wait for all pending hash_page to finish. This is needed
* in case of subpage collapse. When we collapse normal pages
diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c
index 25082ab6018b..fa352da844a9 100644
--- a/arch/powerpc/mm/book3s64/pgtable.c
+++ b/arch/powerpc/mm/book3s64/pgtable.c
@@ -10,6 +10,7 @@
#include <linux/pkeys.h>
#include <linux/debugfs.h>
#include <linux/proc_fs.h>
+#include <linux/page_table_check.h>
#include <misc/cxl-base.h>
#include <asm/pgalloc.h>
@@ -116,6 +117,7 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
WARN_ON(!(pmd_large(pmd)));
#endif
trace_hugepage_set_pmd(addr, pmd_val(pmd));
+ page_table_check_pmd_set(mm, addr, pmdp, pmd);
return set_pte_at_unchecked(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
}
@@ -133,6 +135,7 @@ void set_pud_at(struct mm_struct *mm, unsigned long addr,
WARN_ON(!(pud_large(pud)));
#endif
trace_hugepage_set_pud(addr, pud_val(pud));
+ page_table_check_pud_set(mm, addr, pudp, pud);
return set_pte_at_unchecked(mm, addr, pudp_ptep(pudp), pud_pte(pud));
}
@@ -168,11 +171,13 @@ void serialize_against_pte_lookup(struct mm_struct *mm)
pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp)
{
- unsigned long old_pmd;
+ pmd_t old_pmd;
- old_pmd = pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, _PAGE_INVALID);
+ old_pmd = __pmd(pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, _PAGE_INVALID));
flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
- return __pmd(old_pmd);
+ page_table_check_pmd_clear(vma->vm_mm, address, old_pmd);
+
+ return old_pmd;
}
pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma,
diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index c661e42bb2f1..1fafb9fb6231 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -14,6 +14,7 @@
#include <linux/of.h>
#include <linux/of_fdt.h>
#include <linux/mm.h>
+#include <linux/page_table_check.h>
#include <linux/hugetlb.h>
#include <linux/string_helpers.h>
#include <linux/memory.h>
@@ -1390,6 +1391,8 @@ pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addre
pmd = *pmdp;
pmd_clear(pmdp);
+ page_table_check_pmd_clear(vma->vm_mm, address, pmd);
+
radix__flush_tlb_collapsed_pmd(vma->vm_mm, address);
return pmd;
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index 352679cf2684..e89b28a7b313 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -22,6 +22,7 @@
#include <linux/mm.h>
#include <linux/percpu.h>
#include <linux/hardirq.h>
+#include <linux/page_table_check.h>
#include <linux/hugetlb.h>
#include <asm/tlbflush.h>
#include <asm/tlb.h>
@@ -206,6 +207,9 @@ void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
* and not hw_valid ptes. Hence there is no translation cache flush
* involved that need to be batched.
*/
+
+ page_table_check_ptes_set(mm, addr, ptep, pte, nr);
+
for (;;) {
/*
--
2.44.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf
2024-03-13 4:21 ` [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf Rohan McLure
@ 2024-03-13 10:33 ` Christophe Leroy
0 siblings, 0 replies; 19+ messages in thread
From: Christophe Leroy @ 2024-03-13 10:33 UTC (permalink / raw)
To: Rohan McLure, linuxppc-dev; +Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
Hi,
Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> Replace occurrences of p{u,m,4}d_is_leaf with p{u,m,4}_leaf, as the
> latter is the name given to checking that a higher-level entry in
> multi-level paging contains a page translation entry (pte) throughout
> all other archs.
There's already an equivalent commit in mm-stable, that will likely go
into v6.9:
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/commit/?h=mm-stable&id=bd18b688220c7225fb50498dabd9f9d0c9988e67
>
> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
> Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> ---
> v9: No longer required in order to implement page table check, just a
> refactor.
> v10: Fix more occurances, and just delete p{u,m,4}_is_leaf() stubs as
> equivalent p{u,m,4}_leaf() stubs already exist.
> ---
> arch/powerpc/include/asm/book3s/64/pgtable.h | 10 ++++----
> arch/powerpc/include/asm/pgtable.h | 24 --------------------
> arch/powerpc/kvm/book3s_64_mmu_radix.c | 12 +++++-----
> arch/powerpc/mm/book3s64/radix_pgtable.c | 14 ++++++------
> arch/powerpc/mm/pgtable.c | 6 ++---
> arch/powerpc/mm/pgtable_64.c | 6 ++---
> arch/powerpc/xmon/xmon.c | 6 ++---
> 7 files changed, 26 insertions(+), 52 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
> index 62c43d3d80ec..382724c5e872 100644
> --- a/arch/powerpc/include/asm/book3s/64/pgtable.h
> +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
> @@ -1443,16 +1443,14 @@ static inline bool is_pte_rw_upgrade(unsigned long old_val, unsigned long new_va
> /*
> * Like pmd_huge() and pmd_large(), but works regardless of config options
> */
> -#define pmd_is_leaf pmd_is_leaf
> -#define pmd_leaf pmd_is_leaf
> -static inline bool pmd_is_leaf(pmd_t pmd)
> +#define pmd_leaf pmd_leaf
> +static inline bool pmd_leaf(pmd_t pmd)
> {
> return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE));
> }
>
> -#define pud_is_leaf pud_is_leaf
> -#define pud_leaf pud_is_leaf
> -static inline bool pud_is_leaf(pud_t pud)
> +#define pud_leaf pud_leaf
> +static inline bool pud_leaf(pud_t pud)
> {
> return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE));
> }
> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> index 9224f23065ff..0c0ffbe7a3b5 100644
> --- a/arch/powerpc/include/asm/pgtable.h
> +++ b/arch/powerpc/include/asm/pgtable.h
> @@ -180,30 +180,6 @@ static inline void pte_frag_set(mm_context_t *ctx, void *p)
> }
> #endif
>
> -#ifndef pmd_is_leaf
> -#define pmd_is_leaf pmd_is_leaf
> -static inline bool pmd_is_leaf(pmd_t pmd)
> -{
> - return false;
> -}
> -#endif
> -
> -#ifndef pud_is_leaf
> -#define pud_is_leaf pud_is_leaf
> -static inline bool pud_is_leaf(pud_t pud)
> -{
> - return false;
> -}
> -#endif
> -
> -#ifndef p4d_is_leaf
> -#define p4d_is_leaf p4d_is_leaf
> -static inline bool p4d_is_leaf(p4d_t p4d)
> -{
> - return false;
> -}
> -#endif
> -
> #define pmd_pgtable pmd_pgtable
> static inline pgtable_t pmd_pgtable(pmd_t pmd)
> {
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c
> index 4a1abb9f7c05..408d98f8a514 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c
> @@ -503,7 +503,7 @@ static void kvmppc_unmap_free_pmd(struct kvm *kvm, pmd_t *pmd, bool full,
> for (im = 0; im < PTRS_PER_PMD; ++im, ++p) {
> if (!pmd_present(*p))
> continue;
> - if (pmd_is_leaf(*p)) {
> + if (pmd_leaf(*p)) {
> if (full) {
> pmd_clear(p);
> } else {
> @@ -532,7 +532,7 @@ static void kvmppc_unmap_free_pud(struct kvm *kvm, pud_t *pud,
> for (iu = 0; iu < PTRS_PER_PUD; ++iu, ++p) {
> if (!pud_present(*p))
> continue;
> - if (pud_is_leaf(*p)) {
> + if (pud_leaf(*p)) {
> pud_clear(p);
> } else {
> pmd_t *pmd;
> @@ -635,12 +635,12 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
> new_pud = pud_alloc_one(kvm->mm, gpa);
>
> pmd = NULL;
> - if (pud && pud_present(*pud) && !pud_is_leaf(*pud))
> + if (pud && pud_present(*pud) && !pud_leaf(*pud))
> pmd = pmd_offset(pud, gpa);
> else if (level <= 1)
> new_pmd = kvmppc_pmd_alloc();
>
> - if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_is_leaf(*pmd)))
> + if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_leaf(*pmd)))
> new_ptep = kvmppc_pte_alloc();
>
> /* Check if we might have been invalidated; let the guest retry if so */
> @@ -658,7 +658,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
> new_pud = NULL;
> }
> pud = pud_offset(p4d, gpa);
> - if (pud_is_leaf(*pud)) {
> + if (pud_leaf(*pud)) {
> unsigned long hgpa = gpa & PUD_MASK;
>
> /* Check if we raced and someone else has set the same thing */
> @@ -709,7 +709,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
> new_pmd = NULL;
> }
> pmd = pmd_offset(pud, gpa);
> - if (pmd_is_leaf(*pmd)) {
> + if (pmd_leaf(*pmd)) {
> unsigned long lgpa = gpa & PMD_MASK;
>
> /* Check if we raced and someone else has set the same thing */
> diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
> index e16e2fd104c5..46fa46ce6526 100644
> --- a/arch/powerpc/mm/book3s64/radix_pgtable.c
> +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
> @@ -204,14 +204,14 @@ static void radix__change_memory_range(unsigned long start, unsigned long end,
> pudp = pud_alloc(&init_mm, p4dp, idx);
> if (!pudp)
> continue;
> - if (pud_is_leaf(*pudp)) {
> + if (pud_leaf(*pudp)) {
> ptep = (pte_t *)pudp;
> goto update_the_pte;
> }
> pmdp = pmd_alloc(&init_mm, pudp, idx);
> if (!pmdp)
> continue;
> - if (pmd_is_leaf(*pmdp)) {
> + if (pmd_leaf(*pmdp)) {
> ptep = pmdp_ptep(pmdp);
> goto update_the_pte;
> }
> @@ -767,7 +767,7 @@ static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr,
> if (!pmd_present(*pmd))
> continue;
>
> - if (pmd_is_leaf(*pmd)) {
> + if (pmd_leaf(*pmd)) {
> if (IS_ALIGNED(addr, PMD_SIZE) &&
> IS_ALIGNED(next, PMD_SIZE)) {
> if (!direct)
> @@ -807,7 +807,7 @@ static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr,
> if (!pud_present(*pud))
> continue;
>
> - if (pud_is_leaf(*pud)) {
> + if (pud_leaf(*pud)) {
> if (!IS_ALIGNED(addr, PUD_SIZE) ||
> !IS_ALIGNED(next, PUD_SIZE)) {
> WARN_ONCE(1, "%s: unaligned range\n", __func__);
> @@ -845,7 +845,7 @@ remove_pagetable(unsigned long start, unsigned long end, bool direct,
> if (!p4d_present(*p4d))
> continue;
>
> - if (p4d_is_leaf(*p4d)) {
> + if (p4d_leaf(*p4d)) {
> if (!IS_ALIGNED(addr, P4D_SIZE) ||
> !IS_ALIGNED(next, P4D_SIZE)) {
> WARN_ONCE(1, "%s: unaligned range\n", __func__);
> @@ -1540,7 +1540,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
>
> int pud_clear_huge(pud_t *pud)
> {
> - if (pud_is_leaf(*pud)) {
> + if (pud_leaf(*pud)) {
> pud_clear(pud);
> return 1;
> }
> @@ -1587,7 +1587,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
>
> int pmd_clear_huge(pmd_t *pmd)
> {
> - if (pmd_is_leaf(*pmd)) {
> + if (pmd_leaf(*pmd)) {
> pmd_clear(pmd);
> return 1;
> }
> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> index a04ae4449a02..e8e0289d7ab0 100644
> --- a/arch/powerpc/mm/pgtable.c
> +++ b/arch/powerpc/mm/pgtable.c
> @@ -413,7 +413,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
> if (p4d_none(p4d))
> return NULL;
>
> - if (p4d_is_leaf(p4d)) {
> + if (p4d_leaf(p4d)) {
> ret_pte = (pte_t *)p4dp;
> goto out;
> }
> @@ -435,7 +435,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
> if (pud_none(pud))
> return NULL;
>
> - if (pud_is_leaf(pud)) {
> + if (pud_leaf(pud)) {
> ret_pte = (pte_t *)pudp;
> goto out;
> }
> @@ -474,7 +474,7 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
> goto out;
> }
>
> - if (pmd_is_leaf(pmd)) {
> + if (pmd_leaf(pmd)) {
> ret_pte = (pte_t *)pmdp;
> goto out;
> }
> diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
> index 5ac1fd30341b..0604c80dae66 100644
> --- a/arch/powerpc/mm/pgtable_64.c
> +++ b/arch/powerpc/mm/pgtable_64.c
> @@ -100,7 +100,7 @@ EXPORT_SYMBOL(__pte_frag_size_shift);
> /* 4 level page table */
> struct page *p4d_page(p4d_t p4d)
> {
> - if (p4d_is_leaf(p4d)) {
> + if (p4d_leaf(p4d)) {
> if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
> VM_WARN_ON(!p4d_huge(p4d));
> return pte_page(p4d_pte(p4d));
> @@ -111,7 +111,7 @@ struct page *p4d_page(p4d_t p4d)
>
> struct page *pud_page(pud_t pud)
> {
> - if (pud_is_leaf(pud)) {
> + if (pud_leaf(pud)) {
> if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
> VM_WARN_ON(!pud_huge(pud));
> return pte_page(pud_pte(pud));
> @@ -125,7 +125,7 @@ struct page *pud_page(pud_t pud)
> */
> struct page *pmd_page(pmd_t pmd)
> {
> - if (pmd_is_leaf(pmd)) {
> + if (pmd_leaf(pmd)) {
> /*
> * vmalloc_to_page may be called on any vmap address (not only
> * vmalloc), and it uses pmd_page() etc., when huge vmap is
> diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
> index c85fa3f0dd3b..d79d6633f333 100644
> --- a/arch/powerpc/xmon/xmon.c
> +++ b/arch/powerpc/xmon/xmon.c
> @@ -3340,7 +3340,7 @@ static void show_pte(unsigned long addr)
> return;
> }
>
> - if (p4d_is_leaf(*p4dp)) {
> + if (p4d_leaf(*p4dp)) {
> format_pte(p4dp, p4d_val(*p4dp));
> return;
> }
> @@ -3354,7 +3354,7 @@ static void show_pte(unsigned long addr)
> return;
> }
>
> - if (pud_is_leaf(*pudp)) {
> + if (pud_leaf(*pudp)) {
> format_pte(pudp, pud_val(*pudp));
> return;
> }
> @@ -3368,7 +3368,7 @@ static void show_pte(unsigned long addr)
> return;
> }
>
> - if (pmd_is_leaf(*pmdp)) {
> + if (pmd_leaf(*pmdp)) {
> format_pte(pmdp, pmd_val(*pmdp));
> return;
> }
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms
2024-03-13 4:21 ` [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms Rohan McLure
@ 2024-03-13 11:08 ` Christophe Leroy
2024-03-14 0:18 ` LTC IMAP
0 siblings, 1 reply; 19+ messages in thread
From: Christophe Leroy @ 2024-03-13 11:08 UTC (permalink / raw)
To: Rohan McLure, linuxppc-dev; +Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> Prior to this commit, pud_pfn was implemented with BUILD_BUG as the inline
> function for 64-bit Book3S systems but is never included, as its
> invocations in generic code are guarded by calls to pud_devmap which return
> zero on such systems. A future patch will provide support for page table
> checks, the generic code for which depends on a pud_pfn stub being
> implemented, even while the patch will not interact with puds directly.
>
> Remove the 64-bit Book3S stub and define pud_pfn to warn on all
> platforms. pud_pfn may be defined properly on a per-platform basis
> should it grow real usages in future.
Can you please re-explain why that's needed ? I remember we discussed it
already in the past, but I checked again today and can't see the need:
In mm/page_table_check.c, the call to pud_pfn() is gated by a call to
pud_user_accessible_page(pud). If I look into arm64 version of
pud_user_accessible_page(), it depends on pud_leaf(). When pud_leaf() is
constant 0, pud_user_accessible_page() is always false and the call to
pud_pfn() should be folded away.
>
> Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> ---
> arch/powerpc/include/asm/pgtable.h | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> index 0c0ffbe7a3b5..13f661831333 100644
> --- a/arch/powerpc/include/asm/pgtable.h
> +++ b/arch/powerpc/include/asm/pgtable.h
> @@ -213,6 +213,20 @@ static inline bool arch_supports_memmap_on_memory(unsigned long vmemmap_size)
>
> #endif /* CONFIG_PPC64 */
>
> +/*
> + * Currently only consumed by page_table_check_pud_{set,clear}. Since clears
> + * and sets to page table entries at any level are done through
> + * page_table_check_pte_{set,clear}, provide stub implementation.
> + */
> +#ifndef pud_pfn
> +#define pud_pfn pud_pfn
> +static inline int pud_pfn(pud_t pud)
> +{
> + WARN_ONCE(1, "pud: platform does not use pud entries directly");
> + return 0;
> +}
> +#endif
> +
> #endif /* __ASSEMBLY__ */
>
> #endif /* _ASM_POWERPC_PGTABLE_H */
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes
2024-03-13 4:21 ` [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes Rohan McLure
@ 2024-03-13 11:19 ` Christophe Leroy
0 siblings, 0 replies; 19+ messages in thread
From: Christophe Leroy @ 2024-03-13 11:19 UTC (permalink / raw)
To: Rohan McLure, linuxppc-dev; +Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> Page table checking depends on architectures providing an
> implementation of p{te,md,ud}_user_accessible_page. With
> refactorisations made on powerpc/mm, the pte_access_permitted() and
> similar methods verify whether a userland page is accessible with the
> required permissions.
>
> Since page table checking is the only user of
> p{te,md,ud}_user_accessible_page(), implement these for all platforms,
> using some of the same preliminay checks taken by pte_access_permitted()
> on that platform.
>
> Since Commit 8e9bd41e4ce1 ("powerpc/nohash: Replace pte_user() by pte_read()")
> pte_user() is no longer required to be present on all platforms as it
> may be equivalent to or implied by pte_read(). Hence implementations are
> specialised.
>
> Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> ---
> v9: New implementation
> v10: Let book3s/64 use pte_user(), but otherwise default other platforms
> to using the address provided with the call to infer whether it is a
> user page or not. pmd/pud variants will warn on all other platforms, as
> they should not be used for user page mappings
> ---
> arch/powerpc/include/asm/book3s/64/pgtable.h | 19 ++++++++++++++
> arch/powerpc/include/asm/pgtable.h | 26 ++++++++++++++++++++
> 2 files changed, 45 insertions(+)
>
> diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
> index 382724c5e872..ca765331e21d 100644
> --- a/arch/powerpc/include/asm/book3s/64/pgtable.h
> +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
> @@ -538,6 +538,12 @@ static inline bool pte_access_permitted(pte_t pte, bool write)
> return arch_pte_access_permitted(pte_val(pte), write, 0);
> }
>
> +#define pte_user_accessible_page pte_user_accessible_page
> +static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
> +{
> + return pte_present(pte) && pte_user(pte);
> +}
> +
> /*
> * Conversion functions: convert a page and protection to a page entry,
> * and a page entry and page directory to the page they refer to.
> @@ -881,6 +887,7 @@ static inline int pud_present(pud_t pud)
>
> extern struct page *pud_page(pud_t pud);
> extern struct page *pmd_page(pmd_t pmd);
> +
Garbage ?
> static inline pte_t pud_pte(pud_t pud)
> {
> return __pte_raw(pud_raw(pud));
> @@ -926,6 +933,12 @@ static inline bool pud_access_permitted(pud_t pud, bool write)
> return pte_access_permitted(pud_pte(pud), write);
> }
>
> +#define pud_user_accessible_page pud_user_accessible_page
> +static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
> +{
> + return pte_user_accessible_page(pud_pte(pud), addr);
> +}
> +
If I understand what is done on arm64, you should first check
pud_leaf(). Then this function could be common to all powerpc platforms,
only pte_user_accessible_page() would be platform specific.
> #define __p4d_raw(x) ((p4d_t) { __pgd_raw(x) })
> static inline __be64 p4d_raw(p4d_t x)
> {
> @@ -1091,6 +1104,12 @@ static inline bool pmd_access_permitted(pmd_t pmd, bool write)
> return pte_access_permitted(pmd_pte(pmd), write);
> }
>
> +#define pmd_user_accessible_page pmd_user_accessible_page
> +static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
> +{
> + return pte_user_accessible_page(pmd_pte(pmd), addr);
> +}
Same, pmd_leaf() should be checked.
> +
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot);
> extern pud_t pfn_pud(unsigned long pfn, pgprot_t pgprot);
> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> index 13f661831333..3741a63fb82e 100644
> --- a/arch/powerpc/include/asm/pgtable.h
> +++ b/arch/powerpc/include/asm/pgtable.h
> @@ -227,6 +227,32 @@ static inline int pud_pfn(pud_t pud)
> }
> #endif
>
> +#ifndef pte_user_accessible_page
> +#define pte_user_accessible_page pte_user_accessible_page
> +static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
> +{
> + return pte_present(pte) && !is_kernel_addr(addr);
> +}
> +#endif
I would prefer to see one version in asm/book3s/32/pgtable.h and one in
asm/nohash/pgtable.h and then avoid this game with ifdefs.
> +
> +#ifndef pmd_user_accessible_page
> +#define pmd_user_accessible_page pmd_user_accessible_page
> +static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
> +{
> + WARN_ONCE(1, "pmd: platform does not use pmd entries directly");
> + return false;
> +}
> +#endif
Also check pmd_leaf() and this function on all platforms.
> +
> +#ifndef pud_user_accessible_page
> +#define pud_user_accessible_page pud_user_accessible_page
> +static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
> +{
> + WARN_ONCE(1, "pud: platform does not use pud entries directly");
> + return false;
> +}
Also check pud_leaf() and this function on all platforms.
> +#endif
> +
> #endif /* __ASSEMBLY__ */
>
> #endif /* _ASM_POWERPC_PGTABLE_H */
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages
2024-03-13 4:21 ` [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages Rohan McLure
@ 2024-03-13 11:30 ` Christophe Leroy
2024-03-15 0:45 ` LTC IMAP
0 siblings, 1 reply; 19+ messages in thread
From: Christophe Leroy @ 2024-03-13 11:30 UTC (permalink / raw)
To: Rohan McLure, linuxppc-dev; +Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> In the new set_ptes() API, set_pte_at() (a special case of set_ptes())
> is intended to be instrumented by the page table check facility. There
> are however several other routines that constitute the API for setting
> page table entries, including set_pmd_at() among others. Such routines
> are themselves implemented in terms of set_ptes_at().
>
> A future patch providing support for page table checking on powerpc
> must take care to avoid duplicate calls to
> page_table_check_p{te,md,ud}_set(). Allow for assignment of pte entries
> without instrumentation through the set_pte_at_unchecked() routine
> introduced in this patch.
>
> Cause API-facing routines that call set_pte_at() to instead call
> set_pte_at_unchecked(), which will remain uninstrumented by page
> table check. set_ptes() is itself implemented by calls to
> __set_pte_at(), so this eliminates redundant code.
>
> Also prefer set_pte_at_unchecked() in early-boot usages which should not be
> instrumented.
>
> Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> ---
> v9: New patch
> v10: don't reuse __set_pte_at(), as that will not apply filters. Instead
> use new set_pte_at_unchecked().
Are filters needed at all in those usecases ?
> ---
> arch/powerpc/include/asm/pgtable.h | 2 ++
> arch/powerpc/mm/book3s64/hash_pgtable.c | 2 +-
> arch/powerpc/mm/book3s64/pgtable.c | 6 +++---
> arch/powerpc/mm/book3s64/radix_pgtable.c | 8 ++++----
> arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +-
> arch/powerpc/mm/pgtable.c | 7 +++++++
> arch/powerpc/mm/pgtable_32.c | 2 +-
> 7 files changed, 19 insertions(+), 10 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
> index 3741a63fb82e..6ff1d8cfa216 100644
> --- a/arch/powerpc/include/asm/pgtable.h
> +++ b/arch/powerpc/include/asm/pgtable.h
> @@ -44,6 +44,8 @@ struct mm_struct;
> void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
> pte_t pte, unsigned int nr);
> #define set_ptes set_ptes
> +void set_pte_at_unchecked(struct mm_struct *mm, unsigned long addr,
> + pte_t *ptep, pte_t pte);
> #define update_mmu_cache(vma, addr, ptep) \
> update_mmu_cache_range(NULL, vma, addr, ptep, 1)
>
> diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c
> index 988948d69bc1..871472f99a01 100644
> --- a/arch/powerpc/mm/book3s64/hash_pgtable.c
> +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c
> @@ -165,7 +165,7 @@ int hash__map_kernel_page(unsigned long ea, unsigned long pa, pgprot_t prot)
> ptep = pte_alloc_kernel(pmdp, ea);
> if (!ptep)
> return -ENOMEM;
> - set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
> + set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
> } else {
> /*
> * If the mm subsystem is not fully up, we cannot create a
> diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c
> index 3438ab72c346..25082ab6018b 100644
> --- a/arch/powerpc/mm/book3s64/pgtable.c
> +++ b/arch/powerpc/mm/book3s64/pgtable.c
> @@ -116,7 +116,7 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
> WARN_ON(!(pmd_large(pmd)));
> #endif
> trace_hugepage_set_pmd(addr, pmd_val(pmd));
> - return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
> + return set_pte_at_unchecked(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
> }
>
> void set_pud_at(struct mm_struct *mm, unsigned long addr,
> @@ -133,7 +133,7 @@ void set_pud_at(struct mm_struct *mm, unsigned long addr,
> WARN_ON(!(pud_large(pud)));
> #endif
> trace_hugepage_set_pud(addr, pud_val(pud));
> - return set_pte_at(mm, addr, pudp_ptep(pudp), pud_pte(pud));
> + return set_pte_at_unchecked(mm, addr, pudp_ptep(pudp), pud_pte(pud));
> }
>
> static void do_serialize(void *arg)
> @@ -539,7 +539,7 @@ void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr,
> if (radix_enabled())
> return radix__ptep_modify_prot_commit(vma, addr,
> ptep, old_pte, pte);
> - set_pte_at(vma->vm_mm, addr, ptep, pte);
> + set_pte_at_unchecked(vma->vm_mm, addr, ptep, pte);
> }
>
> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
> index 46fa46ce6526..c661e42bb2f1 100644
> --- a/arch/powerpc/mm/book3s64/radix_pgtable.c
> +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
> @@ -109,7 +109,7 @@ static int early_map_kernel_page(unsigned long ea, unsigned long pa,
> ptep = pte_offset_kernel(pmdp, ea);
>
> set_the_pte:
> - set_pte_at(&init_mm, ea, ptep, pfn_pte(pfn, flags));
> + set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pfn, flags));
> asm volatile("ptesync": : :"memory");
> return 0;
> }
> @@ -1522,7 +1522,7 @@ void radix__ptep_modify_prot_commit(struct vm_area_struct *vma,
> (atomic_read(&mm->context.copros) > 0))
> radix__flush_tlb_page(vma, addr);
>
> - set_pte_at(mm, addr, ptep, pte);
> + set_pte_at_unchecked(mm, addr, ptep, pte);
> }
>
> int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
> @@ -1533,7 +1533,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
> if (!radix_enabled())
> return 0;
>
> - set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pud);
> + set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep, new_pud);
>
> return 1;
> }
> @@ -1580,7 +1580,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
> if (!radix_enabled())
> return 0;
>
> - set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pmd);
> + set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep, new_pmd);
>
> return 1;
> }
> diff --git a/arch/powerpc/mm/nohash/book3e_pgtable.c b/arch/powerpc/mm/nohash/book3e_pgtable.c
> index 1c5e4ecbebeb..10d487b2b991 100644
> --- a/arch/powerpc/mm/nohash/book3e_pgtable.c
> +++ b/arch/powerpc/mm/nohash/book3e_pgtable.c
> @@ -111,7 +111,7 @@ int __ref map_kernel_page(unsigned long ea, phys_addr_t pa, pgprot_t prot)
> }
> ptep = pte_offset_kernel(pmdp, ea);
> }
> - set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
> + set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, prot));
>
> smp_wmb();
> return 0;
> diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> index e8e0289d7ab0..352679cf2684 100644
> --- a/arch/powerpc/mm/pgtable.c
> +++ b/arch/powerpc/mm/pgtable.c
> @@ -227,6 +227,13 @@ void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
> }
> }
>
> +void set_pte_at_unchecked(struct mm_struct *mm, unsigned long addr,
> + pte_t *ptep, pte_t pte)
> +{
No need of the
VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep));
which is in set_ptes() ?
> + pte = set_pte_filter(pte, addr);
> + __set_pte_at(mm, addr, ptep, pte, 0);
> +}
> +
> void unmap_kernel_page(unsigned long va)
> {
> pmd_t *pmdp = pmd_off_k(va);
> diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
> index 4be97b4a44f9..a5a26faf91ec 100644
> --- a/arch/powerpc/mm/pgtable_32.c
> +++ b/arch/powerpc/mm/pgtable_32.c
> @@ -89,7 +89,7 @@ int __ref map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot)
> * hash table
> */
> BUG_ON((pte_present(*pg) | pte_hashpte(*pg)) && pgprot_val(prot));
> - set_pte_at(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT, prot));
> + set_pte_at_unchecked(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT, prot));
> }
> smp_wmb();
> return err;
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms
2024-03-13 11:08 ` Christophe Leroy
@ 2024-03-14 0:18 ` LTC IMAP
0 siblings, 0 replies; 19+ messages in thread
From: LTC IMAP @ 2024-03-14 0:18 UTC (permalink / raw)
To: Christophe Leroy, linuxppc-dev
Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
On Wed, 2024-03-13 at 11:08 +0000, Christophe Leroy wrote:
>
>
> Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> > Prior to this commit, pud_pfn was implemented with BUILD_BUG as the
> > inline
> > function for 64-bit Book3S systems but is never included, as its
> > invocations in generic code are guarded by calls to pud_devmap
> > which return
> > zero on such systems. A future patch will provide support for page
> > table
> > checks, the generic code for which depends on a pud_pfn stub being
> > implemented, even while the patch will not interact with puds
> > directly.
> >
> > Remove the 64-bit Book3S stub and define pud_pfn to warn on all
> > platforms. pud_pfn may be defined properly on a per-platform basis
> > should it grow real usages in future.
Apologies, I don't actually remove the 64-bit, Book3S stub, as it
currently correctly reflects how transparent hugepages should work.
Also the stub that was previously implemented for all platforms has
been removed in commit 27af67f35631 ("powerpc/book3s64/mm: enable
transparent pud hugepage").
>
> Can you please re-explain why that's needed ? I remember we discussed
> it
> already in the past, but I checked again today and can't see the
> need:
>
> In mm/page_table_check.c, the call to pud_pfn() is gated by a call to
> pud_user_accessible_page(pud). If I look into arm64 version of
> pud_user_accessible_page(), it depends on pud_leaf(). When pud_leaf()
> is
> constant 0, pud_user_accessible_page() is always false and the call
> to
> pud_pfn() should be folded away.
As it will be folded away on non 64-bit Book3S platforms, I could
even replace the WARN_ONCE with a BUILD_BUG for your stated reason.
The __page_table_check_pud_set() function will still be included in the
build and references this routine so a fallback stub is still
necessary.
>
> >
> > Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> > ---
> > arch/powerpc/include/asm/pgtable.h | 14 ++++++++++++++
> > 1 file changed, 14 insertions(+)
> >
> > diff --git a/arch/powerpc/include/asm/pgtable.h
> > b/arch/powerpc/include/asm/pgtable.h
> > index 0c0ffbe7a3b5..13f661831333 100644
> > --- a/arch/powerpc/include/asm/pgtable.h
> > +++ b/arch/powerpc/include/asm/pgtable.h
> > @@ -213,6 +213,20 @@ static inline bool
> > arch_supports_memmap_on_memory(unsigned long vmemmap_size)
> >
> > #endif /* CONFIG_PPC64 */
> >
> > +/*
> > + * Currently only consumed by page_table_check_pud_{set,clear}.
> > Since clears
> > + * and sets to page table entries at any level are done through
> > + * page_table_check_pte_{set,clear}, provide stub implementation.
> > + */
> > +#ifndef pud_pfn
> > +#define pud_pfn pud_pfn
> > +static inline int pud_pfn(pud_t pud)
> > +{
> > + WARN_ONCE(1, "pud: platform does not use pud entries
> > directly");
> > + return 0;
> > +}
> > +#endif
> > +
> > #endif /* __ASSEMBLY__ */
> >
> > #endif /* _ASM_POWERPC_PGTABLE_H */
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages
2024-03-13 11:30 ` Christophe Leroy
@ 2024-03-15 0:45 ` LTC IMAP
0 siblings, 0 replies; 19+ messages in thread
From: LTC IMAP @ 2024-03-15 0:45 UTC (permalink / raw)
To: Christophe Leroy, linuxppc-dev
Cc: linux-riscv, x86, linux-arm-kernel, linux-mm
On Wed, 2024-03-13 at 11:30 +0000, Christophe Leroy wrote:
>
>
> Le 13/03/2024 à 05:21, Rohan McLure a écrit :
> > In the new set_ptes() API, set_pte_at() (a special case of
> > set_ptes())
> > is intended to be instrumented by the page table check facility.
> > There
> > are however several other routines that constitute the API for
> > setting
> > page table entries, including set_pmd_at() among others. Such
> > routines
> > are themselves implemented in terms of set_ptes_at().
> >
> > A future patch providing support for page table checking on powerpc
> > must take care to avoid duplicate calls to
> > page_table_check_p{te,md,ud}_set(). Allow for assignment of pte
> > entries
> > without instrumentation through the set_pte_at_unchecked() routine
> > introduced in this patch.
> >
> > Cause API-facing routines that call set_pte_at() to instead call
> > set_pte_at_unchecked(), which will remain uninstrumented by page
> > table check. set_ptes() is itself implemented by calls to
> > __set_pte_at(), so this eliminates redundant code.
> >
> > Also prefer set_pte_at_unchecked() in early-boot usages which
> > should not be
> > instrumented.
> >
> > Signed-off-by: Rohan McLure <rmclure@linux.ibm.com>
> > ---
> > v9: New patch
> > v10: don't reuse __set_pte_at(), as that will not apply filters.
> > Instead
> > use new set_pte_at_unchecked().
>
> Are filters needed at all in those usecases ?
I'm just retaining the original semantics of these calls. I think
another patch can replace this call with __set_pte_at() if filters are
deemed unnecessary.
>
> > ---
> > arch/powerpc/include/asm/pgtable.h | 2 ++
> > arch/powerpc/mm/book3s64/hash_pgtable.c | 2 +-
> > arch/powerpc/mm/book3s64/pgtable.c | 6 +++---
> > arch/powerpc/mm/book3s64/radix_pgtable.c | 8 ++++----
> > arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +-
> > arch/powerpc/mm/pgtable.c | 7 +++++++
> > arch/powerpc/mm/pgtable_32.c | 2 +-
> > 7 files changed, 19 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/powerpc/include/asm/pgtable.h
> > b/arch/powerpc/include/asm/pgtable.h
> > index 3741a63fb82e..6ff1d8cfa216 100644
> > --- a/arch/powerpc/include/asm/pgtable.h
> > +++ b/arch/powerpc/include/asm/pgtable.h
> > @@ -44,6 +44,8 @@ struct mm_struct;
> > void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t
> > *ptep,
> > pte_t pte, unsigned int nr);
> > #define set_ptes set_ptes
> > +void set_pte_at_unchecked(struct mm_struct *mm, unsigned long
> > addr,
> > + pte_t *ptep, pte_t pte);
> > #define update_mmu_cache(vma, addr, ptep) \
> > update_mmu_cache_range(NULL, vma, addr, ptep, 1)
> >
> > diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c
> > b/arch/powerpc/mm/book3s64/hash_pgtable.c
> > index 988948d69bc1..871472f99a01 100644
> > --- a/arch/powerpc/mm/book3s64/hash_pgtable.c
> > +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c
> > @@ -165,7 +165,7 @@ int hash__map_kernel_page(unsigned long ea,
> > unsigned long pa, pgprot_t prot)
> > ptep = pte_alloc_kernel(pmdp, ea);
> > if (!ptep)
> > return -ENOMEM;
> > - set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >>
> > PAGE_SHIFT, prot));
> > + set_pte_at_unchecked(&init_mm, ea, ptep,
> > pfn_pte(pa >> PAGE_SHIFT, prot));
> > } else {
> > /*
> > * If the mm subsystem is not fully up, we cannot
> > create a
> > diff --git a/arch/powerpc/mm/book3s64/pgtable.c
> > b/arch/powerpc/mm/book3s64/pgtable.c
> > index 3438ab72c346..25082ab6018b 100644
> > --- a/arch/powerpc/mm/book3s64/pgtable.c
> > +++ b/arch/powerpc/mm/book3s64/pgtable.c
> > @@ -116,7 +116,7 @@ void set_pmd_at(struct mm_struct *mm, unsigned
> > long addr,
> > WARN_ON(!(pmd_large(pmd)));
> > #endif
> > trace_hugepage_set_pmd(addr, pmd_val(pmd));
> > - return set_pte_at(mm, addr, pmdp_ptep(pmdp),
> > pmd_pte(pmd));
> > + return set_pte_at_unchecked(mm, addr, pmdp_ptep(pmdp),
> > pmd_pte(pmd));
> > }
> >
> > void set_pud_at(struct mm_struct *mm, unsigned long addr,
> > @@ -133,7 +133,7 @@ void set_pud_at(struct mm_struct *mm, unsigned
> > long addr,
> > WARN_ON(!(pud_large(pud)));
> > #endif
> > trace_hugepage_set_pud(addr, pud_val(pud));
> > - return set_pte_at(mm, addr, pudp_ptep(pudp),
> > pud_pte(pud));
> > + return set_pte_at_unchecked(mm, addr, pudp_ptep(pudp),
> > pud_pte(pud));
> > }
> >
> > static void do_serialize(void *arg)
> > @@ -539,7 +539,7 @@ void ptep_modify_prot_commit(struct
> > vm_area_struct *vma, unsigned long addr,
> > if (radix_enabled())
> > return radix__ptep_modify_prot_commit(vma, addr,
> > ptep,
> > old_pte, pte);
> > - set_pte_at(vma->vm_mm, addr, ptep, pte);
> > + set_pte_at_unchecked(vma->vm_mm, addr, ptep, pte);
> > }
> >
> > #ifdef CONFIG_TRANSPARENT_HUGEPAGE
> > diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c
> > b/arch/powerpc/mm/book3s64/radix_pgtable.c
> > index 46fa46ce6526..c661e42bb2f1 100644
> > --- a/arch/powerpc/mm/book3s64/radix_pgtable.c
> > +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
> > @@ -109,7 +109,7 @@ static int early_map_kernel_page(unsigned long
> > ea, unsigned long pa,
> > ptep = pte_offset_kernel(pmdp, ea);
> >
> > set_the_pte:
> > - set_pte_at(&init_mm, ea, ptep, pfn_pte(pfn, flags));
> > + set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pfn,
> > flags));
> > asm volatile("ptesync": : :"memory");
> > return 0;
> > }
> > @@ -1522,7 +1522,7 @@ void radix__ptep_modify_prot_commit(struct
> > vm_area_struct *vma,
> > (atomic_read(&mm->context.copros) > 0))
> > radix__flush_tlb_page(vma, addr);
> >
> > - set_pte_at(mm, addr, ptep, pte);
> > + set_pte_at_unchecked(mm, addr, ptep, pte);
> > }
> >
> > int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
> > @@ -1533,7 +1533,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t
> > addr, pgprot_t prot)
> > if (!radix_enabled())
> > return 0;
> >
> > - set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pud);
> > + set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep,
> > new_pud);
> >
> > return 1;
> > }
> > @@ -1580,7 +1580,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t
> > addr, pgprot_t prot)
> > if (!radix_enabled())
> > return 0;
> >
> > - set_pte_at(&init_mm, 0 /* radix unused */, ptep, new_pmd);
> > + set_pte_at_unchecked(&init_mm, 0 /* radix unused */, ptep,
> > new_pmd);
> >
> > return 1;
> > }
> > diff --git a/arch/powerpc/mm/nohash/book3e_pgtable.c
> > b/arch/powerpc/mm/nohash/book3e_pgtable.c
> > index 1c5e4ecbebeb..10d487b2b991 100644
> > --- a/arch/powerpc/mm/nohash/book3e_pgtable.c
> > +++ b/arch/powerpc/mm/nohash/book3e_pgtable.c
> > @@ -111,7 +111,7 @@ int __ref map_kernel_page(unsigned long ea,
> > phys_addr_t pa, pgprot_t prot)
> > }
> > ptep = pte_offset_kernel(pmdp, ea);
> > }
> > - set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
> > prot));
> > + set_pte_at_unchecked(&init_mm, ea, ptep, pfn_pte(pa >>
> > PAGE_SHIFT, prot));
> >
> > smp_wmb();
> > return 0;
> > diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
> > index e8e0289d7ab0..352679cf2684 100644
> > --- a/arch/powerpc/mm/pgtable.c
> > +++ b/arch/powerpc/mm/pgtable.c
> > @@ -227,6 +227,13 @@ void set_ptes(struct mm_struct *mm, unsigned
> > long addr, pte_t *ptep,
> > }
> > }
> >
> > +void set_pte_at_unchecked(struct mm_struct *mm, unsigned long
> > addr,
> > + pte_t *ptep, pte_t pte)
> > +{
>
> No need of the
>
> VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep));
>
> which is in set_ptes() ?
Good spot, I'll include that check in this routine.
>
> > + pte = set_pte_filter(pte, addr);
> > + __set_pte_at(mm, addr, ptep, pte, 0);
> > +}
> > +
> > void unmap_kernel_page(unsigned long va)
> > {
> > pmd_t *pmdp = pmd_off_k(va);
> > diff --git a/arch/powerpc/mm/pgtable_32.c
> > b/arch/powerpc/mm/pgtable_32.c
> > index 4be97b4a44f9..a5a26faf91ec 100644
> > --- a/arch/powerpc/mm/pgtable_32.c
> > +++ b/arch/powerpc/mm/pgtable_32.c
> > @@ -89,7 +89,7 @@ int __ref map_kernel_page(unsigned long va,
> > phys_addr_t pa, pgprot_t prot)
> > * hash table
> > */
> > BUG_ON((pte_present(*pg) | pte_hashpte(*pg)) &&
> > pgprot_val(prot));
> > - set_pte_at(&init_mm, va, pg, pfn_pte(pa >>
> > PAGE_SHIFT, prot));
> > + set_pte_at_unchecked(&init_mm, va, pg, pfn_pte(pa
> > >> PAGE_SHIFT, prot));
> > }
> > smp_wmb();
> > return err;
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2024-03-15 0:47 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-13 4:21 [PATCH v10 00/12] Support page table check PowerPC Rohan McLure
2024-03-13 4:21 ` [PATCH v10 01/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_set" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 02/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_set" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 03/12] mm: Provide addr parameter to page_table_check_pte_set() Rohan McLure
2024-03-13 4:21 ` [PATCH v10 04/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pud_clear" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 05/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pmd_clear" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 06/12] Revert "mm/page_table_check: remove unused parameter in [__]page_table_check_pte_clear" Rohan McLure
2024-03-13 4:21 ` [PATCH v10 07/12] mm: Provide address parameter to p{te,md,ud}_user_accessible_page() Rohan McLure
2024-03-13 4:21 ` [PATCH v10 08/12] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf Rohan McLure
2024-03-13 10:33 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 09/12] powerpc: mm: Add common pud_pfn stub for all platforms Rohan McLure
2024-03-13 11:08 ` Christophe Leroy
2024-03-14 0:18 ` LTC IMAP
2024-03-13 4:21 ` [PATCH v10 10/12] poweprc: mm: Implement *_user_accessible_page() for ptes Rohan McLure
2024-03-13 11:19 ` Christophe Leroy
2024-03-13 4:21 ` [PATCH v10 11/12] powerpc: mm: Use set_pte_at_unchecked() for early-boot / internal usages Rohan McLure
2024-03-13 11:30 ` Christophe Leroy
2024-03-15 0:45 ` LTC IMAP
2024-03-13 4:21 ` [PATCH v10 12/12] powerpc: mm: Support page table check Rohan McLure
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).