From: Huangzhaoyang <huangzhaoyang@gmail.com>
To: Ard Biesheuvel <ardb@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Anshuman Khandual <anshuman.khandual@arm.com>,
Andrew Morton <akpm@linux-foundation.org>,
Nicholas Piggin <npiggin@gmail.com>,
Mike Rapoport <rppt@kernel.org>,
Pavel Tatashin <pasha.tatashin@soleen.com>,
Christophe Leroy <christophe.leroy@csgroup.eu>,
Jonathan Marek <jonathan@marek.ca>,
Zhaoyang Huang <zhaoyang.huang@unisoc.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH] arch: arm64: introduce RODATA_FULL_USE_PTE_CONT
Date: Mon, 22 Nov 2021 13:28:49 +0800 [thread overview]
Message-ID: <1637558929-22971-1-git-send-email-huangzhaoyang@gmail.com> (raw)
From: Zhaoyang Huang <zhaoyang.huang@unisoc.com>
Kernel linear mapping will be split to the smallest granularity when
RODATA_FULL applied, which could lead to TLB pressure. Introduce a method
to apply PTE_CONT on pte.
Signed-off-by: Zhaoyang Huang <zhaoyang.huang@unisoc.com>
---
arch/arm64/Kconfig | 9 +++++++++
arch/arm64/mm/mmu.c | 10 ++++++++--
arch/arm64/mm/pageattr.c | 9 +++++++++
3 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index fee914c..3f8fbf0 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1198,6 +1198,15 @@ config RODATA_FULL_DEFAULT_ENABLED
This requires the linear region to be mapped down to pages,
which may adversely affect performance in some cases.
+config RODATA_FULL_USE_PTE_CONT
+ bool "Apply PTE_CONT when RODATA_FULL_DEFAULT_ENABLED enabled"
+ depends on RODATA_FULL_DEFAULT_ENABLED
+ default y
+ help
+ Apply PTE_CONT on linear mapping as much as we can when
+ RODATA_FULL_DEFAULT_ENABLED enabled which could decrease the
+ impaction on performance by small pte granularity.
+
config ARM64_SW_TTBR0_PAN
bool "Emulate Privileged Access Never using TTBR0_EL1 switching"
help
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index cfd9deb..8017b17 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -124,15 +124,21 @@ static bool pgattr_change_is_safe(u64 old, u64 new)
* The following mapping attributes may be updated in live
* kernel mappings without the need for break-before-make.
*/
+#ifndef CONFIG_RODATA_FULL_USE_PTE_CONT
pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG;
+#else
+ pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG | PTE_CONT;
+#endif
/* creating or taking down mappings is always safe */
if (old == 0 || new == 0)
return true;
/* live contiguous mappings may not be manipulated at all */
- if ((old | new) & PTE_CONT)
+#ifndef CONFIG_RODATA_FULL_USE_PTE_CONT
+ if (old | new) & PTE_CONT)
return false;
+#endif
/* Transitioning from Non-Global to Global is unsafe */
if (old & ~new & PTE_NG)
@@ -206,7 +212,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
/* use a contiguous mapping if the range is suitably aligned */
if ((((addr | next | phys) & ~CONT_PTE_MASK) == 0) &&
- (flags & NO_CONT_MAPPINGS) == 0)
+ (IS_ENABLED(CONFIG_RODATA_FULL_USE_PTE_CONT) || (flags & NO_CONT_MAPPINGS) == 0))
__prot = __pgprot(pgprot_val(prot) | PTE_CONT);
init_pte(pmdp, addr, next, phys, __prot);
diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c
index a3bacd7..88a87eb 100644
--- a/arch/arm64/mm/pageattr.c
+++ b/arch/arm64/mm/pageattr.c
@@ -99,6 +99,15 @@ static int change_memory_common(unsigned long addr, int numpages,
if (rodata_full && (pgprot_val(set_mask) == PTE_RDONLY ||
pgprot_val(clear_mask) == PTE_RDONLY)) {
for (i = 0; i < area->nr_pages; i++) {
+#ifdef CONFIG_RODATA_FULL_USE_PTE_CONT
+ unsigned long cont_pte_low_bound;
+ unsigned long addr;
+
+ addr = (u64)page_address(area->pages[i]);
+ cont_pte_low_bound = addr & CONT_PTE_MASK;
+ __change_memory_common(cont_pte_low_bound,
+ (~CONT_PTE_MASK + 1), __pgprot(0) , __pgprot(PTE_CONT));
+#endif
__change_memory_common((u64)page_address(area->pages[i]),
PAGE_SIZE, set_mask, clear_mask);
}
--
1.9.1
next reply other threads:[~2021-11-22 5:29 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-22 5:28 Huangzhaoyang [this message]
2021-11-22 8:52 ` [RFC PATCH] arch: arm64: introduce RODATA_FULL_USE_PTE_CONT Ard Biesheuvel
2021-11-22 9:00 ` Zhaoyang Huang
2021-11-22 9:04 ` Ard Biesheuvel
2021-11-22 9:17 ` Zhaoyang Huang
2021-11-22 9:23 ` Ard Biesheuvel
2021-11-22 11:24 ` Zhaoyang Huang
2021-11-22 18:16 ` Ard Biesheuvel
2021-11-23 6:48 ` Zhaoyang Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1637558929-22971-1-git-send-email-huangzhaoyang@gmail.com \
--to=huangzhaoyang@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=ardb@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=christophe.leroy@csgroup.eu \
--cc=jonathan@marek.ca \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=npiggin@gmail.com \
--cc=pasha.tatashin@soleen.com \
--cc=rppt@kernel.org \
--cc=will@kernel.org \
--cc=zhaoyang.huang@unisoc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).