From: Christophe Leroy <christophe.leroy@csgroup.eu>
To: Benjamin Gray <bgray@linux.ibm.com>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org
Subject: [PATCH v1 2/4] powerpc/code-patching: Speed up page mapping/unmapping
Date: Tue, 27 Sep 2022 16:33:05 +0200 [thread overview]
Message-ID: <3a1e34312b5c169cde846174c275241db85f7406.1647962456.git.christophe.leroy@csgroup.eu> (raw)
Message-ID: <20220927143305.wGMoOO-Ywr03EWjy5qGJb1ZxVc6yuZ36KvEE9ZHjZTo@z> (raw)
In-Reply-To: <cover.1647962456.git.christophe.leroy@csgroup.eu>
Since commit 591b4b268435 ("powerpc/code-patching: Pre-map patch area")
the patch area is premapped so intermediate page tables are already
allocated.
Use __set_pte_at() directly instead of the heavy map_kernel_page(),
at for unmapping just do a pte_clear() followed by a flush.
__set_pte_at() can be used directly without the filters in
set_pte_at() because we are mapping a normal page non executable.
Make sure gcc knows text_poke_area is page aligned in order to
optimise the flush.
This change reduces by 66% the time needed to activate ftrace on
an 8xx (588000 tb ticks instead of 1744000).
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/lib/code-patching.c | 27 ++++++++++++++++-----------
1 file changed, 16 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index f970f189875b..62692c6031bc 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -90,17 +90,20 @@ void __init poking_init(void)
text_area_cpu_down));
}
+static unsigned long get_patch_pfn(void *addr)
+{
+ if (IS_ENABLED(CONFIG_MODULES) && is_vmalloc_or_module_addr(addr))
+ return vmalloc_to_pfn(addr);
+ else
+ return __pa_symbol(addr) >> PAGE_SHIFT;
+}
+
/*
* This can be called for kernel text or a module.
*/
static int map_patch_area(void *addr, unsigned long text_poke_addr)
{
- unsigned long pfn;
-
- if (IS_ENABLED(CONFIG_MODULES) && is_vmalloc_or_module_addr(addr))
- pfn = vmalloc_to_pfn(addr);
- else
- pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+ unsigned long pfn = get_patch_pfn(addr);
return map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
}
@@ -145,17 +148,19 @@ static int __do_patch_instruction(u32 *addr, ppc_inst_t instr)
int err;
u32 *patch_addr;
unsigned long text_poke_addr;
+ pte_t *pte;
+ unsigned long pfn = get_patch_pfn(addr);
- text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
+ text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr & PAGE_MASK;
patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr));
- err = map_patch_area(addr, text_poke_addr);
- if (err)
- return err;
+ pte = virt_to_kpte(text_poke_addr);
+ __set_pte_at(&init_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0);
err = __patch_instruction(addr, instr, patch_addr);
- unmap_patch_area(text_poke_addr);
+ pte_clear(&init_mm, text_poke_addr, pte);
+ flush_tlb_kernel_range(text_poke_addr, text_poke_addr + PAGE_SIZE);
return err;
}
--
2.35.1
next prev parent reply other threads:[~2022-09-27 14:35 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-22 15:40 [PATCH v1 0/4] Kill the time spent in patch_instruction() Christophe Leroy
2022-03-22 15:40 ` [PATCH v1 1/4] powerpc/code-patching: Don't call is_vmalloc_or_module_addr() without CONFIG_MODULES Christophe Leroy
2022-09-27 14:33 ` Christophe Leroy
2022-03-22 15:40 ` Christophe Leroy [this message]
2022-09-27 14:33 ` [PATCH v1 2/4] powerpc/code-patching: Speed up page mapping/unmapping Christophe Leroy
2022-03-22 15:40 ` [PATCH v1 3/4] powerpc/code-patching: Use jump_label for testing freed initmem Christophe Leroy
2022-05-19 2:17 ` Guenter Roeck
2022-05-19 6:27 ` Christophe Leroy
2022-05-19 6:53 ` Christophe Leroy
2022-05-19 17:27 ` Christophe Leroy
2022-09-27 14:33 ` Christophe Leroy
2022-03-22 15:40 ` [PATCH v1 4/4] powerpc/code-patching: Use jump_label to check if poking_init() is done Christophe Leroy
2022-09-27 14:33 ` Christophe Leroy
2022-05-15 10:28 ` [PATCH v1 0/4] Kill the time spent in patch_instruction() Michael Ellerman
2022-05-17 6:44 ` Christophe Leroy
2022-05-17 12:37 ` Michael Ellerman
2022-05-31 6:24 ` Christophe Leroy
2022-06-24 7:06 ` Christophe Leroy
2022-09-27 14:33 ` Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 1/6] powerpc/code-patching: Use pte_offset_kernel() instead of virt_to_kpte() Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 2/6] powerpc/code-patching: Remove #ifdef CONFIG_STRICT_KERNEL_RWX Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 3/6] powerpc/feature-fixups: Refactor entry fixups patching Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 4/6] powerpc/feature-fixups: Refactor other " Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 5/6] powerpc/feature-fixups: Do not patch init section after init Christophe Leroy
2022-09-27 14:33 ` [PATCH v1 6/6] powerpc/code-patching: Remove protection against patching init addresses " Christophe Leroy
2022-09-27 14:38 ` [PATCH v1 0/4] Kill the time spent in patch_instruction() Christophe Leroy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3a1e34312b5c169cde846174c275241db85f7406.1647962456.git.christophe.leroy@csgroup.eu \
--to=christophe.leroy@csgroup.eu \
--cc=benh@kernel.crashing.org \
--cc=bgray@linux.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=paulus@samba.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).