* [PATCH V2 1/3] riscv/kprobe: Optimize the performance of patching single-step slot
2022-09-13 3:34 [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Liao Chang
@ 2022-09-13 3:34 ` Liao Chang
2022-09-13 3:34 ` [PATCH V2 2/3] csky/kprobe: " Liao Chang
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Liao Chang @ 2022-09-13 3:34 UTC (permalink / raw)
To: catalin.marinas, will, guoren, paul.walmsley, palmer, aou,
rostedt, mhiramat, liaochang1, maz, alexandru.elisei
Cc: linux-arm-kernel, linux-kernel, linux-csky, linux-riscv
Single-step slot would not be used until kprobe is enabled, that means
no race condition occurs on it under SMP, hence it is safe to pacth ss
slot without stopping machine.
Signed-off-by: Liao Chang <liaochang1@huawei.com>
---
arch/riscv/kernel/probes/kprobes.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c
index e6e950b7cf32..bc1f39b96e41 100644
--- a/arch/riscv/kernel/probes/kprobes.c
+++ b/arch/riscv/kernel/probes/kprobes.c
@@ -24,12 +24,14 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *);
static void __kprobes arch_prepare_ss_slot(struct kprobe *p)
{
unsigned long offset = GET_INSN_LENGTH(p->opcode);
+ kprobe_opcode_t slot[MAX_INSN_SIZE];
p->ainsn.api.restore = (unsigned long)p->addr + offset;
- patch_text(p->ainsn.api.insn, p->opcode);
- patch_text((void *)((unsigned long)(p->ainsn.api.insn) + offset),
- __BUG_INSN_32);
+ memcpy(slot, &p->opcode, offset);
+ *(kprobe_opcode_t *)((unsigned long)slot + offset) = __BUG_INSN_32;
+ patch_text_nosync(p->ainsn.api.insn, slot,
+ offset + GET_INSN_LENGTH(__BUG_INSN_32));
}
static void __kprobes arch_prepare_simulate(struct kprobe *p)
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH V2 2/3] csky/kprobe: Optimize the performance of patching single-step slot
2022-09-13 3:34 [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Liao Chang
2022-09-13 3:34 ` [PATCH V2 1/3] riscv/kprobe: Optimize the performance of patching single-step slot Liao Chang
@ 2022-09-13 3:34 ` Liao Chang
2022-09-13 3:34 ` [PATCH V2 3/3] arm64/kprobe: " Liao Chang
2022-09-13 11:19 ` [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Masami Hiramatsu
3 siblings, 0 replies; 7+ messages in thread
From: Liao Chang @ 2022-09-13 3:34 UTC (permalink / raw)
To: catalin.marinas, will, guoren, paul.walmsley, palmer, aou,
rostedt, mhiramat, liaochang1, maz, alexandru.elisei
Cc: linux-arm-kernel, linux-kernel, linux-csky, linux-riscv
Single-step slot would not be used until kprobe is enabled, that means
no race condition occurs on it under SMP, hence it is safe to pacth ss
slot without stopping machine.
Signed-off-by: Liao Chang <liaochang1@huawei.com>
---
arch/csky/kernel/probes/kprobes.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/csky/kernel/probes/kprobes.c b/arch/csky/kernel/probes/kprobes.c
index 3c6e5c725d81..4feb5ce16264 100644
--- a/arch/csky/kernel/probes/kprobes.c
+++ b/arch/csky/kernel/probes/kprobes.c
@@ -57,7 +57,11 @@ static void __kprobes arch_prepare_ss_slot(struct kprobe *p)
p->ainsn.api.restore = (unsigned long)p->addr + offset;
- patch_text(p->ainsn.api.insn, p->opcode);
+ memcpy(p->ainsn.api.insn, &p->opcode, offset);
+ dcache_wb_range((unsigned long)p->ainsn.api.insn,
+ (unsigned long)p->ainsn.api.insn + offset);
+ icache_inv_range((unsigned long)p->ainsn.api.insn,
+ (unsigned long)p->ainsn.api.insn + offset);
}
static void __kprobes arch_prepare_simulate(struct kprobe *p)
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH V2 3/3] arm64/kprobe: Optimize the performance of patching single-step slot
2022-09-13 3:34 [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Liao Chang
2022-09-13 3:34 ` [PATCH V2 1/3] riscv/kprobe: Optimize the performance of patching single-step slot Liao Chang
2022-09-13 3:34 ` [PATCH V2 2/3] csky/kprobe: " Liao Chang
@ 2022-09-13 3:34 ` Liao Chang
2022-09-22 13:38 ` Will Deacon
2022-09-13 11:19 ` [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Masami Hiramatsu
3 siblings, 1 reply; 7+ messages in thread
From: Liao Chang @ 2022-09-13 3:34 UTC (permalink / raw)
To: catalin.marinas, will, guoren, paul.walmsley, palmer, aou,
rostedt, mhiramat, liaochang1, maz, alexandru.elisei
Cc: linux-arm-kernel, linux-kernel, linux-csky, linux-riscv
Single-step slot would not be used until kprobe is enabled, that means
no race condition occurs on it under SMP, hence it is safe to pacth ss
slot without stopping machine.
Signed-off-by: Liao Chang <liaochang1@huawei.com>
---
arch/arm64/kernel/probes/kprobes.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index d1d182320245..5902e33fd3b6 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -44,11 +44,10 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *);
static void __kprobes arch_prepare_ss_slot(struct kprobe *p)
{
kprobe_opcode_t *addr = p->ainsn.api.insn;
- void *addrs[] = {addr, addr + 1};
- u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS};
/* prepare insn slot */
- aarch64_insn_patch_text(addrs, insns, 2);
+ aarch64_insn_write(addr, p->opcode);
+ aarch64_insn_write(addr + 1, BRK64_OPCODE_KPROBES_SS);
flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE));
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH V2 3/3] arm64/kprobe: Optimize the performance of patching single-step slot
2022-09-13 3:34 ` [PATCH V2 3/3] arm64/kprobe: " Liao Chang
@ 2022-09-22 13:38 ` Will Deacon
2022-09-23 1:24 ` liaochang (A)
0 siblings, 1 reply; 7+ messages in thread
From: Will Deacon @ 2022-09-22 13:38 UTC (permalink / raw)
To: Liao Chang
Cc: catalin.marinas, guoren, paul.walmsley, palmer, aou, rostedt,
mhiramat, maz, alexandru.elisei, linux-arm-kernel, linux-kernel,
linux-csky, linux-riscv
On Tue, Sep 13, 2022 at 11:34:54AM +0800, Liao Chang wrote:
> Single-step slot would not be used until kprobe is enabled, that means
> no race condition occurs on it under SMP, hence it is safe to pacth ss
> slot without stopping machine.
>
> Signed-off-by: Liao Chang <liaochang1@huawei.com>
> ---
> arch/arm64/kernel/probes/kprobes.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
> index d1d182320245..5902e33fd3b6 100644
> --- a/arch/arm64/kernel/probes/kprobes.c
> +++ b/arch/arm64/kernel/probes/kprobes.c
> @@ -44,11 +44,10 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *);
> static void __kprobes arch_prepare_ss_slot(struct kprobe *p)
> {
> kprobe_opcode_t *addr = p->ainsn.api.insn;
> - void *addrs[] = {addr, addr + 1};
> - u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS};
>
> /* prepare insn slot */
> - aarch64_insn_patch_text(addrs, insns, 2);
> + aarch64_insn_write(addr, p->opcode);
> + aarch64_insn_write(addr + 1, BRK64_OPCODE_KPROBES_SS);
>
> flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE));
Hmm, so it looks like prior to your change we were doing the cache
maintebnance twice: once in aarch64_insn_patch_text() from stop machine
context and then again in the flush_icache_range() call above.
I suppose the cleanest thing would be to drop the flush_icache_range()
call and then use aarch64_insn_patch_text_nosync() instead of
aarch64_insn_write() in your change.
Will
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V2 3/3] arm64/kprobe: Optimize the performance of patching single-step slot
2022-09-22 13:38 ` Will Deacon
@ 2022-09-23 1:24 ` liaochang (A)
0 siblings, 0 replies; 7+ messages in thread
From: liaochang (A) @ 2022-09-23 1:24 UTC (permalink / raw)
To: Will Deacon
Cc: catalin.marinas, guoren, paul.walmsley, palmer, aou, rostedt,
mhiramat, maz, alexandru.elisei, linux-arm-kernel, linux-kernel,
linux-csky, linux-riscv
在 2022/9/22 21:38, Will Deacon 写道:
> On Tue, Sep 13, 2022 at 11:34:54AM +0800, Liao Chang wrote:
>> Single-step slot would not be used until kprobe is enabled, that means
>> no race condition occurs on it under SMP, hence it is safe to pacth ss
>> slot without stopping machine.
>>
>> Signed-off-by: Liao Chang <liaochang1@huawei.com>
>> ---
>> arch/arm64/kernel/probes/kprobes.c | 5 ++---
>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
>> index d1d182320245..5902e33fd3b6 100644
>> --- a/arch/arm64/kernel/probes/kprobes.c
>> +++ b/arch/arm64/kernel/probes/kprobes.c
>> @@ -44,11 +44,10 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *);
>> static void __kprobes arch_prepare_ss_slot(struct kprobe *p)
>> {
>> kprobe_opcode_t *addr = p->ainsn.api.insn;
>> - void *addrs[] = {addr, addr + 1};
>> - u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS};
>>
>> /* prepare insn slot */
>> - aarch64_insn_patch_text(addrs, insns, 2);
>> + aarch64_insn_write(addr, p->opcode);
>> + aarch64_insn_write(addr + 1, BRK64_OPCODE_KPROBES_SS);
>>
>> flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE));
>
> Hmm, so it looks like prior to your change we were doing the cache
> maintebnance twice: once in aarch64_insn_patch_text() from stop machine
> context and then again in the flush_icache_range() call above.
>
> I suppose the cleanest thing would be to drop the flush_icache_range()
> call and then use aarch64_insn_patch_text_nosync() instead of
> aarch64_insn_write() in your change.
OK,I will improve it in next version, thanks.
>
> Will
> .
--
BR,
Liao, Chang
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V2 0/3] kprobe: Optimize the performance of patching ss
2022-09-13 3:34 [PATCH V2 0/3] kprobe: Optimize the performance of patching ss Liao Chang
` (2 preceding siblings ...)
2022-09-13 3:34 ` [PATCH V2 3/3] arm64/kprobe: " Liao Chang
@ 2022-09-13 11:19 ` Masami Hiramatsu
3 siblings, 0 replies; 7+ messages in thread
From: Masami Hiramatsu @ 2022-09-13 11:19 UTC (permalink / raw)
To: Liao Chang
Cc: catalin.marinas, will, guoren, paul.walmsley, palmer, aou,
rostedt, maz, alexandru.elisei, linux-arm-kernel, linux-kernel,
linux-csky, linux-riscv
On Tue, 13 Sep 2022 11:34:51 +0800
Liao Chang <liaochang1@huawei.com> wrote:
> Single-step slot would not be used until kprobe is enabled, that means
> no race condition occurs on it under SMP, hence it is safe to pacth ss
> slot without stopping machine.
All these patches look good to me, since single-step slot must not
be executed until it is prepared.
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
for this series.
But I need Ack by each architecture maintainer.
Thank you,
>
> v2:
> Backport riscv patch to cksy and arm64.
>
> Liao Chang (3):
> riscv/kprobe: Optimize the performance of patching single-step slot
> csky/kprobe: Optimize the performance of patching single-step slot
> arm64/kprobe: Optimize the performance of patching single-step slot
>
> arch/arm64/kernel/probes/kprobes.c | 5 ++---
> arch/csky/kernel/probes/kprobes.c | 6 +++++-
> arch/riscv/kernel/probes/kprobes.c | 8 +++++---
> 3 files changed, 12 insertions(+), 7 deletions(-)
>
> --
> 2.17.1
>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 7+ messages in thread