* [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-01-12 9:05 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
- The most horrible bug is preemption panic which found by Andy [1].
Let's disable preemption for ftrace first, and Andy could continue
the ftrace preemption work.
- The "-fpatchable-function-entry= CFLAG" wasted code size
!RISCV_ISA_C.
- The ftrace detour implementation wasted code size.
- When livepatching, the trampoline (ftrace_regs_caller) would not
return to <func_prolog+12> but would rather jump to the new function.
So, "REG_L ra, -SZREG(sp)" would not run and the original return
address would not be restored. The kernel is likely to hang or crash
as a result. (Found by Evgenii Shatokhin [4])
Patches 1,2,3 fixup above problems.
Patches 4,5,6,7 are the features based on reduced detour code
patch, we include them in the series for test and maintenance.
You can directly try it with:
https://github.com/guoren83/linux/tree/ftrace_fixup_v7
Make function graph use ftrace directly [2] (patch 4, 5)
========================================================
In RISC-V architecture, when we enable the ftrace_graph tracer on some
functions, the function tracings on other functions will suffer extra
graph tracing work. In essence, graph_ops isn't limited by its func_hash
due to the global ftrace_graph_[regs]_call label. That should be
corrected.
What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
graph use ftrace directly") that uses graph_ops::func function to
install return_hooker and makes the function called against its
func_hash.
This series of patches makes function graph use ftrace directly for
riscv.
If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
so that it can be replaced with the calling of prepare_ftrace_return by
the enable/disable helper.
As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
necessary regs against the pt_regs layout, so it can reasonably call the
graph_ops::func function - ftrace_graph_func. And
ftrace_graph_[regs]_call
and its enable/disable helper aren't needed.
Test log:
The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
local
qemu-system-riscv64 virt machine. The following is the log during
startup.
```
Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
Nov 15 03:07:13 stage4 kernel: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
```
Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
==============================================
This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
here as the samples for testing DIRECT_CALLS related interface.
First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
register_ftrace_direct[_multi] interfaces allowing user to register
the customed trampoline (direct_caller) as the mcount for one or
more target functions. And modify_ftrace_direct[_multi] are also
provided for modify direct_caller.
At the same time, the samples in ./samples/ftrace/ can be built
as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
and SAMPLE_FTRACE_DIRECT_MULTI selected.
Second, to make the direct_caller and the other ftrace hooks
(eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
register
are nominated to store the address of direct_caller in
ftrace_regs_caller.
After the setting of the address direct_caller by direct_ops->func and
the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.
The following tests have been passed in my local qemu-riscv64 virt
machine.
1. tests with CONFIG_FTRACE_STARTUP_TEST
2. tests of samples/ftrace/ftrace*.ko
3. manual tests with any combination of the following hooks
- function/function_graph tracer
- ftrace*.ko
- kprobe/kretprobe
For your reference, here is the log when function tracer, kretprobe and
ftrace-direct-too.ko co-hooks the handle_mm_fault function.
```
[root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
[root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
[root@stage4 tracing]# echo function > current_tracer
[root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
[root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
[root@stage4 tracing]#
[root@stage4 tracing]# cat trace | tail
cat-388 [000] ...1. 583.051438: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
cat-388 [000] ...2. 583.057930: handle_mm_fault
<-do_page_fault
cat-388 [000] ..... 583.057990: my_direct_func:
handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
cat-388 [000] ...1. 583.058284: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
tail-389 [001] ...2. 583.059062: handle_mm_fault
<-do_page_fault
tail-389 [001] ..... 583.059104: my_direct_func:
handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
tail-389 [001] ...1. 583.059325: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
tail-389 [001] ...2. 583.060371: handle_mm_fault
<-do_page_fault
tail-389 [001] ..... 583.060410: my_direct_func:
handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
tail-389 [001] ...1. 583.060996: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
```
Note1:
The checkpatch.pl will output some warnings on this series, like this
```
WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
this function's name, in a string
111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
+" call my_direct_func2\n"
```
The reason is that checkpatch depends on patch context providing the
function name. In the above warning, my_direct_func2 has some codeline
distance with the changed trunk, so its declaration doesn't come into
the patch, and then the warning jumps out.
You may notice the location of `my_ip` variable changes in the 2nd
patch. I did that for reducing the warnings to some extent. But killing
all the warnings will makes the patch less readable, so I stopped here.
[1] https://lpc.events/event/16/contributions/1171/
[2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
[3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
[4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
Changes in v7:
- Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
- Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
- Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
HAVE_SAMPLE_FTRACE_DIRECT_MULTI
Changes in v6:
https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
- Replace 8 with MCOUNT_INSN_SIZE
- Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
- Add Evgenii Shatokhin comment
Changes in v5:
https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
- Sort Kconfig entries in alphabetical order.
Changes in v4:
https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
- Include [3] for maintenance. [Song Shuai]
Changes in V3:
https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
- Include [2] for maintenance. [Song Shuai]
Changes in V2:
https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
- Add Signed-off for preemption fixup.
Changes in V1:
https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
Andy Chiu (1):
riscv: ftrace: Fixup panic by disabling preemption
Guo Ren (2):
riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
riscv: ftrace: Reduce the detour code size to half
Song Shuai (4):
riscv: ftrace: Add ftrace_graph_func
riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
arch/riscv/Kconfig | 6 +-
arch/riscv/Makefile | 6 +-
arch/riscv/include/asm/ftrace.h | 71 ++++++--
arch/riscv/kernel/ftrace.c | 91 ++++------
arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
samples/ftrace/ftrace-direct-modify.c | 33 ++++
samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
samples/ftrace/ftrace-direct-multi.c | 22 +++
samples/ftrace/ftrace-direct-too.c | 26 +++
samples/ftrace/ftrace-direct.c | 22 +++
10 files changed, 356 insertions(+), 137 deletions(-)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-01-12 9:05 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
- The most horrible bug is preemption panic which found by Andy [1].
Let's disable preemption for ftrace first, and Andy could continue
the ftrace preemption work.
- The "-fpatchable-function-entry= CFLAG" wasted code size
!RISCV_ISA_C.
- The ftrace detour implementation wasted code size.
- When livepatching, the trampoline (ftrace_regs_caller) would not
return to <func_prolog+12> but would rather jump to the new function.
So, "REG_L ra, -SZREG(sp)" would not run and the original return
address would not be restored. The kernel is likely to hang or crash
as a result. (Found by Evgenii Shatokhin [4])
Patches 1,2,3 fixup above problems.
Patches 4,5,6,7 are the features based on reduced detour code
patch, we include them in the series for test and maintenance.
You can directly try it with:
https://github.com/guoren83/linux/tree/ftrace_fixup_v7
Make function graph use ftrace directly [2] (patch 4, 5)
========================================================
In RISC-V architecture, when we enable the ftrace_graph tracer on some
functions, the function tracings on other functions will suffer extra
graph tracing work. In essence, graph_ops isn't limited by its func_hash
due to the global ftrace_graph_[regs]_call label. That should be
corrected.
What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
graph use ftrace directly") that uses graph_ops::func function to
install return_hooker and makes the function called against its
func_hash.
This series of patches makes function graph use ftrace directly for
riscv.
If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
so that it can be replaced with the calling of prepare_ftrace_return by
the enable/disable helper.
As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
necessary regs against the pt_regs layout, so it can reasonably call the
graph_ops::func function - ftrace_graph_func. And
ftrace_graph_[regs]_call
and its enable/disable helper aren't needed.
Test log:
The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
local
qemu-system-riscv64 virt machine. The following is the log during
startup.
```
Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
Nov 15 03:07:13 stage4 kernel: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
```
Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
==============================================
This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
here as the samples for testing DIRECT_CALLS related interface.
First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
register_ftrace_direct[_multi] interfaces allowing user to register
the customed trampoline (direct_caller) as the mcount for one or
more target functions. And modify_ftrace_direct[_multi] are also
provided for modify direct_caller.
At the same time, the samples in ./samples/ftrace/ can be built
as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
and SAMPLE_FTRACE_DIRECT_MULTI selected.
Second, to make the direct_caller and the other ftrace hooks
(eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
register
are nominated to store the address of direct_caller in
ftrace_regs_caller.
After the setting of the address direct_caller by direct_ops->func and
the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.
The following tests have been passed in my local qemu-riscv64 virt
machine.
1. tests with CONFIG_FTRACE_STARTUP_TEST
2. tests of samples/ftrace/ftrace*.ko
3. manual tests with any combination of the following hooks
- function/function_graph tracer
- ftrace*.ko
- kprobe/kretprobe
For your reference, here is the log when function tracer, kretprobe and
ftrace-direct-too.ko co-hooks the handle_mm_fault function.
```
[root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
[root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
[root@stage4 tracing]# echo function > current_tracer
[root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
[root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
[root@stage4 tracing]#
[root@stage4 tracing]# cat trace | tail
cat-388 [000] ...1. 583.051438: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
cat-388 [000] ...2. 583.057930: handle_mm_fault
<-do_page_fault
cat-388 [000] ..... 583.057990: my_direct_func:
handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
cat-388 [000] ...1. 583.058284: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
tail-389 [001] ...2. 583.059062: handle_mm_fault
<-do_page_fault
tail-389 [001] ..... 583.059104: my_direct_func:
handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
tail-389 [001] ...1. 583.059325: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
tail-389 [001] ...2. 583.060371: handle_mm_fault
<-do_page_fault
tail-389 [001] ..... 583.060410: my_direct_func:
handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
tail-389 [001] ...1. 583.060996: myr:
(do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
```
Note1:
The checkpatch.pl will output some warnings on this series, like this
```
WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
this function's name, in a string
111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
+" call my_direct_func2\n"
```
The reason is that checkpatch depends on patch context providing the
function name. In the above warning, my_direct_func2 has some codeline
distance with the changed trunk, so its declaration doesn't come into
the patch, and then the warning jumps out.
You may notice the location of `my_ip` variable changes in the 2nd
patch. I did that for reducing the warnings to some extent. But killing
all the warnings will makes the patch less readable, so I stopped here.
[1] https://lpc.events/event/16/contributions/1171/
[2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
[3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
[4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
Changes in v7:
- Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
- Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
- Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
HAVE_SAMPLE_FTRACE_DIRECT_MULTI
Changes in v6:
https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
- Replace 8 with MCOUNT_INSN_SIZE
- Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
- Add Evgenii Shatokhin comment
Changes in v5:
https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
- Sort Kconfig entries in alphabetical order.
Changes in v4:
https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
- Include [3] for maintenance. [Song Shuai]
Changes in V3:
https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
- Include [2] for maintenance. [Song Shuai]
Changes in V2:
https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
- Add Signed-off for preemption fixup.
Changes in V1:
https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
Andy Chiu (1):
riscv: ftrace: Fixup panic by disabling preemption
Guo Ren (2):
riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
riscv: ftrace: Reduce the detour code size to half
Song Shuai (4):
riscv: ftrace: Add ftrace_graph_func
riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
arch/riscv/Kconfig | 6 +-
arch/riscv/Makefile | 6 +-
arch/riscv/include/asm/ftrace.h | 71 ++++++--
arch/riscv/kernel/ftrace.c | 91 ++++------
arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
samples/ftrace/ftrace-direct-modify.c | 33 ++++
samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
samples/ftrace/ftrace-direct-multi.c | 22 +++
samples/ftrace/ftrace-direct-too.c | 26 +++
samples/ftrace/ftrace-direct.c | 22 +++
10 files changed, 356 insertions(+), 137 deletions(-)
--
2.36.1
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:05 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Andy Chiu <andy.chiu@sifive.com>
In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
forming a jump that jumps to an address over 4K. This may cause errors
if we want to enable kernel preemption and remove dependency from
patching code with stop_machine(). For example, if a task was switched
out on auipc. And, if we changed the ftrace function before it was
switched back, then it would jump to an address that has updated 11:0
bits mixing with previous XLEN:12 part.
p: patched area performed by dynamic ftrace
ftrace_prologue:
p| REG_S ra, -SZREG(sp)
p| auipc ra, 0x? ------------> preempted
...
change ftrace function
...
p| jalr -?(ra) <------------- switched back
p| REG_L ra, -SZREG(sp)
func:
xxx
ret
Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e2b656043abf..ee0d39b26794 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -138,7 +138,7 @@ config RISCV
select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
select HAVE_FUNCTION_GRAPH_TRACER
- select HAVE_FUNCTION_TRACER if !XIP_KERNEL
+ select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
config ARCH_MMAP_RND_BITS_MIN
default 18 if 64BIT
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-12 9:05 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Andy Chiu <andy.chiu@sifive.com>
In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
forming a jump that jumps to an address over 4K. This may cause errors
if we want to enable kernel preemption and remove dependency from
patching code with stop_machine(). For example, if a task was switched
out on auipc. And, if we changed the ftrace function before it was
switched back, then it would jump to an address that has updated 11:0
bits mixing with previous XLEN:12 part.
p: patched area performed by dynamic ftrace
ftrace_prologue:
p| REG_S ra, -SZREG(sp)
p| auipc ra, 0x? ------------> preempted
...
change ftrace function
...
p| jalr -?(ra) <------------- switched back
p| REG_L ra, -SZREG(sp)
func:
xxx
ret
Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e2b656043abf..ee0d39b26794 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -138,7 +138,7 @@ config RISCV
select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
select HAVE_FUNCTION_GRAPH_TRACER
- select HAVE_FUNCTION_TRACER if !XIP_KERNEL
+ select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
config ARCH_MMAP_RND_BITS_MIN
default 18 if 64BIT
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 2/7] riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:05 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
When CONFIG_RISCV_ISA_C=n, -fpatchable-function-entry=8 would generate
more nops than we expect. Because it treat nop opcode as 0x00000013
instead of 0x0001.
Dump of assembler code for function dw_pcie_free_msi:
0xffffffff806fce94 <+0>: sd ra,-8(sp)
0xffffffff806fce98 <+4>: auipc ra,0xff90f
0xffffffff806fce9c <+8>: jalr -684(ra) # 0xffffffff8000bbec
<ftrace_caller>
0xffffffff806fcea0 <+12>: ld ra,-8(sp)
0xffffffff806fcea4 <+16>: nop /* wasted */
0xffffffff806fcea8 <+20>: nop /* wasted */
0xffffffff806fceac <+24>: nop /* wasted */
0xffffffff806fceb0 <+28>: nop /* wasted */
0xffffffff806fceb4 <+0>: addi sp,sp,-48
0xffffffff806fceb8 <+4>: sd s0,32(sp)
0xffffffff806fcebc <+8>: sd s1,24(sp)
0xffffffff806fcec0 <+12>: sd s2,16(sp)
0xffffffff806fcec4 <+16>: sd s3,8(sp)
0xffffffff806fcec8 <+20>: sd ra,40(sp)
0xffffffff806fcecc <+24>: addi s0,sp,48
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 12d91b0a73d8..ea5a91da6897 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -11,7 +11,11 @@ LDFLAGS_vmlinux :=
ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
LDFLAGS_vmlinux := --no-relax
KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+ifeq ($(CONFIG_RISCV_ISA_C),y)
CC_FLAGS_FTRACE := -fpatchable-function-entry=8
+else
+ CC_FLAGS_FTRACE := -fpatchable-function-entry=4
+endif
endif
ifeq ($(CONFIG_CMODEL_MEDLOW),y)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 2/7] riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
@ 2023-01-12 9:05 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
When CONFIG_RISCV_ISA_C=n, -fpatchable-function-entry=8 would generate
more nops than we expect. Because it treat nop opcode as 0x00000013
instead of 0x0001.
Dump of assembler code for function dw_pcie_free_msi:
0xffffffff806fce94 <+0>: sd ra,-8(sp)
0xffffffff806fce98 <+4>: auipc ra,0xff90f
0xffffffff806fce9c <+8>: jalr -684(ra) # 0xffffffff8000bbec
<ftrace_caller>
0xffffffff806fcea0 <+12>: ld ra,-8(sp)
0xffffffff806fcea4 <+16>: nop /* wasted */
0xffffffff806fcea8 <+20>: nop /* wasted */
0xffffffff806fceac <+24>: nop /* wasted */
0xffffffff806fceb0 <+28>: nop /* wasted */
0xffffffff806fceb4 <+0>: addi sp,sp,-48
0xffffffff806fceb8 <+4>: sd s0,32(sp)
0xffffffff806fcebc <+8>: sd s1,24(sp)
0xffffffff806fcec0 <+12>: sd s2,16(sp)
0xffffffff806fcec4 <+16>: sd s3,8(sp)
0xffffffff806fcec8 <+20>: sd ra,40(sp)
0xffffffff806fcecc <+24>: addi s0,sp,48
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 12d91b0a73d8..ea5a91da6897 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -11,7 +11,11 @@ LDFLAGS_vmlinux :=
ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
LDFLAGS_vmlinux := --no-relax
KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+ifeq ($(CONFIG_RISCV_ISA_C),y)
CC_FLAGS_FTRACE := -fpatchable-function-entry=8
+else
+ CC_FLAGS_FTRACE := -fpatchable-function-entry=4
+endif
endif
ifeq ($(CONFIG_CMODEL_MEDLOW),y)
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 3/7] riscv: ftrace: Reduce the detour code size to half
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:05 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Use a temporary register to reduce the size of detour code from 16 bytes to
8 bytes. The previous implementation is from 'commit afc76b8b8011 ("riscv:
Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")'.
Before the patch:
<func_prolog>:
0: REG_S ra, -SZREG(sp)
4: auipc ra, ?
8: jalr ?(ra)
12: REG_L ra, -SZREG(sp)
(func_boddy)
After the patch:
<func_prolog>:
0: auipc t0, ?
4: jalr t0, ?(t0)
(func_boddy)
This patch not just reduces the size of detour code, but also fixes an
important issue:
An Ftrace callback registered with FTRACE_OPS_FL_IPMODIFY flag can
actually change the instruction pointer, e.g. to "replace" the given
kernel function with a new one, which is needed for livepatching, etc.
In this case, the trampoline (ftrace_regs_caller) would not return to
<func_prolog+12> but would rather jump to the new function. So, "REG_L
ra, -SZREG(sp)" would not run and the original return address would not
be restored. The kernel is likely to hang or crash as a result.
This can be easily demonstrated if one tries to "replace", say,
cmdline_proc_show() with a new function with the same signature using
instruction_pointer_set(&fregs->regs, new_func_addr) in the Ftrace
callback.
Link: https://lore.kernel.org/linux-riscv/20221122075440.1165172-1-suagrfillet@gmail.com/
Link: https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
Co-developed-by: Song Shuai <suagrfillet@gmail.com>
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: Evgenii Shatokhin <e.shatokhin@yadro.com>
---
arch/riscv/Makefile | 4 +-
arch/riscv/include/asm/ftrace.h | 50 +++++++++++++++++++------
arch/riscv/kernel/ftrace.c | 65 ++++++++++-----------------------
arch/riscv/kernel/mcount-dyn.S | 42 ++++++++-------------
4 files changed, 75 insertions(+), 86 deletions(-)
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index ea5a91da6897..3c9aaf67ed79 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -12,9 +12,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
LDFLAGS_vmlinux := --no-relax
KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
ifeq ($(CONFIG_RISCV_ISA_C),y)
- CC_FLAGS_FTRACE := -fpatchable-function-entry=8
-else
CC_FLAGS_FTRACE := -fpatchable-function-entry=4
+else
+ CC_FLAGS_FTRACE := -fpatchable-function-entry=2
endif
endif
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 04dad3380041..9e73922e1e2e 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -42,6 +42,14 @@ struct dyn_arch_ftrace {
* 2) jalr: setting low-12 offset to ra, jump to ra, and set ra to
* return address (original pc + 4)
*
+ *<ftrace enable>:
+ * 0: auipc t0/ra, 0x?
+ * 4: jalr t0/ra, ?(t0/ra)
+ *
+ *<ftrace disable>:
+ * 0: nop
+ * 4: nop
+ *
* Dynamic ftrace generates probes to call sites, so we must deal with
* both auipc and jalr at the same time.
*/
@@ -52,25 +60,43 @@ struct dyn_arch_ftrace {
#define AUIPC_OFFSET_MASK (0xfffff000)
#define AUIPC_PAD (0x00001000)
#define JALR_SHIFT 20
-#define JALR_BASIC (0x000080e7)
-#define AUIPC_BASIC (0x00000097)
+#define JALR_RA (0x000080e7)
+#define AUIPC_RA (0x00000097)
+#define JALR_T0 (0x000282e7)
+#define AUIPC_T0 (0x00000297)
#define NOP4 (0x00000013)
-#define make_call(caller, callee, call) \
+#define to_jalr_t0(offset) \
+ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0)
+
+#define to_auipc_t0(offset) \
+ ((offset & JALR_SIGN_MASK) ? \
+ (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_T0) : \
+ ((offset & AUIPC_OFFSET_MASK) | AUIPC_T0))
+
+#define make_call_t0(caller, callee, call) \
do { \
- call[0] = to_auipc_insn((unsigned int)((unsigned long)callee - \
- (unsigned long)caller)); \
- call[1] = to_jalr_insn((unsigned int)((unsigned long)callee - \
- (unsigned long)caller)); \
+ unsigned int offset = \
+ (unsigned long) callee - (unsigned long) caller; \
+ call[0] = to_auipc_t0(offset); \
+ call[1] = to_jalr_t0(offset); \
} while (0)
-#define to_jalr_insn(offset) \
- (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_BASIC)
+#define to_jalr_ra(offset) \
+ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_RA)
-#define to_auipc_insn(offset) \
+#define to_auipc_ra(offset) \
((offset & JALR_SIGN_MASK) ? \
- (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_BASIC) : \
- ((offset & AUIPC_OFFSET_MASK) | AUIPC_BASIC))
+ (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_RA) : \
+ ((offset & AUIPC_OFFSET_MASK) | AUIPC_RA))
+
+#define make_call_ra(caller, callee, call) \
+do { \
+ unsigned int offset = \
+ (unsigned long) callee - (unsigned long) caller; \
+ call[0] = to_auipc_ra(offset); \
+ call[1] = to_jalr_ra(offset); \
+} while (0)
/*
* Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here.
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index 2086f6585773..5bff37af4770 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -55,12 +55,15 @@ static int ftrace_check_current_call(unsigned long hook_pos,
}
static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
- bool enable)
+ bool enable, bool ra)
{
unsigned int call[2];
unsigned int nops[2] = {NOP4, NOP4};
- make_call(hook_pos, target, call);
+ if (ra)
+ make_call_ra(hook_pos, target, call);
+ else
+ make_call_t0(hook_pos, target, call);
/* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
if (patch_text_nosync
@@ -70,42 +73,13 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
return 0;
}
-/*
- * Put 5 instructions with 16 bytes at the front of function within
- * patchable function entry nops' area.
- *
- * 0: REG_S ra, -SZREG(sp)
- * 1: auipc ra, 0x?
- * 2: jalr -?(ra)
- * 3: REG_L ra, -SZREG(sp)
- *
- * So the opcodes is:
- * 0: 0xfe113c23 (sd)/0xfe112e23 (sw)
- * 1: 0x???????? -> auipc
- * 2: 0x???????? -> jalr
- * 3: 0xff813083 (ld)/0xffc12083 (lw)
- */
-#if __riscv_xlen == 64
-#define INSN0 0xfe113c23
-#define INSN3 0xff813083
-#elif __riscv_xlen == 32
-#define INSN0 0xfe112e23
-#define INSN3 0xffc12083
-#endif
-
-#define FUNC_ENTRY_SIZE 16
-#define FUNC_ENTRY_JMP 4
-
int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
- unsigned int call[4] = {INSN0, 0, 0, INSN3};
- unsigned long target = addr;
- unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
+ unsigned int call[2];
- call[1] = to_auipc_insn((unsigned int)(target - caller));
- call[2] = to_jalr_insn((unsigned int)(target - caller));
+ make_call_t0(rec->ip, addr, call);
- if (patch_text_nosync((void *)rec->ip, call, FUNC_ENTRY_SIZE))
+ if (patch_text_nosync((void *)rec->ip, call, MCOUNT_INSN_SIZE))
return -EPERM;
return 0;
@@ -114,15 +88,14 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
unsigned long addr)
{
- unsigned int nops[4] = {NOP4, NOP4, NOP4, NOP4};
+ unsigned int nops[2] = {NOP4, NOP4};
- if (patch_text_nosync((void *)rec->ip, nops, FUNC_ENTRY_SIZE))
+ if (patch_text_nosync((void *)rec->ip, nops, MCOUNT_INSN_SIZE))
return -EPERM;
return 0;
}
-
/*
* This is called early on, and isn't wrapped by
* ftrace_arch_code_modify_{prepare,post_process}() and therefor doesn't hold
@@ -144,10 +117,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
int ftrace_update_ftrace_func(ftrace_func_t func)
{
int ret = __ftrace_modify_call((unsigned long)&ftrace_call,
- (unsigned long)func, true);
+ (unsigned long)func, true, true);
if (!ret) {
ret = __ftrace_modify_call((unsigned long)&ftrace_regs_call,
- (unsigned long)func, true);
+ (unsigned long)func, true, true);
}
return ret;
@@ -159,16 +132,16 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
unsigned long addr)
{
unsigned int call[2];
- unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
+ unsigned long caller = rec->ip;
int ret;
- make_call(caller, old_addr, call);
+ make_call_t0(caller, old_addr, call);
ret = ftrace_check_current_call(caller, call);
if (ret)
return ret;
- return __ftrace_modify_call(caller, addr, true);
+ return __ftrace_modify_call(caller, addr, true, false);
}
#endif
@@ -203,12 +176,12 @@ int ftrace_enable_ftrace_graph_caller(void)
int ret;
ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, true);
+ (unsigned long)&prepare_ftrace_return, true, true);
if (ret)
return ret;
return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
- (unsigned long)&prepare_ftrace_return, true);
+ (unsigned long)&prepare_ftrace_return, true, true);
}
int ftrace_disable_ftrace_graph_caller(void)
@@ -216,12 +189,12 @@ int ftrace_disable_ftrace_graph_caller(void)
int ret;
ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, false);
+ (unsigned long)&prepare_ftrace_return, false, true);
if (ret)
return ret;
return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
- (unsigned long)&prepare_ftrace_return, false);
+ (unsigned long)&prepare_ftrace_return, false, true);
}
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index d171eca623b6..125de818d1ba 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -13,8 +13,8 @@
.text
-#define FENTRY_RA_OFFSET 12
-#define ABI_SIZE_ON_STACK 72
+#define FENTRY_RA_OFFSET 8
+#define ABI_SIZE_ON_STACK 80
#define ABI_A0 0
#define ABI_A1 8
#define ABI_A2 16
@@ -23,10 +23,10 @@
#define ABI_A5 40
#define ABI_A6 48
#define ABI_A7 56
-#define ABI_RA 64
+#define ABI_T0 64
+#define ABI_RA 72
.macro SAVE_ABI
- addi sp, sp, -SZREG
addi sp, sp, -ABI_SIZE_ON_STACK
REG_S a0, ABI_A0(sp)
@@ -37,6 +37,7 @@
REG_S a5, ABI_A5(sp)
REG_S a6, ABI_A6(sp)
REG_S a7, ABI_A7(sp)
+ REG_S t0, ABI_T0(sp)
REG_S ra, ABI_RA(sp)
.endm
@@ -49,24 +50,18 @@
REG_L a5, ABI_A5(sp)
REG_L a6, ABI_A6(sp)
REG_L a7, ABI_A7(sp)
+ REG_L t0, ABI_T0(sp)
REG_L ra, ABI_RA(sp)
addi sp, sp, ABI_SIZE_ON_STACK
- addi sp, sp, SZREG
.endm
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
.macro SAVE_ALL
- addi sp, sp, -SZREG
addi sp, sp, -PT_SIZE_ON_STACK
- REG_S x1, PT_EPC(sp)
- addi sp, sp, PT_SIZE_ON_STACK
- REG_L x1, (sp)
- addi sp, sp, -PT_SIZE_ON_STACK
+ REG_S t0, PT_EPC(sp)
REG_S x1, PT_RA(sp)
- REG_L x1, PT_EPC(sp)
-
REG_S x2, PT_SP(sp)
REG_S x3, PT_GP(sp)
REG_S x4, PT_TP(sp)
@@ -100,15 +95,11 @@
.endm
.macro RESTORE_ALL
+ REG_L t0, PT_EPC(sp)
REG_L x1, PT_RA(sp)
- addi sp, sp, PT_SIZE_ON_STACK
- REG_S x1, (sp)
- addi sp, sp, -PT_SIZE_ON_STACK
- REG_L x1, PT_EPC(sp)
REG_L x2, PT_SP(sp)
REG_L x3, PT_GP(sp)
REG_L x4, PT_TP(sp)
- REG_L x5, PT_T0(sp)
REG_L x6, PT_T1(sp)
REG_L x7, PT_T2(sp)
REG_L x8, PT_S0(sp)
@@ -137,17 +128,16 @@
REG_L x31, PT_T6(sp)
addi sp, sp, PT_SIZE_ON_STACK
- addi sp, sp, SZREG
.endm
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_caller)
SAVE_ABI
- addi a0, ra, -FENTRY_RA_OFFSET
+ addi a0, t0, -FENTRY_RA_OFFSET
la a1, function_trace_op
REG_L a2, 0(a1)
- REG_L a1, ABI_SIZE_ON_STACK(sp)
+ mv a1, ra
mv a3, sp
ftrace_call:
@@ -155,8 +145,8 @@ ftrace_call:
call ftrace_stub
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
- addi a0, sp, ABI_SIZE_ON_STACK
- REG_L a1, ABI_RA(sp)
+ addi a0, sp, ABI_RA
+ REG_L a1, ABI_T0(sp)
addi a1, a1, -FENTRY_RA_OFFSET
#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
mv a2, s0
@@ -166,17 +156,17 @@ ftrace_graph_call:
call ftrace_stub
#endif
RESTORE_ABI
- ret
+ jr t0
ENDPROC(ftrace_caller)
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
ENTRY(ftrace_regs_caller)
SAVE_ALL
- addi a0, ra, -FENTRY_RA_OFFSET
+ addi a0, t0, -FENTRY_RA_OFFSET
la a1, function_trace_op
REG_L a2, 0(a1)
- REG_L a1, PT_SIZE_ON_STACK(sp)
+ mv a1, ra
mv a3, sp
ftrace_regs_call:
@@ -196,6 +186,6 @@ ftrace_graph_regs_call:
#endif
RESTORE_ALL
- ret
+ jr t0
ENDPROC(ftrace_regs_caller)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 3/7] riscv: ftrace: Reduce the detour code size to half
@ 2023-01-12 9:05 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:05 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Use a temporary register to reduce the size of detour code from 16 bytes to
8 bytes. The previous implementation is from 'commit afc76b8b8011 ("riscv:
Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")'.
Before the patch:
<func_prolog>:
0: REG_S ra, -SZREG(sp)
4: auipc ra, ?
8: jalr ?(ra)
12: REG_L ra, -SZREG(sp)
(func_boddy)
After the patch:
<func_prolog>:
0: auipc t0, ?
4: jalr t0, ?(t0)
(func_boddy)
This patch not just reduces the size of detour code, but also fixes an
important issue:
An Ftrace callback registered with FTRACE_OPS_FL_IPMODIFY flag can
actually change the instruction pointer, e.g. to "replace" the given
kernel function with a new one, which is needed for livepatching, etc.
In this case, the trampoline (ftrace_regs_caller) would not return to
<func_prolog+12> but would rather jump to the new function. So, "REG_L
ra, -SZREG(sp)" would not run and the original return address would not
be restored. The kernel is likely to hang or crash as a result.
This can be easily demonstrated if one tries to "replace", say,
cmdline_proc_show() with a new function with the same signature using
instruction_pointer_set(&fregs->regs, new_func_addr) in the Ftrace
callback.
Link: https://lore.kernel.org/linux-riscv/20221122075440.1165172-1-suagrfillet@gmail.com/
Link: https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
Co-developed-by: Song Shuai <suagrfillet@gmail.com>
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: Evgenii Shatokhin <e.shatokhin@yadro.com>
---
arch/riscv/Makefile | 4 +-
arch/riscv/include/asm/ftrace.h | 50 +++++++++++++++++++------
arch/riscv/kernel/ftrace.c | 65 ++++++++++-----------------------
arch/riscv/kernel/mcount-dyn.S | 42 ++++++++-------------
4 files changed, 75 insertions(+), 86 deletions(-)
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index ea5a91da6897..3c9aaf67ed79 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -12,9 +12,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
LDFLAGS_vmlinux := --no-relax
KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
ifeq ($(CONFIG_RISCV_ISA_C),y)
- CC_FLAGS_FTRACE := -fpatchable-function-entry=8
-else
CC_FLAGS_FTRACE := -fpatchable-function-entry=4
+else
+ CC_FLAGS_FTRACE := -fpatchable-function-entry=2
endif
endif
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 04dad3380041..9e73922e1e2e 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -42,6 +42,14 @@ struct dyn_arch_ftrace {
* 2) jalr: setting low-12 offset to ra, jump to ra, and set ra to
* return address (original pc + 4)
*
+ *<ftrace enable>:
+ * 0: auipc t0/ra, 0x?
+ * 4: jalr t0/ra, ?(t0/ra)
+ *
+ *<ftrace disable>:
+ * 0: nop
+ * 4: nop
+ *
* Dynamic ftrace generates probes to call sites, so we must deal with
* both auipc and jalr at the same time.
*/
@@ -52,25 +60,43 @@ struct dyn_arch_ftrace {
#define AUIPC_OFFSET_MASK (0xfffff000)
#define AUIPC_PAD (0x00001000)
#define JALR_SHIFT 20
-#define JALR_BASIC (0x000080e7)
-#define AUIPC_BASIC (0x00000097)
+#define JALR_RA (0x000080e7)
+#define AUIPC_RA (0x00000097)
+#define JALR_T0 (0x000282e7)
+#define AUIPC_T0 (0x00000297)
#define NOP4 (0x00000013)
-#define make_call(caller, callee, call) \
+#define to_jalr_t0(offset) \
+ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0)
+
+#define to_auipc_t0(offset) \
+ ((offset & JALR_SIGN_MASK) ? \
+ (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_T0) : \
+ ((offset & AUIPC_OFFSET_MASK) | AUIPC_T0))
+
+#define make_call_t0(caller, callee, call) \
do { \
- call[0] = to_auipc_insn((unsigned int)((unsigned long)callee - \
- (unsigned long)caller)); \
- call[1] = to_jalr_insn((unsigned int)((unsigned long)callee - \
- (unsigned long)caller)); \
+ unsigned int offset = \
+ (unsigned long) callee - (unsigned long) caller; \
+ call[0] = to_auipc_t0(offset); \
+ call[1] = to_jalr_t0(offset); \
} while (0)
-#define to_jalr_insn(offset) \
- (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_BASIC)
+#define to_jalr_ra(offset) \
+ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_RA)
-#define to_auipc_insn(offset) \
+#define to_auipc_ra(offset) \
((offset & JALR_SIGN_MASK) ? \
- (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_BASIC) : \
- ((offset & AUIPC_OFFSET_MASK) | AUIPC_BASIC))
+ (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_RA) : \
+ ((offset & AUIPC_OFFSET_MASK) | AUIPC_RA))
+
+#define make_call_ra(caller, callee, call) \
+do { \
+ unsigned int offset = \
+ (unsigned long) callee - (unsigned long) caller; \
+ call[0] = to_auipc_ra(offset); \
+ call[1] = to_jalr_ra(offset); \
+} while (0)
/*
* Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here.
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index 2086f6585773..5bff37af4770 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -55,12 +55,15 @@ static int ftrace_check_current_call(unsigned long hook_pos,
}
static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
- bool enable)
+ bool enable, bool ra)
{
unsigned int call[2];
unsigned int nops[2] = {NOP4, NOP4};
- make_call(hook_pos, target, call);
+ if (ra)
+ make_call_ra(hook_pos, target, call);
+ else
+ make_call_t0(hook_pos, target, call);
/* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
if (patch_text_nosync
@@ -70,42 +73,13 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
return 0;
}
-/*
- * Put 5 instructions with 16 bytes at the front of function within
- * patchable function entry nops' area.
- *
- * 0: REG_S ra, -SZREG(sp)
- * 1: auipc ra, 0x?
- * 2: jalr -?(ra)
- * 3: REG_L ra, -SZREG(sp)
- *
- * So the opcodes is:
- * 0: 0xfe113c23 (sd)/0xfe112e23 (sw)
- * 1: 0x???????? -> auipc
- * 2: 0x???????? -> jalr
- * 3: 0xff813083 (ld)/0xffc12083 (lw)
- */
-#if __riscv_xlen == 64
-#define INSN0 0xfe113c23
-#define INSN3 0xff813083
-#elif __riscv_xlen == 32
-#define INSN0 0xfe112e23
-#define INSN3 0xffc12083
-#endif
-
-#define FUNC_ENTRY_SIZE 16
-#define FUNC_ENTRY_JMP 4
-
int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
- unsigned int call[4] = {INSN0, 0, 0, INSN3};
- unsigned long target = addr;
- unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
+ unsigned int call[2];
- call[1] = to_auipc_insn((unsigned int)(target - caller));
- call[2] = to_jalr_insn((unsigned int)(target - caller));
+ make_call_t0(rec->ip, addr, call);
- if (patch_text_nosync((void *)rec->ip, call, FUNC_ENTRY_SIZE))
+ if (patch_text_nosync((void *)rec->ip, call, MCOUNT_INSN_SIZE))
return -EPERM;
return 0;
@@ -114,15 +88,14 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
unsigned long addr)
{
- unsigned int nops[4] = {NOP4, NOP4, NOP4, NOP4};
+ unsigned int nops[2] = {NOP4, NOP4};
- if (patch_text_nosync((void *)rec->ip, nops, FUNC_ENTRY_SIZE))
+ if (patch_text_nosync((void *)rec->ip, nops, MCOUNT_INSN_SIZE))
return -EPERM;
return 0;
}
-
/*
* This is called early on, and isn't wrapped by
* ftrace_arch_code_modify_{prepare,post_process}() and therefor doesn't hold
@@ -144,10 +117,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
int ftrace_update_ftrace_func(ftrace_func_t func)
{
int ret = __ftrace_modify_call((unsigned long)&ftrace_call,
- (unsigned long)func, true);
+ (unsigned long)func, true, true);
if (!ret) {
ret = __ftrace_modify_call((unsigned long)&ftrace_regs_call,
- (unsigned long)func, true);
+ (unsigned long)func, true, true);
}
return ret;
@@ -159,16 +132,16 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
unsigned long addr)
{
unsigned int call[2];
- unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
+ unsigned long caller = rec->ip;
int ret;
- make_call(caller, old_addr, call);
+ make_call_t0(caller, old_addr, call);
ret = ftrace_check_current_call(caller, call);
if (ret)
return ret;
- return __ftrace_modify_call(caller, addr, true);
+ return __ftrace_modify_call(caller, addr, true, false);
}
#endif
@@ -203,12 +176,12 @@ int ftrace_enable_ftrace_graph_caller(void)
int ret;
ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, true);
+ (unsigned long)&prepare_ftrace_return, true, true);
if (ret)
return ret;
return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
- (unsigned long)&prepare_ftrace_return, true);
+ (unsigned long)&prepare_ftrace_return, true, true);
}
int ftrace_disable_ftrace_graph_caller(void)
@@ -216,12 +189,12 @@ int ftrace_disable_ftrace_graph_caller(void)
int ret;
ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, false);
+ (unsigned long)&prepare_ftrace_return, false, true);
if (ret)
return ret;
return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
- (unsigned long)&prepare_ftrace_return, false);
+ (unsigned long)&prepare_ftrace_return, false, true);
}
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index d171eca623b6..125de818d1ba 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -13,8 +13,8 @@
.text
-#define FENTRY_RA_OFFSET 12
-#define ABI_SIZE_ON_STACK 72
+#define FENTRY_RA_OFFSET 8
+#define ABI_SIZE_ON_STACK 80
#define ABI_A0 0
#define ABI_A1 8
#define ABI_A2 16
@@ -23,10 +23,10 @@
#define ABI_A5 40
#define ABI_A6 48
#define ABI_A7 56
-#define ABI_RA 64
+#define ABI_T0 64
+#define ABI_RA 72
.macro SAVE_ABI
- addi sp, sp, -SZREG
addi sp, sp, -ABI_SIZE_ON_STACK
REG_S a0, ABI_A0(sp)
@@ -37,6 +37,7 @@
REG_S a5, ABI_A5(sp)
REG_S a6, ABI_A6(sp)
REG_S a7, ABI_A7(sp)
+ REG_S t0, ABI_T0(sp)
REG_S ra, ABI_RA(sp)
.endm
@@ -49,24 +50,18 @@
REG_L a5, ABI_A5(sp)
REG_L a6, ABI_A6(sp)
REG_L a7, ABI_A7(sp)
+ REG_L t0, ABI_T0(sp)
REG_L ra, ABI_RA(sp)
addi sp, sp, ABI_SIZE_ON_STACK
- addi sp, sp, SZREG
.endm
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
.macro SAVE_ALL
- addi sp, sp, -SZREG
addi sp, sp, -PT_SIZE_ON_STACK
- REG_S x1, PT_EPC(sp)
- addi sp, sp, PT_SIZE_ON_STACK
- REG_L x1, (sp)
- addi sp, sp, -PT_SIZE_ON_STACK
+ REG_S t0, PT_EPC(sp)
REG_S x1, PT_RA(sp)
- REG_L x1, PT_EPC(sp)
-
REG_S x2, PT_SP(sp)
REG_S x3, PT_GP(sp)
REG_S x4, PT_TP(sp)
@@ -100,15 +95,11 @@
.endm
.macro RESTORE_ALL
+ REG_L t0, PT_EPC(sp)
REG_L x1, PT_RA(sp)
- addi sp, sp, PT_SIZE_ON_STACK
- REG_S x1, (sp)
- addi sp, sp, -PT_SIZE_ON_STACK
- REG_L x1, PT_EPC(sp)
REG_L x2, PT_SP(sp)
REG_L x3, PT_GP(sp)
REG_L x4, PT_TP(sp)
- REG_L x5, PT_T0(sp)
REG_L x6, PT_T1(sp)
REG_L x7, PT_T2(sp)
REG_L x8, PT_S0(sp)
@@ -137,17 +128,16 @@
REG_L x31, PT_T6(sp)
addi sp, sp, PT_SIZE_ON_STACK
- addi sp, sp, SZREG
.endm
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_caller)
SAVE_ABI
- addi a0, ra, -FENTRY_RA_OFFSET
+ addi a0, t0, -FENTRY_RA_OFFSET
la a1, function_trace_op
REG_L a2, 0(a1)
- REG_L a1, ABI_SIZE_ON_STACK(sp)
+ mv a1, ra
mv a3, sp
ftrace_call:
@@ -155,8 +145,8 @@ ftrace_call:
call ftrace_stub
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
- addi a0, sp, ABI_SIZE_ON_STACK
- REG_L a1, ABI_RA(sp)
+ addi a0, sp, ABI_RA
+ REG_L a1, ABI_T0(sp)
addi a1, a1, -FENTRY_RA_OFFSET
#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
mv a2, s0
@@ -166,17 +156,17 @@ ftrace_graph_call:
call ftrace_stub
#endif
RESTORE_ABI
- ret
+ jr t0
ENDPROC(ftrace_caller)
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
ENTRY(ftrace_regs_caller)
SAVE_ALL
- addi a0, ra, -FENTRY_RA_OFFSET
+ addi a0, t0, -FENTRY_RA_OFFSET
la a1, function_trace_op
REG_L a2, 0(a1)
- REG_L a1, PT_SIZE_ON_STACK(sp)
+ mv a1, ra
mv a3, sp
ftrace_regs_call:
@@ -196,6 +186,6 @@ ftrace_graph_regs_call:
#endif
RESTORE_ALL
- ret
+ jr t0
ENDPROC(ftrace_regs_caller)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 4/7] riscv: ftrace: Add ftrace_graph_func
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:06 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
Here implements ftrace_graph_func as the function graph tracing function
with FTRACE_WITH_REGS defined.
function_graph_func gets the point of the parent IP and the frame pointer
from fregs and call prepare_ftrace_return for function graph tracing.
If FTRACE_WITH_REGS isn't defined, the enable/disable helpers of
ftrace_graph_[regs]_call are revised for serving only ftrace_graph_call
in the !FTRACE_WITH_REGS version ftrace_caller.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/ftrace.h | 13 ++-
arch/riscv/kernel/ftrace.c | 30 +++----
arch/riscv/kernel/mcount-dyn.S | 139 +++++++++++++++++++++++---------
3 files changed, 126 insertions(+), 56 deletions(-)
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 9e73922e1e2e..84f856a3286e 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -107,8 +107,17 @@ do { \
struct dyn_ftrace;
int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
#define ftrace_init_nop ftrace_init_nop
-#endif
-#endif
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+struct ftrace_ops;
+struct ftrace_regs;
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs);
+#define ftrace_graph_func ftrace_graph_func
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* _ASM_RISCV_FTRACE_H */
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index 5bff37af4770..95e14d8161a4 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -169,32 +169,28 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
}
#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+ struct pt_regs *regs = arch_ftrace_get_regs(fregs);
+ unsigned long *parent = (unsigned long *)®s->ra;
+
+ prepare_ftrace_return(parent, ip, frame_pointer(regs));
+}
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
extern void ftrace_graph_call(void);
-extern void ftrace_graph_regs_call(void);
int ftrace_enable_ftrace_graph_caller(void)
{
- int ret;
-
- ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, true, true);
- if (ret)
- return ret;
-
- return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+ return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
(unsigned long)&prepare_ftrace_return, true, true);
}
int ftrace_disable_ftrace_graph_caller(void)
{
- int ret;
-
- ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, false, true);
- if (ret)
- return ret;
-
- return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+ return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
(unsigned long)&prepare_ftrace_return, false, true);
}
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index 125de818d1ba..f26e9f6e2fed 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -57,19 +57,52 @@
.endm
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
- .macro SAVE_ALL
+
+/**
+* SAVE_ABI_REGS - save regs against the pt_regs struct
+*
+* @all: tell if saving all the regs
+*
+* If all is set, all the regs will be saved, otherwise only ABI
+* related regs (a0-a7,epc,ra and optional s0) will be saved.
+*
+* After the stack is established,
+*
+* 0(sp) stores the PC of the traced function which can be accessed
+* by &(fregs)->regs->epc in tracing function. Note that the real
+* function entry address should be computed with -FENTRY_RA_OFFSET.
+*
+* 8(sp) stores the function return address (i.e. parent IP) that
+* can be accessed by &(fregs)->regs->ra in tracing function.
+*
+* The other regs are saved at the respective localtion and accessed
+* by the respective pt_regs member.
+*
+* Here is the layout of stack for your reference.
+*
+* PT_SIZE_ON_STACK -> +++++++++
+* + ..... +
+* + t3-t6 +
+* + s2-s11+
+* + a0-a7 + --++++-> ftrace_caller saved
+* + s1 + +
+* + s0 + --+
+* + t0-t2 + +
+* + tp + +
+* + gp + +
+* + sp + +
+* + ra + --+ // parent IP
+* sp -> + epc + --+ // PC
+* +++++++++
+**/
+ .macro SAVE_ABI_REGS, all=0
addi sp, sp, -PT_SIZE_ON_STACK
REG_S t0, PT_EPC(sp)
REG_S x1, PT_RA(sp)
- REG_S x2, PT_SP(sp)
- REG_S x3, PT_GP(sp)
- REG_S x4, PT_TP(sp)
- REG_S x5, PT_T0(sp)
- REG_S x6, PT_T1(sp)
- REG_S x7, PT_T2(sp)
- REG_S x8, PT_S0(sp)
- REG_S x9, PT_S1(sp)
+
+ // always save the ABI regs
+
REG_S x10, PT_A0(sp)
REG_S x11, PT_A1(sp)
REG_S x12, PT_A2(sp)
@@ -78,6 +111,18 @@
REG_S x15, PT_A5(sp)
REG_S x16, PT_A6(sp)
REG_S x17, PT_A7(sp)
+
+ // save the leftover regs
+
+ .if \all == 1
+ REG_S x2, PT_SP(sp)
+ REG_S x3, PT_GP(sp)
+ REG_S x4, PT_TP(sp)
+ REG_S x5, PT_T0(sp)
+ REG_S x6, PT_T1(sp)
+ REG_S x7, PT_T2(sp)
+ REG_S x8, PT_S0(sp)
+ REG_S x9, PT_S1(sp)
REG_S x18, PT_S2(sp)
REG_S x19, PT_S3(sp)
REG_S x20, PT_S4(sp)
@@ -92,18 +137,19 @@
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
+
+ // save s0 if FP_TEST defined
+
+ .else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+ REG_S x8, PT_S0(sp)
+#endif
+ .endif
.endm
- .macro RESTORE_ALL
+ .macro RESTORE_ABI_REGS, all=0
REG_L t0, PT_EPC(sp)
REG_L x1, PT_RA(sp)
- REG_L x2, PT_SP(sp)
- REG_L x3, PT_GP(sp)
- REG_L x4, PT_TP(sp)
- REG_L x6, PT_T1(sp)
- REG_L x7, PT_T2(sp)
- REG_L x8, PT_S0(sp)
- REG_L x9, PT_S1(sp)
REG_L x10, PT_A0(sp)
REG_L x11, PT_A1(sp)
REG_L x12, PT_A2(sp)
@@ -112,6 +158,15 @@
REG_L x15, PT_A5(sp)
REG_L x16, PT_A6(sp)
REG_L x17, PT_A7(sp)
+
+ .if \all == 1
+ REG_L x2, PT_SP(sp)
+ REG_L x3, PT_GP(sp)
+ REG_L x4, PT_TP(sp)
+ REG_L x6, PT_T1(sp)
+ REG_L x7, PT_T2(sp)
+ REG_L x8, PT_S0(sp)
+ REG_L x9, PT_S1(sp)
REG_L x18, PT_S2(sp)
REG_L x19, PT_S3(sp)
REG_L x20, PT_S4(sp)
@@ -127,10 +182,25 @@
REG_L x30, PT_T5(sp)
REG_L x31, PT_T6(sp)
+ .else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+ REG_L x8, PT_S0(sp)
+#endif
+ .endif
addi sp, sp, PT_SIZE_ON_STACK
.endm
+
+ .macro PREPARE_ARGS
+ addi a0, t0, -FENTRY_RA_OFFSET // ip
+ la a1, function_trace_op
+ REG_L a2, 0(a1) // op
+ mv a1, ra // parent_ip
+ mv a3, sp // fregs
+ .endm
+
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
ENTRY(ftrace_caller)
SAVE_ABI
@@ -159,33 +229,28 @@ ftrace_graph_call:
jr t0
ENDPROC(ftrace_caller)
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_regs_caller)
- SAVE_ALL
-
- addi a0, t0, -FENTRY_RA_OFFSET
- la a1, function_trace_op
- REG_L a2, 0(a1)
- mv a1, ra
- mv a3, sp
+ SAVE_ABI_REGS 1
+ PREPARE_ARGS
ftrace_regs_call:
.global ftrace_regs_call
call ftrace_stub
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
- addi a0, sp, PT_RA
- REG_L a1, PT_EPC(sp)
- addi a1, a1, -FENTRY_RA_OFFSET
-#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
- mv a2, s0
-#endif
-ftrace_graph_regs_call:
- .global ftrace_graph_regs_call
+ RESTORE_ABI_REGS 1
+ jr t0
+ENDPROC(ftrace_regs_caller)
+
+ENTRY(ftrace_caller)
+ SAVE_ABI_REGS 0
+ PREPARE_ARGS
+
+ftrace_call:
+ .global ftrace_call
call ftrace_stub
-#endif
- RESTORE_ALL
+ RESTORE_ABI_REGS 0
jr t0
-ENDPROC(ftrace_regs_caller)
+ENDPROC(ftrace_caller)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 4/7] riscv: ftrace: Add ftrace_graph_func
@ 2023-01-12 9:06 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
Here implements ftrace_graph_func as the function graph tracing function
with FTRACE_WITH_REGS defined.
function_graph_func gets the point of the parent IP and the frame pointer
from fregs and call prepare_ftrace_return for function graph tracing.
If FTRACE_WITH_REGS isn't defined, the enable/disable helpers of
ftrace_graph_[regs]_call are revised for serving only ftrace_graph_call
in the !FTRACE_WITH_REGS version ftrace_caller.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/ftrace.h | 13 ++-
arch/riscv/kernel/ftrace.c | 30 +++----
arch/riscv/kernel/mcount-dyn.S | 139 +++++++++++++++++++++++---------
3 files changed, 126 insertions(+), 56 deletions(-)
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 9e73922e1e2e..84f856a3286e 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -107,8 +107,17 @@ do { \
struct dyn_ftrace;
int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
#define ftrace_init_nop ftrace_init_nop
-#endif
-#endif
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+struct ftrace_ops;
+struct ftrace_regs;
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs);
+#define ftrace_graph_func ftrace_graph_func
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* _ASM_RISCV_FTRACE_H */
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index 5bff37af4770..95e14d8161a4 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -169,32 +169,28 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
}
#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+ struct pt_regs *regs = arch_ftrace_get_regs(fregs);
+ unsigned long *parent = (unsigned long *)®s->ra;
+
+ prepare_ftrace_return(parent, ip, frame_pointer(regs));
+}
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
extern void ftrace_graph_call(void);
-extern void ftrace_graph_regs_call(void);
int ftrace_enable_ftrace_graph_caller(void)
{
- int ret;
-
- ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, true, true);
- if (ret)
- return ret;
-
- return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+ return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
(unsigned long)&prepare_ftrace_return, true, true);
}
int ftrace_disable_ftrace_graph_caller(void)
{
- int ret;
-
- ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
- (unsigned long)&prepare_ftrace_return, false, true);
- if (ret)
- return ret;
-
- return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+ return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
(unsigned long)&prepare_ftrace_return, false, true);
}
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index 125de818d1ba..f26e9f6e2fed 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -57,19 +57,52 @@
.endm
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
- .macro SAVE_ALL
+
+/**
+* SAVE_ABI_REGS - save regs against the pt_regs struct
+*
+* @all: tell if saving all the regs
+*
+* If all is set, all the regs will be saved, otherwise only ABI
+* related regs (a0-a7,epc,ra and optional s0) will be saved.
+*
+* After the stack is established,
+*
+* 0(sp) stores the PC of the traced function which can be accessed
+* by &(fregs)->regs->epc in tracing function. Note that the real
+* function entry address should be computed with -FENTRY_RA_OFFSET.
+*
+* 8(sp) stores the function return address (i.e. parent IP) that
+* can be accessed by &(fregs)->regs->ra in tracing function.
+*
+* The other regs are saved at the respective localtion and accessed
+* by the respective pt_regs member.
+*
+* Here is the layout of stack for your reference.
+*
+* PT_SIZE_ON_STACK -> +++++++++
+* + ..... +
+* + t3-t6 +
+* + s2-s11+
+* + a0-a7 + --++++-> ftrace_caller saved
+* + s1 + +
+* + s0 + --+
+* + t0-t2 + +
+* + tp + +
+* + gp + +
+* + sp + +
+* + ra + --+ // parent IP
+* sp -> + epc + --+ // PC
+* +++++++++
+**/
+ .macro SAVE_ABI_REGS, all=0
addi sp, sp, -PT_SIZE_ON_STACK
REG_S t0, PT_EPC(sp)
REG_S x1, PT_RA(sp)
- REG_S x2, PT_SP(sp)
- REG_S x3, PT_GP(sp)
- REG_S x4, PT_TP(sp)
- REG_S x5, PT_T0(sp)
- REG_S x6, PT_T1(sp)
- REG_S x7, PT_T2(sp)
- REG_S x8, PT_S0(sp)
- REG_S x9, PT_S1(sp)
+
+ // always save the ABI regs
+
REG_S x10, PT_A0(sp)
REG_S x11, PT_A1(sp)
REG_S x12, PT_A2(sp)
@@ -78,6 +111,18 @@
REG_S x15, PT_A5(sp)
REG_S x16, PT_A6(sp)
REG_S x17, PT_A7(sp)
+
+ // save the leftover regs
+
+ .if \all == 1
+ REG_S x2, PT_SP(sp)
+ REG_S x3, PT_GP(sp)
+ REG_S x4, PT_TP(sp)
+ REG_S x5, PT_T0(sp)
+ REG_S x6, PT_T1(sp)
+ REG_S x7, PT_T2(sp)
+ REG_S x8, PT_S0(sp)
+ REG_S x9, PT_S1(sp)
REG_S x18, PT_S2(sp)
REG_S x19, PT_S3(sp)
REG_S x20, PT_S4(sp)
@@ -92,18 +137,19 @@
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
+
+ // save s0 if FP_TEST defined
+
+ .else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+ REG_S x8, PT_S0(sp)
+#endif
+ .endif
.endm
- .macro RESTORE_ALL
+ .macro RESTORE_ABI_REGS, all=0
REG_L t0, PT_EPC(sp)
REG_L x1, PT_RA(sp)
- REG_L x2, PT_SP(sp)
- REG_L x3, PT_GP(sp)
- REG_L x4, PT_TP(sp)
- REG_L x6, PT_T1(sp)
- REG_L x7, PT_T2(sp)
- REG_L x8, PT_S0(sp)
- REG_L x9, PT_S1(sp)
REG_L x10, PT_A0(sp)
REG_L x11, PT_A1(sp)
REG_L x12, PT_A2(sp)
@@ -112,6 +158,15 @@
REG_L x15, PT_A5(sp)
REG_L x16, PT_A6(sp)
REG_L x17, PT_A7(sp)
+
+ .if \all == 1
+ REG_L x2, PT_SP(sp)
+ REG_L x3, PT_GP(sp)
+ REG_L x4, PT_TP(sp)
+ REG_L x6, PT_T1(sp)
+ REG_L x7, PT_T2(sp)
+ REG_L x8, PT_S0(sp)
+ REG_L x9, PT_S1(sp)
REG_L x18, PT_S2(sp)
REG_L x19, PT_S3(sp)
REG_L x20, PT_S4(sp)
@@ -127,10 +182,25 @@
REG_L x30, PT_T5(sp)
REG_L x31, PT_T6(sp)
+ .else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+ REG_L x8, PT_S0(sp)
+#endif
+ .endif
addi sp, sp, PT_SIZE_ON_STACK
.endm
+
+ .macro PREPARE_ARGS
+ addi a0, t0, -FENTRY_RA_OFFSET // ip
+ la a1, function_trace_op
+ REG_L a2, 0(a1) // op
+ mv a1, ra // parent_ip
+ mv a3, sp // fregs
+ .endm
+
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
ENTRY(ftrace_caller)
SAVE_ABI
@@ -159,33 +229,28 @@ ftrace_graph_call:
jr t0
ENDPROC(ftrace_caller)
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_regs_caller)
- SAVE_ALL
-
- addi a0, t0, -FENTRY_RA_OFFSET
- la a1, function_trace_op
- REG_L a2, 0(a1)
- mv a1, ra
- mv a3, sp
+ SAVE_ABI_REGS 1
+ PREPARE_ARGS
ftrace_regs_call:
.global ftrace_regs_call
call ftrace_stub
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
- addi a0, sp, PT_RA
- REG_L a1, PT_EPC(sp)
- addi a1, a1, -FENTRY_RA_OFFSET
-#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
- mv a2, s0
-#endif
-ftrace_graph_regs_call:
- .global ftrace_graph_regs_call
+ RESTORE_ABI_REGS 1
+ jr t0
+ENDPROC(ftrace_regs_caller)
+
+ENTRY(ftrace_caller)
+ SAVE_ABI_REGS 0
+ PREPARE_ARGS
+
+ftrace_call:
+ .global ftrace_call
call ftrace_stub
-#endif
- RESTORE_ALL
+ RESTORE_ABI_REGS 0
jr t0
-ENDPROC(ftrace_regs_caller)
+ENDPROC(ftrace_caller)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 5/7] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:06 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
This patch adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide the
register_ftrace_direct[_multi] interfaces allowing users to register
the customed trampoline (direct_caller) as the mcount for one or
more target functions. And modify_ftrace_direct[_multi] are also
provided for modifying direct_caller.
To make the direct_caller and the other ftrace hooks (eg. function/fgraph
tracer, k[ret]probes) co-exist, a temporary register is nominated to
store the address of direct_caller in ftrace_regs_caller. After the
setting of the address direct_caller by direct_ops->func and the
RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/ftrace.h | 8 ++++++++
arch/riscv/kernel/mcount-dyn.S | 4 ++++
3 files changed, 13 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index ee0d39b26794..307a9f413edd 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -135,6 +135,7 @@ config RISCV
select UACCESS_MEMCPY if !MMU
select ZONE_DMA32 if 64BIT
select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && $(cc-option,-fpatchable-function-entry=8)
+ select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
select HAVE_FUNCTION_GRAPH_TRACER
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 84f856a3286e..84904c1e4369 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -114,6 +114,14 @@ struct ftrace_regs;
void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct ftrace_regs *fregs);
#define ftrace_graph_func ftrace_graph_func
+
+static inline void
+__arch_ftrace_set_direct_caller(struct pt_regs *regs, unsigned long addr)
+{
+ regs->t1 = addr;
+}
+#define arch_ftrace_set_direct_caller(fregs, addr) \
+ __arch_ftrace_set_direct_caller(&(fregs)->regs, addr)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
#endif /* __ASSEMBLY__ */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index f26e9f6e2fed..7801c1c8bb5a 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -231,6 +231,7 @@ ENDPROC(ftrace_caller)
#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_regs_caller)
+ move t1, zero
SAVE_ABI_REGS 1
PREPARE_ARGS
@@ -239,7 +240,10 @@ ftrace_regs_call:
call ftrace_stub
RESTORE_ABI_REGS 1
+ bnez t1,.Ldirect
jr t0
+.Ldirect:
+ jr t1
ENDPROC(ftrace_regs_caller)
ENTRY(ftrace_caller)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 5/7] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
@ 2023-01-12 9:06 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
This patch adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide the
register_ftrace_direct[_multi] interfaces allowing users to register
the customed trampoline (direct_caller) as the mcount for one or
more target functions. And modify_ftrace_direct[_multi] are also
provided for modifying direct_caller.
To make the direct_caller and the other ftrace hooks (eg. function/fgraph
tracer, k[ret]probes) co-exist, a temporary register is nominated to
store the address of direct_caller in ftrace_regs_caller. After the
setting of the address direct_caller by direct_ops->func and the
RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/ftrace.h | 8 ++++++++
arch/riscv/kernel/mcount-dyn.S | 4 ++++
3 files changed, 13 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index ee0d39b26794..307a9f413edd 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -135,6 +135,7 @@ config RISCV
select UACCESS_MEMCPY if !MMU
select ZONE_DMA32 if 64BIT
select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && $(cc-option,-fpatchable-function-entry=8)
+ select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
select HAVE_FUNCTION_GRAPH_TRACER
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 84f856a3286e..84904c1e4369 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -114,6 +114,14 @@ struct ftrace_regs;
void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct ftrace_regs *fregs);
#define ftrace_graph_func ftrace_graph_func
+
+static inline void
+__arch_ftrace_set_direct_caller(struct pt_regs *regs, unsigned long addr)
+{
+ regs->t1 = addr;
+}
+#define arch_ftrace_set_direct_caller(fregs, addr) \
+ __arch_ftrace_set_direct_caller(&(fregs)->regs, addr)
#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
#endif /* __ASSEMBLY__ */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index f26e9f6e2fed..7801c1c8bb5a 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -231,6 +231,7 @@ ENDPROC(ftrace_caller)
#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
ENTRY(ftrace_regs_caller)
+ move t1, zero
SAVE_ABI_REGS 1
PREPARE_ARGS
@@ -239,7 +240,10 @@ ftrace_regs_call:
call ftrace_stub
RESTORE_ABI_REGS 1
+ bnez t1,.Ldirect
jr t0
+.Ldirect:
+ jr t1
ENDPROC(ftrace_regs_caller)
ENTRY(ftrace_caller)
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:06 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
the ftrace-direct*.c files in samples/ftrace/.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 ++
samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
samples/ftrace/ftrace-direct.c | 22 ++++++++++++
6 files changed, 142 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 307a9f413edd..e944af44f681 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -112,6 +112,8 @@ config RISCV
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_FUNCTION_ARG_ACCESS_API
+ select HAVE_SAMPLE_FTRACE_DIRECT
+ select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_RSEQ
diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
index de5a0f67f320..be7bf472c3c7 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -23,6 +23,39 @@ extern void my_tramp2(void *);
static unsigned long my_ip = (unsigned long)schedule;
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func1\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func2\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
index d52370cad0b6..10884bf418f7 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
extern void my_tramp1(void *);
extern void my_tramp2(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func1\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func2\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
index ec1088922517..a35bf43bf6d7 100644
--- a/samples/ftrace/ftrace-direct-multi.c
+++ b/samples/ftrace/ftrace-direct-multi.c
@@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
index e13fb59a2b47..3b62e33c2e6d 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-40\n"
+" sd a0,0(sp)\n"
+" sd a1,8(sp)\n"
+" sd a2,16(sp)\n"
+" sd t0,24(sp)\n"
+" sd ra,32(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld a1,8(sp)\n"
+" ld a2,16(sp)\n"
+" ld t0,24(sp)\n"
+" ld ra,32(sp)\n"
+" addi sp,sp,40\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
index 1f769d0db20f..2cfe5a7d2d70 100644
--- a/samples/ftrace/ftrace-direct.c
+++ b/samples/ftrace/ftrace-direct.c
@@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-12 9:06 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
the ftrace-direct*.c files in samples/ftrace/.
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 ++
samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
samples/ftrace/ftrace-direct.c | 22 ++++++++++++
6 files changed, 142 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 307a9f413edd..e944af44f681 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -112,6 +112,8 @@ config RISCV
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_FUNCTION_ARG_ACCESS_API
+ select HAVE_SAMPLE_FTRACE_DIRECT
+ select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_RSEQ
diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
index de5a0f67f320..be7bf472c3c7 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -23,6 +23,39 @@ extern void my_tramp2(void *);
static unsigned long my_ip = (unsigned long)schedule;
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func1\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-16\n"
+" sd t0,0(sp)\n"
+" sd ra,8(sp)\n"
+" call my_direct_func2\n"
+" ld t0,0(sp)\n"
+" ld ra,8(sp)\n"
+" addi sp,sp,16\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
index d52370cad0b6..10884bf418f7 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
extern void my_tramp1(void *);
extern void my_tramp2(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
+" my_tramp1:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func1\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp1, .-my_tramp1\n"
+
+" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
+" my_tramp2:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func2\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp2, .-my_tramp2\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
index ec1088922517..a35bf43bf6d7 100644
--- a/samples/ftrace/ftrace-direct-multi.c
+++ b/samples/ftrace/ftrace-direct-multi.c
@@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
index e13fb59a2b47..3b62e33c2e6d 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-40\n"
+" sd a0,0(sp)\n"
+" sd a1,8(sp)\n"
+" sd a2,16(sp)\n"
+" sd t0,24(sp)\n"
+" sd ra,32(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld a1,8(sp)\n"
+" ld a2,16(sp)\n"
+" ld t0,24(sp)\n"
+" ld ra,32(sp)\n"
+" addi sp,sp,40\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
index 1f769d0db20f..2cfe5a7d2d70 100644
--- a/samples/ftrace/ftrace-direct.c
+++ b/samples/ftrace/ftrace-direct.c
@@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
extern void my_tramp(void *);
+#ifdef CONFIG_RISCV
+
+asm (" .pushsection .text, \"ax\", @progbits\n"
+" .type my_tramp, @function\n"
+" .globl my_tramp\n"
+" my_tramp:\n"
+" addi sp,sp,-24\n"
+" sd a0,0(sp)\n"
+" sd t0,8(sp)\n"
+" sd ra,16(sp)\n"
+" call my_direct_func\n"
+" ld a0,0(sp)\n"
+" ld t0,8(sp)\n"
+" ld ra,16(sp)\n"
+" addi sp,sp,24\n"
+" jr t0\n"
+" .size my_tramp, .-my_tramp\n"
+" .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
#ifdef CONFIG_X86_64
#include <asm/ibt.h>
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 7/7] riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
2023-01-12 9:05 ` guoren
@ 2023-01-12 9:06 ` guoren
-1 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
In RISC-V, -fpatchable-function-entry option is used to support
dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
don't have to be called to create the __mcount_loc section before
the vmlinux linking.
Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
Makefile not to run recordmcount.
Link: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
Link: https://lore.kernel.org/linux-riscv/Y4jtfrJt+%2FQ5nMOz@spud/
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e944af44f681..a78d39780b76 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -54,6 +54,7 @@ config RISCV
select COMMON_CLK
select CPU_PM if CPU_IDLE
select EDAC_SUPPORT
+ select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE
select GENERIC_ARCH_TOPOLOGY
select GENERIC_ATOMIC64 if !64BIT
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH -next V7 7/7] riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
@ 2023-01-12 9:06 ` guoren
0 siblings, 0 replies; 88+ messages in thread
From: guoren @ 2023-01-12 9:06 UTC (permalink / raw)
To: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, guoren
Cc: linux-riscv, linux-kernel
From: Song Shuai <suagrfillet@gmail.com>
In RISC-V, -fpatchable-function-entry option is used to support
dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
don't have to be called to create the __mcount_loc section before
the vmlinux linking.
Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
Makefile not to run recordmcount.
Link: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
Link: https://lore.kernel.org/linux-riscv/Y4jtfrJt+%2FQ5nMOz@spud/
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e944af44f681..a78d39780b76 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -54,6 +54,7 @@ config RISCV
select COMMON_CLK
select CPU_PM if CPU_IDLE
select EDAC_SUPPORT
+ select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE
select GENERIC_ARCH_TOPOLOGY
select GENERIC_ATOMIC64 if !64BIT
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
--
2.36.1
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-12 9:05 ` guoren
@ 2023-01-12 12:16 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-12 12:16 UTC (permalink / raw)
To: guoren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
Hi Guo,
On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> From: Andy Chiu <andy.chiu@sifive.com>
>
> In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> forming a jump that jumps to an address over 4K. This may cause errors
> if we want to enable kernel preemption and remove dependency from
> patching code with stop_machine(). For example, if a task was switched
> out on auipc. And, if we changed the ftrace function before it was
> switched back, then it would jump to an address that has updated 11:0
> bits mixing with previous XLEN:12 part.
>
> p: patched area performed by dynamic ftrace
> ftrace_prologue:
> p| REG_S ra, -SZREG(sp)
> p| auipc ra, 0x? ------------> preempted
> ...
> change ftrace function
> ...
> p| jalr -?(ra) <------------- switched back
> p| REG_L ra, -SZREG(sp)
> func:
> xxx
> ret
As mentioned on the last posting, I don't think this is sufficient to fix the
issue. I've replied with more detail there:
https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
executing the ftrace_prologue while another CPU is patching the
ftrace_prologue, you have the exact same issue.
For example, if CPU X is in the prologue fetches the old AUIPC and the new
JALR (because it races with CPU Y modifying those), CPU X will branch to the
wrong address. The race window is much smaller in the absence of preemption,
but it's still there (and will be exacerbated in virtual machines since the
hypervisor can preempt a vCPU at any time).
Note that the above is even assuming that instruction fetches are atomic, which
I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
MODification and eXecutuion of instructions" rules which mean only certain
instructions can be patched atomically.
Either I'm missing something that provides mutual exclusion between the
patching and execution of the ftrace_prologue, or this patch is not sufficient.
Thanks,
Mark.
> Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
> Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e2b656043abf..ee0d39b26794 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -138,7 +138,7 @@ config RISCV
> select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
> select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
> select HAVE_FUNCTION_GRAPH_TRACER
> - select HAVE_FUNCTION_TRACER if !XIP_KERNEL
> + select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
>
> config ARCH_MMAP_RND_BITS_MIN
> default 18 if 64BIT
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-12 12:16 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-12 12:16 UTC (permalink / raw)
To: guoren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
Hi Guo,
On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> From: Andy Chiu <andy.chiu@sifive.com>
>
> In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> forming a jump that jumps to an address over 4K. This may cause errors
> if we want to enable kernel preemption and remove dependency from
> patching code with stop_machine(). For example, if a task was switched
> out on auipc. And, if we changed the ftrace function before it was
> switched back, then it would jump to an address that has updated 11:0
> bits mixing with previous XLEN:12 part.
>
> p: patched area performed by dynamic ftrace
> ftrace_prologue:
> p| REG_S ra, -SZREG(sp)
> p| auipc ra, 0x? ------------> preempted
> ...
> change ftrace function
> ...
> p| jalr -?(ra) <------------- switched back
> p| REG_L ra, -SZREG(sp)
> func:
> xxx
> ret
As mentioned on the last posting, I don't think this is sufficient to fix the
issue. I've replied with more detail there:
https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
executing the ftrace_prologue while another CPU is patching the
ftrace_prologue, you have the exact same issue.
For example, if CPU X is in the prologue fetches the old AUIPC and the new
JALR (because it races with CPU Y modifying those), CPU X will branch to the
wrong address. The race window is much smaller in the absence of preemption,
but it's still there (and will be exacerbated in virtual machines since the
hypervisor can preempt a vCPU at any time).
Note that the above is even assuming that instruction fetches are atomic, which
I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
MODification and eXecutuion of instructions" rules which mean only certain
instructions can be patched atomically.
Either I'm missing something that provides mutual exclusion between the
patching and execution of the ftrace_prologue, or this patch is not sufficient.
Thanks,
Mark.
> Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
> Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e2b656043abf..ee0d39b26794 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -138,7 +138,7 @@ config RISCV
> select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
> select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
> select HAVE_FUNCTION_GRAPH_TRACER
> - select HAVE_FUNCTION_TRACER if !XIP_KERNEL
> + select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
>
> config ARCH_MMAP_RND_BITS_MIN
> default 18 if 64BIT
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-12 12:16 ` Mark Rutland
@ 2023-01-12 12:57 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-12 12:57 UTC (permalink / raw)
To: guoren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 12:16:02PM +0000, Mark Rutland wrote:
> Hi Guo,
>
> On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > From: Andy Chiu <andy.chiu@sifive.com>
> >
> > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > forming a jump that jumps to an address over 4K. This may cause errors
> > if we want to enable kernel preemption and remove dependency from
> > patching code with stop_machine(). For example, if a task was switched
> > out on auipc. And, if we changed the ftrace function before it was
> > switched back, then it would jump to an address that has updated 11:0
> > bits mixing with previous XLEN:12 part.
> >
> > p: patched area performed by dynamic ftrace
> > ftrace_prologue:
> > p| REG_S ra, -SZREG(sp)
> > p| auipc ra, 0x? ------------> preempted
> > ...
> > change ftrace function
> > ...
> > p| jalr -?(ra) <------------- switched back
> > p| REG_L ra, -SZREG(sp)
> > func:
> > xxx
> > ret
>
> As mentioned on the last posting, I don't think this is sufficient to fix the
> issue. I've replied with more detail there:
>
> https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
>
> Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> executing the ftrace_prologue while another CPU is patching the
> ftrace_prologue, you have the exact same issue.
>
> For example, if CPU X is in the prologue fetches the old AUIPC and the new
> JALR (because it races with CPU Y modifying those), CPU X will branch to the
> wrong address. The race window is much smaller in the absence of preemption,
> but it's still there (and will be exacerbated in virtual machines since the
> hypervisor can preempt a vCPU at any time).
With that in mind, I think your current implementation of ftrace_make_call()
and ftrace_make_nop() have a simlar bug. A caller might execute:
NOP // not yet patched to AUIPC
< AUIPC and JALR instructions both patched >
JALR
... and go to the wrong place.
Assuming individual instruction fetches are atomic, and that you only ever
branch to the same trampoline, you could fix that by always leaving the AUIPC
in place, so that you only patch the JALR to enable/disable the callsite.
Depending on your calling convention, if you have two free GPRs, you might be
able to avoid the stacking of RA by always saving it to a GPR in the callsite,
using a different GPR for the address generation, and having the ftrace
trampoline restore the original RA value, e.g.
MV GPR1, ra
AUIPC GPR2, high_bits_of(ftrace_caller)
JALR ra, high_bits(GPR2) // only patch this
... which'd save an instruction per callsite.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-12 12:57 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-12 12:57 UTC (permalink / raw)
To: guoren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 12:16:02PM +0000, Mark Rutland wrote:
> Hi Guo,
>
> On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > From: Andy Chiu <andy.chiu@sifive.com>
> >
> > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > forming a jump that jumps to an address over 4K. This may cause errors
> > if we want to enable kernel preemption and remove dependency from
> > patching code with stop_machine(). For example, if a task was switched
> > out on auipc. And, if we changed the ftrace function before it was
> > switched back, then it would jump to an address that has updated 11:0
> > bits mixing with previous XLEN:12 part.
> >
> > p: patched area performed by dynamic ftrace
> > ftrace_prologue:
> > p| REG_S ra, -SZREG(sp)
> > p| auipc ra, 0x? ------------> preempted
> > ...
> > change ftrace function
> > ...
> > p| jalr -?(ra) <------------- switched back
> > p| REG_L ra, -SZREG(sp)
> > func:
> > xxx
> > ret
>
> As mentioned on the last posting, I don't think this is sufficient to fix the
> issue. I've replied with more detail there:
>
> https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
>
> Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> executing the ftrace_prologue while another CPU is patching the
> ftrace_prologue, you have the exact same issue.
>
> For example, if CPU X is in the prologue fetches the old AUIPC and the new
> JALR (because it races with CPU Y modifying those), CPU X will branch to the
> wrong address. The race window is much smaller in the absence of preemption,
> but it's still there (and will be exacerbated in virtual machines since the
> hypervisor can preempt a vCPU at any time).
With that in mind, I think your current implementation of ftrace_make_call()
and ftrace_make_nop() have a simlar bug. A caller might execute:
NOP // not yet patched to AUIPC
< AUIPC and JALR instructions both patched >
JALR
... and go to the wrong place.
Assuming individual instruction fetches are atomic, and that you only ever
branch to the same trampoline, you could fix that by always leaving the AUIPC
in place, so that you only patch the JALR to enable/disable the callsite.
Depending on your calling convention, if you have two free GPRs, you might be
able to avoid the stacking of RA by always saving it to a GPR in the callsite,
using a different GPR for the address generation, and having the ftrace
trampoline restore the original RA value, e.g.
MV GPR1, ra
AUIPC GPR2, high_bits_of(ftrace_caller)
JALR ra, high_bits(GPR2) // only patch this
... which'd save an instruction per callsite.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 3/7] riscv: ftrace: Reduce the detour code size to half
2023-01-12 9:05 ` guoren
@ 2023-01-16 14:11 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 14:11 UTC (permalink / raw)
To: guoren
Cc: linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
suagrfillet, andy.chiu, linux
Hi,
On 12.01.2023 12:05, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Use a temporary register to reduce the size of detour code from 16 bytes to
> 8 bytes. The previous implementation is from 'commit afc76b8b8011 ("riscv:
> Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")'.
>
> Before the patch:
> <func_prolog>:
> 0: REG_S ra, -SZREG(sp)
> 4: auipc ra, ?
> 8: jalr ?(ra)
> 12: REG_L ra, -SZREG(sp)
> (func_boddy)
>
> After the patch:
> <func_prolog>:
> 0: auipc t0, ?
> 4: jalr t0, ?(t0)
> (func_boddy)
>
> This patch not just reduces the size of detour code, but also fixes an
> important issue:
>
> An Ftrace callback registered with FTRACE_OPS_FL_IPMODIFY flag can
> actually change the instruction pointer, e.g. to "replace" the given
> kernel function with a new one, which is needed for livepatching, etc.
>
> In this case, the trampoline (ftrace_regs_caller) would not return to
> <func_prolog+12> but would rather jump to the new function. So, "REG_L
> ra, -SZREG(sp)" would not run and the original return address would not
> be restored. The kernel is likely to hang or crash as a result.
>
> This can be easily demonstrated if one tries to "replace", say,
> cmdline_proc_show() with a new function with the same signature using
> instruction_pointer_set(&fregs->regs, new_func_addr) in the Ftrace
> callback.
>
> Link: https://lore.kernel.org/linux-riscv/20221122075440.1165172-1-suagrfillet@gmail.com/
> Link: https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
> Co-developed-by: Song Shuai <suagrfillet@gmail.com>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Cc: Evgenii Shatokhin <e.shatokhin@yadro.com>
> ---
> arch/riscv/Makefile | 4 +-
> arch/riscv/include/asm/ftrace.h | 50 +++++++++++++++++++------
> arch/riscv/kernel/ftrace.c | 65 ++++++++++-----------------------
> arch/riscv/kernel/mcount-dyn.S | 42 ++++++++-------------
> 4 files changed, 75 insertions(+), 86 deletions(-)
>
> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> index ea5a91da6897..3c9aaf67ed79 100644
> --- a/arch/riscv/Makefile
> +++ b/arch/riscv/Makefile
> @@ -12,9 +12,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
> LDFLAGS_vmlinux := --no-relax
> KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
> ifeq ($(CONFIG_RISCV_ISA_C),y)
> - CC_FLAGS_FTRACE := -fpatchable-function-entry=8
> -else
> CC_FLAGS_FTRACE := -fpatchable-function-entry=4
> +else
> + CC_FLAGS_FTRACE := -fpatchable-function-entry=2
> endif
> endif
>
> diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
> index 04dad3380041..9e73922e1e2e 100644
> --- a/arch/riscv/include/asm/ftrace.h
> +++ b/arch/riscv/include/asm/ftrace.h
> @@ -42,6 +42,14 @@ struct dyn_arch_ftrace {
> * 2) jalr: setting low-12 offset to ra, jump to ra, and set ra to
> * return address (original pc + 4)
> *
> + *<ftrace enable>:
> + * 0: auipc t0/ra, 0x?
> + * 4: jalr t0/ra, ?(t0/ra)
> + *
> + *<ftrace disable>:
> + * 0: nop
> + * 4: nop
> + *
> * Dynamic ftrace generates probes to call sites, so we must deal with
> * both auipc and jalr at the same time.
> */
> @@ -52,25 +60,43 @@ struct dyn_arch_ftrace {
> #define AUIPC_OFFSET_MASK (0xfffff000)
> #define AUIPC_PAD (0x00001000)
> #define JALR_SHIFT 20
> -#define JALR_BASIC (0x000080e7)
> -#define AUIPC_BASIC (0x00000097)
> +#define JALR_RA (0x000080e7)
> +#define AUIPC_RA (0x00000097)
> +#define JALR_T0 (0x000282e7)
> +#define AUIPC_T0 (0x00000297)
> #define NOP4 (0x00000013)
>
> -#define make_call(caller, callee, call) \
> +#define to_jalr_t0(offset) \
> + (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0)
> +
> +#define to_auipc_t0(offset) \
> + ((offset & JALR_SIGN_MASK) ? \
> + (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_T0) : \
> + ((offset & AUIPC_OFFSET_MASK) | AUIPC_T0))
> +
> +#define make_call_t0(caller, callee, call) \
> do { \
> - call[0] = to_auipc_insn((unsigned int)((unsigned long)callee - \
> - (unsigned long)caller)); \
> - call[1] = to_jalr_insn((unsigned int)((unsigned long)callee - \
> - (unsigned long)caller)); \
> + unsigned int offset = \
> + (unsigned long) callee - (unsigned long) caller; \
> + call[0] = to_auipc_t0(offset); \
> + call[1] = to_jalr_t0(offset); \
> } while (0)
>
> -#define to_jalr_insn(offset) \
> - (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_BASIC)
> +#define to_jalr_ra(offset) \
> + (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_RA)
>
> -#define to_auipc_insn(offset) \
> +#define to_auipc_ra(offset) \
> ((offset & JALR_SIGN_MASK) ? \
> - (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_BASIC) : \
> - ((offset & AUIPC_OFFSET_MASK) | AUIPC_BASIC))
> + (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_RA) : \
> + ((offset & AUIPC_OFFSET_MASK) | AUIPC_RA))
> +
> +#define make_call_ra(caller, callee, call) \
> +do { \
> + unsigned int offset = \
> + (unsigned long) callee - (unsigned long) caller; \
> + call[0] = to_auipc_ra(offset); \
> + call[1] = to_jalr_ra(offset); \
> +} while (0)
>
> /*
> * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here.
> diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
> index 2086f6585773..5bff37af4770 100644
> --- a/arch/riscv/kernel/ftrace.c
> +++ b/arch/riscv/kernel/ftrace.c
> @@ -55,12 +55,15 @@ static int ftrace_check_current_call(unsigned long hook_pos,
> }
>
> static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
> - bool enable)
> + bool enable, bool ra)
> {
> unsigned int call[2];
> unsigned int nops[2] = {NOP4, NOP4};
>
> - make_call(hook_pos, target, call);
> + if (ra)
> + make_call_ra(hook_pos, target, call);
> + else
> + make_call_t0(hook_pos, target, call);
>
> /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
> if (patch_text_nosync
> @@ -70,42 +73,13 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
> return 0;
> }
>
> -/*
> - * Put 5 instructions with 16 bytes at the front of function within
> - * patchable function entry nops' area.
> - *
> - * 0: REG_S ra, -SZREG(sp)
> - * 1: auipc ra, 0x?
> - * 2: jalr -?(ra)
> - * 3: REG_L ra, -SZREG(sp)
> - *
> - * So the opcodes is:
> - * 0: 0xfe113c23 (sd)/0xfe112e23 (sw)
> - * 1: 0x???????? -> auipc
> - * 2: 0x???????? -> jalr
> - * 3: 0xff813083 (ld)/0xffc12083 (lw)
> - */
> -#if __riscv_xlen == 64
> -#define INSN0 0xfe113c23
> -#define INSN3 0xff813083
> -#elif __riscv_xlen == 32
> -#define INSN0 0xfe112e23
> -#define INSN3 0xffc12083
> -#endif
> -
> -#define FUNC_ENTRY_SIZE 16
> -#define FUNC_ENTRY_JMP 4
> -
> int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> {
> - unsigned int call[4] = {INSN0, 0, 0, INSN3};
> - unsigned long target = addr;
> - unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
> + unsigned int call[2];
>
> - call[1] = to_auipc_insn((unsigned int)(target - caller));
> - call[2] = to_jalr_insn((unsigned int)(target - caller));
> + make_call_t0(rec->ip, addr, call);
>
> - if (patch_text_nosync((void *)rec->ip, call, FUNC_ENTRY_SIZE))
> + if (patch_text_nosync((void *)rec->ip, call, MCOUNT_INSN_SIZE))
> return -EPERM;
>
> return 0;
> @@ -114,15 +88,14 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> unsigned long addr)
> {
> - unsigned int nops[4] = {NOP4, NOP4, NOP4, NOP4};
> + unsigned int nops[2] = {NOP4, NOP4};
>
> - if (patch_text_nosync((void *)rec->ip, nops, FUNC_ENTRY_SIZE))
> + if (patch_text_nosync((void *)rec->ip, nops, MCOUNT_INSN_SIZE))
> return -EPERM;
>
> return 0;
> }
>
> -
> /*
> * This is called early on, and isn't wrapped by
> * ftrace_arch_code_modify_{prepare,post_process}() and therefor doesn't hold
> @@ -144,10 +117,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
> int ftrace_update_ftrace_func(ftrace_func_t func)
> {
> int ret = __ftrace_modify_call((unsigned long)&ftrace_call,
> - (unsigned long)func, true);
> + (unsigned long)func, true, true);
> if (!ret) {
> ret = __ftrace_modify_call((unsigned long)&ftrace_regs_call,
> - (unsigned long)func, true);
> + (unsigned long)func, true, true);
> }
>
> return ret;
> @@ -159,16 +132,16 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> unsigned long addr)
> {
> unsigned int call[2];
> - unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
> + unsigned long caller = rec->ip;
> int ret;
>
> - make_call(caller, old_addr, call);
> + make_call_t0(caller, old_addr, call);
> ret = ftrace_check_current_call(caller, call);
>
> if (ret)
> return ret;
>
> - return __ftrace_modify_call(caller, addr, true);
> + return __ftrace_modify_call(caller, addr, true, false);
> }
> #endif
>
> @@ -203,12 +176,12 @@ int ftrace_enable_ftrace_graph_caller(void)
> int ret;
>
> ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
> - (unsigned long)&prepare_ftrace_return, true);
> + (unsigned long)&prepare_ftrace_return, true, true);
> if (ret)
> return ret;
>
> return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
> - (unsigned long)&prepare_ftrace_return, true);
> + (unsigned long)&prepare_ftrace_return, true, true);
> }
>
> int ftrace_disable_ftrace_graph_caller(void)
> @@ -216,12 +189,12 @@ int ftrace_disable_ftrace_graph_caller(void)
> int ret;
>
> ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
> - (unsigned long)&prepare_ftrace_return, false);
> + (unsigned long)&prepare_ftrace_return, false, true);
> if (ret)
> return ret;
>
> return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
> - (unsigned long)&prepare_ftrace_return, false);
> + (unsigned long)&prepare_ftrace_return, false, true);
> }
> #endif /* CONFIG_DYNAMIC_FTRACE */
> #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
> index d171eca623b6..125de818d1ba 100644
> --- a/arch/riscv/kernel/mcount-dyn.S
> +++ b/arch/riscv/kernel/mcount-dyn.S
> @@ -13,8 +13,8 @@
>
> .text
>
> -#define FENTRY_RA_OFFSET 12
> -#define ABI_SIZE_ON_STACK 72
> +#define FENTRY_RA_OFFSET 8
> +#define ABI_SIZE_ON_STACK 80
> #define ABI_A0 0
> #define ABI_A1 8
> #define ABI_A2 16
> @@ -23,10 +23,10 @@
> #define ABI_A5 40
> #define ABI_A6 48
> #define ABI_A7 56
> -#define ABI_RA 64
> +#define ABI_T0 64
> +#define ABI_RA 72
>
> .macro SAVE_ABI
> - addi sp, sp, -SZREG
> addi sp, sp, -ABI_SIZE_ON_STACK
>
> REG_S a0, ABI_A0(sp)
> @@ -37,6 +37,7 @@
> REG_S a5, ABI_A5(sp)
> REG_S a6, ABI_A6(sp)
> REG_S a7, ABI_A7(sp)
> + REG_S t0, ABI_T0(sp)
> REG_S ra, ABI_RA(sp)
> .endm
>
> @@ -49,24 +50,18 @@
> REG_L a5, ABI_A5(sp)
> REG_L a6, ABI_A6(sp)
> REG_L a7, ABI_A7(sp)
> + REG_L t0, ABI_T0(sp)
> REG_L ra, ABI_RA(sp)
>
> addi sp, sp, ABI_SIZE_ON_STACK
> - addi sp, sp, SZREG
> .endm
>
> #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> .macro SAVE_ALL
> - addi sp, sp, -SZREG
> addi sp, sp, -PT_SIZE_ON_STACK
>
> - REG_S x1, PT_EPC(sp)
> - addi sp, sp, PT_SIZE_ON_STACK
> - REG_L x1, (sp)
> - addi sp, sp, -PT_SIZE_ON_STACK
> + REG_S t0, PT_EPC(sp)
> REG_S x1, PT_RA(sp)
> - REG_L x1, PT_EPC(sp)
> -
> REG_S x2, PT_SP(sp)
> REG_S x3, PT_GP(sp)
> REG_S x4, PT_TP(sp)
> @@ -100,15 +95,11 @@
> .endm
>
> .macro RESTORE_ALL
> + REG_L t0, PT_EPC(sp)
> REG_L x1, PT_RA(sp)
> - addi sp, sp, PT_SIZE_ON_STACK
> - REG_S x1, (sp)
> - addi sp, sp, -PT_SIZE_ON_STACK
> - REG_L x1, PT_EPC(sp)
> REG_L x2, PT_SP(sp)
> REG_L x3, PT_GP(sp)
> REG_L x4, PT_TP(sp)
> - REG_L x5, PT_T0(sp)
> REG_L x6, PT_T1(sp)
> REG_L x7, PT_T2(sp)
> REG_L x8, PT_S0(sp)
> @@ -137,17 +128,16 @@
> REG_L x31, PT_T6(sp)
>
> addi sp, sp, PT_SIZE_ON_STACK
> - addi sp, sp, SZREG
> .endm
> #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
>
> ENTRY(ftrace_caller)
> SAVE_ABI
>
> - addi a0, ra, -FENTRY_RA_OFFSET
> + addi a0, t0, -FENTRY_RA_OFFSET
> la a1, function_trace_op
> REG_L a2, 0(a1)
> - REG_L a1, ABI_SIZE_ON_STACK(sp)
> + mv a1, ra
> mv a3, sp
>
> ftrace_call:
> @@ -155,8 +145,8 @@ ftrace_call:
> call ftrace_stub
>
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> - addi a0, sp, ABI_SIZE_ON_STACK
> - REG_L a1, ABI_RA(sp)
> + addi a0, sp, ABI_RA
> + REG_L a1, ABI_T0(sp)
> addi a1, a1, -FENTRY_RA_OFFSET
> #ifdef HAVE_FUNCTION_GRAPH_FP_TEST
> mv a2, s0
> @@ -166,17 +156,17 @@ ftrace_graph_call:
> call ftrace_stub
> #endif
> RESTORE_ABI
> - ret
> + jr t0
> ENDPROC(ftrace_caller)
>
> #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> ENTRY(ftrace_regs_caller)
> SAVE_ALL
>
> - addi a0, ra, -FENTRY_RA_OFFSET
> + addi a0, t0, -FENTRY_RA_OFFSET
> la a1, function_trace_op
> REG_L a2, 0(a1)
> - REG_L a1, PT_SIZE_ON_STACK(sp)
> + mv a1, ra
> mv a3, sp
>
> ftrace_regs_call:
> @@ -196,6 +186,6 @@ ftrace_graph_regs_call:
> #endif
>
> RESTORE_ALL
> - ret
> + jr t0
> ENDPROC(ftrace_regs_caller)
> #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
> --
> 2.36.1
>
>
Looks good to me.
I also re-checked if "replacement" of cmdline_proc_show() with a custom
function via Ftrace works in this case - it does. Rollback to the
original cmdline_proc_show() seems to work OK too.
Reviewed-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
Regards,
Evgenii
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 3/7] riscv: ftrace: Reduce the detour code size to half
@ 2023-01-16 14:11 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 14:11 UTC (permalink / raw)
To: guoren
Cc: linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
suagrfillet, andy.chiu, linux
Hi,
On 12.01.2023 12:05, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Use a temporary register to reduce the size of detour code from 16 bytes to
> 8 bytes. The previous implementation is from 'commit afc76b8b8011 ("riscv:
> Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")'.
>
> Before the patch:
> <func_prolog>:
> 0: REG_S ra, -SZREG(sp)
> 4: auipc ra, ?
> 8: jalr ?(ra)
> 12: REG_L ra, -SZREG(sp)
> (func_boddy)
>
> After the patch:
> <func_prolog>:
> 0: auipc t0, ?
> 4: jalr t0, ?(t0)
> (func_boddy)
>
> This patch not just reduces the size of detour code, but also fixes an
> important issue:
>
> An Ftrace callback registered with FTRACE_OPS_FL_IPMODIFY flag can
> actually change the instruction pointer, e.g. to "replace" the given
> kernel function with a new one, which is needed for livepatching, etc.
>
> In this case, the trampoline (ftrace_regs_caller) would not return to
> <func_prolog+12> but would rather jump to the new function. So, "REG_L
> ra, -SZREG(sp)" would not run and the original return address would not
> be restored. The kernel is likely to hang or crash as a result.
>
> This can be easily demonstrated if one tries to "replace", say,
> cmdline_proc_show() with a new function with the same signature using
> instruction_pointer_set(&fregs->regs, new_func_addr) in the Ftrace
> callback.
>
> Link: https://lore.kernel.org/linux-riscv/20221122075440.1165172-1-suagrfillet@gmail.com/
> Link: https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
> Co-developed-by: Song Shuai <suagrfillet@gmail.com>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Cc: Evgenii Shatokhin <e.shatokhin@yadro.com>
> ---
> arch/riscv/Makefile | 4 +-
> arch/riscv/include/asm/ftrace.h | 50 +++++++++++++++++++------
> arch/riscv/kernel/ftrace.c | 65 ++++++++++-----------------------
> arch/riscv/kernel/mcount-dyn.S | 42 ++++++++-------------
> 4 files changed, 75 insertions(+), 86 deletions(-)
>
> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> index ea5a91da6897..3c9aaf67ed79 100644
> --- a/arch/riscv/Makefile
> +++ b/arch/riscv/Makefile
> @@ -12,9 +12,9 @@ ifeq ($(CONFIG_DYNAMIC_FTRACE),y)
> LDFLAGS_vmlinux := --no-relax
> KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
> ifeq ($(CONFIG_RISCV_ISA_C),y)
> - CC_FLAGS_FTRACE := -fpatchable-function-entry=8
> -else
> CC_FLAGS_FTRACE := -fpatchable-function-entry=4
> +else
> + CC_FLAGS_FTRACE := -fpatchable-function-entry=2
> endif
> endif
>
> diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
> index 04dad3380041..9e73922e1e2e 100644
> --- a/arch/riscv/include/asm/ftrace.h
> +++ b/arch/riscv/include/asm/ftrace.h
> @@ -42,6 +42,14 @@ struct dyn_arch_ftrace {
> * 2) jalr: setting low-12 offset to ra, jump to ra, and set ra to
> * return address (original pc + 4)
> *
> + *<ftrace enable>:
> + * 0: auipc t0/ra, 0x?
> + * 4: jalr t0/ra, ?(t0/ra)
> + *
> + *<ftrace disable>:
> + * 0: nop
> + * 4: nop
> + *
> * Dynamic ftrace generates probes to call sites, so we must deal with
> * both auipc and jalr at the same time.
> */
> @@ -52,25 +60,43 @@ struct dyn_arch_ftrace {
> #define AUIPC_OFFSET_MASK (0xfffff000)
> #define AUIPC_PAD (0x00001000)
> #define JALR_SHIFT 20
> -#define JALR_BASIC (0x000080e7)
> -#define AUIPC_BASIC (0x00000097)
> +#define JALR_RA (0x000080e7)
> +#define AUIPC_RA (0x00000097)
> +#define JALR_T0 (0x000282e7)
> +#define AUIPC_T0 (0x00000297)
> #define NOP4 (0x00000013)
>
> -#define make_call(caller, callee, call) \
> +#define to_jalr_t0(offset) \
> + (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0)
> +
> +#define to_auipc_t0(offset) \
> + ((offset & JALR_SIGN_MASK) ? \
> + (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_T0) : \
> + ((offset & AUIPC_OFFSET_MASK) | AUIPC_T0))
> +
> +#define make_call_t0(caller, callee, call) \
> do { \
> - call[0] = to_auipc_insn((unsigned int)((unsigned long)callee - \
> - (unsigned long)caller)); \
> - call[1] = to_jalr_insn((unsigned int)((unsigned long)callee - \
> - (unsigned long)caller)); \
> + unsigned int offset = \
> + (unsigned long) callee - (unsigned long) caller; \
> + call[0] = to_auipc_t0(offset); \
> + call[1] = to_jalr_t0(offset); \
> } while (0)
>
> -#define to_jalr_insn(offset) \
> - (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_BASIC)
> +#define to_jalr_ra(offset) \
> + (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_RA)
>
> -#define to_auipc_insn(offset) \
> +#define to_auipc_ra(offset) \
> ((offset & JALR_SIGN_MASK) ? \
> - (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_BASIC) : \
> - ((offset & AUIPC_OFFSET_MASK) | AUIPC_BASIC))
> + (((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) | AUIPC_RA) : \
> + ((offset & AUIPC_OFFSET_MASK) | AUIPC_RA))
> +
> +#define make_call_ra(caller, callee, call) \
> +do { \
> + unsigned int offset = \
> + (unsigned long) callee - (unsigned long) caller; \
> + call[0] = to_auipc_ra(offset); \
> + call[1] = to_jalr_ra(offset); \
> +} while (0)
>
> /*
> * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here.
> diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
> index 2086f6585773..5bff37af4770 100644
> --- a/arch/riscv/kernel/ftrace.c
> +++ b/arch/riscv/kernel/ftrace.c
> @@ -55,12 +55,15 @@ static int ftrace_check_current_call(unsigned long hook_pos,
> }
>
> static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
> - bool enable)
> + bool enable, bool ra)
> {
> unsigned int call[2];
> unsigned int nops[2] = {NOP4, NOP4};
>
> - make_call(hook_pos, target, call);
> + if (ra)
> + make_call_ra(hook_pos, target, call);
> + else
> + make_call_t0(hook_pos, target, call);
>
> /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */
> if (patch_text_nosync
> @@ -70,42 +73,13 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target,
> return 0;
> }
>
> -/*
> - * Put 5 instructions with 16 bytes at the front of function within
> - * patchable function entry nops' area.
> - *
> - * 0: REG_S ra, -SZREG(sp)
> - * 1: auipc ra, 0x?
> - * 2: jalr -?(ra)
> - * 3: REG_L ra, -SZREG(sp)
> - *
> - * So the opcodes is:
> - * 0: 0xfe113c23 (sd)/0xfe112e23 (sw)
> - * 1: 0x???????? -> auipc
> - * 2: 0x???????? -> jalr
> - * 3: 0xff813083 (ld)/0xffc12083 (lw)
> - */
> -#if __riscv_xlen == 64
> -#define INSN0 0xfe113c23
> -#define INSN3 0xff813083
> -#elif __riscv_xlen == 32
> -#define INSN0 0xfe112e23
> -#define INSN3 0xffc12083
> -#endif
> -
> -#define FUNC_ENTRY_SIZE 16
> -#define FUNC_ENTRY_JMP 4
> -
> int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> {
> - unsigned int call[4] = {INSN0, 0, 0, INSN3};
> - unsigned long target = addr;
> - unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
> + unsigned int call[2];
>
> - call[1] = to_auipc_insn((unsigned int)(target - caller));
> - call[2] = to_jalr_insn((unsigned int)(target - caller));
> + make_call_t0(rec->ip, addr, call);
>
> - if (patch_text_nosync((void *)rec->ip, call, FUNC_ENTRY_SIZE))
> + if (patch_text_nosync((void *)rec->ip, call, MCOUNT_INSN_SIZE))
> return -EPERM;
>
> return 0;
> @@ -114,15 +88,14 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> unsigned long addr)
> {
> - unsigned int nops[4] = {NOP4, NOP4, NOP4, NOP4};
> + unsigned int nops[2] = {NOP4, NOP4};
>
> - if (patch_text_nosync((void *)rec->ip, nops, FUNC_ENTRY_SIZE))
> + if (patch_text_nosync((void *)rec->ip, nops, MCOUNT_INSN_SIZE))
> return -EPERM;
>
> return 0;
> }
>
> -
> /*
> * This is called early on, and isn't wrapped by
> * ftrace_arch_code_modify_{prepare,post_process}() and therefor doesn't hold
> @@ -144,10 +117,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
> int ftrace_update_ftrace_func(ftrace_func_t func)
> {
> int ret = __ftrace_modify_call((unsigned long)&ftrace_call,
> - (unsigned long)func, true);
> + (unsigned long)func, true, true);
> if (!ret) {
> ret = __ftrace_modify_call((unsigned long)&ftrace_regs_call,
> - (unsigned long)func, true);
> + (unsigned long)func, true, true);
> }
>
> return ret;
> @@ -159,16 +132,16 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> unsigned long addr)
> {
> unsigned int call[2];
> - unsigned long caller = rec->ip + FUNC_ENTRY_JMP;
> + unsigned long caller = rec->ip;
> int ret;
>
> - make_call(caller, old_addr, call);
> + make_call_t0(caller, old_addr, call);
> ret = ftrace_check_current_call(caller, call);
>
> if (ret)
> return ret;
>
> - return __ftrace_modify_call(caller, addr, true);
> + return __ftrace_modify_call(caller, addr, true, false);
> }
> #endif
>
> @@ -203,12 +176,12 @@ int ftrace_enable_ftrace_graph_caller(void)
> int ret;
>
> ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
> - (unsigned long)&prepare_ftrace_return, true);
> + (unsigned long)&prepare_ftrace_return, true, true);
> if (ret)
> return ret;
>
> return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
> - (unsigned long)&prepare_ftrace_return, true);
> + (unsigned long)&prepare_ftrace_return, true, true);
> }
>
> int ftrace_disable_ftrace_graph_caller(void)
> @@ -216,12 +189,12 @@ int ftrace_disable_ftrace_graph_caller(void)
> int ret;
>
> ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
> - (unsigned long)&prepare_ftrace_return, false);
> + (unsigned long)&prepare_ftrace_return, false, true);
> if (ret)
> return ret;
>
> return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
> - (unsigned long)&prepare_ftrace_return, false);
> + (unsigned long)&prepare_ftrace_return, false, true);
> }
> #endif /* CONFIG_DYNAMIC_FTRACE */
> #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
> index d171eca623b6..125de818d1ba 100644
> --- a/arch/riscv/kernel/mcount-dyn.S
> +++ b/arch/riscv/kernel/mcount-dyn.S
> @@ -13,8 +13,8 @@
>
> .text
>
> -#define FENTRY_RA_OFFSET 12
> -#define ABI_SIZE_ON_STACK 72
> +#define FENTRY_RA_OFFSET 8
> +#define ABI_SIZE_ON_STACK 80
> #define ABI_A0 0
> #define ABI_A1 8
> #define ABI_A2 16
> @@ -23,10 +23,10 @@
> #define ABI_A5 40
> #define ABI_A6 48
> #define ABI_A7 56
> -#define ABI_RA 64
> +#define ABI_T0 64
> +#define ABI_RA 72
>
> .macro SAVE_ABI
> - addi sp, sp, -SZREG
> addi sp, sp, -ABI_SIZE_ON_STACK
>
> REG_S a0, ABI_A0(sp)
> @@ -37,6 +37,7 @@
> REG_S a5, ABI_A5(sp)
> REG_S a6, ABI_A6(sp)
> REG_S a7, ABI_A7(sp)
> + REG_S t0, ABI_T0(sp)
> REG_S ra, ABI_RA(sp)
> .endm
>
> @@ -49,24 +50,18 @@
> REG_L a5, ABI_A5(sp)
> REG_L a6, ABI_A6(sp)
> REG_L a7, ABI_A7(sp)
> + REG_L t0, ABI_T0(sp)
> REG_L ra, ABI_RA(sp)
>
> addi sp, sp, ABI_SIZE_ON_STACK
> - addi sp, sp, SZREG
> .endm
>
> #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> .macro SAVE_ALL
> - addi sp, sp, -SZREG
> addi sp, sp, -PT_SIZE_ON_STACK
>
> - REG_S x1, PT_EPC(sp)
> - addi sp, sp, PT_SIZE_ON_STACK
> - REG_L x1, (sp)
> - addi sp, sp, -PT_SIZE_ON_STACK
> + REG_S t0, PT_EPC(sp)
> REG_S x1, PT_RA(sp)
> - REG_L x1, PT_EPC(sp)
> -
> REG_S x2, PT_SP(sp)
> REG_S x3, PT_GP(sp)
> REG_S x4, PT_TP(sp)
> @@ -100,15 +95,11 @@
> .endm
>
> .macro RESTORE_ALL
> + REG_L t0, PT_EPC(sp)
> REG_L x1, PT_RA(sp)
> - addi sp, sp, PT_SIZE_ON_STACK
> - REG_S x1, (sp)
> - addi sp, sp, -PT_SIZE_ON_STACK
> - REG_L x1, PT_EPC(sp)
> REG_L x2, PT_SP(sp)
> REG_L x3, PT_GP(sp)
> REG_L x4, PT_TP(sp)
> - REG_L x5, PT_T0(sp)
> REG_L x6, PT_T1(sp)
> REG_L x7, PT_T2(sp)
> REG_L x8, PT_S0(sp)
> @@ -137,17 +128,16 @@
> REG_L x31, PT_T6(sp)
>
> addi sp, sp, PT_SIZE_ON_STACK
> - addi sp, sp, SZREG
> .endm
> #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
>
> ENTRY(ftrace_caller)
> SAVE_ABI
>
> - addi a0, ra, -FENTRY_RA_OFFSET
> + addi a0, t0, -FENTRY_RA_OFFSET
> la a1, function_trace_op
> REG_L a2, 0(a1)
> - REG_L a1, ABI_SIZE_ON_STACK(sp)
> + mv a1, ra
> mv a3, sp
>
> ftrace_call:
> @@ -155,8 +145,8 @@ ftrace_call:
> call ftrace_stub
>
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> - addi a0, sp, ABI_SIZE_ON_STACK
> - REG_L a1, ABI_RA(sp)
> + addi a0, sp, ABI_RA
> + REG_L a1, ABI_T0(sp)
> addi a1, a1, -FENTRY_RA_OFFSET
> #ifdef HAVE_FUNCTION_GRAPH_FP_TEST
> mv a2, s0
> @@ -166,17 +156,17 @@ ftrace_graph_call:
> call ftrace_stub
> #endif
> RESTORE_ABI
> - ret
> + jr t0
> ENDPROC(ftrace_caller)
>
> #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> ENTRY(ftrace_regs_caller)
> SAVE_ALL
>
> - addi a0, ra, -FENTRY_RA_OFFSET
> + addi a0, t0, -FENTRY_RA_OFFSET
> la a1, function_trace_op
> REG_L a2, 0(a1)
> - REG_L a1, PT_SIZE_ON_STACK(sp)
> + mv a1, ra
> mv a3, sp
>
> ftrace_regs_call:
> @@ -196,6 +186,6 @@ ftrace_graph_regs_call:
> #endif
>
> RESTORE_ALL
> - ret
> + jr t0
> ENDPROC(ftrace_regs_caller)
> #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
> --
> 2.36.1
>
>
Looks good to me.
I also re-checked if "replacement" of cmdline_proc_show() with a custom
function via Ftrace works in this case - it does. Rollback to the
original cmdline_proc_show() seems to work OK too.
Reviewed-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
Regards,
Evgenii
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-12 9:06 ` guoren
@ 2023-01-16 14:30 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 14:30 UTC (permalink / raw)
To: guoren, suagrfillet
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi,
On 12.01.2023 12:06, guoren@kernel.org wrote:
> From: Song Shuai <suagrfillet@gmail.com>
>
> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> the ftrace-direct*.c files in samples/ftrace/.
>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Tested-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 2 ++
> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> 6 files changed, 142 insertions(+)
The samples were built OK now, but ftrace-direct-multi and
ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
the messages in the trace:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
direct func1 ip 0
migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
direct func1 ip ff60000001ba9600
migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
direct func1 ip ff60000003334d00
migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
direct func1 ip 0
rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
direct func1 ip 0
insmod-415 [000] ..... 3858.532270: my_direct_func1: my
direct func1 ip 7fffffffffffffff
<idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
direct func1 ip ff60000001ba9600
<idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
direct func1 ip ff60000001ba9600
rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
direct func1 ip 0
[...]
If I understand it right, my_direct_func1() should print the address of
some location in the code, probably - at the beginning of the traced
functions.
The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
addresses.
The same issue is with ftrace-direct-multi-modify.ko.
Is anything missing here?
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 307a9f413edd..e944af44f681 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -112,6 +112,8 @@ config RISCV
> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> select HAVE_REGS_AND_STACK_ACCESS_API
> select HAVE_FUNCTION_ARG_ACCESS_API
> + select HAVE_SAMPLE_FTRACE_DIRECT
> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> select HAVE_STACKPROTECTOR
> select HAVE_SYSCALL_TRACEPOINTS
> select HAVE_RSEQ
> diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
> index de5a0f67f320..be7bf472c3c7 100644
> --- a/samples/ftrace/ftrace-direct-modify.c
> +++ b/samples/ftrace/ftrace-direct-modify.c
> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>
> static unsigned long my_ip = (unsigned long)schedule;
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp1, @function\n"
> +" .globl my_tramp1\n"
> +" my_tramp1:\n"
> +" addi sp,sp,-16\n"
> +" sd t0,0(sp)\n"
> +" sd ra,8(sp)\n"
> +" call my_direct_func1\n"
> +" ld t0,0(sp)\n"
> +" ld ra,8(sp)\n"
> +" addi sp,sp,16\n"
> +" jr t0\n"
> +" .size my_tramp1, .-my_tramp1\n"
> +
> +" .type my_tramp2, @function\n"
> +" .globl my_tramp2\n"
> +" my_tramp2:\n"
> +" addi sp,sp,-16\n"
> +" sd t0,0(sp)\n"
> +" sd ra,8(sp)\n"
> +" call my_direct_func2\n"
> +" ld t0,0(sp)\n"
> +" ld ra,8(sp)\n"
> +" addi sp,sp,16\n"
> +" jr t0\n"
> +" .size my_tramp2, .-my_tramp2\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
> index d52370cad0b6..10884bf418f7 100644
> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> extern void my_tramp1(void *);
> extern void my_tramp2(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp1, @function\n"
> +" .globl my_tramp1\n"
> +" my_tramp1:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func1\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp1, .-my_tramp1\n"
> +
> +" .type my_tramp2, @function\n"
> +" .globl my_tramp2\n"
> +" my_tramp2:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func2\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp2, .-my_tramp2\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
> index ec1088922517..a35bf43bf6d7 100644
> --- a/samples/ftrace/ftrace-direct-multi.c
> +++ b/samples/ftrace/ftrace-direct-multi.c
> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
> index e13fb59a2b47..3b62e33c2e6d 100644
> --- a/samples/ftrace/ftrace-direct-too.c
> +++ b/samples/ftrace/ftrace-direct-too.c
> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-40\n"
> +" sd a0,0(sp)\n"
> +" sd a1,8(sp)\n"
> +" sd a2,16(sp)\n"
> +" sd t0,24(sp)\n"
> +" sd ra,32(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld a1,8(sp)\n"
> +" ld a2,16(sp)\n"
> +" ld t0,24(sp)\n"
> +" ld ra,32(sp)\n"
> +" addi sp,sp,40\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
> index 1f769d0db20f..2cfe5a7d2d70 100644
> --- a/samples/ftrace/ftrace-direct.c
> +++ b/samples/ftrace/ftrace-direct.c
> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> --
> 2.36.1
>
>
Regards,
Evgenii
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-16 14:30 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 14:30 UTC (permalink / raw)
To: guoren, suagrfillet
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi,
On 12.01.2023 12:06, guoren@kernel.org wrote:
> From: Song Shuai <suagrfillet@gmail.com>
>
> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> the ftrace-direct*.c files in samples/ftrace/.
>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Tested-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 2 ++
> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> 6 files changed, 142 insertions(+)
The samples were built OK now, but ftrace-direct-multi and
ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
the messages in the trace:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
direct func1 ip 0
migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
direct func1 ip ff60000001ba9600
migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
direct func1 ip ff60000003334d00
migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
direct func1 ip 0
rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
direct func1 ip 0
insmod-415 [000] ..... 3858.532270: my_direct_func1: my
direct func1 ip 7fffffffffffffff
<idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
direct func1 ip ff60000001ba9600
<idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
direct func1 ip ff60000001ba9600
rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
direct func1 ip 0
[...]
If I understand it right, my_direct_func1() should print the address of
some location in the code, probably - at the beginning of the traced
functions.
The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
addresses.
The same issue is with ftrace-direct-multi-modify.ko.
Is anything missing here?
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 307a9f413edd..e944af44f681 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -112,6 +112,8 @@ config RISCV
> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> select HAVE_REGS_AND_STACK_ACCESS_API
> select HAVE_FUNCTION_ARG_ACCESS_API
> + select HAVE_SAMPLE_FTRACE_DIRECT
> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> select HAVE_STACKPROTECTOR
> select HAVE_SYSCALL_TRACEPOINTS
> select HAVE_RSEQ
> diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
> index de5a0f67f320..be7bf472c3c7 100644
> --- a/samples/ftrace/ftrace-direct-modify.c
> +++ b/samples/ftrace/ftrace-direct-modify.c
> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>
> static unsigned long my_ip = (unsigned long)schedule;
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp1, @function\n"
> +" .globl my_tramp1\n"
> +" my_tramp1:\n"
> +" addi sp,sp,-16\n"
> +" sd t0,0(sp)\n"
> +" sd ra,8(sp)\n"
> +" call my_direct_func1\n"
> +" ld t0,0(sp)\n"
> +" ld ra,8(sp)\n"
> +" addi sp,sp,16\n"
> +" jr t0\n"
> +" .size my_tramp1, .-my_tramp1\n"
> +
> +" .type my_tramp2, @function\n"
> +" .globl my_tramp2\n"
> +" my_tramp2:\n"
> +" addi sp,sp,-16\n"
> +" sd t0,0(sp)\n"
> +" sd ra,8(sp)\n"
> +" call my_direct_func2\n"
> +" ld t0,0(sp)\n"
> +" ld ra,8(sp)\n"
> +" addi sp,sp,16\n"
> +" jr t0\n"
> +" .size my_tramp2, .-my_tramp2\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
> index d52370cad0b6..10884bf418f7 100644
> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> extern void my_tramp1(void *);
> extern void my_tramp2(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp1, @function\n"
> +" .globl my_tramp1\n"
> +" my_tramp1:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func1\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp1, .-my_tramp1\n"
> +
> +" .type my_tramp2, @function\n"
> +" .globl my_tramp2\n"
> +" my_tramp2:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func2\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp2, .-my_tramp2\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
> index ec1088922517..a35bf43bf6d7 100644
> --- a/samples/ftrace/ftrace-direct-multi.c
> +++ b/samples/ftrace/ftrace-direct-multi.c
> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
> index e13fb59a2b47..3b62e33c2e6d 100644
> --- a/samples/ftrace/ftrace-direct-too.c
> +++ b/samples/ftrace/ftrace-direct-too.c
> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-40\n"
> +" sd a0,0(sp)\n"
> +" sd a1,8(sp)\n"
> +" sd a2,16(sp)\n"
> +" sd t0,24(sp)\n"
> +" sd ra,32(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld a1,8(sp)\n"
> +" ld a2,16(sp)\n"
> +" ld t0,24(sp)\n"
> +" ld ra,32(sp)\n"
> +" addi sp,sp,40\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
> index 1f769d0db20f..2cfe5a7d2d70 100644
> --- a/samples/ftrace/ftrace-direct.c
> +++ b/samples/ftrace/ftrace-direct.c
> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>
> extern void my_tramp(void *);
>
> +#ifdef CONFIG_RISCV
> +
> +asm (" .pushsection .text, \"ax\", @progbits\n"
> +" .type my_tramp, @function\n"
> +" .globl my_tramp\n"
> +" my_tramp:\n"
> +" addi sp,sp,-24\n"
> +" sd a0,0(sp)\n"
> +" sd t0,8(sp)\n"
> +" sd ra,16(sp)\n"
> +" call my_direct_func\n"
> +" ld a0,0(sp)\n"
> +" ld t0,8(sp)\n"
> +" ld ra,16(sp)\n"
> +" addi sp,sp,24\n"
> +" jr t0\n"
> +" .size my_tramp, .-my_tramp\n"
> +" .popsection\n"
> +);
> +
> +#endif /* CONFIG_RISCV */
> +
> #ifdef CONFIG_X86_64
>
> #include <asm/ibt.h>
> --
> 2.36.1
>
>
Regards,
Evgenii
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-01-12 9:05 ` guoren
@ 2023-01-16 15:02 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 15:02 UTC (permalink / raw)
To: guoren, suagrfillet, andy.chiu
Cc: linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
Hi,
On 12.01.2023 12:05, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
>
> - The most horrible bug is preemption panic which found by Andy [1].
> Let's disable preemption for ftrace first, and Andy could continue
> the ftrace preemption work.
It seems, the patches #2-#7 of this series do not require "riscv:
ftrace: Fixup panic by disabling preemption" and can be used without it.
How about moving that patch out of the series and processing it separately?
As it was pointed out in the discussion of that patch, some other
solution to non-atomic changes of the prologue might be needed anyway.
> - The "-fpatchable-function-entry= CFLAG" wasted code size
> !RISCV_ISA_C.
> - The ftrace detour implementation wasted code size.
> - When livepatching, the trampoline (ftrace_regs_caller) would not
> return to <func_prolog+12> but would rather jump to the new function.
> So, "REG_L ra, -SZREG(sp)" would not run and the original return
> address would not be restored. The kernel is likely to hang or crash
> as a result. (Found by Evgenii Shatokhin [4])
>
> Patches 1,2,3 fixup above problems.
>
> Patches 4,5,6,7 are the features based on reduced detour code
> patch, we include them in the series for test and maintenance.
>
> You can directly try it with:
> https://github.com/guoren83/linux/tree/ftrace_fixup_v7
>
> Make function graph use ftrace directly [2] (patch 4, 5)
> ========================================================
>
> In RISC-V architecture, when we enable the ftrace_graph tracer on some
> functions, the function tracings on other functions will suffer extra
> graph tracing work. In essence, graph_ops isn't limited by its func_hash
> due to the global ftrace_graph_[regs]_call label. That should be
> corrected.
>
> What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> graph use ftrace directly") that uses graph_ops::func function to
> install return_hooker and makes the function called against its
> func_hash.
>
> This series of patches makes function graph use ftrace directly for
> riscv.
>
> If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
> so that it can be replaced with the calling of prepare_ftrace_return by
> the enable/disable helper.
>
> As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
> necessary regs against the pt_regs layout, so it can reasonably call the
> graph_ops::func function - ftrace_graph_func. And
> ftrace_graph_[regs]_call
> and its enable/disable helper aren't needed.
>
> Test log:
>
> The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
> local
> qemu-system-riscv64 virt machine. The following is the log during
> startup.
>
> ```
> Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
> Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
> Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
> Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
> Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
> Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
> Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
> Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
> Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
> Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
> Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
> Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
> Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
> Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
> Nov 15 03:07:13 stage4 kernel: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
> ```
>
> Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
> ==============================================
>
> This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> here as the samples for testing DIRECT_CALLS related interface.
>
> First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> register_ftrace_direct[_multi] interfaces allowing user to register
> the customed trampoline (direct_caller) as the mcount for one or
> more target functions. And modify_ftrace_direct[_multi] are also
> provided for modify direct_caller.
>
> At the same time, the samples in ./samples/ftrace/ can be built
> as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> and SAMPLE_FTRACE_DIRECT_MULTI selected.
>
> Second, to make the direct_caller and the other ftrace hooks
> (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> register
> are nominated to store the address of direct_caller in
> ftrace_regs_caller.
> After the setting of the address direct_caller by direct_ops->func and
> the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> by the `jr` inst.
>
> The following tests have been passed in my local qemu-riscv64 virt
> machine.
>
> 1. tests with CONFIG_FTRACE_STARTUP_TEST
> 2. tests of samples/ftrace/ftrace*.ko
> 3. manual tests with any combination of the following hooks
> - function/function_graph tracer
> - ftrace*.ko
> - kprobe/kretprobe
>
> For your reference, here is the log when function tracer, kretprobe and
> ftrace-direct-too.ko co-hooks the handle_mm_fault function.
>
> ```
> [root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
> [root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
> [root@stage4 tracing]# echo function > current_tracer
> [root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
> [root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
> [root@stage4 tracing]#
> [root@stage4 tracing]# cat trace | tail
> cat-388 [000] ...1. 583.051438: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> cat-388 [000] ...2. 583.057930: handle_mm_fault
> <-do_page_fault
> cat-388 [000] ..... 583.057990: my_direct_func:
> handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
> cat-388 [000] ...1. 583.058284: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> tail-389 [001] ...2. 583.059062: handle_mm_fault
> <-do_page_fault
> tail-389 [001] ..... 583.059104: my_direct_func:
> handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
> tail-389 [001] ...1. 583.059325: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> tail-389 [001] ...2. 583.060371: handle_mm_fault
> <-do_page_fault
> tail-389 [001] ..... 583.060410: my_direct_func:
> handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
> tail-389 [001] ...1. 583.060996: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> ```
> Note1:
>
> The checkpatch.pl will output some warnings on this series, like this
>
> ```
> WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
> this function's name, in a string
> 111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
> +" call my_direct_func2\n"
> ```
>
> The reason is that checkpatch depends on patch context providing the
> function name. In the above warning, my_direct_func2 has some codeline
> distance with the changed trunk, so its declaration doesn't come into
> the patch, and then the warning jumps out.
>
> You may notice the location of `my_ip` variable changes in the 2nd
> patch. I did that for reducing the warnings to some extent. But killing
> all the warnings will makes the patch less readable, so I stopped here.
>
> [1] https://lpc.events/event/16/contributions/1171/
> [2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> [3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
> [4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
>
> Changes in v7:
> - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
> - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
> HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>
> Changes in v6:
> https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
> - Replace 8 with MCOUNT_INSN_SIZE
> - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
> - Add Evgenii Shatokhin comment
>
> Changes in v5:
> https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
> - Sort Kconfig entries in alphabetical order.
>
> Changes in v4:
> https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
> - Include [3] for maintenance. [Song Shuai]
>
> Changes in V3:
> https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
> - Include [2] for maintenance. [Song Shuai]
>
> Changes in V2:
> https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
> - Add Signed-off for preemption fixup.
>
> Changes in V1:
> https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
>
> Andy Chiu (1):
> riscv: ftrace: Fixup panic by disabling preemption
>
> Guo Ren (2):
> riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
> riscv: ftrace: Reduce the detour code size to half
>
> Song Shuai (4):
> riscv: ftrace: Add ftrace_graph_func
> riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
> samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
> riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
>
> arch/riscv/Kconfig | 6 +-
> arch/riscv/Makefile | 6 +-
> arch/riscv/include/asm/ftrace.h | 71 ++++++--
> arch/riscv/kernel/ftrace.c | 91 ++++------
> arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
> samples/ftrace/ftrace-direct-modify.c | 33 ++++
> samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
> samples/ftrace/ftrace-direct-multi.c | 22 +++
> samples/ftrace/ftrace-direct-too.c | 26 +++
> samples/ftrace/ftrace-direct.c | 22 +++
> 10 files changed, 356 insertions(+), 137 deletions(-)
>
> --
> 2.36.1
>
>
I tested this series a bit on top of 6.2-rc4, with
https://github.com/sugarfillet/linux/commit/9539a80dc6e7d1137ec7a96ebef2ab912a694bd7.patch
added.
The kernel was built with CONFIG_DYNAMIC_FTRACE_WITH_REGS=y and
CONFIG_RISCV_ISA_C=y.
Here are some results:
* The kernel was built without visible issues.
* Most of the Ftrace startup tests ran and passed. Certain
event-specific tests caused soft lockups in one of the test runs but
they are likely unrelated to this patchset and could happen without it too.
* "function" and "function_graph" tracers worked in my simple use cases.
* "ipmodify" functionality worked.
* kprobe sample modules worked OK, which is good, because they actually
use Ftrace: they plant the probes at the beginning of the resp. functions.
* ftrace-direct-multi and ftrace-direct-multi-modify sample modules
reported possibly invalid data. More info - in my reply to
"[PATCH -next V7 6/7] samples: ftrace: Add riscv support for
SAMPLE_FTRACE_DIRECT[_MULTI]"
Regards,
Evgenii
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-01-16 15:02 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-16 15:02 UTC (permalink / raw)
To: guoren, suagrfillet, andy.chiu
Cc: linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
Hi,
On 12.01.2023 12:05, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
>
> - The most horrible bug is preemption panic which found by Andy [1].
> Let's disable preemption for ftrace first, and Andy could continue
> the ftrace preemption work.
It seems, the patches #2-#7 of this series do not require "riscv:
ftrace: Fixup panic by disabling preemption" and can be used without it.
How about moving that patch out of the series and processing it separately?
As it was pointed out in the discussion of that patch, some other
solution to non-atomic changes of the prologue might be needed anyway.
> - The "-fpatchable-function-entry= CFLAG" wasted code size
> !RISCV_ISA_C.
> - The ftrace detour implementation wasted code size.
> - When livepatching, the trampoline (ftrace_regs_caller) would not
> return to <func_prolog+12> but would rather jump to the new function.
> So, "REG_L ra, -SZREG(sp)" would not run and the original return
> address would not be restored. The kernel is likely to hang or crash
> as a result. (Found by Evgenii Shatokhin [4])
>
> Patches 1,2,3 fixup above problems.
>
> Patches 4,5,6,7 are the features based on reduced detour code
> patch, we include them in the series for test and maintenance.
>
> You can directly try it with:
> https://github.com/guoren83/linux/tree/ftrace_fixup_v7
>
> Make function graph use ftrace directly [2] (patch 4, 5)
> ========================================================
>
> In RISC-V architecture, when we enable the ftrace_graph tracer on some
> functions, the function tracings on other functions will suffer extra
> graph tracing work. In essence, graph_ops isn't limited by its func_hash
> due to the global ftrace_graph_[regs]_call label. That should be
> corrected.
>
> What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> graph use ftrace directly") that uses graph_ops::func function to
> install return_hooker and makes the function called against its
> func_hash.
>
> This series of patches makes function graph use ftrace directly for
> riscv.
>
> If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
> so that it can be replaced with the calling of prepare_ftrace_return by
> the enable/disable helper.
>
> As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
> necessary regs against the pt_regs layout, so it can reasonably call the
> graph_ops::func function - ftrace_graph_func. And
> ftrace_graph_[regs]_call
> and its enable/disable helper aren't needed.
>
> Test log:
>
> The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
> local
> qemu-system-riscv64 virt machine. The following is the log during
> startup.
>
> ```
> Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
> Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
> Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
> Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
> Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
> Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
> Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
> Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
> Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
> Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
> Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
> Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
> Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
> Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
> Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
> Nov 15 03:07:13 stage4 kernel: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
> Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
> ```
>
> Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
> ==============================================
>
> This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> here as the samples for testing DIRECT_CALLS related interface.
>
> First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> register_ftrace_direct[_multi] interfaces allowing user to register
> the customed trampoline (direct_caller) as the mcount for one or
> more target functions. And modify_ftrace_direct[_multi] are also
> provided for modify direct_caller.
>
> At the same time, the samples in ./samples/ftrace/ can be built
> as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> and SAMPLE_FTRACE_DIRECT_MULTI selected.
>
> Second, to make the direct_caller and the other ftrace hooks
> (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> register
> are nominated to store the address of direct_caller in
> ftrace_regs_caller.
> After the setting of the address direct_caller by direct_ops->func and
> the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> by the `jr` inst.
>
> The following tests have been passed in my local qemu-riscv64 virt
> machine.
>
> 1. tests with CONFIG_FTRACE_STARTUP_TEST
> 2. tests of samples/ftrace/ftrace*.ko
> 3. manual tests with any combination of the following hooks
> - function/function_graph tracer
> - ftrace*.ko
> - kprobe/kretprobe
>
> For your reference, here is the log when function tracer, kretprobe and
> ftrace-direct-too.ko co-hooks the handle_mm_fault function.
>
> ```
> [root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
> [root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
> [root@stage4 tracing]# echo function > current_tracer
> [root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
> [root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
> [root@stage4 tracing]#
> [root@stage4 tracing]# cat trace | tail
> cat-388 [000] ...1. 583.051438: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> cat-388 [000] ...2. 583.057930: handle_mm_fault
> <-do_page_fault
> cat-388 [000] ..... 583.057990: my_direct_func:
> handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
> cat-388 [000] ...1. 583.058284: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> tail-389 [001] ...2. 583.059062: handle_mm_fault
> <-do_page_fault
> tail-389 [001] ..... 583.059104: my_direct_func:
> handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
> tail-389 [001] ...1. 583.059325: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> tail-389 [001] ...2. 583.060371: handle_mm_fault
> <-do_page_fault
> tail-389 [001] ..... 583.060410: my_direct_func:
> handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
> tail-389 [001] ...1. 583.060996: myr:
> (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> ```
> Note1:
>
> The checkpatch.pl will output some warnings on this series, like this
>
> ```
> WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
> this function's name, in a string
> 111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
> +" call my_direct_func2\n"
> ```
>
> The reason is that checkpatch depends on patch context providing the
> function name. In the above warning, my_direct_func2 has some codeline
> distance with the changed trunk, so its declaration doesn't come into
> the patch, and then the warning jumps out.
>
> You may notice the location of `my_ip` variable changes in the 2nd
> patch. I did that for reducing the warnings to some extent. But killing
> all the warnings will makes the patch less readable, so I stopped here.
>
> [1] https://lpc.events/event/16/contributions/1171/
> [2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> [3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
> [4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
>
> Changes in v7:
> - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
> - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
> HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>
> Changes in v6:
> https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
> - Replace 8 with MCOUNT_INSN_SIZE
> - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
> - Add Evgenii Shatokhin comment
>
> Changes in v5:
> https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
> - Sort Kconfig entries in alphabetical order.
>
> Changes in v4:
> https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
> - Include [3] for maintenance. [Song Shuai]
>
> Changes in V3:
> https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
> - Include [2] for maintenance. [Song Shuai]
>
> Changes in V2:
> https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
> - Add Signed-off for preemption fixup.
>
> Changes in V1:
> https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
>
> Andy Chiu (1):
> riscv: ftrace: Fixup panic by disabling preemption
>
> Guo Ren (2):
> riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
> riscv: ftrace: Reduce the detour code size to half
>
> Song Shuai (4):
> riscv: ftrace: Add ftrace_graph_func
> riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
> samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
> riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
>
> arch/riscv/Kconfig | 6 +-
> arch/riscv/Makefile | 6 +-
> arch/riscv/include/asm/ftrace.h | 71 ++++++--
> arch/riscv/kernel/ftrace.c | 91 ++++------
> arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
> samples/ftrace/ftrace-direct-modify.c | 33 ++++
> samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
> samples/ftrace/ftrace-direct-multi.c | 22 +++
> samples/ftrace/ftrace-direct-too.c | 26 +++
> samples/ftrace/ftrace-direct.c | 22 +++
> 10 files changed, 356 insertions(+), 137 deletions(-)
>
> --
> 2.36.1
>
>
I tested this series a bit on top of 6.2-rc4, with
https://github.com/sugarfillet/linux/commit/9539a80dc6e7d1137ec7a96ebef2ab912a694bd7.patch
added.
The kernel was built with CONFIG_DYNAMIC_FTRACE_WITH_REGS=y and
CONFIG_RISCV_ISA_C=y.
Here are some results:
* The kernel was built without visible issues.
* Most of the Ftrace startup tests ran and passed. Certain
event-specific tests caused soft lockups in one of the test runs but
they are likely unrelated to this patchset and could happen without it too.
* "function" and "function_graph" tracers worked in my simple use cases.
* "ipmodify" functionality worked.
* kprobe sample modules worked OK, which is good, because they actually
use Ftrace: they plant the probes at the beginning of the resp. functions.
* ftrace-direct-multi and ftrace-direct-multi-modify sample modules
reported possibly invalid data. More info - in my reply to
"[PATCH -next V7 6/7] samples: ftrace: Add riscv support for
SAMPLE_FTRACE_DIRECT[_MULTI]"
Regards,
Evgenii
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-16 14:30 ` Evgenii Shatokhin
@ 2023-01-17 9:32 ` Song Shuai
-1 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-01-17 9:32 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi, Evgenii:
Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>
> Hi,
>
> On 12.01.2023 12:06, guoren@kernel.org wrote:
> > From: Song Shuai <suagrfillet@gmail.com>
> >
> > select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > the ftrace-direct*.c files in samples/ftrace/.
> >
> > Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > Tested-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 2 ++
> > samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> > samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
> > samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> > samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> > samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> > 6 files changed, 142 insertions(+)
>
> The samples were built OK now, but ftrace-direct-multi and
> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>
> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> the messages in the trace:
>
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> direct func1 ip 0
> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> direct func1 ip ff60000003334d00
> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> direct func1 ip 0
> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> direct func1 ip 0
> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> direct func1 ip 7fffffffffffffff
> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> direct func1 ip 0
> [...]
>
> If I understand it right, my_direct_func1() should print the address of
> some location in the code, probably - at the beginning of the traced
> functions.
>
> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> addresses.
>
The invalid code address is only printed by accessing the schedule()
function's first argument whose address stores in a0 register.
While schedule() actually has no parameter declared, so my_direct_func
just prints the a0 in the context of the schedule()'s caller and
the address maybe varies depending on the caller.
I can't really understand why tracing the first argument of the
schedule() function, but it seems nonsense at this point.
As for this patch, it just impls a simple mcount (direct_caller) to
trace kernel functions, and basically saves the necessary ABI,
call the tracing function, and restores the ABI, just like other arches do.
so It shouldn't be blamed.
I started an independent patch to replace schedule with kick_process
to make these samples more reasonable. And It has no conflict with the
current patch, so we can go on.
Link: https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> The same issue is with ftrace-direct-multi-modify.ko.
>
> Is anything missing here?
>
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 307a9f413edd..e944af44f681 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -112,6 +112,8 @@ config RISCV
> > select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> > select HAVE_REGS_AND_STACK_ACCESS_API
> > select HAVE_FUNCTION_ARG_ACCESS_API
> > + select HAVE_SAMPLE_FTRACE_DIRECT
> > + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > select HAVE_STACKPROTECTOR
> > select HAVE_SYSCALL_TRACEPOINTS
> > select HAVE_RSEQ
> > diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
> > index de5a0f67f320..be7bf472c3c7 100644
> > --- a/samples/ftrace/ftrace-direct-modify.c
> > +++ b/samples/ftrace/ftrace-direct-modify.c
> > @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >
> > static unsigned long my_ip = (unsigned long)schedule;
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp1, @function\n"
> > +" .globl my_tramp1\n"
> > +" my_tramp1:\n"
> > +" addi sp,sp,-16\n"
> > +" sd t0,0(sp)\n"
> > +" sd ra,8(sp)\n"
> > +" call my_direct_func1\n"
> > +" ld t0,0(sp)\n"
> > +" ld ra,8(sp)\n"
> > +" addi sp,sp,16\n"
> > +" jr t0\n"
> > +" .size my_tramp1, .-my_tramp1\n"
> > +
> > +" .type my_tramp2, @function\n"
> > +" .globl my_tramp2\n"
> > +" my_tramp2:\n"
> > +" addi sp,sp,-16\n"
> > +" sd t0,0(sp)\n"
> > +" sd ra,8(sp)\n"
> > +" call my_direct_func2\n"
> > +" ld t0,0(sp)\n"
> > +" ld ra,8(sp)\n"
> > +" addi sp,sp,16\n"
> > +" jr t0\n"
> > +" .size my_tramp2, .-my_tramp2\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
> > index d52370cad0b6..10884bf418f7 100644
> > --- a/samples/ftrace/ftrace-direct-multi-modify.c
> > +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> > @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> > extern void my_tramp1(void *);
> > extern void my_tramp2(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp1, @function\n"
> > +" .globl my_tramp1\n"
> > +" my_tramp1:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func1\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp1, .-my_tramp1\n"
> > +
> > +" .type my_tramp2, @function\n"
> > +" .globl my_tramp2\n"
> > +" my_tramp2:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func2\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp2, .-my_tramp2\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
> > index ec1088922517..a35bf43bf6d7 100644
> > --- a/samples/ftrace/ftrace-direct-multi.c
> > +++ b/samples/ftrace/ftrace-direct-multi.c
> > @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
> > index e13fb59a2b47..3b62e33c2e6d 100644
> > --- a/samples/ftrace/ftrace-direct-too.c
> > +++ b/samples/ftrace/ftrace-direct-too.c
> > @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-40\n"
> > +" sd a0,0(sp)\n"
> > +" sd a1,8(sp)\n"
> > +" sd a2,16(sp)\n"
> > +" sd t0,24(sp)\n"
> > +" sd ra,32(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld a1,8(sp)\n"
> > +" ld a2,16(sp)\n"
> > +" ld t0,24(sp)\n"
> > +" ld ra,32(sp)\n"
> > +" addi sp,sp,40\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
> > index 1f769d0db20f..2cfe5a7d2d70 100644
> > --- a/samples/ftrace/ftrace-direct.c
> > +++ b/samples/ftrace/ftrace-direct.c
> > @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > --
> > 2.36.1
> >
> >
>
> Regards,
> Evgenii
>
>
--
Thanks,
Song
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-17 9:32 ` Song Shuai
0 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-01-17 9:32 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi, Evgenii:
Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>
> Hi,
>
> On 12.01.2023 12:06, guoren@kernel.org wrote:
> > From: Song Shuai <suagrfillet@gmail.com>
> >
> > select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > the ftrace-direct*.c files in samples/ftrace/.
> >
> > Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > Tested-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 2 ++
> > samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> > samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
> > samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> > samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> > samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> > 6 files changed, 142 insertions(+)
>
> The samples were built OK now, but ftrace-direct-multi and
> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>
> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> the messages in the trace:
>
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> direct func1 ip 0
> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> direct func1 ip ff60000003334d00
> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> direct func1 ip 0
> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> direct func1 ip 0
> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> direct func1 ip 7fffffffffffffff
> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> direct func1 ip ff60000001ba9600
> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> direct func1 ip 0
> [...]
>
> If I understand it right, my_direct_func1() should print the address of
> some location in the code, probably - at the beginning of the traced
> functions.
>
> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> addresses.
>
The invalid code address is only printed by accessing the schedule()
function's first argument whose address stores in a0 register.
While schedule() actually has no parameter declared, so my_direct_func
just prints the a0 in the context of the schedule()'s caller and
the address maybe varies depending on the caller.
I can't really understand why tracing the first argument of the
schedule() function, but it seems nonsense at this point.
As for this patch, it just impls a simple mcount (direct_caller) to
trace kernel functions, and basically saves the necessary ABI,
call the tracing function, and restores the ABI, just like other arches do.
so It shouldn't be blamed.
I started an independent patch to replace schedule with kick_process
to make these samples more reasonable. And It has no conflict with the
current patch, so we can go on.
Link: https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> The same issue is with ftrace-direct-multi-modify.ko.
>
> Is anything missing here?
>
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 307a9f413edd..e944af44f681 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -112,6 +112,8 @@ config RISCV
> > select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> > select HAVE_REGS_AND_STACK_ACCESS_API
> > select HAVE_FUNCTION_ARG_ACCESS_API
> > + select HAVE_SAMPLE_FTRACE_DIRECT
> > + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > select HAVE_STACKPROTECTOR
> > select HAVE_SYSCALL_TRACEPOINTS
> > select HAVE_RSEQ
> > diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
> > index de5a0f67f320..be7bf472c3c7 100644
> > --- a/samples/ftrace/ftrace-direct-modify.c
> > +++ b/samples/ftrace/ftrace-direct-modify.c
> > @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >
> > static unsigned long my_ip = (unsigned long)schedule;
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp1, @function\n"
> > +" .globl my_tramp1\n"
> > +" my_tramp1:\n"
> > +" addi sp,sp,-16\n"
> > +" sd t0,0(sp)\n"
> > +" sd ra,8(sp)\n"
> > +" call my_direct_func1\n"
> > +" ld t0,0(sp)\n"
> > +" ld ra,8(sp)\n"
> > +" addi sp,sp,16\n"
> > +" jr t0\n"
> > +" .size my_tramp1, .-my_tramp1\n"
> > +
> > +" .type my_tramp2, @function\n"
> > +" .globl my_tramp2\n"
> > +" my_tramp2:\n"
> > +" addi sp,sp,-16\n"
> > +" sd t0,0(sp)\n"
> > +" sd ra,8(sp)\n"
> > +" call my_direct_func2\n"
> > +" ld t0,0(sp)\n"
> > +" ld ra,8(sp)\n"
> > +" addi sp,sp,16\n"
> > +" jr t0\n"
> > +" .size my_tramp2, .-my_tramp2\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
> > index d52370cad0b6..10884bf418f7 100644
> > --- a/samples/ftrace/ftrace-direct-multi-modify.c
> > +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> > @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> > extern void my_tramp1(void *);
> > extern void my_tramp2(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp1, @function\n"
> > +" .globl my_tramp1\n"
> > +" my_tramp1:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func1\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp1, .-my_tramp1\n"
> > +
> > +" .type my_tramp2, @function\n"
> > +" .globl my_tramp2\n"
> > +" my_tramp2:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func2\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp2, .-my_tramp2\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
> > index ec1088922517..a35bf43bf6d7 100644
> > --- a/samples/ftrace/ftrace-direct-multi.c
> > +++ b/samples/ftrace/ftrace-direct-multi.c
> > @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
> > index e13fb59a2b47..3b62e33c2e6d 100644
> > --- a/samples/ftrace/ftrace-direct-too.c
> > +++ b/samples/ftrace/ftrace-direct-too.c
> > @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-40\n"
> > +" sd a0,0(sp)\n"
> > +" sd a1,8(sp)\n"
> > +" sd a2,16(sp)\n"
> > +" sd t0,24(sp)\n"
> > +" sd ra,32(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld a1,8(sp)\n"
> > +" ld a2,16(sp)\n"
> > +" ld t0,24(sp)\n"
> > +" ld ra,32(sp)\n"
> > +" addi sp,sp,40\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
> > index 1f769d0db20f..2cfe5a7d2d70 100644
> > --- a/samples/ftrace/ftrace-direct.c
> > +++ b/samples/ftrace/ftrace-direct.c
> > @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >
> > extern void my_tramp(void *);
> >
> > +#ifdef CONFIG_RISCV
> > +
> > +asm (" .pushsection .text, \"ax\", @progbits\n"
> > +" .type my_tramp, @function\n"
> > +" .globl my_tramp\n"
> > +" my_tramp:\n"
> > +" addi sp,sp,-24\n"
> > +" sd a0,0(sp)\n"
> > +" sd t0,8(sp)\n"
> > +" sd ra,16(sp)\n"
> > +" call my_direct_func\n"
> > +" ld a0,0(sp)\n"
> > +" ld t0,8(sp)\n"
> > +" ld ra,16(sp)\n"
> > +" addi sp,sp,24\n"
> > +" jr t0\n"
> > +" .size my_tramp, .-my_tramp\n"
> > +" .popsection\n"
> > +);
> > +
> > +#endif /* CONFIG_RISCV */
> > +
> > #ifdef CONFIG_X86_64
> >
> > #include <asm/ibt.h>
> > --
> > 2.36.1
> >
> >
>
> Regards,
> Evgenii
>
>
--
Thanks,
Song
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-17 9:32 ` Song Shuai
@ 2023-01-17 13:16 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-17 13:16 UTC (permalink / raw)
To: Song Shuai
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi, Song,
On 17.01.2023 12:32, Song Shuai wrote:
>
> Hi, Evgenii:
>
> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>
>>
>> Hi,
>>
>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>> From: Song Shuai <suagrfillet@gmail.com>
>>>
>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>> the ftrace-direct*.c files in samples/ftrace/.
>>>
>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>> Tested-by: Guo Ren <guoren@kernel.org>
>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>> ---
>>> arch/riscv/Kconfig | 2 ++
>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>> samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>> 6 files changed, 142 insertions(+)
>>
>> The samples were built OK now, but ftrace-direct-multi and
>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>
>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>> the messages in the trace:
>>
>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> # | | | ||||| | |
>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>> direct func1 ip 0
>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>> direct func1 ip ff60000003334d00
>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>> direct func1 ip 0
>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>> direct func1 ip 0
>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>> direct func1 ip 7fffffffffffffff
>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>> direct func1 ip 0
>> [...]
>>
>> If I understand it right, my_direct_func1() should print the address of
>> some location in the code, probably - at the beginning of the traced
>> functions.
>>
>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>> addresses.
>>
> The invalid code address is only printed by accessing the schedule()
> function's first argument whose address stores in a0 register.
> While schedule() actually has no parameter declared, so my_direct_func
> just prints the a0 in the context of the schedule()'s caller and
> the address maybe varies depending on the caller.
>
> I can't really understand why tracing the first argument of the
> schedule() function, but it seems nonsense at this point.
The question is, what should be passed as the argument(s) of
my_direct_func() in this particular sample module. The kernel docs and
commit logs seem to contain no info on that.
With direct functions, I suppose, the trampoline can pass anything it
wants to my_direct_func(), not just the arguments of the traced function.
I'd check what these sample modules do on x86 and would try to match
that behaviour on RISC-V.
>
> As for this patch, it just impls a simple mcount (direct_caller) to
> trace kernel functions, and basically saves the necessary ABI,
> call the tracing function, and restores the ABI, just like other arches do.
> so It shouldn't be blamed.
>
> I started an independent patch to replace schedule with kick_process
> to make these samples more reasonable. And It has no conflict with the
> current patch, so we can go on.
>
> Link: https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>
>> The same issue is with ftrace-direct-multi-modify.ko.
>>
>> Is anything missing here?
>>
>>>
>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>> index 307a9f413edd..e944af44f681 100644
>>> --- a/arch/riscv/Kconfig
>>> +++ b/arch/riscv/Kconfig
>>> @@ -112,6 +112,8 @@ config RISCV
>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>> select HAVE_STACKPROTECTOR
>>> select HAVE_SYSCALL_TRACEPOINTS
>>> select HAVE_RSEQ
>>> diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
>>> index de5a0f67f320..be7bf472c3c7 100644
>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>
>>> static unsigned long my_ip = (unsigned long)schedule;
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp1, @function\n"
>>> +" .globl my_tramp1\n"
>>> +" my_tramp1:\n"
>>> +" addi sp,sp,-16\n"
>>> +" sd t0,0(sp)\n"
>>> +" sd ra,8(sp)\n"
>>> +" call my_direct_func1\n"
>>> +" ld t0,0(sp)\n"
>>> +" ld ra,8(sp)\n"
>>> +" addi sp,sp,16\n"
>>> +" jr t0\n"
>>> +" .size my_tramp1, .-my_tramp1\n"
>>> +
>>> +" .type my_tramp2, @function\n"
>>> +" .globl my_tramp2\n"
>>> +" my_tramp2:\n"
>>> +" addi sp,sp,-16\n"
>>> +" sd t0,0(sp)\n"
>>> +" sd ra,8(sp)\n"
>>> +" call my_direct_func2\n"
>>> +" ld t0,0(sp)\n"
>>> +" ld ra,8(sp)\n"
>>> +" addi sp,sp,16\n"
>>> +" jr t0\n"
>>> +" .size my_tramp2, .-my_tramp2\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
>>> index d52370cad0b6..10884bf418f7 100644
>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>> extern void my_tramp1(void *);
>>> extern void my_tramp2(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp1, @function\n"
>>> +" .globl my_tramp1\n"
>>> +" my_tramp1:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func1\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp1, .-my_tramp1\n"
>>> +
>>> +" .type my_tramp2, @function\n"
>>> +" .globl my_tramp2\n"
>>> +" my_tramp2:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func2\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp2, .-my_tramp2\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
>>> index ec1088922517..a35bf43bf6d7 100644
>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>> --- a/samples/ftrace/ftrace-direct-too.c
>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-40\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd a1,8(sp)\n"
>>> +" sd a2,16(sp)\n"
>>> +" sd t0,24(sp)\n"
>>> +" sd ra,32(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld a1,8(sp)\n"
>>> +" ld a2,16(sp)\n"
>>> +" ld t0,24(sp)\n"
>>> +" ld ra,32(sp)\n"
>>> +" addi sp,sp,40\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>> --- a/samples/ftrace/ftrace-direct.c
>>> +++ b/samples/ftrace/ftrace-direct.c
>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> --
>>> 2.36.1
>
> --
> Thanks,
> Song
>
Regards,
Evgenii
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-17 13:16 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-17 13:16 UTC (permalink / raw)
To: Song Shuai
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Hi, Song,
On 17.01.2023 12:32, Song Shuai wrote:
>
> Hi, Evgenii:
>
> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>
>>
>> Hi,
>>
>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>> From: Song Shuai <suagrfillet@gmail.com>
>>>
>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>> the ftrace-direct*.c files in samples/ftrace/.
>>>
>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>> Tested-by: Guo Ren <guoren@kernel.org>
>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>> ---
>>> arch/riscv/Kconfig | 2 ++
>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>> samples/ftrace/ftrace-direct-multi-modify.c | 37 +++++++++++++++++++++
>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>> 6 files changed, 142 insertions(+)
>>
>> The samples were built OK now, but ftrace-direct-multi and
>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>
>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>> the messages in the trace:
>>
>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> # | | | ||||| | |
>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>> direct func1 ip 0
>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>> direct func1 ip ff60000003334d00
>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>> direct func1 ip 0
>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>> direct func1 ip 0
>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>> direct func1 ip 7fffffffffffffff
>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>> direct func1 ip ff60000001ba9600
>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>> direct func1 ip 0
>> [...]
>>
>> If I understand it right, my_direct_func1() should print the address of
>> some location in the code, probably - at the beginning of the traced
>> functions.
>>
>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>> addresses.
>>
> The invalid code address is only printed by accessing the schedule()
> function's first argument whose address stores in a0 register.
> While schedule() actually has no parameter declared, so my_direct_func
> just prints the a0 in the context of the schedule()'s caller and
> the address maybe varies depending on the caller.
>
> I can't really understand why tracing the first argument of the
> schedule() function, but it seems nonsense at this point.
The question is, what should be passed as the argument(s) of
my_direct_func() in this particular sample module. The kernel docs and
commit logs seem to contain no info on that.
With direct functions, I suppose, the trampoline can pass anything it
wants to my_direct_func(), not just the arguments of the traced function.
I'd check what these sample modules do on x86 and would try to match
that behaviour on RISC-V.
>
> As for this patch, it just impls a simple mcount (direct_caller) to
> trace kernel functions, and basically saves the necessary ABI,
> call the tracing function, and restores the ABI, just like other arches do.
> so It shouldn't be blamed.
>
> I started an independent patch to replace schedule with kick_process
> to make these samples more reasonable. And It has no conflict with the
> current patch, so we can go on.
>
> Link: https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>
>> The same issue is with ftrace-direct-multi-modify.ko.
>>
>> Is anything missing here?
>>
>>>
>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>> index 307a9f413edd..e944af44f681 100644
>>> --- a/arch/riscv/Kconfig
>>> +++ b/arch/riscv/Kconfig
>>> @@ -112,6 +112,8 @@ config RISCV
>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>> select HAVE_STACKPROTECTOR
>>> select HAVE_SYSCALL_TRACEPOINTS
>>> select HAVE_RSEQ
>>> diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
>>> index de5a0f67f320..be7bf472c3c7 100644
>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>
>>> static unsigned long my_ip = (unsigned long)schedule;
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp1, @function\n"
>>> +" .globl my_tramp1\n"
>>> +" my_tramp1:\n"
>>> +" addi sp,sp,-16\n"
>>> +" sd t0,0(sp)\n"
>>> +" sd ra,8(sp)\n"
>>> +" call my_direct_func1\n"
>>> +" ld t0,0(sp)\n"
>>> +" ld ra,8(sp)\n"
>>> +" addi sp,sp,16\n"
>>> +" jr t0\n"
>>> +" .size my_tramp1, .-my_tramp1\n"
>>> +
>>> +" .type my_tramp2, @function\n"
>>> +" .globl my_tramp2\n"
>>> +" my_tramp2:\n"
>>> +" addi sp,sp,-16\n"
>>> +" sd t0,0(sp)\n"
>>> +" sd ra,8(sp)\n"
>>> +" call my_direct_func2\n"
>>> +" ld t0,0(sp)\n"
>>> +" ld ra,8(sp)\n"
>>> +" addi sp,sp,16\n"
>>> +" jr t0\n"
>>> +" .size my_tramp2, .-my_tramp2\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
>>> index d52370cad0b6..10884bf418f7 100644
>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>> extern void my_tramp1(void *);
>>> extern void my_tramp2(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp1, @function\n"
>>> +" .globl my_tramp1\n"
>>> +" my_tramp1:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func1\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp1, .-my_tramp1\n"
>>> +
>>> +" .type my_tramp2, @function\n"
>>> +" .globl my_tramp2\n"
>>> +" my_tramp2:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func2\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp2, .-my_tramp2\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
>>> index ec1088922517..a35bf43bf6d7 100644
>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>> --- a/samples/ftrace/ftrace-direct-too.c
>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-40\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd a1,8(sp)\n"
>>> +" sd a2,16(sp)\n"
>>> +" sd t0,24(sp)\n"
>>> +" sd ra,32(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld a1,8(sp)\n"
>>> +" ld a2,16(sp)\n"
>>> +" ld t0,24(sp)\n"
>>> +" ld ra,32(sp)\n"
>>> +" addi sp,sp,40\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>> --- a/samples/ftrace/ftrace-direct.c
>>> +++ b/samples/ftrace/ftrace-direct.c
>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>
>>> extern void my_tramp(void *);
>>>
>>> +#ifdef CONFIG_RISCV
>>> +
>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>> +" .type my_tramp, @function\n"
>>> +" .globl my_tramp\n"
>>> +" my_tramp:\n"
>>> +" addi sp,sp,-24\n"
>>> +" sd a0,0(sp)\n"
>>> +" sd t0,8(sp)\n"
>>> +" sd ra,16(sp)\n"
>>> +" call my_direct_func\n"
>>> +" ld a0,0(sp)\n"
>>> +" ld t0,8(sp)\n"
>>> +" ld ra,16(sp)\n"
>>> +" addi sp,sp,24\n"
>>> +" jr t0\n"
>>> +" .size my_tramp, .-my_tramp\n"
>>> +" .popsection\n"
>>> +);
>>> +
>>> +#endif /* CONFIG_RISCV */
>>> +
>>> #ifdef CONFIG_X86_64
>>>
>>> #include <asm/ibt.h>
>>> --
>>> 2.36.1
>
> --
> Thanks,
> Song
>
Regards,
Evgenii
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-17 13:16 ` Evgenii Shatokhin
@ 2023-01-17 16:22 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-17 16:22 UTC (permalink / raw)
To: Song Shuai
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> Hi, Song,
>
> On 17.01.2023 12:32, Song Shuai wrote:
>>
>> Hi, Evgenii:
>>
>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>>
>>>
>>> Hi,
>>>
>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>>> From: Song Shuai <suagrfillet@gmail.com>
>>>>
>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>>> the ftrace-direct*.c files in samples/ftrace/.
>>>>
>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>>> Tested-by: Guo Ren <guoren@kernel.org>
>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>> ---
>>>> arch/riscv/Kconfig | 2 ++
>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>>>> +++++++++++++++++++++
>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>>> 6 files changed, 142 insertions(+)
>>>
>>> The samples were built OK now, but ftrace-direct-multi and
>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>>
>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>>> the messages in the trace:
>>>
>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>>> # | | | ||||| | |
>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>>> direct func1 ip 0
>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>>> direct func1 ip ff60000003334d00
>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>>> direct func1 ip 0
>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>>> direct func1 ip 0
>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>>> direct func1 ip 7fffffffffffffff
>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>>> direct func1 ip 0
>>> [...]
>>>
>>> If I understand it right, my_direct_func1() should print the address of
>>> some location in the code, probably - at the beginning of the traced
>>> functions.
>>>
>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>>> addresses.
>>>
>> The invalid code address is only printed by accessing the schedule()
>> function's first argument whose address stores in a0 register.
>> While schedule() actually has no parameter declared, so my_direct_func
>> just prints the a0 in the context of the schedule()'s caller and
>> the address maybe varies depending on the caller.
>>
>> I can't really understand why tracing the first argument of the
>> schedule() function, but it seems nonsense at this point.
>
> The question is, what should be passed as the argument(s) of
> my_direct_func() in this particular sample module. The kernel docs and
> commit logs seem to contain no info on that.
>
> With direct functions, I suppose, the trampoline can pass anything it
> wants to my_direct_func(), not just the arguments of the traced function.
>
> I'd check what these sample modules do on x86 and would try to match
> that behaviour on RISC-V.
I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
be passed to my_direct_func().
ftrace-direct-multi.ko:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
ffffffff87332f45 // wake_up_process+0x5
rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
ffffffff8828d935 // schedule+0x5
insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
ffffffff8828d935
systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
ffffffff8828d935
systemd-1 [000] ..... 1719.519141: my_direct_func: ip
ffffffff8828d935
<idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
ffffffff87332f45
<idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
ffffffff87332f45
rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
ffffffff8828d935
[...]
The ip values are wake_up_process+0x5 and schedule+0x5, the locations
where the execution of the traced functions resumes after the Ftrace
trampoline has finished.
The results with ftrace-direct-multi-modify.ko are similar to that.
The samples look like a demonstration, that one can pass anything
necessary to the handler in case of "direct" functions.
I suppose, the RISC-V-specific asm code in these two sample modules
could be updated to pass the saved pc value to my_direct_func() in a0.
>
>>
>> As for this patch, it just impls a simple mcount (direct_caller) to
>> trace kernel functions, and basically saves the necessary ABI,
>> call the tracing function, and restores the ABI, just like other
>> arches do.
>> so It shouldn't be blamed.
>>
>> I started an independent patch to replace schedule with kick_process
>> to make these samples more reasonable. And It has no conflict with the
>> current patch, so we can go on.
>>
>> Link:
>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>>
>>> The same issue is with ftrace-direct-multi-modify.ko.
>>>
>>> Is anything missing here?
>>>
>>>>
>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>> index 307a9f413edd..e944af44f681 100644
>>>> --- a/arch/riscv/Kconfig
>>>> +++ b/arch/riscv/Kconfig
>>>> @@ -112,6 +112,8 @@ config RISCV
>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>> select HAVE_STACKPROTECTOR
>>>> select HAVE_SYSCALL_TRACEPOINTS
>>>> select HAVE_RSEQ
>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>>>> b/samples/ftrace/ftrace-direct-modify.c
>>>> index de5a0f67f320..be7bf472c3c7 100644
>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>>
>>>> static unsigned long my_ip = (unsigned long)schedule;
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp1, @function\n"
>>>> +" .globl my_tramp1\n"
>>>> +" my_tramp1:\n"
>>>> +" addi sp,sp,-16\n"
>>>> +" sd t0,0(sp)\n"
>>>> +" sd ra,8(sp)\n"
>>>> +" call my_direct_func1\n"
>>>> +" ld t0,0(sp)\n"
>>>> +" ld ra,8(sp)\n"
>>>> +" addi sp,sp,16\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>> +
>>>> +" .type my_tramp2, @function\n"
>>>> +" .globl my_tramp2\n"
>>>> +" my_tramp2:\n"
>>>> +" addi sp,sp,-16\n"
>>>> +" sd t0,0(sp)\n"
>>>> +" sd ra,8(sp)\n"
>>>> +" call my_direct_func2\n"
>>>> +" ld t0,0(sp)\n"
>>>> +" ld ra,8(sp)\n"
>>>> +" addi sp,sp,16\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>>>> index d52370cad0b6..10884bf418f7 100644
>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>>> extern void my_tramp1(void *);
>>>> extern void my_tramp2(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp1, @function\n"
>>>> +" .globl my_tramp1\n"
>>>> +" my_tramp1:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func1\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>> +
>>>> +" .type my_tramp2, @function\n"
>>>> +" .globl my_tramp2\n"
>>>> +" my_tramp2:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func2\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>>>> b/samples/ftrace/ftrace-direct-multi.c
>>>> index ec1088922517..a35bf43bf6d7 100644
>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>>>> b/samples/ftrace/ftrace-direct-too.c
>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>>> --- a/samples/ftrace/ftrace-direct-too.c
>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-40\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd a1,8(sp)\n"
>>>> +" sd a2,16(sp)\n"
>>>> +" sd t0,24(sp)\n"
>>>> +" sd ra,32(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld a1,8(sp)\n"
>>>> +" ld a2,16(sp)\n"
>>>> +" ld t0,24(sp)\n"
>>>> +" ld ra,32(sp)\n"
>>>> +" addi sp,sp,40\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct.c
>>>> b/samples/ftrace/ftrace-direct.c
>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>>> --- a/samples/ftrace/ftrace-direct.c
>>>> +++ b/samples/ftrace/ftrace-direct.c
>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> --
>>>> 2.36.1
>>
>> --
>> Thanks,
>> Song
>>
>
> Regards,
> Evgenii
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-17 16:22 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-17 16:22 UTC (permalink / raw)
To: Song Shuai
Cc: guoren, linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> Hi, Song,
>
> On 17.01.2023 12:32, Song Shuai wrote:
>>
>> Hi, Evgenii:
>>
>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>>
>>>
>>> Hi,
>>>
>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>>> From: Song Shuai <suagrfillet@gmail.com>
>>>>
>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>>> the ftrace-direct*.c files in samples/ftrace/.
>>>>
>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>>> Tested-by: Guo Ren <guoren@kernel.org>
>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>> ---
>>>> arch/riscv/Kconfig | 2 ++
>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>>>> +++++++++++++++++++++
>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>>> 6 files changed, 142 insertions(+)
>>>
>>> The samples were built OK now, but ftrace-direct-multi and
>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>>
>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>>> the messages in the trace:
>>>
>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>>> # | | | ||||| | |
>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>>> direct func1 ip 0
>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>>> direct func1 ip ff60000003334d00
>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>>> direct func1 ip 0
>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>>> direct func1 ip 0
>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>>> direct func1 ip 7fffffffffffffff
>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>>> direct func1 ip ff60000001ba9600
>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>>> direct func1 ip 0
>>> [...]
>>>
>>> If I understand it right, my_direct_func1() should print the address of
>>> some location in the code, probably - at the beginning of the traced
>>> functions.
>>>
>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>>> addresses.
>>>
>> The invalid code address is only printed by accessing the schedule()
>> function's first argument whose address stores in a0 register.
>> While schedule() actually has no parameter declared, so my_direct_func
>> just prints the a0 in the context of the schedule()'s caller and
>> the address maybe varies depending on the caller.
>>
>> I can't really understand why tracing the first argument of the
>> schedule() function, but it seems nonsense at this point.
>
> The question is, what should be passed as the argument(s) of
> my_direct_func() in this particular sample module. The kernel docs and
> commit logs seem to contain no info on that.
>
> With direct functions, I suppose, the trampoline can pass anything it
> wants to my_direct_func(), not just the arguments of the traced function.
>
> I'd check what these sample modules do on x86 and would try to match
> that behaviour on RISC-V.
I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
be passed to my_direct_func().
ftrace-direct-multi.ko:
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
ffffffff87332f45 // wake_up_process+0x5
rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
ffffffff8828d935 // schedule+0x5
insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
ffffffff8828d935
systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
ffffffff8828d935
systemd-1 [000] ..... 1719.519141: my_direct_func: ip
ffffffff8828d935
<idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
ffffffff87332f45
<idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
ffffffff87332f45
rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
ffffffff8828d935
[...]
The ip values are wake_up_process+0x5 and schedule+0x5, the locations
where the execution of the traced functions resumes after the Ftrace
trampoline has finished.
The results with ftrace-direct-multi-modify.ko are similar to that.
The samples look like a demonstration, that one can pass anything
necessary to the handler in case of "direct" functions.
I suppose, the RISC-V-specific asm code in these two sample modules
could be updated to pass the saved pc value to my_direct_func() in a0.
>
>>
>> As for this patch, it just impls a simple mcount (direct_caller) to
>> trace kernel functions, and basically saves the necessary ABI,
>> call the tracing function, and restores the ABI, just like other
>> arches do.
>> so It shouldn't be blamed.
>>
>> I started an independent patch to replace schedule with kick_process
>> to make these samples more reasonable. And It has no conflict with the
>> current patch, so we can go on.
>>
>> Link:
>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>>
>>> The same issue is with ftrace-direct-multi-modify.ko.
>>>
>>> Is anything missing here?
>>>
>>>>
>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>> index 307a9f413edd..e944af44f681 100644
>>>> --- a/arch/riscv/Kconfig
>>>> +++ b/arch/riscv/Kconfig
>>>> @@ -112,6 +112,8 @@ config RISCV
>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>> select HAVE_STACKPROTECTOR
>>>> select HAVE_SYSCALL_TRACEPOINTS
>>>> select HAVE_RSEQ
>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>>>> b/samples/ftrace/ftrace-direct-modify.c
>>>> index de5a0f67f320..be7bf472c3c7 100644
>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>>
>>>> static unsigned long my_ip = (unsigned long)schedule;
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp1, @function\n"
>>>> +" .globl my_tramp1\n"
>>>> +" my_tramp1:\n"
>>>> +" addi sp,sp,-16\n"
>>>> +" sd t0,0(sp)\n"
>>>> +" sd ra,8(sp)\n"
>>>> +" call my_direct_func1\n"
>>>> +" ld t0,0(sp)\n"
>>>> +" ld ra,8(sp)\n"
>>>> +" addi sp,sp,16\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>> +
>>>> +" .type my_tramp2, @function\n"
>>>> +" .globl my_tramp2\n"
>>>> +" my_tramp2:\n"
>>>> +" addi sp,sp,-16\n"
>>>> +" sd t0,0(sp)\n"
>>>> +" sd ra,8(sp)\n"
>>>> +" call my_direct_func2\n"
>>>> +" ld t0,0(sp)\n"
>>>> +" ld ra,8(sp)\n"
>>>> +" addi sp,sp,16\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>>>> index d52370cad0b6..10884bf418f7 100644
>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>>> extern void my_tramp1(void *);
>>>> extern void my_tramp2(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp1, @function\n"
>>>> +" .globl my_tramp1\n"
>>>> +" my_tramp1:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func1\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>> +
>>>> +" .type my_tramp2, @function\n"
>>>> +" .globl my_tramp2\n"
>>>> +" my_tramp2:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func2\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>>>> b/samples/ftrace/ftrace-direct-multi.c
>>>> index ec1088922517..a35bf43bf6d7 100644
>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>>>> b/samples/ftrace/ftrace-direct-too.c
>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>>> --- a/samples/ftrace/ftrace-direct-too.c
>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-40\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd a1,8(sp)\n"
>>>> +" sd a2,16(sp)\n"
>>>> +" sd t0,24(sp)\n"
>>>> +" sd ra,32(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld a1,8(sp)\n"
>>>> +" ld a2,16(sp)\n"
>>>> +" ld t0,24(sp)\n"
>>>> +" ld ra,32(sp)\n"
>>>> +" addi sp,sp,40\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> diff --git a/samples/ftrace/ftrace-direct.c
>>>> b/samples/ftrace/ftrace-direct.c
>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>>> --- a/samples/ftrace/ftrace-direct.c
>>>> +++ b/samples/ftrace/ftrace-direct.c
>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>>
>>>> extern void my_tramp(void *);
>>>>
>>>> +#ifdef CONFIG_RISCV
>>>> +
>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>> +" .type my_tramp, @function\n"
>>>> +" .globl my_tramp\n"
>>>> +" my_tramp:\n"
>>>> +" addi sp,sp,-24\n"
>>>> +" sd a0,0(sp)\n"
>>>> +" sd t0,8(sp)\n"
>>>> +" sd ra,16(sp)\n"
>>>> +" call my_direct_func\n"
>>>> +" ld a0,0(sp)\n"
>>>> +" ld t0,8(sp)\n"
>>>> +" ld ra,16(sp)\n"
>>>> +" addi sp,sp,24\n"
>>>> +" jr t0\n"
>>>> +" .size my_tramp, .-my_tramp\n"
>>>> +" .popsection\n"
>>>> +);
>>>> +
>>>> +#endif /* CONFIG_RISCV */
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>>
>>>> #include <asm/ibt.h>
>>>> --
>>>> 2.36.1
>>
>> --
>> Thanks,
>> Song
>>
>
> Regards,
> Evgenii
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-17 16:22 ` Evgenii Shatokhin
@ 2023-01-18 2:37 ` Song Shuai
-1 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-01-18 2:37 UTC (permalink / raw)
To: Evgenii Shatokhin, guoren
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>
> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> > Hi, Song,
> >
> > On 17.01.2023 12:32, Song Shuai wrote:
> >>
> >> Hi, Evgenii:
> >>
> >> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >>
> >>>
> >>> Hi,
> >>>
> >>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >>>> From: Song Shuai <suagrfillet@gmail.com>
> >>>>
> >>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >>>> the ftrace-direct*.c files in samples/ftrace/.
> >>>>
> >>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >>>> Tested-by: Guo Ren <guoren@kernel.org>
> >>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>> ---
> >>>> arch/riscv/Kconfig | 2 ++
> >>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >>>> +++++++++++++++++++++
> >>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >>>> 6 files changed, 142 insertions(+)
> >>>
> >>> The samples were built OK now, but ftrace-direct-multi and
> >>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >>>
> >>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >>> the messages in the trace:
> >>>
> >>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >>> # | | | ||||| | |
> >>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >>> direct func1 ip 0
> >>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >>> direct func1 ip ff60000003334d00
> >>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >>> direct func1 ip 0
> >>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >>> direct func1 ip 0
> >>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >>> direct func1 ip 7fffffffffffffff
> >>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >>> direct func1 ip 0
> >>> [...]
> >>>
> >>> If I understand it right, my_direct_func1() should print the address of
> >>> some location in the code, probably - at the beginning of the traced
> >>> functions.
> >>>
> >>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >>> addresses.
> >>>
> >> The invalid code address is only printed by accessing the schedule()
> >> function's first argument whose address stores in a0 register.
> >> While schedule() actually has no parameter declared, so my_direct_func
> >> just prints the a0 in the context of the schedule()'s caller and
> >> the address maybe varies depending on the caller.
> >>
> >> I can't really understand why tracing the first argument of the
> >> schedule() function, but it seems nonsense at this point.
> >
> > The question is, what should be passed as the argument(s) of
> > my_direct_func() in this particular sample module. The kernel docs and
> > commit logs seem to contain no info on that.
> >
> > With direct functions, I suppose, the trampoline can pass anything it
> > wants to my_direct_func(), not just the arguments of the traced function.
> >
> > I'd check what these sample modules do on x86 and would try to match
> > that behaviour on RISC-V.
>
> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> be passed to my_direct_func().
>
> ftrace-direct-multi.ko:
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> ffffffff87332f45 // wake_up_process+0x5
> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> ffffffff8828d935 // schedule+0x5
> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> ffffffff8828d935
> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> ffffffff8828d935
> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> ffffffff8828d935
> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> ffffffff87332f45
> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> ffffffff87332f45
> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> ffffffff8828d935
> [...]
>
> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> where the execution of the traced functions resumes after the Ftrace
> trampoline has finished.
>
> The results with ftrace-direct-multi-modify.ko are similar to that.
>
> The samples look like a demonstration, that one can pass anything
> necessary to the handler in case of "direct" functions.
>
> I suppose, the RISC-V-specific asm code in these two sample modules
> could be updated to pass the saved pc value to my_direct_func() in a0.
Yes, you're right.
I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
traced function with mcount offset.
Here is the updated patch for your reference.
https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> >
> >>
> >> As for this patch, it just impls a simple mcount (direct_caller) to
> >> trace kernel functions, and basically saves the necessary ABI,
> >> call the tracing function, and restores the ABI, just like other
> >> arches do.
> >> so It shouldn't be blamed.
> >>
> >> I started an independent patch to replace schedule with kick_process
> >> to make these samples more reasonable. And It has no conflict with the
> >> current patch, so we can go on.
> >>
> >> Link:
> >> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >>
> >>> The same issue is with ftrace-direct-multi-modify.ko.
> >>>
> >>> Is anything missing here?
> >>>
> >>>>
> >>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>>> index 307a9f413edd..e944af44f681 100644
> >>>> --- a/arch/riscv/Kconfig
> >>>> +++ b/arch/riscv/Kconfig
> >>>> @@ -112,6 +112,8 @@ config RISCV
> >>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>> select HAVE_STACKPROTECTOR
> >>>> select HAVE_SYSCALL_TRACEPOINTS
> >>>> select HAVE_RSEQ
> >>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >>>> b/samples/ftrace/ftrace-direct-modify.c
> >>>> index de5a0f67f320..be7bf472c3c7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >>>>
> >>>> static unsigned long my_ip = (unsigned long)schedule;
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp1, @function\n"
> >>>> +" .globl my_tramp1\n"
> >>>> +" my_tramp1:\n"
> >>>> +" addi sp,sp,-16\n"
> >>>> +" sd t0,0(sp)\n"
> >>>> +" sd ra,8(sp)\n"
> >>>> +" call my_direct_func1\n"
> >>>> +" ld t0,0(sp)\n"
> >>>> +" ld ra,8(sp)\n"
> >>>> +" addi sp,sp,16\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>> +
> >>>> +" .type my_tramp2, @function\n"
> >>>> +" .globl my_tramp2\n"
> >>>> +" my_tramp2:\n"
> >>>> +" addi sp,sp,-16\n"
> >>>> +" sd t0,0(sp)\n"
> >>>> +" sd ra,8(sp)\n"
> >>>> +" call my_direct_func2\n"
> >>>> +" ld t0,0(sp)\n"
> >>>> +" ld ra,8(sp)\n"
> >>>> +" addi sp,sp,16\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> index d52370cad0b6..10884bf418f7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >>>> extern void my_tramp1(void *);
> >>>> extern void my_tramp2(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp1, @function\n"
> >>>> +" .globl my_tramp1\n"
> >>>> +" my_tramp1:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func1\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>> +
> >>>> +" .type my_tramp2, @function\n"
> >>>> +" .globl my_tramp2\n"
> >>>> +" my_tramp2:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func2\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >>>> b/samples/ftrace/ftrace-direct-multi.c
> >>>> index ec1088922517..a35bf43bf6d7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >>>> b/samples/ftrace/ftrace-direct-too.c
> >>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >>>> --- a/samples/ftrace/ftrace-direct-too.c
> >>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-40\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd a1,8(sp)\n"
> >>>> +" sd a2,16(sp)\n"
> >>>> +" sd t0,24(sp)\n"
> >>>> +" sd ra,32(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld a1,8(sp)\n"
> >>>> +" ld a2,16(sp)\n"
> >>>> +" ld t0,24(sp)\n"
> >>>> +" ld ra,32(sp)\n"
> >>>> +" addi sp,sp,40\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct.c
> >>>> b/samples/ftrace/ftrace-direct.c
> >>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >>>> --- a/samples/ftrace/ftrace-direct.c
> >>>> +++ b/samples/ftrace/ftrace-direct.c
> >>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> --
> >>>> 2.36.1
> >>
> >> --
> >> Thanks,
> >> Song
> >>
> >
> > Regards,
> > Evgenii
>
>
--
Thanks,
Song
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-18 2:37 ` Song Shuai
0 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-01-18 2:37 UTC (permalink / raw)
To: Evgenii Shatokhin, guoren
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>
> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> > Hi, Song,
> >
> > On 17.01.2023 12:32, Song Shuai wrote:
> >>
> >> Hi, Evgenii:
> >>
> >> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >>
> >>>
> >>> Hi,
> >>>
> >>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >>>> From: Song Shuai <suagrfillet@gmail.com>
> >>>>
> >>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >>>> the ftrace-direct*.c files in samples/ftrace/.
> >>>>
> >>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >>>> Tested-by: Guo Ren <guoren@kernel.org>
> >>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>> ---
> >>>> arch/riscv/Kconfig | 2 ++
> >>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >>>> +++++++++++++++++++++
> >>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >>>> 6 files changed, 142 insertions(+)
> >>>
> >>> The samples were built OK now, but ftrace-direct-multi and
> >>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >>>
> >>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >>> the messages in the trace:
> >>>
> >>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >>> # | | | ||||| | |
> >>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >>> direct func1 ip 0
> >>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >>> direct func1 ip ff60000003334d00
> >>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >>> direct func1 ip 0
> >>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >>> direct func1 ip 0
> >>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >>> direct func1 ip 7fffffffffffffff
> >>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >>> direct func1 ip ff60000001ba9600
> >>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >>> direct func1 ip 0
> >>> [...]
> >>>
> >>> If I understand it right, my_direct_func1() should print the address of
> >>> some location in the code, probably - at the beginning of the traced
> >>> functions.
> >>>
> >>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >>> addresses.
> >>>
> >> The invalid code address is only printed by accessing the schedule()
> >> function's first argument whose address stores in a0 register.
> >> While schedule() actually has no parameter declared, so my_direct_func
> >> just prints the a0 in the context of the schedule()'s caller and
> >> the address maybe varies depending on the caller.
> >>
> >> I can't really understand why tracing the first argument of the
> >> schedule() function, but it seems nonsense at this point.
> >
> > The question is, what should be passed as the argument(s) of
> > my_direct_func() in this particular sample module. The kernel docs and
> > commit logs seem to contain no info on that.
> >
> > With direct functions, I suppose, the trampoline can pass anything it
> > wants to my_direct_func(), not just the arguments of the traced function.
> >
> > I'd check what these sample modules do on x86 and would try to match
> > that behaviour on RISC-V.
>
> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> be passed to my_direct_func().
>
> ftrace-direct-multi.ko:
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> ffffffff87332f45 // wake_up_process+0x5
> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> ffffffff8828d935 // schedule+0x5
> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> ffffffff8828d935
> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> ffffffff8828d935
> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> ffffffff8828d935
> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> ffffffff87332f45
> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> ffffffff87332f45
> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> ffffffff8828d935
> [...]
>
> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> where the execution of the traced functions resumes after the Ftrace
> trampoline has finished.
>
> The results with ftrace-direct-multi-modify.ko are similar to that.
>
> The samples look like a demonstration, that one can pass anything
> necessary to the handler in case of "direct" functions.
>
> I suppose, the RISC-V-specific asm code in these two sample modules
> could be updated to pass the saved pc value to my_direct_func() in a0.
Yes, you're right.
I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
traced function with mcount offset.
Here is the updated patch for your reference.
https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> >
> >>
> >> As for this patch, it just impls a simple mcount (direct_caller) to
> >> trace kernel functions, and basically saves the necessary ABI,
> >> call the tracing function, and restores the ABI, just like other
> >> arches do.
> >> so It shouldn't be blamed.
> >>
> >> I started an independent patch to replace schedule with kick_process
> >> to make these samples more reasonable. And It has no conflict with the
> >> current patch, so we can go on.
> >>
> >> Link:
> >> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >>
> >>> The same issue is with ftrace-direct-multi-modify.ko.
> >>>
> >>> Is anything missing here?
> >>>
> >>>>
> >>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>>> index 307a9f413edd..e944af44f681 100644
> >>>> --- a/arch/riscv/Kconfig
> >>>> +++ b/arch/riscv/Kconfig
> >>>> @@ -112,6 +112,8 @@ config RISCV
> >>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>> select HAVE_STACKPROTECTOR
> >>>> select HAVE_SYSCALL_TRACEPOINTS
> >>>> select HAVE_RSEQ
> >>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >>>> b/samples/ftrace/ftrace-direct-modify.c
> >>>> index de5a0f67f320..be7bf472c3c7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >>>>
> >>>> static unsigned long my_ip = (unsigned long)schedule;
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp1, @function\n"
> >>>> +" .globl my_tramp1\n"
> >>>> +" my_tramp1:\n"
> >>>> +" addi sp,sp,-16\n"
> >>>> +" sd t0,0(sp)\n"
> >>>> +" sd ra,8(sp)\n"
> >>>> +" call my_direct_func1\n"
> >>>> +" ld t0,0(sp)\n"
> >>>> +" ld ra,8(sp)\n"
> >>>> +" addi sp,sp,16\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>> +
> >>>> +" .type my_tramp2, @function\n"
> >>>> +" .globl my_tramp2\n"
> >>>> +" my_tramp2:\n"
> >>>> +" addi sp,sp,-16\n"
> >>>> +" sd t0,0(sp)\n"
> >>>> +" sd ra,8(sp)\n"
> >>>> +" call my_direct_func2\n"
> >>>> +" ld t0,0(sp)\n"
> >>>> +" ld ra,8(sp)\n"
> >>>> +" addi sp,sp,16\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> index d52370cad0b6..10884bf418f7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >>>> extern void my_tramp1(void *);
> >>>> extern void my_tramp2(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp1, @function\n"
> >>>> +" .globl my_tramp1\n"
> >>>> +" my_tramp1:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func1\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>> +
> >>>> +" .type my_tramp2, @function\n"
> >>>> +" .globl my_tramp2\n"
> >>>> +" my_tramp2:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func2\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >>>> b/samples/ftrace/ftrace-direct-multi.c
> >>>> index ec1088922517..a35bf43bf6d7 100644
> >>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >>>> b/samples/ftrace/ftrace-direct-too.c
> >>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >>>> --- a/samples/ftrace/ftrace-direct-too.c
> >>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-40\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd a1,8(sp)\n"
> >>>> +" sd a2,16(sp)\n"
> >>>> +" sd t0,24(sp)\n"
> >>>> +" sd ra,32(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld a1,8(sp)\n"
> >>>> +" ld a2,16(sp)\n"
> >>>> +" ld t0,24(sp)\n"
> >>>> +" ld ra,32(sp)\n"
> >>>> +" addi sp,sp,40\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> diff --git a/samples/ftrace/ftrace-direct.c
> >>>> b/samples/ftrace/ftrace-direct.c
> >>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >>>> --- a/samples/ftrace/ftrace-direct.c
> >>>> +++ b/samples/ftrace/ftrace-direct.c
> >>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >>>>
> >>>> extern void my_tramp(void *);
> >>>>
> >>>> +#ifdef CONFIG_RISCV
> >>>> +
> >>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>> +" .type my_tramp, @function\n"
> >>>> +" .globl my_tramp\n"
> >>>> +" my_tramp:\n"
> >>>> +" addi sp,sp,-24\n"
> >>>> +" sd a0,0(sp)\n"
> >>>> +" sd t0,8(sp)\n"
> >>>> +" sd ra,16(sp)\n"
> >>>> +" call my_direct_func\n"
> >>>> +" ld a0,0(sp)\n"
> >>>> +" ld t0,8(sp)\n"
> >>>> +" ld ra,16(sp)\n"
> >>>> +" addi sp,sp,24\n"
> >>>> +" jr t0\n"
> >>>> +" .size my_tramp, .-my_tramp\n"
> >>>> +" .popsection\n"
> >>>> +);
> >>>> +
> >>>> +#endif /* CONFIG_RISCV */
> >>>> +
> >>>> #ifdef CONFIG_X86_64
> >>>>
> >>>> #include <asm/ibt.h>
> >>>> --
> >>>> 2.36.1
> >>
> >> --
> >> Thanks,
> >> Song
> >>
> >
> > Regards,
> > Evgenii
>
>
--
Thanks,
Song
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-18 2:37 ` Song Shuai
@ 2023-01-18 15:19 ` Evgenii Shatokhin
-1 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-18 15:19 UTC (permalink / raw)
To: Song Shuai, guoren
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
On 18.01.2023 05:37, Song Shuai wrote:
> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>>
>> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
>>> Hi, Song,
>>>
>>> On 17.01.2023 12:32, Song Shuai wrote:
>>>>
>>>> Hi, Evgenii:
>>>>
>>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>>>>> From: Song Shuai <suagrfillet@gmail.com>
>>>>>>
>>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>>>>> the ftrace-direct*.c files in samples/ftrace/.
>>>>>>
>>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>>>>> Tested-by: Guo Ren <guoren@kernel.org>
>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>>>> ---
>>>>>> arch/riscv/Kconfig | 2 ++
>>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>>>>>> +++++++++++++++++++++
>>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>>>>> 6 files changed, 142 insertions(+)
>>>>>
>>>>> The samples were built OK now, but ftrace-direct-multi and
>>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>>>>
>>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>>>>> the messages in the trace:
>>>>>
>>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>>>>> # | | | ||||| | |
>>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>>>>> direct func1 ip ff60000003334d00
>>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>>>>> direct func1 ip 7fffffffffffffff
>>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> [...]
>>>>>
>>>>> If I understand it right, my_direct_func1() should print the address of
>>>>> some location in the code, probably - at the beginning of the traced
>>>>> functions.
>>>>>
>>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>>>>> addresses.
>>>>>
>>>> The invalid code address is only printed by accessing the schedule()
>>>> function's first argument whose address stores in a0 register.
>>>> While schedule() actually has no parameter declared, so my_direct_func
>>>> just prints the a0 in the context of the schedule()'s caller and
>>>> the address maybe varies depending on the caller.
>>>>
>>>> I can't really understand why tracing the first argument of the
>>>> schedule() function, but it seems nonsense at this point.
>>>
>>> The question is, what should be passed as the argument(s) of
>>> my_direct_func() in this particular sample module. The kernel docs and
>>> commit logs seem to contain no info on that.
>>>
>>> With direct functions, I suppose, the trampoline can pass anything it
>>> wants to my_direct_func(), not just the arguments of the traced function.
>>>
>>> I'd check what these sample modules do on x86 and would try to match
>>> that behaviour on RISC-V.
>>
>> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
>> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
>> be passed to my_direct_func().
>>
>> ftrace-direct-multi.ko:
>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> # | | | ||||| | |
>> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
>> ffffffff87332f45 // wake_up_process+0x5
>> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
>> ffffffff8828d935 // schedule+0x5
>> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
>> ffffffff8828d935
>> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
>> ffffffff8828d935
>> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
>> ffffffff8828d935
>> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
>> ffffffff87332f45
>> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
>> ffffffff87332f45
>> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
>> ffffffff8828d935
>> [...]
>>
>> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
>> where the execution of the traced functions resumes after the Ftrace
>> trampoline has finished.
>>
>> The results with ftrace-direct-multi-modify.ko are similar to that.
>>
>> The samples look like a demonstration, that one can pass anything
>> necessary to the handler in case of "direct" functions.
>>
>> I suppose, the RISC-V-specific asm code in these two sample modules
>> could be updated to pass the saved pc value to my_direct_func() in a0.
>
> Yes, you're right.
>
> I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> traced function with mcount offset.
>
> Here is the updated patch for your reference.
> https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
Thank you for the quick fix. This one looks good to me.
ftrace-direct-multi*.ko now report the ip values corresponding to
schedule+0x8 and wake_up_process+0x8, which is what was expected here.
One more thing: please change my "Co-developed-by:" into "Tested-by:" in
your patch, becase this is what I actually did: tested it and reported
the results. I cannot take your credit for development of this patch ;-)
Looking forward for v8 of the series.
>
>
>>
>>>
>>>>
>>>> As for this patch, it just impls a simple mcount (direct_caller) to
>>>> trace kernel functions, and basically saves the necessary ABI,
>>>> call the tracing function, and restores the ABI, just like other
>>>> arches do.
>>>> so It shouldn't be blamed.
>>>>
>>>> I started an independent patch to replace schedule with kick_process
>>>> to make these samples more reasonable. And It has no conflict with the
>>>> current patch, so we can go on.
>>>>
>>>> Link:
>>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>>>>
>>>>> The same issue is with ftrace-direct-multi-modify.ko.
>>>>>
>>>>> Is anything missing here?
>>>>>
>>>>>>
>>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>>>> index 307a9f413edd..e944af44f681 100644
>>>>>> --- a/arch/riscv/Kconfig
>>>>>> +++ b/arch/riscv/Kconfig
>>>>>> @@ -112,6 +112,8 @@ config RISCV
>>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>>>> select HAVE_STACKPROTECTOR
>>>>>> select HAVE_SYSCALL_TRACEPOINTS
>>>>>> select HAVE_RSEQ
>>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>>>>>> b/samples/ftrace/ftrace-direct-modify.c
>>>>>> index de5a0f67f320..be7bf472c3c7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>>>>
>>>>>> static unsigned long my_ip = (unsigned long)schedule;
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp1, @function\n"
>>>>>> +" .globl my_tramp1\n"
>>>>>> +" my_tramp1:\n"
>>>>>> +" addi sp,sp,-16\n"
>>>>>> +" sd t0,0(sp)\n"
>>>>>> +" sd ra,8(sp)\n"
>>>>>> +" call my_direct_func1\n"
>>>>>> +" ld t0,0(sp)\n"
>>>>>> +" ld ra,8(sp)\n"
>>>>>> +" addi sp,sp,16\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>>>> +
>>>>>> +" .type my_tramp2, @function\n"
>>>>>> +" .globl my_tramp2\n"
>>>>>> +" my_tramp2:\n"
>>>>>> +" addi sp,sp,-16\n"
>>>>>> +" sd t0,0(sp)\n"
>>>>>> +" sd ra,8(sp)\n"
>>>>>> +" call my_direct_func2\n"
>>>>>> +" ld t0,0(sp)\n"
>>>>>> +" ld ra,8(sp)\n"
>>>>>> +" addi sp,sp,16\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> index d52370cad0b6..10884bf418f7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>>>>> extern void my_tramp1(void *);
>>>>>> extern void my_tramp2(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp1, @function\n"
>>>>>> +" .globl my_tramp1\n"
>>>>>> +" my_tramp1:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func1\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>>>> +
>>>>>> +" .type my_tramp2, @function\n"
>>>>>> +" .globl my_tramp2\n"
>>>>>> +" my_tramp2:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func2\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>>>>>> b/samples/ftrace/ftrace-direct-multi.c
>>>>>> index ec1088922517..a35bf43bf6d7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>>>>>> b/samples/ftrace/ftrace-direct-too.c
>>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-too.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-40\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd a1,8(sp)\n"
>>>>>> +" sd a2,16(sp)\n"
>>>>>> +" sd t0,24(sp)\n"
>>>>>> +" sd ra,32(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld a1,8(sp)\n"
>>>>>> +" ld a2,16(sp)\n"
>>>>>> +" ld t0,24(sp)\n"
>>>>>> +" ld ra,32(sp)\n"
>>>>>> +" addi sp,sp,40\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct.c
>>>>>> b/samples/ftrace/ftrace-direct.c
>>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>>>>> --- a/samples/ftrace/ftrace-direct.c
>>>>>> +++ b/samples/ftrace/ftrace-direct.c
>>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> --
>>>>>> 2.36.1
>>>>
>>>> --
>>>> Thanks,
>>>> Song
>>>>
>>>
>>> Regards,
>>> Evgenii
>>
>>
>
>
> --
> Thanks,
> Song
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-18 15:19 ` Evgenii Shatokhin
0 siblings, 0 replies; 88+ messages in thread
From: Evgenii Shatokhin @ 2023-01-18 15:19 UTC (permalink / raw)
To: Song Shuai, guoren
Cc: linux-riscv, linux-kernel, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
andy.chiu, linux
On 18.01.2023 05:37, Song Shuai wrote:
> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>>
>> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
>>> Hi, Song,
>>>
>>> On 17.01.2023 12:32, Song Shuai wrote:
>>>>
>>>> Hi, Evgenii:
>>>>
>>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>>>>>> From: Song Shuai <suagrfillet@gmail.com>
>>>>>>
>>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>>>>>> the ftrace-direct*.c files in samples/ftrace/.
>>>>>>
>>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>>>>>> Tested-by: Guo Ren <guoren@kernel.org>
>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>>>> ---
>>>>>> arch/riscv/Kconfig | 2 ++
>>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>>>>>> +++++++++++++++++++++
>>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>>>>>> 6 files changed, 142 insertions(+)
>>>>>
>>>>> The samples were built OK now, but ftrace-direct-multi and
>>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>>>>>
>>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>>>>> the messages in the trace:
>>>>>
>>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>>>>> # | | | ||||| | |
>>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>>>>> direct func1 ip ff60000003334d00
>>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>>>>> direct func1 ip 7fffffffffffffff
>>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>>>>> direct func1 ip ff60000001ba9600
>>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>>>>> direct func1 ip 0
>>>>> [...]
>>>>>
>>>>> If I understand it right, my_direct_func1() should print the address of
>>>>> some location in the code, probably - at the beginning of the traced
>>>>> functions.
>>>>>
>>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>>>>> addresses.
>>>>>
>>>> The invalid code address is only printed by accessing the schedule()
>>>> function's first argument whose address stores in a0 register.
>>>> While schedule() actually has no parameter declared, so my_direct_func
>>>> just prints the a0 in the context of the schedule()'s caller and
>>>> the address maybe varies depending on the caller.
>>>>
>>>> I can't really understand why tracing the first argument of the
>>>> schedule() function, but it seems nonsense at this point.
>>>
>>> The question is, what should be passed as the argument(s) of
>>> my_direct_func() in this particular sample module. The kernel docs and
>>> commit logs seem to contain no info on that.
>>>
>>> With direct functions, I suppose, the trampoline can pass anything it
>>> wants to my_direct_func(), not just the arguments of the traced function.
>>>
>>> I'd check what these sample modules do on x86 and would try to match
>>> that behaviour on RISC-V.
>>
>> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
>> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
>> be passed to my_direct_func().
>>
>> ftrace-direct-multi.ko:
>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> # | | | ||||| | |
>> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
>> ffffffff87332f45 // wake_up_process+0x5
>> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
>> ffffffff8828d935 // schedule+0x5
>> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
>> ffffffff8828d935
>> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
>> ffffffff8828d935
>> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
>> ffffffff8828d935
>> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
>> ffffffff87332f45
>> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
>> ffffffff87332f45
>> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
>> ffffffff8828d935
>> [...]
>>
>> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
>> where the execution of the traced functions resumes after the Ftrace
>> trampoline has finished.
>>
>> The results with ftrace-direct-multi-modify.ko are similar to that.
>>
>> The samples look like a demonstration, that one can pass anything
>> necessary to the handler in case of "direct" functions.
>>
>> I suppose, the RISC-V-specific asm code in these two sample modules
>> could be updated to pass the saved pc value to my_direct_func() in a0.
>
> Yes, you're right.
>
> I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> traced function with mcount offset.
>
> Here is the updated patch for your reference.
> https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
Thank you for the quick fix. This one looks good to me.
ftrace-direct-multi*.ko now report the ip values corresponding to
schedule+0x8 and wake_up_process+0x8, which is what was expected here.
One more thing: please change my "Co-developed-by:" into "Tested-by:" in
your patch, becase this is what I actually did: tested it and reported
the results. I cannot take your credit for development of this patch ;-)
Looking forward for v8 of the series.
>
>
>>
>>>
>>>>
>>>> As for this patch, it just impls a simple mcount (direct_caller) to
>>>> trace kernel functions, and basically saves the necessary ABI,
>>>> call the tracing function, and restores the ABI, just like other
>>>> arches do.
>>>> so It shouldn't be blamed.
>>>>
>>>> I started an independent patch to replace schedule with kick_process
>>>> to make these samples more reasonable. And It has no conflict with the
>>>> current patch, so we can go on.
>>>>
>>>> Link:
>>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>>>>
>>>>> The same issue is with ftrace-direct-multi-modify.ko.
>>>>>
>>>>> Is anything missing here?
>>>>>
>>>>>>
>>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>>>> index 307a9f413edd..e944af44f681 100644
>>>>>> --- a/arch/riscv/Kconfig
>>>>>> +++ b/arch/riscv/Kconfig
>>>>>> @@ -112,6 +112,8 @@ config RISCV
>>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>>>>>> select HAVE_STACKPROTECTOR
>>>>>> select HAVE_SYSCALL_TRACEPOINTS
>>>>>> select HAVE_RSEQ
>>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>>>>>> b/samples/ftrace/ftrace-direct-modify.c
>>>>>> index de5a0f67f320..be7bf472c3c7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>>>>>>
>>>>>> static unsigned long my_ip = (unsigned long)schedule;
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp1, @function\n"
>>>>>> +" .globl my_tramp1\n"
>>>>>> +" my_tramp1:\n"
>>>>>> +" addi sp,sp,-16\n"
>>>>>> +" sd t0,0(sp)\n"
>>>>>> +" sd ra,8(sp)\n"
>>>>>> +" call my_direct_func1\n"
>>>>>> +" ld t0,0(sp)\n"
>>>>>> +" ld ra,8(sp)\n"
>>>>>> +" addi sp,sp,16\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>>>> +
>>>>>> +" .type my_tramp2, @function\n"
>>>>>> +" .globl my_tramp2\n"
>>>>>> +" my_tramp2:\n"
>>>>>> +" addi sp,sp,-16\n"
>>>>>> +" sd t0,0(sp)\n"
>>>>>> +" sd ra,8(sp)\n"
>>>>>> +" call my_direct_func2\n"
>>>>>> +" ld t0,0(sp)\n"
>>>>>> +" ld ra,8(sp)\n"
>>>>>> +" addi sp,sp,16\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> index d52370cad0b6..10884bf418f7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>>>>>> extern void my_tramp1(void *);
>>>>>> extern void my_tramp2(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp1, @function\n"
>>>>>> +" .globl my_tramp1\n"
>>>>>> +" my_tramp1:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func1\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp1, .-my_tramp1\n"
>>>>>> +
>>>>>> +" .type my_tramp2, @function\n"
>>>>>> +" .globl my_tramp2\n"
>>>>>> +" my_tramp2:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func2\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp2, .-my_tramp2\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>>>>>> b/samples/ftrace/ftrace-direct-multi.c
>>>>>> index ec1088922517..a35bf43bf6d7 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>>>>>> b/samples/ftrace/ftrace-direct-too.c
>>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>>>>>> --- a/samples/ftrace/ftrace-direct-too.c
>>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-40\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd a1,8(sp)\n"
>>>>>> +" sd a2,16(sp)\n"
>>>>>> +" sd t0,24(sp)\n"
>>>>>> +" sd ra,32(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld a1,8(sp)\n"
>>>>>> +" ld a2,16(sp)\n"
>>>>>> +" ld t0,24(sp)\n"
>>>>>> +" ld ra,32(sp)\n"
>>>>>> +" addi sp,sp,40\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> diff --git a/samples/ftrace/ftrace-direct.c
>>>>>> b/samples/ftrace/ftrace-direct.c
>>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>>>>>> --- a/samples/ftrace/ftrace-direct.c
>>>>>> +++ b/samples/ftrace/ftrace-direct.c
>>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>>>>>>
>>>>>> extern void my_tramp(void *);
>>>>>>
>>>>>> +#ifdef CONFIG_RISCV
>>>>>> +
>>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>>>>>> +" .type my_tramp, @function\n"
>>>>>> +" .globl my_tramp\n"
>>>>>> +" my_tramp:\n"
>>>>>> +" addi sp,sp,-24\n"
>>>>>> +" sd a0,0(sp)\n"
>>>>>> +" sd t0,8(sp)\n"
>>>>>> +" sd ra,16(sp)\n"
>>>>>> +" call my_direct_func\n"
>>>>>> +" ld a0,0(sp)\n"
>>>>>> +" ld t0,8(sp)\n"
>>>>>> +" ld ra,16(sp)\n"
>>>>>> +" addi sp,sp,24\n"
>>>>>> +" jr t0\n"
>>>>>> +" .size my_tramp, .-my_tramp\n"
>>>>>> +" .popsection\n"
>>>>>> +);
>>>>>> +
>>>>>> +#endif /* CONFIG_RISCV */
>>>>>> +
>>>>>> #ifdef CONFIG_X86_64
>>>>>>
>>>>>> #include <asm/ibt.h>
>>>>>> --
>>>>>> 2.36.1
>>>>
>>>> --
>>>> Thanks,
>>>> Song
>>>>
>>>
>>> Regards,
>>> Evgenii
>>
>>
>
>
> --
> Thanks,
> Song
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-18 15:19 ` Evgenii Shatokhin
@ 2023-01-19 6:05 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-19 6:05 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: Song Shuai, linux-riscv, linux-kernel, anup, paul.walmsley,
palmer, conor.dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
Thx Evgenii & Song,
I got it; it would be put into v8.
On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
<e.shatokhin@yadro.com> wrote:
>
> On 18.01.2023 05:37, Song Shuai wrote:
> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >>
> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >>> Hi, Song,
> >>>
> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >>>>
> >>>> Hi, Evgenii:
> >>>>
> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >>>>>>
> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >>>>>>
> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>>>> ---
> >>>>>> arch/riscv/Kconfig | 2 ++
> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >>>>>> +++++++++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >>>>>> 6 files changed, 142 insertions(+)
> >>>>>
> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >>>>>
> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >>>>> the messages in the trace:
> >>>>>
> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >>>>> # | | | ||||| | |
> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >>>>> direct func1 ip ff60000003334d00
> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >>>>> direct func1 ip 7fffffffffffffff
> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> [...]
> >>>>>
> >>>>> If I understand it right, my_direct_func1() should print the address of
> >>>>> some location in the code, probably - at the beginning of the traced
> >>>>> functions.
> >>>>>
> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >>>>> addresses.
> >>>>>
> >>>> The invalid code address is only printed by accessing the schedule()
> >>>> function's first argument whose address stores in a0 register.
> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >>>> just prints the a0 in the context of the schedule()'s caller and
> >>>> the address maybe varies depending on the caller.
> >>>>
> >>>> I can't really understand why tracing the first argument of the
> >>>> schedule() function, but it seems nonsense at this point.
> >>>
> >>> The question is, what should be passed as the argument(s) of
> >>> my_direct_func() in this particular sample module. The kernel docs and
> >>> commit logs seem to contain no info on that.
> >>>
> >>> With direct functions, I suppose, the trampoline can pass anything it
> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >>>
> >>> I'd check what these sample modules do on x86 and would try to match
> >>> that behaviour on RISC-V.
> >>
> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> be passed to my_direct_func().
> >>
> >> ftrace-direct-multi.ko:
> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> # | | | ||||| | |
> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> ffffffff87332f45 // wake_up_process+0x5
> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> ffffffff8828d935 // schedule+0x5
> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> ffffffff8828d935
> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> ffffffff8828d935
> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> ffffffff8828d935
> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> ffffffff87332f45
> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> ffffffff87332f45
> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> ffffffff8828d935
> >> [...]
> >>
> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> where the execution of the traced functions resumes after the Ftrace
> >> trampoline has finished.
> >>
> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >>
> >> The samples look like a demonstration, that one can pass anything
> >> necessary to the handler in case of "direct" functions.
> >>
> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >
> > Yes, you're right.
> >
> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> > traced function with mcount offset.
> >
> > Here is the updated patch for your reference.
> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> Thank you for the quick fix. This one looks good to me.
>
> ftrace-direct-multi*.ko now report the ip values corresponding to
> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
>
> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> your patch, becase this is what I actually did: tested it and reported
> the results. I cannot take your credit for development of this patch ;-)
>
> Looking forward for v8 of the series.
> >
> >
> >>
> >>>
> >>>>
> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >>>> trace kernel functions, and basically saves the necessary ABI,
> >>>> call the tracing function, and restores the ABI, just like other
> >>>> arches do.
> >>>> so It shouldn't be blamed.
> >>>>
> >>>> I started an independent patch to replace schedule with kick_process
> >>>> to make these samples more reasonable. And It has no conflict with the
> >>>> current patch, so we can go on.
> >>>>
> >>>> Link:
> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >>>>
> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >>>>>
> >>>>> Is anything missing here?
> >>>>>
> >>>>>>
> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>>>>> index 307a9f413edd..e944af44f681 100644
> >>>>>> --- a/arch/riscv/Kconfig
> >>>>>> +++ b/arch/riscv/Kconfig
> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>>>> select HAVE_STACKPROTECTOR
> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >>>>>> select HAVE_RSEQ
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >>>>>>
> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp1, @function\n"
> >>>>>> +" .globl my_tramp1\n"
> >>>>>> +" my_tramp1:\n"
> >>>>>> +" addi sp,sp,-16\n"
> >>>>>> +" sd t0,0(sp)\n"
> >>>>>> +" sd ra,8(sp)\n"
> >>>>>> +" call my_direct_func1\n"
> >>>>>> +" ld t0,0(sp)\n"
> >>>>>> +" ld ra,8(sp)\n"
> >>>>>> +" addi sp,sp,16\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>>>> +
> >>>>>> +" .type my_tramp2, @function\n"
> >>>>>> +" .globl my_tramp2\n"
> >>>>>> +" my_tramp2:\n"
> >>>>>> +" addi sp,sp,-16\n"
> >>>>>> +" sd t0,0(sp)\n"
> >>>>>> +" sd ra,8(sp)\n"
> >>>>>> +" call my_direct_func2\n"
> >>>>>> +" ld t0,0(sp)\n"
> >>>>>> +" ld ra,8(sp)\n"
> >>>>>> +" addi sp,sp,16\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >>>>>> extern void my_tramp1(void *);
> >>>>>> extern void my_tramp2(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp1, @function\n"
> >>>>>> +" .globl my_tramp1\n"
> >>>>>> +" my_tramp1:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func1\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>>>> +
> >>>>>> +" .type my_tramp2, @function\n"
> >>>>>> +" .globl my_tramp2\n"
> >>>>>> +" my_tramp2:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func2\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-40\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd a1,8(sp)\n"
> >>>>>> +" sd a2,16(sp)\n"
> >>>>>> +" sd t0,24(sp)\n"
> >>>>>> +" sd ra,32(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld a1,8(sp)\n"
> >>>>>> +" ld a2,16(sp)\n"
> >>>>>> +" ld t0,24(sp)\n"
> >>>>>> +" ld ra,32(sp)\n"
> >>>>>> +" addi sp,sp,40\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >>>>>> b/samples/ftrace/ftrace-direct.c
> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> --
> >>>>>> 2.36.1
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Song
> >>>>
> >>>
> >>> Regards,
> >>> Evgenii
> >>
> >>
> >
> >
> > --
> > Thanks,
> > Song
> >
>
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-01-19 6:05 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-19 6:05 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: Song Shuai, linux-riscv, linux-kernel, anup, paul.walmsley,
palmer, conor.dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
Thx Evgenii & Song,
I got it; it would be put into v8.
On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
<e.shatokhin@yadro.com> wrote:
>
> On 18.01.2023 05:37, Song Shuai wrote:
> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >>
> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >>> Hi, Song,
> >>>
> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >>>>
> >>>> Hi, Evgenii:
> >>>>
> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >>>>>>
> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >>>>>>
> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>>>> ---
> >>>>>> arch/riscv/Kconfig | 2 ++
> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >>>>>> +++++++++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >>>>>> 6 files changed, 142 insertions(+)
> >>>>>
> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >>>>>
> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >>>>> the messages in the trace:
> >>>>>
> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >>>>> # | | | ||||| | |
> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >>>>> direct func1 ip ff60000003334d00
> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >>>>> direct func1 ip 7fffffffffffffff
> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >>>>> direct func1 ip ff60000001ba9600
> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >>>>> direct func1 ip 0
> >>>>> [...]
> >>>>>
> >>>>> If I understand it right, my_direct_func1() should print the address of
> >>>>> some location in the code, probably - at the beginning of the traced
> >>>>> functions.
> >>>>>
> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >>>>> addresses.
> >>>>>
> >>>> The invalid code address is only printed by accessing the schedule()
> >>>> function's first argument whose address stores in a0 register.
> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >>>> just prints the a0 in the context of the schedule()'s caller and
> >>>> the address maybe varies depending on the caller.
> >>>>
> >>>> I can't really understand why tracing the first argument of the
> >>>> schedule() function, but it seems nonsense at this point.
> >>>
> >>> The question is, what should be passed as the argument(s) of
> >>> my_direct_func() in this particular sample module. The kernel docs and
> >>> commit logs seem to contain no info on that.
> >>>
> >>> With direct functions, I suppose, the trampoline can pass anything it
> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >>>
> >>> I'd check what these sample modules do on x86 and would try to match
> >>> that behaviour on RISC-V.
> >>
> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> be passed to my_direct_func().
> >>
> >> ftrace-direct-multi.ko:
> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> # | | | ||||| | |
> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> ffffffff87332f45 // wake_up_process+0x5
> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> ffffffff8828d935 // schedule+0x5
> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> ffffffff8828d935
> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> ffffffff8828d935
> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> ffffffff8828d935
> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> ffffffff87332f45
> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> ffffffff87332f45
> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> ffffffff8828d935
> >> [...]
> >>
> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> where the execution of the traced functions resumes after the Ftrace
> >> trampoline has finished.
> >>
> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >>
> >> The samples look like a demonstration, that one can pass anything
> >> necessary to the handler in case of "direct" functions.
> >>
> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >
> > Yes, you're right.
> >
> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> > traced function with mcount offset.
> >
> > Here is the updated patch for your reference.
> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> Thank you for the quick fix. This one looks good to me.
>
> ftrace-direct-multi*.ko now report the ip values corresponding to
> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
>
> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> your patch, becase this is what I actually did: tested it and reported
> the results. I cannot take your credit for development of this patch ;-)
>
> Looking forward for v8 of the series.
> >
> >
> >>
> >>>
> >>>>
> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >>>> trace kernel functions, and basically saves the necessary ABI,
> >>>> call the tracing function, and restores the ABI, just like other
> >>>> arches do.
> >>>> so It shouldn't be blamed.
> >>>>
> >>>> I started an independent patch to replace schedule with kick_process
> >>>> to make these samples more reasonable. And It has no conflict with the
> >>>> current patch, so we can go on.
> >>>>
> >>>> Link:
> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >>>>
> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >>>>>
> >>>>> Is anything missing here?
> >>>>>
> >>>>>>
> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>>>>> index 307a9f413edd..e944af44f681 100644
> >>>>>> --- a/arch/riscv/Kconfig
> >>>>>> +++ b/arch/riscv/Kconfig
> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >>>>>> select HAVE_STACKPROTECTOR
> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >>>>>> select HAVE_RSEQ
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >>>>>>
> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp1, @function\n"
> >>>>>> +" .globl my_tramp1\n"
> >>>>>> +" my_tramp1:\n"
> >>>>>> +" addi sp,sp,-16\n"
> >>>>>> +" sd t0,0(sp)\n"
> >>>>>> +" sd ra,8(sp)\n"
> >>>>>> +" call my_direct_func1\n"
> >>>>>> +" ld t0,0(sp)\n"
> >>>>>> +" ld ra,8(sp)\n"
> >>>>>> +" addi sp,sp,16\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>>>> +
> >>>>>> +" .type my_tramp2, @function\n"
> >>>>>> +" .globl my_tramp2\n"
> >>>>>> +" my_tramp2:\n"
> >>>>>> +" addi sp,sp,-16\n"
> >>>>>> +" sd t0,0(sp)\n"
> >>>>>> +" sd ra,8(sp)\n"
> >>>>>> +" call my_direct_func2\n"
> >>>>>> +" ld t0,0(sp)\n"
> >>>>>> +" ld ra,8(sp)\n"
> >>>>>> +" addi sp,sp,16\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >>>>>> extern void my_tramp1(void *);
> >>>>>> extern void my_tramp2(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp1, @function\n"
> >>>>>> +" .globl my_tramp1\n"
> >>>>>> +" my_tramp1:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func1\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >>>>>> +
> >>>>>> +" .type my_tramp2, @function\n"
> >>>>>> +" .globl my_tramp2\n"
> >>>>>> +" my_tramp2:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func2\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-40\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd a1,8(sp)\n"
> >>>>>> +" sd a2,16(sp)\n"
> >>>>>> +" sd t0,24(sp)\n"
> >>>>>> +" sd ra,32(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld a1,8(sp)\n"
> >>>>>> +" ld a2,16(sp)\n"
> >>>>>> +" ld t0,24(sp)\n"
> >>>>>> +" ld ra,32(sp)\n"
> >>>>>> +" addi sp,sp,40\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >>>>>> b/samples/ftrace/ftrace-direct.c
> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >>>>>>
> >>>>>> extern void my_tramp(void *);
> >>>>>>
> >>>>>> +#ifdef CONFIG_RISCV
> >>>>>> +
> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >>>>>> +" .type my_tramp, @function\n"
> >>>>>> +" .globl my_tramp\n"
> >>>>>> +" my_tramp:\n"
> >>>>>> +" addi sp,sp,-24\n"
> >>>>>> +" sd a0,0(sp)\n"
> >>>>>> +" sd t0,8(sp)\n"
> >>>>>> +" sd ra,16(sp)\n"
> >>>>>> +" call my_direct_func\n"
> >>>>>> +" ld a0,0(sp)\n"
> >>>>>> +" ld t0,8(sp)\n"
> >>>>>> +" ld ra,16(sp)\n"
> >>>>>> +" addi sp,sp,24\n"
> >>>>>> +" jr t0\n"
> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >>>>>> +" .popsection\n"
> >>>>>> +);
> >>>>>> +
> >>>>>> +#endif /* CONFIG_RISCV */
> >>>>>> +
> >>>>>> #ifdef CONFIG_X86_64
> >>>>>>
> >>>>>> #include <asm/ibt.h>
> >>>>>> --
> >>>>>> 2.36.1
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Song
> >>>>
> >>>
> >>> Regards,
> >>> Evgenii
> >>
> >>
> >
> >
> > --
> > Thanks,
> > Song
> >
>
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-12 12:16 ` Mark Rutland
@ 2023-01-28 9:37 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-28 9:37 UTC (permalink / raw)
To: Mark Rutland
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> Hi Guo,
>
> On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > From: Andy Chiu <andy.chiu@sifive.com>
> >
> > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > forming a jump that jumps to an address over 4K. This may cause errors
> > if we want to enable kernel preemption and remove dependency from
> > patching code with stop_machine(). For example, if a task was switched
> > out on auipc. And, if we changed the ftrace function before it was
> > switched back, then it would jump to an address that has updated 11:0
> > bits mixing with previous XLEN:12 part.
> >
> > p: patched area performed by dynamic ftrace
> > ftrace_prologue:
> > p| REG_S ra, -SZREG(sp)
> > p| auipc ra, 0x? ------------> preempted
> > ...
> > change ftrace function
> > ...
> > p| jalr -?(ra) <------------- switched back
> > p| REG_L ra, -SZREG(sp)
> > func:
> > xxx
> > ret
>
> As mentioned on the last posting, I don't think this is sufficient to fix the
> issue. I've replied with more detail there:
>
> https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
>
> Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> executing the ftrace_prologue while another CPU is patching the
> ftrace_prologue, you have the exact same issue.
>
> For example, if CPU X is in the prologue fetches the old AUIPC and the new
> JALR (because it races with CPU Y modifying those), CPU X will branch to the
> wrong address. The race window is much smaller in the absence of preemption,
> but it's still there (and will be exacerbated in virtual machines since the
> hypervisor can preempt a vCPU at any time).
>
> Note that the above is even assuming that instruction fetches are atomic, which
> I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> MODification and eXecutuion of instructions" rules which mean only certain
> instructions can be patched atomically.
>
> Either I'm missing something that provides mutual exclusion between the
> patching and execution of the ftrace_prologue, or this patch is not sufficient.
This patch is sufficient because riscv isn't the same as arm64. It
uses default arch_ftrace_update_code, which uses stop_machine.
See kernel/trace/ftrace.c:
void __weak arch_ftrace_update_code(int command)
{
ftrace_run_stop_machine(command);
}
ps:
Yes, it's not good, and it's expensive.
>
> Thanks,
> Mark.
>
> > Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
> > Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e2b656043abf..ee0d39b26794 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -138,7 +138,7 @@ config RISCV
> > select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
> > select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
> > select HAVE_FUNCTION_GRAPH_TRACER
> > - select HAVE_FUNCTION_TRACER if !XIP_KERNEL
> > + select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
> >
> > config ARCH_MMAP_RND_BITS_MIN
> > default 18 if 64BIT
> > --
> > 2.36.1
> >
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-28 9:37 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-28 9:37 UTC (permalink / raw)
To: Mark Rutland
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> Hi Guo,
>
> On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > From: Andy Chiu <andy.chiu@sifive.com>
> >
> > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > forming a jump that jumps to an address over 4K. This may cause errors
> > if we want to enable kernel preemption and remove dependency from
> > patching code with stop_machine(). For example, if a task was switched
> > out on auipc. And, if we changed the ftrace function before it was
> > switched back, then it would jump to an address that has updated 11:0
> > bits mixing with previous XLEN:12 part.
> >
> > p: patched area performed by dynamic ftrace
> > ftrace_prologue:
> > p| REG_S ra, -SZREG(sp)
> > p| auipc ra, 0x? ------------> preempted
> > ...
> > change ftrace function
> > ...
> > p| jalr -?(ra) <------------- switched back
> > p| REG_L ra, -SZREG(sp)
> > func:
> > xxx
> > ret
>
> As mentioned on the last posting, I don't think this is sufficient to fix the
> issue. I've replied with more detail there:
>
> https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
>
> Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> executing the ftrace_prologue while another CPU is patching the
> ftrace_prologue, you have the exact same issue.
>
> For example, if CPU X is in the prologue fetches the old AUIPC and the new
> JALR (because it races with CPU Y modifying those), CPU X will branch to the
> wrong address. The race window is much smaller in the absence of preemption,
> but it's still there (and will be exacerbated in virtual machines since the
> hypervisor can preempt a vCPU at any time).
>
> Note that the above is even assuming that instruction fetches are atomic, which
> I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> MODification and eXecutuion of instructions" rules which mean only certain
> instructions can be patched atomically.
>
> Either I'm missing something that provides mutual exclusion between the
> patching and execution of the ftrace_prologue, or this patch is not sufficient.
This patch is sufficient because riscv isn't the same as arm64. It
uses default arch_ftrace_update_code, which uses stop_machine.
See kernel/trace/ftrace.c:
void __weak arch_ftrace_update_code(int command)
{
ftrace_run_stop_machine(command);
}
ps:
Yes, it's not good, and it's expensive.
>
> Thanks,
> Mark.
>
> > Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")
> > Signed-off-by: Andy Chiu <andy.chiu@sifive.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e2b656043abf..ee0d39b26794 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -138,7 +138,7 @@ config RISCV
> > select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
> > select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
> > select HAVE_FUNCTION_GRAPH_TRACER
> > - select HAVE_FUNCTION_TRACER if !XIP_KERNEL
> > + select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION
> >
> > config ARCH_MMAP_RND_BITS_MIN
> > default 18 if 64BIT
> > --
> > 2.36.1
> >
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-12 12:57 ` Mark Rutland
@ 2023-01-28 9:45 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-28 9:45 UTC (permalink / raw)
To: Mark Rutland
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 8:57 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Thu, Jan 12, 2023 at 12:16:02PM +0000, Mark Rutland wrote:
> > Hi Guo,
> >
> > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > From: Andy Chiu <andy.chiu@sifive.com>
> > >
> > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > forming a jump that jumps to an address over 4K. This may cause errors
> > > if we want to enable kernel preemption and remove dependency from
> > > patching code with stop_machine(). For example, if a task was switched
> > > out on auipc. And, if we changed the ftrace function before it was
> > > switched back, then it would jump to an address that has updated 11:0
> > > bits mixing with previous XLEN:12 part.
> > >
> > > p: patched area performed by dynamic ftrace
> > > ftrace_prologue:
> > > p| REG_S ra, -SZREG(sp)
> > > p| auipc ra, 0x? ------------> preempted
> > > ...
> > > change ftrace function
> > > ...
> > > p| jalr -?(ra) <------------- switched back
> > > p| REG_L ra, -SZREG(sp)
> > > func:
> > > xxx
> > > ret
> >
> > As mentioned on the last posting, I don't think this is sufficient to fix the
> > issue. I've replied with more detail there:
> >
> > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> >
> > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > executing the ftrace_prologue while another CPU is patching the
> > ftrace_prologue, you have the exact same issue.
> >
> > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > wrong address. The race window is much smaller in the absence of preemption,
> > but it's still there (and will be exacerbated in virtual machines since the
> > hypervisor can preempt a vCPU at any time).
>
> With that in mind, I think your current implementation of ftrace_make_call()
> and ftrace_make_nop() have a simlar bug. A caller might execute:
>
> NOP // not yet patched to AUIPC
>
> < AUIPC and JALR instructions both patched >
>
> JALR
>
> ... and go to the wrong place.
>
> Assuming individual instruction fetches are atomic, and that you only ever
> branch to the same trampoline, you could fix that by always leaving the AUIPC
> in place, so that you only patch the JALR to enable/disable the callsite.
Yes, the same trampoline is one of the antidotes.
>
> Depending on your calling convention, if you have two free GPRs, you might be
> able to avoid the stacking of RA by always saving it to a GPR in the callsite,
> using a different GPR for the address generation, and having the ftrace
> trampoline restore the original RA value, e.g.
>
> MV GPR1, ra
> AUIPC GPR2, high_bits_of(ftrace_caller)
> JALR ra, high_bits(GPR2) // only patch this
I think you mean temp registers here. We are at the prologue of a
function, so we have all of them.
But why do you need another "MV GPR1, ra"
AUIPC GPR2, high_bits_of(ftrace_caller)
JALR GPR2, high_bits(GPR2) // only patch this
We could reserve ra on the trampoline.
MV XX, ra
>
> ... which'd save an instruction per callsite.
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-28 9:45 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-01-28 9:45 UTC (permalink / raw)
To: Mark Rutland
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Thu, Jan 12, 2023 at 8:57 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Thu, Jan 12, 2023 at 12:16:02PM +0000, Mark Rutland wrote:
> > Hi Guo,
> >
> > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > From: Andy Chiu <andy.chiu@sifive.com>
> > >
> > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > forming a jump that jumps to an address over 4K. This may cause errors
> > > if we want to enable kernel preemption and remove dependency from
> > > patching code with stop_machine(). For example, if a task was switched
> > > out on auipc. And, if we changed the ftrace function before it was
> > > switched back, then it would jump to an address that has updated 11:0
> > > bits mixing with previous XLEN:12 part.
> > >
> > > p: patched area performed by dynamic ftrace
> > > ftrace_prologue:
> > > p| REG_S ra, -SZREG(sp)
> > > p| auipc ra, 0x? ------------> preempted
> > > ...
> > > change ftrace function
> > > ...
> > > p| jalr -?(ra) <------------- switched back
> > > p| REG_L ra, -SZREG(sp)
> > > func:
> > > xxx
> > > ret
> >
> > As mentioned on the last posting, I don't think this is sufficient to fix the
> > issue. I've replied with more detail there:
> >
> > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> >
> > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > executing the ftrace_prologue while another CPU is patching the
> > ftrace_prologue, you have the exact same issue.
> >
> > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > wrong address. The race window is much smaller in the absence of preemption,
> > but it's still there (and will be exacerbated in virtual machines since the
> > hypervisor can preempt a vCPU at any time).
>
> With that in mind, I think your current implementation of ftrace_make_call()
> and ftrace_make_nop() have a simlar bug. A caller might execute:
>
> NOP // not yet patched to AUIPC
>
> < AUIPC and JALR instructions both patched >
>
> JALR
>
> ... and go to the wrong place.
>
> Assuming individual instruction fetches are atomic, and that you only ever
> branch to the same trampoline, you could fix that by always leaving the AUIPC
> in place, so that you only patch the JALR to enable/disable the callsite.
Yes, the same trampoline is one of the antidotes.
>
> Depending on your calling convention, if you have two free GPRs, you might be
> able to avoid the stacking of RA by always saving it to a GPR in the callsite,
> using a different GPR for the address generation, and having the ftrace
> trampoline restore the original RA value, e.g.
>
> MV GPR1, ra
> AUIPC GPR2, high_bits_of(ftrace_caller)
> JALR ra, high_bits(GPR2) // only patch this
I think you mean temp registers here. We are at the prologue of a
function, so we have all of them.
But why do you need another "MV GPR1, ra"
AUIPC GPR2, high_bits_of(ftrace_caller)
JALR GPR2, high_bits(GPR2) // only patch this
We could reserve ra on the trampoline.
MV XX, ra
>
> ... which'd save an instruction per callsite.
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-28 9:37 ` Guo Ren
@ 2023-01-30 10:54 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-30 10:54 UTC (permalink / raw)
To: Guo Ren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Sat, Jan 28, 2023 at 05:37:46PM +0800, Guo Ren wrote:
> On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
> >
> > Hi Guo,
> >
> > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > From: Andy Chiu <andy.chiu@sifive.com>
> > >
> > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > forming a jump that jumps to an address over 4K. This may cause errors
> > > if we want to enable kernel preemption and remove dependency from
> > > patching code with stop_machine(). For example, if a task was switched
> > > out on auipc. And, if we changed the ftrace function before it was
> > > switched back, then it would jump to an address that has updated 11:0
> > > bits mixing with previous XLEN:12 part.
> > >
> > > p: patched area performed by dynamic ftrace
> > > ftrace_prologue:
> > > p| REG_S ra, -SZREG(sp)
> > > p| auipc ra, 0x? ------------> preempted
> > > ...
> > > change ftrace function
> > > ...
> > > p| jalr -?(ra) <------------- switched back
> > > p| REG_L ra, -SZREG(sp)
> > > func:
> > > xxx
> > > ret
> >
> > As mentioned on the last posting, I don't think this is sufficient to fix the
> > issue. I've replied with more detail there:
> >
> > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> >
> > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > executing the ftrace_prologue while another CPU is patching the
> > ftrace_prologue, you have the exact same issue.
> >
> > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > wrong address. The race window is much smaller in the absence of preemption,
> > but it's still there (and will be exacerbated in virtual machines since the
> > hypervisor can preempt a vCPU at any time).
> >
> > Note that the above is even assuming that instruction fetches are atomic, which
> > I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> > MODification and eXecutuion of instructions" rules which mean only certain
> > instructions can be patched atomically.
> >
> > Either I'm missing something that provides mutual exclusion between the
> > patching and execution of the ftrace_prologue, or this patch is not sufficient.
> This patch is sufficient because riscv isn't the same as arm64. It
> uses default arch_ftrace_update_code, which uses stop_machine.
> See kernel/trace/ftrace.c:
> void __weak arch_ftrace_update_code(int command)
> {
> ftrace_run_stop_machine(command);
> }
Ah; sorry, I had misunderstood here, since the commit message spoke in terms of
removing that.
As long as stop_machine() is used I agree this is safe; sorry for the noise.
> ps:
> Yes, it's not good, and it's expensive.
We can't have everything! :)
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-01-30 10:54 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-01-30 10:54 UTC (permalink / raw)
To: Guo Ren
Cc: anup, paul.walmsley, palmer, conor.dooley, heiko, rostedt,
mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-riscv, linux-kernel
On Sat, Jan 28, 2023 at 05:37:46PM +0800, Guo Ren wrote:
> On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
> >
> > Hi Guo,
> >
> > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > From: Andy Chiu <andy.chiu@sifive.com>
> > >
> > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > forming a jump that jumps to an address over 4K. This may cause errors
> > > if we want to enable kernel preemption and remove dependency from
> > > patching code with stop_machine(). For example, if a task was switched
> > > out on auipc. And, if we changed the ftrace function before it was
> > > switched back, then it would jump to an address that has updated 11:0
> > > bits mixing with previous XLEN:12 part.
> > >
> > > p: patched area performed by dynamic ftrace
> > > ftrace_prologue:
> > > p| REG_S ra, -SZREG(sp)
> > > p| auipc ra, 0x? ------------> preempted
> > > ...
> > > change ftrace function
> > > ...
> > > p| jalr -?(ra) <------------- switched back
> > > p| REG_L ra, -SZREG(sp)
> > > func:
> > > xxx
> > > ret
> >
> > As mentioned on the last posting, I don't think this is sufficient to fix the
> > issue. I've replied with more detail there:
> >
> > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> >
> > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > executing the ftrace_prologue while another CPU is patching the
> > ftrace_prologue, you have the exact same issue.
> >
> > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > wrong address. The race window is much smaller in the absence of preemption,
> > but it's still there (and will be exacerbated in virtual machines since the
> > hypervisor can preempt a vCPU at any time).
> >
> > Note that the above is even assuming that instruction fetches are atomic, which
> > I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> > MODification and eXecutuion of instructions" rules which mean only certain
> > instructions can be patched atomically.
> >
> > Either I'm missing something that provides mutual exclusion between the
> > patching and execution of the ftrace_prologue, or this patch is not sufficient.
> This patch is sufficient because riscv isn't the same as arm64. It
> uses default arch_ftrace_update_code, which uses stop_machine.
> See kernel/trace/ftrace.c:
> void __weak arch_ftrace_update_code(int command)
> {
> ftrace_run_stop_machine(command);
> }
Ah; sorry, I had misunderstood here, since the commit message spoke in terms of
removing that.
As long as stop_machine() is used I agree this is safe; sorry for the noise.
> ps:
> Yes, it's not good, and it's expensive.
We can't have everything! :)
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
2023-01-30 10:54 ` Mark Rutland
@ 2023-02-04 1:19 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-04 1:19 UTC (permalink / raw)
To: palmer, Mark Rutland
Cc: anup, paul.walmsley, conor.dooley, heiko, rostedt, mhiramat,
jolsa, bp, jpoimboe, suagrfillet, andy.chiu, e.shatokhin,
linux-riscv, linux-kernel
On Mon, Jan 30, 2023 at 6:54 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Sat, Jan 28, 2023 at 05:37:46PM +0800, Guo Ren wrote:
> > On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > >
> > > Hi Guo,
> > >
> > > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > > From: Andy Chiu <andy.chiu@sifive.com>
> > > >
> > > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > > forming a jump that jumps to an address over 4K. This may cause errors
> > > > if we want to enable kernel preemption and remove dependency from
> > > > patching code with stop_machine(). For example, if a task was switched
> > > > out on auipc. And, if we changed the ftrace function before it was
> > > > switched back, then it would jump to an address that has updated 11:0
> > > > bits mixing with previous XLEN:12 part.
> > > >
> > > > p: patched area performed by dynamic ftrace
> > > > ftrace_prologue:
> > > > p| REG_S ra, -SZREG(sp)
> > > > p| auipc ra, 0x? ------------> preempted
> > > > ...
> > > > change ftrace function
> > > > ...
> > > > p| jalr -?(ra) <------------- switched back
> > > > p| REG_L ra, -SZREG(sp)
> > > > func:
> > > > xxx
> > > > ret
> > >
> > > As mentioned on the last posting, I don't think this is sufficient to fix the
> > > issue. I've replied with more detail there:
> > >
> > > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> > >
> > > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > > executing the ftrace_prologue while another CPU is patching the
> > > ftrace_prologue, you have the exact same issue.
> > >
> > > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > > wrong address. The race window is much smaller in the absence of preemption,
> > > but it's still there (and will be exacerbated in virtual machines since the
> > > hypervisor can preempt a vCPU at any time).
> > >
> > > Note that the above is even assuming that instruction fetches are atomic, which
> > > I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> > > MODification and eXecutuion of instructions" rules which mean only certain
> > > instructions can be patched atomically.
> > >
> > > Either I'm missing something that provides mutual exclusion between the
> > > patching and execution of the ftrace_prologue, or this patch is not sufficient.
> > This patch is sufficient because riscv isn't the same as arm64. It
> > uses default arch_ftrace_update_code, which uses stop_machine.
> > See kernel/trace/ftrace.c:
> > void __weak arch_ftrace_update_code(int command)
> > {
> > ftrace_run_stop_machine(command);
> > }
>
> Ah; sorry, I had misunderstood here, since the commit message spoke in terms of
> removing that.
>
> As long as stop_machine() is used I agree this is safe; sorry for the noise.
Okay.
Hi Palmer,
Please take Andy's fixup patch. We would continue to find a way for PREEMPTION.
>
> > ps:
> > Yes, it's not good, and it's expensive.
>
> We can't have everything! :)
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption
@ 2023-02-04 1:19 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-04 1:19 UTC (permalink / raw)
To: palmer, Mark Rutland
Cc: anup, paul.walmsley, conor.dooley, heiko, rostedt, mhiramat,
jolsa, bp, jpoimboe, suagrfillet, andy.chiu, e.shatokhin,
linux-riscv, linux-kernel
On Mon, Jan 30, 2023 at 6:54 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Sat, Jan 28, 2023 at 05:37:46PM +0800, Guo Ren wrote:
> > On Thu, Jan 12, 2023 at 8:16 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > >
> > > Hi Guo,
> > >
> > > On Thu, Jan 12, 2023 at 04:05:57AM -0500, guoren@kernel.org wrote:
> > > > From: Andy Chiu <andy.chiu@sifive.com>
> > > >
> > > > In RISCV, we must use an AUIPC + JALR pair to encode an immediate,
> > > > forming a jump that jumps to an address over 4K. This may cause errors
> > > > if we want to enable kernel preemption and remove dependency from
> > > > patching code with stop_machine(). For example, if a task was switched
> > > > out on auipc. And, if we changed the ftrace function before it was
> > > > switched back, then it would jump to an address that has updated 11:0
> > > > bits mixing with previous XLEN:12 part.
> > > >
> > > > p: patched area performed by dynamic ftrace
> > > > ftrace_prologue:
> > > > p| REG_S ra, -SZREG(sp)
> > > > p| auipc ra, 0x? ------------> preempted
> > > > ...
> > > > change ftrace function
> > > > ...
> > > > p| jalr -?(ra) <------------- switched back
> > > > p| REG_L ra, -SZREG(sp)
> > > > func:
> > > > xxx
> > > > ret
> > >
> > > As mentioned on the last posting, I don't think this is sufficient to fix the
> > > issue. I've replied with more detail there:
> > >
> > > https://lore.kernel.org/lkml/Y7%2F3hoFjS49yy52W@FVFF77S0Q05N/
> > >
> > > Even in a non-preemptible SMP kernel, if one CPU can be in the middle of
> > > executing the ftrace_prologue while another CPU is patching the
> > > ftrace_prologue, you have the exact same issue.
> > >
> > > For example, if CPU X is in the prologue fetches the old AUIPC and the new
> > > JALR (because it races with CPU Y modifying those), CPU X will branch to the
> > > wrong address. The race window is much smaller in the absence of preemption,
> > > but it's still there (and will be exacerbated in virtual machines since the
> > > hypervisor can preempt a vCPU at any time).
> > >
> > > Note that the above is even assuming that instruction fetches are atomic, which
> > > I'm not sure is the case; for example arm64 has special CMODX / "Concurrent
> > > MODification and eXecutuion of instructions" rules which mean only certain
> > > instructions can be patched atomically.
> > >
> > > Either I'm missing something that provides mutual exclusion between the
> > > patching and execution of the ftrace_prologue, or this patch is not sufficient.
> > This patch is sufficient because riscv isn't the same as arm64. It
> > uses default arch_ftrace_update_code, which uses stop_machine.
> > See kernel/trace/ftrace.c:
> > void __weak arch_ftrace_update_code(int command)
> > {
> > ftrace_run_stop_machine(command);
> > }
>
> Ah; sorry, I had misunderstood here, since the commit message spoke in terms of
> removing that.
>
> As long as stop_machine() is used I agree this is safe; sorry for the noise.
Okay.
Hi Palmer,
Please take Andy's fixup patch. We would continue to find a way for PREEMPTION.
>
> > ps:
> > Yes, it's not good, and it's expensive.
>
> We can't have everything! :)
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-01-16 15:02 ` Evgenii Shatokhin
@ 2023-02-04 6:40 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-04 6:40 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: suagrfillet, andy.chiu, linux-riscv, linux-kernel, Guo Ren, anup,
paul.walmsley, palmer, conor.dooley, heiko, rostedt, mhiramat,
jolsa, bp, jpoimboe, linux
On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
<e.shatokhin@yadro.com> wrote:
>
> Hi,
>
> On 12.01.2023 12:05, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> >
> > - The most horrible bug is preemption panic which found by Andy [1].
> > Let's disable preemption for ftrace first, and Andy could continue
> > the ftrace preemption work.
>
> It seems, the patches #2-#7 of this series do not require "riscv:
> ftrace: Fixup panic by disabling preemption" and can be used without it.
>
> How about moving that patch out of the series and processing it separately?
Okay.
>
> As it was pointed out in the discussion of that patch, some other
> solution to non-atomic changes of the prologue might be needed anyway.
I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
still needs to be ready. Let's disable PREEMPT for ftrace first.
>
> > - The "-fpatchable-function-entry= CFLAG" wasted code size
> > !RISCV_ISA_C.
> > - The ftrace detour implementation wasted code size.
> > - When livepatching, the trampoline (ftrace_regs_caller) would not
> > return to <func_prolog+12> but would rather jump to the new function.
> > So, "REG_L ra, -SZREG(sp)" would not run and the original return
> > address would not be restored. The kernel is likely to hang or crash
> > as a result. (Found by Evgenii Shatokhin [4])
> >
> > Patches 1,2,3 fixup above problems.
> >
> > Patches 4,5,6,7 are the features based on reduced detour code
> > patch, we include them in the series for test and maintenance.
> >
> > You can directly try it with:
> > https://github.com/guoren83/linux/tree/ftrace_fixup_v7
> >
> > Make function graph use ftrace directly [2] (patch 4, 5)
> > ========================================================
> >
> > In RISC-V architecture, when we enable the ftrace_graph tracer on some
> > functions, the function tracings on other functions will suffer extra
> > graph tracing work. In essence, graph_ops isn't limited by its func_hash
> > due to the global ftrace_graph_[regs]_call label. That should be
> > corrected.
> >
> > What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> > graph use ftrace directly") that uses graph_ops::func function to
> > install return_hooker and makes the function called against its
> > func_hash.
> >
> > This series of patches makes function graph use ftrace directly for
> > riscv.
> >
> > If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
> > so that it can be replaced with the calling of prepare_ftrace_return by
> > the enable/disable helper.
> >
> > As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
> > necessary regs against the pt_regs layout, so it can reasonably call the
> > graph_ops::func function - ftrace_graph_func. And
> > ftrace_graph_[regs]_call
> > and its enable/disable helper aren't needed.
> >
> > Test log:
> >
> > The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
> > local
> > qemu-system-riscv64 virt machine. The following is the log during
> > startup.
> >
> > ```
> > Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
> > Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
> > Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
> > Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
> > Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
> > Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
> > Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
> > Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
> > Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
> > Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
> > Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
> > Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
> > Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
> > Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
> > Nov 15 03:07:13 stage4 kernel: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
> > ```
> >
> > Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
> > ==============================================
> >
> > This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> > SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> > here as the samples for testing DIRECT_CALLS related interface.
> >
> > First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> > register_ftrace_direct[_multi] interfaces allowing user to register
> > the customed trampoline (direct_caller) as the mcount for one or
> > more target functions. And modify_ftrace_direct[_multi] are also
> > provided for modify direct_caller.
> >
> > At the same time, the samples in ./samples/ftrace/ can be built
> > as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> > and SAMPLE_FTRACE_DIRECT_MULTI selected.
> >
> > Second, to make the direct_caller and the other ftrace hooks
> > (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> > register
> > are nominated to store the address of direct_caller in
> > ftrace_regs_caller.
> > After the setting of the address direct_caller by direct_ops->func and
> > the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> > by the `jr` inst.
> >
> > The following tests have been passed in my local qemu-riscv64 virt
> > machine.
> >
> > 1. tests with CONFIG_FTRACE_STARTUP_TEST
> > 2. tests of samples/ftrace/ftrace*.ko
> > 3. manual tests with any combination of the following hooks
> > - function/function_graph tracer
> > - ftrace*.ko
> > - kprobe/kretprobe
> >
> > For your reference, here is the log when function tracer, kretprobe and
> > ftrace-direct-too.ko co-hooks the handle_mm_fault function.
> >
> > ```
> > [root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
> > [root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
> > [root@stage4 tracing]# echo function > current_tracer
> > [root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
> > [root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
> > [root@stage4 tracing]#
> > [root@stage4 tracing]# cat trace | tail
> > cat-388 [000] ...1. 583.051438: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > cat-388 [000] ...2. 583.057930: handle_mm_fault
> > <-do_page_fault
> > cat-388 [000] ..... 583.057990: my_direct_func:
> > handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
> > cat-388 [000] ...1. 583.058284: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > tail-389 [001] ...2. 583.059062: handle_mm_fault
> > <-do_page_fault
> > tail-389 [001] ..... 583.059104: my_direct_func:
> > handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
> > tail-389 [001] ...1. 583.059325: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > tail-389 [001] ...2. 583.060371: handle_mm_fault
> > <-do_page_fault
> > tail-389 [001] ..... 583.060410: my_direct_func:
> > handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
> > tail-389 [001] ...1. 583.060996: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > ```
> > Note1:
> >
> > The checkpatch.pl will output some warnings on this series, like this
> >
> > ```
> > WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
> > this function's name, in a string
> > 111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
> > +" call my_direct_func2\n"
> > ```
> >
> > The reason is that checkpatch depends on patch context providing the
> > function name. In the above warning, my_direct_func2 has some codeline
> > distance with the changed trunk, so its declaration doesn't come into
> > the patch, and then the warning jumps out.
> >
> > You may notice the location of `my_ip` variable changes in the 2nd
> > patch. I did that for reducing the warnings to some extent. But killing
> > all the warnings will makes the patch less readable, so I stopped here.
> >
> > [1] https://lpc.events/event/16/contributions/1171/
> > [2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> > [3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
> > [4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
> >
> > Changes in v7:
> > - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
> > - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> > - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
> > HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >
> > Changes in v6:
> > https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
> > - Replace 8 with MCOUNT_INSN_SIZE
> > - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
> > - Add Evgenii Shatokhin comment
> >
> > Changes in v5:
> > https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
> > - Sort Kconfig entries in alphabetical order.
> >
> > Changes in v4:
> > https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
> > - Include [3] for maintenance. [Song Shuai]
> >
> > Changes in V3:
> > https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
> > - Include [2] for maintenance. [Song Shuai]
> >
> > Changes in V2:
> > https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
> > - Add Signed-off for preemption fixup.
> >
> > Changes in V1:
> > https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
> >
> > Andy Chiu (1):
> > riscv: ftrace: Fixup panic by disabling preemption
> >
> > Guo Ren (2):
> > riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
> > riscv: ftrace: Reduce the detour code size to half
> >
> > Song Shuai (4):
> > riscv: ftrace: Add ftrace_graph_func
> > riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
> > samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
> > riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> >
> > arch/riscv/Kconfig | 6 +-
> > arch/riscv/Makefile | 6 +-
> > arch/riscv/include/asm/ftrace.h | 71 ++++++--
> > arch/riscv/kernel/ftrace.c | 91 ++++------
> > arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
> > samples/ftrace/ftrace-direct-modify.c | 33 ++++
> > samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
> > samples/ftrace/ftrace-direct-multi.c | 22 +++
> > samples/ftrace/ftrace-direct-too.c | 26 +++
> > samples/ftrace/ftrace-direct.c | 22 +++
> > 10 files changed, 356 insertions(+), 137 deletions(-)
> >
> > --
> > 2.36.1
> >
> >
>
> I tested this series a bit on top of 6.2-rc4, with
> https://github.com/sugarfillet/linux/commit/9539a80dc6e7d1137ec7a96ebef2ab912a694bd7.patch
> added.
>
> The kernel was built with CONFIG_DYNAMIC_FTRACE_WITH_REGS=y and
> CONFIG_RISCV_ISA_C=y.
>
> Here are some results:
>
> * The kernel was built without visible issues.
>
> * Most of the Ftrace startup tests ran and passed. Certain
> event-specific tests caused soft lockups in one of the test runs but
> they are likely unrelated to this patchset and could happen without it too.
>
> * "function" and "function_graph" tracers worked in my simple use cases.
>
> * "ipmodify" functionality worked.
>
> * kprobe sample modules worked OK, which is good, because they actually
> use Ftrace: they plant the probes at the beginning of the resp. functions.
Thx for test
>
> * ftrace-direct-multi and ftrace-direct-multi-modify sample modules
> reported possibly invalid data. More info - in my reply to
> "[PATCH -next V7 6/7] samples: ftrace: Add riscv support for
> SAMPLE_FTRACE_DIRECT[_MULTI]"
Good report! We will solve it in the next version.
>
> Regards,
> Evgenii
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-04 6:40 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-04 6:40 UTC (permalink / raw)
To: Evgenii Shatokhin
Cc: suagrfillet, andy.chiu, linux-riscv, linux-kernel, Guo Ren, anup,
paul.walmsley, palmer, conor.dooley, heiko, rostedt, mhiramat,
jolsa, bp, jpoimboe, linux
On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
<e.shatokhin@yadro.com> wrote:
>
> Hi,
>
> On 12.01.2023 12:05, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> >
> > - The most horrible bug is preemption panic which found by Andy [1].
> > Let's disable preemption for ftrace first, and Andy could continue
> > the ftrace preemption work.
>
> It seems, the patches #2-#7 of this series do not require "riscv:
> ftrace: Fixup panic by disabling preemption" and can be used without it.
>
> How about moving that patch out of the series and processing it separately?
Okay.
>
> As it was pointed out in the discussion of that patch, some other
> solution to non-atomic changes of the prologue might be needed anyway.
I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
still needs to be ready. Let's disable PREEMPT for ftrace first.
>
> > - The "-fpatchable-function-entry= CFLAG" wasted code size
> > !RISCV_ISA_C.
> > - The ftrace detour implementation wasted code size.
> > - When livepatching, the trampoline (ftrace_regs_caller) would not
> > return to <func_prolog+12> but would rather jump to the new function.
> > So, "REG_L ra, -SZREG(sp)" would not run and the original return
> > address would not be restored. The kernel is likely to hang or crash
> > as a result. (Found by Evgenii Shatokhin [4])
> >
> > Patches 1,2,3 fixup above problems.
> >
> > Patches 4,5,6,7 are the features based on reduced detour code
> > patch, we include them in the series for test and maintenance.
> >
> > You can directly try it with:
> > https://github.com/guoren83/linux/tree/ftrace_fixup_v7
> >
> > Make function graph use ftrace directly [2] (patch 4, 5)
> > ========================================================
> >
> > In RISC-V architecture, when we enable the ftrace_graph tracer on some
> > functions, the function tracings on other functions will suffer extra
> > graph tracing work. In essence, graph_ops isn't limited by its func_hash
> > due to the global ftrace_graph_[regs]_call label. That should be
> > corrected.
> >
> > What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> > graph use ftrace directly") that uses graph_ops::func function to
> > install return_hooker and makes the function called against its
> > func_hash.
> >
> > This series of patches makes function graph use ftrace directly for
> > riscv.
> >
> > If FTRACE_WITH_REGS isn't defined, ftrace_caller keeps ftrace_graph_call
> > so that it can be replaced with the calling of prepare_ftrace_return by
> > the enable/disable helper.
> >
> > As for defining FTRACE_WITH_REGS, ftrace_caller is adjusted to save the
> > necessary regs against the pt_regs layout, so it can reasonably call the
> > graph_ops::func function - ftrace_graph_func. And
> > ftrace_graph_[regs]_call
> > and its enable/disable helper aren't needed.
> >
> > Test log:
> >
> > The tests generated by CONFIG_FTRACE_STARTUP_TEST have passed in the
> > local
> > qemu-system-riscv64 virt machine. The following is the log during
> > startup.
> >
> > ```
> > Nov 15 03:07:13 stage4 kernel: Testing tracer function: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #1:
> > Nov 15 03:07:13 stage4 kernel: (1 0 1 0 0)
> > Nov 15 03:07:13 stage4 kernel: (1 1 2 0 0)
> > Nov 15 03:07:13 stage4 kernel: (2 1 3 0 365)
> > Nov 15 03:07:13 stage4 kernel: (2 2 4 0 399)
> > Nov 15 03:07:13 stage4 kernel: (3 2 4 0 146071)
> > Nov 15 03:07:13 stage4 kernel: (3 3 5 0 146105) PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing dynamic ftrace ops #2:
> > Nov 15 03:07:13 stage4 kernel: (1 0 1 589 0)
> > Nov 15 03:07:13 stage4 kernel: (1 1 2 635 0)
> > Nov 15 03:07:13 stage4 kernel: (2 1 3 1 2)
> > Nov 15 03:07:13 stage4 kernel: (2 2 4 125 126)
> > Nov 15 03:07:13 stage4 kernel: (3 2 4 146001 146078)
> > Nov 15 03:07:13 stage4 kernel: (3 3 5 146035 146112) PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace recursion safe: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing ftrace regs: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer nop: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer irqsoff: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup:
> > Nov 15 03:07:13 stage4 kernel: sched: DL replenish lagged too much
> > Nov 15 03:07:13 stage4 kernel: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_rt: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer wakeup_dl: PASSED
> > Nov 15 03:07:13 stage4 kernel: Testing tracer function_graph: PASSED
> > ```
> >
> > Add WITH_DIRECT_CALLS support [3] (patch 6, 7)
> > ==============================================
> >
> > This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> > SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> > here as the samples for testing DIRECT_CALLS related interface.
> >
> > First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> > register_ftrace_direct[_multi] interfaces allowing user to register
> > the customed trampoline (direct_caller) as the mcount for one or
> > more target functions. And modify_ftrace_direct[_multi] are also
> > provided for modify direct_caller.
> >
> > At the same time, the samples in ./samples/ftrace/ can be built
> > as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> > and SAMPLE_FTRACE_DIRECT_MULTI selected.
> >
> > Second, to make the direct_caller and the other ftrace hooks
> > (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> > register
> > are nominated to store the address of direct_caller in
> > ftrace_regs_caller.
> > After the setting of the address direct_caller by direct_ops->func and
> > the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> > by the `jr` inst.
> >
> > The following tests have been passed in my local qemu-riscv64 virt
> > machine.
> >
> > 1. tests with CONFIG_FTRACE_STARTUP_TEST
> > 2. tests of samples/ftrace/ftrace*.ko
> > 3. manual tests with any combination of the following hooks
> > - function/function_graph tracer
> > - ftrace*.ko
> > - kprobe/kretprobe
> >
> > For your reference, here is the log when function tracer, kretprobe and
> > ftrace-direct-too.ko co-hooks the handle_mm_fault function.
> >
> > ```
> > [root@stage4 tracing]# echo handle_mm_fault > set_ftrace_filter
> > [root@stage4 tracing]# echo 'r:myr handle_mm_fault' > kprobe_events
> > [root@stage4 tracing]# echo function > current_tracer
> > [root@stage4 tracing]# echo 1 > events/kprobes/myr/enable
> > [root@stage4 tracing]# insmod /root/ftrace-direct-too.ko
> > [root@stage4 tracing]#
> > [root@stage4 tracing]# cat trace | tail
> > cat-388 [000] ...1. 583.051438: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > cat-388 [000] ...2. 583.057930: handle_mm_fault
> > <-do_page_fault
> > cat-388 [000] ..... 583.057990: my_direct_func:
> > handle mm fault vma=000000002d9fe19c address=ffffffae9b7000 flags=215
> > cat-388 [000] ...1. 583.058284: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > tail-389 [001] ...2. 583.059062: handle_mm_fault
> > <-do_page_fault
> > tail-389 [001] ..... 583.059104: my_direct_func:
> > handle mm fault vma=0000000017f3c48e address=aaaaaabebf3000 flags=215
> > tail-389 [001] ...1. 583.059325: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > tail-389 [001] ...2. 583.060371: handle_mm_fault
> > <-do_page_fault
> > tail-389 [001] ..... 583.060410: my_direct_func:
> > handle mm fault vma=0000000017f3c48e address=aaaaaabebf1000 flags=255
> > tail-389 [001] ...1. 583.060996: myr:
> > (do_page_fault+0x16c/0x5f2 <- handle_mm_fault)
> > ```
> > Note1:
> >
> > The checkpatch.pl will output some warnings on this series, like this
> >
> > ```
> > WARNING: Prefer using '"%s...", __func__' to using 'my_direct_func2',
> > this function's name, in a string
> > 111: FILE: samples/ftrace/ftrace-direct-multi-modify.c:48:
> > +" call my_direct_func2\n"
> > ```
> >
> > The reason is that checkpatch depends on patch context providing the
> > function name. In the above warning, my_direct_func2 has some codeline
> > distance with the changed trunk, so its declaration doesn't come into
> > the patch, and then the warning jumps out.
> >
> > You may notice the location of `my_ip` variable changes in the 2nd
> > patch. I did that for reducing the warnings to some extent. But killing
> > all the warnings will makes the patch less readable, so I stopped here.
> >
> > [1] https://lpc.events/event/16/contributions/1171/
> > [2] https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> > [3] https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
> > [4] https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/
> >
> > Changes in v7:
> > - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
> > - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> > - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
> > HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >
> > Changes in v6:
> > https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
> > - Replace 8 with MCOUNT_INSN_SIZE
> > - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
> > - Add Evgenii Shatokhin comment
> >
> > Changes in v5:
> > https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
> > - Sort Kconfig entries in alphabetical order.
> >
> > Changes in v4:
> > https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
> > - Include [3] for maintenance. [Song Shuai]
> >
> > Changes in V3:
> > https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
> > - Include [2] for maintenance. [Song Shuai]
> >
> > Changes in V2:
> > https://lore.kernel.org/linux-riscv/20220921034910.3142465-1-guoren@kernel.org/
> > - Add Signed-off for preemption fixup.
> >
> > Changes in V1:
> > https://lore.kernel.org/linux-riscv/20220916103817.9490-1-guoren@kernel.org/
> >
> > Andy Chiu (1):
> > riscv: ftrace: Fixup panic by disabling preemption
> >
> > Guo Ren (2):
> > riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
> > riscv: ftrace: Reduce the detour code size to half
> >
> > Song Shuai (4):
> > riscv: ftrace: Add ftrace_graph_func
> > riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
> > samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
> > riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> >
> > arch/riscv/Kconfig | 6 +-
> > arch/riscv/Makefile | 6 +-
> > arch/riscv/include/asm/ftrace.h | 71 ++++++--
> > arch/riscv/kernel/ftrace.c | 91 ++++------
> > arch/riscv/kernel/mcount-dyn.S | 179 +++++++++++++-------
> > samples/ftrace/ftrace-direct-modify.c | 33 ++++
> > samples/ftrace/ftrace-direct-multi-modify.c | 37 ++++
> > samples/ftrace/ftrace-direct-multi.c | 22 +++
> > samples/ftrace/ftrace-direct-too.c | 26 +++
> > samples/ftrace/ftrace-direct.c | 22 +++
> > 10 files changed, 356 insertions(+), 137 deletions(-)
> >
> > --
> > 2.36.1
> >
> >
>
> I tested this series a bit on top of 6.2-rc4, with
> https://github.com/sugarfillet/linux/commit/9539a80dc6e7d1137ec7a96ebef2ab912a694bd7.patch
> added.
>
> The kernel was built with CONFIG_DYNAMIC_FTRACE_WITH_REGS=y and
> CONFIG_RISCV_ISA_C=y.
>
> Here are some results:
>
> * The kernel was built without visible issues.
>
> * Most of the Ftrace startup tests ran and passed. Certain
> event-specific tests caused soft lockups in one of the test runs but
> they are likely unrelated to this patchset and could happen without it too.
>
> * "function" and "function_graph" tracers worked in my simple use cases.
>
> * "ipmodify" functionality worked.
>
> * kprobe sample modules worked OK, which is good, because they actually
> use Ftrace: they plant the probes at the beginning of the resp. functions.
Thx for test
>
> * ftrace-direct-multi and ftrace-direct-multi-modify sample modules
> reported possibly invalid data. More info - in my reply to
> "[PATCH -next V7 6/7] samples: ftrace: Add riscv support for
> SAMPLE_FTRACE_DIRECT[_MULTI]"
Good report! We will solve it in the next version.
>
> Regards,
> Evgenii
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-04 6:40 ` Guo Ren
@ 2023-02-06 9:56 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-06 9:56 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Sat, Feb 04, 2023 at 02:40:52PM +0800, Guo Ren wrote:
> On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
> <e.shatokhin@yadro.com> wrote:
> >
> > Hi,
> >
> > On 12.01.2023 12:05, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> > >
> > > - The most horrible bug is preemption panic which found by Andy [1].
> > > Let's disable preemption for ftrace first, and Andy could continue
> > > the ftrace preemption work.
> >
> > It seems, the patches #2-#7 of this series do not require "riscv:
> > ftrace: Fixup panic by disabling preemption" and can be used without it.
> >
> > How about moving that patch out of the series and processing it separately?
> Okay.
>
> >
> > As it was pointed out in the discussion of that patch, some other
> > solution to non-atomic changes of the prologue might be needed anyway.
> I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
> still needs to be ready. Let's disable PREEMPT for ftrace first.
FWIW, taking the patch to disable FTRACE with PREEMPT for now makes sense to
me, too.
The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
queued in the arm64 tree in the for-next/ftrace branch:
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
... and those *should* be in v6.3.
Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
... and if riscv uses the CALL_OPS approach, I believe it can do much the same
there.
If riscv wants to do a single atomic patch to each patch-site (to avoid
stop_machine()), then direct calls would always needs to bounce through the
ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
of the overhead seen with direct calls is when using the list_ops or having to
do a hash lookup, and both of those are avoided with the CALL_OPS approach.
Calling directly from the patch-site is a minor optimization relative to
skipping that work.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-06 9:56 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-06 9:56 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Sat, Feb 04, 2023 at 02:40:52PM +0800, Guo Ren wrote:
> On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
> <e.shatokhin@yadro.com> wrote:
> >
> > Hi,
> >
> > On 12.01.2023 12:05, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> > >
> > > - The most horrible bug is preemption panic which found by Andy [1].
> > > Let's disable preemption for ftrace first, and Andy could continue
> > > the ftrace preemption work.
> >
> > It seems, the patches #2-#7 of this series do not require "riscv:
> > ftrace: Fixup panic by disabling preemption" and can be used without it.
> >
> > How about moving that patch out of the series and processing it separately?
> Okay.
>
> >
> > As it was pointed out in the discussion of that patch, some other
> > solution to non-atomic changes of the prologue might be needed anyway.
> I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
> still needs to be ready. Let's disable PREEMPT for ftrace first.
FWIW, taking the patch to disable FTRACE with PREEMPT for now makes sense to
me, too.
The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
queued in the arm64 tree in the for-next/ftrace branch:
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
... and those *should* be in v6.3.
Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
... and if riscv uses the CALL_OPS approach, I believe it can do much the same
there.
If riscv wants to do a single atomic patch to each patch-site (to avoid
stop_machine()), then direct calls would always needs to bounce through the
ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
of the overhead seen with direct calls is when using the list_ops or having to
do a hash lookup, and both of those are avoided with the CALL_OPS approach.
Calling directly from the patch-site is a minor optimization relative to
skipping that work.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-06 9:56 ` Mark Rutland
@ 2023-02-07 3:57 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-07 3:57 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Sat, Feb 04, 2023 at 02:40:52PM +0800, Guo Ren wrote:
> > On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> > >
> > > Hi,
> > >
> > > On 12.01.2023 12:05, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > > > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> > > >
> > > > - The most horrible bug is preemption panic which found by Andy [1].
> > > > Let's disable preemption for ftrace first, and Andy could continue
> > > > the ftrace preemption work.
> > >
> > > It seems, the patches #2-#7 of this series do not require "riscv:
> > > ftrace: Fixup panic by disabling preemption" and can be used without it.
> > >
> > > How about moving that patch out of the series and processing it separately?
> > Okay.
> >
> > >
> > > As it was pointed out in the discussion of that patch, some other
> > > solution to non-atomic changes of the prologue might be needed anyway.
> > I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
> > still needs to be ready. Let's disable PREEMPT for ftrace first.
>
> FWIW, taking the patch to disable FTRACE with PREEMPT for now makes sense to
> me, too.
Thx, you agree with that.
>
> The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> queued in the arm64 tree in the for-next/ftrace branch:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
>
> ... and those *should* be in v6.3.
Glade to hear that. Great!
>
> Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
>
> https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
Good reference. Thx for sharing.
>
> ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> there.
>
> If riscv wants to do a single atomic patch to each patch-site (to avoid
> stop_machine()), then direct calls would always needs to bounce through the
> ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> of the overhead seen with direct calls is when using the list_ops or having to
> do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> Calling directly from the patch-site is a minor optimization relative to
> skipping that work.
Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
would follow up.
The difference from arm64 is that RISC-V is 16bit/32bit mixed
instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
2048 aligned. Then:
FTRACE_UPDATE_MAKE_CALL:
* addr+00: NOP // Literal (first 32 bits)
* addr+04: NOP // Literal (last 32 bits)
* addr+08: func: auipc t0, ? // All trampolines are in the 2048
aligned place, so this point won't be changed.
* addr+12: jalr ?(t0) // For different trampolines:
ftrace_regs_caller, ftrace_caller
FTRACE_UPDATE_MAKE_NOP:
* addr+00: NOP // Literal (first 32 bits)
* addr+04: NOP // Literal (last 32 bits)
* addr+08: func: c.j // jump to addr + 16 and skip broken insn & jalr
* addr+10: xxx // last half & broken insn of auipc t0, ?
* addr+12: jalr ?(t0) // To be patched to jalr ?<t0> ()
* addr+16: func body
Right? (The call site would be increased from 64bit to 128bit ahead of func.)
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-07 3:57 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-07 3:57 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Sat, Feb 04, 2023 at 02:40:52PM +0800, Guo Ren wrote:
> > On Mon, Jan 16, 2023 at 11:02 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> > >
> > > Hi,
> > >
> > > On 12.01.2023 12:05, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> > > > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
> > > >
> > > > - The most horrible bug is preemption panic which found by Andy [1].
> > > > Let's disable preemption for ftrace first, and Andy could continue
> > > > the ftrace preemption work.
> > >
> > > It seems, the patches #2-#7 of this series do not require "riscv:
> > > ftrace: Fixup panic by disabling preemption" and can be used without it.
> > >
> > > How about moving that patch out of the series and processing it separately?
> > Okay.
> >
> > >
> > > As it was pointed out in the discussion of that patch, some other
> > > solution to non-atomic changes of the prologue might be needed anyway.
> > I think you mean Mark Rutland's DYNAMIC_FTRACE_WITH_CALL_OPS. But that
> > still needs to be ready. Let's disable PREEMPT for ftrace first.
>
> FWIW, taking the patch to disable FTRACE with PREEMPT for now makes sense to
> me, too.
Thx, you agree with that.
>
> The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> queued in the arm64 tree in the for-next/ftrace branch:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
>
> ... and those *should* be in v6.3.
Glade to hear that. Great!
>
> Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
>
> https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
Good reference. Thx for sharing.
>
> ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> there.
>
> If riscv wants to do a single atomic patch to each patch-site (to avoid
> stop_machine()), then direct calls would always needs to bounce through the
> ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> of the overhead seen with direct calls is when using the list_ops or having to
> do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> Calling directly from the patch-site is a minor optimization relative to
> skipping that work.
Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
would follow up.
The difference from arm64 is that RISC-V is 16bit/32bit mixed
instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
2048 aligned. Then:
FTRACE_UPDATE_MAKE_CALL:
* addr+00: NOP // Literal (first 32 bits)
* addr+04: NOP // Literal (last 32 bits)
* addr+08: func: auipc t0, ? // All trampolines are in the 2048
aligned place, so this point won't be changed.
* addr+12: jalr ?(t0) // For different trampolines:
ftrace_regs_caller, ftrace_caller
FTRACE_UPDATE_MAKE_NOP:
* addr+00: NOP // Literal (first 32 bits)
* addr+04: NOP // Literal (last 32 bits)
* addr+08: func: c.j // jump to addr + 16 and skip broken insn & jalr
* addr+10: xxx // last half & broken insn of auipc t0, ?
* addr+12: jalr ?(t0) // To be patched to jalr ?<t0> ()
* addr+16: func body
Right? (The call site would be increased from 64bit to 128bit ahead of func.)
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-07 3:57 ` Guo Ren
@ 2023-02-07 9:16 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-07 9:16 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Tue, Feb 07, 2023 at 11:57:06AM +0800, Guo Ren wrote:
> On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> > queued in the arm64 tree in the for-next/ftrace branch:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> > https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
> >
> > ... and those *should* be in v6.3.
> Glade to hear that. Great!
>
> >
> > Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
> >
> > https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
> Good reference. Thx for sharing.
>
> >
> > ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> > there.
> >
> > If riscv wants to do a single atomic patch to each patch-site (to avoid
> > stop_machine()), then direct calls would always needs to bounce through the
> > ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> > that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> > of the overhead seen with direct calls is when using the list_ops or having to
> > do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> > Calling directly from the patch-site is a minor optimization relative to
> > skipping that work.
> Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
> would follow up.
>
> The difference from arm64 is that RISC-V is 16bit/32bit mixed
> instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
> 2048 aligned. Then:
Where does the 2048-bit alignment requirement come from?
Note that I'm assuming you will *always* go through a common ftrace_caller
trampoline (even for direct calls), with the trampoline responsible for
recovering the direct trampoline (or ops->func) from the ops pointer.
That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
32-bit) to keep the literal naturally-aligned; the rest of the instructions
wouldn't require additional alignment.
For example, I would expect that (for 64-bit) you'd use:
# place 2 NOPs *immediately before* the function, and 3 NOPs at the start
-fpatchable-function-entry=5,2
# Align the function to 8-bytes
-falign=functions=8
... and your trampoline in each function could be initialized to:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
addr+00 func: mv t0, ra
addr+04 auipc t1, ftrace_caller
addr+08 nop
... and when enabled can be set to:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // patched to ops ptr
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 func: mv t0, ra
addr+04 auipc t1, ftrace_caller
addr+08 jalr ftrace_caller(t1)
Note: this *only* requires patching the literal and NOP<->JALR; the MV and
AUIPC aren't harmful and can always be there. This way, you won't need to use
stop_machine().
With that, the ftrace_caller trampoline can recover the `ops` pointer at a
negative offset from `ra`, and can recover the instrumented function's return
address in `t0`. Using the `ops` pointer, it can figure out whether to branch
to a direct trampoline or whether to save/restore the regs around invoking
ops->func.
For 32-bit it would be exactly the same, except you'd only need a single nop
before the function, and the offset would be -0x10.
That's what arm64 does; the only difference is that riscv would *always* need
to go via the trampoline in order to make direct calls.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-07 9:16 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-07 9:16 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Tue, Feb 07, 2023 at 11:57:06AM +0800, Guo Ren wrote:
> On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> > queued in the arm64 tree in the for-next/ftrace branch:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> > https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
> >
> > ... and those *should* be in v6.3.
> Glade to hear that. Great!
>
> >
> > Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
> >
> > https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
> Good reference. Thx for sharing.
>
> >
> > ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> > there.
> >
> > If riscv wants to do a single atomic patch to each patch-site (to avoid
> > stop_machine()), then direct calls would always needs to bounce through the
> > ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> > that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> > of the overhead seen with direct calls is when using the list_ops or having to
> > do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> > Calling directly from the patch-site is a minor optimization relative to
> > skipping that work.
> Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
> would follow up.
>
> The difference from arm64 is that RISC-V is 16bit/32bit mixed
> instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
> 2048 aligned. Then:
Where does the 2048-bit alignment requirement come from?
Note that I'm assuming you will *always* go through a common ftrace_caller
trampoline (even for direct calls), with the trampoline responsible for
recovering the direct trampoline (or ops->func) from the ops pointer.
That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
32-bit) to keep the literal naturally-aligned; the rest of the instructions
wouldn't require additional alignment.
For example, I would expect that (for 64-bit) you'd use:
# place 2 NOPs *immediately before* the function, and 3 NOPs at the start
-fpatchable-function-entry=5,2
# Align the function to 8-bytes
-falign=functions=8
... and your trampoline in each function could be initialized to:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
addr+00 func: mv t0, ra
addr+04 auipc t1, ftrace_caller
addr+08 nop
... and when enabled can be set to:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // patched to ops ptr
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 func: mv t0, ra
addr+04 auipc t1, ftrace_caller
addr+08 jalr ftrace_caller(t1)
Note: this *only* requires patching the literal and NOP<->JALR; the MV and
AUIPC aren't harmful and can always be there. This way, you won't need to use
stop_machine().
With that, the ftrace_caller trampoline can recover the `ops` pointer at a
negative offset from `ra`, and can recover the instrumented function's return
address in `t0`. Using the `ops` pointer, it can figure out whether to branch
to a direct trampoline or whether to save/restore the regs around invoking
ops->func.
For 32-bit it would be exactly the same, except you'd only need a single nop
before the function, and the offset would be -0x10.
That's what arm64 does; the only difference is that riscv would *always* need
to go via the trampoline in order to make direct calls.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-07 9:16 ` Mark Rutland
@ 2023-02-08 2:30 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-08 2:30 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
Hi Mark,
Thx for the thoughtful reply.
On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Tue, Feb 07, 2023 at 11:57:06AM +0800, Guo Ren wrote:
> > On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > > The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> > > queued in the arm64 tree in the for-next/ftrace branch:
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> > > https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
> > >
> > > ... and those *should* be in v6.3.
> > Glade to hear that. Great!
> >
> > >
> > > Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
> > >
> > > https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
> > Good reference. Thx for sharing.
> >
> > >
> > > ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> > > there.
> > >
> > > If riscv wants to do a single atomic patch to each patch-site (to avoid
> > > stop_machine()), then direct calls would always needs to bounce through the
> > > ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> > > that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> > > of the overhead seen with direct calls is when using the list_ops or having to
> > > do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> > > Calling directly from the patch-site is a minor optimization relative to
> > > skipping that work.
> > Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
> > would follow up.
> >
> > The difference from arm64 is that RISC-V is 16bit/32bit mixed
> > instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
> > 2048 aligned. Then:
>
> Where does the 2048-bit alignment requirement come from?
Sorry for the typo. It's one 2048 bytes for keeping two trampolines
(ftrace_caller & ftrace_regs_caller) in one aligned part.
Because the jalr has only +-2048 bytes offset range.
Then the "auipc t1, ftrace(_regs)_caller" is fixed.
>
> Note that I'm assuming you will *always* go through a common ftrace_caller
> trampoline (even for direct calls), with the trampoline responsible for
> recovering the direct trampoline (or ops->func) from the ops pointer.
>
> That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> wouldn't require additional alignment.
>
> For example, I would expect that (for 64-bit) you'd use:
>
> # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> -fpatchable-function-entry=5,2
>
> # Align the function to 8-bytes
> -falign=functions=8
>
> ... and your trampoline in each function could be initialized to:
>
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> addr+00 func: mv t0, ra
> addr+04 auipc t1, ftrace_caller
> addr+08 nop
>
> ... and when enabled can be set to:
>
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 func: mv t0, ra
We needn't "mv t0, ra" here because our "jalr" could work with t0 and
won't affect ra. Let's do it in the trampoline code, and then we can
save another word here.
> addr+04 auipc t1, ftrace_caller
> addr+08 jalr ftrace_caller(t1)
Here is the call-site:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // patched to ops ptr
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 auipc t0, ftrace_caller
addr+04 jalr ftrace_caller(t0)
>
> Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> AUIPC aren't harmful and can always be there. This way, you won't need to use
> stop_machine().
Yes, simplest nop is better than c.j. I confused.
>
> With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> negative offset from `ra`, and can recover the instrumented function's return
> address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> to a direct trampoline or whether to save/restore the regs around invoking
> ops->func.
>
> For 32-bit it would be exactly the same, except you'd only need a single nop
> before the function, and the offset would be -0x10.
Yes, we reduced another 4 bytes & a smaller alignment for better code
size when 32-bit.
# Note: aligned to 4 bytes
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 auipc t0, ftrace_caller
addr+04 jalr ftrace_caller(t0)
>
> That's what arm64 does; the only difference is that riscv would *always* need
> to go via the trampoline in order to make direct calls.
We need one more trampoline here beside ftrace_caller &
ftrace_regs_caller: It's "direct_caller".
addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-08 2:30 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-08 2:30 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
Hi Mark,
Thx for the thoughtful reply.
On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Tue, Feb 07, 2023 at 11:57:06AM +0800, Guo Ren wrote:
> > On Mon, Feb 6, 2023 at 5:56 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > > The DYNAMIC_FTRACE_WITH_CALL_OPS patches should be in v6.3. They're currently
> > > queued in the arm64 tree in the for-next/ftrace branch:
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/ftrace
> > > https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/
> > >
> > > ... and those *should* be in v6.3.
> > Glade to hear that. Great!
> >
> > >
> > > Patches to imeplement DIRECT_CALLS atop that are in review at the moment:
> > >
> > > https://lore.kernel.org/linux-arm-kernel/20230201163420.1579014-1-revest@chromium.org/
> > Good reference. Thx for sharing.
> >
> > >
> > > ... and if riscv uses the CALL_OPS approach, I believe it can do much the same
> > > there.
> > >
> > > If riscv wants to do a single atomic patch to each patch-site (to avoid
> > > stop_machine()), then direct calls would always needs to bounce through the
> > > ftrace_caller trampoline (and acquire the direct call from the ftrace_ops), but
> > > that might not be as bad as it sounds -- from benchmarking on arm64, the bulk
> > > of the overhead seen with direct calls is when using the list_ops or having to
> > > do a hash lookup, and both of those are avoided with the CALL_OPS approach.
> > > Calling directly from the patch-site is a minor optimization relative to
> > > skipping that work.
> > Yes, CALL_OPS could solve the PREEMPTION & stop_machine problems. I
> > would follow up.
> >
> > The difference from arm64 is that RISC-V is 16bit/32bit mixed
> > instruction ISA, so we must keep ftrace_caller & ftrace_regs_caller in
> > 2048 aligned. Then:
>
> Where does the 2048-bit alignment requirement come from?
Sorry for the typo. It's one 2048 bytes for keeping two trampolines
(ftrace_caller & ftrace_regs_caller) in one aligned part.
Because the jalr has only +-2048 bytes offset range.
Then the "auipc t1, ftrace(_regs)_caller" is fixed.
>
> Note that I'm assuming you will *always* go through a common ftrace_caller
> trampoline (even for direct calls), with the trampoline responsible for
> recovering the direct trampoline (or ops->func) from the ops pointer.
>
> That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> wouldn't require additional alignment.
>
> For example, I would expect that (for 64-bit) you'd use:
>
> # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> -fpatchable-function-entry=5,2
>
> # Align the function to 8-bytes
> -falign=functions=8
>
> ... and your trampoline in each function could be initialized to:
>
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> addr+00 func: mv t0, ra
> addr+04 auipc t1, ftrace_caller
> addr+08 nop
>
> ... and when enabled can be set to:
>
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 func: mv t0, ra
We needn't "mv t0, ra" here because our "jalr" could work with t0 and
won't affect ra. Let's do it in the trampoline code, and then we can
save another word here.
> addr+04 auipc t1, ftrace_caller
> addr+08 jalr ftrace_caller(t1)
Here is the call-site:
# Note: aligned to 8 bytes
addr-08 // Literal (first 32-bits) // patched to ops ptr
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 auipc t0, ftrace_caller
addr+04 jalr ftrace_caller(t0)
>
> Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> AUIPC aren't harmful and can always be there. This way, you won't need to use
> stop_machine().
Yes, simplest nop is better than c.j. I confused.
>
> With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> negative offset from `ra`, and can recover the instrumented function's return
> address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> to a direct trampoline or whether to save/restore the regs around invoking
> ops->func.
>
> For 32-bit it would be exactly the same, except you'd only need a single nop
> before the function, and the offset would be -0x10.
Yes, we reduced another 4 bytes & a smaller alignment for better code
size when 32-bit.
# Note: aligned to 4 bytes
addr-04 // Literal (last 32-bits) // patched to ops ptr
addr+00 auipc t0, ftrace_caller
addr+04 jalr ftrace_caller(t0)
>
> That's what arm64 does; the only difference is that riscv would *always* need
> to go via the trampoline in order to make direct calls.
We need one more trampoline here beside ftrace_caller &
ftrace_regs_caller: It's "direct_caller".
addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-08 2:30 ` Guo Ren
@ 2023-02-08 14:46 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-08 14:46 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Wed, Feb 08, 2023 at 10:30:56AM +0800, Guo Ren wrote:
> Hi Mark,
>
> Thx for the thoughtful reply.
>
> On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > Note that I'm assuming you will *always* go through a common ftrace_caller
> > trampoline (even for direct calls), with the trampoline responsible for
> > recovering the direct trampoline (or ops->func) from the ops pointer.
> >
> > That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> > 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> > wouldn't require additional alignment.
> >
> > For example, I would expect that (for 64-bit) you'd use:
> >
> > # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> > -fpatchable-function-entry=5,2
> >
> > # Align the function to 8-bytes
> > -falign=functions=8
> >
> > ... and your trampoline in each function could be initialized to:
> >
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> > addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> > addr+00 func: mv t0, ra
> > addr+04 auipc t1, ftrace_caller
> > addr+08 nop
> >
> > ... and when enabled can be set to:
> >
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 func: mv t0, ra
> We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> won't affect ra. Let's do it in the trampoline code, and then we can
> save another word here.
Ah; I thought JALR always clobbered ra? Or can that specify the register to
save the link address to?
I'm not that familiar with riscv asm, so I've probably just got that wrong.
> > addr+04 auipc t1, ftrace_caller
> > addr+08 jalr ftrace_caller(t1)
>
> Here is the call-site:
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
generating additional bits that the AUPIC didn't -- have I got that wrong?
What specifies which register the JALR will write the link address to?
> > Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> > AUIPC aren't harmful and can always be there. This way, you won't need to use
> > stop_machine().
> Yes, simplest nop is better than c.j. I confused.
>
> >
> > With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> > negative offset from `ra`, and can recover the instrumented function's return
> > address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> > to a direct trampoline or whether to save/restore the regs around invoking
> > ops->func.
> >
> > For 32-bit it would be exactly the same, except you'd only need a single nop
> > before the function, and the offset would be -0x10.
> Yes, we reduced another 4 bytes & a smaller alignment for better code
> size when 32-bit.
> # Note: aligned to 4 bytes
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
> >
> > That's what arm64 does; the only difference is that riscv would *always* need
> > to go via the trampoline in order to make direct calls.
> We need one more trampoline here beside ftrace_caller &
> ftrace_regs_caller: It's "direct_caller".
>
> addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
I'd strongly recommend that you instead implement FTRACE_WITH_ARGS and
deprecate FTRACE_WITH_REGS, like arm64 has done, then you only need a single
ftrace_caller, as I mentioned above. That way there's no risk that you need to
patch the AUIPC after initialization.
The arm64 FTRACE_WITH_ARGS conversion is in mainline, and arm64's
FTRACE_WITH_CALL_OPS is based upon that. Florent's DIRECT_CALLS patches add the
direct call logic to the same ftrace_caller trampoline.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-08 14:46 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-08 14:46 UTC (permalink / raw)
To: Guo Ren
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Wed, Feb 08, 2023 at 10:30:56AM +0800, Guo Ren wrote:
> Hi Mark,
>
> Thx for the thoughtful reply.
>
> On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > Note that I'm assuming you will *always* go through a common ftrace_caller
> > trampoline (even for direct calls), with the trampoline responsible for
> > recovering the direct trampoline (or ops->func) from the ops pointer.
> >
> > That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> > 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> > wouldn't require additional alignment.
> >
> > For example, I would expect that (for 64-bit) you'd use:
> >
> > # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> > -fpatchable-function-entry=5,2
> >
> > # Align the function to 8-bytes
> > -falign=functions=8
> >
> > ... and your trampoline in each function could be initialized to:
> >
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> > addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> > addr+00 func: mv t0, ra
> > addr+04 auipc t1, ftrace_caller
> > addr+08 nop
> >
> > ... and when enabled can be set to:
> >
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 func: mv t0, ra
> We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> won't affect ra. Let's do it in the trampoline code, and then we can
> save another word here.
Ah; I thought JALR always clobbered ra? Or can that specify the register to
save the link address to?
I'm not that familiar with riscv asm, so I've probably just got that wrong.
> > addr+04 auipc t1, ftrace_caller
> > addr+08 jalr ftrace_caller(t1)
>
> Here is the call-site:
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
generating additional bits that the AUPIC didn't -- have I got that wrong?
What specifies which register the JALR will write the link address to?
> > Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> > AUIPC aren't harmful and can always be there. This way, you won't need to use
> > stop_machine().
> Yes, simplest nop is better than c.j. I confused.
>
> >
> > With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> > negative offset from `ra`, and can recover the instrumented function's return
> > address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> > to a direct trampoline or whether to save/restore the regs around invoking
> > ops->func.
> >
> > For 32-bit it would be exactly the same, except you'd only need a single nop
> > before the function, and the offset would be -0x10.
> Yes, we reduced another 4 bytes & a smaller alignment for better code
> size when 32-bit.
> # Note: aligned to 4 bytes
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
> >
> > That's what arm64 does; the only difference is that riscv would *always* need
> > to go via the trampoline in order to make direct calls.
> We need one more trampoline here beside ftrace_caller &
> ftrace_regs_caller: It's "direct_caller".
>
> addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
I'd strongly recommend that you instead implement FTRACE_WITH_ARGS and
deprecate FTRACE_WITH_REGS, like arm64 has done, then you only need a single
ftrace_caller, as I mentioned above. That way there's no risk that you need to
patch the AUIPC after initialization.
The arm64 FTRACE_WITH_ARGS conversion is in mainline, and arm64's
FTRACE_WITH_CALL_OPS is based upon that. Florent's DIRECT_CALLS patches add the
direct call logic to the same ftrace_caller trampoline.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-08 2:30 ` Guo Ren
@ 2023-02-08 22:29 ` David Laight
-1 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-08 22:29 UTC (permalink / raw)
To: 'Guo Ren', Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 func: mv t0, ra
> We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> won't affect ra. Let's do it in the trampoline code, and then we can
> save another word here.
> > addr+04 auipc t1, ftrace_caller
> > addr+08 jalr ftrace_caller(t1)
Is that some kind of 'load high' and 'add offset' pair?
I guess 64bit kernels guarantee to put all module code
within +-2G of the main kernel?
> Here is the call-site:
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
Could you even do something like:
addr-n call ftrace-function
addr-n+x literals
addr+0 nop or jmp addr-n
addr+4 function_code
So that all the code executed when tracing is enabled
is before the label and only one 'nop' is in the body.
The called code can use the return address to find the
literals and then modify it to return to addr+4.
The code cost when trace is enabled is probably irrelevant
here - dominated by what happens later.
It probably isn't even worth aligning a 64bit constant.
Doing two reads probably won't be noticable.
What you do want to ensure is that the initial patch is
overwriting nop - just in case the gap isn't there.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-08 22:29 ` David Laight
0 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-08 22:29 UTC (permalink / raw)
To: 'Guo Ren', Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 func: mv t0, ra
> We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> won't affect ra. Let's do it in the trampoline code, and then we can
> save another word here.
> > addr+04 auipc t1, ftrace_caller
> > addr+08 jalr ftrace_caller(t1)
Is that some kind of 'load high' and 'add offset' pair?
I guess 64bit kernels guarantee to put all module code
within +-2G of the main kernel?
> Here is the call-site:
> # Note: aligned to 8 bytes
> addr-08 // Literal (first 32-bits) // patched to ops ptr
> addr-04 // Literal (last 32-bits) // patched to ops ptr
> addr+00 auipc t0, ftrace_caller
> addr+04 jalr ftrace_caller(t0)
Could you even do something like:
addr-n call ftrace-function
addr-n+x literals
addr+0 nop or jmp addr-n
addr+4 function_code
So that all the code executed when tracing is enabled
is before the label and only one 'nop' is in the body.
The called code can use the return address to find the
literals and then modify it to return to addr+4.
The code cost when trace is enabled is probably irrelevant
here - dominated by what happens later.
It probably isn't even worth aligning a 64bit constant.
Doing two reads probably won't be noticable.
What you do want to ensure is that the initial patch is
overwriting nop - just in case the gap isn't there.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-08 14:46 ` Mark Rutland
@ 2023-02-09 1:31 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:31 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Wed, Feb 8, 2023 at 10:46 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Wed, Feb 08, 2023 at 10:30:56AM +0800, Guo Ren wrote:
> > Hi Mark,
> >
> > Thx for the thoughtful reply.
> >
> > On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > > Note that I'm assuming you will *always* go through a common ftrace_caller
> > > trampoline (even for direct calls), with the trampoline responsible for
> > > recovering the direct trampoline (or ops->func) from the ops pointer.
> > >
> > > That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> > > 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> > > wouldn't require additional alignment.
> > >
> > > For example, I would expect that (for 64-bit) you'd use:
> > >
> > > # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> > > -fpatchable-function-entry=5,2
> > >
> > > # Align the function to 8-bytes
> > > -falign=functions=8
> > >
> > > ... and your trampoline in each function could be initialized to:
> > >
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> > > addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> > > addr+00 func: mv t0, ra
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 nop
> > >
> > > ... and when enabled can be set to:
> > >
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 func: mv t0, ra
> > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > won't affect ra. Let's do it in the trampoline code, and then we can
> > save another word here.
>
> Ah; I thought JALR always clobbered ra? Or can that specify the register to
> save the link address to?
Yes, that's the feature of riscv :) We could use any register to save
the link address.
>
> I'm not that familiar with riscv asm, so I've probably just got that wrong.
>
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 jalr ftrace_caller(t1)
> >
> > Here is the call-site:
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
Sorry, it should be:
addr+04 jalr t0, ftrace_caller(t0)
>
> I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> generating additional bits that the AUPIC didn't -- have I got that wrong?
>
> What specifies which register the JALR will write the link address to?
According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
stored in t0 + 0x0 and stores the return address to t0.
That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
>
> > > Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> > > AUIPC aren't harmful and can always be there. This way, you won't need to use
> > > stop_machine().
> > Yes, simplest nop is better than c.j. I confused.
> >
> > >
> > > With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> > > negative offset from `ra`, and can recover the instrumented function's return
> > > address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> > > to a direct trampoline or whether to save/restore the regs around invoking
> > > ops->func.
> > >
> > > For 32-bit it would be exactly the same, except you'd only need a single nop
> > > before the function, and the offset would be -0x10.
> > Yes, we reduced another 4 bytes & a smaller alignment for better code
> > size when 32-bit.
> > # Note: aligned to 4 bytes
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
addr+04 jalr t0, ftrace_caller(t0)
> > >
>
> > > That's what arm64 does; the only difference is that riscv would *always* need
> > > to go via the trampoline in order to make direct calls.
> > We need one more trampoline here beside ftrace_caller &
> > ftrace_regs_caller: It's "direct_caller".
> >
> > addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
>
> I'd strongly recommend that you instead implement FTRACE_WITH_ARGS and
> deprecate FTRACE_WITH_REGS, like arm64 has done, then you only need a single
> ftrace_caller, as I mentioned above. That way there's no risk that you need to
> patch the AUIPC after initialization.
>
> The arm64 FTRACE_WITH_ARGS conversion is in mainline, and arm64's
> FTRACE_WITH_CALL_OPS is based upon that. Florent's DIRECT_CALLS patches add the
> direct call logic to the same ftrace_caller trampoline.
Thx for the suggestion of only keeping the ftrace_caller idea, but
it's another topic.
What I want to point out:
If we keep "auipc (addr+00)" fixed, we could use the different
trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
aligned area).
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 1:31 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:31 UTC (permalink / raw)
To: Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
On Wed, Feb 8, 2023 at 10:46 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Wed, Feb 08, 2023 at 10:30:56AM +0800, Guo Ren wrote:
> > Hi Mark,
> >
> > Thx for the thoughtful reply.
> >
> > On Tue, Feb 7, 2023 at 5:17 PM Mark Rutland <mark.rutland@arm.com> wrote:
> > > Note that I'm assuming you will *always* go through a common ftrace_caller
> > > trampoline (even for direct calls), with the trampoline responsible for
> > > recovering the direct trampoline (or ops->func) from the ops pointer.
> > >
> > > That would only require 64-bit alignment on 64-bit (or 32-bit alignment on
> > > 32-bit) to keep the literal naturally-aligned; the rest of the instructions
> > > wouldn't require additional alignment.
> > >
> > > For example, I would expect that (for 64-bit) you'd use:
> > >
> > > # place 2 NOPs *immediately before* the function, and 3 NOPs at the start
> > > -fpatchable-function-entry=5,2
> > >
> > > # Align the function to 8-bytes
> > > -falign=functions=8
> > >
> > > ... and your trampoline in each function could be initialized to:
> > >
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // set to ftrace_nop_ops
> > > addr-04 // Literal (last 32-bits) // set to ftrace_nop_ops
> > > addr+00 func: mv t0, ra
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 nop
> > >
> > > ... and when enabled can be set to:
> > >
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 func: mv t0, ra
> > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > won't affect ra. Let's do it in the trampoline code, and then we can
> > save another word here.
>
> Ah; I thought JALR always clobbered ra? Or can that specify the register to
> save the link address to?
Yes, that's the feature of riscv :) We could use any register to save
the link address.
>
> I'm not that familiar with riscv asm, so I've probably just got that wrong.
>
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 jalr ftrace_caller(t1)
> >
> > Here is the call-site:
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
Sorry, it should be:
addr+04 jalr t0, ftrace_caller(t0)
>
> I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> generating additional bits that the AUPIC didn't -- have I got that wrong?
>
> What specifies which register the JALR will write the link address to?
According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
stored in t0 + 0x0 and stores the return address to t0.
That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
>
> > > Note: this *only* requires patching the literal and NOP<->JALR; the MV and
> > > AUIPC aren't harmful and can always be there. This way, you won't need to use
> > > stop_machine().
> > Yes, simplest nop is better than c.j. I confused.
> >
> > >
> > > With that, the ftrace_caller trampoline can recover the `ops` pointer at a
> > > negative offset from `ra`, and can recover the instrumented function's return
> > > address in `t0`. Using the `ops` pointer, it can figure out whether to branch
> > > to a direct trampoline or whether to save/restore the regs around invoking
> > > ops->func.
> > >
> > > For 32-bit it would be exactly the same, except you'd only need a single nop
> > > before the function, and the offset would be -0x10.
> > Yes, we reduced another 4 bytes & a smaller alignment for better code
> > size when 32-bit.
> > # Note: aligned to 4 bytes
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
addr+04 jalr t0, ftrace_caller(t0)
> > >
>
> > > That's what arm64 does; the only difference is that riscv would *always* need
> > > to go via the trampoline in order to make direct calls.
> > We need one more trampoline here beside ftrace_caller &
> > ftrace_regs_caller: It's "direct_caller".
> >
> > addr+04 nop -> direct_caller/ftrace_caller/ftrace_regs_caller
>
> I'd strongly recommend that you instead implement FTRACE_WITH_ARGS and
> deprecate FTRACE_WITH_REGS, like arm64 has done, then you only need a single
> ftrace_caller, as I mentioned above. That way there's no risk that you need to
> patch the AUIPC after initialization.
>
> The arm64 FTRACE_WITH_ARGS conversion is in mainline, and arm64's
> FTRACE_WITH_CALL_OPS is based upon that. Florent's DIRECT_CALLS patches add the
> direct call logic to the same ftrace_caller trampoline.
Thx for the suggestion of only keeping the ftrace_caller idea, but
it's another topic.
What I want to point out:
If we keep "auipc (addr+00)" fixed, we could use the different
trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
aligned area).
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-08 22:29 ` David Laight
@ 2023-02-09 1:51 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:51 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
>
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 func: mv t0, ra
> > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > won't affect ra. Let's do it in the trampoline code, and then we can
> > save another word here.
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 jalr ftrace_caller(t1)
>
> Is that some kind of 'load high' and 'add offset' pair?
Yes.
> I guess 64bit kernels guarantee to put all module code
> within +-2G of the main kernel?
Yes, 32-bit is enough. So we only need one 32-bit literal size for the
current rv64, just like CONFIG_32BIT.
>
> > Here is the call-site:
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
>
> Could you even do something like:
> addr-n call ftrace-function
> addr-n+x literals
> addr+0 nop or jmp addr-n
> addr+4 function_code
Yours cost one more instruction, right?
addr-12 auipc
addr-8 jalr
addr-4 // Literal (32-bits)
addr+0 nop or jmp addr-n // one more?
addr+4 function_code
> So that all the code executed when tracing is enabled
> is before the label and only one 'nop' is in the body.
> The called code can use the return address to find the
> literals and then modify it to return to addr+4.
> The code cost when trace is enabled is probably irrelevant
> here - dominated by what happens later.
> It probably isn't even worth aligning a 64bit constant.
> Doing two reads probably won't be noticable.
>
> What you do want to ensure is that the initial patch is
> overwriting nop - just in case the gap isn't there.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 1:51 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:51 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
>
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 func: mv t0, ra
> > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > won't affect ra. Let's do it in the trampoline code, and then we can
> > save another word here.
> > > addr+04 auipc t1, ftrace_caller
> > > addr+08 jalr ftrace_caller(t1)
>
> Is that some kind of 'load high' and 'add offset' pair?
Yes.
> I guess 64bit kernels guarantee to put all module code
> within +-2G of the main kernel?
Yes, 32-bit is enough. So we only need one 32-bit literal size for the
current rv64, just like CONFIG_32BIT.
>
> > Here is the call-site:
> > # Note: aligned to 8 bytes
> > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > addr+00 auipc t0, ftrace_caller
> > addr+04 jalr ftrace_caller(t0)
>
> Could you even do something like:
> addr-n call ftrace-function
> addr-n+x literals
> addr+0 nop or jmp addr-n
> addr+4 function_code
Yours cost one more instruction, right?
addr-12 auipc
addr-8 jalr
addr-4 // Literal (32-bits)
addr+0 nop or jmp addr-n // one more?
addr+4 function_code
> So that all the code executed when tracing is enabled
> is before the label and only one 'nop' is in the body.
> The called code can use the return address to find the
> literals and then modify it to return to addr+4.
> The code cost when trace is enabled is probably irrelevant
> here - dominated by what happens later.
> It probably isn't even worth aligning a 64bit constant.
> Doing two reads probably won't be noticable.
>
> What you do want to ensure is that the initial patch is
> overwriting nop - just in case the gap isn't there.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 1:51 ` Guo Ren
@ 2023-02-09 1:59 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:59 UTC (permalink / raw)
To: Mark Rutland
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
>
> On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> >
> > > > # Note: aligned to 8 bytes
> > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > addr+00 func: mv t0, ra
> > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > save another word here.
> > > > addr+04 auipc t1, ftrace_caller
> > > > addr+08 jalr ftrace_caller(t1)
> >
> > Is that some kind of 'load high' and 'add offset' pair?
> Yes.
>
> > I guess 64bit kernels guarantee to put all module code
> > within +-2G of the main kernel?
> Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> current rv64, just like CONFIG_32BIT.
We need kernel_addr_base + this 32-bit Literal.
@Mark Rutland
What do you think the idea about reducing one more 32-bit in
call-site? (It also sould work for arm64.)
>
> >
> > > Here is the call-site:
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 auipc t0, ftrace_caller
> > > addr+04 jalr ftrace_caller(t0)
> >
> > Could you even do something like:
> > addr-n call ftrace-function
> > addr-n+x literals
> > addr+0 nop or jmp addr-n
> > addr+4 function_code
> Yours cost one more instruction, right?
> addr-12 auipc
> addr-8 jalr
> addr-4 // Literal (32-bits)
> addr+0 nop or jmp addr-n // one more?
> addr+4 function_code
>
> > So that all the code executed when tracing is enabled
> > is before the label and only one 'nop' is in the body.
> > The called code can use the return address to find the
> > literals and then modify it to return to addr+4.
> > The code cost when trace is enabled is probably irrelevant
> > here - dominated by what happens later.
> > It probably isn't even worth aligning a 64bit constant.
> > Doing two reads probably won't be noticable.
> >
> > What you do want to ensure is that the initial patch is
> > overwriting nop - just in case the gap isn't there.
> >
> > David
> >
> > -
> > Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> > Registration No: 1397386 (Wales)
>
>
>
> --
> Best Regards
> Guo Ren
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 1:59 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 1:59 UTC (permalink / raw)
To: Mark Rutland
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
>
> On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> >
> > > > # Note: aligned to 8 bytes
> > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > addr+00 func: mv t0, ra
> > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > save another word here.
> > > > addr+04 auipc t1, ftrace_caller
> > > > addr+08 jalr ftrace_caller(t1)
> >
> > Is that some kind of 'load high' and 'add offset' pair?
> Yes.
>
> > I guess 64bit kernels guarantee to put all module code
> > within +-2G of the main kernel?
> Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> current rv64, just like CONFIG_32BIT.
We need kernel_addr_base + this 32-bit Literal.
@Mark Rutland
What do you think the idea about reducing one more 32-bit in
call-site? (It also sould work for arm64.)
>
> >
> > > Here is the call-site:
> > > # Note: aligned to 8 bytes
> > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > addr+00 auipc t0, ftrace_caller
> > > addr+04 jalr ftrace_caller(t0)
> >
> > Could you even do something like:
> > addr-n call ftrace-function
> > addr-n+x literals
> > addr+0 nop or jmp addr-n
> > addr+4 function_code
> Yours cost one more instruction, right?
> addr-12 auipc
> addr-8 jalr
> addr-4 // Literal (32-bits)
> addr+0 nop or jmp addr-n // one more?
> addr+4 function_code
>
> > So that all the code executed when tracing is enabled
> > is before the label and only one 'nop' is in the body.
> > The called code can use the return address to find the
> > literals and then modify it to return to addr+4.
> > The code cost when trace is enabled is probably irrelevant
> > here - dominated by what happens later.
> > It probably isn't even worth aligning a 64bit constant.
> > Doing two reads probably won't be noticable.
> >
> > What you do want to ensure is that the initial patch is
> > overwriting nop - just in case the gap isn't there.
> >
> > David
> >
> > -
> > Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> > Registration No: 1397386 (Wales)
>
>
>
> --
> Best Regards
> Guo Ren
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 1:51 ` Guo Ren
@ 2023-02-09 9:00 ` David Laight
-1 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-09 9:00 UTC (permalink / raw)
To: 'Guo Ren'
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
From: Guo Ren
> Sent: 09 February 2023 01:51
...
> Yours cost one more instruction, right?
> addr-12 auipc
> addr-8 jalr
> addr-4 // Literal (32-bits)
> addr+0 nop or jmp addr-n // one more?
> addr+4 function_code
Yes, it is 4 bytes larger but there is one less
instruction executed (only one nop) when ftrace is disabled.
That probably matters more than anything in the ftrace
'prologue' code.
I also suspect that you can use a 32bit integer as
a table index in 64bit mode to save a word there.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 9:00 ` David Laight
0 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-09 9:00 UTC (permalink / raw)
To: 'Guo Ren'
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
From: Guo Ren
> Sent: 09 February 2023 01:51
...
> Yours cost one more instruction, right?
> addr-12 auipc
> addr-8 jalr
> addr-4 // Literal (32-bits)
> addr+0 nop or jmp addr-n // one more?
> addr+4 function_code
Yes, it is 4 bytes larger but there is one less
instruction executed (only one nop) when ftrace is disabled.
That probably matters more than anything in the ftrace
'prologue' code.
I also suspect that you can use a 32bit integer as
a table index in 64bit mode to save a word there.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 9:00 ` David Laight
@ 2023-02-09 9:11 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 9:11 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 5:00 PM David Laight <David.Laight@aculab.com> wrote:
>
> From: Guo Ren
> > Sent: 09 February 2023 01:51
> ...
> > Yours cost one more instruction, right?
> > addr-12 auipc
> > addr-8 jalr
> > addr-4 // Literal (32-bits)
> > addr+0 nop or jmp addr-n // one more?
> > addr+4 function_code
>
> Yes, it is 4 bytes larger but there is one less
> instruction executed (only one nop) when ftrace is disabled.
> That probably matters more than anything in the ftrace
> 'prologue' code.
I've got your point, thx. I would consider your advice and make the tradeoff.
>
> I also suspect that you can use a 32bit integer as
> a table index in 64bit mode to save a word there.
Yes, good idea. I've asked Mark Rutland. Let's wait for his feedback.
It also helps arm64.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 9:11 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-09 9:11 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 5:00 PM David Laight <David.Laight@aculab.com> wrote:
>
> From: Guo Ren
> > Sent: 09 February 2023 01:51
> ...
> > Yours cost one more instruction, right?
> > addr-12 auipc
> > addr-8 jalr
> > addr-4 // Literal (32-bits)
> > addr+0 nop or jmp addr-n // one more?
> > addr+4 function_code
>
> Yes, it is 4 bytes larger but there is one less
> instruction executed (only one nop) when ftrace is disabled.
> That probably matters more than anything in the ftrace
> 'prologue' code.
I've got your point, thx. I would consider your advice and make the tradeoff.
>
> I also suspect that you can use a 32bit integer as
> a table index in 64bit mode to save a word there.
Yes, good idea. I've asked Mark Rutland. Let's wait for his feedback.
It also helps arm64.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 1:59 ` Guo Ren
@ 2023-02-09 9:54 ` Mark Rutland
-1 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-09 9:54 UTC (permalink / raw)
To: Guo Ren
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 09, 2023 at 09:59:33AM +0800, Guo Ren wrote:
> On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
> >
> > On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> > >
> > > > > # Note: aligned to 8 bytes
> > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > addr+00 func: mv t0, ra
> > > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > > save another word here.
> > > > > addr+04 auipc t1, ftrace_caller
> > > > > addr+08 jalr ftrace_caller(t1)
> > >
> > > Is that some kind of 'load high' and 'add offset' pair?
> > Yes.
> >
> > > I guess 64bit kernels guarantee to put all module code
> > > within +-2G of the main kernel?
> > Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> > current rv64, just like CONFIG_32BIT.
> We need kernel_addr_base + this 32-bit Literal.
>
> @Mark Rutland
> What do you think the idea about reducing one more 32-bit in
> call-site? (It also sould work for arm64.)
The literal pointer is for a struct ftrace_ops, which is data, not code.
An ftrace_ops can be allocated from anywhere (e.g. core kernel data, module
data, linear map, vmalloc space), and so is not guaranteed to be within 2GiB of
all code. The literal needs to be able to address the entire kernel addresss
range, and since it can be modified concurrently (with PREEMPT and not using
stop_machine()) it needs to be possible to read/write atomically. So
practically speaking it needs to be the native pointer size (i.e. 64-bit on a
64-bit kernel).
Other schemes for compressing that (e.g. using an integer into an array of
pointers) is possible, but uses more memory and gets more complicated for
concurrent manipulation, so I would strongly recommend keeping this simple and
using a native pointer size here.
> > > > Here is the call-site:
> > > > # Note: aligned to 8 bytes
> > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > addr+00 auipc t0, ftrace_caller
> > > > addr+04 jalr ftrace_caller(t0)
> > >
> > > Could you even do something like:
> > > addr-n call ftrace-function
> > > addr-n+x literals
> > > addr+0 nop or jmp addr-n
> > > addr+4 function_code
> > Yours cost one more instruction, right?
> > addr-12 auipc
> > addr-8 jalr
> > addr-4 // Literal (32-bits)
> > addr+0 nop or jmp addr-n // one more?
> > addr+4 function_code
Placing instructions before the entry point is going to confuse symbol
resolution and unwind code, so I would not recommend that. It also means the
trampoline will need to re-adjust the return address back into the function,
but that is relatively simple.
I also think that this is micro-optimizing. The AUPIC *should* be cheap, so
executing that unconditionally should be fine. I think the form that Guo
suggested with AUIPC + {JALR || NOP} in the function (and 64-bits reserved
immediately bfore the function) is the way to go, so long as that does the
right thing with ra.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 9:54 ` Mark Rutland
0 siblings, 0 replies; 88+ messages in thread
From: Mark Rutland @ 2023-02-09 9:54 UTC (permalink / raw)
To: Guo Ren
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 09, 2023 at 09:59:33AM +0800, Guo Ren wrote:
> On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
> >
> > On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> > >
> > > > > # Note: aligned to 8 bytes
> > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > addr+00 func: mv t0, ra
> > > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > > save another word here.
> > > > > addr+04 auipc t1, ftrace_caller
> > > > > addr+08 jalr ftrace_caller(t1)
> > >
> > > Is that some kind of 'load high' and 'add offset' pair?
> > Yes.
> >
> > > I guess 64bit kernels guarantee to put all module code
> > > within +-2G of the main kernel?
> > Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> > current rv64, just like CONFIG_32BIT.
> We need kernel_addr_base + this 32-bit Literal.
>
> @Mark Rutland
> What do you think the idea about reducing one more 32-bit in
> call-site? (It also sould work for arm64.)
The literal pointer is for a struct ftrace_ops, which is data, not code.
An ftrace_ops can be allocated from anywhere (e.g. core kernel data, module
data, linear map, vmalloc space), and so is not guaranteed to be within 2GiB of
all code. The literal needs to be able to address the entire kernel addresss
range, and since it can be modified concurrently (with PREEMPT and not using
stop_machine()) it needs to be possible to read/write atomically. So
practically speaking it needs to be the native pointer size (i.e. 64-bit on a
64-bit kernel).
Other schemes for compressing that (e.g. using an integer into an array of
pointers) is possible, but uses more memory and gets more complicated for
concurrent manipulation, so I would strongly recommend keeping this simple and
using a native pointer size here.
> > > > Here is the call-site:
> > > > # Note: aligned to 8 bytes
> > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > addr+00 auipc t0, ftrace_caller
> > > > addr+04 jalr ftrace_caller(t0)
> > >
> > > Could you even do something like:
> > > addr-n call ftrace-function
> > > addr-n+x literals
> > > addr+0 nop or jmp addr-n
> > > addr+4 function_code
> > Yours cost one more instruction, right?
> > addr-12 auipc
> > addr-8 jalr
> > addr-4 // Literal (32-bits)
> > addr+0 nop or jmp addr-n // one more?
> > addr+4 function_code
Placing instructions before the entry point is going to confuse symbol
resolution and unwind code, so I would not recommend that. It also means the
trampoline will need to re-adjust the return address back into the function,
but that is relatively simple.
I also think that this is micro-optimizing. The AUPIC *should* be cheap, so
executing that unconditionally should be fine. I think the form that Guo
suggested with AUIPC + {JALR || NOP} in the function (and 64-bits reserved
immediately bfore the function) is the way to go, so long as that does the
right thing with ra.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 1:31 ` Guo Ren
@ 2023-02-09 22:46 ` David Laight
-1 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-09 22:46 UTC (permalink / raw)
To: 'Guo Ren', Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
From: Guo Ren
> Sent: 09 February 2023 01:31
...
> > I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> > generating additional bits that the AUPIC didn't -- have I got that wrong?
> >
> > What specifies which register the JALR will write the link address to?
>
> According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
> is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
> stored in t0 + 0x0 and stores the return address to t0.
>
> That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
...
> What I want to point out:
> If we keep "auipc (addr+00)" fixed, we could use the different
> trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
> aligned area).
I looked up auipc:
"AUIPC is used to build PC-relative addresses and uses the U-type format.
AUIPC forms a 32-bit offset from the U-immediate, filling in the lowest
12 bits with zeros, adds this offset to the address of the AUIPC instruction,
then places the result in rd."
So it generates 'pc + (val << 12)'.
And the jalr then adds in a 12bit offset.
I think that means that if you have two trampolines you might need
to change both instructions even if the two trampolines are actually
adjacent instructions.
It is the distance from the call site that mustn't cross a 2k
boundary - not the absolute address of the trampoline itself.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
^ permalink raw reply [flat|nested] 88+ messages in thread
* RE: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-09 22:46 ` David Laight
0 siblings, 0 replies; 88+ messages in thread
From: David Laight @ 2023-02-09 22:46 UTC (permalink / raw)
To: 'Guo Ren', Mark Rutland
Cc: Evgenii Shatokhin, suagrfillet, andy.chiu, linux-riscv,
linux-kernel, Guo Ren, anup, paul.walmsley, palmer, conor.dooley,
heiko, rostedt, mhiramat, jolsa, bp, jpoimboe, linux
From: Guo Ren
> Sent: 09 February 2023 01:31
...
> > I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> > generating additional bits that the AUPIC didn't -- have I got that wrong?
> >
> > What specifies which register the JALR will write the link address to?
>
> According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
> is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
> stored in t0 + 0x0 and stores the return address to t0.
>
> That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
...
> What I want to point out:
> If we keep "auipc (addr+00)" fixed, we could use the different
> trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
> aligned area).
I looked up auipc:
"AUIPC is used to build PC-relative addresses and uses the U-type format.
AUIPC forms a 32-bit offset from the U-immediate, filling in the lowest
12 bits with zeros, adds this offset to the address of the AUIPC instruction,
then places the result in rd."
So it generates 'pc + (val << 12)'.
And the jalr then adds in a 12bit offset.
I think that means that if you have two trampolines you might need
to change both instructions even if the two trampolines are actually
adjacent instructions.
It is the distance from the call site that mustn't cross a 2k
boundary - not the absolute address of the trampoline itself.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 22:46 ` David Laight
@ 2023-02-10 2:18 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-10 2:18 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Fri, Feb 10, 2023 at 6:47 AM David Laight <David.Laight@aculab.com> wrote:
>
> From: Guo Ren
> > Sent: 09 February 2023 01:31
> ...
> > > I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> > > generating additional bits that the AUPIC didn't -- have I got that wrong?
> > >
> > > What specifies which register the JALR will write the link address to?
> >
> > According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
> > is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
> > stored in t0 + 0x0 and stores the return address to t0.
> >
> > That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
>
> ...
> > What I want to point out:
> > If we keep "auipc (addr+00)" fixed, we could use the different
> > trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
> > aligned area).
>
> I looked up auipc:
> "AUIPC is used to build PC-relative addresses and uses the U-type format.
> AUIPC forms a 32-bit offset from the U-immediate, filling in the lowest
> 12 bits with zeros, adds this offset to the address of the AUIPC instruction,
> then places the result in rd."
>
> So it generates 'pc + (val << 12)'.
> And the jalr then adds in a 12bit offset.
Correct!
>
> I think that means that if you have two trampolines you might need
> to change both instructions even if the two trampolines are actually
> adjacent instructions.
> It is the distance from the call site that mustn't cross a 2k
> boundary - not the absolute address of the trampoline itself.
We could have multiple adjacent trampolines, which must be kept in the
same 2k boundary area. Then we needn't change auipc part when
modifying.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-10 2:18 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-10 2:18 UTC (permalink / raw)
To: David Laight
Cc: Mark Rutland, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Fri, Feb 10, 2023 at 6:47 AM David Laight <David.Laight@aculab.com> wrote:
>
> From: Guo Ren
> > Sent: 09 February 2023 01:31
> ...
> > > I'm a bit confused there; I thought that the `symbol(reg)` addressing mode was
> > > generating additional bits that the AUPIC didn't -- have I got that wrong?
> > >
> > > What specifies which register the JALR will write the link address to?
> >
> > According to the spec, auipc t1,0x0 should write PC + 0x0<<12 (which
> > is equal to PC) to t1 and then jalr t0, (t0)0 jumps to the address
> > stored in t0 + 0x0 and stores the return address to t0.
> >
> > That means auipc defines xxx << 12 bits, jalr defines lowest 12 bits.
>
> ...
> > What I want to point out:
> > If we keep "auipc (addr+00)" fixed, we could use the different
> > trampolines at "jalr (addr+0x4)" (All of them must be in one 2k
> > aligned area).
>
> I looked up auipc:
> "AUIPC is used to build PC-relative addresses and uses the U-type format.
> AUIPC forms a 32-bit offset from the U-immediate, filling in the lowest
> 12 bits with zeros, adds this offset to the address of the AUIPC instruction,
> then places the result in rd."
>
> So it generates 'pc + (val << 12)'.
> And the jalr then adds in a 12bit offset.
Correct!
>
> I think that means that if you have two trampolines you might need
> to change both instructions even if the two trampolines are actually
> adjacent instructions.
> It is the distance from the call site that mustn't cross a 2k
> boundary - not the absolute address of the trampoline itself.
We could have multiple adjacent trampolines, which must be kept in the
same 2k boundary area. Then we needn't change auipc part when
modifying.
>
> David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-02-09 9:54 ` Mark Rutland
@ 2023-02-10 2:21 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-10 2:21 UTC (permalink / raw)
To: Mark Rutland
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 5:54 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Thu, Feb 09, 2023 at 09:59:33AM +0800, Guo Ren wrote:
> > On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
> > >
> > > On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> > > >
> > > > > > # Note: aligned to 8 bytes
> > > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > > addr+00 func: mv t0, ra
> > > > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > > > save another word here.
> > > > > > addr+04 auipc t1, ftrace_caller
> > > > > > addr+08 jalr ftrace_caller(t1)
> > > >
> > > > Is that some kind of 'load high' and 'add offset' pair?
> > > Yes.
> > >
> > > > I guess 64bit kernels guarantee to put all module code
> > > > within +-2G of the main kernel?
> > > Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> > > current rv64, just like CONFIG_32BIT.
> > We need kernel_addr_base + this 32-bit Literal.
> >
> > @Mark Rutland
> > What do you think the idea about reducing one more 32-bit in
> > call-site? (It also sould work for arm64.)
>
> The literal pointer is for a struct ftrace_ops, which is data, not code.
>
> An ftrace_ops can be allocated from anywhere (e.g. core kernel data, module
> data, linear map, vmalloc space), and so is not guaranteed to be within 2GiB of
> all code. The literal needs to be able to address the entire kernel addresss
> range, and since it can be modified concurrently (with PREEMPT and not using
> stop_machine()) it needs to be possible to read/write atomically. So
> practically speaking it needs to be the native pointer size (i.e. 64-bit on a
> 64-bit kernel).
Got it, thx. Let's use an absolute pointer as the beginning.
>
> Other schemes for compressing that (e.g. using an integer into an array of
> pointers) is possible, but uses more memory and gets more complicated for
> concurrent manipulation, so I would strongly recommend keeping this simple and
> using a native pointer size here.
>
> > > > > Here is the call-site:
> > > > > # Note: aligned to 8 bytes
> > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > addr+00 auipc t0, ftrace_caller
> > > > > addr+04 jalr ftrace_caller(t0)
> > > >
> > > > Could you even do something like:
> > > > addr-n call ftrace-function
> > > > addr-n+x literals
> > > > addr+0 nop or jmp addr-n
> > > > addr+4 function_code
> > > Yours cost one more instruction, right?
> > > addr-12 auipc
> > > addr-8 jalr
> > > addr-4 // Literal (32-bits)
> > > addr+0 nop or jmp addr-n // one more?
> > > addr+4 function_code
>
> Placing instructions before the entry point is going to confuse symbol
> resolution and unwind code, so I would not recommend that. It also means the
> trampoline will need to re-adjust the return address back into the function,
> but that is relatively simple.
>
> I also think that this is micro-optimizing. The AUPIC *should* be cheap, so
> executing that unconditionally should be fine. I think the form that Guo
> suggested with AUIPC + {JALR || NOP} in the function (and 64-bits reserved
> immediately bfore the function) is the way to go, so long as that does the
> right thing with ra.
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-10 2:21 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-10 2:21 UTC (permalink / raw)
To: Mark Rutland
Cc: David Laight, Evgenii Shatokhin, suagrfillet, andy.chiu,
linux-riscv, linux-kernel, Guo Ren, anup, paul.walmsley, palmer,
conor.dooley, heiko, rostedt, mhiramat, jolsa, bp, jpoimboe,
linux
On Thu, Feb 9, 2023 at 5:54 PM Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Thu, Feb 09, 2023 at 09:59:33AM +0800, Guo Ren wrote:
> > On Thu, Feb 9, 2023 at 9:51 AM Guo Ren <guoren@kernel.org> wrote:
> > >
> > > On Thu, Feb 9, 2023 at 6:29 AM David Laight <David.Laight@aculab.com> wrote:
> > > >
> > > > > > # Note: aligned to 8 bytes
> > > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > > addr+00 func: mv t0, ra
> > > > > We needn't "mv t0, ra" here because our "jalr" could work with t0 and
> > > > > won't affect ra. Let's do it in the trampoline code, and then we can
> > > > > save another word here.
> > > > > > addr+04 auipc t1, ftrace_caller
> > > > > > addr+08 jalr ftrace_caller(t1)
> > > >
> > > > Is that some kind of 'load high' and 'add offset' pair?
> > > Yes.
> > >
> > > > I guess 64bit kernels guarantee to put all module code
> > > > within +-2G of the main kernel?
> > > Yes, 32-bit is enough. So we only need one 32-bit literal size for the
> > > current rv64, just like CONFIG_32BIT.
> > We need kernel_addr_base + this 32-bit Literal.
> >
> > @Mark Rutland
> > What do you think the idea about reducing one more 32-bit in
> > call-site? (It also sould work for arm64.)
>
> The literal pointer is for a struct ftrace_ops, which is data, not code.
>
> An ftrace_ops can be allocated from anywhere (e.g. core kernel data, module
> data, linear map, vmalloc space), and so is not guaranteed to be within 2GiB of
> all code. The literal needs to be able to address the entire kernel addresss
> range, and since it can be modified concurrently (with PREEMPT and not using
> stop_machine()) it needs to be possible to read/write atomically. So
> practically speaking it needs to be the native pointer size (i.e. 64-bit on a
> 64-bit kernel).
Got it, thx. Let's use an absolute pointer as the beginning.
>
> Other schemes for compressing that (e.g. using an integer into an array of
> pointers) is possible, but uses more memory and gets more complicated for
> concurrent manipulation, so I would strongly recommend keeping this simple and
> using a native pointer size here.
>
> > > > > Here is the call-site:
> > > > > # Note: aligned to 8 bytes
> > > > > addr-08 // Literal (first 32-bits) // patched to ops ptr
> > > > > addr-04 // Literal (last 32-bits) // patched to ops ptr
> > > > > addr+00 auipc t0, ftrace_caller
> > > > > addr+04 jalr ftrace_caller(t0)
> > > >
> > > > Could you even do something like:
> > > > addr-n call ftrace-function
> > > > addr-n+x literals
> > > > addr+0 nop or jmp addr-n
> > > > addr+4 function_code
> > > Yours cost one more instruction, right?
> > > addr-12 auipc
> > > addr-8 jalr
> > > addr-4 // Literal (32-bits)
> > > addr+0 nop or jmp addr-n // one more?
> > > addr+4 function_code
>
> Placing instructions before the entry point is going to confuse symbol
> resolution and unwind code, so I would not recommend that. It also means the
> trampoline will need to re-adjust the return address back into the function,
> but that is relatively simple.
>
> I also think that this is micro-optimizing. The AUPIC *should* be cheap, so
> executing that unconditionally should be fine. I think the form that Guo
> suggested with AUIPC + {JALR || NOP} in the function (and 64-bits reserved
> immediately bfore the function) is the way to go, so long as that does the
> right thing with ra.
>
> Thanks,
> Mark.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-01-19 6:05 ` Guo Ren
@ 2023-02-18 21:30 ` Palmer Dabbelt
-1 siblings, 0 replies; 88+ messages in thread
From: Palmer Dabbelt @ 2023-02-18 21:30 UTC (permalink / raw)
To: guoren
Cc: e.shatokhin, suagrfillet, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> Thx Evgenii & Song,
>
> I got it; it would be put into v8.
Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
the first three patches onto for-next as they seem like pretty
independent fixes/cleanups, I'll hold off on the others until v8.
Thanks!
>
> On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> <e.shatokhin@yadro.com> wrote:
>>
>> On 18.01.2023 05:37, Song Shuai wrote:
>> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>> >>
>> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
>> >>> Hi, Song,
>> >>>
>> >>> On 17.01.2023 12:32, Song Shuai wrote:
>> >>>>
>> >>>> Hi, Evgenii:
>> >>>>
>> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>> >>>>
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
>> >>>>>>
>> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
>> >>>>>>
>> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
>> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>> >>>>>> ---
>> >>>>>> arch/riscv/Kconfig | 2 ++
>> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>> >>>>>> +++++++++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>> >>>>>> 6 files changed, 142 insertions(+)
>> >>>>>
>> >>>>> The samples were built OK now, but ftrace-direct-multi and
>> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>> >>>>>
>> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>> >>>>> the messages in the trace:
>> >>>>>
>> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> >>>>> # | | | ||||| | |
>> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>> >>>>> direct func1 ip ff60000003334d00
>> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>> >>>>> direct func1 ip 7fffffffffffffff
>> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> [...]
>> >>>>>
>> >>>>> If I understand it right, my_direct_func1() should print the address of
>> >>>>> some location in the code, probably - at the beginning of the traced
>> >>>>> functions.
>> >>>>>
>> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>> >>>>> addresses.
>> >>>>>
>> >>>> The invalid code address is only printed by accessing the schedule()
>> >>>> function's first argument whose address stores in a0 register.
>> >>>> While schedule() actually has no parameter declared, so my_direct_func
>> >>>> just prints the a0 in the context of the schedule()'s caller and
>> >>>> the address maybe varies depending on the caller.
>> >>>>
>> >>>> I can't really understand why tracing the first argument of the
>> >>>> schedule() function, but it seems nonsense at this point.
>> >>>
>> >>> The question is, what should be passed as the argument(s) of
>> >>> my_direct_func() in this particular sample module. The kernel docs and
>> >>> commit logs seem to contain no info on that.
>> >>>
>> >>> With direct functions, I suppose, the trampoline can pass anything it
>> >>> wants to my_direct_func(), not just the arguments of the traced function.
>> >>>
>> >>> I'd check what these sample modules do on x86 and would try to match
>> >>> that behaviour on RISC-V.
>> >>
>> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
>> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
>> >> be passed to my_direct_func().
>> >>
>> >> ftrace-direct-multi.ko:
>> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> >> # | | | ||||| | |
>> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
>> >> ffffffff87332f45 // wake_up_process+0x5
>> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
>> >> ffffffff8828d935 // schedule+0x5
>> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
>> >> ffffffff8828d935
>> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
>> >> ffffffff8828d935
>> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
>> >> ffffffff8828d935
>> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
>> >> ffffffff87332f45
>> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
>> >> ffffffff87332f45
>> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
>> >> ffffffff8828d935
>> >> [...]
>> >>
>> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
>> >> where the execution of the traced functions resumes after the Ftrace
>> >> trampoline has finished.
>> >>
>> >> The results with ftrace-direct-multi-modify.ko are similar to that.
>> >>
>> >> The samples look like a demonstration, that one can pass anything
>> >> necessary to the handler in case of "direct" functions.
>> >>
>> >> I suppose, the RISC-V-specific asm code in these two sample modules
>> >> could be updated to pass the saved pc value to my_direct_func() in a0.
>> >
>> > Yes, you're right.
>> >
>> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
>> > traced function with mcount offset.
>> >
>> > Here is the updated patch for your reference.
>> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>>
>> Thank you for the quick fix. This one looks good to me.
>>
>> ftrace-direct-multi*.ko now report the ip values corresponding to
>> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
>>
>> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
>> your patch, becase this is what I actually did: tested it and reported
>> the results. I cannot take your credit for development of this patch ;-)
>>
>> Looking forward for v8 of the series.
>> >
>> >
>> >>
>> >>>
>> >>>>
>> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
>> >>>> trace kernel functions, and basically saves the necessary ABI,
>> >>>> call the tracing function, and restores the ABI, just like other
>> >>>> arches do.
>> >>>> so It shouldn't be blamed.
>> >>>>
>> >>>> I started an independent patch to replace schedule with kick_process
>> >>>> to make these samples more reasonable. And It has no conflict with the
>> >>>> current patch, so we can go on.
>> >>>>
>> >>>> Link:
>> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>> >>>>
>> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
>> >>>>>
>> >>>>> Is anything missing here?
>> >>>>>
>> >>>>>>
>> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> >>>>>> index 307a9f413edd..e944af44f681 100644
>> >>>>>> --- a/arch/riscv/Kconfig
>> >>>>>> +++ b/arch/riscv/Kconfig
>> >>>>>> @@ -112,6 +112,8 @@ config RISCV
>> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>> >>>>>> select HAVE_STACKPROTECTOR
>> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
>> >>>>>> select HAVE_RSEQ
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>> >>>>>>
>> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp1, @function\n"
>> >>>>>> +" .globl my_tramp1\n"
>> >>>>>> +" my_tramp1:\n"
>> >>>>>> +" addi sp,sp,-16\n"
>> >>>>>> +" sd t0,0(sp)\n"
>> >>>>>> +" sd ra,8(sp)\n"
>> >>>>>> +" call my_direct_func1\n"
>> >>>>>> +" ld t0,0(sp)\n"
>> >>>>>> +" ld ra,8(sp)\n"
>> >>>>>> +" addi sp,sp,16\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
>> >>>>>> +
>> >>>>>> +" .type my_tramp2, @function\n"
>> >>>>>> +" .globl my_tramp2\n"
>> >>>>>> +" my_tramp2:\n"
>> >>>>>> +" addi sp,sp,-16\n"
>> >>>>>> +" sd t0,0(sp)\n"
>> >>>>>> +" sd ra,8(sp)\n"
>> >>>>>> +" call my_direct_func2\n"
>> >>>>>> +" ld t0,0(sp)\n"
>> >>>>>> +" ld ra,8(sp)\n"
>> >>>>>> +" addi sp,sp,16\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> index d52370cad0b6..10884bf418f7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>> >>>>>> extern void my_tramp1(void *);
>> >>>>>> extern void my_tramp2(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp1, @function\n"
>> >>>>>> +" .globl my_tramp1\n"
>> >>>>>> +" my_tramp1:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func1\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
>> >>>>>> +
>> >>>>>> +" .type my_tramp2, @function\n"
>> >>>>>> +" .globl my_tramp2\n"
>> >>>>>> +" my_tramp2:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func2\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> index ec1088922517..a35bf43bf6d7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>> >>>>>> b/samples/ftrace/ftrace-direct-too.c
>> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-40\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd a1,8(sp)\n"
>> >>>>>> +" sd a2,16(sp)\n"
>> >>>>>> +" sd t0,24(sp)\n"
>> >>>>>> +" sd ra,32(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld a1,8(sp)\n"
>> >>>>>> +" ld a2,16(sp)\n"
>> >>>>>> +" ld t0,24(sp)\n"
>> >>>>>> +" ld ra,32(sp)\n"
>> >>>>>> +" addi sp,sp,40\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
>> >>>>>> b/samples/ftrace/ftrace-direct.c
>> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
>> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> --
>> >>>>>> 2.36.1
>> >>>>
>> >>>> --
>> >>>> Thanks,
>> >>>> Song
>> >>>>
>> >>>
>> >>> Regards,
>> >>> Evgenii
>> >>
>> >>
>> >
>> >
>> > --
>> > Thanks,
>> > Song
>> >
>>
>>
>
>
> --
> Best Regards
> Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-02-18 21:30 ` Palmer Dabbelt
0 siblings, 0 replies; 88+ messages in thread
From: Palmer Dabbelt @ 2023-02-18 21:30 UTC (permalink / raw)
To: guoren
Cc: e.shatokhin, suagrfillet, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> Thx Evgenii & Song,
>
> I got it; it would be put into v8.
Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
the first three patches onto for-next as they seem like pretty
independent fixes/cleanups, I'll hold off on the others until v8.
Thanks!
>
> On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> <e.shatokhin@yadro.com> wrote:
>>
>> On 18.01.2023 05:37, Song Shuai wrote:
>> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
>> >>
>> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
>> >>> Hi, Song,
>> >>>
>> >>> On 17.01.2023 12:32, Song Shuai wrote:
>> >>>>
>> >>>> Hi, Evgenii:
>> >>>>
>> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
>> >>>>
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
>> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
>> >>>>>>
>> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
>> >>>>>>
>> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
>> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>> >>>>>> ---
>> >>>>>> arch/riscv/Kconfig | 2 ++
>> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
>> >>>>>> +++++++++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
>> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
>> >>>>>> 6 files changed, 142 insertions(+)
>> >>>>>
>> >>>>> The samples were built OK now, but ftrace-direct-multi and
>> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
>> >>>>>
>> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
>> >>>>> the messages in the trace:
>> >>>>>
>> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> >>>>> # | | | ||||| | |
>> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
>> >>>>> direct func1 ip ff60000003334d00
>> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
>> >>>>> direct func1 ip 7fffffffffffffff
>> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
>> >>>>> direct func1 ip ff60000001ba9600
>> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
>> >>>>> direct func1 ip 0
>> >>>>> [...]
>> >>>>>
>> >>>>> If I understand it right, my_direct_func1() should print the address of
>> >>>>> some location in the code, probably - at the beginning of the traced
>> >>>>> functions.
>> >>>>>
>> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
>> >>>>> addresses.
>> >>>>>
>> >>>> The invalid code address is only printed by accessing the schedule()
>> >>>> function's first argument whose address stores in a0 register.
>> >>>> While schedule() actually has no parameter declared, so my_direct_func
>> >>>> just prints the a0 in the context of the schedule()'s caller and
>> >>>> the address maybe varies depending on the caller.
>> >>>>
>> >>>> I can't really understand why tracing the first argument of the
>> >>>> schedule() function, but it seems nonsense at this point.
>> >>>
>> >>> The question is, what should be passed as the argument(s) of
>> >>> my_direct_func() in this particular sample module. The kernel docs and
>> >>> commit logs seem to contain no info on that.
>> >>>
>> >>> With direct functions, I suppose, the trampoline can pass anything it
>> >>> wants to my_direct_func(), not just the arguments of the traced function.
>> >>>
>> >>> I'd check what these sample modules do on x86 and would try to match
>> >>> that behaviour on RISC-V.
>> >>
>> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
>> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
>> >> be passed to my_direct_func().
>> >>
>> >> ftrace-direct-multi.ko:
>> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> >> # | | | ||||| | |
>> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
>> >> ffffffff87332f45 // wake_up_process+0x5
>> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
>> >> ffffffff8828d935 // schedule+0x5
>> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
>> >> ffffffff8828d935
>> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
>> >> ffffffff8828d935
>> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
>> >> ffffffff8828d935
>> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
>> >> ffffffff87332f45
>> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
>> >> ffffffff87332f45
>> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
>> >> ffffffff8828d935
>> >> [...]
>> >>
>> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
>> >> where the execution of the traced functions resumes after the Ftrace
>> >> trampoline has finished.
>> >>
>> >> The results with ftrace-direct-multi-modify.ko are similar to that.
>> >>
>> >> The samples look like a demonstration, that one can pass anything
>> >> necessary to the handler in case of "direct" functions.
>> >>
>> >> I suppose, the RISC-V-specific asm code in these two sample modules
>> >> could be updated to pass the saved pc value to my_direct_func() in a0.
>> >
>> > Yes, you're right.
>> >
>> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
>> > traced function with mcount offset.
>> >
>> > Here is the updated patch for your reference.
>> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>>
>> Thank you for the quick fix. This one looks good to me.
>>
>> ftrace-direct-multi*.ko now report the ip values corresponding to
>> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
>>
>> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
>> your patch, becase this is what I actually did: tested it and reported
>> the results. I cannot take your credit for development of this patch ;-)
>>
>> Looking forward for v8 of the series.
>> >
>> >
>> >>
>> >>>
>> >>>>
>> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
>> >>>> trace kernel functions, and basically saves the necessary ABI,
>> >>>> call the tracing function, and restores the ABI, just like other
>> >>>> arches do.
>> >>>> so It shouldn't be blamed.
>> >>>>
>> >>>> I started an independent patch to replace schedule with kick_process
>> >>>> to make these samples more reasonable. And It has no conflict with the
>> >>>> current patch, so we can go on.
>> >>>>
>> >>>> Link:
>> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
>> >>>>
>> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
>> >>>>>
>> >>>>> Is anything missing here?
>> >>>>>
>> >>>>>>
>> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> >>>>>> index 307a9f413edd..e944af44f681 100644
>> >>>>>> --- a/arch/riscv/Kconfig
>> >>>>>> +++ b/arch/riscv/Kconfig
>> >>>>>> @@ -112,6 +112,8 @@ config RISCV
>> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
>> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
>> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
>> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
>> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>> >>>>>> select HAVE_STACKPROTECTOR
>> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
>> >>>>>> select HAVE_RSEQ
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
>> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
>> >>>>>>
>> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp1, @function\n"
>> >>>>>> +" .globl my_tramp1\n"
>> >>>>>> +" my_tramp1:\n"
>> >>>>>> +" addi sp,sp,-16\n"
>> >>>>>> +" sd t0,0(sp)\n"
>> >>>>>> +" sd ra,8(sp)\n"
>> >>>>>> +" call my_direct_func1\n"
>> >>>>>> +" ld t0,0(sp)\n"
>> >>>>>> +" ld ra,8(sp)\n"
>> >>>>>> +" addi sp,sp,16\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
>> >>>>>> +
>> >>>>>> +" .type my_tramp2, @function\n"
>> >>>>>> +" .globl my_tramp2\n"
>> >>>>>> +" my_tramp2:\n"
>> >>>>>> +" addi sp,sp,-16\n"
>> >>>>>> +" sd t0,0(sp)\n"
>> >>>>>> +" sd ra,8(sp)\n"
>> >>>>>> +" call my_direct_func2\n"
>> >>>>>> +" ld t0,0(sp)\n"
>> >>>>>> +" ld ra,8(sp)\n"
>> >>>>>> +" addi sp,sp,16\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> index d52370cad0b6..10884bf418f7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
>> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
>> >>>>>> extern void my_tramp1(void *);
>> >>>>>> extern void my_tramp2(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp1, @function\n"
>> >>>>>> +" .globl my_tramp1\n"
>> >>>>>> +" my_tramp1:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func1\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
>> >>>>>> +
>> >>>>>> +" .type my_tramp2, @function\n"
>> >>>>>> +" .globl my_tramp2\n"
>> >>>>>> +" my_tramp2:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func2\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> index ec1088922517..a35bf43bf6d7 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
>> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
>> >>>>>> b/samples/ftrace/ftrace-direct-too.c
>> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
>> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-40\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd a1,8(sp)\n"
>> >>>>>> +" sd a2,16(sp)\n"
>> >>>>>> +" sd t0,24(sp)\n"
>> >>>>>> +" sd ra,32(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld a1,8(sp)\n"
>> >>>>>> +" ld a2,16(sp)\n"
>> >>>>>> +" ld t0,24(sp)\n"
>> >>>>>> +" ld ra,32(sp)\n"
>> >>>>>> +" addi sp,sp,40\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
>> >>>>>> b/samples/ftrace/ftrace-direct.c
>> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
>> >>>>>> --- a/samples/ftrace/ftrace-direct.c
>> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
>> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
>> >>>>>>
>> >>>>>> extern void my_tramp(void *);
>> >>>>>>
>> >>>>>> +#ifdef CONFIG_RISCV
>> >>>>>> +
>> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
>> >>>>>> +" .type my_tramp, @function\n"
>> >>>>>> +" .globl my_tramp\n"
>> >>>>>> +" my_tramp:\n"
>> >>>>>> +" addi sp,sp,-24\n"
>> >>>>>> +" sd a0,0(sp)\n"
>> >>>>>> +" sd t0,8(sp)\n"
>> >>>>>> +" sd ra,16(sp)\n"
>> >>>>>> +" call my_direct_func\n"
>> >>>>>> +" ld a0,0(sp)\n"
>> >>>>>> +" ld t0,8(sp)\n"
>> >>>>>> +" ld ra,16(sp)\n"
>> >>>>>> +" addi sp,sp,24\n"
>> >>>>>> +" jr t0\n"
>> >>>>>> +" .size my_tramp, .-my_tramp\n"
>> >>>>>> +" .popsection\n"
>> >>>>>> +);
>> >>>>>> +
>> >>>>>> +#endif /* CONFIG_RISCV */
>> >>>>>> +
>> >>>>>> #ifdef CONFIG_X86_64
>> >>>>>>
>> >>>>>> #include <asm/ibt.h>
>> >>>>>> --
>> >>>>>> 2.36.1
>> >>>>
>> >>>> --
>> >>>> Thanks,
>> >>>> Song
>> >>>>
>> >>>
>> >>> Regards,
>> >>> Evgenii
>> >>
>> >>
>> >
>> >
>> > --
>> > Thanks,
>> > Song
>> >
>>
>>
>
>
> --
> Best Regards
> Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
2023-01-12 9:05 ` guoren
@ 2023-02-18 21:42 ` patchwork-bot+linux-riscv
-1 siblings, 0 replies; 88+ messages in thread
From: patchwork-bot+linux-riscv @ 2023-02-18 21:42 UTC (permalink / raw)
To: Guo Ren
Cc: linux-riscv, anup, paul.walmsley, palmer, conor.dooley, heiko,
rostedt, mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-kernel, guoren
Hello:
This series was applied to riscv/linux.git (for-next)
by Palmer Dabbelt <palmer@rivosinc.com>:
On Thu, 12 Jan 2023 04:05:56 -0500 you wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
>
> - The most horrible bug is preemption panic which found by Andy [1].
> Let's disable preemption for ftrace first, and Andy could continue
> the ftrace preemption work.
> - The "-fpatchable-function-entry= CFLAG" wasted code size
> !RISCV_ISA_C.
> - The ftrace detour implementation wasted code size.
> - When livepatching, the trampoline (ftrace_regs_caller) would not
> return to <func_prolog+12> but would rather jump to the new function.
> So, "REG_L ra, -SZREG(sp)" would not run and the original return
> address would not be restored. The kernel is likely to hang or crash
> as a result. (Found by Evgenii Shatokhin [4])
>
> [...]
Here is the summary with links:
- [-next,V7,1/7] riscv: ftrace: Fixup panic by disabling preemption
https://git.kernel.org/riscv/c/8547649981e6
- [-next,V7,2/7] riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
https://git.kernel.org/riscv/c/409c8fb20c66
- [-next,V7,3/7] riscv: ftrace: Reduce the detour code size to half
https://git.kernel.org/riscv/c/6724a76cff85
- [-next,V7,4/7] riscv: ftrace: Add ftrace_graph_func
(no matching commit)
- [-next,V7,5/7] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
(no matching commit)
- [-next,V7,6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
(no matching commit)
- [-next,V7,7/7] riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
(no matching commit)
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 0/7] riscv: Optimize function trace
@ 2023-02-18 21:42 ` patchwork-bot+linux-riscv
0 siblings, 0 replies; 88+ messages in thread
From: patchwork-bot+linux-riscv @ 2023-02-18 21:42 UTC (permalink / raw)
To: Guo Ren
Cc: linux-riscv, anup, paul.walmsley, palmer, conor.dooley, heiko,
rostedt, mhiramat, jolsa, bp, jpoimboe, suagrfillet, andy.chiu,
e.shatokhin, linux-kernel, guoren
Hello:
This series was applied to riscv/linux.git (for-next)
by Palmer Dabbelt <palmer@rivosinc.com>:
On Thu, 12 Jan 2023 04:05:56 -0500 you wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The previous ftrace detour implementation fc76b8b8011 ("riscv: Using
> PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems.
>
> - The most horrible bug is preemption panic which found by Andy [1].
> Let's disable preemption for ftrace first, and Andy could continue
> the ftrace preemption work.
> - The "-fpatchable-function-entry= CFLAG" wasted code size
> !RISCV_ISA_C.
> - The ftrace detour implementation wasted code size.
> - When livepatching, the trampoline (ftrace_regs_caller) would not
> return to <func_prolog+12> but would rather jump to the new function.
> So, "REG_L ra, -SZREG(sp)" would not run and the original return
> address would not be restored. The kernel is likely to hang or crash
> as a result. (Found by Evgenii Shatokhin [4])
>
> [...]
Here is the summary with links:
- [-next,V7,1/7] riscv: ftrace: Fixup panic by disabling preemption
https://git.kernel.org/riscv/c/8547649981e6
- [-next,V7,2/7] riscv: ftrace: Remove wasted nops for !RISCV_ISA_C
https://git.kernel.org/riscv/c/409c8fb20c66
- [-next,V7,3/7] riscv: ftrace: Reduce the detour code size to half
https://git.kernel.org/riscv/c/6724a76cff85
- [-next,V7,4/7] riscv: ftrace: Add ftrace_graph_func
(no matching commit)
- [-next,V7,5/7] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
(no matching commit)
- [-next,V7,6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
(no matching commit)
- [-next,V7,7/7] riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
(no matching commit)
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-02-18 21:30 ` Palmer Dabbelt
@ 2023-02-20 2:46 ` Song Shuai
-1 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-02-20 2:46 UTC (permalink / raw)
To: guoren
Cc: Palmer Dabbelt, e.shatokhin, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
Hi,Guo:
Palmer Dabbelt <palmer@dabbelt.com> 于2023年2月18日周六 21:30写道：
>
> On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > Thx Evgenii & Song,
> >
> > I got it; it would be put into v8.
>
There were 2 problems for this patch in its V6 and V7 version:
1. build error resulted by including of nospec-branch.h file in V6
- this common fixup patch had been sent to the linux-trace mail
list and would show up in Linux v6.3
- here is the link:
https://lore.kernel.org/lkml/20230208015633.791198913@goodmis.org/
2. invalid code address reported by Evgenii in V7
- for your convenience, the link to this updated patch is
https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
Is it time to launch the V8 with this updated patch, or waiting for
the common fixup patch to be merged?
> Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> the first three patches onto for-next as they seem like pretty
> independent fixes/cleanups, I'll hold off on the others until v8.
>
> Thanks!
>
> >
> > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> >>
> >> On 18.01.2023 05:37, Song Shuai wrote:
> >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >> >>
> >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >> >>> Hi, Song,
> >> >>>
> >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >> >>>>
> >> >>>> Hi, Evgenii:
> >> >>>>
> >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >> >>>>
> >> >>>>>
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>>
> >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >> >>>>>>
> >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> ---
> >> >>>>>> arch/riscv/Kconfig | 2 ++
> >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >> >>>>>> +++++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >> >>>>>> 6 files changed, 142 insertions(+)
> >> >>>>>
> >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >> >>>>>
> >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >> >>>>> the messages in the trace:
> >> >>>>>
> >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >>>>> # | | | ||||| | |
> >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000003334d00
> >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >> >>>>> direct func1 ip 7fffffffffffffff
> >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> [...]
> >> >>>>>
> >> >>>>> If I understand it right, my_direct_func1() should print the address of
> >> >>>>> some location in the code, probably - at the beginning of the traced
> >> >>>>> functions.
> >> >>>>>
> >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >> >>>>> addresses.
> >> >>>>>
> >> >>>> The invalid code address is only printed by accessing the schedule()
> >> >>>> function's first argument whose address stores in a0 register.
> >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >> >>>> just prints the a0 in the context of the schedule()'s caller and
> >> >>>> the address maybe varies depending on the caller.
> >> >>>>
> >> >>>> I can't really understand why tracing the first argument of the
> >> >>>> schedule() function, but it seems nonsense at this point.
> >> >>>
> >> >>> The question is, what should be passed as the argument(s) of
> >> >>> my_direct_func() in this particular sample module. The kernel docs and
> >> >>> commit logs seem to contain no info on that.
> >> >>>
> >> >>> With direct functions, I suppose, the trampoline can pass anything it
> >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >> >>>
> >> >>> I'd check what these sample modules do on x86 and would try to match
> >> >>> that behaviour on RISC-V.
> >> >>
> >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> >> be passed to my_direct_func().
> >> >>
> >> >> ftrace-direct-multi.ko:
> >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >> # | | | ||||| | |
> >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> >> ffffffff87332f45 // wake_up_process+0x5
> >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> >> ffffffff8828d935 // schedule+0x5
> >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> [...]
> >> >>
> >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> >> where the execution of the traced functions resumes after the Ftrace
> >> >> trampoline has finished.
> >> >>
> >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >> >>
> >> >> The samples look like a demonstration, that one can pass anything
> >> >> necessary to the handler in case of "direct" functions.
> >> >>
> >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >> >
> >> > Yes, you're right.
> >> >
> >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> >> > traced function with mcount offset.
> >> >
> >> > Here is the updated patch for your reference.
> >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> >>
> >> Thank you for the quick fix. This one looks good to me.
> >>
> >> ftrace-direct-multi*.ko now report the ip values corresponding to
> >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> >>
> >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> >> your patch, becase this is what I actually did: tested it and reported
> >> the results. I cannot take your credit for development of this patch ;-)
> >>
> >> Looking forward for v8 of the series.
> >> >
> >> >
> >> >>
> >> >>>
> >> >>>>
> >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >> >>>> trace kernel functions, and basically saves the necessary ABI,
> >> >>>> call the tracing function, and restores the ABI, just like other
> >> >>>> arches do.
> >> >>>> so It shouldn't be blamed.
> >> >>>>
> >> >>>> I started an independent patch to replace schedule with kick_process
> >> >>>> to make these samples more reasonable. And It has no conflict with the
> >> >>>> current patch, so we can go on.
> >> >>>>
> >> >>>> Link:
> >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >> >>>>
> >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >> >>>>>
> >> >>>>> Is anything missing here?
> >> >>>>>
> >> >>>>>>
> >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >> >>>>>> index 307a9f413edd..e944af44f681 100644
> >> >>>>>> --- a/arch/riscv/Kconfig
> >> >>>>>> +++ b/arch/riscv/Kconfig
> >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> select HAVE_STACKPROTECTOR
> >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >> >>>>>> select HAVE_RSEQ
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >> >>>>>> extern void my_tramp1(void *);
> >> >>>>>> extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-40\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd a1,8(sp)\n"
> >> >>>>>> +" sd a2,16(sp)\n"
> >> >>>>>> +" sd t0,24(sp)\n"
> >> >>>>>> +" sd ra,32(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld a1,8(sp)\n"
> >> >>>>>> +" ld a2,16(sp)\n"
> >> >>>>>> +" ld t0,24(sp)\n"
> >> >>>>>> +" ld ra,32(sp)\n"
> >> >>>>>> +" addi sp,sp,40\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >> >>>>>> b/samples/ftrace/ftrace-direct.c
> >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> --
> >> >>>>>> 2.36.1
> >> >>>>
> >> >>>> --
> >> >>>> Thanks,
> >> >>>> Song
> >> >>>>
> >> >>>
> >> >>> Regards,
> >> >>> Evgenii
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Song
> >> >
> >>
> >>
> >
> >
> > --
> > Best Regards
> > Guo Ren
--
Thanks,
Song
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-02-20 2:46 ` Song Shuai
0 siblings, 0 replies; 88+ messages in thread
From: Song Shuai @ 2023-02-20 2:46 UTC (permalink / raw)
To: guoren
Cc: Palmer Dabbelt, e.shatokhin, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
Hi,Guo:
Palmer Dabbelt <palmer@dabbelt.com> 于2023年2月18日周六 21:30写道：
>
> On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > Thx Evgenii & Song,
> >
> > I got it; it would be put into v8.
>
There were 2 problems for this patch in its V6 and V7 version:
1. build error resulted by including of nospec-branch.h file in V6
- this common fixup patch had been sent to the linux-trace mail
list and would show up in Linux v6.3
- here is the link:
https://lore.kernel.org/lkml/20230208015633.791198913@goodmis.org/
2. invalid code address reported by Evgenii in V7
- for your convenience, the link to this updated patch is
https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
Is it time to launch the V8 with this updated patch, or waiting for
the common fixup patch to be merged?
> Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> the first three patches onto for-next as they seem like pretty
> independent fixes/cleanups, I'll hold off on the others until v8.
>
> Thanks!
>
> >
> > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> >>
> >> On 18.01.2023 05:37, Song Shuai wrote:
> >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >> >>
> >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >> >>> Hi, Song,
> >> >>>
> >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >> >>>>
> >> >>>> Hi, Evgenii:
> >> >>>>
> >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >> >>>>
> >> >>>>>
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>>
> >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >> >>>>>>
> >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> ---
> >> >>>>>> arch/riscv/Kconfig | 2 ++
> >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >> >>>>>> +++++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >> >>>>>> 6 files changed, 142 insertions(+)
> >> >>>>>
> >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >> >>>>>
> >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >> >>>>> the messages in the trace:
> >> >>>>>
> >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >>>>> # | | | ||||| | |
> >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000003334d00
> >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >> >>>>> direct func1 ip 7fffffffffffffff
> >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> [...]
> >> >>>>>
> >> >>>>> If I understand it right, my_direct_func1() should print the address of
> >> >>>>> some location in the code, probably - at the beginning of the traced
> >> >>>>> functions.
> >> >>>>>
> >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >> >>>>> addresses.
> >> >>>>>
> >> >>>> The invalid code address is only printed by accessing the schedule()
> >> >>>> function's first argument whose address stores in a0 register.
> >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >> >>>> just prints the a0 in the context of the schedule()'s caller and
> >> >>>> the address maybe varies depending on the caller.
> >> >>>>
> >> >>>> I can't really understand why tracing the first argument of the
> >> >>>> schedule() function, but it seems nonsense at this point.
> >> >>>
> >> >>> The question is, what should be passed as the argument(s) of
> >> >>> my_direct_func() in this particular sample module. The kernel docs and
> >> >>> commit logs seem to contain no info on that.
> >> >>>
> >> >>> With direct functions, I suppose, the trampoline can pass anything it
> >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >> >>>
> >> >>> I'd check what these sample modules do on x86 and would try to match
> >> >>> that behaviour on RISC-V.
> >> >>
> >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> >> be passed to my_direct_func().
> >> >>
> >> >> ftrace-direct-multi.ko:
> >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >> # | | | ||||| | |
> >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> >> ffffffff87332f45 // wake_up_process+0x5
> >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> >> ffffffff8828d935 // schedule+0x5
> >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> [...]
> >> >>
> >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> >> where the execution of the traced functions resumes after the Ftrace
> >> >> trampoline has finished.
> >> >>
> >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >> >>
> >> >> The samples look like a demonstration, that one can pass anything
> >> >> necessary to the handler in case of "direct" functions.
> >> >>
> >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >> >
> >> > Yes, you're right.
> >> >
> >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> >> > traced function with mcount offset.
> >> >
> >> > Here is the updated patch for your reference.
> >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> >>
> >> Thank you for the quick fix. This one looks good to me.
> >>
> >> ftrace-direct-multi*.ko now report the ip values corresponding to
> >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> >>
> >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> >> your patch, becase this is what I actually did: tested it and reported
> >> the results. I cannot take your credit for development of this patch ;-)
> >>
> >> Looking forward for v8 of the series.
> >> >
> >> >
> >> >>
> >> >>>
> >> >>>>
> >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >> >>>> trace kernel functions, and basically saves the necessary ABI,
> >> >>>> call the tracing function, and restores the ABI, just like other
> >> >>>> arches do.
> >> >>>> so It shouldn't be blamed.
> >> >>>>
> >> >>>> I started an independent patch to replace schedule with kick_process
> >> >>>> to make these samples more reasonable. And It has no conflict with the
> >> >>>> current patch, so we can go on.
> >> >>>>
> >> >>>> Link:
> >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >> >>>>
> >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >> >>>>>
> >> >>>>> Is anything missing here?
> >> >>>>>
> >> >>>>>>
> >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >> >>>>>> index 307a9f413edd..e944af44f681 100644
> >> >>>>>> --- a/arch/riscv/Kconfig
> >> >>>>>> +++ b/arch/riscv/Kconfig
> >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> select HAVE_STACKPROTECTOR
> >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >> >>>>>> select HAVE_RSEQ
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >> >>>>>> extern void my_tramp1(void *);
> >> >>>>>> extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-40\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd a1,8(sp)\n"
> >> >>>>>> +" sd a2,16(sp)\n"
> >> >>>>>> +" sd t0,24(sp)\n"
> >> >>>>>> +" sd ra,32(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld a1,8(sp)\n"
> >> >>>>>> +" ld a2,16(sp)\n"
> >> >>>>>> +" ld t0,24(sp)\n"
> >> >>>>>> +" ld ra,32(sp)\n"
> >> >>>>>> +" addi sp,sp,40\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >> >>>>>> b/samples/ftrace/ftrace-direct.c
> >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> --
> >> >>>>>> 2.36.1
> >> >>>>
> >> >>>> --
> >> >>>> Thanks,
> >> >>>> Song
> >> >>>>
> >> >>>
> >> >>> Regards,
> >> >>> Evgenii
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Song
> >> >
> >>
> >>
> >
> >
> > --
> > Best Regards
> > Guo Ren
--
Thanks,
Song
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-02-20 2:46 ` Song Shuai
@ 2023-02-21 3:56 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-21 3:56 UTC (permalink / raw)
To: Song Shuai
Cc: Palmer Dabbelt, e.shatokhin, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Mon, Feb 20, 2023 at 10:46 AM Song Shuai <suagrfillet@gmail.com> wrote:
>
> Hi,Guo:
>
> Palmer Dabbelt <palmer@dabbelt.com> 于2023年2月18日周六 21:30写道：
> >
> > On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > > Thx Evgenii & Song,
> > >
> > > I got it; it would be put into v8.
> >
> There were 2 problems for this patch in its V6 and V7 version:
>
> 1. build error resulted by including of nospec-branch.h file in V6
> - this common fixup patch had been sent to the linux-trace mail
> list and would show up in Linux v6.3
> - here is the link:
> https://lore.kernel.org/lkml/20230208015633.791198913@goodmis.org/
>
> 2. invalid code address reported by Evgenii in V7
> - for your convenience, the link to this updated patch is
> https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> Is it time to launch the V8 with this updated patch, or waiting for
> the common fixup patch to be merged?
I don't know.
I've made v8 based on the newest remotes/palmer/for-next with your
newest branch:
https://github.com/guoren83/linux/tree/ftrace_fixup_v8
I fixed some conflicts with the palmer/for-next, You could send them any time.
>
> > Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> > the first three patches onto for-next as they seem like pretty
> > independent fixes/cleanups, I'll hold off on the others until v8.
> >
> > Thanks!
> >
> > >
> > > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > > <e.shatokhin@yadro.com> wrote:
> > >>
> > >> On 18.01.2023 05:37, Song Shuai wrote:
> > >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> > >> >>
> > >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> > >> >>> Hi, Song,
> > >> >>>
> > >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> > >> >>>>
> > >> >>>> Hi, Evgenii:
> > >> >>>>
> > >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> > >> >>>>
> > >> >>>>>
> > >> >>>>> Hi,
> > >> >>>>>
> > >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> > >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> > >> >>>>>>
> > >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> > >> >>>>>>
> > >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> > >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > >> >>>>>> ---
> > >> >>>>>> arch/riscv/Kconfig | 2 ++
> > >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> > >> >>>>>> +++++++++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> > >> >>>>>> 6 files changed, 142 insertions(+)
> > >> >>>>>
> > >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> > >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> > >> >>>>>
> > >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> > >> >>>>> the messages in the trace:
> > >> >>>>>
> > >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> > >> >>>>> # | | | ||||| | |
> > >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000003334d00
> > >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> > >> >>>>> direct func1 ip 7fffffffffffffff
> > >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> [...]
> > >> >>>>>
> > >> >>>>> If I understand it right, my_direct_func1() should print the address of
> > >> >>>>> some location in the code, probably - at the beginning of the traced
> > >> >>>>> functions.
> > >> >>>>>
> > >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> > >> >>>>> addresses.
> > >> >>>>>
> > >> >>>> The invalid code address is only printed by accessing the schedule()
> > >> >>>> function's first argument whose address stores in a0 register.
> > >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> > >> >>>> just prints the a0 in the context of the schedule()'s caller and
> > >> >>>> the address maybe varies depending on the caller.
> > >> >>>>
> > >> >>>> I can't really understand why tracing the first argument of the
> > >> >>>> schedule() function, but it seems nonsense at this point.
> > >> >>>
> > >> >>> The question is, what should be passed as the argument(s) of
> > >> >>> my_direct_func() in this particular sample module. The kernel docs and
> > >> >>> commit logs seem to contain no info on that.
> > >> >>>
> > >> >>> With direct functions, I suppose, the trampoline can pass anything it
> > >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> > >> >>>
> > >> >>> I'd check what these sample modules do on x86 and would try to match
> > >> >>> that behaviour on RISC-V.
> > >> >>
> > >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> > >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> > >> >> be passed to my_direct_func().
> > >> >>
> > >> >> ftrace-direct-multi.ko:
> > >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> > >> >> # | | | ||||| | |
> > >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> > >> >> ffffffff87332f45 // wake_up_process+0x5
> > >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> > >> >> ffffffff8828d935 // schedule+0x5
> > >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> > >> >> ffffffff87332f45
> > >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> > >> >> ffffffff87332f45
> > >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> [...]
> > >> >>
> > >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> > >> >> where the execution of the traced functions resumes after the Ftrace
> > >> >> trampoline has finished.
> > >> >>
> > >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> > >> >>
> > >> >> The samples look like a demonstration, that one can pass anything
> > >> >> necessary to the handler in case of "direct" functions.
> > >> >>
> > >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> > >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> > >> >
> > >> > Yes, you're right.
> > >> >
> > >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> > >> > traced function with mcount offset.
> > >> >
> > >> > Here is the updated patch for your reference.
> > >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> > >>
> > >> Thank you for the quick fix. This one looks good to me.
> > >>
> > >> ftrace-direct-multi*.ko now report the ip values corresponding to
> > >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> > >>
> > >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> > >> your patch, becase this is what I actually did: tested it and reported
> > >> the results. I cannot take your credit for development of this patch ;-)
> > >>
> > >> Looking forward for v8 of the series.
> > >> >
> > >> >
> > >> >>
> > >> >>>
> > >> >>>>
> > >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> > >> >>>> trace kernel functions, and basically saves the necessary ABI,
> > >> >>>> call the tracing function, and restores the ABI, just like other
> > >> >>>> arches do.
> > >> >>>> so It shouldn't be blamed.
> > >> >>>>
> > >> >>>> I started an independent patch to replace schedule with kick_process
> > >> >>>> to make these samples more reasonable. And It has no conflict with the
> > >> >>>> current patch, so we can go on.
> > >> >>>>
> > >> >>>> Link:
> > >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> > >> >>>>
> > >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> > >> >>>>>
> > >> >>>>> Is anything missing here?
> > >> >>>>>
> > >> >>>>>>
> > >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > >> >>>>>> index 307a9f413edd..e944af44f681 100644
> > >> >>>>>> --- a/arch/riscv/Kconfig
> > >> >>>>>> +++ b/arch/riscv/Kconfig
> > >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> > >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> > >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> > >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> > >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> > >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > >> >>>>>> select HAVE_STACKPROTECTOR
> > >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> > >> >>>>>> select HAVE_RSEQ
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> > >> >>>>>>
> > >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp1, @function\n"
> > >> >>>>>> +" .globl my_tramp1\n"
> > >> >>>>>> +" my_tramp1:\n"
> > >> >>>>>> +" addi sp,sp,-16\n"
> > >> >>>>>> +" sd t0,0(sp)\n"
> > >> >>>>>> +" sd ra,8(sp)\n"
> > >> >>>>>> +" call my_direct_func1\n"
> > >> >>>>>> +" ld t0,0(sp)\n"
> > >> >>>>>> +" ld ra,8(sp)\n"
> > >> >>>>>> +" addi sp,sp,16\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> > >> >>>>>> +
> > >> >>>>>> +" .type my_tramp2, @function\n"
> > >> >>>>>> +" .globl my_tramp2\n"
> > >> >>>>>> +" my_tramp2:\n"
> > >> >>>>>> +" addi sp,sp,-16\n"
> > >> >>>>>> +" sd t0,0(sp)\n"
> > >> >>>>>> +" sd ra,8(sp)\n"
> > >> >>>>>> +" call my_direct_func2\n"
> > >> >>>>>> +" ld t0,0(sp)\n"
> > >> >>>>>> +" ld ra,8(sp)\n"
> > >> >>>>>> +" addi sp,sp,16\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> > >> >>>>>> extern void my_tramp1(void *);
> > >> >>>>>> extern void my_tramp2(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp1, @function\n"
> > >> >>>>>> +" .globl my_tramp1\n"
> > >> >>>>>> +" my_tramp1:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func1\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> > >> >>>>>> +
> > >> >>>>>> +" .type my_tramp2, @function\n"
> > >> >>>>>> +" .globl my_tramp2\n"
> > >> >>>>>> +" my_tramp2:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func2\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-40\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd a1,8(sp)\n"
> > >> >>>>>> +" sd a2,16(sp)\n"
> > >> >>>>>> +" sd t0,24(sp)\n"
> > >> >>>>>> +" sd ra,32(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld a1,8(sp)\n"
> > >> >>>>>> +" ld a2,16(sp)\n"
> > >> >>>>>> +" ld t0,24(sp)\n"
> > >> >>>>>> +" ld ra,32(sp)\n"
> > >> >>>>>> +" addi sp,sp,40\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct.c
> > >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> > >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> --
> > >> >>>>>> 2.36.1
> > >> >>>>
> > >> >>>> --
> > >> >>>> Thanks,
> > >> >>>> Song
> > >> >>>>
> > >> >>>
> > >> >>> Regards,
> > >> >>> Evgenii
> > >> >>
> > >> >>
> > >> >
> > >> >
> > >> > --
> > >> > Thanks,
> > >> > Song
> > >> >
> > >>
> > >>
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
>
>
>
> --
> Thanks,
> Song
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-02-21 3:56 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-21 3:56 UTC (permalink / raw)
To: Song Shuai
Cc: Palmer Dabbelt, e.shatokhin, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Mon, Feb 20, 2023 at 10:46 AM Song Shuai <suagrfillet@gmail.com> wrote:
>
> Hi,Guo:
>
> Palmer Dabbelt <palmer@dabbelt.com> 于2023年2月18日周六 21:30写道：
> >
> > On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > > Thx Evgenii & Song,
> > >
> > > I got it; it would be put into v8.
> >
> There were 2 problems for this patch in its V6 and V7 version:
>
> 1. build error resulted by including of nospec-branch.h file in V6
> - this common fixup patch had been sent to the linux-trace mail
> list and would show up in Linux v6.3
> - here is the link:
> https://lore.kernel.org/lkml/20230208015633.791198913@goodmis.org/
>
> 2. invalid code address reported by Evgenii in V7
> - for your convenience, the link to this updated patch is
> https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
>
> Is it time to launch the V8 with this updated patch, or waiting for
> the common fixup patch to be merged?
I don't know.
I've made v8 based on the newest remotes/palmer/for-next with your
newest branch:
https://github.com/guoren83/linux/tree/ftrace_fixup_v8
I fixed some conflicts with the palmer/for-next, You could send them any time.
>
> > Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> > the first three patches onto for-next as they seem like pretty
> > independent fixes/cleanups, I'll hold off on the others until v8.
> >
> > Thanks!
> >
> > >
> > > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > > <e.shatokhin@yadro.com> wrote:
> > >>
> > >> On 18.01.2023 05:37, Song Shuai wrote:
> > >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> > >> >>
> > >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> > >> >>> Hi, Song,
> > >> >>>
> > >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> > >> >>>>
> > >> >>>> Hi, Evgenii:
> > >> >>>>
> > >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> > >> >>>>
> > >> >>>>>
> > >> >>>>> Hi,
> > >> >>>>>
> > >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> > >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> > >> >>>>>>
> > >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> > >> >>>>>>
> > >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> > >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > >> >>>>>> ---
> > >> >>>>>> arch/riscv/Kconfig | 2 ++
> > >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> > >> >>>>>> +++++++++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> > >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> > >> >>>>>> 6 files changed, 142 insertions(+)
> > >> >>>>>
> > >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> > >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> > >> >>>>>
> > >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> > >> >>>>> the messages in the trace:
> > >> >>>>>
> > >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> > >> >>>>> # | | | ||||| | |
> > >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000003334d00
> > >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> > >> >>>>> direct func1 ip 7fffffffffffffff
> > >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> > >> >>>>> direct func1 ip ff60000001ba9600
> > >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> > >> >>>>> direct func1 ip 0
> > >> >>>>> [...]
> > >> >>>>>
> > >> >>>>> If I understand it right, my_direct_func1() should print the address of
> > >> >>>>> some location in the code, probably - at the beginning of the traced
> > >> >>>>> functions.
> > >> >>>>>
> > >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> > >> >>>>> addresses.
> > >> >>>>>
> > >> >>>> The invalid code address is only printed by accessing the schedule()
> > >> >>>> function's first argument whose address stores in a0 register.
> > >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> > >> >>>> just prints the a0 in the context of the schedule()'s caller and
> > >> >>>> the address maybe varies depending on the caller.
> > >> >>>>
> > >> >>>> I can't really understand why tracing the first argument of the
> > >> >>>> schedule() function, but it seems nonsense at this point.
> > >> >>>
> > >> >>> The question is, what should be passed as the argument(s) of
> > >> >>> my_direct_func() in this particular sample module. The kernel docs and
> > >> >>> commit logs seem to contain no info on that.
> > >> >>>
> > >> >>> With direct functions, I suppose, the trampoline can pass anything it
> > >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> > >> >>>
> > >> >>> I'd check what these sample modules do on x86 and would try to match
> > >> >>> that behaviour on RISC-V.
> > >> >>
> > >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> > >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> > >> >> be passed to my_direct_func().
> > >> >>
> > >> >> ftrace-direct-multi.ko:
> > >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> > >> >> # | | | ||||| | |
> > >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> > >> >> ffffffff87332f45 // wake_up_process+0x5
> > >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> > >> >> ffffffff8828d935 // schedule+0x5
> > >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> > >> >> ffffffff87332f45
> > >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> > >> >> ffffffff87332f45
> > >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> > >> >> ffffffff8828d935
> > >> >> [...]
> > >> >>
> > >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> > >> >> where the execution of the traced functions resumes after the Ftrace
> > >> >> trampoline has finished.
> > >> >>
> > >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> > >> >>
> > >> >> The samples look like a demonstration, that one can pass anything
> > >> >> necessary to the handler in case of "direct" functions.
> > >> >>
> > >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> > >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> > >> >
> > >> > Yes, you're right.
> > >> >
> > >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> > >> > traced function with mcount offset.
> > >> >
> > >> > Here is the updated patch for your reference.
> > >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> > >>
> > >> Thank you for the quick fix. This one looks good to me.
> > >>
> > >> ftrace-direct-multi*.ko now report the ip values corresponding to
> > >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> > >>
> > >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> > >> your patch, becase this is what I actually did: tested it and reported
> > >> the results. I cannot take your credit for development of this patch ;-)
> > >>
> > >> Looking forward for v8 of the series.
> > >> >
> > >> >
> > >> >>
> > >> >>>
> > >> >>>>
> > >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> > >> >>>> trace kernel functions, and basically saves the necessary ABI,
> > >> >>>> call the tracing function, and restores the ABI, just like other
> > >> >>>> arches do.
> > >> >>>> so It shouldn't be blamed.
> > >> >>>>
> > >> >>>> I started an independent patch to replace schedule with kick_process
> > >> >>>> to make these samples more reasonable. And It has no conflict with the
> > >> >>>> current patch, so we can go on.
> > >> >>>>
> > >> >>>> Link:
> > >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> > >> >>>>
> > >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> > >> >>>>>
> > >> >>>>> Is anything missing here?
> > >> >>>>>
> > >> >>>>>>
> > >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > >> >>>>>> index 307a9f413edd..e944af44f681 100644
> > >> >>>>>> --- a/arch/riscv/Kconfig
> > >> >>>>>> +++ b/arch/riscv/Kconfig
> > >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> > >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> > >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> > >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> > >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> > >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > >> >>>>>> select HAVE_STACKPROTECTOR
> > >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> > >> >>>>>> select HAVE_RSEQ
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> > >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> > >> >>>>>>
> > >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp1, @function\n"
> > >> >>>>>> +" .globl my_tramp1\n"
> > >> >>>>>> +" my_tramp1:\n"
> > >> >>>>>> +" addi sp,sp,-16\n"
> > >> >>>>>> +" sd t0,0(sp)\n"
> > >> >>>>>> +" sd ra,8(sp)\n"
> > >> >>>>>> +" call my_direct_func1\n"
> > >> >>>>>> +" ld t0,0(sp)\n"
> > >> >>>>>> +" ld ra,8(sp)\n"
> > >> >>>>>> +" addi sp,sp,16\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> > >> >>>>>> +
> > >> >>>>>> +" .type my_tramp2, @function\n"
> > >> >>>>>> +" .globl my_tramp2\n"
> > >> >>>>>> +" my_tramp2:\n"
> > >> >>>>>> +" addi sp,sp,-16\n"
> > >> >>>>>> +" sd t0,0(sp)\n"
> > >> >>>>>> +" sd ra,8(sp)\n"
> > >> >>>>>> +" call my_direct_func2\n"
> > >> >>>>>> +" ld t0,0(sp)\n"
> > >> >>>>>> +" ld ra,8(sp)\n"
> > >> >>>>>> +" addi sp,sp,16\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> > >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> > >> >>>>>> extern void my_tramp1(void *);
> > >> >>>>>> extern void my_tramp2(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp1, @function\n"
> > >> >>>>>> +" .globl my_tramp1\n"
> > >> >>>>>> +" my_tramp1:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func1\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> > >> >>>>>> +
> > >> >>>>>> +" .type my_tramp2, @function\n"
> > >> >>>>>> +" .globl my_tramp2\n"
> > >> >>>>>> +" my_tramp2:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func2\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> > >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> > >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-40\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd a1,8(sp)\n"
> > >> >>>>>> +" sd a2,16(sp)\n"
> > >> >>>>>> +" sd t0,24(sp)\n"
> > >> >>>>>> +" sd ra,32(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld a1,8(sp)\n"
> > >> >>>>>> +" ld a2,16(sp)\n"
> > >> >>>>>> +" ld t0,24(sp)\n"
> > >> >>>>>> +" ld ra,32(sp)\n"
> > >> >>>>>> +" addi sp,sp,40\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> > >> >>>>>> b/samples/ftrace/ftrace-direct.c
> > >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> > >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> > >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> > >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> > >> >>>>>>
> > >> >>>>>> extern void my_tramp(void *);
> > >> >>>>>>
> > >> >>>>>> +#ifdef CONFIG_RISCV
> > >> >>>>>> +
> > >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> > >> >>>>>> +" .type my_tramp, @function\n"
> > >> >>>>>> +" .globl my_tramp\n"
> > >> >>>>>> +" my_tramp:\n"
> > >> >>>>>> +" addi sp,sp,-24\n"
> > >> >>>>>> +" sd a0,0(sp)\n"
> > >> >>>>>> +" sd t0,8(sp)\n"
> > >> >>>>>> +" sd ra,16(sp)\n"
> > >> >>>>>> +" call my_direct_func\n"
> > >> >>>>>> +" ld a0,0(sp)\n"
> > >> >>>>>> +" ld t0,8(sp)\n"
> > >> >>>>>> +" ld ra,16(sp)\n"
> > >> >>>>>> +" addi sp,sp,24\n"
> > >> >>>>>> +" jr t0\n"
> > >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> > >> >>>>>> +" .popsection\n"
> > >> >>>>>> +);
> > >> >>>>>> +
> > >> >>>>>> +#endif /* CONFIG_RISCV */
> > >> >>>>>> +
> > >> >>>>>> #ifdef CONFIG_X86_64
> > >> >>>>>>
> > >> >>>>>> #include <asm/ibt.h>
> > >> >>>>>> --
> > >> >>>>>> 2.36.1
> > >> >>>>
> > >> >>>> --
> > >> >>>> Thanks,
> > >> >>>> Song
> > >> >>>>
> > >> >>>
> > >> >>> Regards,
> > >> >>> Evgenii
> > >> >>
> > >> >>
> > >> >
> > >> >
> > >> > --
> > >> > Thanks,
> > >> > Song
> > >> >
> > >>
> > >>
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
>
>
>
> --
> Thanks,
> Song
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
2023-02-18 21:30 ` Palmer Dabbelt
@ 2023-02-21 4:02 ` Guo Ren
-1 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-21 4:02 UTC (permalink / raw)
To: Palmer Dabbelt
Cc: e.shatokhin, suagrfillet, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Sun, Feb 19, 2023 at 5:30 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > Thx Evgenii & Song,
> >
> > I got it; it would be put into v8.
>
> Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> the first three patches onto for-next as they seem like pretty
> independent fixes/cleanups, I'll hold off on the others until v8.
Thx for approving. I would keep the patch series more independent next time.
>
> Thanks!
>
> >
> > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> >>
> >> On 18.01.2023 05:37, Song Shuai wrote:
> >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >> >>
> >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >> >>> Hi, Song,
> >> >>>
> >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >> >>>>
> >> >>>> Hi, Evgenii:
> >> >>>>
> >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >> >>>>
> >> >>>>>
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>>
> >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >> >>>>>>
> >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> ---
> >> >>>>>> arch/riscv/Kconfig | 2 ++
> >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >> >>>>>> +++++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >> >>>>>> 6 files changed, 142 insertions(+)
> >> >>>>>
> >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >> >>>>>
> >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >> >>>>> the messages in the trace:
> >> >>>>>
> >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >>>>> # | | | ||||| | |
> >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000003334d00
> >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >> >>>>> direct func1 ip 7fffffffffffffff
> >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> [...]
> >> >>>>>
> >> >>>>> If I understand it right, my_direct_func1() should print the address of
> >> >>>>> some location in the code, probably - at the beginning of the traced
> >> >>>>> functions.
> >> >>>>>
> >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >> >>>>> addresses.
> >> >>>>>
> >> >>>> The invalid code address is only printed by accessing the schedule()
> >> >>>> function's first argument whose address stores in a0 register.
> >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >> >>>> just prints the a0 in the context of the schedule()'s caller and
> >> >>>> the address maybe varies depending on the caller.
> >> >>>>
> >> >>>> I can't really understand why tracing the first argument of the
> >> >>>> schedule() function, but it seems nonsense at this point.
> >> >>>
> >> >>> The question is, what should be passed as the argument(s) of
> >> >>> my_direct_func() in this particular sample module. The kernel docs and
> >> >>> commit logs seem to contain no info on that.
> >> >>>
> >> >>> With direct functions, I suppose, the trampoline can pass anything it
> >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >> >>>
> >> >>> I'd check what these sample modules do on x86 and would try to match
> >> >>> that behaviour on RISC-V.
> >> >>
> >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> >> be passed to my_direct_func().
> >> >>
> >> >> ftrace-direct-multi.ko:
> >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >> # | | | ||||| | |
> >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> >> ffffffff87332f45 // wake_up_process+0x5
> >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> >> ffffffff8828d935 // schedule+0x5
> >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> [...]
> >> >>
> >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> >> where the execution of the traced functions resumes after the Ftrace
> >> >> trampoline has finished.
> >> >>
> >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >> >>
> >> >> The samples look like a demonstration, that one can pass anything
> >> >> necessary to the handler in case of "direct" functions.
> >> >>
> >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >> >
> >> > Yes, you're right.
> >> >
> >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> >> > traced function with mcount offset.
> >> >
> >> > Here is the updated patch for your reference.
> >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> >>
> >> Thank you for the quick fix. This one looks good to me.
> >>
> >> ftrace-direct-multi*.ko now report the ip values corresponding to
> >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> >>
> >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> >> your patch, becase this is what I actually did: tested it and reported
> >> the results. I cannot take your credit for development of this patch ;-)
> >>
> >> Looking forward for v8 of the series.
> >> >
> >> >
> >> >>
> >> >>>
> >> >>>>
> >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >> >>>> trace kernel functions, and basically saves the necessary ABI,
> >> >>>> call the tracing function, and restores the ABI, just like other
> >> >>>> arches do.
> >> >>>> so It shouldn't be blamed.
> >> >>>>
> >> >>>> I started an independent patch to replace schedule with kick_process
> >> >>>> to make these samples more reasonable. And It has no conflict with the
> >> >>>> current patch, so we can go on.
> >> >>>>
> >> >>>> Link:
> >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >> >>>>
> >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >> >>>>>
> >> >>>>> Is anything missing here?
> >> >>>>>
> >> >>>>>>
> >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >> >>>>>> index 307a9f413edd..e944af44f681 100644
> >> >>>>>> --- a/arch/riscv/Kconfig
> >> >>>>>> +++ b/arch/riscv/Kconfig
> >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> select HAVE_STACKPROTECTOR
> >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >> >>>>>> select HAVE_RSEQ
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >> >>>>>> extern void my_tramp1(void *);
> >> >>>>>> extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-40\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd a1,8(sp)\n"
> >> >>>>>> +" sd a2,16(sp)\n"
> >> >>>>>> +" sd t0,24(sp)\n"
> >> >>>>>> +" sd ra,32(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld a1,8(sp)\n"
> >> >>>>>> +" ld a2,16(sp)\n"
> >> >>>>>> +" ld t0,24(sp)\n"
> >> >>>>>> +" ld ra,32(sp)\n"
> >> >>>>>> +" addi sp,sp,40\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >> >>>>>> b/samples/ftrace/ftrace-direct.c
> >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> --
> >> >>>>>> 2.36.1
> >> >>>>
> >> >>>> --
> >> >>>> Thanks,
> >> >>>> Song
> >> >>>>
> >> >>>
> >> >>> Regards,
> >> >>> Evgenii
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Song
> >> >
> >>
> >>
> >
> >
> > --
> > Best Regards
> > Guo Ren
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
@ 2023-02-21 4:02 ` Guo Ren
0 siblings, 0 replies; 88+ messages in thread
From: Guo Ren @ 2023-02-21 4:02 UTC (permalink / raw)
To: Palmer Dabbelt
Cc: e.shatokhin, suagrfillet, linux-riscv, linux-kernel, anup,
Paul Walmsley, Conor Dooley, heiko, rostedt, mhiramat, jolsa, bp,
jpoimboe, andy.chiu, linux
On Sun, Feb 19, 2023 at 5:30 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Wed, 18 Jan 2023 22:05:04 PST (-0800), guoren@kernel.org wrote:
> > Thx Evgenii & Song,
> >
> > I got it; it would be put into v8.
>
> Sorry if I missed it, but I don't see a v8 on patchwork. I queued up
> the first three patches onto for-next as they seem like pretty
> independent fixes/cleanups, I'll hold off on the others until v8.
Thx for approving. I would keep the patch series more independent next time.
>
> Thanks!
>
> >
> > On Wed, Jan 18, 2023 at 11:19 PM Evgenii Shatokhin
> > <e.shatokhin@yadro.com> wrote:
> >>
> >> On 18.01.2023 05:37, Song Shuai wrote:
> >> > Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月17日周二 16:22写道：
> >> >>
> >> >> On 17.01.2023 16:16, Evgenii Shatokhin wrote:
> >> >>> Hi, Song,
> >> >>>
> >> >>> On 17.01.2023 12:32, Song Shuai wrote:
> >> >>>>
> >> >>>> Hi, Evgenii:
> >> >>>>
> >> >>>> Evgenii Shatokhin <e.shatokhin@yadro.com> 于2023年1月16日周一 14:30写道：
> >> >>>>
> >> >>>>>
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> On 12.01.2023 12:06, guoren@kernel.org wrote:
> >> >>>>>> From: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>>
> >> >>>>>> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> >> >>>>>> the ftrace-direct*.c files in samples/ftrace/.
> >> >>>>>>
> >> >>>>>> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> >> >>>>>> Tested-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >> >>>>>> ---
> >> >>>>>> arch/riscv/Kconfig | 2 ++
> >> >>>>>> samples/ftrace/ftrace-direct-modify.c | 33 ++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi-modify.c | 37
> >> >>>>>> +++++++++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-multi.c | 22 ++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct-too.c | 26 +++++++++++++++
> >> >>>>>> samples/ftrace/ftrace-direct.c | 22 ++++++++++++
> >> >>>>>> 6 files changed, 142 insertions(+)
> >> >>>>>
> >> >>>>> The samples were built OK now, but ftrace-direct-multi and
> >> >>>>> ftrace-direct-multi-modify report incorrect values of ip/pc it seems.
> >> >>>>>
> >> >>>>> I ran 'insmod ftrace-direct-multi.ko', waited a little and then checked
> >> >>>>> the messages in the trace:
> >> >>>>>
> >> >>>>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >>>>> # | | | ||||| | |
> >> >>>>> migration/1-19 [001] ..... 3858.532131: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> migration/0-15 [000] d.s2. 3858.532136: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> migration/0-15 [000] d..2. 3858.532204: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000003334d00
> >> >>>>> migration/0-15 [000] ..... 3858.532232: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> rcu_sched-14 [001] ..... 3858.532257: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> insmod-415 [000] ..... 3858.532270: my_direct_func1: my
> >> >>>>> direct func1 ip 7fffffffffffffff
> >> >>>>> <idle>-0 [001] ..s1. 3858.539051: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> <idle>-0 [001] dns2. 3858.539124: my_direct_func1: my
> >> >>>>> direct func1 ip ff60000001ba9600
> >> >>>>> rcu_sched-14 [001] ..... 3858.539208: my_direct_func1: my
> >> >>>>> direct func1 ip 0
> >> >>>>> [...]
> >> >>>>>
> >> >>>>> If I understand it right, my_direct_func1() should print the address of
> >> >>>>> some location in the code, probably - at the beginning of the traced
> >> >>>>> functions.
> >> >>>>>
> >> >>>>> The printed values (0x0, 0x7fffffffffffffff, ...) are not valid code
> >> >>>>> addresses.
> >> >>>>>
> >> >>>> The invalid code address is only printed by accessing the schedule()
> >> >>>> function's first argument whose address stores in a0 register.
> >> >>>> While schedule() actually has no parameter declared, so my_direct_func
> >> >>>> just prints the a0 in the context of the schedule()'s caller and
> >> >>>> the address maybe varies depending on the caller.
> >> >>>>
> >> >>>> I can't really understand why tracing the first argument of the
> >> >>>> schedule() function, but it seems nonsense at this point.
> >> >>>
> >> >>> The question is, what should be passed as the argument(s) of
> >> >>> my_direct_func() in this particular sample module. The kernel docs and
> >> >>> commit logs seem to contain no info on that.
> >> >>>
> >> >>> With direct functions, I suppose, the trampoline can pass anything it
> >> >>> wants to my_direct_func(), not just the arguments of the traced function.
> >> >>>
> >> >>> I'd check what these sample modules do on x86 and would try to match
> >> >>> that behaviour on RISC-V.
> >> >>
> >> >> I have checked ftrace-direct-multi.ko and ftrace-direct-multi-modify.ko
> >> >> on 6.2-rc4 built for x86-64 - yes, ip/pc in the traced function should
> >> >> be passed to my_direct_func().
> >> >>
> >> >> ftrace-direct-multi.ko:
> >> >> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> >> >> # | | | ||||| | |
> >> >> insmod-10829 [000] d.h1. 1719.518535: my_direct_func: ip
> >> >> ffffffff87332f45 // wake_up_process+0x5
> >> >> rcu_tasks_kthre-11 [000] ..... 1719.518696: my_direct_func: ip
> >> >> ffffffff8828d935 // schedule+0x5
> >> >> insmod-10829 [000] ..... 1719.518708: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-journal-293 [001] ..... 1719.518823: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> systemd-1 [000] ..... 1719.519141: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> <idle>-0 [001] ..s1. 1719.521889: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> <idle>-0 [000] d.s2. 1719.521901: my_direct_func: ip
> >> >> ffffffff87332f45
> >> >> rcu_preempt-15 [001] ..... 1719.521917: my_direct_func: ip
> >> >> ffffffff8828d935
> >> >> [...]
> >> >>
> >> >> The ip values are wake_up_process+0x5 and schedule+0x5, the locations
> >> >> where the execution of the traced functions resumes after the Ftrace
> >> >> trampoline has finished.
> >> >>
> >> >> The results with ftrace-direct-multi-modify.ko are similar to that.
> >> >>
> >> >> The samples look like a demonstration, that one can pass anything
> >> >> necessary to the handler in case of "direct" functions.
> >> >>
> >> >> I suppose, the RISC-V-specific asm code in these two sample modules
> >> >> could be updated to pass the saved pc value to my_direct_func() in a0.
> >> >
> >> > Yes, you're right.
> >> >
> >> > I added 'mv a0,t0' in front of `call my_direct_func` to pass the address of
> >> > traced function with mcount offset.
> >> >
> >> > Here is the updated patch for your reference.
> >> > https://github.com/sugarfillet/linux/commit/95b174fb104dd970b982ee6fa19879393e229318
> >>
> >> Thank you for the quick fix. This one looks good to me.
> >>
> >> ftrace-direct-multi*.ko now report the ip values corresponding to
> >> schedule+0x8 and wake_up_process+0x8, which is what was expected here.
> >>
> >> One more thing: please change my "Co-developed-by:" into "Tested-by:" in
> >> your patch, becase this is what I actually did: tested it and reported
> >> the results. I cannot take your credit for development of this patch ;-)
> >>
> >> Looking forward for v8 of the series.
> >> >
> >> >
> >> >>
> >> >>>
> >> >>>>
> >> >>>> As for this patch, it just impls a simple mcount (direct_caller) to
> >> >>>> trace kernel functions, and basically saves the necessary ABI,
> >> >>>> call the tracing function, and restores the ABI, just like other
> >> >>>> arches do.
> >> >>>> so It shouldn't be blamed.
> >> >>>>
> >> >>>> I started an independent patch to replace schedule with kick_process
> >> >>>> to make these samples more reasonable. And It has no conflict with the
> >> >>>> current patch, so we can go on.
> >> >>>>
> >> >>>> Link:
> >> >>>> https://lore.kernel.org/linux-kernel/20230117091101.3669996-1-suagrfillet@gmail.com/T/#u
> >> >>>>
> >> >>>>> The same issue is with ftrace-direct-multi-modify.ko.
> >> >>>>>
> >> >>>>> Is anything missing here?
> >> >>>>>
> >> >>>>>>
> >> >>>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >> >>>>>> index 307a9f413edd..e944af44f681 100644
> >> >>>>>> --- a/arch/riscv/Kconfig
> >> >>>>>> +++ b/arch/riscv/Kconfig
> >> >>>>>> @@ -112,6 +112,8 @@ config RISCV
> >> >>>>>> select HAVE_POSIX_CPU_TIMERS_TASK_WORK
> >> >>>>>> select HAVE_REGS_AND_STACK_ACCESS_API
> >> >>>>>> select HAVE_FUNCTION_ARG_ACCESS_API
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT
> >> >>>>>> + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >> >>>>>> select HAVE_STACKPROTECTOR
> >> >>>>>> select HAVE_SYSCALL_TRACEPOINTS
> >> >>>>>> select HAVE_RSEQ
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> index de5a0f67f320..be7bf472c3c7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-modify.c
> >> >>>>>> @@ -23,6 +23,39 @@ extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> static unsigned long my_ip = (unsigned long)schedule;
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-16\n"
> >> >>>>>> +" sd t0,0(sp)\n"
> >> >>>>>> +" sd ra,8(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld t0,0(sp)\n"
> >> >>>>>> +" ld ra,8(sp)\n"
> >> >>>>>> +" addi sp,sp,16\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> index d52370cad0b6..10884bf418f7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi-modify.c
> >> >>>>>> @@ -21,6 +21,43 @@ void my_direct_func2(unsigned long ip)
> >> >>>>>> extern void my_tramp1(void *);
> >> >>>>>> extern void my_tramp2(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp1, @function\n"
> >> >>>>>> +" .globl my_tramp1\n"
> >> >>>>>> +" my_tramp1:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func1\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp1, .-my_tramp1\n"
> >> >>>>>> +
> >> >>>>>> +" .type my_tramp2, @function\n"
> >> >>>>>> +" .globl my_tramp2\n"
> >> >>>>>> +" my_tramp2:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func2\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp2, .-my_tramp2\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> index ec1088922517..a35bf43bf6d7 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-multi.c
> >> >>>>>> @@ -16,6 +16,28 @@ void my_direct_func(unsigned long ip)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> index e13fb59a2b47..3b62e33c2e6d 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct-too.c
> >> >>>>>> @@ -18,6 +18,32 @@ void my_direct_func(struct vm_area_struct *vma,
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-40\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd a1,8(sp)\n"
> >> >>>>>> +" sd a2,16(sp)\n"
> >> >>>>>> +" sd t0,24(sp)\n"
> >> >>>>>> +" sd ra,32(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld a1,8(sp)\n"
> >> >>>>>> +" ld a2,16(sp)\n"
> >> >>>>>> +" ld t0,24(sp)\n"
> >> >>>>>> +" ld ra,32(sp)\n"
> >> >>>>>> +" addi sp,sp,40\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> diff --git a/samples/ftrace/ftrace-direct.c
> >> >>>>>> b/samples/ftrace/ftrace-direct.c
> >> >>>>>> index 1f769d0db20f..2cfe5a7d2d70 100644
> >> >>>>>> --- a/samples/ftrace/ftrace-direct.c
> >> >>>>>> +++ b/samples/ftrace/ftrace-direct.c
> >> >>>>>> @@ -15,6 +15,28 @@ void my_direct_func(struct task_struct *p)
> >> >>>>>>
> >> >>>>>> extern void my_tramp(void *);
> >> >>>>>>
> >> >>>>>> +#ifdef CONFIG_RISCV
> >> >>>>>> +
> >> >>>>>> +asm (" .pushsection .text, \"ax\", @progbits\n"
> >> >>>>>> +" .type my_tramp, @function\n"
> >> >>>>>> +" .globl my_tramp\n"
> >> >>>>>> +" my_tramp:\n"
> >> >>>>>> +" addi sp,sp,-24\n"
> >> >>>>>> +" sd a0,0(sp)\n"
> >> >>>>>> +" sd t0,8(sp)\n"
> >> >>>>>> +" sd ra,16(sp)\n"
> >> >>>>>> +" call my_direct_func\n"
> >> >>>>>> +" ld a0,0(sp)\n"
> >> >>>>>> +" ld t0,8(sp)\n"
> >> >>>>>> +" ld ra,16(sp)\n"
> >> >>>>>> +" addi sp,sp,24\n"
> >> >>>>>> +" jr t0\n"
> >> >>>>>> +" .size my_tramp, .-my_tramp\n"
> >> >>>>>> +" .popsection\n"
> >> >>>>>> +);
> >> >>>>>> +
> >> >>>>>> +#endif /* CONFIG_RISCV */
> >> >>>>>> +
> >> >>>>>> #ifdef CONFIG_X86_64
> >> >>>>>>
> >> >>>>>> #include <asm/ibt.h>
> >> >>>>>> --
> >> >>>>>> 2.36.1
> >> >>>>
> >> >>>> --
> >> >>>> Thanks,
> >> >>>> Song
> >> >>>>
> >> >>>
> >> >>> Regards,
> >> >>> Evgenii
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Song
> >> >
> >>
> >>
> >
> >
> > --
> > Best Regards
> > Guo Ren
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 88+ messages in thread
end of thread, other threads:[~2023-02-21 4:02 UTC | newest]
Thread overview: 88+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-12 9:05 [PATCH -next V7 0/7] riscv: Optimize function trace guoren
2023-01-12 9:05 ` guoren
2023-01-12 9:05 ` [PATCH -next V7 1/7] riscv: ftrace: Fixup panic by disabling preemption guoren
2023-01-12 9:05 ` guoren
2023-01-12 12:16 ` Mark Rutland
2023-01-12 12:16 ` Mark Rutland
2023-01-12 12:57 ` Mark Rutland
2023-01-12 12:57 ` Mark Rutland
2023-01-28 9:45 ` Guo Ren
2023-01-28 9:45 ` Guo Ren
2023-01-28 9:37 ` Guo Ren
2023-01-28 9:37 ` Guo Ren
2023-01-30 10:54 ` Mark Rutland
2023-01-30 10:54 ` Mark Rutland
2023-02-04 1:19 ` Guo Ren
2023-02-04 1:19 ` Guo Ren
2023-01-12 9:05 ` [PATCH -next V7 2/7] riscv: ftrace: Remove wasted nops for !RISCV_ISA_C guoren
2023-01-12 9:05 ` guoren
2023-01-12 9:05 ` [PATCH -next V7 3/7] riscv: ftrace: Reduce the detour code size to half guoren
2023-01-12 9:05 ` guoren
2023-01-16 14:11 ` Evgenii Shatokhin
2023-01-16 14:11 ` Evgenii Shatokhin
2023-01-12 9:06 ` [PATCH -next V7 4/7] riscv: ftrace: Add ftrace_graph_func guoren
2023-01-12 9:06 ` guoren
2023-01-12 9:06 ` [PATCH -next V7 5/7] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support guoren
2023-01-12 9:06 ` guoren
2023-01-12 9:06 ` [PATCH -next V7 6/7] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI] guoren
2023-01-12 9:06 ` guoren
2023-01-16 14:30 ` Evgenii Shatokhin
2023-01-16 14:30 ` Evgenii Shatokhin
2023-01-17 9:32 ` Song Shuai
2023-01-17 9:32 ` Song Shuai
2023-01-17 13:16 ` Evgenii Shatokhin
2023-01-17 13:16 ` Evgenii Shatokhin
2023-01-17 16:22 ` Evgenii Shatokhin
2023-01-17 16:22 ` Evgenii Shatokhin
2023-01-18 2:37 ` Song Shuai
2023-01-18 2:37 ` Song Shuai
2023-01-18 15:19 ` Evgenii Shatokhin
2023-01-18 15:19 ` Evgenii Shatokhin
2023-01-19 6:05 ` Guo Ren
2023-01-19 6:05 ` Guo Ren
2023-02-18 21:30 ` Palmer Dabbelt
2023-02-18 21:30 ` Palmer Dabbelt
2023-02-20 2:46 ` Song Shuai
2023-02-20 2:46 ` Song Shuai
2023-02-21 3:56 ` Guo Ren
2023-02-21 3:56 ` Guo Ren
2023-02-21 4:02 ` Guo Ren
2023-02-21 4:02 ` Guo Ren
2023-01-12 9:06 ` [PATCH -next V7 7/7] riscv : select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY guoren
2023-01-12 9:06 ` guoren
2023-01-16 15:02 ` [PATCH -next V7 0/7] riscv: Optimize function trace Evgenii Shatokhin
2023-01-16 15:02 ` Evgenii Shatokhin
2023-02-04 6:40 ` Guo Ren
2023-02-04 6:40 ` Guo Ren
2023-02-06 9:56 ` Mark Rutland
2023-02-06 9:56 ` Mark Rutland
2023-02-07 3:57 ` Guo Ren
2023-02-07 3:57 ` Guo Ren
2023-02-07 9:16 ` Mark Rutland
2023-02-07 9:16 ` Mark Rutland
2023-02-08 2:30 ` Guo Ren
2023-02-08 2:30 ` Guo Ren
2023-02-08 14:46 ` Mark Rutland
2023-02-08 14:46 ` Mark Rutland
2023-02-09 1:31 ` Guo Ren
2023-02-09 1:31 ` Guo Ren
2023-02-09 22:46 ` David Laight
2023-02-09 22:46 ` David Laight
2023-02-10 2:18 ` Guo Ren
2023-02-10 2:18 ` Guo Ren
2023-02-08 22:29 ` David Laight
2023-02-08 22:29 ` David Laight
2023-02-09 1:51 ` Guo Ren
2023-02-09 1:51 ` Guo Ren
2023-02-09 1:59 ` Guo Ren
2023-02-09 1:59 ` Guo Ren
2023-02-09 9:54 ` Mark Rutland
2023-02-09 9:54 ` Mark Rutland
2023-02-10 2:21 ` Guo Ren
2023-02-10 2:21 ` Guo Ren
2023-02-09 9:00 ` David Laight
2023-02-09 9:00 ` David Laight
2023-02-09 9:11 ` Guo Ren
2023-02-09 9:11 ` Guo Ren
2023-02-18 21:42 ` patchwork-bot+linux-riscv
2023-02-18 21:42 ` patchwork-bot+linux-riscv
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.