* [PATCH v6 0/9] LoongArch: Add ftrace support
@ 2022-11-15 2:55 Qing Zhang
2022-11-15 2:55 ` [PATCH v6 1/9] LoongArch/ftrace: Add basic support Qing Zhang
` (8 more replies)
0 siblings, 9 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
This patch series to support basic and dynamic ftrace.
1) -pg
Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
Thus, it can be seen store RA and open stack before `call _mcount`.
Find `open stack` at first, and then find `store RA`.
2) -fpatchable-function-entry=2
The compiler has inserted 2 NOPs before the regular function prologue.
T series registers are available and safe because of LoongArch psABI.
At runtime, replace nop with bl to enable ftrace call and replace bl with
nop to disable ftrace call. The bl requires us to save the original RA value,
so here it saves RA at t0.
details are:
| Compiled | Disabled | Enabled |
+------------+------------------------+------------------------+
| nop | move t0, ra | move t0, ra |
| nop | nop | bl ftrace_caller |
| func_body | func_body | func_body |
The RA value will be recovered by ftrace_regs_entry, and restored into RA
before returning to the regular function prologue. When a function is not
being traced, the move t0, ra is not harmful.
performs a series of startup tests on ftrace and The test cases in selftests
has passed on LoongArch.
Changes in v2:
- Remove patch "LoongArch: ftrace: Add CALLER_ADDRx macros" there are other
better ways
Suggested by Steve:
- Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support (6/10)
Suggested by Jinyang:
- Change addu16id to lu12iw and Adjust module_finalize return value (7/10)
- Use the "jr" pseudo-instruction where applicable (1/10)
- Use the "la.pcrel" instead of "la" (3/10)
Changes in v3:
Reported by Jeff:
- Fix unwind state when option func_stack_trace (10/10)
Changes in v4:
- No comments. Just resend the series.
- Rebased onto v6.0.0-rc4.
Changes in v5:
- Modify the indentation of Kconfig and small changes
Changes in v6:
Suggested by Huacai:
- Adjusting the patch Sequence
- renamed mcount-dyn.S for consistency
Qing Zhang (9):
LoongArch/ftrace: Add basic support
LoongArch/ftrace: Add recordmcount support
LoongArch/ftrace: Add dynamic function tracer support
LoongArch/ftrace: Add dynamic function graph tracer support
LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
LoongArch/ftrace: Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support
LoongArch/ftrace: Add HAVE_FUNCTION_GRAPH_RET_ADDR_PTR support
LoongArch: modules/ftrace: Initialize PLT at load time
LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS
arch/loongarch/Kconfig | 7 +
arch/loongarch/Makefile | 5 +
arch/loongarch/configs/loongson3_defconfig | 2 +
arch/loongarch/include/asm/ftrace.h | 61 +++++
arch/loongarch/include/asm/inst.h | 18 ++
arch/loongarch/include/asm/module.h | 5 +-
arch/loongarch/include/asm/module.lds.h | 1 +
arch/loongarch/include/asm/unwind.h | 3 +-
arch/loongarch/kernel/Makefile | 13 +
arch/loongarch/kernel/ftrace.c | 74 ++++++
arch/loongarch/kernel/ftrace_dyn.c | 264 +++++++++++++++++++++
arch/loongarch/kernel/inst.c | 127 ++++++++++
arch/loongarch/kernel/mcount-dyn.S | 154 ++++++++++++
arch/loongarch/kernel/mcount.S | 94 ++++++++
arch/loongarch/kernel/module-sections.c | 11 +
arch/loongarch/kernel/module.c | 22 ++
arch/loongarch/kernel/unwind_guess.c | 4 +-
arch/loongarch/kernel/unwind_prologue.c | 46 +++-
scripts/recordmcount.c | 23 ++
19 files changed, 924 insertions(+), 10 deletions(-)
create mode 100644 arch/loongarch/include/asm/ftrace.h
create mode 100644 arch/loongarch/kernel/ftrace.c
create mode 100644 arch/loongarch/kernel/ftrace_dyn.c
create mode 100644 arch/loongarch/kernel/mcount-dyn.S
create mode 100644 arch/loongarch/kernel/mcount.S
--
2.36.0
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v6 1/9] LoongArch/ftrace: Add basic support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 8:25 ` Huacai Chen
2022-11-15 2:55 ` [PATCH v6 2/9] LoongArch/ftrace: Add recordmcount support Qing Zhang
` (7 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar
Cc: loongarch, linux-kernel, Jinyang He
This patch contains basic ftrace support for LoongArch.
Specifically, function tracer (HAVE_FUNCTION_TRACER), function graph
tracer (HAVE_FUNCTION_GRAPH_TRACER) are implemented following the
instructions in Documentation/trace/ftrace-design.txt.
Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
Thus, it can be seen store RA and open stack before `call _mcount`.
Find `open stack` at first, and then find `store RA`
Note that the functions in both inst.c and time.c should not be
hooked with the compiler's -pg option: to prevent infinite self-
referencing for the former, and to ignore early setup stuff for the
latter.
Co-developed-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/Kconfig | 2 +
arch/loongarch/Makefile | 5 ++
arch/loongarch/include/asm/ftrace.h | 18 ++++++
arch/loongarch/kernel/Makefile | 8 +++
arch/loongarch/kernel/ftrace.c | 74 +++++++++++++++++++++++
arch/loongarch/kernel/mcount.S | 94 +++++++++++++++++++++++++++++
6 files changed, 201 insertions(+)
create mode 100644 arch/loongarch/include/asm/ftrace.h
create mode 100644 arch/loongarch/kernel/ftrace.c
create mode 100644 arch/loongarch/kernel/mcount.S
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 1943f840e494..92c4ec8c8527 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -91,6 +91,8 @@ config LOONGARCH
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
select HAVE_FAST_GUP
+ select HAVE_FUNCTION_GRAPH_TRACER
+ select HAVE_FUNCTION_TRACER
select HAVE_GENERIC_VDSO
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK
diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
index a0fc1f9980e3..6832a8f891fd 100644
--- a/arch/loongarch/Makefile
+++ b/arch/loongarch/Makefile
@@ -36,6 +36,11 @@ ifneq ($(SUBARCH),$(ARCH))
endif
endif
+ifdef CONFIG_DYNAMIC_FTRACE
+KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+CC_FLAGS_FTRACE := -fpatchable-function-entry=2
+endif
+
ifdef CONFIG_64BIT
ld-emul = $(64bit-emul)
cflags-y += -mabi=lp64s
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
new file mode 100644
index 000000000000..6a3e76234618
--- /dev/null
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#ifndef _ASM_LOONGARCH_FTRACE_H
+#define _ASM_LOONGARCH_FTRACE_H
+
+#ifdef CONFIG_FUNCTION_TRACER
+#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
+
+#ifndef __ASSEMBLY__
+extern void _mcount(void);
+#define mcount _mcount
+
+#endif /* __ASSEMBLY__ */
+#endif /* CONFIG_FUNCTION_TRACER */
+#endif /* _ASM_LOONGARCH_FTRACE_H */
diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
index 86744531b100..3f71bce1c7ce 100644
--- a/arch/loongarch/kernel/Makefile
+++ b/arch/loongarch/kernel/Makefile
@@ -15,6 +15,14 @@ obj-$(CONFIG_EFI) += efi.o
obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
+ifdef CONFIG_FUNCTION_TRACER
+obj-y += mcount.o ftrace.o
+CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
+endif
+
obj-$(CONFIG_MODULES) += module.o module-sections.o
obj-$(CONFIG_STACKTRACE) += stacktrace.o
diff --git a/arch/loongarch/kernel/ftrace.c b/arch/loongarch/kernel/ftrace.c
new file mode 100644
index 000000000000..c8ddc5f11f32
--- /dev/null
+++ b/arch/loongarch/kernel/ftrace.c
@@ -0,0 +1,74 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/uaccess.h>
+#include <linux/init.h>
+#include <linux/ftrace.h>
+#include <linux/syscalls.h>
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/cacheflush.h>
+#include <asm/inst.h>
+#include <asm/loongarch.h>
+#include <asm/syscall.h>
+#include <asm/unistd.h>
+
+#include <asm-generic/sections.h>
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+
+/*
+ * As `call _mcount` follows LoongArch psABI, ra-saved operation and
+ * stack operation can be found before this insn.
+ */
+
+static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
+{
+ union loongarch_instruction *insn;
+ int limit = 32;
+
+ insn = (union loongarch_instruction *)insn_addr;
+
+ do {
+ insn--;
+ limit--;
+
+ if (is_ra_save_ins(insn))
+ *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
+
+ } while (!is_stack_alloc_ins(insn) && limit);
+
+ if (!limit)
+ return -EINVAL;
+
+ return 0;
+}
+
+void prepare_ftrace_return(unsigned long self_addr,
+ unsigned long callsite_sp, unsigned long old)
+{
+ int ra_off;
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+
+ if (unlikely(ftrace_graph_is_dead()))
+ return;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
+ goto out;
+
+ if (!function_graph_enter(old, self_addr, 0, NULL))
+ *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
+
+ return;
+
+out:
+ ftrace_graph_stop();
+ WARN_ON(1);
+}
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/loongarch/kernel/mcount.S b/arch/loongarch/kernel/mcount.S
new file mode 100644
index 000000000000..3de7c2d7fd12
--- /dev/null
+++ b/arch/loongarch/kernel/mcount.S
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * LoongArch specific _mcount support
+ *
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/export.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+#include <asm/ftrace.h>
+
+ .text
+
+#define MCOUNT_STACK_SIZE (2 * SZREG)
+#define MCOUNT_S0_OFFSET (0)
+#define MCOUNT_RA_OFFSET (SZREG)
+
+ .macro MCOUNT_SAVE_REGS
+ PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
+ PTR_S s0, sp, MCOUNT_S0_OFFSET
+ PTR_S ra, sp, MCOUNT_RA_OFFSET
+ move s0, a0
+ .endm
+
+ .macro MCOUNT_RESTORE_REGS
+ move a0, s0
+ PTR_L ra, sp, MCOUNT_RA_OFFSET
+ PTR_L s0, sp, MCOUNT_S0_OFFSET
+ PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
+ .endm
+
+
+SYM_FUNC_START(_mcount)
+ la.pcrel t1, ftrace_stub
+ la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
+ PTR_L t2, t2, 0
+ beq t1, t2, fgraph_trace
+
+ MCOUNT_SAVE_REGS
+
+ move a0, ra /* arg0: self return address */
+ move a1, s0 /* arg1: parent's return address */
+ jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
+
+ MCOUNT_RESTORE_REGS
+
+fgraph_trace:
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ la.pcrel t1, ftrace_stub
+ la.pcrel t3, ftrace_graph_return
+ PTR_L t3, t3, 0
+ bne t1, t3, ftrace_graph_caller
+ la.pcrel t1, ftrace_graph_entry_stub
+ la.pcrel t3, ftrace_graph_entry
+ PTR_L t3, t3, 0
+ bne t1, t3, ftrace_graph_caller
+#endif
+
+ .globl ftrace_stub
+ftrace_stub:
+ jr ra
+SYM_FUNC_END(_mcount)
+EXPORT_SYMBOL(_mcount)
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_FUNC_START(ftrace_graph_caller)
+ MCOUNT_SAVE_REGS
+
+ PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
+ PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
+ move a2, s0 /* arg2: Callsite parent ra */
+ bl prepare_ftrace_return
+
+ MCOUNT_RESTORE_REGS
+ jr ra
+SYM_FUNC_END(ftrace_graph_caller)
+
+SYM_FUNC_START(return_to_handler)
+ PTR_ADDI sp, sp, -2 * SZREG
+ PTR_S a0, sp, 0
+ PTR_S a1, sp, SZREG
+
+ bl ftrace_return_to_handler
+
+ /* restore the real parent address: a0 -> ra */
+ move ra, a0
+
+ PTR_L a0, sp, 0
+ PTR_L a1, sp, SZREG
+ PTR_ADDI sp, sp, 2 * SZREG
+ jr ra
+SYM_FUNC_END(return_to_handler)
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 2/9] LoongArch/ftrace: Add recordmcount support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 1/9] LoongArch/ftrace: Add basic support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support Qing Zhang
` (6 subsequent siblings)
8 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
Recordmcount utility under scripts is run, after compiling each object,
to find out all the locations of calling _mcount() and put them into
specific seciton named __mcount_loc.
Then linker collects all such information into a table in the kernel image
(between __start_mcount_loc and __stop_mcount_loc) for later use by ftrace.
This patch adds loongarch specific definitions to identify such locations.
On loongarch, only C version is used to build the kernel now that
CONFIG_HAVE_C_RECORDMCOUNT is on.
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/Kconfig | 2 ++
scripts/recordmcount.c | 23 +++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 92c4ec8c8527..e6b1defca1f1 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -86,11 +86,13 @@ config LOONGARCH
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select HAVE_ASM_MODVERSIONS
select HAVE_CONTEXT_TRACKING_USER
+ select HAVE_C_RECORDMCOUNT
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DMA_CONTIGUOUS
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
select HAVE_FAST_GUP
+ select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
select HAVE_GENERIC_VDSO
diff --git a/scripts/recordmcount.c b/scripts/recordmcount.c
index cce12e1971d8..ae42f9c5dc42 100644
--- a/scripts/recordmcount.c
+++ b/scripts/recordmcount.c
@@ -38,6 +38,13 @@
#define R_AARCH64_ABS64 257
#endif
+#ifndef EM_LOONGARCH
+#define EM_LOONGARCH 258
+#define R_LARCH_64 2
+#define R_LARCH_MARK_LA 20
+#define R_LARCH_SOP_PUSH_PLT_PCREL 29
+#endif
+
#define R_ARM_PC24 1
#define R_ARM_THM_CALL 10
#define R_ARM_CALL 28
@@ -441,6 +448,17 @@ static int arm64_is_fake_mcount(Elf64_Rel const *rp)
return ELF64_R_TYPE(w8(rp->r_info)) != R_AARCH64_CALL26;
}
+static int LARCH64_is_fake_mcount(Elf64_Rel const *rp)
+{
+ switch (ELF64_R_TYPE(w(rp->r_info))) {
+ case R_LARCH_MARK_LA:
+ case R_LARCH_SOP_PUSH_PLT_PCREL:
+ return 0;
+ }
+
+ return 1;
+}
+
/* 64-bit EM_MIPS has weird ELF64_Rela.r_info.
* http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf
* We interpret Table 29 Relocation Operation (Elf64_Rel, Elf64_Rela) [p.40]
@@ -558,6 +576,7 @@ static int do_file(char const *const fname)
break;
case EM_IA_64: reltype = R_IA64_IMM64; break;
case EM_MIPS: /* reltype: e_class */ break;
+ case EM_LOONGARCH: /* reltype: e_class */ break;
case EM_PPC: reltype = R_PPC_ADDR32; break;
case EM_PPC64: reltype = R_PPC64_ADDR64; break;
case EM_S390: /* reltype: e_class */ break;
@@ -610,6 +629,10 @@ static int do_file(char const *const fname)
Elf64_r_info = MIPS64_r_info;
is_fake_mcount64 = MIPS64_is_fake_mcount;
}
+ if (w2(ghdr->e_machine) == EM_LOONGARCH) {
+ reltype = R_LARCH_64;
+ is_fake_mcount64 = LARCH64_is_fake_mcount;
+ }
if (do64(ghdr, fname, reltype) < 0)
goto out;
break;
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 1/9] LoongArch/ftrace: Add basic support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 2/9] LoongArch/ftrace: Add recordmcount support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 10:54 ` Huacai Chen
2022-11-15 2:55 ` [PATCH v6 4/9] LoongArch/ftrace: Add dynamic function graph " Qing Zhang
` (5 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar
Cc: loongarch, linux-kernel, Jinyang He
The compiler has inserted 2 NOPs before the regular function prologue.
T series registers are available and safe because of LoongArch psABI.
At runtime, replace nop with bl to enable ftrace call and replace bl with
nop to disable ftrace call. The bl requires us to save the original RA
value, so here it saves RA at t0.
details are:
| Compiled | Disabled | Enabled |
+------------+------------------------+------------------------+
| nop | move t0, ra | move t0, ra |
| nop | nop | bl ftrace_caller |
| func_body | func_body | func_body |
The RA value will be recovered by ftrace_regs_entry, and restored into RA
before returning to the regular function prologue. When a function is not
being traced, the move t0, ra is not harmful.
1) ftrace_make_call, ftrace_make_nop (in kernel/ftrace.c)
The two functions turn each recorded call site of filtered functions
into a call to ftrace_caller or nops.
2) ftracce_update_ftrace_func (in kernel/ftrace.c)
turns the nops at ftrace_call into a call to a generic entry for
function tracers.
3) ftrace_caller (in kernel/mcount-dyn.S)
The entry where each _mcount call sites calls to once they are
filtered to be traced.
Co-developed-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/Kconfig | 1 +
arch/loongarch/include/asm/ftrace.h | 16 ++++
arch/loongarch/include/asm/inst.h | 15 ++++
arch/loongarch/include/asm/unwind.h | 2 +-
arch/loongarch/kernel/Makefile | 5 ++
arch/loongarch/kernel/ftrace_dyn.c | 111 ++++++++++++++++++++++++
arch/loongarch/kernel/inst.c | 92 ++++++++++++++++++++
arch/loongarch/kernel/mcount-dyn.S | 89 +++++++++++++++++++
arch/loongarch/kernel/unwind_prologue.c | 35 ++++++--
9 files changed, 360 insertions(+), 6 deletions(-)
create mode 100644 arch/loongarch/kernel/ftrace_dyn.c
create mode 100644 arch/loongarch/kernel/mcount-dyn.S
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index e6b1defca1f1..615ce62422b8 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -89,6 +89,7 @@ config LOONGARCH
select HAVE_C_RECORDMCOUNT
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DMA_CONTIGUOUS
+ select HAVE_DYNAMIC_FTRACE
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
select HAVE_FAST_GUP
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
index 6a3e76234618..76ca58767f4d 100644
--- a/arch/loongarch/include/asm/ftrace.h
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -10,9 +10,25 @@
#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
#ifndef __ASSEMBLY__
+#ifndef CONFIG_DYNAMIC_FTRACE
extern void _mcount(void);
#define mcount _mcount
+#endif
+#ifdef CONFIG_DYNAMIC_FTRACE
+static inline unsigned long ftrace_call_adjust(unsigned long addr)
+{
+ return addr;
+}
+
+struct dyn_arch_ftrace {
+};
+
+struct dyn_ftrace;
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
+#define ftrace_init_nop ftrace_init_nop
+
+#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* __ASSEMBLY__ */
#endif /* CONFIG_FUNCTION_TRACER */
#endif /* _ASM_LOONGARCH_FTRACE_H */
diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
index a52913787183..0ec775d39ca5 100644
--- a/arch/loongarch/include/asm/inst.h
+++ b/arch/loongarch/include/asm/inst.h
@@ -11,6 +11,9 @@
#define INSN_NOP 0x03400000
#define INSN_BREAK 0x002a0000
+#define INSN_NOP 0x03400000
+#define INSN_BREAK 0x002a0000
+
#define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
#define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
#define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
@@ -349,6 +352,18 @@ static inline bool is_stack_alloc_ins(union loongarch_instruction *ip)
is_imm12_negative(ip->reg2i12_format.immediate);
}
+int larch_insn_read(void *addr, u32 *insnp);
+int larch_insn_write(void *addr, u32 insn);
+int larch_insn_patch_text(void *addr, u32 insn);
+
+u32 larch_insn_gen_nop(void);
+u32 larch_insn_gen_b(unsigned long pc, unsigned long dest);
+u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest);
+
+u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj,
+ enum loongarch_gpr rk);
+u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
+
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
diff --git a/arch/loongarch/include/asm/unwind.h b/arch/loongarch/include/asm/unwind.h
index 6af4718bdf01..a51eec00efb8 100644
--- a/arch/loongarch/include/asm/unwind.h
+++ b/arch/loongarch/include/asm/unwind.h
@@ -20,7 +20,7 @@ struct unwind_state {
char type; /* UNWINDER_XXX */
struct stack_info stack_info;
struct task_struct *task;
- bool first, error;
+ bool first, error, is_ftrace;
unsigned long sp, pc, ra;
};
diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
index 3f71bce1c7ce..c5e2bfd8247d 100644
--- a/arch/loongarch/kernel/Makefile
+++ b/arch/loongarch/kernel/Makefile
@@ -16,8 +16,13 @@ obj-$(CONFIG_EFI) += efi.o
obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
ifdef CONFIG_FUNCTION_TRACER
+ifndef CONFIG_DYNAMIC_FTRACE
obj-y += mcount.o ftrace.o
CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+else
+obj-y += mcount-dyn.o ftrace_dyn.o
+CFLAGS_REMOVE_ftrace_dyn.o = $(CC_FLAGS_FTRACE)
+endif
CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
new file mode 100644
index 000000000000..1f8955be8b64
--- /dev/null
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Based on arch/arm64/kernel/ftrace.c
+ *
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <linux/ftrace.h>
+#include <linux/uaccess.h>
+
+#include <asm/inst.h>
+
+static int ftrace_modify_code(unsigned long pc, u32 old, u32 new,
+ bool validate)
+{
+ u32 replaced;
+
+ if (validate) {
+ if (larch_insn_read((void *)pc, &replaced))
+ return -EFAULT;
+
+ if (replaced != old)
+ return -EINVAL;
+ }
+
+ if (larch_insn_patch_text((void *)pc, new))
+ return -EPERM;
+
+ return 0;
+}
+
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+ unsigned long pc;
+ u32 new;
+
+ pc = (unsigned long)&ftrace_call;
+ new = larch_insn_gen_bl(pc, (unsigned long)func);
+
+ return ftrace_modify_code(pc, 0, new, false);
+}
+
+/*
+ * The compiler has inserted 2 NOPs before the regular function prologue.
+ * T series registers are available and safe because of LoongArch psABI.
+ *
+ * At runtime, replace nop with bl to enable ftrace call and replace bl with
+ * nop to disable ftrace call. The bl requires us to save the original RA value,
+ * so here it saves RA at t0.
+ * details are:
+ *
+ * | Compiled | Disabled | Enabled |
+ * +------------+------------------------+------------------------+
+ * | nop | move t0, ra | move t0, ra |
+ * | nop | nop | bl ftrace_caller |
+ * | func_body | func_body | func_body |
+ *
+ * The RA value will be recovered by ftrace_regs_entry, and restored into RA
+ * before returning to the regular function prologue. When a function is not
+ * being traced, the move t0, ra is not harmful.
+ */
+
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
+{
+ unsigned long pc;
+ u32 old, new;
+
+ pc = rec->ip;
+ old = larch_insn_gen_nop();
+ new = larch_insn_gen_move(LOONGARCH_GPR_T0, LOONGARCH_GPR_RA);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long pc;
+ u32 old, new;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+ old = larch_insn_gen_nop();
+ new = larch_insn_gen_bl(pc, addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
+ unsigned long addr)
+{
+ unsigned long pc;
+ u32 old, new;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+ new = larch_insn_gen_nop();
+ old = larch_insn_gen_bl(pc, addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+
+void arch_ftrace_update_code(int command)
+{
+ command |= FTRACE_MAY_SLEEP;
+ ftrace_modify_all_code(command);
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
+ return 0;
+}
diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
index b1df0ec34bd1..d62cdf4a9ffb 100644
--- a/arch/loongarch/kernel/inst.c
+++ b/arch/loongarch/kernel/inst.c
@@ -2,8 +2,83 @@
/*
* Copyright (C) 2020-2022 Loongson Technology Corporation Limited
*/
+#include <linux/sizes.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
#include <asm/inst.h>
+static DEFINE_RAW_SPINLOCK(patch_lock);
+
+int larch_insn_read(void *addr, u32 *insnp)
+{
+ int ret;
+ u32 val;
+
+ ret = copy_from_kernel_nofault(&val, addr, LOONGARCH_INSN_SIZE);
+ if (!ret)
+ *insnp = val;
+
+ return ret;
+}
+
+int larch_insn_write(void *addr, u32 insn)
+{
+ int ret;
+ unsigned long flags = 0;
+
+ raw_spin_lock_irqsave(&patch_lock, flags);
+ ret = copy_to_kernel_nofault(addr, &insn, LOONGARCH_INSN_SIZE);
+ raw_spin_unlock_irqrestore(&patch_lock, flags);
+
+ return ret;
+}
+
+int larch_insn_patch_text(void *addr, u32 insn)
+{
+ int ret;
+ u32 *tp = addr;
+
+ if ((unsigned long)tp & 3)
+ return -EINVAL;
+
+ ret = larch_insn_write(tp, insn);
+ if (!ret)
+ flush_icache_range((unsigned long)tp,
+ (unsigned long)tp + LOONGARCH_INSN_SIZE);
+
+ return ret;
+}
+
+u32 larch_insn_gen_nop(void)
+{
+ return INSN_NOP;
+}
+
+u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
+{
+ unsigned int immediate_l, immediate_h;
+ union loongarch_instruction insn;
+ long offset = dest - pc;
+
+ if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
+ pr_warn("The generated bl instruction is out of range.\n");
+ return INSN_BREAK;
+ }
+
+ offset >>= 2;
+
+ immediate_l = offset & 0xffff;
+ offset >>= 16;
+ immediate_h = offset & 0x3ff;
+
+ insn.reg0i26_format.opcode = bl_op;
+ insn.reg0i26_format.immediate_l = immediate_l;
+ insn.reg0i26_format.immediate_h = immediate_h;
+
+ return insn.word;
+}
+
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
{
union loongarch_instruction insn;
@@ -38,3 +113,20 @@ u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned l
return insn.word;
}
+
+u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk)
+{
+ union loongarch_instruction insn;
+
+ insn.reg3_format.opcode = or_op;
+ insn.reg3_format.rd = rd;
+ insn.reg3_format.rj = rj;
+ insn.reg3_format.rk = rk;
+
+ return insn.word;
+}
+
+u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj)
+{
+ return larch_insn_gen_or(rd, rj, 0);
+}
diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
new file mode 100644
index 000000000000..205925bc3822
--- /dev/null
+++ b/arch/loongarch/kernel/mcount-dyn.S
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2022 Loongson Technology Corporation Limited
+ */
+
+#include <asm/export.h>
+#include <asm/regdef.h>
+#include <asm/stackframe.h>
+#include <asm/ftrace.h>
+
+ .text
+/*
+ * Due to -fpatchable-function-entry=2: the compiler inserted 2 NOPs before the
+ * regular C function prologue. When PC arrived here, the last 2 instructions
+ * as follows,
+ * move t0, ra
+ * bl callsite (for modules, callsite is a tramplione)
+ *
+ * modules tramplione as follows,
+ * lu12i.w t1, callsite[31:12]
+ * lu32i.d t1, callsite[51:32]
+ * lu52i.d t1, t1, callsite[63:52]
+ * jirl zero, t1, callsite[11:0] >> 2
+ *
+ * See arch/loongarch/kernel/ftrace_dyn.c for details. Here, pay attention to
+ * that the T series regs are available and safe because each C functions
+ * follows the LoongArch psABI well.
+ */
+
+ .macro ftrace_regs_entry
+ PTR_ADDI sp, sp, -PT_SIZE
+ /* Save trace function ra at PT_ERA */
+ PTR_S ra, sp, PT_ERA
+ /* Save parent ra at PT_R1(RA) */
+ PTR_S t0, sp, PT_R1
+ PTR_S a0, sp, PT_R4
+ PTR_S a1, sp, PT_R5
+ PTR_S a2, sp, PT_R6
+ PTR_S a3, sp, PT_R7
+ PTR_S a4, sp, PT_R8
+ PTR_S a5, sp, PT_R9
+ PTR_S a6, sp, PT_R10
+ PTR_S a7, sp, PT_R11
+ PTR_S fp, sp, PT_R22
+
+ PTR_ADDI t8, sp, PT_SIZE
+ PTR_S t8, sp, PT_R3
+
+ .endm
+
+SYM_CODE_START(ftrace_caller)
+ ftrace_regs_entry
+ b ftrace_common
+SYM_CODE_END(ftrace_caller)
+
+SYM_CODE_START(ftrace_common)
+ PTR_ADDI a0, ra, -8 /* arg0: ip */
+ move a1, t0 /* arg1: parent_ip */
+ la.pcrel t1, function_trace_op
+ PTR_L a2, t1, 0 /* arg2: op */
+ move a3, sp /* arg3: regs */
+ .globl ftrace_call
+ftrace_call:
+ bl ftrace_stub
+/*
+ * As we didn't use S series regs in this assmembly code and all calls
+ * are C function which will save S series regs by themselves, there is
+ * no need to restore S series regs. The T series is available and safe
+ * at the callsite, so there is no need to restore the T series regs.
+ */
+ftrace_common_return:
+ PTR_L a0, sp, PT_R4
+ PTR_L a1, sp, PT_R5
+ PTR_L a2, sp, PT_R6
+ PTR_L a3, sp, PT_R7
+ PTR_L a4, sp, PT_R8
+ PTR_L a5, sp, PT_R9
+ PTR_L a6, sp, PT_R10
+ PTR_L a7, sp, PT_R11
+ PTR_L fp, sp, PT_R22
+ PTR_L ra, sp, PT_R1
+ PTR_L t0, sp, PT_ERA
+ PTR_ADDI sp, sp, PT_SIZE
+ jr t0
+SYM_CODE_END(ftrace_common)
+
+SYM_FUNC_START(ftrace_stub)
+ jr ra
+SYM_FUNC_END(ftrace_stub)
diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c
index b206d9159205..c5df4ae73e0d 100644
--- a/arch/loongarch/kernel/unwind_prologue.c
+++ b/arch/loongarch/kernel/unwind_prologue.c
@@ -13,9 +13,7 @@ unsigned long unwind_get_return_address(struct unwind_state *state)
if (unwind_done(state))
return 0;
- else if (state->type)
- return state->pc;
- else if (state->first)
+ else if (state->type || state->first)
return state->pc;
return *(unsigned long *)(state->sp);
@@ -39,16 +37,41 @@ static bool unwind_by_guess(struct unwind_state *state)
return false;
}
+static inline void unwind_state_fixup(struct unwind_state *state)
+{
+#ifdef CONFIG_DYNAMIC_FTRACE
+ static unsigned long ftrace_case = (unsigned long)ftrace_call + 4;
+
+ if (state->pc == ftrace_case)
+ state->is_ftrace = true;
+#endif
+}
+
static bool unwind_by_prologue(struct unwind_state *state)
{
struct stack_info *info = &state->stack_info;
union loongarch_instruction *ip, *ip_end;
unsigned long frame_size = 0, frame_ra = -1;
unsigned long size, offset, pc = state->pc;
+ struct pt_regs *regs;
if (state->sp >= info->end || state->sp < info->begin)
return false;
+ if (state->is_ftrace) {
+ /*
+ * As we meet ftrace_regs_entry, reset first flag like first doing
+ * tracing, Prologue analysis will stop soon because PC is at entry.
+ */
+ regs = (struct pt_regs *)state->sp;
+ state->pc = regs->csr_era;
+ state->ra = regs->regs[1];
+ state->sp = regs->regs[3];
+ state->first = true;
+ state->is_ftrace = false;
+ return true;
+ }
+
if (!kallsyms_lookup_size_offset(pc, &size, &offset))
return false;
@@ -94,7 +117,7 @@ static bool unwind_by_prologue(struct unwind_state *state)
state->pc = *(unsigned long *)(state->sp + frame_ra);
state->sp = state->sp + frame_size;
- return !!__kernel_text_address(state->pc);
+ goto out;
first:
state->first = false;
@@ -103,7 +126,9 @@ static bool unwind_by_prologue(struct unwind_state *state)
state->pc = state->ra;
- return !!__kernel_text_address(state->ra);
+out:
+ unwind_state_fixup(state);
+ return !!__kernel_text_address(state->pc);
}
void unwind_start(struct unwind_state *state, struct task_struct *task,
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 4/9] LoongArch/ftrace: Add dynamic function graph tracer support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (2 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support Qing Zhang
` (4 subsequent siblings)
8 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar
Cc: loongarch, linux-kernel, Jinyang He
Once the function_graph tracer is enabled, a filtered function has the
following call sequence:
1) ftracer_caller ==> on/off by ftrace_make_call/ftrace_make_nop
2) ftrace_graph_caller
3) ftrace_graph_call ==> on/off by ftrace_en/disable_ftrace_graph_caller
4) prepare_ftrace_return
Considering the following DYNAMIC_FTRACE_WITH_REGS feature, it would be
more extendable to have a ftrace_graph_caller function, instead of
calling prepare_ftrace_return directly in ftrace_caller.
Co-developed-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/kernel/ftrace_dyn.c | 45 ++++++++++++++++++++++++++++++
arch/loongarch/kernel/inst.c | 24 ++++++++++++++++
arch/loongarch/kernel/mcount-dyn.S | 33 ++++++++++++++++++++++
3 files changed, 102 insertions(+)
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
index 1f8955be8b64..3fe791b6783e 100644
--- a/arch/loongarch/kernel/ftrace_dyn.c
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -109,3 +109,48 @@ int __init ftrace_dyn_arch_init(void)
{
return 0;
}
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+extern void ftrace_graph_call(void);
+
+void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent)
+{
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+ unsigned long old;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ old = *parent;
+
+ if (!function_graph_enter(old, self_addr, 0, NULL))
+ *parent = return_hooker;
+}
+
+static int ftrace_modify_graph_caller(bool enable)
+{
+ unsigned long pc, func;
+ u32 branch, nop;
+
+ pc = (unsigned long)&ftrace_graph_call;
+ func = (unsigned long)&ftrace_graph_caller;
+
+ branch = larch_insn_gen_b(pc, func);
+ nop = larch_insn_gen_nop();
+
+ if (enable)
+ return ftrace_modify_code(pc, nop, branch, true);
+ else
+ return ftrace_modify_code(pc, branch, nop, true);
+}
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(true);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_graph_caller(false);
+}
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
index d62cdf4a9ffb..2d2e942eb06a 100644
--- a/arch/loongarch/kernel/inst.c
+++ b/arch/loongarch/kernel/inst.c
@@ -55,6 +55,30 @@ u32 larch_insn_gen_nop(void)
return INSN_NOP;
}
+u32 larch_insn_gen_b(unsigned long pc, unsigned long dest)
+{
+ unsigned int immediate_l, immediate_h;
+ union loongarch_instruction insn;
+ long offset = dest - pc;
+
+ if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
+ pr_warn("The generated b instruction is out of range.\n");
+ return INSN_BREAK;
+ }
+
+ offset >>= 2;
+
+ immediate_l = offset & 0xffff;
+ offset >>= 16;
+ immediate_h = offset & 0x3ff;
+
+ insn.reg0i26_format.opcode = b_op;
+ insn.reg0i26_format.immediate_l = immediate_l;
+ insn.reg0i26_format.immediate_h = immediate_h;
+
+ return insn.word;
+}
+
u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
{
unsigned int immediate_l, immediate_h;
diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
index 205925bc3822..0c12cc108e6f 100644
--- a/arch/loongarch/kernel/mcount-dyn.S
+++ b/arch/loongarch/kernel/mcount-dyn.S
@@ -62,6 +62,11 @@ SYM_CODE_START(ftrace_common)
.globl ftrace_call
ftrace_call:
bl ftrace_stub
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ .globl ftrace_graph_call
+ftrace_graph_call:
+ nop /* b ftrace_graph_caller */
+#endif
/*
* As we didn't use S series regs in this assmembly code and all calls
* are C function which will save S series regs by themselves, there is
@@ -84,6 +89,34 @@ ftrace_common_return:
jr t0
SYM_CODE_END(ftrace_common)
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+SYM_CODE_START(ftrace_graph_caller)
+ PTR_L a0, sp, PT_ERA
+ PTR_ADDI a0, a0, -8 /* arg0: self_addr */
+ PTR_ADDI a1, sp, PT_R1 /* arg1: parent */
+ bl prepare_ftrace_return
+ b ftrace_common_return
+SYM_CODE_END(ftrace_graph_caller)
+
+SYM_CODE_START(return_to_handler)
+ /* save return value regs */
+ PTR_ADDI sp, sp, -2 * SZREG
+ PTR_S a0, sp, 0
+ PTR_S a1, sp, SZREG
+
+ move a0, zero /* Has no check FP now. */
+ bl ftrace_return_to_handler
+ move ra, a0 /* parent ra */
+
+ /* restore return value regs */
+ PTR_L a0, sp, 0
+ PTR_L a1, sp, SZREG
+ PTR_ADDI sp, sp, 2 * SZREG
+
+ jr ra
+SYM_CODE_END(return_to_handler)
+#endif
+
SYM_FUNC_START(ftrace_stub)
jr ra
SYM_FUNC_END(ftrace_stub)
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (3 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 4/9] LoongArch/ftrace: Add dynamic function graph " Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 8:41 ` Huacai Chen
2022-11-15 2:55 ` [PATCH v6 6/9] LoongArch/ftrace: Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support Qing Zhang
` (3 subsequent siblings)
8 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar
Cc: loongarch, linux-kernel, Jinyang He
This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
a traced function's arguments (and some other registers) to be captured
into a struct pt_regs, allowing these to be inspected and modified.
Co-developed-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Jinyang He <hejinyang@loongson.cn>
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/Kconfig | 1 +
arch/loongarch/include/asm/ftrace.h | 3 +++
arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
4 files changed, 55 insertions(+), 2 deletions(-)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 615ce62422b8..12e3e91a68ae 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -90,6 +90,7 @@ config LOONGARCH
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
+ select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
select HAVE_FAST_GUP
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
index 76ca58767f4d..a3f974a7a5ce 100644
--- a/arch/loongarch/include/asm/ftrace.h
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -28,6 +28,9 @@ struct dyn_ftrace;
int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
#define ftrace_init_nop ftrace_init_nop
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#define ARCH_SUPPORTS_FTRACE_OPS 1
+#endif
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* __ASSEMBLY__ */
#endif /* CONFIG_FUNCTION_TRACER */
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
index 3fe791b6783e..ec3d951be50c 100644
--- a/arch/loongarch/kernel/ftrace_dyn.c
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
return ftrace_modify_code(pc, old, new, true);
}
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr)
+{
+ unsigned long pc;
+ long offset;
+ u32 old, new;
+
+ pc = rec->ip + LOONGARCH_INSN_SIZE;
+
+ old = larch_insn_gen_bl(pc, old_addr);
+ new = larch_insn_gen_bl(pc, addr);
+
+ return ftrace_modify_code(pc, old, new, true);
+}
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
void arch_ftrace_update_code(int command)
{
command |= FTRACE_MAY_SLEEP;
diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
index 0c12cc108e6f..02835186b463 100644
--- a/arch/loongarch/kernel/mcount-dyn.S
+++ b/arch/loongarch/kernel/mcount-dyn.S
@@ -27,7 +27,7 @@
* follows the LoongArch psABI well.
*/
- .macro ftrace_regs_entry
+ .macro ftrace_regs_entry allregs=0
PTR_ADDI sp, sp, -PT_SIZE
/* Save trace function ra at PT_ERA */
PTR_S ra, sp, PT_ERA
@@ -43,16 +43,48 @@
PTR_S a7, sp, PT_R11
PTR_S fp, sp, PT_R22
+ .if \allregs
+ PTR_S t0, sp, PT_R12
+ PTR_S t1, sp, PT_R13
+ PTR_S t2, sp, PT_R14
+ PTR_S t3, sp, PT_R15
+ PTR_S t4, sp, PT_R16
+ PTR_S t5, sp, PT_R17
+ PTR_S t6, sp, PT_R18
+ PTR_S t7, sp, PT_R19
+ PTR_S t8, sp, PT_R20
+ PTR_S s0, sp, PT_R23
+ PTR_S s1, sp, PT_R24
+ PTR_S s2, sp, PT_R25
+ PTR_S s3, sp, PT_R26
+ PTR_S s4, sp, PT_R27
+ PTR_S s5, sp, PT_R28
+ PTR_S s6, sp, PT_R29
+ PTR_S s7, sp, PT_R30
+ PTR_S s8, sp, PT_R31
+ PTR_S tp, sp, PT_R2
+ /* Clear it for later use as a flag sometimes. */
+ PTR_S zero, sp, PT_R0
+ PTR_S $r21, sp, PT_R21
+ .endif
+
PTR_ADDI t8, sp, PT_SIZE
PTR_S t8, sp, PT_R3
.endm
SYM_CODE_START(ftrace_caller)
- ftrace_regs_entry
+ ftrace_regs_entry allregs=0
b ftrace_common
SYM_CODE_END(ftrace_caller)
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+SYM_CODE_START(ftrace_regs_caller)
+ ftrace_regs_entry allregs=1
+ b ftrace_common
+SYM_CODE_END(ftrace_regs_caller)
+#endif
+
SYM_CODE_START(ftrace_common)
PTR_ADDI a0, ra, -8 /* arg0: ip */
move a1, t0 /* arg1: parent_ip */
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 6/9] LoongArch/ftrace: Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (4 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 7/9] LoongArch/ftrace: Add HAVE_FUNCTION_GRAPH_RET_ADDR_PTR support Qing Zhang
` (2 subsequent siblings)
8 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
Allow for arguments to be passed in to ftrace_regs by default,
If this is set, then arguments and stack can be found from
the pt_regs.
1. HAVE_DYNAMIC_FTRACE_WITH_ARGS don't need special hook for graph
tracer entry point, but instead we can use graph_ops::func function
to install the return_hooker.
2. Livepatch requires this option in the future.
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/Kconfig | 1 +
arch/loongarch/include/asm/ftrace.h | 17 +++++++++++++++++
arch/loongarch/kernel/ftrace_dyn.c | 12 ++++++++++++
3 files changed, 30 insertions(+)
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 12e3e91a68ae..5c4f1dc87f84 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -90,6 +90,7 @@ config LOONGARCH
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DMA_CONTIGUOUS
select HAVE_DYNAMIC_FTRACE
+ select HAVE_DYNAMIC_FTRACE_WITH_ARGS
select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_EBPF_JIT
select HAVE_EXIT_THREAD
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
index a3f974a7a5ce..5cc13ae48164 100644
--- a/arch/loongarch/include/asm/ftrace.h
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -15,6 +15,23 @@ extern void _mcount(void);
#define mcount _mcount
#endif
+#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS
+struct ftrace_ops;
+
+struct ftrace_regs {
+ struct pt_regs regs;
+};
+
+static __always_inline struct pt_regs *arch_ftrace_get_regs(struct ftrace_regs *fregs)
+{
+ return &fregs->regs;
+}
+
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs);
+#define ftrace_graph_func ftrace_graph_func
+#endif
+
#ifdef CONFIG_DYNAMIC_FTRACE
static inline unsigned long ftrace_call_adjust(unsigned long addr)
{
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
index ec3d951be50c..f538829312d7 100644
--- a/arch/loongarch/kernel/ftrace_dyn.c
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -144,6 +144,17 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent)
*parent = return_hooker;
}
+#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+ struct pt_regs *regs = &fregs->regs;
+ unsigned long *parent = (unsigned long *)®s->regs[1];
+
+ prepare_ftrace_return(ip, (unsigned long *)parent);
+}
+#else
+
static int ftrace_modify_graph_caller(bool enable)
{
unsigned long pc, func;
@@ -170,4 +181,5 @@ int ftrace_disable_ftrace_graph_caller(void)
{
return ftrace_modify_graph_caller(false);
}
+#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 7/9] LoongArch/ftrace: Add HAVE_FUNCTION_GRAPH_RET_ADDR_PTR support
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (5 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 6/9] LoongArch/ftrace: Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 8/9] LoongArch: modules/ftrace: Initialize PLT at load time Qing Zhang
2022-11-15 2:55 ` [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS Qing Zhang
8 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
Ftrace_graph_ret_addr can be called by stack unwinding code to convert
a found stack return address ('ret') to its original value, in case the
function graph tracer has modified it to be 'return_to_handler',If the
hasn't been modified, the unchanged value of 'ret' is returned.
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/include/asm/ftrace.h | 3 +++
arch/loongarch/include/asm/unwind.h | 1 +
arch/loongarch/kernel/ftrace_dyn.c | 2 +-
arch/loongarch/kernel/unwind_guess.c | 4 +++-
arch/loongarch/kernel/unwind_prologue.c | 11 +++++++++--
5 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
index 5cc13ae48164..1583ec2171fc 100644
--- a/arch/loongarch/include/asm/ftrace.h
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -6,6 +6,8 @@
#ifndef _ASM_LOONGARCH_FTRACE_H
#define _ASM_LOONGARCH_FTRACE_H
+#define GRAPH_FAKE_OFFSET (sizeof(struct pt_regs) - offsetof(struct pt_regs, regs[1]))
+
#ifdef CONFIG_FUNCTION_TRACER
#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
@@ -33,6 +35,7 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
#endif
#ifdef CONFIG_DYNAMIC_FTRACE
+#define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
static inline unsigned long ftrace_call_adjust(unsigned long addr)
{
return addr;
diff --git a/arch/loongarch/include/asm/unwind.h b/arch/loongarch/include/asm/unwind.h
index a51eec00efb8..f2b52b9ea93d 100644
--- a/arch/loongarch/include/asm/unwind.h
+++ b/arch/loongarch/include/asm/unwind.h
@@ -21,6 +21,7 @@ struct unwind_state {
struct stack_info stack_info;
struct task_struct *task;
bool first, error, is_ftrace;
+ int graph_idx;
unsigned long sp, pc, ra;
};
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
index f538829312d7..cd64887e26b5 100644
--- a/arch/loongarch/kernel/ftrace_dyn.c
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -140,7 +140,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent)
old = *parent;
- if (!function_graph_enter(old, self_addr, 0, NULL))
+ if (!function_graph_enter(old, self_addr, 0, parent))
*parent = return_hooker;
}
diff --git a/arch/loongarch/kernel/unwind_guess.c b/arch/loongarch/kernel/unwind_guess.c
index 5afa6064d73e..229ba014cea0 100644
--- a/arch/loongarch/kernel/unwind_guess.c
+++ b/arch/loongarch/kernel/unwind_guess.c
@@ -2,6 +2,7 @@
/*
* Copyright (C) 2022 Loongson Technology Corporation Limited
*/
+#include <linux/ftrace.h>
#include <linux/kernel.h>
#include <asm/unwind.h>
@@ -53,7 +54,8 @@ bool unwind_next_frame(struct unwind_state *state)
state->sp < info->end;
state->sp += sizeof(unsigned long)) {
addr = *(unsigned long *)(state->sp);
-
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
if (__kernel_text_address(addr))
return true;
}
diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c
index c5df4ae73e0d..48f7a120ec27 100644
--- a/arch/loongarch/kernel/unwind_prologue.c
+++ b/arch/loongarch/kernel/unwind_prologue.c
@@ -2,6 +2,7 @@
/*
* Copyright (C) 2022 Loongson Technology Corporation Limited
*/
+#include <linux/ftrace.h>
#include <linux/kallsyms.h>
#include <asm/inst.h>
@@ -30,6 +31,8 @@ static bool unwind_by_guess(struct unwind_state *state)
state->sp < info->end;
state->sp += sizeof(unsigned long)) {
addr = *(unsigned long *)(state->sp);
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ addr, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
if (__kernel_text_address(addr))
return true;
}
@@ -171,8 +174,11 @@ bool unwind_next_frame(struct unwind_state *state)
break;
case UNWINDER_PROLOGUE:
- if (unwind_by_prologue(state))
+ if (unwind_by_prologue(state)) {
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ state->pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
return true;
+ }
if (info->type == STACK_TYPE_IRQ &&
info->end == state->sp) {
@@ -182,8 +188,9 @@ bool unwind_next_frame(struct unwind_state *state)
if (user_mode(regs) || !__kernel_text_address(pc))
return false;
- state->pc = pc;
state->sp = regs->regs[3];
+ state->pc = ftrace_graph_ret_addr(state->task, &state->graph_idx,
+ pc, (unsigned long *)(state->sp - GRAPH_FAKE_OFFSET));
state->ra = regs->regs[1];
state->first = true;
get_stack_info(state->sp, state->task, info);
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 8/9] LoongArch: modules/ftrace: Initialize PLT at load time
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (6 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 7/9] LoongArch/ftrace: Add HAVE_FUNCTION_GRAPH_RET_ADDR_PTR support Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS Qing Zhang
8 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
To Implement ftrace trampiones through plt entry.
Tested by forcing ftrace_make_call() to use the module PLT, and then
loading up a module after setting up ftrace with:
| echo ":mod:<module-name>" > set_ftrace_filter;
| echo function > current_tracer;
| modprobe <module-name>
Since FTRACE_ADDR/FTRACE_REGS_ADDR is only defined when CONFIG_DYNAMIC_FTRACE
is selected, we wrap its use along with most of module_init_ftrace_plt() with
ifdeffery rather than using IS_ENABLED().
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/include/asm/ftrace.h | 4 ++
arch/loongarch/include/asm/inst.h | 3 +
arch/loongarch/include/asm/module.h | 5 +-
arch/loongarch/include/asm/module.lds.h | 1 +
arch/loongarch/kernel/ftrace_dyn.c | 79 +++++++++++++++++++++++++
arch/loongarch/kernel/inst.c | 11 ++++
arch/loongarch/kernel/module-sections.c | 11 ++++
arch/loongarch/kernel/module.c | 22 +++++++
8 files changed, 135 insertions(+), 1 deletion(-)
diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
index 1583ec2171fc..8d194f6da8af 100644
--- a/arch/loongarch/include/asm/ftrace.h
+++ b/arch/loongarch/include/asm/ftrace.h
@@ -8,6 +8,10 @@
#define GRAPH_FAKE_OFFSET (sizeof(struct pt_regs) - offsetof(struct pt_regs, regs[1]))
+#define FTRACE_PLT_IDX 0
+#define FTRACE_REGS_PLT_IDX 1
+#define NR_FTRACE_PLTS 2
+
#ifdef CONFIG_FUNCTION_TRACER
#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
index 0ec775d39ca5..3c13fb673f1e 100644
--- a/arch/loongarch/include/asm/inst.h
+++ b/arch/loongarch/include/asm/inst.h
@@ -16,10 +16,12 @@
#define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
#define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
+#define ADDR_IMMMASK_LU12IW 0x00000000FFFFF000
#define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
#define ADDR_IMMSHIFT_LU52ID 52
#define ADDR_IMMSHIFT_LU32ID 32
+#define ADDR_IMMSHIFT_LU12IW 12
#define ADDR_IMMSHIFT_ADDU16ID 16
#define ADDR_IMM(addr, INSN) ((addr & ADDR_IMMMASK_##INSN) >> ADDR_IMMSHIFT_##INSN)
@@ -364,6 +366,7 @@ u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj,
enum loongarch_gpr rk);
u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
+u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
diff --git a/arch/loongarch/include/asm/module.h b/arch/loongarch/include/asm/module.h
index b29b19a46f42..a311cfec2b23 100644
--- a/arch/loongarch/include/asm/module.h
+++ b/arch/loongarch/include/asm/module.h
@@ -20,6 +20,9 @@ struct mod_arch_specific {
struct mod_section got;
struct mod_section plt;
struct mod_section plt_idx;
+
+ /* for CONFIG_DYNAMIC_FTRACE */
+ struct plt_entry *ftrace_trampolines;
};
struct got_entry {
@@ -49,7 +52,7 @@ static inline struct plt_entry emit_plt_entry(unsigned long val)
{
u32 lu12iw, lu32id, lu52id, jirl;
- lu12iw = (lu12iw_op << 25 | (((val >> 12) & 0xfffff) << 5) | LOONGARCH_GPR_T1);
+ lu12iw = larch_insn_gen_lu12iw(LOONGARCH_GPR_T1, ADDR_IMM(val, LU12IW));
lu32id = larch_insn_gen_lu32id(LOONGARCH_GPR_T1, ADDR_IMM(val, LU32ID));
lu52id = larch_insn_gen_lu52id(LOONGARCH_GPR_T1, LOONGARCH_GPR_T1, ADDR_IMM(val, LU52ID));
jirl = larch_insn_gen_jirl(0, LOONGARCH_GPR_T1, 0, (val & 0xfff));
diff --git a/arch/loongarch/include/asm/module.lds.h b/arch/loongarch/include/asm/module.lds.h
index a3d1bc0fcc72..438f09d4ccf4 100644
--- a/arch/loongarch/include/asm/module.lds.h
+++ b/arch/loongarch/include/asm/module.lds.h
@@ -5,4 +5,5 @@ SECTIONS {
.got : { BYTE(0) }
.plt : { BYTE(0) }
.plt.idx : { BYTE(0) }
+ .ftrace_trampoline : { BYTE(0) }
}
diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
index cd64887e26b5..b070ce225989 100644
--- a/arch/loongarch/kernel/ftrace_dyn.c
+++ b/arch/loongarch/kernel/ftrace_dyn.c
@@ -9,6 +9,7 @@
#include <linux/uaccess.h>
#include <asm/inst.h>
+#include <asm/module.h>
static int ftrace_modify_code(unsigned long pc, u32 old, u32 new,
bool validate)
@@ -72,12 +73,63 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
return ftrace_modify_code(pc, old, new, true);
}
+static inline int __get_mod(struct module **mod, unsigned long addr)
+{
+ preempt_disable();
+ *mod = __module_text_address(addr);
+ preempt_enable();
+
+ if (WARN_ON(!(*mod)))
+ return -EINVAL;
+
+ return 0;
+}
+
+static struct plt_entry *get_ftrace_plt(struct module *mod, unsigned long addr)
+{
+ struct plt_entry *plt = mod->arch.ftrace_trampolines;
+
+ if (addr == FTRACE_ADDR)
+ return &plt[FTRACE_PLT_IDX];
+ if (addr == FTRACE_REGS_ADDR &&
+ IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ return &plt[FTRACE_REGS_PLT_IDX];
+
+ return NULL;
+}
+
+static unsigned long get_plt_addr(struct module *mod, unsigned long addr)
+{
+ struct plt_entry *plt;
+
+ plt = get_ftrace_plt(mod, addr);
+ if (!plt) {
+ pr_err("ftrace: no module PLT for %ps\n", (void *)addr);
+ return -EINVAL;
+ }
+
+ return (unsigned long)plt;
+}
+
int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
unsigned long pc;
+ long offset;
u32 old, new;
pc = rec->ip + LOONGARCH_INSN_SIZE;
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+ }
old = larch_insn_gen_nop();
new = larch_insn_gen_bl(pc, addr);
@@ -89,9 +141,22 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
unsigned long addr)
{
unsigned long pc;
+ long offset;
u32 old, new;
pc = rec->ip + LOONGARCH_INSN_SIZE;
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+ }
new = larch_insn_gen_nop();
old = larch_insn_gen_bl(pc, addr);
@@ -108,6 +173,20 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
u32 old, new;
pc = rec->ip + LOONGARCH_INSN_SIZE;
+ offset = (long)pc - (long)addr;
+
+ if (offset < -SZ_128M || offset >= SZ_128M) {
+ int ret;
+ struct module *mod;
+
+ ret = __get_mod(&mod, pc);
+ if (ret)
+ return ret;
+
+ addr = get_plt_addr(mod, addr);
+
+ old_addr = get_plt_addr(mod, old_addr);
+ }
old = larch_insn_gen_bl(pc, old_addr);
new = larch_insn_gen_bl(pc, addr);
diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
index 2d2e942eb06a..0d6bd7000ba6 100644
--- a/arch/loongarch/kernel/inst.c
+++ b/arch/loongarch/kernel/inst.c
@@ -103,6 +103,17 @@ u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
return insn.word;
}
+u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int imm)
+{
+ union loongarch_instruction insn;
+
+ insn.reg1i20_format.opcode = lu12iw_op;
+ insn.reg1i20_format.rd = rd;
+ insn.reg1i20_format.immediate = imm;
+
+ return insn.word;
+}
+
u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
{
union loongarch_instruction insn;
diff --git a/arch/loongarch/kernel/module-sections.c b/arch/loongarch/kernel/module-sections.c
index d296a70b758f..bd1a96691c98 100644
--- a/arch/loongarch/kernel/module-sections.c
+++ b/arch/loongarch/kernel/module-sections.c
@@ -4,6 +4,7 @@
*/
#include <linux/elf.h>
+#include <linux/ftrace.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -103,6 +104,7 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
char *secstrings, struct module *mod)
{
unsigned int i, num_plts = 0, num_gots = 0;
+ Elf_Shdr *tramp = NULL;
/*
* Find the empty .plt sections.
@@ -114,6 +116,8 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
mod->arch.plt.shdr = sechdrs + i;
else if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt.idx"))
mod->arch.plt_idx.shdr = sechdrs + i;
+ else if (!strcmp(secstrings + sechdrs[i].sh_name, ".ftrace_trampoline"))
+ tramp = sechdrs + i;
}
if (!mod->arch.got.shdr) {
@@ -166,5 +170,12 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
mod->arch.plt_idx.num_entries = 0;
mod->arch.plt_idx.max_entries = num_plts;
+ if (tramp) {
+ tramp->sh_type = SHT_NOBITS;
+ tramp->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ tramp->sh_addralign = __alignof__(struct plt_entry);
+ tramp->sh_size = NR_FTRACE_PLTS * sizeof(struct plt_entry);
+ }
+
return 0;
}
diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c
index 825fcf77f9e7..a606a0cab968 100644
--- a/arch/loongarch/kernel/module.c
+++ b/arch/loongarch/kernel/module.c
@@ -10,6 +10,7 @@
#include <linux/moduleloader.h>
#include <linux/elf.h>
+#include <linux/ftrace.h>
#include <linux/mm.h>
#include <linux/numa.h>
#include <linux/vmalloc.h>
@@ -18,6 +19,7 @@
#include <linux/string.h>
#include <linux/kernel.h>
#include <asm/alternative.h>
+#include <asm/inst.h>
static int rela_stack_push(s64 stack_value, s64 *rela_stack, size_t *rela_stack_top)
{
@@ -458,6 +460,24 @@ void *module_alloc(unsigned long size)
GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0));
}
+static int module_init_ftrace_plt(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs,
+ struct module *mod)
+{
+#ifdef CONFIG_DYNAMIC_FTRACE
+ struct plt_entry *ftrace_plts;
+
+ ftrace_plts = (void *)sechdrs->sh_addr;
+
+ ftrace_plts[FTRACE_PLT_IDX] = emit_plt_entry(FTRACE_ADDR);
+
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ ftrace_plts[FTRACE_REGS_PLT_IDX] = emit_plt_entry(FTRACE_REGS_ADDR);
+
+ mod->arch.ftrace_trampolines = ftrace_plts;
+#endif
+ return 0;
+}
+
int module_finalize(const Elf_Ehdr *hdr,
const Elf_Shdr *sechdrs, struct module *mod)
{
@@ -467,6 +487,8 @@ int module_finalize(const Elf_Ehdr *hdr,
for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) {
if (!strcmp(".altinstructions", secstrs + s->sh_name))
apply_alternatives((void *)s->sh_addr, (void *)s->sh_addr + s->sh_size);
+ if (!strcmp(".ftrace_trampoline", secstrs + s->sh_name))
+ return module_init_ftrace_plt(hdr, s, mod);
}
return 0;
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
` (7 preceding siblings ...)
2022-11-15 2:55 ` [PATCH v6 8/9] LoongArch: modules/ftrace: Initialize PLT at load time Qing Zhang
@ 2022-11-15 2:55 ` Qing Zhang
2022-11-15 8:31 ` Huacai Chen
8 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 2:55 UTC (permalink / raw)
To: Huacai Chen, Steven Rostedt, Ingo Molnar; +Cc: loongarch, linux-kernel
Defaults enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS to convenient
ftrace tests.
Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
---
arch/loongarch/configs/loongson3_defconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig
index 2d4678e6189a..0bbab17609b0 100644
--- a/arch/loongarch/configs/loongson3_defconfig
+++ b/arch/loongarch/configs/loongson3_defconfig
@@ -34,6 +34,7 @@ CONFIG_SYSFS_DEPRECATED=y
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_EXPERT=y
+CONFIG_KALLSYMS_ALL=y
CONFIG_USERFAULTFD=y
CONFIG_PERF_EVENTS=y
# CONFIG_COMPAT_BRK is not set
@@ -845,6 +846,7 @@ CONFIG_CRYPTO_DEV_VIRTIO=m
CONFIG_PRINTK_TIME=y
CONFIG_STRIP_ASM_SYMS=y
CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_FS=y
# CONFIG_SCHED_DEBUG is not set
CONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_PREEMPT is not set
--
2.36.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v6 1/9] LoongArch/ftrace: Add basic support
2022-11-15 2:55 ` [PATCH v6 1/9] LoongArch/ftrace: Add basic support Qing Zhang
@ 2022-11-15 8:25 ` Huacai Chen
2022-11-15 8:50 ` Qing Zhang
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 8:25 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, Qing,
On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> This patch contains basic ftrace support for LoongArch.
> Specifically, function tracer (HAVE_FUNCTION_TRACER), function graph
> tracer (HAVE_FUNCTION_GRAPH_TRACER) are implemented following the
> instructions in Documentation/trace/ftrace-design.txt.
>
> Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
> Thus, it can be seen store RA and open stack before `call _mcount`.
> Find `open stack` at first, and then find `store RA`
>
> Note that the functions in both inst.c and time.c should not be
> hooked with the compiler's -pg option: to prevent infinite self-
> referencing for the former, and to ignore early setup stuff for the
> latter.
>
> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> ---
> arch/loongarch/Kconfig | 2 +
> arch/loongarch/Makefile | 5 ++
> arch/loongarch/include/asm/ftrace.h | 18 ++++++
> arch/loongarch/kernel/Makefile | 8 +++
> arch/loongarch/kernel/ftrace.c | 74 +++++++++++++++++++++++
> arch/loongarch/kernel/mcount.S | 94 +++++++++++++++++++++++++++++
> 6 files changed, 201 insertions(+)
> create mode 100644 arch/loongarch/include/asm/ftrace.h
> create mode 100644 arch/loongarch/kernel/ftrace.c
> create mode 100644 arch/loongarch/kernel/mcount.S
>
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index 1943f840e494..92c4ec8c8527 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -91,6 +91,8 @@ config LOONGARCH
> select HAVE_EBPF_JIT
> select HAVE_EXIT_THREAD
> select HAVE_FAST_GUP
> + select HAVE_FUNCTION_GRAPH_TRACER
This line should be moved to another patch.
> + select HAVE_FUNCTION_TRACER
> select HAVE_GENERIC_VDSO
> select HAVE_IOREMAP_PROT
> select HAVE_IRQ_EXIT_ON_IRQ_STACK
> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
> index a0fc1f9980e3..6832a8f891fd 100644
> --- a/arch/loongarch/Makefile
> +++ b/arch/loongarch/Makefile
> @@ -36,6 +36,11 @@ ifneq ($(SUBARCH),$(ARCH))
> endif
> endif
>
> +ifdef CONFIG_DYNAMIC_FTRACE
> +KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
> +CC_FLAGS_FTRACE := -fpatchable-function-entry=2
> +endif
> +
> ifdef CONFIG_64BIT
> ld-emul = $(64bit-emul)
> cflags-y += -mabi=lp64s
> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> new file mode 100644
> index 000000000000..6a3e76234618
> --- /dev/null
> +++ b/arch/loongarch/include/asm/ftrace.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> + */
> +
> +#ifndef _ASM_LOONGARCH_FTRACE_H
> +#define _ASM_LOONGARCH_FTRACE_H
> +
> +#ifdef CONFIG_FUNCTION_TRACER
> +#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
> +
> +#ifndef __ASSEMBLY__
> +extern void _mcount(void);
> +#define mcount _mcount
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* CONFIG_FUNCTION_TRACER */
> +#endif /* _ASM_LOONGARCH_FTRACE_H */
> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
> index 86744531b100..3f71bce1c7ce 100644
> --- a/arch/loongarch/kernel/Makefile
> +++ b/arch/loongarch/kernel/Makefile
> @@ -15,6 +15,14 @@ obj-$(CONFIG_EFI) += efi.o
>
> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
>
> +ifdef CONFIG_FUNCTION_TRACER
> +obj-y += mcount.o ftrace.o
> +CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
> +endif
> +
> obj-$(CONFIG_MODULES) += module.o module-sections.o
> obj-$(CONFIG_STACKTRACE) += stacktrace.o
>
> diff --git a/arch/loongarch/kernel/ftrace.c b/arch/loongarch/kernel/ftrace.c
> new file mode 100644
> index 000000000000..c8ddc5f11f32
> --- /dev/null
> +++ b/arch/loongarch/kernel/ftrace.c
> @@ -0,0 +1,74 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/uaccess.h>
> +#include <linux/init.h>
> +#include <linux/ftrace.h>
> +#include <linux/syscalls.h>
> +
> +#include <asm/asm.h>
> +#include <asm/asm-offsets.h>
> +#include <asm/cacheflush.h>
> +#include <asm/inst.h>
> +#include <asm/loongarch.h>
> +#include <asm/syscall.h>
> +#include <asm/unistd.h>
> +
> +#include <asm-generic/sections.h>
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
This should also be moved to another patch.
> +
> +/*
> + * As `call _mcount` follows LoongArch psABI, ra-saved operation and
> + * stack operation can be found before this insn.
> + */
> +
> +static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
> +{
> + union loongarch_instruction *insn;
> + int limit = 32;
> +
> + insn = (union loongarch_instruction *)insn_addr;
> +
> + do {
> + insn--;
> + limit--;
> +
> + if (is_ra_save_ins(insn))
> + *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
> +
> + } while (!is_stack_alloc_ins(insn) && limit);
> +
> + if (!limit)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +void prepare_ftrace_return(unsigned long self_addr,
> + unsigned long callsite_sp, unsigned long old)
> +{
> + int ra_off;
> + unsigned long return_hooker = (unsigned long)&return_to_handler;
> +
> + if (unlikely(ftrace_graph_is_dead()))
> + return;
> +
> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
> + return;
> +
> + if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
> + goto out;
> +
> + if (!function_graph_enter(old, self_addr, 0, NULL))
> + *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
> +
> + return;
> +
> +out:
> + ftrace_graph_stop();
> + WARN_ON(1);
> +}
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> diff --git a/arch/loongarch/kernel/mcount.S b/arch/loongarch/kernel/mcount.S
> new file mode 100644
> index 000000000000..3de7c2d7fd12
> --- /dev/null
> +++ b/arch/loongarch/kernel/mcount.S
> @@ -0,0 +1,94 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * LoongArch specific _mcount support
> + *
> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <asm/export.h>
> +#include <asm/regdef.h>
> +#include <asm/stackframe.h>
> +#include <asm/ftrace.h>
> +
> + .text
> +
> +#define MCOUNT_STACK_SIZE (2 * SZREG)
> +#define MCOUNT_S0_OFFSET (0)
> +#define MCOUNT_RA_OFFSET (SZREG)
> +
> + .macro MCOUNT_SAVE_REGS
> + PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
> + PTR_S s0, sp, MCOUNT_S0_OFFSET
> + PTR_S ra, sp, MCOUNT_RA_OFFSET
> + move s0, a0
> + .endm
> +
> + .macro MCOUNT_RESTORE_REGS
> + move a0, s0
> + PTR_L ra, sp, MCOUNT_RA_OFFSET
> + PTR_L s0, sp, MCOUNT_S0_OFFSET
> + PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
> + .endm
> +
> +
> +SYM_FUNC_START(_mcount)
> + la.pcrel t1, ftrace_stub
> + la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
> + PTR_L t2, t2, 0
> + beq t1, t2, fgraph_trace
> +
> + MCOUNT_SAVE_REGS
> +
> + move a0, ra /* arg0: self return address */
> + move a1, s0 /* arg1: parent's return address */
> + jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
> +
> + MCOUNT_RESTORE_REGS
> +
> +fgraph_trace:
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
The same as above.
Huacai
> + la.pcrel t1, ftrace_stub
> + la.pcrel t3, ftrace_graph_return
> + PTR_L t3, t3, 0
> + bne t1, t3, ftrace_graph_caller
> + la.pcrel t1, ftrace_graph_entry_stub
> + la.pcrel t3, ftrace_graph_entry
> + PTR_L t3, t3, 0
> + bne t1, t3, ftrace_graph_caller
> +#endif
> +
> + .globl ftrace_stub
> +ftrace_stub:
> + jr ra
> +SYM_FUNC_END(_mcount)
> +EXPORT_SYMBOL(_mcount)
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +SYM_FUNC_START(ftrace_graph_caller)
> + MCOUNT_SAVE_REGS
> +
> + PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
> + PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
> + move a2, s0 /* arg2: Callsite parent ra */
> + bl prepare_ftrace_return
> +
> + MCOUNT_RESTORE_REGS
> + jr ra
> +SYM_FUNC_END(ftrace_graph_caller)
> +
> +SYM_FUNC_START(return_to_handler)
> + PTR_ADDI sp, sp, -2 * SZREG
> + PTR_S a0, sp, 0
> + PTR_S a1, sp, SZREG
> +
> + bl ftrace_return_to_handler
> +
> + /* restore the real parent address: a0 -> ra */
> + move ra, a0
> +
> + PTR_L a0, sp, 0
> + PTR_L a1, sp, SZREG
> + PTR_ADDI sp, sp, 2 * SZREG
> + jr ra
> +SYM_FUNC_END(return_to_handler)
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> --
> 2.36.0
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS
2022-11-15 2:55 ` [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS Qing Zhang
@ 2022-11-15 8:31 ` Huacai Chen
2022-11-15 8:47 ` Arnd Bergmann
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 8:31 UTC (permalink / raw)
To: Qing Zhang; +Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel
Hi, Qing,
I suggest not enabling debug mechanisms in the default config file,
distribution configs can make their own decisions.
Huacai
On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> Defaults enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS to convenient
> ftrace tests.
>
> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> ---
> arch/loongarch/configs/loongson3_defconfig | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig
> index 2d4678e6189a..0bbab17609b0 100644
> --- a/arch/loongarch/configs/loongson3_defconfig
> +++ b/arch/loongarch/configs/loongson3_defconfig
> @@ -34,6 +34,7 @@ CONFIG_SYSFS_DEPRECATED=y
> CONFIG_RELAY=y
> CONFIG_BLK_DEV_INITRD=y
> CONFIG_EXPERT=y
> +CONFIG_KALLSYMS_ALL=y
> CONFIG_USERFAULTFD=y
> CONFIG_PERF_EVENTS=y
> # CONFIG_COMPAT_BRK is not set
> @@ -845,6 +846,7 @@ CONFIG_CRYPTO_DEV_VIRTIO=m
> CONFIG_PRINTK_TIME=y
> CONFIG_STRIP_ASM_SYMS=y
> CONFIG_MAGIC_SYSRQ=y
> +CONFIG_DEBUG_FS=y
> # CONFIG_SCHED_DEBUG is not set
> CONFIG_SCHEDSTATS=y
> # CONFIG_DEBUG_PREEMPT is not set
> --
> 2.36.0
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 2:55 ` [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support Qing Zhang
@ 2022-11-15 8:41 ` Huacai Chen
2022-11-15 9:22 ` Qing Zhang
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 8:41 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, Qing,
Patch5 and Patch6 are small and related, maybe they can be combined
with a name "LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_{ARGS,REGS}
support"
Huacai
On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
> a traced function's arguments (and some other registers) to be captured
> into a struct pt_regs, allowing these to be inspected and modified.
>
> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> ---
> arch/loongarch/Kconfig | 1 +
> arch/loongarch/include/asm/ftrace.h | 3 +++
> arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
> arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
> 4 files changed, 55 insertions(+), 2 deletions(-)
>
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index 615ce62422b8..12e3e91a68ae 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -90,6 +90,7 @@ config LOONGARCH
> select HAVE_DEBUG_STACKOVERFLOW
> select HAVE_DMA_CONTIGUOUS
> select HAVE_DYNAMIC_FTRACE
> + select HAVE_DYNAMIC_FTRACE_WITH_REGS
> select HAVE_EBPF_JIT
> select HAVE_EXIT_THREAD
> select HAVE_FAST_GUP
> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> index 76ca58767f4d..a3f974a7a5ce 100644
> --- a/arch/loongarch/include/asm/ftrace.h
> +++ b/arch/loongarch/include/asm/ftrace.h
> @@ -28,6 +28,9 @@ struct dyn_ftrace;
> int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
> #define ftrace_init_nop ftrace_init_nop
>
> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> +#define ARCH_SUPPORTS_FTRACE_OPS 1
> +#endif
> #endif /* CONFIG_DYNAMIC_FTRACE */
> #endif /* __ASSEMBLY__ */
> #endif /* CONFIG_FUNCTION_TRACER */
> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
> index 3fe791b6783e..ec3d951be50c 100644
> --- a/arch/loongarch/kernel/ftrace_dyn.c
> +++ b/arch/loongarch/kernel/ftrace_dyn.c
> @@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> return ftrace_modify_code(pc, old, new, true);
> }
>
> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> + unsigned long addr)
> +{
> + unsigned long pc;
> + long offset;
> + u32 old, new;
> +
> + pc = rec->ip + LOONGARCH_INSN_SIZE;
> +
> + old = larch_insn_gen_bl(pc, old_addr);
> + new = larch_insn_gen_bl(pc, addr);
> +
> + return ftrace_modify_code(pc, old, new, true);
> +}
> +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
> +
> void arch_ftrace_update_code(int command)
> {
> command |= FTRACE_MAY_SLEEP;
> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
> index 0c12cc108e6f..02835186b463 100644
> --- a/arch/loongarch/kernel/mcount-dyn.S
> +++ b/arch/loongarch/kernel/mcount-dyn.S
> @@ -27,7 +27,7 @@
> * follows the LoongArch psABI well.
> */
>
> - .macro ftrace_regs_entry
> + .macro ftrace_regs_entry allregs=0
> PTR_ADDI sp, sp, -PT_SIZE
> /* Save trace function ra at PT_ERA */
> PTR_S ra, sp, PT_ERA
> @@ -43,16 +43,48 @@
> PTR_S a7, sp, PT_R11
> PTR_S fp, sp, PT_R22
>
> + .if \allregs
> + PTR_S t0, sp, PT_R12
> + PTR_S t1, sp, PT_R13
> + PTR_S t2, sp, PT_R14
> + PTR_S t3, sp, PT_R15
> + PTR_S t4, sp, PT_R16
> + PTR_S t5, sp, PT_R17
> + PTR_S t6, sp, PT_R18
> + PTR_S t7, sp, PT_R19
> + PTR_S t8, sp, PT_R20
> + PTR_S s0, sp, PT_R23
> + PTR_S s1, sp, PT_R24
> + PTR_S s2, sp, PT_R25
> + PTR_S s3, sp, PT_R26
> + PTR_S s4, sp, PT_R27
> + PTR_S s5, sp, PT_R28
> + PTR_S s6, sp, PT_R29
> + PTR_S s7, sp, PT_R30
> + PTR_S s8, sp, PT_R31
> + PTR_S tp, sp, PT_R2
> + /* Clear it for later use as a flag sometimes. */
> + PTR_S zero, sp, PT_R0
> + PTR_S $r21, sp, PT_R21
> + .endif
> +
> PTR_ADDI t8, sp, PT_SIZE
> PTR_S t8, sp, PT_R3
>
> .endm
>
> SYM_CODE_START(ftrace_caller)
> - ftrace_regs_entry
> + ftrace_regs_entry allregs=0
> b ftrace_common
> SYM_CODE_END(ftrace_caller)
>
> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> +SYM_CODE_START(ftrace_regs_caller)
> + ftrace_regs_entry allregs=1
> + b ftrace_common
> +SYM_CODE_END(ftrace_regs_caller)
> +#endif
> +
> SYM_CODE_START(ftrace_common)
> PTR_ADDI a0, ra, -8 /* arg0: ip */
> move a1, t0 /* arg1: parent_ip */
> --
> 2.36.0
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS
2022-11-15 8:31 ` Huacai Chen
@ 2022-11-15 8:47 ` Arnd Bergmann
2022-11-15 10:56 ` Huacai Chen
0 siblings, 1 reply; 24+ messages in thread
From: Arnd Bergmann @ 2022-11-15 8:47 UTC (permalink / raw)
To: Huacai Chen, Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel
On Tue, Nov 15, 2022, at 09:31, Huacai Chen wrote:
>
> I suggest not enabling debug mechanisms in the default config file,
> distribution configs can make their own decisions.
CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS are rather fundamental, and
are enabled in most other architectures as well. I would recommend
everything that is needed to run on the most common hardware
configurations and that is needed for booting into a general-purpose
distribution, but not much beyond that.
Other options like CONFIG_DEBUG_INFO and CONFIG_DEBUG_KERNEL (including
options based on it) are probably better left disabled though, those
are the ones with a significant runtime or buld-time overhead.
Arnd
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 1/9] LoongArch/ftrace: Add basic support
2022-11-15 8:25 ` Huacai Chen
@ 2022-11-15 8:50 ` Qing Zhang
2022-11-15 9:04 ` Huacai Chen
0 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 8:50 UTC (permalink / raw)
To: Huacai Chen
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, Huacai
On 2022/11/15 下午4:25, Huacai Chen wrote:
> Hi, Qing,
>
> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>>
>> This patch contains basic ftrace support for LoongArch.
>> Specifically, function tracer (HAVE_FUNCTION_TRACER), function graph
>> tracer (HAVE_FUNCTION_GRAPH_TRACER) are implemented following the
>> instructions in Documentation/trace/ftrace-design.txt.
>>
>> Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
>> Thus, it can be seen store RA and open stack before `call _mcount`.
>> Find `open stack` at first, and then find `store RA`
>>
>> Note that the functions in both inst.c and time.c should not be
>> hooked with the compiler's -pg option: to prevent infinite self-
>> referencing for the former, and to ignore early setup stuff for the
>> latter.
>>
>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
>> ---
>> arch/loongarch/Kconfig | 2 +
>> arch/loongarch/Makefile | 5 ++
>> arch/loongarch/include/asm/ftrace.h | 18 ++++++
>> arch/loongarch/kernel/Makefile | 8 +++
>> arch/loongarch/kernel/ftrace.c | 74 +++++++++++++++++++++++
>> arch/loongarch/kernel/mcount.S | 94 +++++++++++++++++++++++++++++
>> 6 files changed, 201 insertions(+)
>> create mode 100644 arch/loongarch/include/asm/ftrace.h
>> create mode 100644 arch/loongarch/kernel/ftrace.c
>> create mode 100644 arch/loongarch/kernel/mcount.S
>>
>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
>> index 1943f840e494..92c4ec8c8527 100644
>> --- a/arch/loongarch/Kconfig
>> +++ b/arch/loongarch/Kconfig
>> @@ -91,6 +91,8 @@ config LOONGARCH
>> select HAVE_EBPF_JIT
>> select HAVE_EXIT_THREAD
>> select HAVE_FAST_GUP
>> + select HAVE_FUNCTION_GRAPH_TRACER
> This line should be moved to another patch.
>
>> + select HAVE_FUNCTION_TRACER
>> select HAVE_GENERIC_VDSO
>> select HAVE_IOREMAP_PROT
>> select HAVE_IRQ_EXIT_ON_IRQ_STACK
>> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
>> index a0fc1f9980e3..6832a8f891fd 100644
>> --- a/arch/loongarch/Makefile
>> +++ b/arch/loongarch/Makefile
>> @@ -36,6 +36,11 @@ ifneq ($(SUBARCH),$(ARCH))
>> endif
>> endif
>>
>> +ifdef CONFIG_DYNAMIC_FTRACE
>> +KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
>> +CC_FLAGS_FTRACE := -fpatchable-function-entry=2
>> +endif
>> +
>> ifdef CONFIG_64BIT
>> ld-emul = $(64bit-emul)
>> cflags-y += -mabi=lp64s
>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
>> new file mode 100644
>> index 000000000000..6a3e76234618
>> --- /dev/null
>> +++ b/arch/loongarch/include/asm/ftrace.h
>> @@ -0,0 +1,18 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>> + */
>> +
>> +#ifndef _ASM_LOONGARCH_FTRACE_H
>> +#define _ASM_LOONGARCH_FTRACE_H
>> +
>> +#ifdef CONFIG_FUNCTION_TRACER
>> +#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
>> +
>> +#ifndef __ASSEMBLY__
>> +extern void _mcount(void);
>> +#define mcount _mcount
>> +
>> +#endif /* __ASSEMBLY__ */
>> +#endif /* CONFIG_FUNCTION_TRACER */
>> +#endif /* _ASM_LOONGARCH_FTRACE_H */
>> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
>> index 86744531b100..3f71bce1c7ce 100644
>> --- a/arch/loongarch/kernel/Makefile
>> +++ b/arch/loongarch/kernel/Makefile
>> @@ -15,6 +15,14 @@ obj-$(CONFIG_EFI) += efi.o
>>
>> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
>>
>> +ifdef CONFIG_FUNCTION_TRACER
>> +obj-y += mcount.o ftrace.o
>> +CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
>> +CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
>> +CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
>> +CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
>> +endif
>> +
>> obj-$(CONFIG_MODULES) += module.o module-sections.o
>> obj-$(CONFIG_STACKTRACE) += stacktrace.o
>>
>> diff --git a/arch/loongarch/kernel/ftrace.c b/arch/loongarch/kernel/ftrace.c
>> new file mode 100644
>> index 000000000000..c8ddc5f11f32
>> --- /dev/null
>> +++ b/arch/loongarch/kernel/ftrace.c
>> @@ -0,0 +1,74 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>> + */
>> +
>> +#include <linux/uaccess.h>
>> +#include <linux/init.h>
>> +#include <linux/ftrace.h>
>> +#include <linux/syscalls.h>
>> +
>> +#include <asm/asm.h>
>> +#include <asm/asm-offsets.h>
>> +#include <asm/cacheflush.h>
>> +#include <asm/inst.h>
>> +#include <asm/loongarch.h>
>> +#include <asm/syscall.h>
>> +#include <asm/unistd.h>
>> +
>> +#include <asm-generic/sections.h>
>> +
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> This should also be moved to another patch.
No, CONFIG_FUNCTION_GRAPH_TRACER is the basic feature, as the first
patch for non-dynamic ftrace, unless it is a separate patch, which is
obviously not necessary (Refer to the first submission of riscv and
arm64 here).
Thanks,
- Qing
>
>> +
>> +/*
>> + * As `call _mcount` follows LoongArch psABI, ra-saved operation and
>> + * stack operation can be found before this insn.
>> + */
>> +
>> +static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
>> +{
>> + union loongarch_instruction *insn;
>> + int limit = 32;
>> +
>> + insn = (union loongarch_instruction *)insn_addr;
>> +
>> + do {
>> + insn--;
>> + limit--;
>> +
>> + if (is_ra_save_ins(insn))
>> + *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
>> +
>> + } while (!is_stack_alloc_ins(insn) && limit);
>> +
>> + if (!limit)
>> + return -EINVAL;
>> +
>> + return 0;
>> +}
>> +
>> +void prepare_ftrace_return(unsigned long self_addr,
>> + unsigned long callsite_sp, unsigned long old)
>> +{
>> + int ra_off;
>> + unsigned long return_hooker = (unsigned long)&return_to_handler;
>> +
>> + if (unlikely(ftrace_graph_is_dead()))
>> + return;
>> +
>> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
>> + return;
>> +
>> + if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
>> + goto out;
>> +
>> + if (!function_graph_enter(old, self_addr, 0, NULL))
>> + *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
>> +
>> + return;
>> +
>> +out:
>> + ftrace_graph_stop();
>> + WARN_ON(1);
>> +}
>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>> diff --git a/arch/loongarch/kernel/mcount.S b/arch/loongarch/kernel/mcount.S
>> new file mode 100644
>> index 000000000000..3de7c2d7fd12
>> --- /dev/null
>> +++ b/arch/loongarch/kernel/mcount.S
>> @@ -0,0 +1,94 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * LoongArch specific _mcount support
>> + *
>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>> + */
>> +
>> +#include <asm/export.h>
>> +#include <asm/regdef.h>
>> +#include <asm/stackframe.h>
>> +#include <asm/ftrace.h>
>> +
>> + .text
>> +
>> +#define MCOUNT_STACK_SIZE (2 * SZREG)
>> +#define MCOUNT_S0_OFFSET (0)
>> +#define MCOUNT_RA_OFFSET (SZREG)
>> +
>> + .macro MCOUNT_SAVE_REGS
>> + PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
>> + PTR_S s0, sp, MCOUNT_S0_OFFSET
>> + PTR_S ra, sp, MCOUNT_RA_OFFSET
>> + move s0, a0
>> + .endm
>> +
>> + .macro MCOUNT_RESTORE_REGS
>> + move a0, s0
>> + PTR_L ra, sp, MCOUNT_RA_OFFSET
>> + PTR_L s0, sp, MCOUNT_S0_OFFSET
>> + PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
>> + .endm
>> +
>> +
>> +SYM_FUNC_START(_mcount)
>> + la.pcrel t1, ftrace_stub
>> + la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
>> + PTR_L t2, t2, 0
>> + beq t1, t2, fgraph_trace
>> +
>> + MCOUNT_SAVE_REGS
>> +
>> + move a0, ra /* arg0: self return address */
>> + move a1, s0 /* arg1: parent's return address */
>> + jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
>> +
>> + MCOUNT_RESTORE_REGS
>> +
>> +fgraph_trace:
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> The same as above.
>
> Huacai
>> + la.pcrel t1, ftrace_stub
>> + la.pcrel t3, ftrace_graph_return
>> + PTR_L t3, t3, 0
>> + bne t1, t3, ftrace_graph_caller
>> + la.pcrel t1, ftrace_graph_entry_stub
>> + la.pcrel t3, ftrace_graph_entry
>> + PTR_L t3, t3, 0
>> + bne t1, t3, ftrace_graph_caller
>> +#endif
>> +
>> + .globl ftrace_stub
>> +ftrace_stub:
>> + jr ra
>> +SYM_FUNC_END(_mcount)
>> +EXPORT_SYMBOL(_mcount)
>> +
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> +SYM_FUNC_START(ftrace_graph_caller)
>> + MCOUNT_SAVE_REGS
>> +
>> + PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
>> + PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
>> + move a2, s0 /* arg2: Callsite parent ra */
>> + bl prepare_ftrace_return
>> +
>> + MCOUNT_RESTORE_REGS
>> + jr ra
>> +SYM_FUNC_END(ftrace_graph_caller)
>> +
>> +SYM_FUNC_START(return_to_handler)
>> + PTR_ADDI sp, sp, -2 * SZREG
>> + PTR_S a0, sp, 0
>> + PTR_S a1, sp, SZREG
>> +
>> + bl ftrace_return_to_handler
>> +
>> + /* restore the real parent address: a0 -> ra */
>> + move ra, a0
>> +
>> + PTR_L a0, sp, 0
>> + PTR_L a1, sp, SZREG
>> + PTR_ADDI sp, sp, 2 * SZREG
>> + jr ra
>> +SYM_FUNC_END(return_to_handler)
>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>> --
>> 2.36.0
>>
>>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 1/9] LoongArch/ftrace: Add basic support
2022-11-15 8:50 ` Qing Zhang
@ 2022-11-15 9:04 ` Huacai Chen
2022-11-15 11:06 ` Qing Zhang
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 9:04 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
On Tue, Nov 15, 2022 at 4:51 PM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> Hi, Huacai
>
> On 2022/11/15 下午4:25, Huacai Chen wrote:
> > Hi, Qing,
> >
> > On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
> >>
> >> This patch contains basic ftrace support for LoongArch.
> >> Specifically, function tracer (HAVE_FUNCTION_TRACER), function graph
> >> tracer (HAVE_FUNCTION_GRAPH_TRACER) are implemented following the
> >> instructions in Documentation/trace/ftrace-design.txt.
> >>
> >> Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
> >> Thus, it can be seen store RA and open stack before `call _mcount`.
> >> Find `open stack` at first, and then find `store RA`
> >>
> >> Note that the functions in both inst.c and time.c should not be
> >> hooked with the compiler's -pg option: to prevent infinite self-
> >> referencing for the former, and to ignore early setup stuff for the
> >> latter.
> >>
> >> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> >> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> >> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> >> ---
> >> arch/loongarch/Kconfig | 2 +
> >> arch/loongarch/Makefile | 5 ++
> >> arch/loongarch/include/asm/ftrace.h | 18 ++++++
> >> arch/loongarch/kernel/Makefile | 8 +++
> >> arch/loongarch/kernel/ftrace.c | 74 +++++++++++++++++++++++
> >> arch/loongarch/kernel/mcount.S | 94 +++++++++++++++++++++++++++++
> >> 6 files changed, 201 insertions(+)
> >> create mode 100644 arch/loongarch/include/asm/ftrace.h
> >> create mode 100644 arch/loongarch/kernel/ftrace.c
> >> create mode 100644 arch/loongarch/kernel/mcount.S
> >>
> >> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> >> index 1943f840e494..92c4ec8c8527 100644
> >> --- a/arch/loongarch/Kconfig
> >> +++ b/arch/loongarch/Kconfig
> >> @@ -91,6 +91,8 @@ config LOONGARCH
> >> select HAVE_EBPF_JIT
> >> select HAVE_EXIT_THREAD
> >> select HAVE_FAST_GUP
> >> + select HAVE_FUNCTION_GRAPH_TRACER
> > This line should be moved to another patch.
> >
> >> + select HAVE_FUNCTION_TRACER
> >> select HAVE_GENERIC_VDSO
> >> select HAVE_IOREMAP_PROT
> >> select HAVE_IRQ_EXIT_ON_IRQ_STACK
> >> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
> >> index a0fc1f9980e3..6832a8f891fd 100644
> >> --- a/arch/loongarch/Makefile
> >> +++ b/arch/loongarch/Makefile
> >> @@ -36,6 +36,11 @@ ifneq ($(SUBARCH),$(ARCH))
> >> endif
> >> endif
> >>
> >> +ifdef CONFIG_DYNAMIC_FTRACE
> >> +KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
> >> +CC_FLAGS_FTRACE := -fpatchable-function-entry=2
> >> +endif
> >> +
> >> ifdef CONFIG_64BIT
> >> ld-emul = $(64bit-emul)
> >> cflags-y += -mabi=lp64s
> >> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> >> new file mode 100644
> >> index 000000000000..6a3e76234618
> >> --- /dev/null
> >> +++ b/arch/loongarch/include/asm/ftrace.h
> >> @@ -0,0 +1,18 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> +/*
> >> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> >> + */
> >> +
> >> +#ifndef _ASM_LOONGARCH_FTRACE_H
> >> +#define _ASM_LOONGARCH_FTRACE_H
> >> +
> >> +#ifdef CONFIG_FUNCTION_TRACER
> >> +#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
> >> +
> >> +#ifndef __ASSEMBLY__
> >> +extern void _mcount(void);
> >> +#define mcount _mcount
> >> +
> >> +#endif /* __ASSEMBLY__ */
> >> +#endif /* CONFIG_FUNCTION_TRACER */
> >> +#endif /* _ASM_LOONGARCH_FTRACE_H */
> >> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
> >> index 86744531b100..3f71bce1c7ce 100644
> >> --- a/arch/loongarch/kernel/Makefile
> >> +++ b/arch/loongarch/kernel/Makefile
> >> @@ -15,6 +15,14 @@ obj-$(CONFIG_EFI) += efi.o
> >>
> >> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
> >>
> >> +ifdef CONFIG_FUNCTION_TRACER
> >> +obj-y += mcount.o ftrace.o
> >> +CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
> >> +CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
> >> +CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
> >> +CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
> >> +endif
> >> +
> >> obj-$(CONFIG_MODULES) += module.o module-sections.o
> >> obj-$(CONFIG_STACKTRACE) += stacktrace.o
> >>
> >> diff --git a/arch/loongarch/kernel/ftrace.c b/arch/loongarch/kernel/ftrace.c
> >> new file mode 100644
> >> index 000000000000..c8ddc5f11f32
> >> --- /dev/null
> >> +++ b/arch/loongarch/kernel/ftrace.c
> >> @@ -0,0 +1,74 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> >> + */
> >> +
> >> +#include <linux/uaccess.h>
> >> +#include <linux/init.h>
> >> +#include <linux/ftrace.h>
> >> +#include <linux/syscalls.h>
> >> +
> >> +#include <asm/asm.h>
> >> +#include <asm/asm-offsets.h>
> >> +#include <asm/cacheflush.h>
> >> +#include <asm/inst.h>
> >> +#include <asm/loongarch.h>
> >> +#include <asm/syscall.h>
> >> +#include <asm/unistd.h>
> >> +
> >> +#include <asm-generic/sections.h>
> >> +
> >> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> > This should also be moved to another patch.
>
> No, CONFIG_FUNCTION_GRAPH_TRACER is the basic feature, as the first
> patch for non-dynamic ftrace, unless it is a separate patch, which is
> obviously not necessary (Refer to the first submission of riscv and
> arm64 here).
But FUNCTION_GRAPH_TRACER is added until Patch-4.
Huacai
>
> Thanks,
> - Qing
> >
> >> +
> >> +/*
> >> + * As `call _mcount` follows LoongArch psABI, ra-saved operation and
> >> + * stack operation can be found before this insn.
> >> + */
> >> +
> >> +static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
> >> +{
> >> + union loongarch_instruction *insn;
> >> + int limit = 32;
> >> +
> >> + insn = (union loongarch_instruction *)insn_addr;
> >> +
> >> + do {
> >> + insn--;
> >> + limit--;
> >> +
> >> + if (is_ra_save_ins(insn))
> >> + *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
> >> +
> >> + } while (!is_stack_alloc_ins(insn) && limit);
> >> +
> >> + if (!limit)
> >> + return -EINVAL;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +void prepare_ftrace_return(unsigned long self_addr,
> >> + unsigned long callsite_sp, unsigned long old)
> >> +{
> >> + int ra_off;
> >> + unsigned long return_hooker = (unsigned long)&return_to_handler;
> >> +
> >> + if (unlikely(ftrace_graph_is_dead()))
> >> + return;
> >> +
> >> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
> >> + return;
> >> +
> >> + if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
> >> + goto out;
> >> +
> >> + if (!function_graph_enter(old, self_addr, 0, NULL))
> >> + *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
> >> +
> >> + return;
> >> +
> >> +out:
> >> + ftrace_graph_stop();
> >> + WARN_ON(1);
> >> +}
> >> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> >> diff --git a/arch/loongarch/kernel/mcount.S b/arch/loongarch/kernel/mcount.S
> >> new file mode 100644
> >> index 000000000000..3de7c2d7fd12
> >> --- /dev/null
> >> +++ b/arch/loongarch/kernel/mcount.S
> >> @@ -0,0 +1,94 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> +/*
> >> + * LoongArch specific _mcount support
> >> + *
> >> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> >> + */
> >> +
> >> +#include <asm/export.h>
> >> +#include <asm/regdef.h>
> >> +#include <asm/stackframe.h>
> >> +#include <asm/ftrace.h>
> >> +
> >> + .text
> >> +
> >> +#define MCOUNT_STACK_SIZE (2 * SZREG)
> >> +#define MCOUNT_S0_OFFSET (0)
> >> +#define MCOUNT_RA_OFFSET (SZREG)
> >> +
> >> + .macro MCOUNT_SAVE_REGS
> >> + PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
> >> + PTR_S s0, sp, MCOUNT_S0_OFFSET
> >> + PTR_S ra, sp, MCOUNT_RA_OFFSET
> >> + move s0, a0
> >> + .endm
> >> +
> >> + .macro MCOUNT_RESTORE_REGS
> >> + move a0, s0
> >> + PTR_L ra, sp, MCOUNT_RA_OFFSET
> >> + PTR_L s0, sp, MCOUNT_S0_OFFSET
> >> + PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
> >> + .endm
> >> +
> >> +
> >> +SYM_FUNC_START(_mcount)
> >> + la.pcrel t1, ftrace_stub
> >> + la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
> >> + PTR_L t2, t2, 0
> >> + beq t1, t2, fgraph_trace
> >> +
> >> + MCOUNT_SAVE_REGS
> >> +
> >> + move a0, ra /* arg0: self return address */
> >> + move a1, s0 /* arg1: parent's return address */
> >> + jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
> >> +
> >> + MCOUNT_RESTORE_REGS
> >> +
> >> +fgraph_trace:
> >> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> > The same as above.
> >
> > Huacai
> >> + la.pcrel t1, ftrace_stub
> >> + la.pcrel t3, ftrace_graph_return
> >> + PTR_L t3, t3, 0
> >> + bne t1, t3, ftrace_graph_caller
> >> + la.pcrel t1, ftrace_graph_entry_stub
> >> + la.pcrel t3, ftrace_graph_entry
> >> + PTR_L t3, t3, 0
> >> + bne t1, t3, ftrace_graph_caller
> >> +#endif
> >> +
> >> + .globl ftrace_stub
> >> +ftrace_stub:
> >> + jr ra
> >> +SYM_FUNC_END(_mcount)
> >> +EXPORT_SYMBOL(_mcount)
> >> +
> >> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> >> +SYM_FUNC_START(ftrace_graph_caller)
> >> + MCOUNT_SAVE_REGS
> >> +
> >> + PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
> >> + PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
> >> + move a2, s0 /* arg2: Callsite parent ra */
> >> + bl prepare_ftrace_return
> >> +
> >> + MCOUNT_RESTORE_REGS
> >> + jr ra
> >> +SYM_FUNC_END(ftrace_graph_caller)
> >> +
> >> +SYM_FUNC_START(return_to_handler)
> >> + PTR_ADDI sp, sp, -2 * SZREG
> >> + PTR_S a0, sp, 0
> >> + PTR_S a1, sp, SZREG
> >> +
> >> + bl ftrace_return_to_handler
> >> +
> >> + /* restore the real parent address: a0 -> ra */
> >> + move ra, a0
> >> +
> >> + PTR_L a0, sp, 0
> >> + PTR_L a1, sp, SZREG
> >> + PTR_ADDI sp, sp, 2 * SZREG
> >> + jr ra
> >> +SYM_FUNC_END(return_to_handler)
> >> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> >> --
> >> 2.36.0
> >>
> >>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 8:41 ` Huacai Chen
@ 2022-11-15 9:22 ` Qing Zhang
2022-11-15 9:24 ` Huacai Chen
0 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 9:22 UTC (permalink / raw)
To: Huacai Chen
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, Huacai
On 2022/11/15 下午4:41, Huacai Chen wrote:
> Hi, Qing,
>
> Patch5 and Patch6 are small and related, maybe they can be combined
> with a name "LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_{ARGS,REGS}
> support"
I think they should be independent due to different use scenarios.
Thanks,
-Qing
>
> Huacai
>
> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>>
>> This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
>> a traced function's arguments (and some other registers) to be captured
>> into a struct pt_regs, allowing these to be inspected and modified.
>>
>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
>> ---
>> arch/loongarch/Kconfig | 1 +
>> arch/loongarch/include/asm/ftrace.h | 3 +++
>> arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
>> arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
>> 4 files changed, 55 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
>> index 615ce62422b8..12e3e91a68ae 100644
>> --- a/arch/loongarch/Kconfig
>> +++ b/arch/loongarch/Kconfig
>> @@ -90,6 +90,7 @@ config LOONGARCH
>> select HAVE_DEBUG_STACKOVERFLOW
>> select HAVE_DMA_CONTIGUOUS
>> select HAVE_DYNAMIC_FTRACE
>> + select HAVE_DYNAMIC_FTRACE_WITH_REGS
>> select HAVE_EBPF_JIT
>> select HAVE_EXIT_THREAD
>> select HAVE_FAST_GUP
>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
>> index 76ca58767f4d..a3f974a7a5ce 100644
>> --- a/arch/loongarch/include/asm/ftrace.h
>> +++ b/arch/loongarch/include/asm/ftrace.h
>> @@ -28,6 +28,9 @@ struct dyn_ftrace;
>> int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
>> #define ftrace_init_nop ftrace_init_nop
>>
>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>> +#define ARCH_SUPPORTS_FTRACE_OPS 1
>> +#endif
>> #endif /* CONFIG_DYNAMIC_FTRACE */
>> #endif /* __ASSEMBLY__ */
>> #endif /* CONFIG_FUNCTION_TRACER */
>> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
>> index 3fe791b6783e..ec3d951be50c 100644
>> --- a/arch/loongarch/kernel/ftrace_dyn.c
>> +++ b/arch/loongarch/kernel/ftrace_dyn.c
>> @@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
>> return ftrace_modify_code(pc, old, new, true);
>> }
>>
>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>> + unsigned long addr)
>> +{
>> + unsigned long pc;
>> + long offset;
>> + u32 old, new;
>> +
>> + pc = rec->ip + LOONGARCH_INSN_SIZE;
>> +
>> + old = larch_insn_gen_bl(pc, old_addr);
>> + new = larch_insn_gen_bl(pc, addr);
>> +
>> + return ftrace_modify_code(pc, old, new, true);
>> +}
>> +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
>> +
>> void arch_ftrace_update_code(int command)
>> {
>> command |= FTRACE_MAY_SLEEP;
>> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
>> index 0c12cc108e6f..02835186b463 100644
>> --- a/arch/loongarch/kernel/mcount-dyn.S
>> +++ b/arch/loongarch/kernel/mcount-dyn.S
>> @@ -27,7 +27,7 @@
>> * follows the LoongArch psABI well.
>> */
>>
>> - .macro ftrace_regs_entry
>> + .macro ftrace_regs_entry allregs=0
>> PTR_ADDI sp, sp, -PT_SIZE
>> /* Save trace function ra at PT_ERA */
>> PTR_S ra, sp, PT_ERA
>> @@ -43,16 +43,48 @@
>> PTR_S a7, sp, PT_R11
>> PTR_S fp, sp, PT_R22
>>
>> + .if \allregs
>> + PTR_S t0, sp, PT_R12
>> + PTR_S t1, sp, PT_R13
>> + PTR_S t2, sp, PT_R14
>> + PTR_S t3, sp, PT_R15
>> + PTR_S t4, sp, PT_R16
>> + PTR_S t5, sp, PT_R17
>> + PTR_S t6, sp, PT_R18
>> + PTR_S t7, sp, PT_R19
>> + PTR_S t8, sp, PT_R20
>> + PTR_S s0, sp, PT_R23
>> + PTR_S s1, sp, PT_R24
>> + PTR_S s2, sp, PT_R25
>> + PTR_S s3, sp, PT_R26
>> + PTR_S s4, sp, PT_R27
>> + PTR_S s5, sp, PT_R28
>> + PTR_S s6, sp, PT_R29
>> + PTR_S s7, sp, PT_R30
>> + PTR_S s8, sp, PT_R31
>> + PTR_S tp, sp, PT_R2
>> + /* Clear it for later use as a flag sometimes. */
>> + PTR_S zero, sp, PT_R0
>> + PTR_S $r21, sp, PT_R21
>> + .endif
>> +
>> PTR_ADDI t8, sp, PT_SIZE
>> PTR_S t8, sp, PT_R3
>>
>> .endm
>>
>> SYM_CODE_START(ftrace_caller)
>> - ftrace_regs_entry
>> + ftrace_regs_entry allregs=0
>> b ftrace_common
>> SYM_CODE_END(ftrace_caller)
>>
>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>> +SYM_CODE_START(ftrace_regs_caller)
>> + ftrace_regs_entry allregs=1
>> + b ftrace_common
>> +SYM_CODE_END(ftrace_regs_caller)
>> +#endif
>> +
>> SYM_CODE_START(ftrace_common)
>> PTR_ADDI a0, ra, -8 /* arg0: ip */
>> move a1, t0 /* arg1: parent_ip */
>> --
>> 2.36.0
>>
>>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 9:22 ` Qing Zhang
@ 2022-11-15 9:24 ` Huacai Chen
2022-11-15 10:44 ` Qing Zhang
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 9:24 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
On Tue, Nov 15, 2022 at 5:22 PM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> Hi, Huacai
>
> On 2022/11/15 下午4:41, Huacai Chen wrote:
> > Hi, Qing,
> >
> > Patch5 and Patch6 are small and related, maybe they can be combined
> > with a name "LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_{ARGS,REGS}
> > support"
> I think they should be independent due to different use scenarios.
From the commit message it seems DYNAMIC_FTRACE_WITH_ARGS depend on
DYNAMIC_FTRACE_WITH_REGS.
Huacai
>
> Thanks,
> -Qing
> >
> > Huacai
> >
> > On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
> >>
> >> This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
> >> a traced function's arguments (and some other registers) to be captured
> >> into a struct pt_regs, allowing these to be inspected and modified.
> >>
> >> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> >> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> >> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> >> ---
> >> arch/loongarch/Kconfig | 1 +
> >> arch/loongarch/include/asm/ftrace.h | 3 +++
> >> arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
> >> arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
> >> 4 files changed, 55 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> >> index 615ce62422b8..12e3e91a68ae 100644
> >> --- a/arch/loongarch/Kconfig
> >> +++ b/arch/loongarch/Kconfig
> >> @@ -90,6 +90,7 @@ config LOONGARCH
> >> select HAVE_DEBUG_STACKOVERFLOW
> >> select HAVE_DMA_CONTIGUOUS
> >> select HAVE_DYNAMIC_FTRACE
> >> + select HAVE_DYNAMIC_FTRACE_WITH_REGS
> >> select HAVE_EBPF_JIT
> >> select HAVE_EXIT_THREAD
> >> select HAVE_FAST_GUP
> >> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> >> index 76ca58767f4d..a3f974a7a5ce 100644
> >> --- a/arch/loongarch/include/asm/ftrace.h
> >> +++ b/arch/loongarch/include/asm/ftrace.h
> >> @@ -28,6 +28,9 @@ struct dyn_ftrace;
> >> int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
> >> #define ftrace_init_nop ftrace_init_nop
> >>
> >> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >> +#define ARCH_SUPPORTS_FTRACE_OPS 1
> >> +#endif
> >> #endif /* CONFIG_DYNAMIC_FTRACE */
> >> #endif /* __ASSEMBLY__ */
> >> #endif /* CONFIG_FUNCTION_TRACER */
> >> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
> >> index 3fe791b6783e..ec3d951be50c 100644
> >> --- a/arch/loongarch/kernel/ftrace_dyn.c
> >> +++ b/arch/loongarch/kernel/ftrace_dyn.c
> >> @@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> >> return ftrace_modify_code(pc, old, new, true);
> >> }
> >>
> >> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> >> + unsigned long addr)
> >> +{
> >> + unsigned long pc;
> >> + long offset;
> >> + u32 old, new;
> >> +
> >> + pc = rec->ip + LOONGARCH_INSN_SIZE;
> >> +
> >> + old = larch_insn_gen_bl(pc, old_addr);
> >> + new = larch_insn_gen_bl(pc, addr);
> >> +
> >> + return ftrace_modify_code(pc, old, new, true);
> >> +}
> >> +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
> >> +
> >> void arch_ftrace_update_code(int command)
> >> {
> >> command |= FTRACE_MAY_SLEEP;
> >> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
> >> index 0c12cc108e6f..02835186b463 100644
> >> --- a/arch/loongarch/kernel/mcount-dyn.S
> >> +++ b/arch/loongarch/kernel/mcount-dyn.S
> >> @@ -27,7 +27,7 @@
> >> * follows the LoongArch psABI well.
> >> */
> >>
> >> - .macro ftrace_regs_entry
> >> + .macro ftrace_regs_entry allregs=0
> >> PTR_ADDI sp, sp, -PT_SIZE
> >> /* Save trace function ra at PT_ERA */
> >> PTR_S ra, sp, PT_ERA
> >> @@ -43,16 +43,48 @@
> >> PTR_S a7, sp, PT_R11
> >> PTR_S fp, sp, PT_R22
> >>
> >> + .if \allregs
> >> + PTR_S t0, sp, PT_R12
> >> + PTR_S t1, sp, PT_R13
> >> + PTR_S t2, sp, PT_R14
> >> + PTR_S t3, sp, PT_R15
> >> + PTR_S t4, sp, PT_R16
> >> + PTR_S t5, sp, PT_R17
> >> + PTR_S t6, sp, PT_R18
> >> + PTR_S t7, sp, PT_R19
> >> + PTR_S t8, sp, PT_R20
> >> + PTR_S s0, sp, PT_R23
> >> + PTR_S s1, sp, PT_R24
> >> + PTR_S s2, sp, PT_R25
> >> + PTR_S s3, sp, PT_R26
> >> + PTR_S s4, sp, PT_R27
> >> + PTR_S s5, sp, PT_R28
> >> + PTR_S s6, sp, PT_R29
> >> + PTR_S s7, sp, PT_R30
> >> + PTR_S s8, sp, PT_R31
> >> + PTR_S tp, sp, PT_R2
> >> + /* Clear it for later use as a flag sometimes. */
> >> + PTR_S zero, sp, PT_R0
> >> + PTR_S $r21, sp, PT_R21
> >> + .endif
> >> +
> >> PTR_ADDI t8, sp, PT_SIZE
> >> PTR_S t8, sp, PT_R3
> >>
> >> .endm
> >>
> >> SYM_CODE_START(ftrace_caller)
> >> - ftrace_regs_entry
> >> + ftrace_regs_entry allregs=0
> >> b ftrace_common
> >> SYM_CODE_END(ftrace_caller)
> >>
> >> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >> +SYM_CODE_START(ftrace_regs_caller)
> >> + ftrace_regs_entry allregs=1
> >> + b ftrace_common
> >> +SYM_CODE_END(ftrace_regs_caller)
> >> +#endif
> >> +
> >> SYM_CODE_START(ftrace_common)
> >> PTR_ADDI a0, ra, -8 /* arg0: ip */
> >> move a1, t0 /* arg1: parent_ip */
> >> --
> >> 2.36.0
> >>
> >>
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 9:24 ` Huacai Chen
@ 2022-11-15 10:44 ` Qing Zhang
2022-11-15 10:55 ` Huacai Chen
0 siblings, 1 reply; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 10:44 UTC (permalink / raw)
To: Huacai Chen
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, huacai
On 2022/11/15 下午5:24, Huacai Chen wrote:
> On Tue, Nov 15, 2022 at 5:22 PM Qing Zhang <zhangqing@loongson.cn> wrote:
>>
>> Hi, Huacai
>>
>> On 2022/11/15 下午4:41, Huacai Chen wrote:
>>> Hi, Qing,
>>>
>>> Patch5 and Patch6 are small and related, maybe they can be combined
>>> with a name "LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_{ARGS,REGS}
>>> support"
>> I think they should be independent due to different use scenarios.
> From the commit message it seems DYNAMIC_FTRACE_WITH_ARGS depend on
> DYNAMIC_FTRACE_WITH_REGS.
DYNAMIC_FTRACE_WITH_REGS : which allows trace function saved registers,
It means that these registers can be modified, eg: Kprobe、Livepatch.
DYNAMIC_FTRACE_WITH_ARGS: uses pt_regs as a parameter for get sp,
which means it can be used to access registers on the stack, and
use graph_ops::func to install return_hooker to replace the special
hook, eg: Livepatch.
From the following Kconfig, they are reasonable as separate patches.
config LIVEPATCH
bool "Kernel Live Patching"
depends on DYNAMIC_FTRACE_WITH_REGS || DYNAMIC_FTRACE_WITH_ARGS
Thanks
- Qing
>
> Huacai
>>
>> Thanks,
>> -Qing
>>>
>>> Huacai
>>>
>>> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>>>>
>>>> This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
>>>> a traced function's arguments (and some other registers) to be captured
>>>> into a struct pt_regs, allowing these to be inspected and modified.
>>>>
>>>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
>>>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>>>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
>>>> ---
>>>> arch/loongarch/Kconfig | 1 +
>>>> arch/loongarch/include/asm/ftrace.h | 3 +++
>>>> arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
>>>> arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
>>>> 4 files changed, 55 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
>>>> index 615ce62422b8..12e3e91a68ae 100644
>>>> --- a/arch/loongarch/Kconfig
>>>> +++ b/arch/loongarch/Kconfig
>>>> @@ -90,6 +90,7 @@ config LOONGARCH
>>>> select HAVE_DEBUG_STACKOVERFLOW
>>>> select HAVE_DMA_CONTIGUOUS
>>>> select HAVE_DYNAMIC_FTRACE
>>>> + select HAVE_DYNAMIC_FTRACE_WITH_REGS
>>>> select HAVE_EBPF_JIT
>>>> select HAVE_EXIT_THREAD
>>>> select HAVE_FAST_GUP
>>>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
>>>> index 76ca58767f4d..a3f974a7a5ce 100644
>>>> --- a/arch/loongarch/include/asm/ftrace.h
>>>> +++ b/arch/loongarch/include/asm/ftrace.h
>>>> @@ -28,6 +28,9 @@ struct dyn_ftrace;
>>>> int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
>>>> #define ftrace_init_nop ftrace_init_nop
>>>>
>>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>>>> +#define ARCH_SUPPORTS_FTRACE_OPS 1
>>>> +#endif
>>>> #endif /* CONFIG_DYNAMIC_FTRACE */
>>>> #endif /* __ASSEMBLY__ */
>>>> #endif /* CONFIG_FUNCTION_TRACER */
>>>> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
>>>> index 3fe791b6783e..ec3d951be50c 100644
>>>> --- a/arch/loongarch/kernel/ftrace_dyn.c
>>>> +++ b/arch/loongarch/kernel/ftrace_dyn.c
>>>> @@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
>>>> return ftrace_modify_code(pc, old, new, true);
>>>> }
>>>>
>>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>>>> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>>>> + unsigned long addr)
>>>> +{
>>>> + unsigned long pc;
>>>> + long offset;
>>>> + u32 old, new;
>>>> +
>>>> + pc = rec->ip + LOONGARCH_INSN_SIZE;
>>>> +
>>>> + old = larch_insn_gen_bl(pc, old_addr);
>>>> + new = larch_insn_gen_bl(pc, addr);
>>>> +
>>>> + return ftrace_modify_code(pc, old, new, true);
>>>> +}
>>>> +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
>>>> +
>>>> void arch_ftrace_update_code(int command)
>>>> {
>>>> command |= FTRACE_MAY_SLEEP;
>>>> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
>>>> index 0c12cc108e6f..02835186b463 100644
>>>> --- a/arch/loongarch/kernel/mcount-dyn.S
>>>> +++ b/arch/loongarch/kernel/mcount-dyn.S
>>>> @@ -27,7 +27,7 @@
>>>> * follows the LoongArch psABI well.
>>>> */
>>>>
>>>> - .macro ftrace_regs_entry
>>>> + .macro ftrace_regs_entry allregs=0
>>>> PTR_ADDI sp, sp, -PT_SIZE
>>>> /* Save trace function ra at PT_ERA */
>>>> PTR_S ra, sp, PT_ERA
>>>> @@ -43,16 +43,48 @@
>>>> PTR_S a7, sp, PT_R11
>>>> PTR_S fp, sp, PT_R22
>>>>
>>>> + .if \allregs
>>>> + PTR_S t0, sp, PT_R12
>>>> + PTR_S t1, sp, PT_R13
>>>> + PTR_S t2, sp, PT_R14
>>>> + PTR_S t3, sp, PT_R15
>>>> + PTR_S t4, sp, PT_R16
>>>> + PTR_S t5, sp, PT_R17
>>>> + PTR_S t6, sp, PT_R18
>>>> + PTR_S t7, sp, PT_R19
>>>> + PTR_S t8, sp, PT_R20
>>>> + PTR_S s0, sp, PT_R23
>>>> + PTR_S s1, sp, PT_R24
>>>> + PTR_S s2, sp, PT_R25
>>>> + PTR_S s3, sp, PT_R26
>>>> + PTR_S s4, sp, PT_R27
>>>> + PTR_S s5, sp, PT_R28
>>>> + PTR_S s6, sp, PT_R29
>>>> + PTR_S s7, sp, PT_R30
>>>> + PTR_S s8, sp, PT_R31
>>>> + PTR_S tp, sp, PT_R2
>>>> + /* Clear it for later use as a flag sometimes. */
>>>> + PTR_S zero, sp, PT_R0
>>>> + PTR_S $r21, sp, PT_R21
>>>> + .endif
>>>> +
>>>> PTR_ADDI t8, sp, PT_SIZE
>>>> PTR_S t8, sp, PT_R3
>>>>
>>>> .endm
>>>>
>>>> SYM_CODE_START(ftrace_caller)
>>>> - ftrace_regs_entry
>>>> + ftrace_regs_entry allregs=0
>>>> b ftrace_common
>>>> SYM_CODE_END(ftrace_caller)
>>>>
>>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>>>> +SYM_CODE_START(ftrace_regs_caller)
>>>> + ftrace_regs_entry allregs=1
>>>> + b ftrace_common
>>>> +SYM_CODE_END(ftrace_regs_caller)
>>>> +#endif
>>>> +
>>>> SYM_CODE_START(ftrace_common)
>>>> PTR_ADDI a0, ra, -8 /* arg0: ip */
>>>> move a1, t0 /* arg1: parent_ip */
>>>> --
>>>> 2.36.0
>>>>
>>>>
>>
>>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support
2022-11-15 2:55 ` [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support Qing Zhang
@ 2022-11-15 10:54 ` Huacai Chen
2022-11-15 11:11 ` Qing Zhang
0 siblings, 1 reply; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 10:54 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, Qing,
On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>
> The compiler has inserted 2 NOPs before the regular function prologue.
> T series registers are available and safe because of LoongArch psABI.
>
> At runtime, replace nop with bl to enable ftrace call and replace bl with
> nop to disable ftrace call. The bl requires us to save the original RA
> value, so here it saves RA at t0.
> details are:
>
> | Compiled | Disabled | Enabled |
> +------------+------------------------+------------------------+
> | nop | move t0, ra | move t0, ra |
> | nop | nop | bl ftrace_caller |
> | func_body | func_body | func_body |
>
> The RA value will be recovered by ftrace_regs_entry, and restored into RA
> before returning to the regular function prologue. When a function is not
> being traced, the move t0, ra is not harmful.
>
> 1) ftrace_make_call, ftrace_make_nop (in kernel/ftrace.c)
> The two functions turn each recorded call site of filtered functions
> into a call to ftrace_caller or nops.
>
> 2) ftracce_update_ftrace_func (in kernel/ftrace.c)
> turns the nops at ftrace_call into a call to a generic entry for
> function tracers.
>
> 3) ftrace_caller (in kernel/mcount-dyn.S)
> The entry where each _mcount call sites calls to once they are
> filtered to be traced.
>
> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> ---
> arch/loongarch/Kconfig | 1 +
> arch/loongarch/include/asm/ftrace.h | 16 ++++
> arch/loongarch/include/asm/inst.h | 15 ++++
> arch/loongarch/include/asm/unwind.h | 2 +-
> arch/loongarch/kernel/Makefile | 5 ++
> arch/loongarch/kernel/ftrace_dyn.c | 111 ++++++++++++++++++++++++
> arch/loongarch/kernel/inst.c | 92 ++++++++++++++++++++
> arch/loongarch/kernel/mcount-dyn.S | 89 +++++++++++++++++++
> arch/loongarch/kernel/unwind_prologue.c | 35 ++++++--
> 9 files changed, 360 insertions(+), 6 deletions(-)
> create mode 100644 arch/loongarch/kernel/ftrace_dyn.c
> create mode 100644 arch/loongarch/kernel/mcount-dyn.S
>
> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> index e6b1defca1f1..615ce62422b8 100644
> --- a/arch/loongarch/Kconfig
> +++ b/arch/loongarch/Kconfig
> @@ -89,6 +89,7 @@ config LOONGARCH
> select HAVE_C_RECORDMCOUNT
> select HAVE_DEBUG_STACKOVERFLOW
> select HAVE_DMA_CONTIGUOUS
> + select HAVE_DYNAMIC_FTRACE
> select HAVE_EBPF_JIT
> select HAVE_EXIT_THREAD
> select HAVE_FAST_GUP
> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> index 6a3e76234618..76ca58767f4d 100644
> --- a/arch/loongarch/include/asm/ftrace.h
> +++ b/arch/loongarch/include/asm/ftrace.h
> @@ -10,9 +10,25 @@
> #define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
>
> #ifndef __ASSEMBLY__
> +#ifndef CONFIG_DYNAMIC_FTRACE
> extern void _mcount(void);
> #define mcount _mcount
> +#endif
>
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +static inline unsigned long ftrace_call_adjust(unsigned long addr)
> +{
> + return addr;
> +}
> +
> +struct dyn_arch_ftrace {
> +};
> +
> +struct dyn_ftrace;
> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
> +#define ftrace_init_nop ftrace_init_nop
> +
> +#endif /* CONFIG_DYNAMIC_FTRACE */
> #endif /* __ASSEMBLY__ */
> #endif /* CONFIG_FUNCTION_TRACER */
> #endif /* _ASM_LOONGARCH_FTRACE_H */
> diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
> index a52913787183..0ec775d39ca5 100644
> --- a/arch/loongarch/include/asm/inst.h
> +++ b/arch/loongarch/include/asm/inst.h
> @@ -11,6 +11,9 @@
> #define INSN_NOP 0x03400000
> #define INSN_BREAK 0x002a0000
>
> +#define INSN_NOP 0x03400000
> +#define INSN_BREAK 0x002a0000
They are already defined.
Huacai
> +
> #define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
> #define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
> #define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
> @@ -349,6 +352,18 @@ static inline bool is_stack_alloc_ins(union loongarch_instruction *ip)
> is_imm12_negative(ip->reg2i12_format.immediate);
> }
>
> +int larch_insn_read(void *addr, u32 *insnp);
> +int larch_insn_write(void *addr, u32 insn);
> +int larch_insn_patch_text(void *addr, u32 insn);
> +
> +u32 larch_insn_gen_nop(void);
> +u32 larch_insn_gen_b(unsigned long pc, unsigned long dest);
> +u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest);
> +
> +u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj,
> + enum loongarch_gpr rk);
> +u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
> +
> u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
> u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
> u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
> diff --git a/arch/loongarch/include/asm/unwind.h b/arch/loongarch/include/asm/unwind.h
> index 6af4718bdf01..a51eec00efb8 100644
> --- a/arch/loongarch/include/asm/unwind.h
> +++ b/arch/loongarch/include/asm/unwind.h
> @@ -20,7 +20,7 @@ struct unwind_state {
> char type; /* UNWINDER_XXX */
> struct stack_info stack_info;
> struct task_struct *task;
> - bool first, error;
> + bool first, error, is_ftrace;
> unsigned long sp, pc, ra;
> };
>
> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
> index 3f71bce1c7ce..c5e2bfd8247d 100644
> --- a/arch/loongarch/kernel/Makefile
> +++ b/arch/loongarch/kernel/Makefile
> @@ -16,8 +16,13 @@ obj-$(CONFIG_EFI) += efi.o
> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
>
> ifdef CONFIG_FUNCTION_TRACER
> +ifndef CONFIG_DYNAMIC_FTRACE
> obj-y += mcount.o ftrace.o
> CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
> +else
> +obj-y += mcount-dyn.o ftrace_dyn.o
> +CFLAGS_REMOVE_ftrace_dyn.o = $(CC_FLAGS_FTRACE)
> +endif
> CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
> CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
> CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
> new file mode 100644
> index 000000000000..1f8955be8b64
> --- /dev/null
> +++ b/arch/loongarch/kernel/ftrace_dyn.c
> @@ -0,0 +1,111 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Based on arch/arm64/kernel/ftrace.c
> + *
> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <linux/ftrace.h>
> +#include <linux/uaccess.h>
> +
> +#include <asm/inst.h>
> +
> +static int ftrace_modify_code(unsigned long pc, u32 old, u32 new,
> + bool validate)
> +{
> + u32 replaced;
> +
> + if (validate) {
> + if (larch_insn_read((void *)pc, &replaced))
> + return -EFAULT;
> +
> + if (replaced != old)
> + return -EINVAL;
> + }
> +
> + if (larch_insn_patch_text((void *)pc, new))
> + return -EPERM;
> +
> + return 0;
> +}
> +
> +int ftrace_update_ftrace_func(ftrace_func_t func)
> +{
> + unsigned long pc;
> + u32 new;
> +
> + pc = (unsigned long)&ftrace_call;
> + new = larch_insn_gen_bl(pc, (unsigned long)func);
> +
> + return ftrace_modify_code(pc, 0, new, false);
> +}
> +
> +/*
> + * The compiler has inserted 2 NOPs before the regular function prologue.
> + * T series registers are available and safe because of LoongArch psABI.
> + *
> + * At runtime, replace nop with bl to enable ftrace call and replace bl with
> + * nop to disable ftrace call. The bl requires us to save the original RA value,
> + * so here it saves RA at t0.
> + * details are:
> + *
> + * | Compiled | Disabled | Enabled |
> + * +------------+------------------------+------------------------+
> + * | nop | move t0, ra | move t0, ra |
> + * | nop | nop | bl ftrace_caller |
> + * | func_body | func_body | func_body |
> + *
> + * The RA value will be recovered by ftrace_regs_entry, and restored into RA
> + * before returning to the regular function prologue. When a function is not
> + * being traced, the move t0, ra is not harmful.
> + */
> +
> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
> +{
> + unsigned long pc;
> + u32 old, new;
> +
> + pc = rec->ip;
> + old = larch_insn_gen_nop();
> + new = larch_insn_gen_move(LOONGARCH_GPR_T0, LOONGARCH_GPR_RA);
> +
> + return ftrace_modify_code(pc, old, new, true);
> +}
> +
> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> + unsigned long pc;
> + u32 old, new;
> +
> + pc = rec->ip + LOONGARCH_INSN_SIZE;
> +
> + old = larch_insn_gen_nop();
> + new = larch_insn_gen_bl(pc, addr);
> +
> + return ftrace_modify_code(pc, old, new, true);
> +}
> +
> +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> + unsigned long addr)
> +{
> + unsigned long pc;
> + u32 old, new;
> +
> + pc = rec->ip + LOONGARCH_INSN_SIZE;
> +
> + new = larch_insn_gen_nop();
> + old = larch_insn_gen_bl(pc, addr);
> +
> + return ftrace_modify_code(pc, old, new, true);
> +}
> +
> +void arch_ftrace_update_code(int command)
> +{
> + command |= FTRACE_MAY_SLEEP;
> + ftrace_modify_all_code(command);
> +}
> +
> +int __init ftrace_dyn_arch_init(void)
> +{
> + return 0;
> +}
> diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
> index b1df0ec34bd1..d62cdf4a9ffb 100644
> --- a/arch/loongarch/kernel/inst.c
> +++ b/arch/loongarch/kernel/inst.c
> @@ -2,8 +2,83 @@
> /*
> * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
> */
> +#include <linux/sizes.h>
> +#include <linux/uaccess.h>
> +
> +#include <asm/cacheflush.h>
> #include <asm/inst.h>
>
> +static DEFINE_RAW_SPINLOCK(patch_lock);
> +
> +int larch_insn_read(void *addr, u32 *insnp)
> +{
> + int ret;
> + u32 val;
> +
> + ret = copy_from_kernel_nofault(&val, addr, LOONGARCH_INSN_SIZE);
> + if (!ret)
> + *insnp = val;
> +
> + return ret;
> +}
> +
> +int larch_insn_write(void *addr, u32 insn)
> +{
> + int ret;
> + unsigned long flags = 0;
> +
> + raw_spin_lock_irqsave(&patch_lock, flags);
> + ret = copy_to_kernel_nofault(addr, &insn, LOONGARCH_INSN_SIZE);
> + raw_spin_unlock_irqrestore(&patch_lock, flags);
> +
> + return ret;
> +}
> +
> +int larch_insn_patch_text(void *addr, u32 insn)
> +{
> + int ret;
> + u32 *tp = addr;
> +
> + if ((unsigned long)tp & 3)
> + return -EINVAL;
> +
> + ret = larch_insn_write(tp, insn);
> + if (!ret)
> + flush_icache_range((unsigned long)tp,
> + (unsigned long)tp + LOONGARCH_INSN_SIZE);
> +
> + return ret;
> +}
> +
> +u32 larch_insn_gen_nop(void)
> +{
> + return INSN_NOP;
> +}
> +
> +u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
> +{
> + unsigned int immediate_l, immediate_h;
> + union loongarch_instruction insn;
> + long offset = dest - pc;
> +
> + if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
> + pr_warn("The generated bl instruction is out of range.\n");
> + return INSN_BREAK;
> + }
> +
> + offset >>= 2;
> +
> + immediate_l = offset & 0xffff;
> + offset >>= 16;
> + immediate_h = offset & 0x3ff;
> +
> + insn.reg0i26_format.opcode = bl_op;
> + insn.reg0i26_format.immediate_l = immediate_l;
> + insn.reg0i26_format.immediate_h = immediate_h;
> +
> + return insn.word;
> +}
> +
> u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
> {
> union loongarch_instruction insn;
> @@ -38,3 +113,20 @@ u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned l
>
> return insn.word;
> }
> +
> +u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk)
> +{
> + union loongarch_instruction insn;
> +
> + insn.reg3_format.opcode = or_op;
> + insn.reg3_format.rd = rd;
> + insn.reg3_format.rj = rj;
> + insn.reg3_format.rk = rk;
> +
> + return insn.word;
> +}
> +
> +u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj)
> +{
> + return larch_insn_gen_or(rd, rj, 0);
> +}
> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
> new file mode 100644
> index 000000000000..205925bc3822
> --- /dev/null
> +++ b/arch/loongarch/kernel/mcount-dyn.S
> @@ -0,0 +1,89 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2022 Loongson Technology Corporation Limited
> + */
> +
> +#include <asm/export.h>
> +#include <asm/regdef.h>
> +#include <asm/stackframe.h>
> +#include <asm/ftrace.h>
> +
> + .text
> +/*
> + * Due to -fpatchable-function-entry=2: the compiler inserted 2 NOPs before the
> + * regular C function prologue. When PC arrived here, the last 2 instructions
> + * as follows,
> + * move t0, ra
> + * bl callsite (for modules, callsite is a tramplione)
> + *
> + * modules tramplione as follows,
> + * lu12i.w t1, callsite[31:12]
> + * lu32i.d t1, callsite[51:32]
> + * lu52i.d t1, t1, callsite[63:52]
> + * jirl zero, t1, callsite[11:0] >> 2
> + *
> + * See arch/loongarch/kernel/ftrace_dyn.c for details. Here, pay attention to
> + * that the T series regs are available and safe because each C functions
> + * follows the LoongArch psABI well.
> + */
> +
> + .macro ftrace_regs_entry
> + PTR_ADDI sp, sp, -PT_SIZE
> + /* Save trace function ra at PT_ERA */
> + PTR_S ra, sp, PT_ERA
> + /* Save parent ra at PT_R1(RA) */
> + PTR_S t0, sp, PT_R1
> + PTR_S a0, sp, PT_R4
> + PTR_S a1, sp, PT_R5
> + PTR_S a2, sp, PT_R6
> + PTR_S a3, sp, PT_R7
> + PTR_S a4, sp, PT_R8
> + PTR_S a5, sp, PT_R9
> + PTR_S a6, sp, PT_R10
> + PTR_S a7, sp, PT_R11
> + PTR_S fp, sp, PT_R22
> +
> + PTR_ADDI t8, sp, PT_SIZE
> + PTR_S t8, sp, PT_R3
> +
> + .endm
> +
> +SYM_CODE_START(ftrace_caller)
> + ftrace_regs_entry
> + b ftrace_common
> +SYM_CODE_END(ftrace_caller)
> +
> +SYM_CODE_START(ftrace_common)
> + PTR_ADDI a0, ra, -8 /* arg0: ip */
> + move a1, t0 /* arg1: parent_ip */
> + la.pcrel t1, function_trace_op
> + PTR_L a2, t1, 0 /* arg2: op */
> + move a3, sp /* arg3: regs */
> + .globl ftrace_call
> +ftrace_call:
> + bl ftrace_stub
> +/*
> + * As we didn't use S series regs in this assmembly code and all calls
> + * are C function which will save S series regs by themselves, there is
> + * no need to restore S series regs. The T series is available and safe
> + * at the callsite, so there is no need to restore the T series regs.
> + */
> +ftrace_common_return:
> + PTR_L a0, sp, PT_R4
> + PTR_L a1, sp, PT_R5
> + PTR_L a2, sp, PT_R6
> + PTR_L a3, sp, PT_R7
> + PTR_L a4, sp, PT_R8
> + PTR_L a5, sp, PT_R9
> + PTR_L a6, sp, PT_R10
> + PTR_L a7, sp, PT_R11
> + PTR_L fp, sp, PT_R22
> + PTR_L ra, sp, PT_R1
> + PTR_L t0, sp, PT_ERA
> + PTR_ADDI sp, sp, PT_SIZE
> + jr t0
> +SYM_CODE_END(ftrace_common)
> +
> +SYM_FUNC_START(ftrace_stub)
> + jr ra
> +SYM_FUNC_END(ftrace_stub)
> diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c
> index b206d9159205..c5df4ae73e0d 100644
> --- a/arch/loongarch/kernel/unwind_prologue.c
> +++ b/arch/loongarch/kernel/unwind_prologue.c
> @@ -13,9 +13,7 @@ unsigned long unwind_get_return_address(struct unwind_state *state)
>
> if (unwind_done(state))
> return 0;
> - else if (state->type)
> - return state->pc;
> - else if (state->first)
> + else if (state->type || state->first)
> return state->pc;
>
> return *(unsigned long *)(state->sp);
> @@ -39,16 +37,41 @@ static bool unwind_by_guess(struct unwind_state *state)
> return false;
> }
>
> +static inline void unwind_state_fixup(struct unwind_state *state)
> +{
> +#ifdef CONFIG_DYNAMIC_FTRACE
> + static unsigned long ftrace_case = (unsigned long)ftrace_call + 4;
> +
> + if (state->pc == ftrace_case)
> + state->is_ftrace = true;
> +#endif
> +}
> +
> static bool unwind_by_prologue(struct unwind_state *state)
> {
> struct stack_info *info = &state->stack_info;
> union loongarch_instruction *ip, *ip_end;
> unsigned long frame_size = 0, frame_ra = -1;
> unsigned long size, offset, pc = state->pc;
> + struct pt_regs *regs;
>
> if (state->sp >= info->end || state->sp < info->begin)
> return false;
>
> + if (state->is_ftrace) {
> + /*
> + * As we meet ftrace_regs_entry, reset first flag like first doing
> + * tracing, Prologue analysis will stop soon because PC is at entry.
> + */
> + regs = (struct pt_regs *)state->sp;
> + state->pc = regs->csr_era;
> + state->ra = regs->regs[1];
> + state->sp = regs->regs[3];
> + state->first = true;
> + state->is_ftrace = false;
> + return true;
> + }
> +
> if (!kallsyms_lookup_size_offset(pc, &size, &offset))
> return false;
>
> @@ -94,7 +117,7 @@ static bool unwind_by_prologue(struct unwind_state *state)
>
> state->pc = *(unsigned long *)(state->sp + frame_ra);
> state->sp = state->sp + frame_size;
> - return !!__kernel_text_address(state->pc);
> + goto out;
>
> first:
> state->first = false;
> @@ -103,7 +126,9 @@ static bool unwind_by_prologue(struct unwind_state *state)
>
> state->pc = state->ra;
>
> - return !!__kernel_text_address(state->ra);
> +out:
> + unwind_state_fixup(state);
> + return !!__kernel_text_address(state->pc);
> }
>
> void unwind_start(struct unwind_state *state, struct task_struct *task,
> --
> 2.36.0
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support
2022-11-15 10:44 ` Qing Zhang
@ 2022-11-15 10:55 ` Huacai Chen
0 siblings, 0 replies; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 10:55 UTC (permalink / raw)
To: Qing Zhang
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
On Tue, Nov 15, 2022 at 6:44 PM Qing Zhang <zhangqing@loongson.cn> wrote:
>
>
> Hi, huacai
> On 2022/11/15 下午5:24, Huacai Chen wrote:
> > On Tue, Nov 15, 2022 at 5:22 PM Qing Zhang <zhangqing@loongson.cn> wrote:
> >>
> >> Hi, Huacai
> >>
> >> On 2022/11/15 下午4:41, Huacai Chen wrote:
> >>> Hi, Qing,
> >>>
> >>> Patch5 and Patch6 are small and related, maybe they can be combined
> >>> with a name "LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_{ARGS,REGS}
> >>> support"
> >> I think they should be independent due to different use scenarios.
> > From the commit message it seems DYNAMIC_FTRACE_WITH_ARGS depend on
> > DYNAMIC_FTRACE_WITH_REGS.
>
> DYNAMIC_FTRACE_WITH_REGS : which allows trace function saved registers,
> It means that these registers can be modified, eg: Kprobe、Livepatch.
>
> DYNAMIC_FTRACE_WITH_ARGS: uses pt_regs as a parameter for get sp,
> which means it can be used to access registers on the stack, and
> use graph_ops::func to install return_hooker to replace the special
> hook, eg: Livepatch.
>
> From the following Kconfig, they are reasonable as separate patches.
> config LIVEPATCH
> bool "Kernel Live Patching"
> depends on DYNAMIC_FTRACE_WITH_REGS || DYNAMIC_FTRACE_WITH_ARGS
>
OK, then keeping them separately is reasonable.
Huacai
> Thanks
> - Qing
> >
> > Huacai
> >>
> >> Thanks,
> >> -Qing
> >>>
> >>> Huacai
> >>>
> >>> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
> >>>>
> >>>> This patch implements DYNAMIC_FTRACE_WITH_REGS on LoongArch, which allows
> >>>> a traced function's arguments (and some other registers) to be captured
> >>>> into a struct pt_regs, allowing these to be inspected and modified.
> >>>>
> >>>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
> >>>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
> >>>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
> >>>> ---
> >>>> arch/loongarch/Kconfig | 1 +
> >>>> arch/loongarch/include/asm/ftrace.h | 3 +++
> >>>> arch/loongarch/kernel/ftrace_dyn.c | 17 ++++++++++++++
> >>>> arch/loongarch/kernel/mcount-dyn.S | 36 +++++++++++++++++++++++++++--
> >>>> 4 files changed, 55 insertions(+), 2 deletions(-)
> >>>>
> >>>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
> >>>> index 615ce62422b8..12e3e91a68ae 100644
> >>>> --- a/arch/loongarch/Kconfig
> >>>> +++ b/arch/loongarch/Kconfig
> >>>> @@ -90,6 +90,7 @@ config LOONGARCH
> >>>> select HAVE_DEBUG_STACKOVERFLOW
> >>>> select HAVE_DMA_CONTIGUOUS
> >>>> select HAVE_DYNAMIC_FTRACE
> >>>> + select HAVE_DYNAMIC_FTRACE_WITH_REGS
> >>>> select HAVE_EBPF_JIT
> >>>> select HAVE_EXIT_THREAD
> >>>> select HAVE_FAST_GUP
> >>>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
> >>>> index 76ca58767f4d..a3f974a7a5ce 100644
> >>>> --- a/arch/loongarch/include/asm/ftrace.h
> >>>> +++ b/arch/loongarch/include/asm/ftrace.h
> >>>> @@ -28,6 +28,9 @@ struct dyn_ftrace;
> >>>> int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
> >>>> #define ftrace_init_nop ftrace_init_nop
> >>>>
> >>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >>>> +#define ARCH_SUPPORTS_FTRACE_OPS 1
> >>>> +#endif
> >>>> #endif /* CONFIG_DYNAMIC_FTRACE */
> >>>> #endif /* __ASSEMBLY__ */
> >>>> #endif /* CONFIG_FUNCTION_TRACER */
> >>>> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
> >>>> index 3fe791b6783e..ec3d951be50c 100644
> >>>> --- a/arch/loongarch/kernel/ftrace_dyn.c
> >>>> +++ b/arch/loongarch/kernel/ftrace_dyn.c
> >>>> @@ -99,6 +99,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> >>>> return ftrace_modify_code(pc, old, new, true);
> >>>> }
> >>>>
> >>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >>>> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> >>>> + unsigned long addr)
> >>>> +{
> >>>> + unsigned long pc;
> >>>> + long offset;
> >>>> + u32 old, new;
> >>>> +
> >>>> + pc = rec->ip + LOONGARCH_INSN_SIZE;
> >>>> +
> >>>> + old = larch_insn_gen_bl(pc, old_addr);
> >>>> + new = larch_insn_gen_bl(pc, addr);
> >>>> +
> >>>> + return ftrace_modify_code(pc, old, new, true);
> >>>> +}
> >>>> +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
> >>>> +
> >>>> void arch_ftrace_update_code(int command)
> >>>> {
> >>>> command |= FTRACE_MAY_SLEEP;
> >>>> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
> >>>> index 0c12cc108e6f..02835186b463 100644
> >>>> --- a/arch/loongarch/kernel/mcount-dyn.S
> >>>> +++ b/arch/loongarch/kernel/mcount-dyn.S
> >>>> @@ -27,7 +27,7 @@
> >>>> * follows the LoongArch psABI well.
> >>>> */
> >>>>
> >>>> - .macro ftrace_regs_entry
> >>>> + .macro ftrace_regs_entry allregs=0
> >>>> PTR_ADDI sp, sp, -PT_SIZE
> >>>> /* Save trace function ra at PT_ERA */
> >>>> PTR_S ra, sp, PT_ERA
> >>>> @@ -43,16 +43,48 @@
> >>>> PTR_S a7, sp, PT_R11
> >>>> PTR_S fp, sp, PT_R22
> >>>>
> >>>> + .if \allregs
> >>>> + PTR_S t0, sp, PT_R12
> >>>> + PTR_S t1, sp, PT_R13
> >>>> + PTR_S t2, sp, PT_R14
> >>>> + PTR_S t3, sp, PT_R15
> >>>> + PTR_S t4, sp, PT_R16
> >>>> + PTR_S t5, sp, PT_R17
> >>>> + PTR_S t6, sp, PT_R18
> >>>> + PTR_S t7, sp, PT_R19
> >>>> + PTR_S t8, sp, PT_R20
> >>>> + PTR_S s0, sp, PT_R23
> >>>> + PTR_S s1, sp, PT_R24
> >>>> + PTR_S s2, sp, PT_R25
> >>>> + PTR_S s3, sp, PT_R26
> >>>> + PTR_S s4, sp, PT_R27
> >>>> + PTR_S s5, sp, PT_R28
> >>>> + PTR_S s6, sp, PT_R29
> >>>> + PTR_S s7, sp, PT_R30
> >>>> + PTR_S s8, sp, PT_R31
> >>>> + PTR_S tp, sp, PT_R2
> >>>> + /* Clear it for later use as a flag sometimes. */
> >>>> + PTR_S zero, sp, PT_R0
> >>>> + PTR_S $r21, sp, PT_R21
> >>>> + .endif
> >>>> +
> >>>> PTR_ADDI t8, sp, PT_SIZE
> >>>> PTR_S t8, sp, PT_R3
> >>>>
> >>>> .endm
> >>>>
> >>>> SYM_CODE_START(ftrace_caller)
> >>>> - ftrace_regs_entry
> >>>> + ftrace_regs_entry allregs=0
> >>>> b ftrace_common
> >>>> SYM_CODE_END(ftrace_caller)
> >>>>
> >>>> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> >>>> +SYM_CODE_START(ftrace_regs_caller)
> >>>> + ftrace_regs_entry allregs=1
> >>>> + b ftrace_common
> >>>> +SYM_CODE_END(ftrace_regs_caller)
> >>>> +#endif
> >>>> +
> >>>> SYM_CODE_START(ftrace_common)
> >>>> PTR_ADDI a0, ra, -8 /* arg0: ip */
> >>>> move a1, t0 /* arg1: parent_ip */
> >>>> --
> >>>> 2.36.0
> >>>>
> >>>>
> >>
> >>
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS
2022-11-15 8:47 ` Arnd Bergmann
@ 2022-11-15 10:56 ` Huacai Chen
0 siblings, 0 replies; 24+ messages in thread
From: Huacai Chen @ 2022-11-15 10:56 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Qing Zhang, Steven Rostedt, Ingo Molnar, loongarch, linux-kernel
On Tue, Nov 15, 2022 at 4:48 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Tue, Nov 15, 2022, at 09:31, Huacai Chen wrote:
> >
> > I suggest not enabling debug mechanisms in the default config file,
> > distribution configs can make their own decisions.
>
> CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS are rather fundamental, and
> are enabled in most other architectures as well. I would recommend
> everything that is needed to run on the most common hardware
> configurations and that is needed for booting into a general-purpose
> distribution, but not much beyond that.
>
> Other options like CONFIG_DEBUG_INFO and CONFIG_DEBUG_KERNEL (including
> options based on it) are probably better left disabled though, those
> are the ones with a significant runtime or buld-time overhead.
OK, got it.
Huacai
>
> Arnd
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 1/9] LoongArch/ftrace: Add basic support
2022-11-15 9:04 ` Huacai Chen
@ 2022-11-15 11:06 ` Qing Zhang
0 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 11:06 UTC (permalink / raw)
To: Huacai Chen
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, huacai
On 2022/11/15 下午5:04, Huacai Chen wrote:
> On Tue, Nov 15, 2022 at 4:51 PM Qing Zhang <zhangqing@loongson.cn> wrote:
>>
>> Hi, Huacai
>>
>> On 2022/11/15 下午4:25, Huacai Chen wrote:
>>> Hi, Qing,
>>>
>>> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>>>>
>>>> This patch contains basic ftrace support for LoongArch.
>>>> Specifically, function tracer (HAVE_FUNCTION_TRACER), function graph
>>>> tracer (HAVE_FUNCTION_GRAPH_TRACER) are implemented following the
>>>> instructions in Documentation/trace/ftrace-design.txt.
>>>>
>>>> Use `-pg` makes stub like a child function `void _mcount(void *ra)`.
>>>> Thus, it can be seen store RA and open stack before `call _mcount`.
>>>> Find `open stack` at first, and then find `store RA`
>>>>
>>>> Note that the functions in both inst.c and time.c should not be
>>>> hooked with the compiler's -pg option: to prevent infinite self-
>>>> referencing for the former, and to ignore early setup stuff for the
>>>> latter.
>>>>
>>>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
>>>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>>>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
>>>> ---
>>>> arch/loongarch/Kconfig | 2 +
>>>> arch/loongarch/Makefile | 5 ++
>>>> arch/loongarch/include/asm/ftrace.h | 18 ++++++
>>>> arch/loongarch/kernel/Makefile | 8 +++
>>>> arch/loongarch/kernel/ftrace.c | 74 +++++++++++++++++++++++
>>>> arch/loongarch/kernel/mcount.S | 94 +++++++++++++++++++++++++++++
>>>> 6 files changed, 201 insertions(+)
>>>> create mode 100644 arch/loongarch/include/asm/ftrace.h
>>>> create mode 100644 arch/loongarch/kernel/ftrace.c
>>>> create mode 100644 arch/loongarch/kernel/mcount.S
>>>>
>>>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
>>>> index 1943f840e494..92c4ec8c8527 100644
>>>> --- a/arch/loongarch/Kconfig
>>>> +++ b/arch/loongarch/Kconfig
>>>> @@ -91,6 +91,8 @@ config LOONGARCH
>>>> select HAVE_EBPF_JIT
>>>> select HAVE_EXIT_THREAD
>>>> select HAVE_FAST_GUP
>>>> + select HAVE_FUNCTION_GRAPH_TRACER
>>> This line should be moved to another patch.
>>>
>>>> + select HAVE_FUNCTION_TRACER
>>>> select HAVE_GENERIC_VDSO
>>>> select HAVE_IOREMAP_PROT
>>>> select HAVE_IRQ_EXIT_ON_IRQ_STACK
>>>> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
>>>> index a0fc1f9980e3..6832a8f891fd 100644
>>>> --- a/arch/loongarch/Makefile
>>>> +++ b/arch/loongarch/Makefile
>>>> @@ -36,6 +36,11 @@ ifneq ($(SUBARCH),$(ARCH))
>>>> endif
>>>> endif
>>>>
>>>> +ifdef CONFIG_DYNAMIC_FTRACE
>>>> +KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
>>>> +CC_FLAGS_FTRACE := -fpatchable-function-entry=2
>>>> +endif
>>>> +
>>>> ifdef CONFIG_64BIT
>>>> ld-emul = $(64bit-emul)
>>>> cflags-y += -mabi=lp64s
>>>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
>>>> new file mode 100644
>>>> index 000000000000..6a3e76234618
>>>> --- /dev/null
>>>> +++ b/arch/loongarch/include/asm/ftrace.h
>>>> @@ -0,0 +1,18 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>> +/*
>>>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>>>> + */
>>>> +
>>>> +#ifndef _ASM_LOONGARCH_FTRACE_H
>>>> +#define _ASM_LOONGARCH_FTRACE_H
>>>> +
>>>> +#ifdef CONFIG_FUNCTION_TRACER
>>>> +#define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
>>>> +
>>>> +#ifndef __ASSEMBLY__
>>>> +extern void _mcount(void);
>>>> +#define mcount _mcount
>>>> +
>>>> +#endif /* __ASSEMBLY__ */
>>>> +#endif /* CONFIG_FUNCTION_TRACER */
>>>> +#endif /* _ASM_LOONGARCH_FTRACE_H */
>>>> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
>>>> index 86744531b100..3f71bce1c7ce 100644
>>>> --- a/arch/loongarch/kernel/Makefile
>>>> +++ b/arch/loongarch/kernel/Makefile
>>>> @@ -15,6 +15,14 @@ obj-$(CONFIG_EFI) += efi.o
>>>>
>>>> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
>>>>
>>>> +ifdef CONFIG_FUNCTION_TRACER
>>>> +obj-y += mcount.o ftrace.o
>>>> +CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
>>>> +CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
>>>> +CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
>>>> +CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
>>>> +endif
>>>> +
>>>> obj-$(CONFIG_MODULES) += module.o module-sections.o
>>>> obj-$(CONFIG_STACKTRACE) += stacktrace.o
>>>>
>>>> diff --git a/arch/loongarch/kernel/ftrace.c b/arch/loongarch/kernel/ftrace.c
>>>> new file mode 100644
>>>> index 000000000000..c8ddc5f11f32
>>>> --- /dev/null
>>>> +++ b/arch/loongarch/kernel/ftrace.c
>>>> @@ -0,0 +1,74 @@
>>>> +// SPDX-License-Identifier: GPL-2.0
>>>> +/*
>>>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>>>> + */
>>>> +
>>>> +#include <linux/uaccess.h>
>>>> +#include <linux/init.h>
>>>> +#include <linux/ftrace.h>
>>>> +#include <linux/syscalls.h>
>>>> +
>>>> +#include <asm/asm.h>
>>>> +#include <asm/asm-offsets.h>
>>>> +#include <asm/cacheflush.h>
>>>> +#include <asm/inst.h>
>>>> +#include <asm/loongarch.h>
>>>> +#include <asm/syscall.h>
>>>> +#include <asm/unistd.h>
>>>> +
>>>> +#include <asm-generic/sections.h>
>>>> +
>>>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>> This should also be moved to another patch.
>>
>> No, CONFIG_FUNCTION_GRAPH_TRACER is the basic feature, as the first
>> patch for non-dynamic ftrace, unless it is a separate patch, which is
>> obviously not necessary (Refer to the first submission of riscv and
>> arm64 here).
> But FUNCTION_GRAPH_TRACER is added until Patch-4.
Separate non dynamic FUNCTION_ GRAPH_ TRACER and dynamic
FUNCTION_GRAPH_TRACER
can be placed in a patch, However, since the dynamic can completely
replace the
non dynamic, my idea is to put the non dynamic content in the first
patch, which
is more consistent with our idea of gradual evolution at the beginning,
How do you think?
Thanks,
- Qing
>
> Huacai
>>
>> Thanks,
>> - Qing
>>>
>>>> +
>>>> +/*
>>>> + * As `call _mcount` follows LoongArch psABI, ra-saved operation and
>>>> + * stack operation can be found before this insn.
>>>> + */
>>>> +
>>>> +static int ftrace_get_parent_ra_addr(unsigned long insn_addr, int *ra_off)
>>>> +{
>>>> + union loongarch_instruction *insn;
>>>> + int limit = 32;
>>>> +
>>>> + insn = (union loongarch_instruction *)insn_addr;
>>>> +
>>>> + do {
>>>> + insn--;
>>>> + limit--;
>>>> +
>>>> + if (is_ra_save_ins(insn))
>>>> + *ra_off = -((1 << 12) - insn->reg2i12_format.immediate);
>>>> +
>>>> + } while (!is_stack_alloc_ins(insn) && limit);
>>>> +
>>>> + if (!limit)
>>>> + return -EINVAL;
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> +void prepare_ftrace_return(unsigned long self_addr,
>>>> + unsigned long callsite_sp, unsigned long old)
>>>> +{
>>>> + int ra_off;
>>>> + unsigned long return_hooker = (unsigned long)&return_to_handler;
>>>> +
>>>> + if (unlikely(ftrace_graph_is_dead()))
>>>> + return;
>>>> +
>>>> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
>>>> + return;
>>>> +
>>>> + if (ftrace_get_parent_ra_addr(self_addr, &ra_off))
>>>> + goto out;
>>>> +
>>>> + if (!function_graph_enter(old, self_addr, 0, NULL))
>>>> + *(unsigned long *)(callsite_sp + ra_off) = return_hooker;
>>>> +
>>>> + return;
>>>> +
>>>> +out:
>>>> + ftrace_graph_stop();
>>>> + WARN_ON(1);
>>>> +}
>>>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>>>> diff --git a/arch/loongarch/kernel/mcount.S b/arch/loongarch/kernel/mcount.S
>>>> new file mode 100644
>>>> index 000000000000..3de7c2d7fd12
>>>> --- /dev/null
>>>> +++ b/arch/loongarch/kernel/mcount.S
>>>> @@ -0,0 +1,94 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>> +/*
>>>> + * LoongArch specific _mcount support
>>>> + *
>>>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>>>> + */
>>>> +
>>>> +#include <asm/export.h>
>>>> +#include <asm/regdef.h>
>>>> +#include <asm/stackframe.h>
>>>> +#include <asm/ftrace.h>
>>>> +
>>>> + .text
>>>> +
>>>> +#define MCOUNT_STACK_SIZE (2 * SZREG)
>>>> +#define MCOUNT_S0_OFFSET (0)
>>>> +#define MCOUNT_RA_OFFSET (SZREG)
>>>> +
>>>> + .macro MCOUNT_SAVE_REGS
>>>> + PTR_ADDI sp, sp, -MCOUNT_STACK_SIZE
>>>> + PTR_S s0, sp, MCOUNT_S0_OFFSET
>>>> + PTR_S ra, sp, MCOUNT_RA_OFFSET
>>>> + move s0, a0
>>>> + .endm
>>>> +
>>>> + .macro MCOUNT_RESTORE_REGS
>>>> + move a0, s0
>>>> + PTR_L ra, sp, MCOUNT_RA_OFFSET
>>>> + PTR_L s0, sp, MCOUNT_S0_OFFSET
>>>> + PTR_ADDI sp, sp, MCOUNT_STACK_SIZE
>>>> + .endm
>>>> +
>>>> +
>>>> +SYM_FUNC_START(_mcount)
>>>> + la.pcrel t1, ftrace_stub
>>>> + la.pcrel t2, ftrace_trace_function /* Prepare t2 for (1) */
>>>> + PTR_L t2, t2, 0
>>>> + beq t1, t2, fgraph_trace
>>>> +
>>>> + MCOUNT_SAVE_REGS
>>>> +
>>>> + move a0, ra /* arg0: self return address */
>>>> + move a1, s0 /* arg1: parent's return address */
>>>> + jirl ra, t2, 0 /* (1) call *ftrace_trace_function */
>>>> +
>>>> + MCOUNT_RESTORE_REGS
>>>> +
>>>> +fgraph_trace:
>>>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>> The same as above.
>>>
>>> Huacai
>>>> + la.pcrel t1, ftrace_stub
>>>> + la.pcrel t3, ftrace_graph_return
>>>> + PTR_L t3, t3, 0
>>>> + bne t1, t3, ftrace_graph_caller
>>>> + la.pcrel t1, ftrace_graph_entry_stub
>>>> + la.pcrel t3, ftrace_graph_entry
>>>> + PTR_L t3, t3, 0
>>>> + bne t1, t3, ftrace_graph_caller
>>>> +#endif
>>>> +
>>>> + .globl ftrace_stub
>>>> +ftrace_stub:
>>>> + jr ra
>>>> +SYM_FUNC_END(_mcount)
>>>> +EXPORT_SYMBOL(_mcount)
>>>> +
>>>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>>>> +SYM_FUNC_START(ftrace_graph_caller)
>>>> + MCOUNT_SAVE_REGS
>>>> +
>>>> + PTR_ADDI a0, ra, -4 /* arg0: Callsite self return addr */
>>>> + PTR_ADDI a1, sp, MCOUNT_STACK_SIZE /* arg1: Callsite sp */
>>>> + move a2, s0 /* arg2: Callsite parent ra */
>>>> + bl prepare_ftrace_return
>>>> +
>>>> + MCOUNT_RESTORE_REGS
>>>> + jr ra
>>>> +SYM_FUNC_END(ftrace_graph_caller)
>>>> +
>>>> +SYM_FUNC_START(return_to_handler)
>>>> + PTR_ADDI sp, sp, -2 * SZREG
>>>> + PTR_S a0, sp, 0
>>>> + PTR_S a1, sp, SZREG
>>>> +
>>>> + bl ftrace_return_to_handler
>>>> +
>>>> + /* restore the real parent address: a0 -> ra */
>>>> + move ra, a0
>>>> +
>>>> + PTR_L a0, sp, 0
>>>> + PTR_L a1, sp, SZREG
>>>> + PTR_ADDI sp, sp, 2 * SZREG
>>>> + jr ra
>>>> +SYM_FUNC_END(return_to_handler)
>>>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>>>> --
>>>> 2.36.0
>>>>
>>>>
>>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support
2022-11-15 10:54 ` Huacai Chen
@ 2022-11-15 11:11 ` Qing Zhang
0 siblings, 0 replies; 24+ messages in thread
From: Qing Zhang @ 2022-11-15 11:11 UTC (permalink / raw)
To: Huacai Chen
Cc: Steven Rostedt, Ingo Molnar, loongarch, linux-kernel, Jinyang He
Hi, huacai
On 2022/11/15 下午6:54, Huacai Chen wrote:
> Hi, Qing,
>
> On Tue, Nov 15, 2022 at 10:55 AM Qing Zhang <zhangqing@loongson.cn> wrote:
>>
>> The compiler has inserted 2 NOPs before the regular function prologue.
>> T series registers are available and safe because of LoongArch psABI.
>>
>> At runtime, replace nop with bl to enable ftrace call and replace bl with
>> nop to disable ftrace call. The bl requires us to save the original RA
>> value, so here it saves RA at t0.
>> details are:
>>
>> | Compiled | Disabled | Enabled |
>> +------------+------------------------+------------------------+
>> | nop | move t0, ra | move t0, ra |
>> | nop | nop | bl ftrace_caller |
>> | func_body | func_body | func_body |
>>
>> The RA value will be recovered by ftrace_regs_entry, and restored into RA
>> before returning to the regular function prologue. When a function is not
>> being traced, the move t0, ra is not harmful.
>>
>> 1) ftrace_make_call, ftrace_make_nop (in kernel/ftrace.c)
>> The two functions turn each recorded call site of filtered functions
>> into a call to ftrace_caller or nops.
>>
>> 2) ftracce_update_ftrace_func (in kernel/ftrace.c)
>> turns the nops at ftrace_call into a call to a generic entry for
>> function tracers.
>>
>> 3) ftrace_caller (in kernel/mcount-dyn.S)
>> The entry where each _mcount call sites calls to once they are
>> filtered to be traced.
>>
>> Co-developed-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>> Signed-off-by: Qing Zhang <zhangqing@loongson.cn>
>> ---
>> arch/loongarch/Kconfig | 1 +
>> arch/loongarch/include/asm/ftrace.h | 16 ++++
>> arch/loongarch/include/asm/inst.h | 15 ++++
>> arch/loongarch/include/asm/unwind.h | 2 +-
>> arch/loongarch/kernel/Makefile | 5 ++
>> arch/loongarch/kernel/ftrace_dyn.c | 111 ++++++++++++++++++++++++
>> arch/loongarch/kernel/inst.c | 92 ++++++++++++++++++++
>> arch/loongarch/kernel/mcount-dyn.S | 89 +++++++++++++++++++
>> arch/loongarch/kernel/unwind_prologue.c | 35 ++++++--
>> 9 files changed, 360 insertions(+), 6 deletions(-)
>> create mode 100644 arch/loongarch/kernel/ftrace_dyn.c
>> create mode 100644 arch/loongarch/kernel/mcount-dyn.S
>>
>> diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
>> index e6b1defca1f1..615ce62422b8 100644
>> --- a/arch/loongarch/Kconfig
>> +++ b/arch/loongarch/Kconfig
>> @@ -89,6 +89,7 @@ config LOONGARCH
>> select HAVE_C_RECORDMCOUNT
>> select HAVE_DEBUG_STACKOVERFLOW
>> select HAVE_DMA_CONTIGUOUS
>> + select HAVE_DYNAMIC_FTRACE
>> select HAVE_EBPF_JIT
>> select HAVE_EXIT_THREAD
>> select HAVE_FAST_GUP
>> diff --git a/arch/loongarch/include/asm/ftrace.h b/arch/loongarch/include/asm/ftrace.h
>> index 6a3e76234618..76ca58767f4d 100644
>> --- a/arch/loongarch/include/asm/ftrace.h
>> +++ b/arch/loongarch/include/asm/ftrace.h
>> @@ -10,9 +10,25 @@
>> #define MCOUNT_INSN_SIZE 4 /* sizeof mcount call */
>>
>> #ifndef __ASSEMBLY__
>> +#ifndef CONFIG_DYNAMIC_FTRACE
>> extern void _mcount(void);
>> #define mcount _mcount
>> +#endif
>>
>> +#ifdef CONFIG_DYNAMIC_FTRACE
>> +static inline unsigned long ftrace_call_adjust(unsigned long addr)
>> +{
>> + return addr;
>> +}
>> +
>> +struct dyn_arch_ftrace {
>> +};
>> +
>> +struct dyn_ftrace;
>> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
>> +#define ftrace_init_nop ftrace_init_nop
>> +
>> +#endif /* CONFIG_DYNAMIC_FTRACE */
>> #endif /* __ASSEMBLY__ */
>> #endif /* CONFIG_FUNCTION_TRACER */
>> #endif /* _ASM_LOONGARCH_FTRACE_H */
>> diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm/inst.h
>> index a52913787183..0ec775d39ca5 100644
>> --- a/arch/loongarch/include/asm/inst.h
>> +++ b/arch/loongarch/include/asm/inst.h
>> @@ -11,6 +11,9 @@
>> #define INSN_NOP 0x03400000
>> #define INSN_BREAK 0x002a0000
>>
>> +#define INSN_NOP 0x03400000
>> +#define INSN_BREAK 0x002a0000
> They are already defined.
Sorry :( , I will resend the next version.
Thanks
-Qing
>
> Huacai
>> +
>> #define ADDR_IMMMASK_LU52ID 0xFFF0000000000000
>> #define ADDR_IMMMASK_LU32ID 0x000FFFFF00000000
>> #define ADDR_IMMMASK_ADDU16ID 0x00000000FFFF0000
>> @@ -349,6 +352,18 @@ static inline bool is_stack_alloc_ins(union loongarch_instruction *ip)
>> is_imm12_negative(ip->reg2i12_format.immediate);
>> }
>>
>> +int larch_insn_read(void *addr, u32 *insnp);
>> +int larch_insn_write(void *addr, u32 insn);
>> +int larch_insn_patch_text(void *addr, u32 insn);
>> +
>> +u32 larch_insn_gen_nop(void);
>> +u32 larch_insn_gen_b(unsigned long pc, unsigned long dest);
>> +u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest);
>> +
>> +u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj,
>> + enum loongarch_gpr rk);
>> +u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj);
>> +
>> u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm);
>> u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, int imm);
>> u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned long pc, unsigned long dest);
>> diff --git a/arch/loongarch/include/asm/unwind.h b/arch/loongarch/include/asm/unwind.h
>> index 6af4718bdf01..a51eec00efb8 100644
>> --- a/arch/loongarch/include/asm/unwind.h
>> +++ b/arch/loongarch/include/asm/unwind.h
>> @@ -20,7 +20,7 @@ struct unwind_state {
>> char type; /* UNWINDER_XXX */
>> struct stack_info stack_info;
>> struct task_struct *task;
>> - bool first, error;
>> + bool first, error, is_ftrace;
>> unsigned long sp, pc, ra;
>> };
>>
>> diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile
>> index 3f71bce1c7ce..c5e2bfd8247d 100644
>> --- a/arch/loongarch/kernel/Makefile
>> +++ b/arch/loongarch/kernel/Makefile
>> @@ -16,8 +16,13 @@ obj-$(CONFIG_EFI) += efi.o
>> obj-$(CONFIG_CPU_HAS_FPU) += fpu.o
>>
>> ifdef CONFIG_FUNCTION_TRACER
>> +ifndef CONFIG_DYNAMIC_FTRACE
>> obj-y += mcount.o ftrace.o
>> CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
>> +else
>> +obj-y += mcount-dyn.o ftrace_dyn.o
>> +CFLAGS_REMOVE_ftrace_dyn.o = $(CC_FLAGS_FTRACE)
>> +endif
>> CFLAGS_REMOVE_inst.o = $(CC_FLAGS_FTRACE)
>> CFLAGS_REMOVE_time.o = $(CC_FLAGS_FTRACE)
>> CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE)
>> diff --git a/arch/loongarch/kernel/ftrace_dyn.c b/arch/loongarch/kernel/ftrace_dyn.c
>> new file mode 100644
>> index 000000000000..1f8955be8b64
>> --- /dev/null
>> +++ b/arch/loongarch/kernel/ftrace_dyn.c
>> @@ -0,0 +1,111 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Based on arch/arm64/kernel/ftrace.c
>> + *
>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>> + */
>> +
>> +#include <linux/ftrace.h>
>> +#include <linux/uaccess.h>
>> +
>> +#include <asm/inst.h>
>> +
>> +static int ftrace_modify_code(unsigned long pc, u32 old, u32 new,
>> + bool validate)
>> +{
>> + u32 replaced;
>> +
>> + if (validate) {
>> + if (larch_insn_read((void *)pc, &replaced))
>> + return -EFAULT;
>> +
>> + if (replaced != old)
>> + return -EINVAL;
>> + }
>> +
>> + if (larch_insn_patch_text((void *)pc, new))
>> + return -EPERM;
>> +
>> + return 0;
>> +}
>> +
>> +int ftrace_update_ftrace_func(ftrace_func_t func)
>> +{
>> + unsigned long pc;
>> + u32 new;
>> +
>> + pc = (unsigned long)&ftrace_call;
>> + new = larch_insn_gen_bl(pc, (unsigned long)func);
>> +
>> + return ftrace_modify_code(pc, 0, new, false);
>> +}
>> +
>> +/*
>> + * The compiler has inserted 2 NOPs before the regular function prologue.
>> + * T series registers are available and safe because of LoongArch psABI.
>> + *
>> + * At runtime, replace nop with bl to enable ftrace call and replace bl with
>> + * nop to disable ftrace call. The bl requires us to save the original RA value,
>> + * so here it saves RA at t0.
>> + * details are:
>> + *
>> + * | Compiled | Disabled | Enabled |
>> + * +------------+------------------------+------------------------+
>> + * | nop | move t0, ra | move t0, ra |
>> + * | nop | nop | bl ftrace_caller |
>> + * | func_body | func_body | func_body |
>> + *
>> + * The RA value will be recovered by ftrace_regs_entry, and restored into RA
>> + * before returning to the regular function prologue. When a function is not
>> + * being traced, the move t0, ra is not harmful.
>> + */
>> +
>> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
>> +{
>> + unsigned long pc;
>> + u32 old, new;
>> +
>> + pc = rec->ip;
>> + old = larch_insn_gen_nop();
>> + new = larch_insn_gen_move(LOONGARCH_GPR_T0, LOONGARCH_GPR_RA);
>> +
>> + return ftrace_modify_code(pc, old, new, true);
>> +}
>> +
>> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
>> +{
>> + unsigned long pc;
>> + u32 old, new;
>> +
>> + pc = rec->ip + LOONGARCH_INSN_SIZE;
>> +
>> + old = larch_insn_gen_nop();
>> + new = larch_insn_gen_bl(pc, addr);
>> +
>> + return ftrace_modify_code(pc, old, new, true);
>> +}
>> +
>> +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
>> + unsigned long addr)
>> +{
>> + unsigned long pc;
>> + u32 old, new;
>> +
>> + pc = rec->ip + LOONGARCH_INSN_SIZE;
>> +
>> + new = larch_insn_gen_nop();
>> + old = larch_insn_gen_bl(pc, addr);
>> +
>> + return ftrace_modify_code(pc, old, new, true);
>> +}
>> +
>> +void arch_ftrace_update_code(int command)
>> +{
>> + command |= FTRACE_MAY_SLEEP;
>> + ftrace_modify_all_code(command);
>> +}
>> +
>> +int __init ftrace_dyn_arch_init(void)
>> +{
>> + return 0;
>> +}
>> diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c
>> index b1df0ec34bd1..d62cdf4a9ffb 100644
>> --- a/arch/loongarch/kernel/inst.c
>> +++ b/arch/loongarch/kernel/inst.c
>> @@ -2,8 +2,83 @@
>> /*
>> * Copyright (C) 2020-2022 Loongson Technology Corporation Limited
>> */
>> +#include <linux/sizes.h>
>> +#include <linux/uaccess.h>
>> +
>> +#include <asm/cacheflush.h>
>> #include <asm/inst.h>
>>
>> +static DEFINE_RAW_SPINLOCK(patch_lock);
>> +
>> +int larch_insn_read(void *addr, u32 *insnp)
>> +{
>> + int ret;
>> + u32 val;
>> +
>> + ret = copy_from_kernel_nofault(&val, addr, LOONGARCH_INSN_SIZE);
>> + if (!ret)
>> + *insnp = val;
>> +
>> + return ret;
>> +}
>> +
>> +int larch_insn_write(void *addr, u32 insn)
>> +{
>> + int ret;
>> + unsigned long flags = 0;
>> +
>> + raw_spin_lock_irqsave(&patch_lock, flags);
>> + ret = copy_to_kernel_nofault(addr, &insn, LOONGARCH_INSN_SIZE);
>> + raw_spin_unlock_irqrestore(&patch_lock, flags);
>> +
>> + return ret;
>> +}
>> +
>> +int larch_insn_patch_text(void *addr, u32 insn)
>> +{
>> + int ret;
>> + u32 *tp = addr;
>> +
>> + if ((unsigned long)tp & 3)
>> + return -EINVAL;
>> +
>> + ret = larch_insn_write(tp, insn);
>> + if (!ret)
>> + flush_icache_range((unsigned long)tp,
>> + (unsigned long)tp + LOONGARCH_INSN_SIZE);
>> +
>> + return ret;
>> +}
>> +
>> +u32 larch_insn_gen_nop(void)
>> +{
>> + return INSN_NOP;
>> +}
>> +
>> +u32 larch_insn_gen_bl(unsigned long pc, unsigned long dest)
>> +{
>> + unsigned int immediate_l, immediate_h;
>> + union loongarch_instruction insn;
>> + long offset = dest - pc;
>> +
>> + if ((offset & 3) || offset < -SZ_128M || offset >= SZ_128M) {
>> + pr_warn("The generated bl instruction is out of range.\n");
>> + return INSN_BREAK;
>> + }
>> +
>> + offset >>= 2;
>> +
>> + immediate_l = offset & 0xffff;
>> + offset >>= 16;
>> + immediate_h = offset & 0x3ff;
>> +
>> + insn.reg0i26_format.opcode = bl_op;
>> + insn.reg0i26_format.immediate_l = immediate_l;
>> + insn.reg0i26_format.immediate_h = immediate_h;
>> +
>> + return insn.word;
>> +}
>> +
>> u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm)
>> {
>> union loongarch_instruction insn;
>> @@ -38,3 +113,20 @@ u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, unsigned l
>>
>> return insn.word;
>> }
>> +
>> +u32 larch_insn_gen_or(enum loongarch_gpr rd, enum loongarch_gpr rj, enum loongarch_gpr rk)
>> +{
>> + union loongarch_instruction insn;
>> +
>> + insn.reg3_format.opcode = or_op;
>> + insn.reg3_format.rd = rd;
>> + insn.reg3_format.rj = rj;
>> + insn.reg3_format.rk = rk;
>> +
>> + return insn.word;
>> +}
>> +
>> +u32 larch_insn_gen_move(enum loongarch_gpr rd, enum loongarch_gpr rj)
>> +{
>> + return larch_insn_gen_or(rd, rj, 0);
>> +}
>> diff --git a/arch/loongarch/kernel/mcount-dyn.S b/arch/loongarch/kernel/mcount-dyn.S
>> new file mode 100644
>> index 000000000000..205925bc3822
>> --- /dev/null
>> +++ b/arch/loongarch/kernel/mcount-dyn.S
>> @@ -0,0 +1,89 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * Copyright (C) 2022 Loongson Technology Corporation Limited
>> + */
>> +
>> +#include <asm/export.h>
>> +#include <asm/regdef.h>
>> +#include <asm/stackframe.h>
>> +#include <asm/ftrace.h>
>> +
>> + .text
>> +/*
>> + * Due to -fpatchable-function-entry=2: the compiler inserted 2 NOPs before the
>> + * regular C function prologue. When PC arrived here, the last 2 instructions
>> + * as follows,
>> + * move t0, ra
>> + * bl callsite (for modules, callsite is a tramplione)
>> + *
>> + * modules tramplione as follows,
>> + * lu12i.w t1, callsite[31:12]
>> + * lu32i.d t1, callsite[51:32]
>> + * lu52i.d t1, t1, callsite[63:52]
>> + * jirl zero, t1, callsite[11:0] >> 2
>> + *
>> + * See arch/loongarch/kernel/ftrace_dyn.c for details. Here, pay attention to
>> + * that the T series regs are available and safe because each C functions
>> + * follows the LoongArch psABI well.
>> + */
>> +
>> + .macro ftrace_regs_entry
>> + PTR_ADDI sp, sp, -PT_SIZE
>> + /* Save trace function ra at PT_ERA */
>> + PTR_S ra, sp, PT_ERA
>> + /* Save parent ra at PT_R1(RA) */
>> + PTR_S t0, sp, PT_R1
>> + PTR_S a0, sp, PT_R4
>> + PTR_S a1, sp, PT_R5
>> + PTR_S a2, sp, PT_R6
>> + PTR_S a3, sp, PT_R7
>> + PTR_S a4, sp, PT_R8
>> + PTR_S a5, sp, PT_R9
>> + PTR_S a6, sp, PT_R10
>> + PTR_S a7, sp, PT_R11
>> + PTR_S fp, sp, PT_R22
>> +
>> + PTR_ADDI t8, sp, PT_SIZE
>> + PTR_S t8, sp, PT_R3
>> +
>> + .endm
>> +
>> +SYM_CODE_START(ftrace_caller)
>> + ftrace_regs_entry
>> + b ftrace_common
>> +SYM_CODE_END(ftrace_caller)
>> +
>> +SYM_CODE_START(ftrace_common)
>> + PTR_ADDI a0, ra, -8 /* arg0: ip */
>> + move a1, t0 /* arg1: parent_ip */
>> + la.pcrel t1, function_trace_op
>> + PTR_L a2, t1, 0 /* arg2: op */
>> + move a3, sp /* arg3: regs */
>> + .globl ftrace_call
>> +ftrace_call:
>> + bl ftrace_stub
>> +/*
>> + * As we didn't use S series regs in this assmembly code and all calls
>> + * are C function which will save S series regs by themselves, there is
>> + * no need to restore S series regs. The T series is available and safe
>> + * at the callsite, so there is no need to restore the T series regs.
>> + */
>> +ftrace_common_return:
>> + PTR_L a0, sp, PT_R4
>> + PTR_L a1, sp, PT_R5
>> + PTR_L a2, sp, PT_R6
>> + PTR_L a3, sp, PT_R7
>> + PTR_L a4, sp, PT_R8
>> + PTR_L a5, sp, PT_R9
>> + PTR_L a6, sp, PT_R10
>> + PTR_L a7, sp, PT_R11
>> + PTR_L fp, sp, PT_R22
>> + PTR_L ra, sp, PT_R1
>> + PTR_L t0, sp, PT_ERA
>> + PTR_ADDI sp, sp, PT_SIZE
>> + jr t0
>> +SYM_CODE_END(ftrace_common)
>> +
>> +SYM_FUNC_START(ftrace_stub)
>> + jr ra
>> +SYM_FUNC_END(ftrace_stub)
>> diff --git a/arch/loongarch/kernel/unwind_prologue.c b/arch/loongarch/kernel/unwind_prologue.c
>> index b206d9159205..c5df4ae73e0d 100644
>> --- a/arch/loongarch/kernel/unwind_prologue.c
>> +++ b/arch/loongarch/kernel/unwind_prologue.c
>> @@ -13,9 +13,7 @@ unsigned long unwind_get_return_address(struct unwind_state *state)
>>
>> if (unwind_done(state))
>> return 0;
>> - else if (state->type)
>> - return state->pc;
>> - else if (state->first)
>> + else if (state->type || state->first)
>> return state->pc;
>>
>> return *(unsigned long *)(state->sp);
>> @@ -39,16 +37,41 @@ static bool unwind_by_guess(struct unwind_state *state)
>> return false;
>> }
>>
>> +static inline void unwind_state_fixup(struct unwind_state *state)
>> +{
>> +#ifdef CONFIG_DYNAMIC_FTRACE
>> + static unsigned long ftrace_case = (unsigned long)ftrace_call + 4;
>> +
>> + if (state->pc == ftrace_case)
>> + state->is_ftrace = true;
>> +#endif
>> +}
>> +
>> static bool unwind_by_prologue(struct unwind_state *state)
>> {
>> struct stack_info *info = &state->stack_info;
>> union loongarch_instruction *ip, *ip_end;
>> unsigned long frame_size = 0, frame_ra = -1;
>> unsigned long size, offset, pc = state->pc;
>> + struct pt_regs *regs;
>>
>> if (state->sp >= info->end || state->sp < info->begin)
>> return false;
>>
>> + if (state->is_ftrace) {
>> + /*
>> + * As we meet ftrace_regs_entry, reset first flag like first doing
>> + * tracing, Prologue analysis will stop soon because PC is at entry.
>> + */
>> + regs = (struct pt_regs *)state->sp;
>> + state->pc = regs->csr_era;
>> + state->ra = regs->regs[1];
>> + state->sp = regs->regs[3];
>> + state->first = true;
>> + state->is_ftrace = false;
>> + return true;
>> + }
>> +
>> if (!kallsyms_lookup_size_offset(pc, &size, &offset))
>> return false;
>>
>> @@ -94,7 +117,7 @@ static bool unwind_by_prologue(struct unwind_state *state)
>>
>> state->pc = *(unsigned long *)(state->sp + frame_ra);
>> state->sp = state->sp + frame_size;
>> - return !!__kernel_text_address(state->pc);
>> + goto out;
>>
>> first:
>> state->first = false;
>> @@ -103,7 +126,9 @@ static bool unwind_by_prologue(struct unwind_state *state)
>>
>> state->pc = state->ra;
>>
>> - return !!__kernel_text_address(state->ra);
>> +out:
>> + unwind_state_fixup(state);
>> + return !!__kernel_text_address(state->pc);
>> }
>>
>> void unwind_start(struct unwind_state *state, struct task_struct *task,
>> --
>> 2.36.0
>>
>>
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2022-11-15 11:11 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-15 2:55 [PATCH v6 0/9] LoongArch: Add ftrace support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 1/9] LoongArch/ftrace: Add basic support Qing Zhang
2022-11-15 8:25 ` Huacai Chen
2022-11-15 8:50 ` Qing Zhang
2022-11-15 9:04 ` Huacai Chen
2022-11-15 11:06 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 2/9] LoongArch/ftrace: Add recordmcount support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 3/9] LoongArch/ftrace: Add dynamic function tracer support Qing Zhang
2022-11-15 10:54 ` Huacai Chen
2022-11-15 11:11 ` Qing Zhang
2022-11-15 2:55 ` [PATCH v6 4/9] LoongArch/ftrace: Add dynamic function graph " Qing Zhang
2022-11-15 2:55 ` [PATCH v6 5/9] LoongArch/ftrace: Add DYNAMIC_FTRACE_WITH_REGS support Qing Zhang
2022-11-15 8:41 ` Huacai Chen
2022-11-15 9:22 ` Qing Zhang
2022-11-15 9:24 ` Huacai Chen
2022-11-15 10:44 ` Qing Zhang
2022-11-15 10:55 ` Huacai Chen
2022-11-15 2:55 ` [PATCH v6 6/9] LoongArch/ftrace: Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 7/9] LoongArch/ftrace: Add HAVE_FUNCTION_GRAPH_RET_ADDR_PTR support Qing Zhang
2022-11-15 2:55 ` [PATCH v6 8/9] LoongArch: modules/ftrace: Initialize PLT at load time Qing Zhang
2022-11-15 2:55 ` [PATCH v6 9/9] LoongArch: Enable CONFIG_KALLSYMS_ALL and CONFIG_DEBUG_FS Qing Zhang
2022-11-15 8:31 ` Huacai Chen
2022-11-15 8:47 ` Arnd Bergmann
2022-11-15 10:56 ` Huacai Chen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.