linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH V10 0/4] riscv: Optimize function trace
@ 2023-05-11  9:32 Song Shuai
  2023-05-11  9:32 ` [PATCH V10 1/4] riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY Song Shuai
                   ` (4 more replies)
  0 siblings, 5 replies; 12+ messages in thread
From: Song Shuai @ 2023-05-11  9:32 UTC (permalink / raw)
  To: paul.walmsley, palmer, aou, rostedt, mhiramat, mark.rutland,
	guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

Changes in v10:

- add Acked-by from Björn Töpel in patch 2 and patch 4 
- replace `move` with `mv` in patch3 
- prettify patch 2/4 with proper tabs

You can directly try it with:

https://github.com/sugarfillet/linux/tree/6.4-rc1-rv-ftrace-v10


This series optimizes function trace. The first 3 independent 
patches has been picked in the V7 version of this series, the
subsequent version continues the following 4 patches:

select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1] (patch 1)
==========================================================

In RISC-V, -fpatchable-function-entry option is used to support
dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
don't have to be called to create the __mcount_loc section before
the vmlinux linking.

Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
Makefile not to run recordmcount.

Make function graph use ftrace directly [2] (patch 2)
======================================================== 

In RISC-V architecture, when we enable the ftrace_graph tracer on some
functions, the function tracings on other functions will suffer extra
graph tracing work. In essence, graph_ops isn't limited by its func_hash
due to the global ftrace_graph_[regs]_call label. That should be
corrected.

What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
graph use ftrace directly") that uses graph_ops::func function to
install return_hooker and makes the function called against its
func_hash.

Add WITH_DIRECT_CALLS support [3] (patch 3, 4)
==============================================

This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
here as the samples for testing DIRECT_CALLS related interface.

First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide 
register_ftrace_direct[_multi] interfaces allowing user to register 
the customed trampoline (direct_caller) as the mcount for one or 
more target functions. And modify_ftrace_direct[_multi] are also 
provided for modify direct_caller.

At the same time, the samples in ./samples/ftrace/ can be built
as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
and SAMPLE_FTRACE_DIRECT_MULTI selected.

Second, to make the direct_caller and the other ftrace hooks
(eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
register
are nominated to store the address of direct_caller in
ftrace_regs_caller.
After the setting of the address direct_caller by direct_ops->func and
the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.

The series's old changes related these patches
==========================================

Changes in v9:
https://lore.kernel.org/linux-riscv/20230510101857.2953955-1-suagrfillet@gmail.com/

1. add Acked-by from Björn Töpel in patch 1

2. rebase patch2/patch3 on Linux v6.4-rc1

  - patch 2: to make the `SAVE_ABI_REGS` configurable, revert the
    modification of mcount-dyn.S from commit (45b32b946a97 "riscv:
entry: Consolidate general regs saving/restoring")

  - patch 3: to pass the trace_selftest, add the implement of
    `ftrace_stub_direct_tramp` from commit (fee86a4ed536 "ftrace:
selftest: remove broken trace_direct_tramp") ; and fixup the context
conflict in Kconfig 

Changes in v8:
https://lore.kernel.org/linux-riscv/20230324033342.3177979-1-suagrfillet@gmail.com/
 - Fix incorrect address values in the 4nd patch 
 - Rebased on v6.3-rc2

Changes in v7:
https://lore.kernel.org/linux-riscv/20230112090603.1295340-1-guoren@kernel.org/
 - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
 - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1]
 - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
   HAVE_SAMPLE_FTRACE_DIRECT_MULTI

Changes in v6:
https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
 - Replace 8 with MCOUNT_INSN_SIZE
 - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
 - Add Evgenii Shatokhin comment

Changes in v5:
https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
 - Sort Kconfig entries in alphabetical order.

Changes in v4:
https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
 - Include [3] for maintenance. [Song Shuai]

Changes in V3:
https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
 - Include [2] for maintenance. [Song Shuai]

[1]: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
[2]: https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
[3]: https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/ 

Song Shuai (4):
  riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
  riscv: ftrace: Add ftrace_graph_func
  riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
  samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]

 arch/riscv/Kconfig                          |   4 +
 arch/riscv/include/asm/ftrace.h             |  19 +-
 arch/riscv/kernel/ftrace.c                  |  30 ++-
 arch/riscv/kernel/mcount-dyn.S              | 200 ++++++++++++++++----
 samples/ftrace/ftrace-direct-modify.c       |  34 ++++
 samples/ftrace/ftrace-direct-multi-modify.c |  40 ++++
 samples/ftrace/ftrace-direct-multi.c        |  24 +++
 samples/ftrace/ftrace-direct-too.c          |  27 +++
 samples/ftrace/ftrace-direct.c              |  23 +++
 9 files changed, 345 insertions(+), 56 deletions(-)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 12+ messages in thread
* [PATCH V10 1/4] riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
  2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
@ 2023-05-11  9:32 ` Song Shuai
  2023-05-11  9:32 ` [PATCH V10 2/4] riscv: ftrace: Add ftrace_graph_func Song Shuai
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 12+ messages in thread
From: Song Shuai @ 2023-05-11  9:32 UTC (permalink / raw)
  To: paul.walmsley, palmer, aou, rostedt, mhiramat, mark.rutland,
	guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

In RISC-V, -fpatchable-function-entry option is used to support
dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
don't have to be called to create the __mcount_loc section before
the vmlinux linking.

Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
Makefile not to run recordmcount.

Link: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
Link: https://lore.kernel.org/linux-riscv/Y4jtfrJt+%2FQ5nMOz@spud/
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
Acked-by: Björn Töpel <bjorn@rivosinc.com>
---
 arch/riscv/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 348c0fa1fc8c..e0632493482f 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -58,6 +58,7 @@ config RISCV
 	select COMMON_CLK
 	select CPU_PM if CPU_IDLE || HIBERNATION
 	select EDAC_SUPPORT
+	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE
 	select GENERIC_ARCH_TOPOLOGY
 	select GENERIC_ATOMIC64 if !64BIT
 	select GENERIC_CLOCKEVENTS_BROADCAST if SMP
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* [PATCH V10 2/4] riscv: ftrace: Add ftrace_graph_func
  2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
  2023-05-11  9:32 ` [PATCH V10 1/4] riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY Song Shuai
@ 2023-05-11  9:32 ` Song Shuai
  2023-05-11  9:32 ` [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support Song Shuai
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 12+ messages in thread
From: Song Shuai @ 2023-05-11  9:32 UTC (permalink / raw)
  To: paul.walmsley, palmer, aou, rostedt, mhiramat, mark.rutland,
	guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

Here implements ftrace_graph_func as the function graph tracing function
with FTRACE_WITH_REGS defined.

function_graph_func gets the point of the parent IP and the frame pointer
from fregs and call prepare_ftrace_return for function graph tracing.

If FTRACE_WITH_REGS isn't defined, the enable/disable helpers of
ftrace_graph_[regs]_call are revised for serving only ftrace_graph_call
in the !FTRACE_WITH_REGS version ftrace_caller.

Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
Acked-by: Björn Töpel <bjorn@rivosinc.com>
---
 arch/riscv/include/asm/ftrace.h |  11 +-
 arch/riscv/kernel/ftrace.c      |  30 +++--
 arch/riscv/kernel/mcount-dyn.S  | 190 +++++++++++++++++++++++++-------
 3 files changed, 175 insertions(+), 56 deletions(-)

diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index d47d87c2d7e3..84f856a3286e 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -107,7 +107,16 @@ do {									\
 struct dyn_ftrace;
 int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
 #define ftrace_init_nop ftrace_init_nop
-#endif
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+struct ftrace_ops;
+struct ftrace_regs;
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+		       struct ftrace_ops *op, struct ftrace_regs *fregs);
+#define ftrace_graph_func ftrace_graph_func
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
+#endif /* __ASSEMBLY__ */
 
 #endif /* CONFIG_DYNAMIC_FTRACE */
 
diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c
index 03a6434a8cdd..f5aa24d9e1c1 100644
--- a/arch/riscv/kernel/ftrace.c
+++ b/arch/riscv/kernel/ftrace.c
@@ -178,32 +178,28 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
 }
 
 #ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+		       struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+	struct pt_regs *regs = arch_ftrace_get_regs(fregs);
+	unsigned long *parent = (unsigned long *)&regs->ra;
+
+	prepare_ftrace_return(parent, ip, frame_pointer(regs));
+}
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 extern void ftrace_graph_call(void);
-extern void ftrace_graph_regs_call(void);
 int ftrace_enable_ftrace_graph_caller(void)
 {
-	int ret;
-
-	ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
-				    (unsigned long)&prepare_ftrace_return, true, true);
-	if (ret)
-		return ret;
-
-	return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+	return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
 				    (unsigned long)&prepare_ftrace_return, true, true);
 }
 
 int ftrace_disable_ftrace_graph_caller(void)
 {
-	int ret;
-
-	ret = __ftrace_modify_call((unsigned long)&ftrace_graph_call,
-				    (unsigned long)&prepare_ftrace_return, false, true);
-	if (ret)
-		return ret;
-
-	return __ftrace_modify_call((unsigned long)&ftrace_graph_regs_call,
+	return __ftrace_modify_call((unsigned long)&ftrace_graph_call,
 				    (unsigned long)&prepare_ftrace_return, false, true);
 }
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 #endif /* CONFIG_DYNAMIC_FTRACE */
 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index 669b8697aa38..fb8286b80cfc 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -57,31 +57,150 @@
 	.endm
 
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
-	.macro SAVE_ALL
+
+/**
+* SAVE_ABI_REGS - save regs against the pt_regs struct
+*
+* @all: tell if saving all the regs
+*
+* If all is set, all the regs will be saved, otherwise only ABI
+* related regs (a0-a7,epc,ra and optional s0) will be saved.
+*
+* After the stack is established,
+*
+* 0(sp) stores the PC of the traced function which can be accessed
+* by &(fregs)->regs->epc in tracing function. Note that the real
+* function entry address should be computed with -FENTRY_RA_OFFSET.
+*
+* 8(sp) stores the function return address (i.e. parent IP) that
+* can be accessed by &(fregs)->regs->ra in tracing function.
+*
+* The other regs are saved at the respective localtion and accessed
+* by the respective pt_regs member.
+*
+* Here is the layout of stack for your reference.
+*
+* PT_SIZE_ON_STACK  ->  +++++++++
+*                       + ..... +
+*                       + t3-t6 +
+*                       + s2-s11+
+*                       + a0-a7 + --++++-> ftrace_caller saved
+*                       + s1    +   +
+*                       + s0    + --+
+*                       + t0-t2 +   +
+*                       + tp    +   +
+*                       + gp    +   +
+*                       + sp    +   +
+*                       + ra    + --+ // parent IP
+*               sp  ->  + epc   + --+ // PC
+*                       +++++++++
+**/
+	.macro SAVE_ABI_REGS, all=0
 	addi	sp, sp, -PT_SIZE_ON_STACK
 
-	REG_S t0,  PT_EPC(sp)
-	REG_S x1,  PT_RA(sp)
-	REG_S x2,  PT_SP(sp)
-	REG_S x3,  PT_GP(sp)
-	REG_S x4,  PT_TP(sp)
-	REG_S x5,  PT_T0(sp)
-	save_from_x6_to_x31
+	REG_S	t0,  PT_EPC(sp)
+	REG_S	x1,  PT_RA(sp)
+
+	// save the ABI regs
+
+	REG_S	x10, PT_A0(sp)
+	REG_S	x11, PT_A1(sp)
+	REG_S	x12, PT_A2(sp)
+	REG_S	x13, PT_A3(sp)
+	REG_S	x14, PT_A4(sp)
+	REG_S	x15, PT_A5(sp)
+	REG_S	x16, PT_A6(sp)
+	REG_S	x17, PT_A7(sp)
+
+	// save the leftover regs
+
+	.if \all == 1
+	REG_S	x2, PT_SP(sp)
+	REG_S	x3, PT_GP(sp)
+	REG_S	x4, PT_TP(sp)
+	REG_S	x5, PT_T0(sp)
+	REG_S	x6, PT_T1(sp)
+	REG_S	x7, PT_T2(sp)
+	REG_S	x8, PT_S0(sp)
+	REG_S	x9, PT_S1(sp)
+	REG_S	x18, PT_S2(sp)
+	REG_S	x19, PT_S3(sp)
+	REG_S	x20, PT_S4(sp)
+	REG_S	x21, PT_S5(sp)
+	REG_S	x22, PT_S6(sp)
+	REG_S	x23, PT_S7(sp)
+	REG_S	x24, PT_S8(sp)
+	REG_S	x25, PT_S9(sp)
+	REG_S	x26, PT_S10(sp)
+	REG_S	x27, PT_S11(sp)
+	REG_S	x28, PT_T3(sp)
+	REG_S	x29, PT_T4(sp)
+	REG_S	x30, PT_T5(sp)
+	REG_S	x31, PT_T6(sp)
+
+	// save s0 if FP_TEST defined
+
+	.else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+	REG_S	x8, PT_S0(sp)
+#endif
+	.endif
 	.endm
 
-	.macro RESTORE_ALL
-	REG_L x1,  PT_RA(sp)
-	REG_L x2,  PT_SP(sp)
-	REG_L x3,  PT_GP(sp)
-	REG_L x4,  PT_TP(sp)
-	/* Restore t0 with PT_EPC */
-	REG_L x5,  PT_EPC(sp)
-	restore_from_x6_to_x31
+	.macro RESTORE_ABI_REGS, all=0
+	REG_L	t0, PT_EPC(sp)
+	REG_L	x1, PT_RA(sp)
+	REG_L	x10, PT_A0(sp)
+	REG_L	x11, PT_A1(sp)
+	REG_L	x12, PT_A2(sp)
+	REG_L	x13, PT_A3(sp)
+	REG_L	x14, PT_A4(sp)
+	REG_L	x15, PT_A5(sp)
+	REG_L	x16, PT_A6(sp)
+	REG_L	x17, PT_A7(sp)
 
+	.if \all == 1
+	REG_L	x2, PT_SP(sp)
+	REG_L	x3, PT_GP(sp)
+	REG_L	x4, PT_TP(sp)
+	REG_L	x6, PT_T1(sp)
+	REG_L	x7, PT_T2(sp)
+	REG_L	x8, PT_S0(sp)
+	REG_L	x9, PT_S1(sp)
+	REG_L	x18, PT_S2(sp)
+	REG_L	x19, PT_S3(sp)
+	REG_L	x20, PT_S4(sp)
+	REG_L	x21, PT_S5(sp)
+	REG_L	x22, PT_S6(sp)
+	REG_L	x23, PT_S7(sp)
+	REG_L	x24, PT_S8(sp)
+	REG_L	x25, PT_S9(sp)
+	REG_L	x26, PT_S10(sp)
+	REG_L	x27, PT_S11(sp)
+	REG_L	x28, PT_T3(sp)
+	REG_L	x29, PT_T4(sp)
+	REG_L	x30, PT_T5(sp)
+	REG_L	x31, PT_T6(sp)
+
+	.else
+#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
+	REG_L	x8, PT_S0(sp)
+#endif
+	.endif
 	addi	sp, sp, PT_SIZE_ON_STACK
 	.endm
+
+	.macro PREPARE_ARGS
+	addi	a0, t0, -FENTRY_RA_OFFSET
+	la	a1, function_trace_op
+	REG_L	a2, 0(a1)
+	mv	a1, ra
+	mv	a3, sp
+	.endm
+
 #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 
+#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
 ENTRY(ftrace_caller)
 	SAVE_ABI
 
@@ -107,36 +226,31 @@ ftrace_graph_call:
 	call	ftrace_stub
 #endif
 	RESTORE_ABI
-	jr t0
+	jr	t0
 ENDPROC(ftrace_caller)
 
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 ENTRY(ftrace_regs_caller)
-	SAVE_ALL
-
-	addi	a0, t0, -FENTRY_RA_OFFSET
-	la	a1, function_trace_op
-	REG_L	a2, 0(a1)
-	mv	a1, ra
-	mv	a3, sp
+	SAVE_ABI_REGS 1
+	PREPARE_ARGS
 
 ftrace_regs_call:
 	.global ftrace_regs_call
 	call	ftrace_stub
 
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-	addi	a0, sp, PT_RA
-	REG_L	a1, PT_EPC(sp)
-	addi	a1, a1, -FENTRY_RA_OFFSET
-#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
-	mv	a2, s0
-#endif
-ftrace_graph_regs_call:
-	.global ftrace_graph_regs_call
+	RESTORE_ABI_REGS 1
+	jr	t0
+ENDPROC(ftrace_regs_caller)
+
+ENTRY(ftrace_caller)
+	SAVE_ABI_REGS 0
+	PREPARE_ARGS
+
+ftrace_call:
+	.global ftrace_call
 	call	ftrace_stub
-#endif
 
-	RESTORE_ALL
-	jr t0
-ENDPROC(ftrace_regs_caller)
+	RESTORE_ABI_REGS 0
+	jr	t0
+ENDPROC(ftrace_caller)
 #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
  2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
  2023-05-11  9:32 ` [PATCH V10 1/4] riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY Song Shuai
  2023-05-11  9:32 ` [PATCH V10 2/4] riscv: ftrace: Add ftrace_graph_func Song Shuai
@ 2023-05-11  9:32 ` Song Shuai
  2023-05-11 19:42   ` Björn Töpel
  2023-05-11  9:32 ` [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI] Song Shuai
  2023-05-16  9:50 ` [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
  4 siblings, 1 reply; 12+ messages in thread
From: Song Shuai @ 2023-05-11  9:32 UTC (permalink / raw)
  To: paul.walmsley, palmer, aou, rostedt, mhiramat, mark.rutland,
	guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

This patch adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.

select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide the
register_ftrace_direct[_multi] interfaces allowing users to register
the customed trampoline (direct_caller) as the mcount for one or
more target functions. And modify_ftrace_direct[_multi] are also
provided for modifying direct_caller.

To make the direct_caller and the other ftrace hooks (eg. function/fgraph
tracer, k[ret]probes) co-exist, a temporary register is nominated to
store the address of direct_caller in ftrace_regs_caller. After the
setting of the address direct_caller by direct_ops->func and the
RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
by the `jr` inst.

Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
 arch/riscv/Kconfig              |  1 +
 arch/riscv/include/asm/ftrace.h |  8 ++++++++
 arch/riscv/kernel/mcount-dyn.S  | 10 ++++++++++
 3 files changed, 19 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e0632493482f..fdf0b219a02c 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -144,6 +144,7 @@ config RISCV
 	select UACCESS_MEMCPY if !MMU
 	select ZONE_DMA32 if 64BIT
 	select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && (CLANG_SUPPORTS_DYNAMIC_FTRACE || GCC_SUPPORTS_DYNAMIC_FTRACE)
+	select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER
diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h
index 84f856a3286e..84904c1e4369 100644
--- a/arch/riscv/include/asm/ftrace.h
+++ b/arch/riscv/include/asm/ftrace.h
@@ -114,6 +114,14 @@ struct ftrace_regs;
 void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
 		       struct ftrace_ops *op, struct ftrace_regs *fregs);
 #define ftrace_graph_func ftrace_graph_func
+
+static inline void
+__arch_ftrace_set_direct_caller(struct pt_regs *regs, unsigned long addr)
+{
+		regs->t1 = addr;
+}
+#define arch_ftrace_set_direct_caller(fregs, addr) \
+	__arch_ftrace_set_direct_caller(&(fregs)->regs, addr)
 #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S
index fb8286b80cfc..b6f4e1847d61 100644
--- a/arch/riscv/kernel/mcount-dyn.S
+++ b/arch/riscv/kernel/mcount-dyn.S
@@ -231,6 +231,7 @@ ENDPROC(ftrace_caller)
 
 #else /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
 ENTRY(ftrace_regs_caller)
+	mv	t1, zero
 	SAVE_ABI_REGS 1
 	PREPARE_ARGS
 
@@ -239,7 +240,10 @@ ftrace_regs_call:
 	call	ftrace_stub
 
 	RESTORE_ABI_REGS 1
+	bnez	t1,.Ldirect
 	jr	t0
+.Ldirect:
+	jr	t1
 ENDPROC(ftrace_regs_caller)
 
 ENTRY(ftrace_caller)
@@ -254,3 +258,9 @@ ftrace_call:
 	jr	t0
 ENDPROC(ftrace_caller)
 #endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
+SYM_CODE_START(ftrace_stub_direct_tramp)
+	jr	t0
+SYM_CODE_END(ftrace_stub_direct_tramp)
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
  2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
                   ` (2 preceding siblings ...)
  2023-05-11  9:32 ` [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support Song Shuai
@ 2023-05-11  9:32 ` Song Shuai
  2023-05-15 18:37   ` Steven Rostedt
  2023-05-16  9:50 ` [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
  4 siblings, 1 reply; 12+ messages in thread
From: Song Shuai @ 2023-05-11  9:32 UTC (permalink / raw)
  To: paul.walmsley, palmer, aou, rostedt, mhiramat, mark.rutland,
	guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
the ftrace-direct*.c files in samples/ftrace/.

Link: https://lore.kernel.org/linux-riscv/c68bac83-5c88-80b1-bac9-e1fd4ea8f07e@yadro.com/T/#ma13012560331c66b051b580b3ab4a04ba44455ec
Tested-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
Signed-off-by: Song Shuai <suagrfillet@gmail.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
Acked-by: Björn Töpel <bjorn@rivosinc.com>
---
 arch/riscv/Kconfig                          |  2 ++
 samples/ftrace/ftrace-direct-modify.c       | 34 ++++++++++++++++++
 samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++
 samples/ftrace/ftrace-direct-multi.c        | 24 +++++++++++++
 samples/ftrace/ftrace-direct-too.c          | 27 ++++++++++++++
 samples/ftrace/ftrace-direct.c              | 23 ++++++++++++
 6 files changed, 150 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index fdf0b219a02c..cb94ef086f0c 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -121,6 +121,8 @@ config RISCV
 	select HAVE_POSIX_CPU_TIMERS_TASK_WORK
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RSEQ
+	select HAVE_SAMPLE_FTRACE_DIRECT
+	select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
 	select HAVE_STACKPROTECTOR
 	select HAVE_SYSCALL_TRACEPOINTS
 	select IRQ_DOMAIN
diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
index 06d889149012..e90ca7b68314 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -22,6 +22,40 @@ extern void my_tramp2(void *);
 
 static unsigned long my_ip = (unsigned long)schedule;
 
+#ifdef CONFIG_RISCV
+
+asm (
+"	.pushsection    .text, \"ax\", @progbits\n"
+"	.type		my_tramp1, @function\n"
+"	.globl		my_tramp1\n"
+"   my_tramp1:\n"
+"	addi	sp,sp,-16\n"
+"	sd	t0,0(sp)\n"
+"	sd	ra,8(sp)\n"
+"	call	my_direct_func1\n"
+"	ld	t0,0(sp)\n"
+"	ld	ra,8(sp)\n"
+"	addi	sp,sp,16\n"
+"	jr	t0\n"
+"	.size		my_tramp1, .-my_tramp1\n"
+
+"	.type		my_tramp2, @function\n"
+"	.globl		my_tramp2\n"
+"   my_tramp2:\n"
+"	addi	sp,sp,-16\n"
+"	sd	t0,0(sp)\n"
+"	sd	ra,8(sp)\n"
+"	call	my_direct_func2\n"
+"	ld	t0,0(sp)\n"
+"	ld	ra,8(sp)\n"
+"	addi	sp,sp,16\n"
+"	jr	t0\n"
+"	.size		my_tramp2, .-my_tramp2\n"
+"	.popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
 #ifdef CONFIG_X86_64
 
 #include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
index 62f6b681999e..5a81af7b3af3 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -20,6 +20,46 @@ void my_direct_func2(unsigned long ip)
 extern void my_tramp1(void *);
 extern void my_tramp2(void *);
 
+#ifdef CONFIG_RISCV
+
+asm (
+"	.pushsection    .text, \"ax\", @progbits\n"
+"	.type		my_tramp1, @function\n"
+"	.globl		my_tramp1\n"
+"   my_tramp1:\n"
+"       addi	sp,sp,-24\n"
+"       sd	a0,0(sp)\n"
+"       sd	t0,8(sp)\n"
+"       sd	ra,16(sp)\n"
+"       mv	a0,t0\n"
+"       call	my_direct_func1\n"
+"       ld	a0,0(sp)\n"
+"       ld	t0,8(sp)\n"
+"       ld	ra,16(sp)\n"
+"       addi	sp,sp,24\n"
+"	jr	t0\n"
+"	.size		my_tramp1, .-my_tramp1\n"
+
+"	.type		my_tramp2, @function\n"
+"	.globl		my_tramp2\n"
+"   my_tramp2:\n"
+"       addi	sp,sp,-24\n"
+"       sd	a0,0(sp)\n"
+"       sd	t0,8(sp)\n"
+"       sd	ra,16(sp)\n"
+"       mv	a0,t0\n"
+"       call	my_direct_func2\n"
+"       ld	a0,0(sp)\n"
+"       ld	t0,8(sp)\n"
+"       ld	ra,16(sp)\n"
+"       addi	sp,sp,24\n"
+"	jr	t0\n"
+"	.size		my_tramp2, .-my_tramp2\n"
+"	.popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
 #ifdef CONFIG_X86_64
 
 #include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c
index 5482cf616b43..0e9bb94edade 100644
--- a/samples/ftrace/ftrace-direct-multi.c
+++ b/samples/ftrace/ftrace-direct-multi.c
@@ -15,6 +15,30 @@ void my_direct_func(unsigned long ip)
 
 extern void my_tramp(void *);
 
+#ifdef CONFIG_RISCV
+
+asm (
+"       .pushsection    .text, \"ax\", @progbits\n"
+"       .type           my_tramp, @function\n"
+"       .globl          my_tramp\n"
+"   my_tramp:\n"
+"       addi	sp,sp,-24\n"
+"       sd	a0,0(sp)\n"
+"       sd	t0,8(sp)\n"
+"       sd	ra,16(sp)\n"
+"       mv	a0,t0\n"
+"       call	my_direct_func\n"
+"       ld	a0,0(sp)\n"
+"       ld	t0,8(sp)\n"
+"       ld	ra,16(sp)\n"
+"       addi	sp,sp,24\n"
+"       jr	t0\n"
+"       .size           my_tramp, .-my_tramp\n"
+"       .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
 #ifdef CONFIG_X86_64
 
 #include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
index a05bc2cc2261..5c319db48af2 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -17,6 +17,33 @@ void my_direct_func(struct vm_area_struct *vma,
 
 extern void my_tramp(void *);
 
+#ifdef CONFIG_RISCV
+
+asm (
+"       .pushsection    .text, \"ax\", @progbits\n"
+"       .type           my_tramp, @function\n"
+"       .globl          my_tramp\n"
+"   my_tramp:\n"
+"       addi	sp,sp,-40\n"
+"       sd	a0,0(sp)\n"
+"       sd	a1,8(sp)\n"
+"       sd	a2,16(sp)\n"
+"       sd	t0,24(sp)\n"
+"       sd	ra,32(sp)\n"
+"       call	my_direct_func\n"
+"       ld	a0,0(sp)\n"
+"       ld	a1,8(sp)\n"
+"       ld	a2,16(sp)\n"
+"       ld	t0,24(sp)\n"
+"       ld	ra,32(sp)\n"
+"       addi	sp,sp,40\n"
+"       jr	t0\n"
+"       .size           my_tramp, .-my_tramp\n"
+"       .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
 #ifdef CONFIG_X86_64
 
 #include <asm/ibt.h>
diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c
index 06879bbd3399..ca95506b0350 100644
--- a/samples/ftrace/ftrace-direct.c
+++ b/samples/ftrace/ftrace-direct.c
@@ -14,6 +14,29 @@ void my_direct_func(struct task_struct *p)
 
 extern void my_tramp(void *);
 
+#ifdef CONFIG_RISCV
+
+asm (
+"       .pushsection    .text, \"ax\", @progbits\n"
+"       .type           my_tramp, @function\n"
+"       .globl          my_tramp\n"
+"   my_tramp:\n"
+"       addi	sp,sp,-24\n"
+"       sd	a0,0(sp)\n"
+"       sd	t0,8(sp)\n"
+"       sd	ra,16(sp)\n"
+"       call	my_direct_func\n"
+"       ld	a0,0(sp)\n"
+"       ld	t0,8(sp)\n"
+"       ld	ra,16(sp)\n"
+"       addi	sp,sp,24\n"
+"       jr	t0\n"
+"       .size           my_tramp, .-my_tramp\n"
+"       .popsection\n"
+);
+
+#endif /* CONFIG_RISCV */
+
 #ifdef CONFIG_X86_64
 
 #include <asm/ibt.h>
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
  2023-05-11  9:32 ` [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support Song Shuai
@ 2023-05-11 19:42   ` Björn Töpel
  0 siblings, 0 replies; 12+ messages in thread
From: Björn Töpel @ 2023-05-11 19:42 UTC (permalink / raw)
  To: Song Shuai, paul.walmsley, palmer, aou, rostedt, mhiramat,
	mark.rutland, guoren, suagrfillet, jszhang, e.shatokhin, bjorn
  Cc: linux-riscv, linux-kernel, linux-trace-kernel

Song Shuai <suagrfillet@gmail.com> writes:

> This patch adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
>
> select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide the
> register_ftrace_direct[_multi] interfaces allowing users to register
> the customed trampoline (direct_caller) as the mcount for one or
> more target functions. And modify_ftrace_direct[_multi] are also
> provided for modifying direct_caller.
>
> To make the direct_caller and the other ftrace hooks (eg. function/fgraph
> tracer, k[ret]probes) co-exist, a temporary register is nominated to
> store the address of direct_caller in ftrace_regs_caller. After the
> setting of the address direct_caller by direct_ops->func and the
> RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> by the `jr` inst.
>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Tested-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@kernel.org>

Thank you for addressing my comments, and for your work on
WITH_DIRECT_CALLS!

Acked-by: Björn Töpel <bjorn@rivosinc.com>

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
  2023-05-11  9:32 ` [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI] Song Shuai
@ 2023-05-15 18:37   ` Steven Rostedt
  2023-05-15 18:38     ` Steven Rostedt
  2023-05-16  9:37     ` Song Shuai
  0 siblings, 2 replies; 12+ messages in thread
From: Steven Rostedt @ 2023-05-15 18:37 UTC (permalink / raw)
  To: Song Shuai
  Cc: paul.walmsley, palmer, aou, mhiramat, mark.rutland, guoren,
	jszhang, e.shatokhin, bjorn, linux-riscv, linux-kernel,
	linux-trace-kernel

On Thu, 11 May 2023 17:32:34 +0800
Song Shuai <suagrfillet@gmail.com> wrote:

> select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> the ftrace-direct*.c files in samples/ftrace/.
> 
> Link: https://lore.kernel.org/linux-riscv/c68bac83-5c88-80b1-bac9-e1fd4ea8f07e@yadro.com/T/#ma13012560331c66b051b580b3ab4a04ba44455ec
> Tested-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
> Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> Tested-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Acked-by: Björn Töpel <bjorn@rivosinc.com>
> ---
>  arch/riscv/Kconfig                          |  2 ++
>  samples/ftrace/ftrace-direct-modify.c       | 34 ++++++++++++++++++
>  samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++
>  samples/ftrace/ftrace-direct-multi.c        | 24 +++++++++++++
>  samples/ftrace/ftrace-direct-too.c          | 27 ++++++++++++++
>  samples/ftrace/ftrace-direct.c              | 23 ++++++++++++
>  6 files changed, 150 insertions(+)

I acked v8 (which you will need my ack to get these in). Unless this patch
changed since v8, you should include my ack in further versions.

Or you may be waiting longer for its acceptance.

If the patch did change, please acknowledge that in the next version (or
this one if it did change), asking me to ack it again.

-- Steve

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
  2023-05-15 18:37   ` Steven Rostedt
@ 2023-05-15 18:38     ` Steven Rostedt
  2023-05-16  9:37     ` Song Shuai
  1 sibling, 0 replies; 12+ messages in thread
From: Steven Rostedt @ 2023-05-15 18:38 UTC (permalink / raw)
  To: Song Shuai
  Cc: paul.walmsley, palmer, aou, mhiramat, mark.rutland, guoren,
	jszhang, e.shatokhin, bjorn, linux-riscv, linux-kernel,
	linux-trace-kernel

On Mon, 15 May 2023 14:37:00 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Thu, 11 May 2023 17:32:34 +0800
> Song Shuai <suagrfillet@gmail.com> wrote:
> 
> > select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > the ftrace-direct*.c files in samples/ftrace/.
> > 
> > Link: https://lore.kernel.org/linux-riscv/c68bac83-5c88-80b1-bac9-e1fd4ea8f07e@yadro.com/T/#ma13012560331c66b051b580b3ab4a04ba44455ec
> > Tested-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
> > Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > Tested-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Acked-by: Björn Töpel <bjorn@rivosinc.com>
> > ---
> >  arch/riscv/Kconfig                          |  2 ++
> >  samples/ftrace/ftrace-direct-modify.c       | 34 ++++++++++++++++++
> >  samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++
> >  samples/ftrace/ftrace-direct-multi.c        | 24 +++++++++++++
> >  samples/ftrace/ftrace-direct-too.c          | 27 ++++++++++++++
> >  samples/ftrace/ftrace-direct.c              | 23 ++++++++++++
> >  6 files changed, 150 insertions(+)  
> 
> I acked v8 (which you will need my ack to get these in). Unless this patch
> changed since v8, you should include my ack in further versions.
> 
> Or you may be waiting longer for its acceptance.
> 
> If the patch did change, please acknowledge that in the next version (or
> this one if it did change), asking me to ack it again.
> 

Nevermind, I'm an idiot (and my inbox is not in date order). You sent this
*after* I acked v8, when I should have been acking this one. ;-)

Sorry about that! I've been traveling and my inbox is a mess :-p

-- Steve

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
  2023-05-15 18:37   ` Steven Rostedt
  2023-05-15 18:38     ` Steven Rostedt
@ 2023-05-16  9:37     ` Song Shuai
  2023-06-25 20:32       ` Palmer Dabbelt
  1 sibling, 1 reply; 12+ messages in thread
From: Song Shuai @ 2023-05-16  9:37 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: paul.walmsley, palmer, aou, mhiramat, mark.rutland, guoren,
	jszhang, e.shatokhin, bjorn, linux-riscv, linux-kernel,
	linux-trace-kernel

Steven Rostedt <rostedt@goodmis.org> 于2023年5月15日周一 18:37写道：
>
> On Thu, 11 May 2023 17:32:34 +0800
> Song Shuai <suagrfillet@gmail.com> wrote:
>
> > select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> > for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
> > the ftrace-direct*.c files in samples/ftrace/.
> >
> > Link: https://lore.kernel.org/linux-riscv/c68bac83-5c88-80b1-bac9-e1fd4ea8f07e@yadro.com/T/#ma13012560331c66b051b580b3ab4a04ba44455ec
> > Tested-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
> > Signed-off-by: Song Shuai <suagrfillet@gmail.com>
> > Tested-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Acked-by: Björn Töpel <bjorn@rivosinc.com>
> > ---
> >  arch/riscv/Kconfig                          |  2 ++
> >  samples/ftrace/ftrace-direct-modify.c       | 34 ++++++++++++++++++
> >  samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++
> >  samples/ftrace/ftrace-direct-multi.c        | 24 +++++++++++++
> >  samples/ftrace/ftrace-direct-too.c          | 27 ++++++++++++++
> >  samples/ftrace/ftrace-direct.c              | 23 ++++++++++++
> >  6 files changed, 150 insertions(+)
>
> I acked v8 (which you will need my ack to get these in). Unless this patch
> changed since v8, you should include my ack in further versions.
>
There were no changes in this patch, except for a few tab pretties since v8.
your Acked-by will still be included.
> Or you may be waiting longer for its acceptance.
>
> If the patch did change, please acknowledge that in the next version (or
> this one if it did change), asking me to ack it again.
>
> -- Steve



-- 
Thanks,
Song

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 0/4] riscv: Optimize function trace
  2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
                   ` (3 preceding siblings ...)
  2023-05-11  9:32 ` [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI] Song Shuai
@ 2023-05-16  9:50 ` Song Shuai
  2023-05-17 11:09   ` Song Shuai
  4 siblings, 1 reply; 12+ messages in thread
From: Song Shuai @ 2023-05-16  9:50 UTC (permalink / raw)
  To: palmer
  Cc: mark.rutland, bjorn, e.shatokhin, jszhang, suagrfillet, guoren,
	mhiramat, rostedt, aou, paul.walmsley, linux-riscv, linux-kernel,
	linux-trace-kernel

Song Shuai <suagrfillet@gmail.com> 于2023年5月11日周四 09:32写道：

>
> Changes in v10:
>
> - add Acked-by from Björn Töpel in patch 2 and patch 4
> - replace `move` with `mv` in patch3
> - prettify patch 2/4 with proper tabs
>
> You can directly try it with:
>
> https://github.com/sugarfillet/linux/tree/6.4-rc1-rv-ftrace-v10
>
>
Hi, plamer:
This series is ready for its acceptance, you can pick it up.

> This series optimizes function trace. The first 3 independent
> patches has been picked in the V7 version of this series, the
> subsequent version continues the following 4 patches:
>
> select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1] (patch 1)
> ==========================================================
>
> In RISC-V, -fpatchable-function-entry option is used to support
> dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
> PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
> don't have to be called to create the __mcount_loc section before
> the vmlinux linking.
>
> Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
> Makefile not to run recordmcount.
>
> Make function graph use ftrace directly [2] (patch 2)
> ========================================================
>
> In RISC-V architecture, when we enable the ftrace_graph tracer on some
> functions, the function tracings on other functions will suffer extra
> graph tracing work. In essence, graph_ops isn't limited by its func_hash
> due to the global ftrace_graph_[regs]_call label. That should be
> corrected.
>
> What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> graph use ftrace directly") that uses graph_ops::func function to
> install return_hooker and makes the function called against its
> func_hash.
>
> Add WITH_DIRECT_CALLS support [3] (patch 3, 4)
> ==============================================
>
> This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> here as the samples for testing DIRECT_CALLS related interface.
>
> First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> register_ftrace_direct[_multi] interfaces allowing user to register
> the customed trampoline (direct_caller) as the mcount for one or
> more target functions. And modify_ftrace_direct[_multi] are also
> provided for modify direct_caller.
>
> At the same time, the samples in ./samples/ftrace/ can be built
> as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> and SAMPLE_FTRACE_DIRECT_MULTI selected.
>
> Second, to make the direct_caller and the other ftrace hooks
> (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> register
> are nominated to store the address of direct_caller in
> ftrace_regs_caller.
> After the setting of the address direct_caller by direct_ops->func and
> the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> by the `jr` inst.
>
> The series's old changes related these patches
> ==========================================
>
> Changes in v9:
> https://lore.kernel.org/linux-riscv/20230510101857.2953955-1-suagrfillet@gmail.com/
>
> 1. add Acked-by from Björn Töpel in patch 1
>
> 2. rebase patch2/patch3 on Linux v6.4-rc1
>
>   - patch 2: to make the `SAVE_ABI_REGS` configurable, revert the
>     modification of mcount-dyn.S from commit (45b32b946a97 "riscv:
> entry: Consolidate general regs saving/restoring")
>
>   - patch 3: to pass the trace_selftest, add the implement of
>     `ftrace_stub_direct_tramp` from commit (fee86a4ed536 "ftrace:
> selftest: remove broken trace_direct_tramp") ; and fixup the context
> conflict in Kconfig
>
> Changes in v8:
> https://lore.kernel.org/linux-riscv/20230324033342.3177979-1-suagrfillet@gmail.com/
>  - Fix incorrect address values in the 4nd patch
>  - Rebased on v6.3-rc2
>
> Changes in v7:
> https://lore.kernel.org/linux-riscv/20230112090603.1295340-1-guoren@kernel.org/
>  - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
>  - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1]
>  - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
>    HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>
> Changes in v6:
> https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
>  - Replace 8 with MCOUNT_INSN_SIZE
>  - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
>  - Add Evgenii Shatokhin comment
>
> Changes in v5:
> https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
>  - Sort Kconfig entries in alphabetical order.
>
> Changes in v4:
> https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
>  - Include [3] for maintenance. [Song Shuai]
>
> Changes in V3:
> https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
>  - Include [2] for maintenance. [Song Shuai]
>
> [1]: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
> [2]: https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> [3]: https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
>
> Song Shuai (4):
>   riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
>   riscv: ftrace: Add ftrace_graph_func
>   riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
>   samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
>
>  arch/riscv/Kconfig                          |   4 +
>  arch/riscv/include/asm/ftrace.h             |  19 +-
>  arch/riscv/kernel/ftrace.c                  |  30 ++-
>  arch/riscv/kernel/mcount-dyn.S              | 200 ++++++++++++++++----
>  samples/ftrace/ftrace-direct-modify.c       |  34 ++++
>  samples/ftrace/ftrace-direct-multi-modify.c |  40 ++++
>  samples/ftrace/ftrace-direct-multi.c        |  24 +++
>  samples/ftrace/ftrace-direct-too.c          |  27 +++
>  samples/ftrace/ftrace-direct.c              |  23 +++
>  9 files changed, 345 insertions(+), 56 deletions(-)
>
> --
> 2.20.1
>


--
Thanks,
Song

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 0/4] riscv: Optimize function trace
  2023-05-16  9:50 ` [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
@ 2023-05-17 11:09   ` Song Shuai
  0 siblings, 0 replies; 12+ messages in thread
From: Song Shuai @ 2023-05-17 11:09 UTC (permalink / raw)
  To: palmer
  Cc: mark.rutland, bjorn, e.shatokhin, jszhang, guoren, mhiramat,
	rostedt, aou, paul.walmsley, linux-riscv, linux-kernel,
	linux-trace-kernel

Song Shuai <suagrfillet@gmail.com> 于2023年5月16日周二 09:50写道：
>
> Song Shuai <suagrfillet@gmail.com> 于2023年5月11日周四 09:32写道：
>
> >
> > Changes in v10:
> >
> > - add Acked-by from Björn Töpel in patch 2 and patch 4
> > - replace `move` with `mv` in patch3
> > - prettify patch 2/4 with proper tabs
> >
> > You can directly try it with:
> >
> > https://github.com/sugarfillet/linux/tree/6.4-rc1-rv-ftrace-v10
> >
> >
> Hi, plamer:
> This series is ready for its acceptance, you can pick it up.
>
Apologize for the typo in your name, Palmer.

> > This series optimizes function trace. The first 3 independent
> > patches has been picked in the V7 version of this series, the
> > subsequent version continues the following 4 patches:
> >
> > select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1] (patch 1)
> > ==========================================================
> >
> > In RISC-V, -fpatchable-function-entry option is used to support
> > dynamic ftrace in this commit afc76b8b8011 ("riscv: Using
> > PATCHABLE_FUNCTION_ENTRY instead of MCOUNT"). So recordmcount
> > don't have to be called to create the __mcount_loc section before
> > the vmlinux linking.
> >
> > Here selects FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY to tell
> > Makefile not to run recordmcount.
> >
> > Make function graph use ftrace directly [2] (patch 2)
> > ========================================================
> >
> > In RISC-V architecture, when we enable the ftrace_graph tracer on some
> > functions, the function tracings on other functions will suffer extra
> > graph tracing work. In essence, graph_ops isn't limited by its func_hash
> > due to the global ftrace_graph_[regs]_call label. That should be
> > corrected.
> >
> > What inspires me is the commit 0c0593b45c9b ("x86/ftrace: Make function
> > graph use ftrace directly") that uses graph_ops::func function to
> > install return_hooker and makes the function called against its
> > func_hash.
> >
> > Add WITH_DIRECT_CALLS support [3] (patch 3, 4)
> > ==============================================
> >
> > This series adds DYNAMIC_FTRACE_WITH_DIRECT_CALLS support for RISC-V.
> > SAMPLE_FTRACE_DIRECT and SAMPLE_FTRACE_DIRECT_MULTI are also included
> > here as the samples for testing DIRECT_CALLS related interface.
> >
> > First, select the DYNAMIC_FTRACE_WITH_DIRECT_CALLS to provide
> > register_ftrace_direct[_multi] interfaces allowing user to register
> > the customed trampoline (direct_caller) as the mcount for one or
> > more target functions. And modify_ftrace_direct[_multi] are also
> > provided for modify direct_caller.
> >
> > At the same time, the samples in ./samples/ftrace/ can be built
> > as kerenl module for testing these interfaces with SAMPLE_FTRACE_DIRECT
> > and SAMPLE_FTRACE_DIRECT_MULTI selected.
> >
> > Second, to make the direct_caller and the other ftrace hooks
> > (eg. function/fgraph tracer, k[ret]probes) co-exist, a temporary
> > register
> > are nominated to store the address of direct_caller in
> > ftrace_regs_caller.
> > After the setting of the address direct_caller by direct_ops->func and
> > the RESTORE_REGS in ftrace_regs_caller, direct_caller will be jumped to
> > by the `jr` inst.
> >
> > The series's old changes related these patches
> > ==========================================
> >
> > Changes in v9:
> > https://lore.kernel.org/linux-riscv/20230510101857.2953955-1-suagrfillet@gmail.com/
> >
> > 1. add Acked-by from Björn Töpel in patch 1
> >
> > 2. rebase patch2/patch3 on Linux v6.4-rc1
> >
> >   - patch 2: to make the `SAVE_ABI_REGS` configurable, revert the
> >     modification of mcount-dyn.S from commit (45b32b946a97 "riscv:
> > entry: Consolidate general regs saving/restoring")
> >
> >   - patch 3: to pass the trace_selftest, add the implement of
> >     `ftrace_stub_direct_tramp` from commit (fee86a4ed536 "ftrace:
> > selftest: remove broken trace_direct_tramp") ; and fixup the context
> > conflict in Kconfig
> >
> > Changes in v8:
> > https://lore.kernel.org/linux-riscv/20230324033342.3177979-1-suagrfillet@gmail.com/
> >  - Fix incorrect address values in the 4nd patch
> >  - Rebased on v6.3-rc2
> >
> > Changes in v7:
> > https://lore.kernel.org/linux-riscv/20230112090603.1295340-1-guoren@kernel.org/
> >  - Fixup RESTORE_ABI_REGS by remove PT_T0(sp) overwrite.
> >  - Add FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY [1]
> >  - Fixup kconfig with HAVE_SAMPLE_FTRACE_DIRECT &
> >    HAVE_SAMPLE_FTRACE_DIRECT_MULTI
> >
> > Changes in v6:
> > https://lore.kernel.org/linux-riscv/20230107133549.4192639-1-guoren@kernel.org/
> >  - Replace 8 with MCOUNT_INSN_SIZE
> >  - Replace "REG_L a1, PT_RA(sp)" with "mv a1, ra"
> >  - Add Evgenii Shatokhin comment
> >
> > Changes in v5:
> > https://lore.kernel.org/linux-riscv/20221208091244.203407-1-guoren@kernel.org/
> >  - Sort Kconfig entries in alphabetical order.
> >
> > Changes in v4:
> > https://lore.kernel.org/linux-riscv/20221129033230.255947-1-guoren@kernel.org/
> >  - Include [3] for maintenance. [Song Shuai]
> >
> > Changes in V3:
> > https://lore.kernel.org/linux-riscv/20221123153950.2911981-1-guoren@kernel.org/
> >  - Include [2] for maintenance. [Song Shuai]
> >
> > [1]: https://lore.kernel.org/linux-riscv/CAAYs2=j3Eak9vU6xbAw0zPuoh00rh8v5C2U3fePkokZFibWs2g@mail.gmail.com/T/#t
> > [2]: https://lore.kernel.org/lkml/20221120084230.910152-1-suagrfillet@gmail.com/
> > [3]: https://lore.kernel.org/linux-riscv/20221123142025.1504030-1-suagrfillet@gmail.com/
> >
> > Song Shuai (4):
> >   riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
> >   riscv: ftrace: Add ftrace_graph_func
> >   riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support
> >   samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
> >
> >  arch/riscv/Kconfig                          |   4 +
> >  arch/riscv/include/asm/ftrace.h             |  19 +-
> >  arch/riscv/kernel/ftrace.c                  |  30 ++-
> >  arch/riscv/kernel/mcount-dyn.S              | 200 ++++++++++++++++----
> >  samples/ftrace/ftrace-direct-modify.c       |  34 ++++
> >  samples/ftrace/ftrace-direct-multi-modify.c |  40 ++++
> >  samples/ftrace/ftrace-direct-multi.c        |  24 +++
> >  samples/ftrace/ftrace-direct-too.c          |  27 +++
> >  samples/ftrace/ftrace-direct.c              |  23 +++
> >  9 files changed, 345 insertions(+), 56 deletions(-)
> >
> > --
> > 2.20.1
> >
>
>
> --
> Thanks,
> Song



-- 
Thanks,
Song

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI]
  2023-05-16  9:37     ` Song Shuai
@ 2023-06-25 20:32       ` Palmer Dabbelt
  0 siblings, 0 replies; 12+ messages in thread
From: Palmer Dabbelt @ 2023-06-25 20:32 UTC (permalink / raw)
  To: suagrfillet
  Cc: rostedt, Paul Walmsley, aou, mhiramat, Mark Rutland, guoren,
	jszhang, e.shatokhin, Bjorn Topel, linux-riscv, linux-kernel,
	linux-trace-kernel

On Tue, 16 May 2023 02:37:53 PDT (-0700), suagrfillet@gmail.com wrote:
> Steven Rostedt <rostedt@goodmis.org> 于2023年5月15日周一 18:37写道：
>>
>> On Thu, 11 May 2023 17:32:34 +0800
>> Song Shuai <suagrfillet@gmail.com> wrote:
>>
>> > select HAVE_SAMPLE_FTRACE_DIRECT and HAVE_SAMPLE_FTRACE_DIRECT_MULTI
>> > for ARCH_RV64I in arch/riscv/Kconfig. And add riscv asm code for
>> > the ftrace-direct*.c files in samples/ftrace/.
>> >
>> > Link: https://lore.kernel.org/linux-riscv/c68bac83-5c88-80b1-bac9-e1fd4ea8f07e@yadro.com/T/#ma13012560331c66b051b580b3ab4a04ba44455ec
>> > Tested-by: Evgenii Shatokhin <e.shatokhin@yadro.com>
>> > Signed-off-by: Song Shuai <suagrfillet@gmail.com>
>> > Tested-by: Guo Ren <guoren@kernel.org>
>> > Signed-off-by: Guo Ren <guoren@kernel.org>
>> > Acked-by: Björn Töpel <bjorn@rivosinc.com>
>> > ---
>> >  arch/riscv/Kconfig                          |  2 ++
>> >  samples/ftrace/ftrace-direct-modify.c       | 34 ++++++++++++++++++
>> >  samples/ftrace/ftrace-direct-multi-modify.c | 40 +++++++++++++++++++++
>> >  samples/ftrace/ftrace-direct-multi.c        | 24 +++++++++++++
>> >  samples/ftrace/ftrace-direct-too.c          | 27 ++++++++++++++
>> >  samples/ftrace/ftrace-direct.c              | 23 ++++++++++++
>> >  6 files changed, 150 insertions(+)
>>
>> I acked v8 (which you will need my ack to get these in). Unless this patch
>> changed since v8, you should include my ack in further versions.
>>
> There were no changes in this patch, except for a few tab pretties since v8.
> your Acked-by will still be included.
>> Or you may be waiting longer for its acceptance.
>>
>> If the patch did change, please acknowledge that in the next version (or
>> this one if it did change), asking me to ack it again.

The samples fail to build on rv32, they're rv64-specific.  I think 
something like this should do it

diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c
index 5c319db48af2..3707c447a42d 100644
--- a/samples/ftrace/ftrace-direct-too.c
+++ b/samples/ftrace/ftrace-direct-too.c
@@ -24,19 +24,19 @@ asm (
 "       .type           my_tramp, @function\n"
 "       .globl          my_tramp\n"
 "   my_tramp:\n"
-"       addi	sp,sp,-40\n"
-"       sd	a0,0(sp)\n"
-"       sd	a1,8(sp)\n"
-"       sd	a2,16(sp)\n"
-"       sd	t0,24(sp)\n"
-"       sd	ra,32(sp)\n"
+"       addi	sp,sp,-8*SZREG\n"
+"       REG_S 	a0,0*SZREG(sp)\n"
+"       REG_S 	a1,1*SZREG(sp)\n"
+"       REG_S 	a2,2*SZREG(sp)\n"
+"       REG_S 	t0,3*SZREG(sp)\n"
+"       REG_S 	ra,4*SZREG(sp)\n"
 "       call	my_direct_func\n"
-"       ld	a0,0(sp)\n"
-"       ld	a1,8(sp)\n"
-"       ld	a2,16(sp)\n"
-"       ld	t0,24(sp)\n"
-"       ld	ra,32(sp)\n"
-"       addi	sp,sp,40\n"
+"       REG_L 	a0,0*SZREG(sp)\n"
+"       REG_L 	a1,1*SZREG(sp)\n"
+"       REG_L 	a2,2*SZREG(sp)\n"
+"       REG_L 	t0,3*SZREG(sp)\n"
+"       REG_L 	ra,4*SZREG(sp)\n"
+"       addi	sp,sp,8*SZREG\n"
 "       jr	t0\n"
 "       .size           my_tramp, .-my_tramp\n"
 "       .popsection\n"

probably for all of them.  Do you mind sending a v11 with that fixed?  

>>
>> -- Steve
>
>
>
> -- 
> Thanks,
> Song

^ permalink raw reply related	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2023-06-25 20:32 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-11  9:32 [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
2023-05-11  9:32 ` [PATCH V10 1/4] riscv: select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY Song Shuai
2023-05-11  9:32 ` [PATCH V10 2/4] riscv: ftrace: Add ftrace_graph_func Song Shuai
2023-05-11  9:32 ` [PATCH V10 3/4] riscv: ftrace: Add DYNAMIC_FTRACE_WITH_DIRECT_CALLS support Song Shuai
2023-05-11 19:42   ` Björn Töpel
2023-05-11  9:32 ` [PATCH V10 4/4] samples: ftrace: Add riscv support for SAMPLE_FTRACE_DIRECT[_MULTI] Song Shuai
2023-05-15 18:37   ` Steven Rostedt
2023-05-15 18:38     ` Steven Rostedt
2023-05-16  9:37     ` Song Shuai
2023-06-25 20:32       ` Palmer Dabbelt
2023-05-16  9:50 ` [PATCH V10 0/4] riscv: Optimize function trace Song Shuai
2023-05-17 11:09   ` Song Shuai

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).