[PATCH 00/17] powerpc/ftrace: refactor and add support for -fpatchable-function-entry

[PATCH 00/17] powerpc/ftrace: refactor and add support for -fpatchable-function-entry

[Naveen N Rao]

Since RFC (*):
- Patches 1 and 17 have been included in this series due to 
  dependencies. Both had been posted out separately.
- Patch 10 has a small change to not throw errors when checking 
  instruction sequence generated by older toolchains.

This has had more testing since and this looks good to me. Christophe 
mentioned that this results in a slowdown with ftrace [de-]activation on 
ppc32, but that isn't performance critical and we can address that 
separately.


(*) http://lore.kernel.org/cover.1686151854.git.naveen@kernel.org


- Naveen


Naveen N Rao (17):
  powerpc/ftrace: Fix dropping weak symbols with older toolchains
  powerpc/module: Remove unused .ftrace.tramp section
  powerpc64/ftrace: Move ELFv1 and -pg support code into a separate file
  powerpc/ftrace: Simplify function_graph support in ftrace.c
  powerpc/ftrace: Use FTRACE_REGS_ADDR to identify the correct ftrace
    trampoline
  powerpc/ftrace: Extend ftrace support for large kernels to ppc32
  powerpc/ftrace: Consolidate ftrace support into fewer files
  powerpc/ftrace: Refactor ftrace_modify_code()
  powerpc/ftrace: Stop re-purposing linker generated long branches for
    ftrace
  powerpc/ftrace: Add separate ftrace_init_nop() with additional
    validation
  powerpc/ftrace: Simplify ftrace_make_nop()
  powerpc/ftrace: Simplify ftrace_make_call()
  powerpc/ftrace: Simplify ftrace_modify_call()
  powerpc/ftrace: Replace use of ftrace_call_replace() with
    ftrace_create_branch_inst()
  powerpc/ftrace: Implement ftrace_replace_code()
  powerpc/ftrace: Add support for -fpatchable-function-entry
  powerpc/ftrace: Create a dummy stackframe to fix stack unwind

 arch/powerpc/Kconfig                          |  14 +-
 arch/powerpc/Makefile                         |   5 +
 arch/powerpc/include/asm/ftrace.h             |  24 +-
 arch/powerpc/include/asm/module.h             |   4 -
 arch/powerpc/include/asm/sections.h           |   2 +
 arch/powerpc/include/asm/vermagic.h           |   4 +-
 arch/powerpc/kernel/module_64.c               |   2 +-
 arch/powerpc/kernel/trace/Makefile            |  12 +-
 arch/powerpc/kernel/trace/ftrace.c            | 910 +++++-------------
 arch/powerpc/kernel/trace/ftrace_64_pg.S      |  67 --
 arch/powerpc/kernel/trace/ftrace_64_pg.c      | 846 ++++++++++++++++
 .../{ftrace_low.S => ftrace_64_pg_entry.S}    |  64 +-
 .../{ftrace_mprofile.S => ftrace_entry.S}     |  78 +-
 arch/powerpc/kernel/vmlinux.lds.S             |   4 -
 .../gcc-check-fpatchable-function-entry.sh    |  26 +
 15 files changed, 1288 insertions(+), 774 deletions(-)
 delete mode 100644 arch/powerpc/kernel/trace/ftrace_64_pg.S
 create mode 100644 arch/powerpc/kernel/trace/ftrace_64_pg.c
 rename arch/powerpc/kernel/trace/{ftrace_low.S => ftrace_64_pg_entry.S} (54%)
 rename arch/powerpc/kernel/trace/{ftrace_mprofile.S => ftrace_entry.S} (79%)
 create mode 100755 arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh


base-commit: 12ffddc6444780aec83fa5086673ec005c0bace4
-- 
2.40.1

[PATCH 01/17] powerpc/ftrace: Fix dropping weak symbols with older toolchains

[Naveen N Rao]

The minimum level of gcc supported for building the kernel is v5.1.
v5.x releases of gcc emitted a three instruction sequence for
-mprofile-kernel:
	mflr	r0
	std	r0, 16(r1)
	bl	_mcount

It is only with the v6.x releases that gcc started emitting the two
instruction sequence for -mprofile-kernel, omitting the second store
instruction.

With the older three instruction sequence, the actual ftrace location
can be the 5th instruction into a function. Update the allowed offset
for ftrace location from 12 to 16 to accommodate the same.

Cc: stable@vger.kernel.org
Fixes: 7af82ff90a2b06 ("powerpc/ftrace: Ignore weak functions")
Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/include/asm/ftrace.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index 91c049d51d0e10..2edc6269b1a357 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -12,7 +12,7 @@
 
 /* Ignore unused weak functions which will have larger offsets */
 #ifdef CONFIG_MPROFILE_KERNEL
-#define FTRACE_MCOUNT_MAX_OFFSET	12
+#define FTRACE_MCOUNT_MAX_OFFSET	16
 #elif defined(CONFIG_PPC32)
 #define FTRACE_MCOUNT_MAX_OFFSET	8
 #endif
-- 
2.40.1

[PATCH 02/17] powerpc/module: Remove unused .ftrace.tramp section

[Naveen N Rao]

.ftrace.tramp section is not used for any purpose. This code was added
all the way back in the original commit introducing support for dynamic
ftrace on ppc64 modules. Remove it.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/include/asm/module.h | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/arch/powerpc/include/asm/module.h b/arch/powerpc/include/asm/module.h
index ac53606c259430..a8e2e8339fb7f4 100644
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -75,10 +75,6 @@ struct mod_arch_specific {
 #endif
 
 #ifdef CONFIG_DYNAMIC_FTRACE
-#    ifdef MODULE
-	asm(".section .ftrace.tramp,\"ax\",@nobits; .align 3; .previous");
-#    endif	/* MODULE */
-
 int module_trampoline_target(struct module *mod, unsigned long trampoline,
 			     unsigned long *target);
 int module_finalize_ftrace(struct module *mod, const Elf_Shdr *sechdrs);
-- 
2.40.1

[PATCH 03/17] powerpc64/ftrace: Move ELFv1 and -pg support code into a separate file

[Naveen N Rao]

ELFv1 support is deprecated and on the way out. Pre -mprofile-kernel
ftrace support (-pg only) is very limited and is retained primarily for
clang builds. It won't be necessary once clang lands support for
-fpatchable-function-entry.

Copy the existing ftrace code supporting these into ftrace_pg.c.
ftrace.c can then be refactored and enhanced with a focus on ppc32 and
ppc64 ELFv2.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/Makefile    |  13 +-
 arch/powerpc/kernel/trace/ftrace.c    |  10 -
 arch/powerpc/kernel/trace/ftrace_pg.c | 846 ++++++++++++++++++++++++++
 3 files changed, 855 insertions(+), 14 deletions(-)
 create mode 100644 arch/powerpc/kernel/trace/ftrace_pg.c

diff --git a/arch/powerpc/kernel/trace/Makefile b/arch/powerpc/kernel/trace/Makefile
index b16a9f9c0b35f2..342a2d1ae86cd0 100644
--- a/arch/powerpc/kernel/trace/Makefile
+++ b/arch/powerpc/kernel/trace/Makefile
@@ -6,15 +6,16 @@
 ifdef CONFIG_FUNCTION_TRACER
 # do not trace tracer code
 CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_ftrace_pg.o = $(CC_FLAGS_FTRACE)
 endif
 
-obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o
+obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
 ifdef CONFIG_MPROFILE_KERNEL
-obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o
+obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
 else
-obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o
+obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_pg.o
 endif
-obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o ftrace.o
+obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o
 obj-$(CONFIG_TRACING)			+= trace_clock.o
 
 obj-$(CONFIG_PPC64)			+= $(obj64-y)
@@ -25,3 +26,7 @@ GCOV_PROFILE_ftrace.o := n
 KCOV_INSTRUMENT_ftrace.o := n
 KCSAN_SANITIZE_ftrace.o := n
 UBSAN_SANITIZE_ftrace.o := n
+GCOV_PROFILE_ftrace_pg.o := n
+KCOV_INSTRUMENT_ftrace_pg.o := n
+KCSAN_SANITIZE_ftrace_pg.o := n
+UBSAN_SANITIZE_ftrace_pg.o := n
diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index a47f303734233b..81a121b56c4d7f 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -864,13 +864,3 @@ unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
 }
 #endif
 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
-
-#ifdef CONFIG_PPC64_ELF_ABI_V1
-char *arch_ftrace_match_adjust(char *str, const char *search)
-{
-	if (str[0] == '.' && search[0] != '.')
-		return str + 1;
-	else
-		return str;
-}
-#endif /* CONFIG_PPC64_ELF_ABI_V1 */
diff --git a/arch/powerpc/kernel/trace/ftrace_pg.c b/arch/powerpc/kernel/trace/ftrace_pg.c
new file mode 100644
index 00000000000000..7b85c3b460a3c0
--- /dev/null
+++ b/arch/powerpc/kernel/trace/ftrace_pg.c
@@ -0,0 +1,846 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Code for replacing ftrace calls with jumps.
+ *
+ * Copyright (C) 2007-2008 Steven Rostedt <srostedt@redhat.com>
+ *
+ * Thanks goes out to P.A. Semi, Inc for supplying me with a PPC64 box.
+ *
+ * Added function graph tracer code, taken from x86 that was written
+ * by Frederic Weisbecker, and ported to PPC by Steven Rostedt.
+ *
+ */
+
+#define pr_fmt(fmt) "ftrace-powerpc: " fmt
+
+#include <linux/spinlock.h>
+#include <linux/hardirq.h>
+#include <linux/uaccess.h>
+#include <linux/module.h>
+#include <linux/ftrace.h>
+#include <linux/percpu.h>
+#include <linux/init.h>
+#include <linux/list.h>
+
+#include <asm/cacheflush.h>
+#include <asm/code-patching.h>
+#include <asm/ftrace.h>
+#include <asm/syscall.h>
+#include <asm/inst.h>
+
+/*
+ * We generally only have a single long_branch tramp and at most 2 or 3 plt
+ * tramps generated. But, we don't use the plt tramps currently. We also allot
+ * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
+ * tramps in total. Set aside 8 just to be sure.
+ */
+#define	NUM_FTRACE_TRAMPS	8
+static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
+
+static ppc_inst_t
+ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
+{
+	ppc_inst_t op;
+
+	addr = ppc_function_entry((void *)addr);
+
+	/* if (link) set op to 'bl' else 'b' */
+	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
+
+	return op;
+}
+
+static inline int
+ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
+{
+	ppc_inst_t replaced;
+
+	/*
+	 * Note:
+	 * We are paranoid about modifying text, as if a bug was to happen, it
+	 * could cause us to read or write to someplace that could cause harm.
+	 * Carefully read and modify the code with probe_kernel_*(), and make
+	 * sure what we read is what we expected it to be before modifying it.
+	 */
+
+	/* read the text we want to modify */
+	if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
+		return -EFAULT;
+
+	/* Make sure it is what we expect it to be */
+	if (!ppc_inst_equal(replaced, old)) {
+		pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
+		       ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
+		return -EINVAL;
+	}
+
+	/* replace the text with the new text */
+	return patch_instruction((u32 *)ip, new);
+}
+
+/*
+ * Helper functions that are the same for both PPC64 and PPC32.
+ */
+static int test_24bit_addr(unsigned long ip, unsigned long addr)
+{
+	addr = ppc_function_entry((void *)addr);
+
+	return is_offset_in_branch_range(addr - ip);
+}
+
+static int is_bl_op(ppc_inst_t op)
+{
+	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
+}
+
+static int is_b_op(ppc_inst_t op)
+{
+	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
+}
+
+static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
+{
+	int offset;
+
+	offset = PPC_LI(ppc_inst_val(op));
+	/* make it signed */
+	if (offset & 0x02000000)
+		offset |= 0xfe000000;
+
+	return ip + (long)offset;
+}
+
+#ifdef CONFIG_MODULES
+static int
+__ftrace_make_nop(struct module *mod,
+		  struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long entry, ptr, tramp;
+	unsigned long ip = rec->ip;
+	ppc_inst_t op, pop;
+
+	/* read where this goes */
+	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+		pr_err("Fetching opcode failed.\n");
+		return -EFAULT;
+	}
+
+	/* Make sure that this is still a 24bit jump */
+	if (!is_bl_op(op)) {
+		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+		return -EINVAL;
+	}
+
+	/* lets find where the pointer goes */
+	tramp = find_bl_target(ip, op);
+
+	pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+	if (module_trampoline_target(mod, tramp, &ptr)) {
+		pr_err("Failed to get trampoline target\n");
+		return -EFAULT;
+	}
+
+	pr_devel("trampoline target %lx", ptr);
+
+	entry = ppc_global_function_entry((void *)addr);
+	/* This should match what was called */
+	if (ptr != entry) {
+		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+		return -EINVAL;
+	}
+
+	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
+		if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
+			pr_err("Fetching instruction at %lx failed.\n", ip - 4);
+			return -EFAULT;
+		}
+
+		/* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
+		if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
+		    !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
+			pr_err("Unexpected instruction %08lx around bl _mcount\n",
+			       ppc_inst_as_ulong(op));
+			return -EINVAL;
+		}
+	} else if (IS_ENABLED(CONFIG_PPC64)) {
+		/*
+		 * Check what is in the next instruction. We can see ld r2,40(r1), but
+		 * on first pass after boot we will see mflr r0.
+		 */
+		if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
+			pr_err("Fetching op failed.\n");
+			return -EFAULT;
+		}
+
+		if (!ppc_inst_equal(op,  ppc_inst(PPC_INST_LD_TOC))) {
+			pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
+			       ppc_inst_as_ulong(op));
+			return -EINVAL;
+		}
+	}
+
+	/*
+	 * When using -mprofile-kernel or PPC32 there is no load to jump over.
+	 *
+	 * Otherwise our original call site looks like:
+	 *
+	 * bl <tramp>
+	 * ld r2,XX(r1)
+	 *
+	 * Milton Miller pointed out that we can not simply nop the branch.
+	 * If a task was preempted when calling a trace function, the nops
+	 * will remove the way to restore the TOC in r2 and the r2 TOC will
+	 * get corrupted.
+	 *
+	 * Use a b +8 to jump over the load.
+	 */
+	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
+		pop = ppc_inst(PPC_RAW_NOP());
+	else
+		pop = ppc_inst(PPC_RAW_BRANCH(8));	/* b +8 */
+
+	if (patch_instruction((u32 *)ip, pop)) {
+		pr_err("Patching NOP failed.\n");
+		return -EPERM;
+	}
+
+	return 0;
+}
+#else
+static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
+{
+	return 0;
+}
+#endif /* CONFIG_MODULES */
+
+static unsigned long find_ftrace_tramp(unsigned long ip)
+{
+	int i;
+
+	/*
+	 * We have the compiler generated long_branch tramps at the end
+	 * and we prefer those
+	 */
+	for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
+		if (!ftrace_tramps[i])
+			continue;
+		else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
+			return ftrace_tramps[i];
+
+	return 0;
+}
+
+static int add_ftrace_tramp(unsigned long tramp)
+{
+	int i;
+
+	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+		if (!ftrace_tramps[i]) {
+			ftrace_tramps[i] = tramp;
+			return 0;
+		}
+
+	return -1;
+}
+
+/*
+ * If this is a compiler generated long_branch trampoline (essentially, a
+ * trampoline that has a branch to _mcount()), we re-write the branch to
+ * instead go to ftrace_[regs_]caller() and note down the location of this
+ * trampoline.
+ */
+static int setup_mcount_compiler_tramp(unsigned long tramp)
+{
+	int i;
+	ppc_inst_t op;
+	unsigned long ptr;
+
+	/* Is this a known long jump tramp? */
+	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+		if (ftrace_tramps[i] == tramp)
+			return 0;
+
+	/* New trampoline -- read where this goes */
+	if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
+		pr_debug("Fetching opcode failed.\n");
+		return -1;
+	}
+
+	/* Is this a 24 bit branch? */
+	if (!is_b_op(op)) {
+		pr_debug("Trampoline is not a long branch tramp.\n");
+		return -1;
+	}
+
+	/* lets find where the pointer goes */
+	ptr = find_bl_target(tramp, op);
+
+	if (ptr != ppc_global_function_entry((void *)_mcount)) {
+		pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
+		return -1;
+	}
+
+	/* Let's re-write the tramp to go to ftrace_[regs_]caller */
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+		ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
+	else
+		ptr = ppc_global_function_entry((void *)ftrace_caller);
+
+	if (patch_branch((u32 *)tramp, ptr, 0)) {
+		pr_debug("REL24 out of range!\n");
+		return -1;
+	}
+
+	if (add_ftrace_tramp(tramp)) {
+		pr_debug("No tramp locations left\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long tramp, ip = rec->ip;
+	ppc_inst_t op;
+
+	/* Read where this goes */
+	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+		pr_err("Fetching opcode failed.\n");
+		return -EFAULT;
+	}
+
+	/* Make sure that this is still a 24bit jump */
+	if (!is_bl_op(op)) {
+		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+		return -EINVAL;
+	}
+
+	/* Let's find where the pointer goes */
+	tramp = find_bl_target(ip, op);
+
+	pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+	if (setup_mcount_compiler_tramp(tramp)) {
+		/* Are other trampolines reachable? */
+		if (!find_ftrace_tramp(ip)) {
+			pr_err("No ftrace trampolines reachable from %ps\n",
+					(void *)ip);
+			return -EINVAL;
+		}
+	}
+
+	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
+		pr_err("Patching NOP failed.\n");
+		return -EPERM;
+	}
+
+	return 0;
+}
+
+int ftrace_make_nop(struct module *mod,
+		    struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long ip = rec->ip;
+	ppc_inst_t old, new;
+
+	/*
+	 * If the calling address is more that 24 bits away,
+	 * then we had to use a trampoline to make the call.
+	 * Otherwise just update the call site.
+	 */
+	if (test_24bit_addr(ip, addr)) {
+		/* within range */
+		old = ftrace_call_replace(ip, addr, 1);
+		new = ppc_inst(PPC_RAW_NOP());
+		return ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip)) {
+		return __ftrace_make_nop_kernel(rec, addr);
+	} else if (!IS_ENABLED(CONFIG_MODULES)) {
+		return -EINVAL;
+	}
+
+	/*
+	 * Out of range jumps are called from modules.
+	 * We should either already have a pointer to the module
+	 * or it has been passed in.
+	 */
+	if (!rec->arch.mod) {
+		if (!mod) {
+			pr_err("No module loaded addr=%lx\n", addr);
+			return -EFAULT;
+		}
+		rec->arch.mod = mod;
+	} else if (mod) {
+		if (mod != rec->arch.mod) {
+			pr_err("Record mod %p not equal to passed in mod %p\n",
+			       rec->arch.mod, mod);
+			return -EINVAL;
+		}
+		/* nothing to do if mod == rec->arch.mod */
+	} else
+		mod = rec->arch.mod;
+
+	return __ftrace_make_nop(mod, rec, addr);
+}
+
+#ifdef CONFIG_MODULES
+/*
+ * Examine the existing instructions for __ftrace_make_call.
+ * They should effectively be a NOP, and follow formal constraints,
+ * depending on the ABI. Return false if they don't.
+ */
+static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
+{
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
+	else
+		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
+		       ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
+}
+
+static int
+__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+	ppc_inst_t op[2];
+	void *ip = (void *)rec->ip;
+	unsigned long entry, ptr, tramp;
+	struct module *mod = rec->arch.mod;
+
+	/* read where this goes */
+	if (copy_inst_from_kernel_nofault(op, ip))
+		return -EFAULT;
+
+	if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
+	    copy_inst_from_kernel_nofault(op + 1, ip + 4))
+		return -EFAULT;
+
+	if (!expected_nop_sequence(ip, op[0], op[1])) {
+		pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
+		       ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
+		return -EINVAL;
+	}
+
+	/* If we never set up ftrace trampoline(s), then bail */
+	if (!mod->arch.tramp ||
+	    (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
+		pr_err("No ftrace trampoline\n");
+		return -EINVAL;
+	}
+
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
+		tramp = mod->arch.tramp_regs;
+	else
+		tramp = mod->arch.tramp;
+
+	if (module_trampoline_target(mod, tramp, &ptr)) {
+		pr_err("Failed to get trampoline target\n");
+		return -EFAULT;
+	}
+
+	pr_devel("trampoline target %lx", ptr);
+
+	entry = ppc_global_function_entry((void *)addr);
+	/* This should match what was called */
+	if (ptr != entry) {
+		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+		return -EINVAL;
+	}
+
+	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
+		pr_err("REL24 out of range!\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+#else
+static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+	return 0;
+}
+#endif /* CONFIG_MODULES */
+
+static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
+{
+	ppc_inst_t op;
+	void *ip = (void *)rec->ip;
+	unsigned long tramp, entry, ptr;
+
+	/* Make sure we're being asked to patch branch to a known ftrace addr */
+	entry = ppc_global_function_entry((void *)ftrace_caller);
+	ptr = ppc_global_function_entry((void *)addr);
+
+	if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+		entry = ppc_global_function_entry((void *)ftrace_regs_caller);
+
+	if (ptr != entry) {
+		pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
+		return -EINVAL;
+	}
+
+	/* Make sure we have a nop */
+	if (copy_inst_from_kernel_nofault(&op, ip)) {
+		pr_err("Unable to read ftrace location %p\n", ip);
+		return -EFAULT;
+	}
+
+	if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
+		pr_err("Unexpected call sequence at %p: %08lx\n",
+		       ip, ppc_inst_as_ulong(op));
+		return -EINVAL;
+	}
+
+	tramp = find_ftrace_tramp((unsigned long)ip);
+	if (!tramp) {
+		pr_err("No ftrace trampolines reachable from %ps\n", ip);
+		return -EINVAL;
+	}
+
+	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
+		pr_err("Error patching branch to ftrace tramp!\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long ip = rec->ip;
+	ppc_inst_t old, new;
+
+	/*
+	 * If the calling address is more that 24 bits away,
+	 * then we had to use a trampoline to make the call.
+	 * Otherwise just update the call site.
+	 */
+	if (test_24bit_addr(ip, addr)) {
+		/* within range */
+		old = ppc_inst(PPC_RAW_NOP());
+		new = ftrace_call_replace(ip, addr, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip)) {
+		return __ftrace_make_call_kernel(rec, addr);
+	} else if (!IS_ENABLED(CONFIG_MODULES)) {
+		/* We should not get here without modules */
+		return -EINVAL;
+	}
+
+	/*
+	 * Out of range jumps are called from modules.
+	 * Being that we are converting from nop, it had better
+	 * already have a module defined.
+	 */
+	if (!rec->arch.mod) {
+		pr_err("No module loaded\n");
+		return -EINVAL;
+	}
+
+	return __ftrace_make_call(rec, addr);
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#ifdef CONFIG_MODULES
+static int
+__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+					unsigned long addr)
+{
+	ppc_inst_t op;
+	unsigned long ip = rec->ip;
+	unsigned long entry, ptr, tramp;
+	struct module *mod = rec->arch.mod;
+
+	/* If we never set up ftrace trampolines, then bail */
+	if (!mod->arch.tramp || !mod->arch.tramp_regs) {
+		pr_err("No ftrace trampoline\n");
+		return -EINVAL;
+	}
+
+	/* read where this goes */
+	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+		pr_err("Fetching opcode failed.\n");
+		return -EFAULT;
+	}
+
+	/* Make sure that this is still a 24bit jump */
+	if (!is_bl_op(op)) {
+		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+		return -EINVAL;
+	}
+
+	/* lets find where the pointer goes */
+	tramp = find_bl_target(ip, op);
+	entry = ppc_global_function_entry((void *)old_addr);
+
+	pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+	if (tramp != entry) {
+		/* old_addr is not within range, so we must have used a trampoline */
+		if (module_trampoline_target(mod, tramp, &ptr)) {
+			pr_err("Failed to get trampoline target\n");
+			return -EFAULT;
+		}
+
+		pr_devel("trampoline target %lx", ptr);
+
+		/* This should match what was called */
+		if (ptr != entry) {
+			pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+			return -EINVAL;
+		}
+	}
+
+	/* The new target may be within range */
+	if (test_24bit_addr(ip, addr)) {
+		/* within range */
+		if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
+			pr_err("REL24 out of range!\n");
+			return -EINVAL;
+		}
+
+		return 0;
+	}
+
+	if (rec->flags & FTRACE_FL_REGS)
+		tramp = mod->arch.tramp_regs;
+	else
+		tramp = mod->arch.tramp;
+
+	if (module_trampoline_target(mod, tramp, &ptr)) {
+		pr_err("Failed to get trampoline target\n");
+		return -EFAULT;
+	}
+
+	pr_devel("trampoline target %lx", ptr);
+
+	entry = ppc_global_function_entry((void *)addr);
+	/* This should match what was called */
+	if (ptr != entry) {
+		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+		return -EINVAL;
+	}
+
+	if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
+		pr_err("REL24 out of range!\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+#else
+static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
+{
+	return 0;
+}
+#endif
+
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+			unsigned long addr)
+{
+	unsigned long ip = rec->ip;
+	ppc_inst_t old, new;
+
+	/*
+	 * If the calling address is more that 24 bits away,
+	 * then we had to use a trampoline to make the call.
+	 * Otherwise just update the call site.
+	 */
+	if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
+		/* within range */
+		old = ftrace_call_replace(ip, old_addr, 1);
+		new = ftrace_call_replace(ip, addr, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip)) {
+		/*
+		 * We always patch out of range locations to go to the regs
+		 * variant, so there is nothing to do here
+		 */
+		return 0;
+	} else if (!IS_ENABLED(CONFIG_MODULES)) {
+		/* We should not get here without modules */
+		return -EINVAL;
+	}
+
+	/*
+	 * Out of range jumps are called from modules.
+	 */
+	if (!rec->arch.mod) {
+		pr_err("No module loaded\n");
+		return -EINVAL;
+	}
+
+	return __ftrace_modify_call(rec, old_addr, addr);
+}
+#endif
+
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+	unsigned long ip = (unsigned long)(&ftrace_call);
+	ppc_inst_t old, new;
+	int ret;
+
+	old = ppc_inst_read((u32 *)&ftrace_call);
+	new = ftrace_call_replace(ip, (unsigned long)func, 1);
+	ret = ftrace_modify_code(ip, old, new);
+
+	/* Also update the regs callback function */
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
+		ip = (unsigned long)(&ftrace_regs_call);
+		old = ppc_inst_read((u32 *)&ftrace_regs_call);
+		new = ftrace_call_replace(ip, (unsigned long)func, 1);
+		ret = ftrace_modify_code(ip, old, new);
+	}
+
+	return ret;
+}
+
+/*
+ * Use the default ftrace_modify_all_code, but without
+ * stop_machine().
+ */
+void arch_ftrace_update_code(int command)
+{
+	ftrace_modify_all_code(command);
+}
+
+#ifdef CONFIG_PPC64
+#define PACATOC offsetof(struct paca_struct, kernel_toc)
+
+extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
+
+void ftrace_free_init_tramp(void)
+{
+	int i;
+
+	for (i = 0; i < NUM_FTRACE_TRAMPS && ftrace_tramps[i]; i++)
+		if (ftrace_tramps[i] == (unsigned long)ftrace_tramp_init) {
+			ftrace_tramps[i] = 0;
+			return;
+		}
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
+	int i;
+	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
+	u32 stub_insns[] = {
+		PPC_RAW_LD(_R12, _R13, PACATOC),
+		PPC_RAW_ADDIS(_R12, _R12, 0),
+		PPC_RAW_ADDI(_R12, _R12, 0),
+		PPC_RAW_MTCTR(_R12),
+		PPC_RAW_BCTR()
+	};
+	unsigned long addr;
+	long reladdr;
+
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+		addr = ppc_global_function_entry((void *)ftrace_regs_caller);
+	else
+		addr = ppc_global_function_entry((void *)ftrace_caller);
+
+	reladdr = addr - kernel_toc_addr();
+
+	if (reladdr >= SZ_2G || reladdr < -(long)SZ_2G) {
+		pr_err("Address of %ps out of range of kernel_toc.\n",
+				(void *)addr);
+		return -1;
+	}
+
+	for (i = 0; i < 2; i++) {
+		memcpy(tramp[i], stub_insns, sizeof(stub_insns));
+		tramp[i][1] |= PPC_HA(reladdr);
+		tramp[i][2] |= PPC_LO(reladdr);
+		add_ftrace_tramp((unsigned long)tramp[i]);
+	}
+
+	return 0;
+}
+#endif
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+
+extern void ftrace_graph_call(void);
+extern void ftrace_graph_stub(void);
+
+static int ftrace_modify_ftrace_graph_caller(bool enable)
+{
+	unsigned long ip = (unsigned long)(&ftrace_graph_call);
+	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
+	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
+	ppc_inst_t old, new;
+
+	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
+		return 0;
+
+	old = ftrace_call_replace(ip, enable ? stub : addr, 0);
+	new = ftrace_call_replace(ip, enable ? addr : stub, 0);
+
+	return ftrace_modify_code(ip, old, new);
+}
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+	return ftrace_modify_ftrace_graph_caller(true);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+	return ftrace_modify_ftrace_graph_caller(false);
+}
+
+/*
+ * Hook the return address and push it in the stack of return addrs
+ * in current thread info. Return the address we want to divert to.
+ */
+static unsigned long
+__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
+{
+	unsigned long return_hooker;
+	int bit;
+
+	if (unlikely(ftrace_graph_is_dead()))
+		goto out;
+
+	if (unlikely(atomic_read(&current->tracing_graph_pause)))
+		goto out;
+
+	bit = ftrace_test_recursion_trylock(ip, parent);
+	if (bit < 0)
+		goto out;
+
+	return_hooker = ppc_function_entry(return_to_handler);
+
+	if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
+		parent = return_hooker;
+
+	ftrace_test_recursion_unlock(bit);
+out:
+	return parent;
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+		       struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+	fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
+}
+#else
+unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
+				    unsigned long sp)
+{
+	return __prepare_ftrace_return(parent, ip, sp);
+}
+#endif
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
+#ifdef CONFIG_PPC64_ELF_ABI_V1
+char *arch_ftrace_match_adjust(char *str, const char *search)
+{
+	if (str[0] == '.' && search[0] != '.')
+		return str + 1;
+	else
+		return str;
+}
+#endif /* CONFIG_PPC64_ELF_ABI_V1 */
-- 
2.40.1

[PATCH 04/17] powerpc/ftrace: Simplify function_graph support in ftrace.c

[Naveen N Rao]

Since we now support DYNAMIC_FTRACE_WITH_ARGS across ppc32 and ppc64
ELFv2, we can simplify function_graph tracer support code in ftrace.c

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 64 ++++--------------------------
 1 file changed, 7 insertions(+), 57 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 81a121b56c4d7f..f117124c30325f 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -790,44 +790,10 @@ int __init ftrace_dyn_arch_init(void)
 #endif
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
-
-extern void ftrace_graph_call(void);
-extern void ftrace_graph_stub(void);
-
-static int ftrace_modify_ftrace_graph_caller(bool enable)
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+		       struct ftrace_ops *op, struct ftrace_regs *fregs)
 {
-	unsigned long ip = (unsigned long)(&ftrace_graph_call);
-	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
-	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
-	ppc_inst_t old, new;
-
-	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
-		return 0;
-
-	old = ftrace_call_replace(ip, enable ? stub : addr, 0);
-	new = ftrace_call_replace(ip, enable ? addr : stub, 0);
-
-	return ftrace_modify_code(ip, old, new);
-}
-
-int ftrace_enable_ftrace_graph_caller(void)
-{
-	return ftrace_modify_ftrace_graph_caller(true);
-}
-
-int ftrace_disable_ftrace_graph_caller(void)
-{
-	return ftrace_modify_ftrace_graph_caller(false);
-}
-
-/*
- * Hook the return address and push it in the stack of return addrs
- * in current thread info. Return the address we want to divert to.
- */
-static unsigned long
-__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
-{
-	unsigned long return_hooker;
+	unsigned long sp = fregs->regs.gpr[1];
 	int bit;
 
 	if (unlikely(ftrace_graph_is_dead()))
@@ -836,31 +802,15 @@ __prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp
 	if (unlikely(atomic_read(&current->tracing_graph_pause)))
 		goto out;
 
-	bit = ftrace_test_recursion_trylock(ip, parent);
+	bit = ftrace_test_recursion_trylock(ip, parent_ip);
 	if (bit < 0)
 		goto out;
 
-	return_hooker = ppc_function_entry(return_to_handler);
-
-	if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
-		parent = return_hooker;
+	if (!function_graph_enter(parent_ip, ip, 0, (unsigned long *)sp))
+		parent_ip = ppc_function_entry(return_to_handler);
 
 	ftrace_test_recursion_unlock(bit);
 out:
-	return parent;
+	fregs->regs.link = parent_ip;
 }
-
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
-void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
-		       struct ftrace_ops *op, struct ftrace_regs *fregs)
-{
-	fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
-}
-#else
-unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
-				    unsigned long sp)
-{
-	return __prepare_ftrace_return(parent, ip, sp);
-}
-#endif
 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
-- 
2.40.1

[PATCH 05/17] powerpc/ftrace: Use FTRACE_REGS_ADDR to identify the correct ftrace trampoline

[Naveen N Rao]

Instead of keying off DYNAMIC_FTRACE_WITH_REGS, use FTRACE_REGS_ADDR to
identify the proper ftrace trampoline address to use.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index f117124c30325f..5aa36272617a03 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -745,14 +745,9 @@ int __init ftrace_dyn_arch_init(void)
 	};
 #endif
 
-	unsigned long addr;
+	unsigned long addr = FTRACE_REGS_ADDR;
 	long reladdr;
 
-	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
-		addr = ppc_global_function_entry((void *)ftrace_regs_caller);
-	else
-		addr = ppc_global_function_entry((void *)ftrace_caller);
-
 	if (IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) {
 		for (i = 0; i < 2; i++) {
 			reladdr = addr - (unsigned long)tramp[i];
-- 
2.40.1

[PATCH 06/17] powerpc/ftrace: Extend ftrace support for large kernels to ppc32

[Naveen N Rao]

Commit 67361cf8071286 ("powerpc/ftrace: Handle large kernel configs")
added ftrace support for ppc64 kernel images with a text section larger
than 32MB. The approach itself isn't specific to ppc64, so extend the
same to also work on ppc32.

While at it, reduce the space reserved for the stub from 64 bytes to 32
bytes since the different stub variants are all less than 8
instructions.

To reduce use of #ifdef, a stub implementation is provided for
kernel_toc_address() and -SZ_2G is cast to 'long long' to prevent
errors on ppc32.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/include/asm/ftrace.h      | 10 +++++--
 arch/powerpc/include/asm/sections.h    |  2 ++
 arch/powerpc/kernel/trace/ftrace.c     | 39 ++++++++++++++------------
 arch/powerpc/kernel/trace/ftrace_low.S |  6 ++--
 arch/powerpc/kernel/vmlinux.lds.S      |  4 ---
 5 files changed, 32 insertions(+), 29 deletions(-)

diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index 2edc6269b1a357..702aaf2efa966c 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -124,15 +124,19 @@ static inline u8 this_cpu_get_ftrace_enabled(void)
 {
 	return get_paca()->ftrace_enabled;
 }
-
-void ftrace_free_init_tramp(void);
 #else /* CONFIG_PPC64 */
 static inline void this_cpu_disable_ftrace(void) { }
 static inline void this_cpu_enable_ftrace(void) { }
 static inline void this_cpu_set_ftrace_enabled(u8 ftrace_enabled) { }
 static inline u8 this_cpu_get_ftrace_enabled(void) { return 1; }
-static inline void ftrace_free_init_tramp(void) { }
 #endif /* CONFIG_PPC64 */
+
+#ifdef CONFIG_FUNCTION_TRACER
+extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
+void ftrace_free_init_tramp(void);
+#else
+static inline void ftrace_free_init_tramp(void) { }
+#endif
 #endif /* !__ASSEMBLY__ */
 
 #endif /* _ASM_POWERPC_FTRACE */
diff --git a/arch/powerpc/include/asm/sections.h b/arch/powerpc/include/asm/sections.h
index 4e1f548c8d373d..ea26665f82cfc8 100644
--- a/arch/powerpc/include/asm/sections.h
+++ b/arch/powerpc/include/asm/sections.h
@@ -74,6 +74,8 @@ static inline int overlaps_kernel_text(unsigned long start, unsigned long end)
 		(unsigned long)_stext < end;
 }
 
+#else
+static inline unsigned long kernel_toc_addr(void) { BUILD_BUG(); return -1UL; }
 #endif
 
 #endif /* __KERNEL__ */
diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 5aa36272617a03..913c7aa63d3fa3 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -707,11 +707,6 @@ void arch_ftrace_update_code(int command)
 	ftrace_modify_all_code(command);
 }
 
-#ifdef CONFIG_PPC64
-#define PACATOC offsetof(struct paca_struct, kernel_toc)
-
-extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
-
 void ftrace_free_init_tramp(void)
 {
 	int i;
@@ -725,28 +720,30 @@ void ftrace_free_init_tramp(void)
 
 int __init ftrace_dyn_arch_init(void)
 {
-	int i;
 	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
-#ifdef CONFIG_PPC_KERNEL_PCREL
+	unsigned long addr = FTRACE_REGS_ADDR;
+	long reladdr;
+	int i;
 	u32 stub_insns[] = {
+#ifdef CONFIG_PPC_KERNEL_PCREL
 		/* pla r12,addr */
 		PPC_PREFIX_MLS | __PPC_PRFX_R(1),
 		PPC_INST_PADDI | ___PPC_RT(_R12),
 		PPC_RAW_MTCTR(_R12),
 		PPC_RAW_BCTR()
-	};
-#else
-	u32 stub_insns[] = {
-		PPC_RAW_LD(_R12, _R13, PACATOC),
+#elif defined(CONFIG_PPC64)
+		PPC_RAW_LD(_R12, _R13, offsetof(struct paca_struct, kernel_toc)),
 		PPC_RAW_ADDIS(_R12, _R12, 0),
 		PPC_RAW_ADDI(_R12, _R12, 0),
 		PPC_RAW_MTCTR(_R12),
 		PPC_RAW_BCTR()
-	};
+#else
+		PPC_RAW_LIS(_R12, 0),
+		PPC_RAW_ADDI(_R12, _R12, 0),
+		PPC_RAW_MTCTR(_R12),
+		PPC_RAW_BCTR()
 #endif
-
-	unsigned long addr = FTRACE_REGS_ADDR;
-	long reladdr;
+	};
 
 	if (IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) {
 		for (i = 0; i < 2; i++) {
@@ -763,10 +760,10 @@ int __init ftrace_dyn_arch_init(void)
 			tramp[i][1] |= IMM_L(reladdr);
 			add_ftrace_tramp((unsigned long)tramp[i]);
 		}
-	} else {
+	} else if (IS_ENABLED(CONFIG_PPC64)) {
 		reladdr = addr - kernel_toc_addr();
 
-		if (reladdr >= (long)SZ_2G || reladdr < -(long)SZ_2G) {
+		if (reladdr >= (long)SZ_2G || reladdr < -(long long)SZ_2G) {
 			pr_err("Address of %ps out of range of kernel_toc.\n",
 				(void *)addr);
 			return -1;
@@ -778,11 +775,17 @@ int __init ftrace_dyn_arch_init(void)
 			tramp[i][2] |= PPC_LO(reladdr);
 			add_ftrace_tramp((unsigned long)tramp[i]);
 		}
+	} else {
+		for (i = 0; i < 2; i++) {
+			memcpy(tramp[i], stub_insns, sizeof(stub_insns));
+			tramp[i][0] |= PPC_HA(addr);
+			tramp[i][1] |= PPC_LO(addr);
+			add_ftrace_tramp((unsigned long)tramp[i]);
+		}
 	}
 
 	return 0;
 }
-#endif
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
diff --git a/arch/powerpc/kernel/trace/ftrace_low.S b/arch/powerpc/kernel/trace/ftrace_low.S
index 294d1e05958aae..2fc7dd0a5ae968 100644
--- a/arch/powerpc/kernel/trace/ftrace_low.S
+++ b/arch/powerpc/kernel/trace/ftrace_low.S
@@ -10,19 +10,17 @@
 #include <asm/ppc-opcode.h>
 #include <asm/export.h>
 
-#ifdef CONFIG_PPC64
 .pushsection ".tramp.ftrace.text","aw",@progbits;
 .globl ftrace_tramp_text
 ftrace_tramp_text:
-	.space 64
+	.space 32
 .popsection
 
 .pushsection ".tramp.ftrace.init","aw",@progbits;
 .globl ftrace_tramp_init
 ftrace_tramp_init:
-	.space 64
+	.space 32
 .popsection
-#endif
 
 _GLOBAL(mcount)
 _GLOBAL(_mcount)
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index 13614f0b269cf4..1c5970df323366 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -107,9 +107,7 @@ SECTIONS
 #endif
 		/* careful! __ftr_alt_* sections need to be close to .text */
 		*(.text.hot .text.hot.* TEXT_MAIN .text.fixup .text.unlikely .text.unlikely.* .fixup __ftr_alt_* .ref.text);
-#ifdef CONFIG_PPC64
 		*(.tramp.ftrace.text);
-#endif
 		NOINSTR_TEXT
 		SCHED_TEXT
 		LOCK_TEXT
@@ -276,9 +274,7 @@ SECTIONS
 		 */
 		. = ALIGN(PAGE_SIZE);
 		_einittext = .;
-#ifdef CONFIG_PPC64
 		*(.tramp.ftrace.init);
-#endif
 	} :text
 
 	/* .exit.text is discarded at runtime, not link time,
-- 
2.40.1

[PATCH 07/17] powerpc/ftrace: Consolidate ftrace support into fewer files

[Naveen N Rao]

ftrace_low.S has just the _mcount stub and return_to_handler(). Merge
this back into ftrace_mprofile.S and ftrace_64_pg.S to keep all ftrace
code together, and to allow those to evolve independently.

ftrace_mprofile.S is also not an entirely accurate name since this also
holds ppc32 code. This will be all the more incorrect once support for
-fpatchable-function-entry is added. Rename files here to more
accurately describe the code:
- ftrace_mprofile.S is renamed to ftrace_entry.S
- ftrace_pg.c is renamed to ftrace_64_pg.c
- ftrace_64_pg.S is rename to ftrace_64_pg_entry.S

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/Makefile            | 17 +++--
 arch/powerpc/kernel/trace/ftrace_64_pg.S      | 67 -------------------
 .../trace/{ftrace_pg.c => ftrace_64_pg.c}     |  0
 .../{ftrace_low.S => ftrace_64_pg_entry.S}    | 58 +++++++++++++++-
 .../{ftrace_mprofile.S => ftrace_entry.S}     | 65 ++++++++++++++++++
 5 files changed, 130 insertions(+), 77 deletions(-)
 delete mode 100644 arch/powerpc/kernel/trace/ftrace_64_pg.S
 rename arch/powerpc/kernel/trace/{ftrace_pg.c => ftrace_64_pg.c} (100%)
 rename arch/powerpc/kernel/trace/{ftrace_low.S => ftrace_64_pg_entry.S} (55%)
 rename arch/powerpc/kernel/trace/{ftrace_mprofile.S => ftrace_entry.S} (83%)

diff --git a/arch/powerpc/kernel/trace/Makefile b/arch/powerpc/kernel/trace/Makefile
index 342a2d1ae86cd0..125f4ca588b98a 100644
--- a/arch/powerpc/kernel/trace/Makefile
+++ b/arch/powerpc/kernel/trace/Makefile
@@ -6,16 +6,15 @@
 ifdef CONFIG_FUNCTION_TRACER
 # do not trace tracer code
 CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
-CFLAGS_REMOVE_ftrace_pg.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_ftrace_64_pg.o = $(CC_FLAGS_FTRACE)
 endif
 
-obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
+obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
 ifdef CONFIG_MPROFILE_KERNEL
-obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
+obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
 else
-obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_pg.o
+obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_64_pg_entry.o
 endif
-obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o
 obj-$(CONFIG_TRACING)			+= trace_clock.o
 
 obj-$(CONFIG_PPC64)			+= $(obj64-y)
@@ -26,7 +25,7 @@ GCOV_PROFILE_ftrace.o := n
 KCOV_INSTRUMENT_ftrace.o := n
 KCSAN_SANITIZE_ftrace.o := n
 UBSAN_SANITIZE_ftrace.o := n
-GCOV_PROFILE_ftrace_pg.o := n
-KCOV_INSTRUMENT_ftrace_pg.o := n
-KCSAN_SANITIZE_ftrace_pg.o := n
-UBSAN_SANITIZE_ftrace_pg.o := n
+GCOV_PROFILE_ftrace_64_pg.o := n
+KCOV_INSTRUMENT_ftrace_64_pg.o := n
+KCSAN_SANITIZE_ftrace_64_pg.o := n
+UBSAN_SANITIZE_ftrace_64_pg.o := n
diff --git a/arch/powerpc/kernel/trace/ftrace_64_pg.S b/arch/powerpc/kernel/trace/ftrace_64_pg.S
deleted file mode 100644
index 6708e24db0aba8..00000000000000
--- a/arch/powerpc/kernel/trace/ftrace_64_pg.S
+++ /dev/null
@@ -1,67 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Split from ftrace_64.S
- */
-
-#include <linux/magic.h>
-#include <asm/ppc_asm.h>
-#include <asm/asm-offsets.h>
-#include <asm/ftrace.h>
-#include <asm/ppc-opcode.h>
-#include <asm/export.h>
-
-_GLOBAL_TOC(ftrace_caller)
-	lbz	r3, PACA_FTRACE_ENABLED(r13)
-	cmpdi	r3, 0
-	beqlr
-
-	/* Taken from output of objdump from lib64/glibc */
-	mflr	r3
-	ld	r11, 0(r1)
-	stdu	r1, -112(r1)
-	std	r3, 128(r1)
-	ld	r4, 16(r11)
-	subi	r3, r3, MCOUNT_INSN_SIZE
-.globl ftrace_call
-ftrace_call:
-	bl	ftrace_stub
-	nop
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-.globl ftrace_graph_call
-ftrace_graph_call:
-	b	ftrace_graph_stub
-_GLOBAL(ftrace_graph_stub)
-#endif
-	ld	r0, 128(r1)
-	mtlr	r0
-	addi	r1, r1, 112
-
-_GLOBAL(ftrace_stub)
-	blr
-
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-_GLOBAL(ftrace_graph_caller)
-	addi	r5, r1, 112
-	/* load r4 with local address */
-	ld	r4, 128(r1)
-	subi	r4, r4, MCOUNT_INSN_SIZE
-
-	/* Grab the LR out of the caller stack frame */
-	ld	r11, 112(r1)
-	ld	r3, 16(r11)
-
-	bl	prepare_ftrace_return
-	nop
-
-	/*
-	 * prepare_ftrace_return gives us the address we divert to.
-	 * Change the LR in the callers stack frame to this.
-	 */
-	ld	r11, 112(r1)
-	std	r3, 16(r11)
-
-	ld	r0, 128(r1)
-	mtlr	r0
-	addi	r1, r1, 112
-	blr
-#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
diff --git a/arch/powerpc/kernel/trace/ftrace_pg.c b/arch/powerpc/kernel/trace/ftrace_64_pg.c
similarity index 100%
rename from arch/powerpc/kernel/trace/ftrace_pg.c
rename to arch/powerpc/kernel/trace/ftrace_64_pg.c
diff --git a/arch/powerpc/kernel/trace/ftrace_low.S b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
similarity index 55%
rename from arch/powerpc/kernel/trace/ftrace_low.S
rename to arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
index 2fc7dd0a5ae968..81dbaf70b1513a 100644
--- a/arch/powerpc/kernel/trace/ftrace_low.S
+++ b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
 /*
- * Split from entry_64.S
+ * Split from ftrace_64.S
  */
 
 #include <linux/magic.h>
@@ -10,6 +10,62 @@
 #include <asm/ppc-opcode.h>
 #include <asm/export.h>
 
+_GLOBAL_TOC(ftrace_caller)
+	lbz	r3, PACA_FTRACE_ENABLED(r13)
+	cmpdi	r3, 0
+	beqlr
+
+	/* Taken from output of objdump from lib64/glibc */
+	mflr	r3
+	ld	r11, 0(r1)
+	stdu	r1, -112(r1)
+	std	r3, 128(r1)
+	ld	r4, 16(r11)
+	subi	r3, r3, MCOUNT_INSN_SIZE
+.globl ftrace_call
+ftrace_call:
+	bl	ftrace_stub
+	nop
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+.globl ftrace_graph_call
+ftrace_graph_call:
+	b	ftrace_graph_stub
+_GLOBAL(ftrace_graph_stub)
+#endif
+	ld	r0, 128(r1)
+	mtlr	r0
+	addi	r1, r1, 112
+
+_GLOBAL(ftrace_stub)
+	blr
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+_GLOBAL(ftrace_graph_caller)
+	addi	r5, r1, 112
+	/* load r4 with local address */
+	ld	r4, 128(r1)
+	subi	r4, r4, MCOUNT_INSN_SIZE
+
+	/* Grab the LR out of the caller stack frame */
+	ld	r11, 112(r1)
+	ld	r3, 16(r11)
+
+	bl	prepare_ftrace_return
+	nop
+
+	/*
+	 * prepare_ftrace_return gives us the address we divert to.
+	 * Change the LR in the callers stack frame to this.
+	 */
+	ld	r11, 112(r1)
+	std	r3, 16(r11)
+
+	ld	r0, 128(r1)
+	mtlr	r0
+	addi	r1, r1, 112
+	blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
 .pushsection ".tramp.ftrace.text","aw",@progbits;
 .globl ftrace_tramp_text
 ftrace_tramp_text:
diff --git a/arch/powerpc/kernel/trace/ftrace_mprofile.S b/arch/powerpc/kernel/trace/ftrace_entry.S
similarity index 83%
rename from arch/powerpc/kernel/trace/ftrace_mprofile.S
rename to arch/powerpc/kernel/trace/ftrace_entry.S
index ffb1db38684998..e8339706e735b1 100644
--- a/arch/powerpc/kernel/trace/ftrace_mprofile.S
+++ b/arch/powerpc/kernel/trace/ftrace_entry.S
@@ -249,3 +249,68 @@ livepatch_handler:
 	/* Return to original caller of live patched function */
 	blr
 #endif /* CONFIG_LIVEPATCH */
+
+_GLOBAL(mcount)
+_GLOBAL(_mcount)
+EXPORT_SYMBOL(_mcount)
+	mflr	r12
+	mtctr	r12
+	mtlr	r0
+	bctr
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+_GLOBAL(return_to_handler)
+	/* need to save return values */
+#ifdef CONFIG_PPC64
+	std	r4,  -32(r1)
+	std	r3,  -24(r1)
+	/* save TOC */
+	std	r2,  -16(r1)
+	std	r31, -8(r1)
+	mr	r31, r1
+	stdu	r1, -112(r1)
+
+	/*
+	 * We might be called from a module.
+	 * Switch to our TOC to run inside the core kernel.
+	 */
+	LOAD_PACA_TOC()
+#else
+	stwu	r1, -16(r1)
+	stw	r3, 8(r1)
+	stw	r4, 12(r1)
+#endif
+
+	bl	ftrace_return_to_handler
+	nop
+
+	/* return value has real return address */
+	mtlr	r3
+
+#ifdef CONFIG_PPC64
+	ld	r1, 0(r1)
+	ld	r4,  -32(r1)
+	ld	r3,  -24(r1)
+	ld	r2,  -16(r1)
+	ld	r31, -8(r1)
+#else
+	lwz	r3, 8(r1)
+	lwz	r4, 12(r1)
+	addi	r1, r1, 16
+#endif
+
+	/* Jump back to real return address */
+	blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
+.pushsection ".tramp.ftrace.text","aw",@progbits;
+.globl ftrace_tramp_text
+ftrace_tramp_text:
+	.space 32
+.popsection
+
+.pushsection ".tramp.ftrace.init","aw",@progbits;
+.globl ftrace_tramp_init
+ftrace_tramp_init:
+	.space 32
+.popsection
-- 
2.40.1

[PATCH 08/17] powerpc/ftrace: Refactor ftrace_modify_code()

[Naveen N Rao]

Split up ftrace_modify_code() into a few helpers for future use. Also
update error messages accordingly.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 51 +++++++++++++++++-------------
 1 file changed, 29 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 913c7aa63d3fa3..ef4e49c2c37781 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -50,32 +50,39 @@ ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
 	return op;
 }
 
-static inline int
-ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
+static inline int ftrace_read_inst(unsigned long ip, ppc_inst_t *op)
 {
-	ppc_inst_t replaced;
-
-	/*
-	 * Note:
-	 * We are paranoid about modifying text, as if a bug was to happen, it
-	 * could cause us to read or write to someplace that could cause harm.
-	 * Carefully read and modify the code with probe_kernel_*(), and make
-	 * sure what we read is what we expected it to be before modifying it.
-	 */
-
-	/* read the text we want to modify */
-	if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
+	if (copy_inst_from_kernel_nofault(op, (void *)ip)) {
+		pr_err("0x%lx: fetching instruction failed\n", ip);
 		return -EFAULT;
-
-	/* Make sure it is what we expect it to be */
-	if (!ppc_inst_equal(replaced, old)) {
-		pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
-		       ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
-		return -EINVAL;
 	}
 
-	/* replace the text with the new text */
-	return patch_instruction((u32 *)ip, new);
+	return 0;
+}
+
+static inline int ftrace_validate_inst(unsigned long ip, ppc_inst_t inst)
+{
+	ppc_inst_t op;
+	int ret;
+
+	ret = ftrace_read_inst(ip, &op);
+	if (!ret && !ppc_inst_equal(op, inst)) {
+		pr_err("0x%lx: expected (%08lx) != found (%08lx)\n",
+		       ip, ppc_inst_as_ulong(inst), ppc_inst_as_ulong(op));
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
+static inline int ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
+{
+	int ret = ftrace_validate_inst(ip, old);
+
+	if (!ret)
+		ret = patch_instruction((u32 *)ip, new);
+
+	return ret;
 }
 
 /*
-- 
2.40.1

[PATCH 09/17] powerpc/ftrace: Stop re-purposing linker generated long branches for ftrace

[Naveen N Rao]

Commit 67361cf8071286 ("powerpc/ftrace: Handle large kernel configs")
added ftrace support for ppc64 kernel images with a text section larger
than 32MB. The patch did two things:
1. Add stubs at the end of .text to branch into ftrace_[regs_]caller for
   functions that were out of branch range.
2. Re-purpose linker-generated long branches to _mcount to instead branch
   to ftrace_[regs_]caller.

Before that, we only supported kernel .text up to ~32MB. With the above,
we now support up to ~96MB:
- The first 32MB of kernel text can branch directly into
  ftrace_[regs_]caller since that symbol is usually at the beginning.
- The modified long_branch from (2) above is used by the next 32MB of
  kernel text.
- The next 32MB of kernel text can use the stub at the end of text to
  branch back to ftrace_[regs_]caller.

While re-purposing the long branch works in practice, it still restricts
ftrace to kernel text up to ~96MB. The stub at the end of kernel text
from (1) already enables us to extend ftrace support for kernel text
up to 64MB, which fulfils the original requirement. Further, once we
switch to -fpatchable-function-entry, there will not be a long branch
that we can use.

Stop re-purposing the linker-generated long branches for ftrace to
simplify the code. If there are good reasons to support ftrace on
kernels beyond 64MB, we can consider adding support by using
-fpatchable-function-entry.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 110 +++++------------------------
 1 file changed, 17 insertions(+), 93 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index ef4e49c2c37781..278bf8e52b6e89 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -28,13 +28,7 @@
 #include <asm/syscall.h>
 #include <asm/inst.h>
 
-/*
- * We generally only have a single long_branch tramp and at most 2 or 3 plt
- * tramps generated. But, we don't use the plt tramps currently. We also allot
- * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
- * tramps in total. Set aside 8 just to be sure.
- */
-#define	NUM_FTRACE_TRAMPS	8
+#define	NUM_FTRACE_TRAMPS	2
 static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
 
 static ppc_inst_t
@@ -100,11 +94,6 @@ static int is_bl_op(ppc_inst_t op)
 	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
 }
 
-static int is_b_op(ppc_inst_t op)
-{
-	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
-}
-
 static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
 {
 	int offset;
@@ -227,11 +216,7 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 {
 	int i;
 
-	/*
-	 * We have the compiler generated long_branch tramps at the end
-	 * and we prefer those
-	 */
-	for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
+	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
 		if (!ftrace_tramps[i])
 			continue;
 		else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
@@ -240,75 +225,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 	return 0;
 }
 
-static int add_ftrace_tramp(unsigned long tramp)
-{
-	int i;
-
-	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
-		if (!ftrace_tramps[i]) {
-			ftrace_tramps[i] = tramp;
-			return 0;
-		}
-
-	return -1;
-}
-
-/*
- * If this is a compiler generated long_branch trampoline (essentially, a
- * trampoline that has a branch to _mcount()), we re-write the branch to
- * instead go to ftrace_[regs_]caller() and note down the location of this
- * trampoline.
- */
-static int setup_mcount_compiler_tramp(unsigned long tramp)
-{
-	int i;
-	ppc_inst_t op;
-	unsigned long ptr;
-
-	/* Is this a known long jump tramp? */
-	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
-		if (ftrace_tramps[i] == tramp)
-			return 0;
-
-	/* New trampoline -- read where this goes */
-	if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
-		pr_debug("Fetching opcode failed.\n");
-		return -1;
-	}
-
-	/* Is this a 24 bit branch? */
-	if (!is_b_op(op)) {
-		pr_debug("Trampoline is not a long branch tramp.\n");
-		return -1;
-	}
-
-	/* lets find where the pointer goes */
-	ptr = find_bl_target(tramp, op);
-
-	if (ptr != ppc_global_function_entry((void *)_mcount)) {
-		pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
-		return -1;
-	}
-
-	/* Let's re-write the tramp to go to ftrace_[regs_]caller */
-	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
-		ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
-	else
-		ptr = ppc_global_function_entry((void *)ftrace_caller);
-
-	if (patch_branch((u32 *)tramp, ptr, 0)) {
-		pr_debug("REL24 out of range!\n");
-		return -1;
-	}
-
-	if (add_ftrace_tramp(tramp)) {
-		pr_debug("No tramp locations left\n");
-		return -1;
-	}
-
-	return 0;
-}
-
 static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
 {
 	unsigned long tramp, ip = rec->ip;
@@ -331,13 +247,10 @@ static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
 
 	pr_devel("ip:%lx jumps to %lx", ip, tramp);
 
-	if (setup_mcount_compiler_tramp(tramp)) {
-		/* Are other trampolines reachable? */
-		if (!find_ftrace_tramp(ip)) {
-			pr_err("No ftrace trampolines reachable from %ps\n",
-					(void *)ip);
-			return -EINVAL;
-		}
+	/* Are ftrace trampolines reachable? */
+	if (!find_ftrace_tramp(ip)) {
+		pr_err("No ftrace trampolines reachable from %ps\n", (void *)ip);
+		return -EINVAL;
 	}
 
 	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
@@ -725,6 +638,17 @@ void ftrace_free_init_tramp(void)
 		}
 }
 
+static void __init add_ftrace_tramp(unsigned long tramp)
+{
+	int i;
+
+	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+		if (!ftrace_tramps[i]) {
+			ftrace_tramps[i] = tramp;
+			return;
+		}
+}
+
 int __init ftrace_dyn_arch_init(void)
 {
 	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
-- 
2.40.1

[PATCH 10/17] powerpc/ftrace: Add separate ftrace_init_nop() with additional validation

[Naveen N Rao]

Currently, we validate instructions around the ftrace location every
time we have to enable/disable ftrace. Introduce ftrace_init_nop() to
instead perform all the validation during ftrace initialization. This
allows us to simply patch the necessary instructions during
enabling/disabling ftrace.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/include/asm/ftrace.h  |  6 +++
 arch/powerpc/kernel/trace/ftrace.c | 71 ++++++++++++++++++++++++++++++
 2 files changed, 77 insertions(+)

diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index 702aaf2efa966c..ef9f0b97670d1c 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -29,11 +29,17 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr)
 unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
 				    unsigned long sp);
 
+struct module;
+struct dyn_ftrace;
 struct dyn_arch_ftrace {
 	struct module *mod;
 };
 
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
+#define ftrace_need_init_nop()	(true)
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
+#define ftrace_init_nop ftrace_init_nop
+
 struct ftrace_regs {
 	struct pt_regs regs;
 };
diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 278bf8e52b6e89..98bd099c428ee0 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -31,6 +31,16 @@
 #define	NUM_FTRACE_TRAMPS	2
 static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
 
+static ppc_inst_t ftrace_create_branch_inst(unsigned long ip, unsigned long addr, int link)
+{
+	ppc_inst_t op;
+
+	WARN_ON(!is_offset_in_branch_range(addr - ip));
+	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
+
+	return op;
+}
+
 static ppc_inst_t
 ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
 {
@@ -597,6 +607,67 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
 }
 #endif
 
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
+{
+	unsigned long addr, ip = rec->ip;
+	ppc_inst_t old, new;
+	int ret = 0;
+
+	/* Verify instructions surrounding the ftrace location */
+	if (IS_ENABLED(CONFIG_PPC32)) {
+		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
+		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
+		if (!ret)
+			ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)));
+	} else if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
+		/* Expected sequence: 'mflr r0', ['std r0,16(r1)'], 'bl _mcount' */
+		ret = ftrace_read_inst(ip - 4, &old);
+		if (!ret && !ppc_inst_equal(old, ppc_inst(PPC_RAW_MFLR(_R0)))) {
+			ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
+			ret |= ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)));
+		}
+	} else {
+		return -EINVAL;
+	}
+
+	if (ret)
+		return ret;
+
+	if (!core_kernel_text(ip)) {
+		if (!mod) {
+			pr_err("0x%lx: No module provided for non-kernel address\n", ip);
+			return -EFAULT;
+		}
+		rec->arch.mod = mod;
+	}
+
+	/* Nop-out the ftrace location */
+	new = ppc_inst(PPC_RAW_NOP());
+	addr = MCOUNT_ADDR;
+	if (is_offset_in_branch_range(addr - ip)) {
+		/* Within range */
+		old = ftrace_create_branch_inst(ip, addr, 1);
+		ret = ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip) || (IS_ENABLED(CONFIG_MODULES) && mod)) {
+		/*
+		 * We would be branching to a linker-generated stub, or to the module _mcount
+		 * stub. Let's just confirm we have a 'bl' here.
+		 */
+		ret = ftrace_read_inst(ip, &old);
+		if (ret)
+			return ret;
+		if (!is_bl_op(old)) {
+			pr_err("0x%lx: expected (bl) != found (%08lx)\n", ip, ppc_inst_as_ulong(old));
+			return -EINVAL;
+		}
+		ret = patch_instruction((u32 *)ip, new);
+	} else {
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
 int ftrace_update_ftrace_func(ftrace_func_t func)
 {
 	unsigned long ip = (unsigned long)(&ftrace_call);
-- 
2.40.1

[PATCH 11/17] powerpc/ftrace: Simplify ftrace_make_nop()

[Naveen N Rao]

Now that we validate the ftrace location during initialization in
ftrace_init_nop(), we can simplify ftrace_make_nop() to patch-in the nop
without worrying about the instructions surrounding the ftrace location.
Note that we continue to ensure that we have a bl to
ftrace_[regs_]caller at the ftrace location before nop-ing it out.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 220 +++++------------------------
 1 file changed, 32 insertions(+), 188 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 98bd099c428ee0..05153a1038fdff 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -116,112 +116,6 @@ static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
 	return ip + (long)offset;
 }
 
-#ifdef CONFIG_MODULES
-static int
-__ftrace_make_nop(struct module *mod,
-		  struct dyn_ftrace *rec, unsigned long addr)
-{
-	unsigned long entry, ptr, tramp;
-	unsigned long ip = rec->ip;
-	ppc_inst_t op, pop;
-
-	/* read where this goes */
-	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
-		pr_err("Fetching opcode failed.\n");
-		return -EFAULT;
-	}
-
-	/* Make sure that this is still a 24bit jump */
-	if (!is_bl_op(op)) {
-		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
-		return -EINVAL;
-	}
-
-	/* lets find where the pointer goes */
-	tramp = find_bl_target(ip, op);
-
-	pr_devel("ip:%lx jumps to %lx", ip, tramp);
-
-	if (module_trampoline_target(mod, tramp, &ptr)) {
-		pr_err("Failed to get trampoline target\n");
-		return -EFAULT;
-	}
-
-	pr_devel("trampoline target %lx", ptr);
-
-	entry = ppc_global_function_entry((void *)addr);
-	/* This should match what was called */
-	if (ptr != entry) {
-		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
-		return -EINVAL;
-	}
-
-	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
-		if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
-			pr_err("Fetching instruction at %lx failed.\n", ip - 4);
-			return -EFAULT;
-		}
-
-		/* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
-		if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
-		    !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
-			pr_err("Unexpected instruction %08lx around bl _mcount\n",
-			       ppc_inst_as_ulong(op));
-			return -EINVAL;
-		}
-	} else if (IS_ENABLED(CONFIG_PPC64)) {
-		/*
-		 * Check what is in the next instruction. We can see ld r2,40(r1), but
-		 * on first pass after boot we will see mflr r0.
-		 */
-		if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
-			pr_err("Fetching op failed.\n");
-			return -EFAULT;
-		}
-
-		if (!ppc_inst_equal(op,  ppc_inst(PPC_INST_LD_TOC))) {
-			pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
-			       ppc_inst_as_ulong(op));
-			return -EINVAL;
-		}
-	}
-
-	/*
-	 * When using -mprofile-kernel or PPC32 there is no load to jump over.
-	 *
-	 * Otherwise our original call site looks like:
-	 *
-	 * bl <tramp>
-	 * ld r2,XX(r1)
-	 *
-	 * Milton Miller pointed out that we can not simply nop the branch.
-	 * If a task was preempted when calling a trace function, the nops
-	 * will remove the way to restore the TOC in r2 and the r2 TOC will
-	 * get corrupted.
-	 *
-	 * Use a b +8 to jump over the load.
-	 * XXX: could make PCREL depend on MPROFILE_KERNEL
-	 * XXX: check PCREL && MPROFILE_KERNEL calling sequence
-	 */
-	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
-		pop = ppc_inst(PPC_RAW_NOP());
-	else
-		pop = ppc_inst(PPC_RAW_BRANCH(8));	/* b +8 */
-
-	if (patch_instruction((u32 *)ip, pop)) {
-		pr_err("Patching NOP failed.\n");
-		return -EPERM;
-	}
-
-	return 0;
-}
-#else
-static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
-{
-	return 0;
-}
-#endif /* CONFIG_MODULES */
-
 static unsigned long find_ftrace_tramp(unsigned long ip)
 {
 	int i;
@@ -235,88 +129,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 	return 0;
 }
 
-static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
-{
-	unsigned long tramp, ip = rec->ip;
-	ppc_inst_t op;
-
-	/* Read where this goes */
-	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
-		pr_err("Fetching opcode failed.\n");
-		return -EFAULT;
-	}
-
-	/* Make sure that this is still a 24bit jump */
-	if (!is_bl_op(op)) {
-		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
-		return -EINVAL;
-	}
-
-	/* Let's find where the pointer goes */
-	tramp = find_bl_target(ip, op);
-
-	pr_devel("ip:%lx jumps to %lx", ip, tramp);
-
-	/* Are ftrace trampolines reachable? */
-	if (!find_ftrace_tramp(ip)) {
-		pr_err("No ftrace trampolines reachable from %ps\n", (void *)ip);
-		return -EINVAL;
-	}
-
-	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
-		pr_err("Patching NOP failed.\n");
-		return -EPERM;
-	}
-
-	return 0;
-}
-
-int ftrace_make_nop(struct module *mod,
-		    struct dyn_ftrace *rec, unsigned long addr)
-{
-	unsigned long ip = rec->ip;
-	ppc_inst_t old, new;
-
-	/*
-	 * If the calling address is more that 24 bits away,
-	 * then we had to use a trampoline to make the call.
-	 * Otherwise just update the call site.
-	 */
-	if (test_24bit_addr(ip, addr)) {
-		/* within range */
-		old = ftrace_call_replace(ip, addr, 1);
-		new = ppc_inst(PPC_RAW_NOP());
-		return ftrace_modify_code(ip, old, new);
-	} else if (core_kernel_text(ip)) {
-		return __ftrace_make_nop_kernel(rec, addr);
-	} else if (!IS_ENABLED(CONFIG_MODULES)) {
-		return -EINVAL;
-	}
-
-	/*
-	 * Out of range jumps are called from modules.
-	 * We should either already have a pointer to the module
-	 * or it has been passed in.
-	 */
-	if (!rec->arch.mod) {
-		if (!mod) {
-			pr_err("No module loaded addr=%lx\n", addr);
-			return -EFAULT;
-		}
-		rec->arch.mod = mod;
-	} else if (mod) {
-		if (mod != rec->arch.mod) {
-			pr_err("Record mod %p not equal to passed in mod %p\n",
-			       rec->arch.mod, mod);
-			return -EINVAL;
-		}
-		/* nothing to do if mod == rec->arch.mod */
-	} else
-		mod = rec->arch.mod;
-
-	return __ftrace_make_nop(mod, rec, addr);
-}
-
 #ifdef CONFIG_MODULES
 /*
  * Examine the existing instructions for __ftrace_make_call.
@@ -607,6 +419,38 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
 }
 #endif
 
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long tramp, ip = rec->ip;
+	ppc_inst_t old, new;
+
+	/* Nop-out the ftrace location */
+	new = ppc_inst(PPC_RAW_NOP());
+	if (is_offset_in_branch_range(addr - ip)) {
+		/* Within range */
+		old = ftrace_create_branch_inst(ip, addr, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip)) {
+		/* We would be branching to one of our ftrace tramps */
+		tramp = find_ftrace_tramp(ip);
+		if (!tramp) {
+			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
+			return -EINVAL;
+		}
+		old = ftrace_create_branch_inst(ip, tramp, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (IS_ENABLED(CONFIG_MODULES)) {
+		/* Module code would be going to one of the module stubs */
+		if (!mod)
+			mod = rec->arch.mod;
+		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
+		old = ftrace_create_branch_inst(ip, tramp, 1);
+		return ftrace_modify_code(ip, old, new);
+	}
+
+	return -EINVAL;
+}
+
 int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
 {
 	unsigned long addr, ip = rec->ip;
-- 
2.40.1

[PATCH 12/17] powerpc/ftrace: Simplify ftrace_make_call()

[Naveen N Rao]

Now that we validate the ftrace location during initialization in
ftrace_init_nop(), we can simplify ftrace_make_call() to replace the nop
without worrying about the instructions surrounding the ftrace location.
Note that we continue to ensure that we have a nop at the ftrace
location before patching it.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 187 +++++------------------------
 1 file changed, 31 insertions(+), 156 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 05153a1038fdff..6ea8b90246a540 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -129,162 +129,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 	return 0;
 }
 
-#ifdef CONFIG_MODULES
-/*
- * Examine the existing instructions for __ftrace_make_call.
- * They should effectively be a NOP, and follow formal constraints,
- * depending on the ABI. Return false if they don't.
- */
-static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
-{
-	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
-		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
-	else
-		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
-		       ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
-}
-
-static int
-__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
-{
-	ppc_inst_t op[2];
-	void *ip = (void *)rec->ip;
-	unsigned long entry, ptr, tramp;
-	struct module *mod = rec->arch.mod;
-
-	/* read where this goes */
-	if (copy_inst_from_kernel_nofault(op, ip))
-		return -EFAULT;
-
-	if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
-	    copy_inst_from_kernel_nofault(op + 1, ip + 4))
-		return -EFAULT;
-
-	if (!expected_nop_sequence(ip, op[0], op[1])) {
-		pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
-		       ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
-		return -EINVAL;
-	}
-
-	/* If we never set up ftrace trampoline(s), then bail */
-	if (!mod->arch.tramp ||
-	    (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
-		pr_err("No ftrace trampoline\n");
-		return -EINVAL;
-	}
-
-	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
-		tramp = mod->arch.tramp_regs;
-	else
-		tramp = mod->arch.tramp;
-
-	if (module_trampoline_target(mod, tramp, &ptr)) {
-		pr_err("Failed to get trampoline target\n");
-		return -EFAULT;
-	}
-
-	pr_devel("trampoline target %lx", ptr);
-
-	entry = ppc_global_function_entry((void *)addr);
-	/* This should match what was called */
-	if (ptr != entry) {
-		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
-		return -EINVAL;
-	}
-
-	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
-		pr_err("REL24 out of range!\n");
-		return -EINVAL;
-	}
-
-	return 0;
-}
-#else
-static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
-{
-	return 0;
-}
-#endif /* CONFIG_MODULES */
-
-static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
-{
-	ppc_inst_t op;
-	void *ip = (void *)rec->ip;
-	unsigned long tramp, entry, ptr;
-
-	/* Make sure we're being asked to patch branch to a known ftrace addr */
-	entry = ppc_global_function_entry((void *)ftrace_caller);
-	ptr = ppc_global_function_entry((void *)addr);
-
-	if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
-		entry = ppc_global_function_entry((void *)ftrace_regs_caller);
-
-	if (ptr != entry) {
-		pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
-		return -EINVAL;
-	}
-
-	/* Make sure we have a nop */
-	if (copy_inst_from_kernel_nofault(&op, ip)) {
-		pr_err("Unable to read ftrace location %p\n", ip);
-		return -EFAULT;
-	}
-
-	if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
-		pr_err("Unexpected call sequence at %p: %08lx\n",
-		       ip, ppc_inst_as_ulong(op));
-		return -EINVAL;
-	}
-
-	tramp = find_ftrace_tramp((unsigned long)ip);
-	if (!tramp) {
-		pr_err("No ftrace trampolines reachable from %ps\n", ip);
-		return -EINVAL;
-	}
-
-	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
-		pr_err("Error patching branch to ftrace tramp!\n");
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
-{
-	unsigned long ip = rec->ip;
-	ppc_inst_t old, new;
-
-	/*
-	 * If the calling address is more that 24 bits away,
-	 * then we had to use a trampoline to make the call.
-	 * Otherwise just update the call site.
-	 */
-	if (test_24bit_addr(ip, addr)) {
-		/* within range */
-		old = ppc_inst(PPC_RAW_NOP());
-		new = ftrace_call_replace(ip, addr, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (core_kernel_text(ip)) {
-		return __ftrace_make_call_kernel(rec, addr);
-	} else if (!IS_ENABLED(CONFIG_MODULES)) {
-		/* We should not get here without modules */
-		return -EINVAL;
-	}
-
-	/*
-	 * Out of range jumps are called from modules.
-	 * Being that we are converting from nop, it had better
-	 * already have a module defined.
-	 */
-	if (!rec->arch.mod) {
-		pr_err("No module loaded\n");
-		return -EINVAL;
-	}
-
-	return __ftrace_make_call(rec, addr);
-}
-
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
 #ifdef CONFIG_MODULES
 static int
@@ -419,6 +263,37 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
 }
 #endif
 
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+	unsigned long tramp, ip = rec->ip;
+	ppc_inst_t old, new;
+	struct module *mod;
+
+	old = ppc_inst(PPC_RAW_NOP());
+	if (is_offset_in_branch_range(addr - ip)) {
+		/* Within range */
+		new = ftrace_create_branch_inst(ip, addr, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (core_kernel_text(ip)) {
+		/* We would be branching to one of our ftrace tramps */
+		tramp = find_ftrace_tramp(ip);
+		if (!tramp) {
+			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
+			return -EINVAL;
+		}
+		new = ftrace_create_branch_inst(ip, tramp, 1);
+		return ftrace_modify_code(ip, old, new);
+	} else if (IS_ENABLED(CONFIG_MODULES)) {
+		/* Module code would be going to one of the module stubs */
+		mod = rec->arch.mod;
+		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
+		new = ftrace_create_branch_inst(ip, tramp, 1);
+		return ftrace_modify_code(ip, old, new);
+	}
+
+	return -EINVAL;
+}
+
 int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
 {
 	unsigned long tramp, ip = rec->ip;
-- 
2.40.1

[PATCH 13/17] powerpc/ftrace: Simplify ftrace_modify_call()

[Naveen N Rao]

Now that we validate the ftrace location during initialization in
ftrace_init_nop(), we can simplify ftrace_modify_call() to patch-in the
updated branch instruction without worrying about the instructions
surrounding the ftrace location. Note that we continue to ensure we
have the expected branch instruction at the ftrace location before
patching it with the updated branch destination.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 161 ++++-------------------------
 1 file changed, 21 insertions(+), 140 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 6ea8b90246a540..c37e22c6c26521 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -89,33 +89,11 @@ static inline int ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_
 	return ret;
 }
 
-/*
- * Helper functions that are the same for both PPC64 and PPC32.
- */
-static int test_24bit_addr(unsigned long ip, unsigned long addr)
-{
-	addr = ppc_function_entry((void *)addr);
-
-	return is_offset_in_branch_range(addr - ip);
-}
-
 static int is_bl_op(ppc_inst_t op)
 {
 	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
 }
 
-static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
-{
-	int offset;
-
-	offset = PPC_LI(ppc_inst_val(op));
-	/* make it signed */
-	if (offset & 0x02000000)
-		offset |= 0xfe000000;
-
-	return ip + (long)offset;
-}
-
 static unsigned long find_ftrace_tramp(unsigned long ip)
 {
 	int i;
@@ -130,115 +108,16 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 }
 
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
-#ifdef CONFIG_MODULES
-static int
-__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
-					unsigned long addr)
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
 {
-	ppc_inst_t op;
-	unsigned long ip = rec->ip;
-	unsigned long entry, ptr, tramp;
-	struct module *mod = rec->arch.mod;
-
-	/* If we never set up ftrace trampolines, then bail */
-	if (!mod->arch.tramp || !mod->arch.tramp_regs) {
-		pr_err("No ftrace trampoline\n");
-		return -EINVAL;
-	}
-
-	/* read where this goes */
-	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
-		pr_err("Fetching opcode failed.\n");
-		return -EFAULT;
-	}
-
-	/* Make sure that this is still a 24bit jump */
-	if (!is_bl_op(op)) {
-		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
-		return -EINVAL;
-	}
-
-	/* lets find where the pointer goes */
-	tramp = find_bl_target(ip, op);
-	entry = ppc_global_function_entry((void *)old_addr);
-
-	pr_devel("ip:%lx jumps to %lx", ip, tramp);
-
-	if (tramp != entry) {
-		/* old_addr is not within range, so we must have used a trampoline */
-		if (module_trampoline_target(mod, tramp, &ptr)) {
-			pr_err("Failed to get trampoline target\n");
-			return -EFAULT;
-		}
-
-		pr_devel("trampoline target %lx", ptr);
-
-		/* This should match what was called */
-		if (ptr != entry) {
-			pr_err("addr %lx does not match expected %lx\n", ptr, entry);
-			return -EINVAL;
-		}
-	}
-
-	/* The new target may be within range */
-	if (test_24bit_addr(ip, addr)) {
-		/* within range */
-		if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
-			pr_err("REL24 out of range!\n");
-			return -EINVAL;
-		}
-
-		return 0;
-	}
-
-	if (rec->flags & FTRACE_FL_REGS)
-		tramp = mod->arch.tramp_regs;
-	else
-		tramp = mod->arch.tramp;
-
-	if (module_trampoline_target(mod, tramp, &ptr)) {
-		pr_err("Failed to get trampoline target\n");
-		return -EFAULT;
-	}
-
-	pr_devel("trampoline target %lx", ptr);
-
-	entry = ppc_global_function_entry((void *)addr);
-	/* This should match what was called */
-	if (ptr != entry) {
-		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
-		return -EINVAL;
-	}
-
-	if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
-		pr_err("REL24 out of range!\n");
-		return -EINVAL;
-	}
-
-	return 0;
-}
-#else
-static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
-{
-	return 0;
-}
-#endif
-
-int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
-			unsigned long addr)
-{
-	unsigned long ip = rec->ip;
+	unsigned long tramp, tramp_old, ip = rec->ip;
 	ppc_inst_t old, new;
+	struct module *mod;
 
-	/*
-	 * If the calling address is more that 24 bits away,
-	 * then we had to use a trampoline to make the call.
-	 * Otherwise just update the call site.
-	 */
-	if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
-		/* within range */
-		old = ftrace_call_replace(ip, old_addr, 1);
-		new = ftrace_call_replace(ip, addr, 1);
+	if (is_offset_in_branch_range(old_addr - ip) && is_offset_in_branch_range(addr - ip)) {
+		/* Within range */
+		old = ftrace_create_branch_inst(ip, old_addr, 1);
+		new = ftrace_create_branch_inst(ip, addr, 1);
 		return ftrace_modify_code(ip, old, new);
 	} else if (core_kernel_text(ip)) {
 		/*
@@ -246,20 +125,22 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
 		 * variant, so there is nothing to do here
 		 */
 		return 0;
-	} else if (!IS_ENABLED(CONFIG_MODULES)) {
-		/* We should not get here without modules */
-		return -EINVAL;
+	} else if (IS_ENABLED(CONFIG_MODULES)) {
+		/* Module code would be going to one of the module stubs */
+		mod = rec->arch.mod;
+		if (addr == (unsigned long)ftrace_caller) {
+			tramp_old = mod->arch.tramp_regs;
+			tramp = mod->arch.tramp;
+		} else {
+			tramp_old = mod->arch.tramp;
+			tramp = mod->arch.tramp_regs;
+		}
+		old = ftrace_create_branch_inst(ip, tramp_old, 1);
+		new = ftrace_create_branch_inst(ip, tramp, 1);
+		return ftrace_modify_code(ip, old, new);
 	}
 
-	/*
-	 * Out of range jumps are called from modules.
-	 */
-	if (!rec->arch.mod) {
-		pr_err("No module loaded\n");
-		return -EINVAL;
-	}
-
-	return __ftrace_modify_call(rec, old_addr, addr);
+	return -EINVAL;
 }
 #endif
 
-- 
2.40.1

[PATCH 14/17] powerpc/ftrace: Replace use of ftrace_call_replace() with ftrace_create_branch_inst()

[Naveen N Rao]

ftrace_create_branch_inst() is clearer about its intent than
ftrace_call_replace().

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 17 ++---------------
 1 file changed, 2 insertions(+), 15 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index c37e22c6c26521..422dd760fbe013 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -41,19 +41,6 @@ static ppc_inst_t ftrace_create_branch_inst(unsigned long ip, unsigned long addr
 	return op;
 }
 
-static ppc_inst_t
-ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
-{
-	ppc_inst_t op;
-
-	addr = ppc_function_entry((void *)addr);
-
-	/* if (link) set op to 'bl' else 'b' */
-	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
-
-	return op;
-}
-
 static inline int ftrace_read_inst(unsigned long ip, ppc_inst_t *op)
 {
 	if (copy_inst_from_kernel_nofault(op, (void *)ip)) {
@@ -275,14 +262,14 @@ int ftrace_update_ftrace_func(ftrace_func_t func)
 	int ret;
 
 	old = ppc_inst_read((u32 *)&ftrace_call);
-	new = ftrace_call_replace(ip, (unsigned long)func, 1);
+	new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
 	ret = ftrace_modify_code(ip, old, new);
 
 	/* Also update the regs callback function */
 	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
 		ip = (unsigned long)(&ftrace_regs_call);
 		old = ppc_inst_read((u32 *)&ftrace_regs_call);
-		new = ftrace_call_replace(ip, (unsigned long)func, 1);
+		new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
 		ret = ftrace_modify_code(ip, old, new);
 	}
 
-- 
2.40.1

[PATCH 15/17] powerpc/ftrace: Implement ftrace_replace_code()

[Naveen N Rao]

Implement ftrace_replace_code() to consolidate logic from the different
ftrace patching routines: ftrace_make_nop(), ftrace_make_call() and
ftrace_modify_call(). Note that ftrace_make_call() is still required
primarily to handle patching modules during their load time. The other
two routines should no longer be called.

This lays the groundwork to enable better control in patching ftrace
locations, including the ability to nop-out preceding profiling
instructions when ftrace is disabled.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c | 173 ++++++++++++++++-------------
 1 file changed, 96 insertions(+), 77 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 422dd760fbe013..cf9dce77527920 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -94,104 +94,123 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
 	return 0;
 }
 
+static int ftrace_get_call_inst(struct dyn_ftrace *rec, unsigned long addr, ppc_inst_t *call_inst)
+{
+	unsigned long ip = rec->ip;
+	unsigned long stub;
+
+	if (is_offset_in_branch_range(addr - ip)) {
+		/* Within range */
+		stub = addr;
+#ifdef CONFIG_MODULES
+	} else if (rec->arch.mod) {
+		/* Module code would be going to one of the module stubs */
+		stub = (addr == (unsigned long)ftrace_caller ? rec->arch.mod->arch.tramp :
+							       rec->arch.mod->arch.tramp_regs);
+#endif
+	} else if (core_kernel_text(ip)) {
+		/* We would be branching to one of our ftrace stubs */
+		stub = find_ftrace_tramp(ip);
+		if (!stub) {
+			pr_err("0x%lx: No ftrace stubs reachable\n", ip);
+			return -EINVAL;
+		}
+	} else {
+		return -EINVAL;
+	}
+
+	*call_inst = ftrace_create_branch_inst(ip, stub, 1);
+	return 0;
+}
+
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
 int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
 {
-	unsigned long tramp, tramp_old, ip = rec->ip;
-	ppc_inst_t old, new;
-	struct module *mod;
-
-	if (is_offset_in_branch_range(old_addr - ip) && is_offset_in_branch_range(addr - ip)) {
-		/* Within range */
-		old = ftrace_create_branch_inst(ip, old_addr, 1);
-		new = ftrace_create_branch_inst(ip, addr, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (core_kernel_text(ip)) {
-		/*
-		 * We always patch out of range locations to go to the regs
-		 * variant, so there is nothing to do here
-		 */
-		return 0;
-	} else if (IS_ENABLED(CONFIG_MODULES)) {
-		/* Module code would be going to one of the module stubs */
-		mod = rec->arch.mod;
-		if (addr == (unsigned long)ftrace_caller) {
-			tramp_old = mod->arch.tramp_regs;
-			tramp = mod->arch.tramp;
-		} else {
-			tramp_old = mod->arch.tramp;
-			tramp = mod->arch.tramp_regs;
-		}
-		old = ftrace_create_branch_inst(ip, tramp_old, 1);
-		new = ftrace_create_branch_inst(ip, tramp, 1);
-		return ftrace_modify_code(ip, old, new);
-	}
-
+	/* This should never be called since we override ftrace_replace_code() */
+	WARN_ON(1);
 	return -EINVAL;
 }
 #endif
 
 int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
 {
-	unsigned long tramp, ip = rec->ip;
 	ppc_inst_t old, new;
-	struct module *mod;
+	int ret;
+
+	/* This can only ever be called during module load */
+	if (WARN_ON(!IS_ENABLED(CONFIG_MODULES) || core_kernel_text(rec->ip)))
+		return -EINVAL;
 
 	old = ppc_inst(PPC_RAW_NOP());
-	if (is_offset_in_branch_range(addr - ip)) {
-		/* Within range */
-		new = ftrace_create_branch_inst(ip, addr, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (core_kernel_text(ip)) {
-		/* We would be branching to one of our ftrace tramps */
-		tramp = find_ftrace_tramp(ip);
-		if (!tramp) {
-			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
-			return -EINVAL;
-		}
-		new = ftrace_create_branch_inst(ip, tramp, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (IS_ENABLED(CONFIG_MODULES)) {
-		/* Module code would be going to one of the module stubs */
-		mod = rec->arch.mod;
-		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
-		new = ftrace_create_branch_inst(ip, tramp, 1);
-		return ftrace_modify_code(ip, old, new);
-	}
+	ret = ftrace_get_call_inst(rec, addr, &new);
+	if (ret)
+		return ret;
 
-	return -EINVAL;
+	return ftrace_modify_code(rec->ip, old, new);
 }
 
 int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
 {
-	unsigned long tramp, ip = rec->ip;
-	ppc_inst_t old, new;
+	/*
+	 * This should never be called since we override ftrace_replace_code(),
+	 * as well as ftrace_init_nop()
+	 */
+	WARN_ON(1);
+	return -EINVAL;
+}
 
-	/* Nop-out the ftrace location */
-	new = ppc_inst(PPC_RAW_NOP());
-	if (is_offset_in_branch_range(addr - ip)) {
-		/* Within range */
-		old = ftrace_create_branch_inst(ip, addr, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (core_kernel_text(ip)) {
-		/* We would be branching to one of our ftrace tramps */
-		tramp = find_ftrace_tramp(ip);
-		if (!tramp) {
-			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
-			return -EINVAL;
+void ftrace_replace_code(int enable)
+{
+	ppc_inst_t old, new, call_inst, new_call_inst;
+	ppc_inst_t nop_inst = ppc_inst(PPC_RAW_NOP());
+	unsigned long ip, new_addr, addr;
+	struct ftrace_rec_iter *iter;
+	struct dyn_ftrace *rec;
+	int ret = 0, update;
+
+	for_ftrace_rec_iter(iter) {
+		rec = ftrace_rec_iter_record(iter);
+		ip = rec->ip;
+
+		if (rec->flags & FTRACE_FL_DISABLED && !(rec->flags & FTRACE_FL_ENABLED))
+			continue;
+
+		addr = ftrace_get_addr_curr(rec);
+		new_addr = ftrace_get_addr_new(rec);
+		update = ftrace_update_record(rec, enable);
+
+		switch (update) {
+		case FTRACE_UPDATE_IGNORE:
+		default:
+			continue;
+		case FTRACE_UPDATE_MODIFY_CALL:
+			ret = ftrace_get_call_inst(rec, new_addr, &new_call_inst);
+			ret |= ftrace_get_call_inst(rec, addr, &call_inst);
+			old = call_inst;
+			new = new_call_inst;
+			break;
+		case FTRACE_UPDATE_MAKE_NOP:
+			ret = ftrace_get_call_inst(rec, addr, &call_inst);
+			old = call_inst;
+			new = nop_inst;
+			break;
+		case FTRACE_UPDATE_MAKE_CALL:
+			ret = ftrace_get_call_inst(rec, new_addr, &call_inst);
+			old = nop_inst;
+			new = call_inst;
+			break;
 		}
-		old = ftrace_create_branch_inst(ip, tramp, 1);
-		return ftrace_modify_code(ip, old, new);
-	} else if (IS_ENABLED(CONFIG_MODULES)) {
-		/* Module code would be going to one of the module stubs */
-		if (!mod)
-			mod = rec->arch.mod;
-		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
-		old = ftrace_create_branch_inst(ip, tramp, 1);
-		return ftrace_modify_code(ip, old, new);
+
+		if (!ret)
+			ret = ftrace_modify_code(ip, old, new);
+		if (ret)
+			goto out;
 	}
 
-	return -EINVAL;
+out:
+	if (ret)
+		ftrace_bug(ret, rec);
+	return;
 }
 
 int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
-- 
2.40.1

[PATCH 16/17] powerpc/ftrace: Add support for -fpatchable-function-entry

[Naveen N Rao]

GCC v13.1 updated support for -fpatchable-function-entry on ppc64le to
emit nops after the local entry point, rather than before it. This
allows us to use this in the kernel for ftrace purposes. A new script is
added under arch/powerpc/tools/ to help detect if nops are emitted after
the function local entry point, or before the global entry point.

With -fpatchable-function-entry, we no longer have the profiling
instructions generated at function entry, so we only need to validate
the presence of two nops at the ftrace location in ftrace_init_nop(). We
patch the preceding instruction with 'mflr r0' to match the
-mprofile-kernel ABI for subsequent ftrace use.

This changes the profiling instructions used on ppc32. The default -pg
option emits an additional 'stw' instruction after 'mflr r0' and before
the branch to _mcount 'bl _mcount'. This is very similar to the original
-mprofile-kernel implementation on ppc64le, where an additional 'std'
instruction was used to save LR to its save location in the caller's
stackframe. Subsequently, this additional store was removed in later
compiler versions for performance reasons. The same reasons apply for
ppc32 so we only patch in a 'mflr r0'.

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/Kconfig                          | 14 +++++++---
 arch/powerpc/Makefile                         |  5 ++++
 arch/powerpc/include/asm/ftrace.h             |  6 +++--
 arch/powerpc/include/asm/vermagic.h           |  4 ++-
 arch/powerpc/kernel/module_64.c               |  2 +-
 arch/powerpc/kernel/trace/ftrace.c            | 14 ++++++++--
 arch/powerpc/kernel/trace/ftrace_entry.S      |  2 ++
 .../gcc-check-fpatchable-function-entry.sh    | 26 +++++++++++++++++++
 8 files changed, 64 insertions(+), 9 deletions(-)
 create mode 100755 arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index bff5820b7cda14..9352d8e68152e1 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -187,6 +187,7 @@ config PPC
 	select DYNAMIC_FTRACE			if FUNCTION_TRACER
 	select EDAC_ATOMIC_SCRUB
 	select EDAC_SUPPORT
+	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if ARCH_USING_PATCHABLE_FUNCTION_ENTRY
 	select GENERIC_ATOMIC64			if PPC32
 	select GENERIC_CLOCKEVENTS_BROADCAST	if SMP
 	select GENERIC_CMOS_UPDATE
@@ -227,8 +228,8 @@ config PPC
 	select HAVE_DEBUG_KMEMLEAK
 	select HAVE_DEBUG_STACKOVERFLOW
 	select HAVE_DYNAMIC_FTRACE
-	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if MPROFILE_KERNEL || PPC32
-	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if MPROFILE_KERNEL || PPC32
+	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
+	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
 	select HAVE_EBPF_JIT
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS
 	select HAVE_FAST_GUP
@@ -256,7 +257,7 @@ config PPC
 	select HAVE_MOD_ARCH_SPECIFIC
 	select HAVE_NMI				if PERF_EVENTS || (PPC64 && PPC_BOOK3S)
 	select HAVE_OPTPROBES
-	select HAVE_OBJTOOL			if PPC32 || MPROFILE_KERNEL
+	select HAVE_OBJTOOL			if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
 	select HAVE_OBJTOOL_MCOUNT		if HAVE_OBJTOOL
 	select HAVE_PERF_EVENTS
 	select HAVE_PERF_EVENTS_NMI		if PPC64
@@ -550,6 +551,13 @@ config MPROFILE_KERNEL
 	depends on PPC64 && CPU_LITTLE_ENDIAN && FUNCTION_TRACER
 	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-mprofile-kernel.sh $(CC) -I$(srctree)/include -D__KERNEL__)
 
+config ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+	depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2)
+	depends on $(cc-option,-fpatchable-function-entry=2)
+	def_bool y if PPC32
+	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN
+	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN
+
 config HOTPLUG_CPU
 	bool "Support for enabling/disabling CPUs"
 	depends on SMP && (PPC_PSERIES || \
diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
index dca73f673d7046..de39478b1c9e9f 100644
--- a/arch/powerpc/Makefile
+++ b/arch/powerpc/Makefile
@@ -148,11 +148,16 @@ CFLAGS-$(CONFIG_PPC32)	+= $(call cc-option, $(MULTIPLEWORD))
 CFLAGS-$(CONFIG_PPC32)	+= $(call cc-option,-mno-readonly-in-sdata)
 
 ifdef CONFIG_FUNCTION_TRACER
+ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+KBUILD_CPPFLAGS	+= -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+CC_FLAGS_FTRACE := -fpatchable-function-entry=2
+else
 CC_FLAGS_FTRACE := -pg
 ifdef CONFIG_MPROFILE_KERNEL
 CC_FLAGS_FTRACE += -mprofile-kernel
 endif
 endif
+endif
 
 CFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
 AFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
index ef9f0b97670d1c..9e5a39b6a3114b 100644
--- a/arch/powerpc/include/asm/ftrace.h
+++ b/arch/powerpc/include/asm/ftrace.h
@@ -11,7 +11,7 @@
 #define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
 
 /* Ignore unused weak functions which will have larger offsets */
-#ifdef CONFIG_MPROFILE_KERNEL
+#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
 #define FTRACE_MCOUNT_MAX_OFFSET	16
 #elif defined(CONFIG_PPC32)
 #define FTRACE_MCOUNT_MAX_OFFSET	8
@@ -22,7 +22,9 @@ extern void _mcount(void);
 
 static inline unsigned long ftrace_call_adjust(unsigned long addr)
 {
-       /* relocation of mcount call site is the same as the address */
+	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY))
+		addr += MCOUNT_INSN_SIZE;
+
        return addr;
 }
 
diff --git a/arch/powerpc/include/asm/vermagic.h b/arch/powerpc/include/asm/vermagic.h
index b054a8576e5deb..6f250fe506bd1c 100644
--- a/arch/powerpc/include/asm/vermagic.h
+++ b/arch/powerpc/include/asm/vermagic.h
@@ -2,7 +2,9 @@
 #ifndef _ASM_VERMAGIC_H
 #define _ASM_VERMAGIC_H
 
-#ifdef CONFIG_MPROFILE_KERNEL
+#ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+#define MODULE_ARCH_VERMAGIC_FTRACE	"patchable-function-entry "
+#elif defined(CONFIG_MPROFILE_KERNEL)
 #define MODULE_ARCH_VERMAGIC_FTRACE	"mprofile-kernel "
 #else
 #define MODULE_ARCH_VERMAGIC_FTRACE	""
diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
index 92570289ce08f5..7112adc597a80b 100644
--- a/arch/powerpc/kernel/module_64.c
+++ b/arch/powerpc/kernel/module_64.c
@@ -465,7 +465,7 @@ int module_frob_arch_sections(Elf64_Ehdr *hdr,
 	return 0;
 }
 
-#ifdef CONFIG_MPROFILE_KERNEL
+#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
 
 static u32 stub_insns[] = {
 #ifdef CONFIG_PPC_KERNEL_PCREL
diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index cf9dce77527920..82010629cf887c 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -220,7 +220,12 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
 	int ret = 0;
 
 	/* Verify instructions surrounding the ftrace location */
-	if (IS_ENABLED(CONFIG_PPC32)) {
+	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
+		/* Expect nops */
+		ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_NOP()));
+		if (!ret)
+			ret = ftrace_validate_inst(ip, ppc_inst(PPC_RAW_NOP()));
+	} else if (IS_ENABLED(CONFIG_PPC32)) {
 		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
 		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
 		if (!ret)
@@ -250,7 +255,12 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
 	/* Nop-out the ftrace location */
 	new = ppc_inst(PPC_RAW_NOP());
 	addr = MCOUNT_ADDR;
-	if (is_offset_in_branch_range(addr - ip)) {
+	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
+		/* we instead patch-in the 'mflr r0' */
+		old = ppc_inst(PPC_RAW_NOP());
+		new = ppc_inst(PPC_RAW_MFLR(_R0));
+		ret = ftrace_modify_code(ip - 4, old, new);
+	} else if (is_offset_in_branch_range(addr - ip)) {
 		/* Within range */
 		old = ftrace_create_branch_inst(ip, addr, 1);
 		ret = ftrace_modify_code(ip, old, new);
diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S
index e8339706e735b1..bab3ab1368a33f 100644
--- a/arch/powerpc/kernel/trace/ftrace_entry.S
+++ b/arch/powerpc/kernel/trace/ftrace_entry.S
@@ -250,6 +250,7 @@ livepatch_handler:
 	blr
 #endif /* CONFIG_LIVEPATCH */
 
+#ifndef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
 _GLOBAL(mcount)
 _GLOBAL(_mcount)
 EXPORT_SYMBOL(_mcount)
@@ -257,6 +258,7 @@ EXPORT_SYMBOL(_mcount)
 	mtctr	r12
 	mtlr	r0
 	bctr
+#endif
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 _GLOBAL(return_to_handler)
diff --git a/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh b/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
new file mode 100755
index 00000000000000..06706903503b6c
--- /dev/null
+++ b/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+set -o pipefail
+
+# To debug, uncomment the following line
+# set -x
+
+# Output from -fpatchable-function-entry can only vary on ppc64 elfv2, so this
+# should not be invoked for other targets. Therefore we can pass in -m64 and
+# -mabi explicitly, to take care of toolchains defaulting to other targets.
+
+# Test whether the compile option -fpatchable-function-entry exists and
+# generates appropriate code
+echo "int func() { return 0; }" | \
+    $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
+    grep -q "__patchable_function_entries"
+
+# Test whether nops are generated after the local entry point
+echo "int x; int func() { return x; }" | \
+    $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
+    awk 'BEGIN { RS = ";" } /\.localentry.*nop.*\n[[:space:]]*nop/ { print $0 }' | \
+    grep -q "func:"
+
+exit 0
-- 
2.40.1

[PATCH 17/17] powerpc/ftrace: Create a dummy stackframe to fix stack unwind

[Naveen N Rao]

With ppc64 -mprofile-kernel and ppc32 -pg, profiling instructions to
call into ftrace are emitted right at function entry. The instruction
sequence used is minimal to reduce overhead. Crucially, a stackframe is
not created for the function being traced. This breaks stack unwinding
since the function being traced does not have a stackframe for itself.
As such, it never shows up in the backtrace:

/sys/kernel/debug/tracing # echo 1 > /proc/sys/kernel/stack_tracer_enabled
/sys/kernel/debug/tracing # cat stack_trace
        Depth    Size   Location    (17 entries)
        -----    ----   --------
  0)     4144      32   ftrace_call+0x4/0x44
  1)     4112     432   get_page_from_freelist+0x26c/0x1ad0
  2)     3680     496   __alloc_pages+0x290/0x1280
  3)     3184     336   __folio_alloc+0x34/0x90
  4)     2848     176   vma_alloc_folio+0xd8/0x540
  5)     2672     272   __handle_mm_fault+0x700/0x1cc0
  6)     2400     208   handle_mm_fault+0xf0/0x3f0
  7)     2192      80   ___do_page_fault+0x3e4/0xbe0
  8)     2112     160   do_page_fault+0x30/0xc0
  9)     1952     256   data_access_common_virt+0x210/0x220
 10)     1696     400   0xc00000000f16b100
 11)     1296     384   load_elf_binary+0x804/0x1b80
 12)      912     208   bprm_execve+0x2d8/0x7e0
 13)      704      64   do_execveat_common+0x1d0/0x2f0
 14)      640     160   sys_execve+0x54/0x70
 15)      480      64   system_call_exception+0x138/0x350
 16)      416     416   system_call_common+0x160/0x2c4

Fix this by having ftrace create a dummy stackframe for the function
being traced. Since this is only relevant when ftrace is active, we nop
out the instruction to store LR in the LR save area in the profiling
instruction sequence on ppc32 (and in ppc64 with older gcc versions).
Instead, in ftrace, we store LR in the LR save area of the previous
stackframe, and create a minimal stackframe to represent the function
being traced. With this, backtraces now capture the function being
traced:

/sys/kernel/debug/tracing # cat stack_trace
        Depth    Size   Location    (17 entries)
        -----    ----   --------
  0)     3888      32   _raw_spin_trylock+0x8/0x70
  1)     3856     576   get_page_from_freelist+0x26c/0x1ad0
  2)     3280      64   __alloc_pages+0x290/0x1280
  3)     3216     336   __folio_alloc+0x34/0x90
  4)     2880     176   vma_alloc_folio+0xd8/0x540
  5)     2704     416   __handle_mm_fault+0x700/0x1cc0
  6)     2288      96   handle_mm_fault+0xf0/0x3f0
  7)     2192      48   ___do_page_fault+0x3e4/0xbe0
  8)     2144     192   do_page_fault+0x30/0xc0
  9)     1952     608   data_access_common_virt+0x210/0x220
 10)     1344      16   0xc0000000334bbb50
 11)     1328     416   load_elf_binary+0x804/0x1b80
 12)      912      64   bprm_execve+0x2d8/0x7e0
 13)      848     176   do_execveat_common+0x1d0/0x2f0
 14)      672     192   sys_execve+0x54/0x70
 15)      480      64   system_call_exception+0x138/0x350
 16)      416     416   system_call_common+0x160/0x2c4

This results in two additional stores in the ftrace entry code, but
produces reliable backtraces. Note that this change now aligns with
other architectures (arm64, s390, x86).

Signed-off-by: Naveen N Rao <naveen@kernel.org>
---
 arch/powerpc/kernel/trace/ftrace.c       |  6 ++++--
 arch/powerpc/kernel/trace/ftrace_entry.S | 11 ++++++++---
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
index 82010629cf887c..2956196c98ffdc 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -229,13 +229,15 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
 		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
 		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
 		if (!ret)
-			ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)));
+			ret = ftrace_modify_code(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)),
+						 ppc_inst(PPC_RAW_NOP()));
 	} else if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
 		/* Expected sequence: 'mflr r0', ['std r0,16(r1)'], 'bl _mcount' */
 		ret = ftrace_read_inst(ip - 4, &old);
 		if (!ret && !ppc_inst_equal(old, ppc_inst(PPC_RAW_MFLR(_R0)))) {
 			ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
-			ret |= ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)));
+			ret |= ftrace_modify_code(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)),
+						  ppc_inst(PPC_RAW_NOP()));
 		}
 	} else {
 		return -EINVAL;
diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S
index bab3ab1368a33f..05e981fb526c2e 100644
--- a/arch/powerpc/kernel/trace/ftrace_entry.S
+++ b/arch/powerpc/kernel/trace/ftrace_entry.S
@@ -33,6 +33,11 @@
  * and then arrange for the ftrace function to be called.
  */
 .macro	ftrace_regs_entry allregs
+	/* Create a minimal stack frame for representing B */
+	PPC_STLU	r1, -STACK_FRAME_MIN_SIZE(r1)
+	/* Save the original return address in A's stack frame */
+	PPC_STL		r0, LRSAVE+STACK_FRAME_MIN_SIZE(r1)
+
 	/* Create our stack frame + pt_regs */
 	PPC_STLU	r1,-SWITCH_FRAME_SIZE(r1)
 
@@ -41,8 +46,6 @@
 	SAVE_GPRS(3, 10, r1)
 
 #ifdef CONFIG_PPC64
-	/* Save the original return address in A's stack frame */
-	std	r0, LRSAVE+SWITCH_FRAME_SIZE(r1)
 	/* Ok to continue? */
 	lbz	r3, PACA_FTRACE_ENABLED(r13)
 	cmpdi	r3, 0
@@ -77,6 +80,8 @@
 	mflr	r7
 	/* Save it as pt_regs->nip */
 	PPC_STL	r7, _NIP(r1)
+	/* Also save it in B's stackframe header for proper unwind */
+	PPC_STL	r7, LRSAVE+SWITCH_FRAME_SIZE(r1)
 	/* Save the read LR in pt_regs->link */
 	PPC_STL	r0, _LINK(r1)
 
@@ -142,7 +147,7 @@
 #endif
 
 	/* Pop our stack frame */
-	addi r1, r1, SWITCH_FRAME_SIZE
+	addi r1, r1, SWITCH_FRAME_SIZE+STACK_FRAME_MIN_SIZE
 
 #ifdef CONFIG_LIVEPATCH_64
         /* Based on the cmpd above, if the NIP was altered handle livepatch */
-- 
2.40.1

Re: [PATCH 01/17] powerpc/ftrace: Fix dropping weak symbols with older toolchains

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> The minimum level of gcc supported for building the kernel is v5.1.
> v5.x releases of gcc emitted a three instruction sequence for
> -mprofile-kernel:
> 	mflr	r0
> 	std	r0, 16(r1)
> 	bl	_mcount
> 
> It is only with the v6.x releases that gcc started emitting the two
> instruction sequence for -mprofile-kernel, omitting the second store
> instruction.
> 
> With the older three instruction sequence, the actual ftrace location
> can be the 5th instruction into a function. Update the allowed offset
> for ftrace location from 12 to 16 to accommodate the same.
> 
> Cc: stable@vger.kernel.org
> Fixes: 7af82ff90a2b06 ("powerpc/ftrace: Ignore weak functions")
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/include/asm/ftrace.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
> index 91c049d51d0e10..2edc6269b1a357 100644
> --- a/arch/powerpc/include/asm/ftrace.h
> +++ b/arch/powerpc/include/asm/ftrace.h
> @@ -12,7 +12,7 @@
>   
>   /* Ignore unused weak functions which will have larger offsets */
>   #ifdef CONFIG_MPROFILE_KERNEL
> -#define FTRACE_MCOUNT_MAX_OFFSET	12
> +#define FTRACE_MCOUNT_MAX_OFFSET	16
>   #elif defined(CONFIG_PPC32)
>   #define FTRACE_MCOUNT_MAX_OFFSET	8
>   #endif

Re: [PATCH 02/17] powerpc/module: Remove unused .ftrace.tramp section

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> .ftrace.tramp section is not used for any purpose. This code was added
> all the way back in the original commit introducing support for dynamic
> ftrace on ppc64 modules. Remove it.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/include/asm/module.h | 4 ----
>   1 file changed, 4 deletions(-)
> 
> diff --git a/arch/powerpc/include/asm/module.h b/arch/powerpc/include/asm/module.h
> index ac53606c259430..a8e2e8339fb7f4 100644
> --- a/arch/powerpc/include/asm/module.h
> +++ b/arch/powerpc/include/asm/module.h
> @@ -75,10 +75,6 @@ struct mod_arch_specific {
>   #endif
>   
>   #ifdef CONFIG_DYNAMIC_FTRACE
> -#    ifdef MODULE
> -	asm(".section .ftrace.tramp,\"ax\",@nobits; .align 3; .previous");
> -#    endif	/* MODULE */
> -
>   int module_trampoline_target(struct module *mod, unsigned long trampoline,
>   			     unsigned long *target);
>   int module_finalize_ftrace(struct module *mod, const Elf_Shdr *sechdrs);

Re: [PATCH 03/17] powerpc64/ftrace: Move ELFv1 and -pg support code into a separate file

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> ELFv1 support is deprecated and on the way out. Pre -mprofile-kernel
> ftrace support (-pg only) is very limited and is retained primarily for
> clang builds. It won't be necessary once clang lands support for
> -fpatchable-function-entry.
> 
> Copy the existing ftrace code supporting these into ftrace_pg.c.
> ftrace.c can then be refactored and enhanced with a focus on ppc32 and
> ppc64 ELFv2.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/Makefile    |  13 +-
>   arch/powerpc/kernel/trace/ftrace.c    |  10 -
>   arch/powerpc/kernel/trace/ftrace_pg.c | 846 ++++++++++++++++++++++++++
>   3 files changed, 855 insertions(+), 14 deletions(-)
>   create mode 100644 arch/powerpc/kernel/trace/ftrace_pg.c
> 
> diff --git a/arch/powerpc/kernel/trace/Makefile b/arch/powerpc/kernel/trace/Makefile
> index b16a9f9c0b35f2..342a2d1ae86cd0 100644
> --- a/arch/powerpc/kernel/trace/Makefile
> +++ b/arch/powerpc/kernel/trace/Makefile
> @@ -6,15 +6,16 @@
>   ifdef CONFIG_FUNCTION_TRACER
>   # do not trace tracer code
>   CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_ftrace_pg.o = $(CC_FLAGS_FTRACE)
>   endif
>   
> -obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o
> +obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
>   ifdef CONFIG_MPROFILE_KERNEL
> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o
> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
>   else
> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o
> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_pg.o
>   endif
> -obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o ftrace.o
> +obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o
>   obj-$(CONFIG_TRACING)			+= trace_clock.o
>   
>   obj-$(CONFIG_PPC64)			+= $(obj64-y)
> @@ -25,3 +26,7 @@ GCOV_PROFILE_ftrace.o := n
>   KCOV_INSTRUMENT_ftrace.o := n
>   KCSAN_SANITIZE_ftrace.o := n
>   UBSAN_SANITIZE_ftrace.o := n
> +GCOV_PROFILE_ftrace_pg.o := n
> +KCOV_INSTRUMENT_ftrace_pg.o := n
> +KCSAN_SANITIZE_ftrace_pg.o := n
> +UBSAN_SANITIZE_ftrace_pg.o := n
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index a47f303734233b..81a121b56c4d7f 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -864,13 +864,3 @@ unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
>   }
>   #endif
>   #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> -
> -#ifdef CONFIG_PPC64_ELF_ABI_V1
> -char *arch_ftrace_match_adjust(char *str, const char *search)
> -{
> -	if (str[0] == '.' && search[0] != '.')
> -		return str + 1;
> -	else
> -		return str;
> -}
> -#endif /* CONFIG_PPC64_ELF_ABI_V1 */
> diff --git a/arch/powerpc/kernel/trace/ftrace_pg.c b/arch/powerpc/kernel/trace/ftrace_pg.c
> new file mode 100644
> index 00000000000000..7b85c3b460a3c0
> --- /dev/null
> +++ b/arch/powerpc/kernel/trace/ftrace_pg.c
> @@ -0,0 +1,846 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Code for replacing ftrace calls with jumps.
> + *
> + * Copyright (C) 2007-2008 Steven Rostedt <srostedt@redhat.com>
> + *
> + * Thanks goes out to P.A. Semi, Inc for supplying me with a PPC64 box.
> + *
> + * Added function graph tracer code, taken from x86 that was written
> + * by Frederic Weisbecker, and ported to PPC by Steven Rostedt.
> + *
> + */
> +
> +#define pr_fmt(fmt) "ftrace-powerpc: " fmt
> +
> +#include <linux/spinlock.h>
> +#include <linux/hardirq.h>
> +#include <linux/uaccess.h>
> +#include <linux/module.h>
> +#include <linux/ftrace.h>
> +#include <linux/percpu.h>
> +#include <linux/init.h>
> +#include <linux/list.h>
> +
> +#include <asm/cacheflush.h>
> +#include <asm/code-patching.h>
> +#include <asm/ftrace.h>
> +#include <asm/syscall.h>
> +#include <asm/inst.h>
> +
> +/*
> + * We generally only have a single long_branch tramp and at most 2 or 3 plt
> + * tramps generated. But, we don't use the plt tramps currently. We also allot
> + * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
> + * tramps in total. Set aside 8 just to be sure.
> + */
> +#define	NUM_FTRACE_TRAMPS	8
> +static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
> +
> +static ppc_inst_t
> +ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
> +{
> +	ppc_inst_t op;
> +
> +	addr = ppc_function_entry((void *)addr);
> +
> +	/* if (link) set op to 'bl' else 'b' */
> +	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
> +
> +	return op;
> +}
> +
> +static inline int
> +ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
> +{
> +	ppc_inst_t replaced;
> +
> +	/*
> +	 * Note:
> +	 * We are paranoid about modifying text, as if a bug was to happen, it
> +	 * could cause us to read or write to someplace that could cause harm.
> +	 * Carefully read and modify the code with probe_kernel_*(), and make
> +	 * sure what we read is what we expected it to be before modifying it.
> +	 */
> +
> +	/* read the text we want to modify */
> +	if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
> +		return -EFAULT;
> +
> +	/* Make sure it is what we expect it to be */
> +	if (!ppc_inst_equal(replaced, old)) {
> +		pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
> +		       ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
> +		return -EINVAL;
> +	}
> +
> +	/* replace the text with the new text */
> +	return patch_instruction((u32 *)ip, new);
> +}
> +
> +/*
> + * Helper functions that are the same for both PPC64 and PPC32.
> + */
> +static int test_24bit_addr(unsigned long ip, unsigned long addr)
> +{
> +	addr = ppc_function_entry((void *)addr);
> +
> +	return is_offset_in_branch_range(addr - ip);
> +}
> +
> +static int is_bl_op(ppc_inst_t op)
> +{
> +	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
> +}
> +
> +static int is_b_op(ppc_inst_t op)
> +{
> +	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
> +}
> +
> +static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
> +{
> +	int offset;
> +
> +	offset = PPC_LI(ppc_inst_val(op));
> +	/* make it signed */
> +	if (offset & 0x02000000)
> +		offset |= 0xfe000000;
> +
> +	return ip + (long)offset;
> +}
> +
> +#ifdef CONFIG_MODULES
> +static int
> +__ftrace_make_nop(struct module *mod,
> +		  struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long entry, ptr, tramp;
> +	unsigned long ip = rec->ip;
> +	ppc_inst_t op, pop;
> +
> +	/* read where this goes */
> +	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> +		pr_err("Fetching opcode failed.\n");
> +		return -EFAULT;
> +	}
> +
> +	/* Make sure that this is still a 24bit jump */
> +	if (!is_bl_op(op)) {
> +		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> +		return -EINVAL;
> +	}
> +
> +	/* lets find where the pointer goes */
> +	tramp = find_bl_target(ip, op);
> +
> +	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> +
> +	if (module_trampoline_target(mod, tramp, &ptr)) {
> +		pr_err("Failed to get trampoline target\n");
> +		return -EFAULT;
> +	}
> +
> +	pr_devel("trampoline target %lx", ptr);
> +
> +	entry = ppc_global_function_entry((void *)addr);
> +	/* This should match what was called */
> +	if (ptr != entry) {
> +		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> +		return -EINVAL;
> +	}
> +
> +	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
> +		if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
> +			pr_err("Fetching instruction at %lx failed.\n", ip - 4);
> +			return -EFAULT;
> +		}
> +
> +		/* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
> +		if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
> +		    !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
> +			pr_err("Unexpected instruction %08lx around bl _mcount\n",
> +			       ppc_inst_as_ulong(op));
> +			return -EINVAL;
> +		}
> +	} else if (IS_ENABLED(CONFIG_PPC64)) {
> +		/*
> +		 * Check what is in the next instruction. We can see ld r2,40(r1), but
> +		 * on first pass after boot we will see mflr r0.
> +		 */
> +		if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
> +			pr_err("Fetching op failed.\n");
> +			return -EFAULT;
> +		}
> +
> +		if (!ppc_inst_equal(op,  ppc_inst(PPC_INST_LD_TOC))) {
> +			pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
> +			       ppc_inst_as_ulong(op));
> +			return -EINVAL;
> +		}
> +	}
> +
> +	/*
> +	 * When using -mprofile-kernel or PPC32 there is no load to jump over.
> +	 *
> +	 * Otherwise our original call site looks like:
> +	 *
> +	 * bl <tramp>
> +	 * ld r2,XX(r1)
> +	 *
> +	 * Milton Miller pointed out that we can not simply nop the branch.
> +	 * If a task was preempted when calling a trace function, the nops
> +	 * will remove the way to restore the TOC in r2 and the r2 TOC will
> +	 * get corrupted.
> +	 *
> +	 * Use a b +8 to jump over the load.
> +	 */
> +	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
> +		pop = ppc_inst(PPC_RAW_NOP());
> +	else
> +		pop = ppc_inst(PPC_RAW_BRANCH(8));	/* b +8 */
> +
> +	if (patch_instruction((u32 *)ip, pop)) {
> +		pr_err("Patching NOP failed.\n");
> +		return -EPERM;
> +	}
> +
> +	return 0;
> +}
> +#else
> +static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	return 0;
> +}
> +#endif /* CONFIG_MODULES */
> +
> +static unsigned long find_ftrace_tramp(unsigned long ip)
> +{
> +	int i;
> +
> +	/*
> +	 * We have the compiler generated long_branch tramps at the end
> +	 * and we prefer those
> +	 */
> +	for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
> +		if (!ftrace_tramps[i])
> +			continue;
> +		else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
> +			return ftrace_tramps[i];
> +
> +	return 0;
> +}
> +
> +static int add_ftrace_tramp(unsigned long tramp)
> +{
> +	int i;
> +
> +	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
> +		if (!ftrace_tramps[i]) {
> +			ftrace_tramps[i] = tramp;
> +			return 0;
> +		}
> +
> +	return -1;
> +}
> +
> +/*
> + * If this is a compiler generated long_branch trampoline (essentially, a
> + * trampoline that has a branch to _mcount()), we re-write the branch to
> + * instead go to ftrace_[regs_]caller() and note down the location of this
> + * trampoline.
> + */
> +static int setup_mcount_compiler_tramp(unsigned long tramp)
> +{
> +	int i;
> +	ppc_inst_t op;
> +	unsigned long ptr;
> +
> +	/* Is this a known long jump tramp? */
> +	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
> +		if (ftrace_tramps[i] == tramp)
> +			return 0;
> +
> +	/* New trampoline -- read where this goes */
> +	if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
> +		pr_debug("Fetching opcode failed.\n");
> +		return -1;
> +	}
> +
> +	/* Is this a 24 bit branch? */
> +	if (!is_b_op(op)) {
> +		pr_debug("Trampoline is not a long branch tramp.\n");
> +		return -1;
> +	}
> +
> +	/* lets find where the pointer goes */
> +	ptr = find_bl_target(tramp, op);
> +
> +	if (ptr != ppc_global_function_entry((void *)_mcount)) {
> +		pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
> +		return -1;
> +	}
> +
> +	/* Let's re-write the tramp to go to ftrace_[regs_]caller */
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> +		ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
> +	else
> +		ptr = ppc_global_function_entry((void *)ftrace_caller);
> +
> +	if (patch_branch((u32 *)tramp, ptr, 0)) {
> +		pr_debug("REL24 out of range!\n");
> +		return -1;
> +	}
> +
> +	if (add_ftrace_tramp(tramp)) {
> +		pr_debug("No tramp locations left\n");
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long tramp, ip = rec->ip;
> +	ppc_inst_t op;
> +
> +	/* Read where this goes */
> +	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> +		pr_err("Fetching opcode failed.\n");
> +		return -EFAULT;
> +	}
> +
> +	/* Make sure that this is still a 24bit jump */
> +	if (!is_bl_op(op)) {
> +		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> +		return -EINVAL;
> +	}
> +
> +	/* Let's find where the pointer goes */
> +	tramp = find_bl_target(ip, op);
> +
> +	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> +
> +	if (setup_mcount_compiler_tramp(tramp)) {
> +		/* Are other trampolines reachable? */
> +		if (!find_ftrace_tramp(ip)) {
> +			pr_err("No ftrace trampolines reachable from %ps\n",
> +					(void *)ip);
> +			return -EINVAL;
> +		}
> +	}
> +
> +	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
> +		pr_err("Patching NOP failed.\n");
> +		return -EPERM;
> +	}
> +
> +	return 0;
> +}
> +
> +int ftrace_make_nop(struct module *mod,
> +		    struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long ip = rec->ip;
> +	ppc_inst_t old, new;
> +
> +	/*
> +	 * If the calling address is more that 24 bits away,
> +	 * then we had to use a trampoline to make the call.
> +	 * Otherwise just update the call site.
> +	 */
> +	if (test_24bit_addr(ip, addr)) {
> +		/* within range */
> +		old = ftrace_call_replace(ip, addr, 1);
> +		new = ppc_inst(PPC_RAW_NOP());
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip)) {
> +		return __ftrace_make_nop_kernel(rec, addr);
> +	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Out of range jumps are called from modules.
> +	 * We should either already have a pointer to the module
> +	 * or it has been passed in.
> +	 */
> +	if (!rec->arch.mod) {
> +		if (!mod) {
> +			pr_err("No module loaded addr=%lx\n", addr);
> +			return -EFAULT;
> +		}
> +		rec->arch.mod = mod;
> +	} else if (mod) {
> +		if (mod != rec->arch.mod) {
> +			pr_err("Record mod %p not equal to passed in mod %p\n",
> +			       rec->arch.mod, mod);
> +			return -EINVAL;
> +		}
> +		/* nothing to do if mod == rec->arch.mod */
> +	} else
> +		mod = rec->arch.mod;
> +
> +	return __ftrace_make_nop(mod, rec, addr);
> +}
> +
> +#ifdef CONFIG_MODULES
> +/*
> + * Examine the existing instructions for __ftrace_make_call.
> + * They should effectively be a NOP, and follow formal constraints,
> + * depending on the ABI. Return false if they don't.
> + */
> +static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
> +{
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> +		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
> +	else
> +		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
> +		       ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
> +}
> +
> +static int
> +__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	ppc_inst_t op[2];
> +	void *ip = (void *)rec->ip;
> +	unsigned long entry, ptr, tramp;
> +	struct module *mod = rec->arch.mod;
> +
> +	/* read where this goes */
> +	if (copy_inst_from_kernel_nofault(op, ip))
> +		return -EFAULT;
> +
> +	if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
> +	    copy_inst_from_kernel_nofault(op + 1, ip + 4))
> +		return -EFAULT;
> +
> +	if (!expected_nop_sequence(ip, op[0], op[1])) {
> +		pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
> +		       ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
> +		return -EINVAL;
> +	}
> +
> +	/* If we never set up ftrace trampoline(s), then bail */
> +	if (!mod->arch.tramp ||
> +	    (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
> +		pr_err("No ftrace trampoline\n");
> +		return -EINVAL;
> +	}
> +
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
> +		tramp = mod->arch.tramp_regs;
> +	else
> +		tramp = mod->arch.tramp;
> +
> +	if (module_trampoline_target(mod, tramp, &ptr)) {
> +		pr_err("Failed to get trampoline target\n");
> +		return -EFAULT;
> +	}
> +
> +	pr_devel("trampoline target %lx", ptr);
> +
> +	entry = ppc_global_function_entry((void *)addr);
> +	/* This should match what was called */
> +	if (ptr != entry) {
> +		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> +		return -EINVAL;
> +	}
> +
> +	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
> +		pr_err("REL24 out of range!\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +#else
> +static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	return 0;
> +}
> +#endif /* CONFIG_MODULES */
> +
> +static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	ppc_inst_t op;
> +	void *ip = (void *)rec->ip;
> +	unsigned long tramp, entry, ptr;
> +
> +	/* Make sure we're being asked to patch branch to a known ftrace addr */
> +	entry = ppc_global_function_entry((void *)ftrace_caller);
> +	ptr = ppc_global_function_entry((void *)addr);
> +
> +	if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> +		entry = ppc_global_function_entry((void *)ftrace_regs_caller);
> +
> +	if (ptr != entry) {
> +		pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
> +		return -EINVAL;
> +	}
> +
> +	/* Make sure we have a nop */
> +	if (copy_inst_from_kernel_nofault(&op, ip)) {
> +		pr_err("Unable to read ftrace location %p\n", ip);
> +		return -EFAULT;
> +	}
> +
> +	if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
> +		pr_err("Unexpected call sequence at %p: %08lx\n",
> +		       ip, ppc_inst_as_ulong(op));
> +		return -EINVAL;
> +	}
> +
> +	tramp = find_ftrace_tramp((unsigned long)ip);
> +	if (!tramp) {
> +		pr_err("No ftrace trampolines reachable from %ps\n", ip);
> +		return -EINVAL;
> +	}
> +
> +	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
> +		pr_err("Error patching branch to ftrace tramp!\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long ip = rec->ip;
> +	ppc_inst_t old, new;
> +
> +	/*
> +	 * If the calling address is more that 24 bits away,
> +	 * then we had to use a trampoline to make the call.
> +	 * Otherwise just update the call site.
> +	 */
> +	if (test_24bit_addr(ip, addr)) {
> +		/* within range */
> +		old = ppc_inst(PPC_RAW_NOP());
> +		new = ftrace_call_replace(ip, addr, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip)) {
> +		return __ftrace_make_call_kernel(rec, addr);
> +	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> +		/* We should not get here without modules */
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Out of range jumps are called from modules.
> +	 * Being that we are converting from nop, it had better
> +	 * already have a module defined.
> +	 */
> +	if (!rec->arch.mod) {
> +		pr_err("No module loaded\n");
> +		return -EINVAL;
> +	}
> +
> +	return __ftrace_make_call(rec, addr);
> +}
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> +#ifdef CONFIG_MODULES
> +static int
> +__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> +					unsigned long addr)
> +{
> +	ppc_inst_t op;
> +	unsigned long ip = rec->ip;
> +	unsigned long entry, ptr, tramp;
> +	struct module *mod = rec->arch.mod;
> +
> +	/* If we never set up ftrace trampolines, then bail */
> +	if (!mod->arch.tramp || !mod->arch.tramp_regs) {
> +		pr_err("No ftrace trampoline\n");
> +		return -EINVAL;
> +	}
> +
> +	/* read where this goes */
> +	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> +		pr_err("Fetching opcode failed.\n");
> +		return -EFAULT;
> +	}
> +
> +	/* Make sure that this is still a 24bit jump */
> +	if (!is_bl_op(op)) {
> +		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> +		return -EINVAL;
> +	}
> +
> +	/* lets find where the pointer goes */
> +	tramp = find_bl_target(ip, op);
> +	entry = ppc_global_function_entry((void *)old_addr);
> +
> +	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> +
> +	if (tramp != entry) {
> +		/* old_addr is not within range, so we must have used a trampoline */
> +		if (module_trampoline_target(mod, tramp, &ptr)) {
> +			pr_err("Failed to get trampoline target\n");
> +			return -EFAULT;
> +		}
> +
> +		pr_devel("trampoline target %lx", ptr);
> +
> +		/* This should match what was called */
> +		if (ptr != entry) {
> +			pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> +			return -EINVAL;
> +		}
> +	}
> +
> +	/* The new target may be within range */
> +	if (test_24bit_addr(ip, addr)) {
> +		/* within range */
> +		if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
> +			pr_err("REL24 out of range!\n");
> +			return -EINVAL;
> +		}
> +
> +		return 0;
> +	}
> +
> +	if (rec->flags & FTRACE_FL_REGS)
> +		tramp = mod->arch.tramp_regs;
> +	else
> +		tramp = mod->arch.tramp;
> +
> +	if (module_trampoline_target(mod, tramp, &ptr)) {
> +		pr_err("Failed to get trampoline target\n");
> +		return -EFAULT;
> +	}
> +
> +	pr_devel("trampoline target %lx", ptr);
> +
> +	entry = ppc_global_function_entry((void *)addr);
> +	/* This should match what was called */
> +	if (ptr != entry) {
> +		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> +		return -EINVAL;
> +	}
> +
> +	if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
> +		pr_err("REL24 out of range!\n");
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +#else
> +static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
> +{
> +	return 0;
> +}
> +#endif
> +
> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> +			unsigned long addr)
> +{
> +	unsigned long ip = rec->ip;
> +	ppc_inst_t old, new;
> +
> +	/*
> +	 * If the calling address is more that 24 bits away,
> +	 * then we had to use a trampoline to make the call.
> +	 * Otherwise just update the call site.
> +	 */
> +	if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
> +		/* within range */
> +		old = ftrace_call_replace(ip, old_addr, 1);
> +		new = ftrace_call_replace(ip, addr, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip)) {
> +		/*
> +		 * We always patch out of range locations to go to the regs
> +		 * variant, so there is nothing to do here
> +		 */
> +		return 0;
> +	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> +		/* We should not get here without modules */
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * Out of range jumps are called from modules.
> +	 */
> +	if (!rec->arch.mod) {
> +		pr_err("No module loaded\n");
> +		return -EINVAL;
> +	}
> +
> +	return __ftrace_modify_call(rec, old_addr, addr);
> +}
> +#endif
> +
> +int ftrace_update_ftrace_func(ftrace_func_t func)
> +{
> +	unsigned long ip = (unsigned long)(&ftrace_call);
> +	ppc_inst_t old, new;
> +	int ret;
> +
> +	old = ppc_inst_read((u32 *)&ftrace_call);
> +	new = ftrace_call_replace(ip, (unsigned long)func, 1);
> +	ret = ftrace_modify_code(ip, old, new);
> +
> +	/* Also update the regs callback function */
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
> +		ip = (unsigned long)(&ftrace_regs_call);
> +		old = ppc_inst_read((u32 *)&ftrace_regs_call);
> +		new = ftrace_call_replace(ip, (unsigned long)func, 1);
> +		ret = ftrace_modify_code(ip, old, new);
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * Use the default ftrace_modify_all_code, but without
> + * stop_machine().
> + */
> +void arch_ftrace_update_code(int command)
> +{
> +	ftrace_modify_all_code(command);
> +}
> +
> +#ifdef CONFIG_PPC64
> +#define PACATOC offsetof(struct paca_struct, kernel_toc)
> +
> +extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
> +
> +void ftrace_free_init_tramp(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < NUM_FTRACE_TRAMPS && ftrace_tramps[i]; i++)
> +		if (ftrace_tramps[i] == (unsigned long)ftrace_tramp_init) {
> +			ftrace_tramps[i] = 0;
> +			return;
> +		}
> +}
> +
> +int __init ftrace_dyn_arch_init(void)
> +{
> +	int i;
> +	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
> +	u32 stub_insns[] = {
> +		PPC_RAW_LD(_R12, _R13, PACATOC),
> +		PPC_RAW_ADDIS(_R12, _R12, 0),
> +		PPC_RAW_ADDI(_R12, _R12, 0),
> +		PPC_RAW_MTCTR(_R12),
> +		PPC_RAW_BCTR()
> +	};
> +	unsigned long addr;
> +	long reladdr;
> +
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> +		addr = ppc_global_function_entry((void *)ftrace_regs_caller);
> +	else
> +		addr = ppc_global_function_entry((void *)ftrace_caller);
> +
> +	reladdr = addr - kernel_toc_addr();
> +
> +	if (reladdr >= SZ_2G || reladdr < -(long)SZ_2G) {
> +		pr_err("Address of %ps out of range of kernel_toc.\n",
> +				(void *)addr);
> +		return -1;
> +	}
> +
> +	for (i = 0; i < 2; i++) {
> +		memcpy(tramp[i], stub_insns, sizeof(stub_insns));
> +		tramp[i][1] |= PPC_HA(reladdr);
> +		tramp[i][2] |= PPC_LO(reladdr);
> +		add_ftrace_tramp((unsigned long)tramp[i]);
> +	}
> +
> +	return 0;
> +}
> +#endif
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +
> +extern void ftrace_graph_call(void);
> +extern void ftrace_graph_stub(void);
> +
> +static int ftrace_modify_ftrace_graph_caller(bool enable)
> +{
> +	unsigned long ip = (unsigned long)(&ftrace_graph_call);
> +	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
> +	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
> +	ppc_inst_t old, new;
> +
> +	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
> +		return 0;
> +
> +	old = ftrace_call_replace(ip, enable ? stub : addr, 0);
> +	new = ftrace_call_replace(ip, enable ? addr : stub, 0);
> +
> +	return ftrace_modify_code(ip, old, new);
> +}
> +
> +int ftrace_enable_ftrace_graph_caller(void)
> +{
> +	return ftrace_modify_ftrace_graph_caller(true);
> +}
> +
> +int ftrace_disable_ftrace_graph_caller(void)
> +{
> +	return ftrace_modify_ftrace_graph_caller(false);
> +}
> +
> +/*
> + * Hook the return address and push it in the stack of return addrs
> + * in current thread info. Return the address we want to divert to.
> + */
> +static unsigned long
> +__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
> +{
> +	unsigned long return_hooker;
> +	int bit;
> +
> +	if (unlikely(ftrace_graph_is_dead()))
> +		goto out;
> +
> +	if (unlikely(atomic_read(&current->tracing_graph_pause)))
> +		goto out;
> +
> +	bit = ftrace_test_recursion_trylock(ip, parent);
> +	if (bit < 0)
> +		goto out;
> +
> +	return_hooker = ppc_function_entry(return_to_handler);
> +
> +	if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
> +		parent = return_hooker;
> +
> +	ftrace_test_recursion_unlock(bit);
> +out:
> +	return parent;
> +}
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
> +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
> +		       struct ftrace_ops *op, struct ftrace_regs *fregs)
> +{
> +	fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
> +}
> +#else
> +unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
> +				    unsigned long sp)
> +{
> +	return __prepare_ftrace_return(parent, ip, sp);
> +}
> +#endif
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
> +#ifdef CONFIG_PPC64_ELF_ABI_V1
> +char *arch_ftrace_match_adjust(char *str, const char *search)
> +{
> +	if (str[0] == '.' && search[0] != '.')
> +		return str + 1;
> +	else
> +		return str;
> +}
> +#endif /* CONFIG_PPC64_ELF_ABI_V1 */

Re: [PATCH 04/17] powerpc/ftrace: Simplify function_graph support in ftrace.c

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Since we now support DYNAMIC_FTRACE_WITH_ARGS across ppc32 and ppc64
> ELFv2, we can simplify function_graph tracer support code in ftrace.c
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 64 ++++--------------------------
>   1 file changed, 7 insertions(+), 57 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 81a121b56c4d7f..f117124c30325f 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -790,44 +790,10 @@ int __init ftrace_dyn_arch_init(void)
>   #endif
>   
>   #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> -
> -extern void ftrace_graph_call(void);
> -extern void ftrace_graph_stub(void);
> -
> -static int ftrace_modify_ftrace_graph_caller(bool enable)
> +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
> +		       struct ftrace_ops *op, struct ftrace_regs *fregs)
>   {
> -	unsigned long ip = (unsigned long)(&ftrace_graph_call);
> -	unsigned long addr = (unsigned long)(&ftrace_graph_caller);
> -	unsigned long stub = (unsigned long)(&ftrace_graph_stub);
> -	ppc_inst_t old, new;
> -
> -	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
> -		return 0;
> -
> -	old = ftrace_call_replace(ip, enable ? stub : addr, 0);
> -	new = ftrace_call_replace(ip, enable ? addr : stub, 0);
> -
> -	return ftrace_modify_code(ip, old, new);
> -}
> -
> -int ftrace_enable_ftrace_graph_caller(void)
> -{
> -	return ftrace_modify_ftrace_graph_caller(true);
> -}
> -
> -int ftrace_disable_ftrace_graph_caller(void)
> -{
> -	return ftrace_modify_ftrace_graph_caller(false);
> -}
> -
> -/*
> - * Hook the return address and push it in the stack of return addrs
> - * in current thread info. Return the address we want to divert to.
> - */
> -static unsigned long
> -__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
> -{
> -	unsigned long return_hooker;
> +	unsigned long sp = fregs->regs.gpr[1];
>   	int bit;
>   
>   	if (unlikely(ftrace_graph_is_dead()))
> @@ -836,31 +802,15 @@ __prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp
>   	if (unlikely(atomic_read(&current->tracing_graph_pause)))
>   		goto out;
>   
> -	bit = ftrace_test_recursion_trylock(ip, parent);
> +	bit = ftrace_test_recursion_trylock(ip, parent_ip);
>   	if (bit < 0)
>   		goto out;
>   
> -	return_hooker = ppc_function_entry(return_to_handler);
> -
> -	if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
> -		parent = return_hooker;
> +	if (!function_graph_enter(parent_ip, ip, 0, (unsigned long *)sp))
> +		parent_ip = ppc_function_entry(return_to_handler);
>   
>   	ftrace_test_recursion_unlock(bit);
>   out:
> -	return parent;
> +	fregs->regs.link = parent_ip;
>   }
> -
> -#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
> -void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
> -		       struct ftrace_ops *op, struct ftrace_regs *fregs)
> -{
> -	fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
> -}
> -#else
> -unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
> -				    unsigned long sp)
> -{
> -	return __prepare_ftrace_return(parent, ip, sp);
> -}
> -#endif
>   #endif /* CONFIG_FUNCTION_GRAPH_TRACER */

Re: [PATCH 05/17] powerpc/ftrace: Use FTRACE_REGS_ADDR to identify the correct ftrace trampoline

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Instead of keying off DYNAMIC_FTRACE_WITH_REGS, use FTRACE_REGS_ADDR to
> identify the proper ftrace trampoline address to use.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 7 +------
>   1 file changed, 1 insertion(+), 6 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index f117124c30325f..5aa36272617a03 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -745,14 +745,9 @@ int __init ftrace_dyn_arch_init(void)
>   	};
>   #endif
>   
> -	unsigned long addr;
> +	unsigned long addr = FTRACE_REGS_ADDR;
>   	long reladdr;
>   
> -	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> -		addr = ppc_global_function_entry((void *)ftrace_regs_caller);
> -	else
> -		addr = ppc_global_function_entry((void *)ftrace_caller);
> -
>   	if (IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) {
>   		for (i = 0; i < 2; i++) {
>   			reladdr = addr - (unsigned long)tramp[i];

Re: [PATCH 06/17] powerpc/ftrace: Extend ftrace support for large kernels to ppc32

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Commit 67361cf8071286 ("powerpc/ftrace: Handle large kernel configs")
> added ftrace support for ppc64 kernel images with a text section larger
> than 32MB. The approach itself isn't specific to ppc64, so extend the
> same to also work on ppc32.
> 
> While at it, reduce the space reserved for the stub from 64 bytes to 32
> bytes since the different stub variants are all less than 8
> instructions.
> 
> To reduce use of #ifdef, a stub implementation is provided for
> kernel_toc_address() and -SZ_2G is cast to 'long long' to prevent
> errors on ppc32.
> 

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> Signed-off-by: Naveen N Rao <naveen@kernel.org>
> ---
>   arch/powerpc/include/asm/ftrace.h      | 10 +++++--
>   arch/powerpc/include/asm/sections.h    |  2 ++
>   arch/powerpc/kernel/trace/ftrace.c     | 39 ++++++++++++++------------
>   arch/powerpc/kernel/trace/ftrace_low.S |  6 ++--
>   arch/powerpc/kernel/vmlinux.lds.S      |  4 ---
>   5 files changed, 32 insertions(+), 29 deletions(-)
> 
> diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
> index 2edc6269b1a357..702aaf2efa966c 100644
> --- a/arch/powerpc/include/asm/ftrace.h
> +++ b/arch/powerpc/include/asm/ftrace.h
> @@ -124,15 +124,19 @@ static inline u8 this_cpu_get_ftrace_enabled(void)
>   {
>   	return get_paca()->ftrace_enabled;
>   }
> -
> -void ftrace_free_init_tramp(void);
>   #else /* CONFIG_PPC64 */
>   static inline void this_cpu_disable_ftrace(void) { }
>   static inline void this_cpu_enable_ftrace(void) { }
>   static inline void this_cpu_set_ftrace_enabled(u8 ftrace_enabled) { }
>   static inline u8 this_cpu_get_ftrace_enabled(void) { return 1; }
> -static inline void ftrace_free_init_tramp(void) { }
>   #endif /* CONFIG_PPC64 */
> +
> +#ifdef CONFIG_FUNCTION_TRACER
> +extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
> +void ftrace_free_init_tramp(void);
> +#else
> +static inline void ftrace_free_init_tramp(void) { }
> +#endif
>   #endif /* !__ASSEMBLY__ */
>   
>   #endif /* _ASM_POWERPC_FTRACE */
> diff --git a/arch/powerpc/include/asm/sections.h b/arch/powerpc/include/asm/sections.h
> index 4e1f548c8d373d..ea26665f82cfc8 100644
> --- a/arch/powerpc/include/asm/sections.h
> +++ b/arch/powerpc/include/asm/sections.h
> @@ -74,6 +74,8 @@ static inline int overlaps_kernel_text(unsigned long start, unsigned long end)
>   		(unsigned long)_stext < end;
>   }
>   
> +#else
> +static inline unsigned long kernel_toc_addr(void) { BUILD_BUG(); return -1UL; }
>   #endif
>   
>   #endif /* __KERNEL__ */
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 5aa36272617a03..913c7aa63d3fa3 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -707,11 +707,6 @@ void arch_ftrace_update_code(int command)
>   	ftrace_modify_all_code(command);
>   }
>   
> -#ifdef CONFIG_PPC64
> -#define PACATOC offsetof(struct paca_struct, kernel_toc)
> -
> -extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
> -
>   void ftrace_free_init_tramp(void)
>   {
>   	int i;
> @@ -725,28 +720,30 @@ void ftrace_free_init_tramp(void)
>   
>   int __init ftrace_dyn_arch_init(void)
>   {
> -	int i;
>   	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
> -#ifdef CONFIG_PPC_KERNEL_PCREL
> +	unsigned long addr = FTRACE_REGS_ADDR;
> +	long reladdr;
> +	int i;
>   	u32 stub_insns[] = {
> +#ifdef CONFIG_PPC_KERNEL_PCREL
>   		/* pla r12,addr */
>   		PPC_PREFIX_MLS | __PPC_PRFX_R(1),
>   		PPC_INST_PADDI | ___PPC_RT(_R12),
>   		PPC_RAW_MTCTR(_R12),
>   		PPC_RAW_BCTR()
> -	};
> -#else
> -	u32 stub_insns[] = {
> -		PPC_RAW_LD(_R12, _R13, PACATOC),
> +#elif defined(CONFIG_PPC64)
> +		PPC_RAW_LD(_R12, _R13, offsetof(struct paca_struct, kernel_toc)),
>   		PPC_RAW_ADDIS(_R12, _R12, 0),
>   		PPC_RAW_ADDI(_R12, _R12, 0),
>   		PPC_RAW_MTCTR(_R12),
>   		PPC_RAW_BCTR()
> -	};
> +#else
> +		PPC_RAW_LIS(_R12, 0),
> +		PPC_RAW_ADDI(_R12, _R12, 0),
> +		PPC_RAW_MTCTR(_R12),
> +		PPC_RAW_BCTR()
>   #endif
> -
> -	unsigned long addr = FTRACE_REGS_ADDR;
> -	long reladdr;
> +	};
>   
>   	if (IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) {
>   		for (i = 0; i < 2; i++) {
> @@ -763,10 +760,10 @@ int __init ftrace_dyn_arch_init(void)
>   			tramp[i][1] |= IMM_L(reladdr);
>   			add_ftrace_tramp((unsigned long)tramp[i]);
>   		}
> -	} else {
> +	} else if (IS_ENABLED(CONFIG_PPC64)) {
>   		reladdr = addr - kernel_toc_addr();
>   
> -		if (reladdr >= (long)SZ_2G || reladdr < -(long)SZ_2G) {
> +		if (reladdr >= (long)SZ_2G || reladdr < -(long long)SZ_2G) {
>   			pr_err("Address of %ps out of range of kernel_toc.\n",
>   				(void *)addr);
>   			return -1;
> @@ -778,11 +775,17 @@ int __init ftrace_dyn_arch_init(void)
>   			tramp[i][2] |= PPC_LO(reladdr);
>   			add_ftrace_tramp((unsigned long)tramp[i]);
>   		}
> +	} else {
> +		for (i = 0; i < 2; i++) {
> +			memcpy(tramp[i], stub_insns, sizeof(stub_insns));
> +			tramp[i][0] |= PPC_HA(addr);
> +			tramp[i][1] |= PPC_LO(addr);
> +			add_ftrace_tramp((unsigned long)tramp[i]);
> +		}
>   	}
>   
>   	return 0;
>   }
> -#endif
>   
>   #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>   void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
> diff --git a/arch/powerpc/kernel/trace/ftrace_low.S b/arch/powerpc/kernel/trace/ftrace_low.S
> index 294d1e05958aae..2fc7dd0a5ae968 100644
> --- a/arch/powerpc/kernel/trace/ftrace_low.S
> +++ b/arch/powerpc/kernel/trace/ftrace_low.S
> @@ -10,19 +10,17 @@
>   #include <asm/ppc-opcode.h>
>   #include <asm/export.h>
>   
> -#ifdef CONFIG_PPC64
>   .pushsection ".tramp.ftrace.text","aw",@progbits;
>   .globl ftrace_tramp_text
>   ftrace_tramp_text:
> -	.space 64
> +	.space 32
>   .popsection
>   
>   .pushsection ".tramp.ftrace.init","aw",@progbits;
>   .globl ftrace_tramp_init
>   ftrace_tramp_init:
> -	.space 64
> +	.space 32
>   .popsection
> -#endif
>   
>   _GLOBAL(mcount)
>   _GLOBAL(_mcount)
> diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
> index 13614f0b269cf4..1c5970df323366 100644
> --- a/arch/powerpc/kernel/vmlinux.lds.S
> +++ b/arch/powerpc/kernel/vmlinux.lds.S
> @@ -107,9 +107,7 @@ SECTIONS
>   #endif
>   		/* careful! __ftr_alt_* sections need to be close to .text */
>   		*(.text.hot .text.hot.* TEXT_MAIN .text.fixup .text.unlikely .text.unlikely.* .fixup __ftr_alt_* .ref.text);
> -#ifdef CONFIG_PPC64
>   		*(.tramp.ftrace.text);
> -#endif
>   		NOINSTR_TEXT
>   		SCHED_TEXT
>   		LOCK_TEXT
> @@ -276,9 +274,7 @@ SECTIONS
>   		 */
>   		. = ALIGN(PAGE_SIZE);
>   		_einittext = .;
> -#ifdef CONFIG_PPC64
>   		*(.tramp.ftrace.init);
> -#endif
>   	} :text
>   
>   	/* .exit.text is discarded at runtime, not link time,

Re: [PATCH 07/17] powerpc/ftrace: Consolidate ftrace support into fewer files

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> ftrace_low.S has just the _mcount stub and return_to_handler(). Merge
> this back into ftrace_mprofile.S and ftrace_64_pg.S to keep all ftrace
> code together, and to allow those to evolve independently.
> 
> ftrace_mprofile.S is also not an entirely accurate name since this also
> holds ppc32 code. This will be all the more incorrect once support for
> -fpatchable-function-entry is added. Rename files here to more
> accurately describe the code:
> - ftrace_mprofile.S is renamed to ftrace_entry.S
> - ftrace_pg.c is renamed to ftrace_64_pg.c
> - ftrace_64_pg.S is rename to ftrace_64_pg_entry.S

The stats from git do not match., it says {ftrace_low.S => 
ftrace_64_pg_entry.S

> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/Makefile            | 17 +++--
>   arch/powerpc/kernel/trace/ftrace_64_pg.S      | 67 -------------------
>   .../trace/{ftrace_pg.c => ftrace_64_pg.c}     |  0
>   .../{ftrace_low.S => ftrace_64_pg_entry.S}    | 58 +++++++++++++++-
>   .../{ftrace_mprofile.S => ftrace_entry.S}     | 65 ++++++++++++++++++
>   5 files changed, 130 insertions(+), 77 deletions(-)
>   delete mode 100644 arch/powerpc/kernel/trace/ftrace_64_pg.S
>   rename arch/powerpc/kernel/trace/{ftrace_pg.c => ftrace_64_pg.c} (100%)
>   rename arch/powerpc/kernel/trace/{ftrace_low.S => ftrace_64_pg_entry.S} (55%)
>   rename arch/powerpc/kernel/trace/{ftrace_mprofile.S => ftrace_entry.S} (83%)
> 
> diff --git a/arch/powerpc/kernel/trace/Makefile b/arch/powerpc/kernel/trace/Makefile
> index 342a2d1ae86cd0..125f4ca588b98a 100644
> --- a/arch/powerpc/kernel/trace/Makefile
> +++ b/arch/powerpc/kernel/trace/Makefile
> @@ -6,16 +6,15 @@
>   ifdef CONFIG_FUNCTION_TRACER
>   # do not trace tracer code
>   CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
> -CFLAGS_REMOVE_ftrace_pg.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_ftrace_64_pg.o = $(CC_FLAGS_FTRACE)
>   endif
>   
> -obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
> +obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
>   ifdef CONFIG_MPROFILE_KERNEL
> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
>   else
> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_pg.o
> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_64_pg_entry.o
>   endif
> -obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o
>   obj-$(CONFIG_TRACING)			+= trace_clock.o
>   
>   obj-$(CONFIG_PPC64)			+= $(obj64-y)
> @@ -26,7 +25,7 @@ GCOV_PROFILE_ftrace.o := n
>   KCOV_INSTRUMENT_ftrace.o := n
>   KCSAN_SANITIZE_ftrace.o := n
>   UBSAN_SANITIZE_ftrace.o := n
> -GCOV_PROFILE_ftrace_pg.o := n
> -KCOV_INSTRUMENT_ftrace_pg.o := n
> -KCSAN_SANITIZE_ftrace_pg.o := n
> -UBSAN_SANITIZE_ftrace_pg.o := n
> +GCOV_PROFILE_ftrace_64_pg.o := n
> +KCOV_INSTRUMENT_ftrace_64_pg.o := n
> +KCSAN_SANITIZE_ftrace_64_pg.o := n
> +UBSAN_SANITIZE_ftrace_64_pg.o := n
> diff --git a/arch/powerpc/kernel/trace/ftrace_64_pg.S b/arch/powerpc/kernel/trace/ftrace_64_pg.S
> deleted file mode 100644
> index 6708e24db0aba8..00000000000000
> --- a/arch/powerpc/kernel/trace/ftrace_64_pg.S
> +++ /dev/null
> @@ -1,67 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0-or-later */
> -/*
> - * Split from ftrace_64.S
> - */
> -
> -#include <linux/magic.h>
> -#include <asm/ppc_asm.h>
> -#include <asm/asm-offsets.h>
> -#include <asm/ftrace.h>
> -#include <asm/ppc-opcode.h>
> -#include <asm/export.h>
> -
> -_GLOBAL_TOC(ftrace_caller)
> -	lbz	r3, PACA_FTRACE_ENABLED(r13)
> -	cmpdi	r3, 0
> -	beqlr
> -
> -	/* Taken from output of objdump from lib64/glibc */
> -	mflr	r3
> -	ld	r11, 0(r1)
> -	stdu	r1, -112(r1)
> -	std	r3, 128(r1)
> -	ld	r4, 16(r11)
> -	subi	r3, r3, MCOUNT_INSN_SIZE
> -.globl ftrace_call
> -ftrace_call:
> -	bl	ftrace_stub
> -	nop
> -#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> -.globl ftrace_graph_call
> -ftrace_graph_call:
> -	b	ftrace_graph_stub
> -_GLOBAL(ftrace_graph_stub)
> -#endif
> -	ld	r0, 128(r1)
> -	mtlr	r0
> -	addi	r1, r1, 112
> -
> -_GLOBAL(ftrace_stub)
> -	blr
> -
> -#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> -_GLOBAL(ftrace_graph_caller)
> -	addi	r5, r1, 112
> -	/* load r4 with local address */
> -	ld	r4, 128(r1)
> -	subi	r4, r4, MCOUNT_INSN_SIZE
> -
> -	/* Grab the LR out of the caller stack frame */
> -	ld	r11, 112(r1)
> -	ld	r3, 16(r11)
> -
> -	bl	prepare_ftrace_return
> -	nop
> -
> -	/*
> -	 * prepare_ftrace_return gives us the address we divert to.
> -	 * Change the LR in the callers stack frame to this.
> -	 */
> -	ld	r11, 112(r1)
> -	std	r3, 16(r11)
> -
> -	ld	r0, 128(r1)
> -	mtlr	r0
> -	addi	r1, r1, 112
> -	blr
> -#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> diff --git a/arch/powerpc/kernel/trace/ftrace_pg.c b/arch/powerpc/kernel/trace/ftrace_64_pg.c
> similarity index 100%
> rename from arch/powerpc/kernel/trace/ftrace_pg.c
> rename to arch/powerpc/kernel/trace/ftrace_64_pg.c
> diff --git a/arch/powerpc/kernel/trace/ftrace_low.S b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
> similarity index 55%
> rename from arch/powerpc/kernel/trace/ftrace_low.S
> rename to arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
> index 2fc7dd0a5ae968..81dbaf70b1513a 100644
> --- a/arch/powerpc/kernel/trace/ftrace_low.S
> +++ b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
> @@ -1,6 +1,6 @@
>   /* SPDX-License-Identifier: GPL-2.0-or-later */
>   /*
> - * Split from entry_64.S
> + * Split from ftrace_64.S
>    */
>   
>   #include <linux/magic.h>
> @@ -10,6 +10,62 @@
>   #include <asm/ppc-opcode.h>
>   #include <asm/export.h>
>   
> +_GLOBAL_TOC(ftrace_caller)
> +	lbz	r3, PACA_FTRACE_ENABLED(r13)
> +	cmpdi	r3, 0
> +	beqlr
> +
> +	/* Taken from output of objdump from lib64/glibc */
> +	mflr	r3
> +	ld	r11, 0(r1)
> +	stdu	r1, -112(r1)
> +	std	r3, 128(r1)
> +	ld	r4, 16(r11)
> +	subi	r3, r3, MCOUNT_INSN_SIZE
> +.globl ftrace_call
> +ftrace_call:
> +	bl	ftrace_stub
> +	nop
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +.globl ftrace_graph_call
> +ftrace_graph_call:
> +	b	ftrace_graph_stub
> +_GLOBAL(ftrace_graph_stub)
> +#endif
> +	ld	r0, 128(r1)
> +	mtlr	r0
> +	addi	r1, r1, 112
> +
> +_GLOBAL(ftrace_stub)
> +	blr
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +_GLOBAL(ftrace_graph_caller)
> +	addi	r5, r1, 112
> +	/* load r4 with local address */
> +	ld	r4, 128(r1)
> +	subi	r4, r4, MCOUNT_INSN_SIZE
> +
> +	/* Grab the LR out of the caller stack frame */
> +	ld	r11, 112(r1)
> +	ld	r3, 16(r11)
> +
> +	bl	prepare_ftrace_return
> +	nop
> +
> +	/*
> +	 * prepare_ftrace_return gives us the address we divert to.
> +	 * Change the LR in the callers stack frame to this.
> +	 */
> +	ld	r11, 112(r1)
> +	std	r3, 16(r11)
> +
> +	ld	r0, 128(r1)
> +	mtlr	r0
> +	addi	r1, r1, 112
> +	blr
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
>   .pushsection ".tramp.ftrace.text","aw",@progbits;
>   .globl ftrace_tramp_text
>   ftrace_tramp_text:
> diff --git a/arch/powerpc/kernel/trace/ftrace_mprofile.S b/arch/powerpc/kernel/trace/ftrace_entry.S
> similarity index 83%
> rename from arch/powerpc/kernel/trace/ftrace_mprofile.S
> rename to arch/powerpc/kernel/trace/ftrace_entry.S
> index ffb1db38684998..e8339706e735b1 100644
> --- a/arch/powerpc/kernel/trace/ftrace_mprofile.S
> +++ b/arch/powerpc/kernel/trace/ftrace_entry.S
> @@ -249,3 +249,68 @@ livepatch_handler:
>   	/* Return to original caller of live patched function */
>   	blr
>   #endif /* CONFIG_LIVEPATCH */
> +
> +_GLOBAL(mcount)
> +_GLOBAL(_mcount)
> +EXPORT_SYMBOL(_mcount)
> +	mflr	r12
> +	mtctr	r12
> +	mtlr	r0
> +	bctr
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +_GLOBAL(return_to_handler)
> +	/* need to save return values */
> +#ifdef CONFIG_PPC64
> +	std	r4,  -32(r1)
> +	std	r3,  -24(r1)
> +	/* save TOC */
> +	std	r2,  -16(r1)
> +	std	r31, -8(r1)
> +	mr	r31, r1
> +	stdu	r1, -112(r1)
> +
> +	/*
> +	 * We might be called from a module.
> +	 * Switch to our TOC to run inside the core kernel.
> +	 */
> +	LOAD_PACA_TOC()
> +#else
> +	stwu	r1, -16(r1)
> +	stw	r3, 8(r1)
> +	stw	r4, 12(r1)
> +#endif
> +
> +	bl	ftrace_return_to_handler
> +	nop
> +
> +	/* return value has real return address */
> +	mtlr	r3
> +
> +#ifdef CONFIG_PPC64
> +	ld	r1, 0(r1)
> +	ld	r4,  -32(r1)
> +	ld	r3,  -24(r1)
> +	ld	r2,  -16(r1)
> +	ld	r31, -8(r1)
> +#else
> +	lwz	r3, 8(r1)
> +	lwz	r4, 12(r1)
> +	addi	r1, r1, 16
> +#endif
> +
> +	/* Jump back to real return address */
> +	blr
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
> +.pushsection ".tramp.ftrace.text","aw",@progbits;
> +.globl ftrace_tramp_text
> +ftrace_tramp_text:
> +	.space 32
> +.popsection
> +
> +.pushsection ".tramp.ftrace.init","aw",@progbits;
> +.globl ftrace_tramp_init
> +ftrace_tramp_init:
> +	.space 32
> +.popsection

Re: [PATCH 08/17] powerpc/ftrace: Refactor ftrace_modify_code()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Split up ftrace_modify_code() into a few helpers for future use. Also
> update error messages accordingly.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 51 +++++++++++++++++-------------
>   1 file changed, 29 insertions(+), 22 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 913c7aa63d3fa3..ef4e49c2c37781 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -50,32 +50,39 @@ ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
>   	return op;
>   }
>   
> -static inline int
> -ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
> +static inline int ftrace_read_inst(unsigned long ip, ppc_inst_t *op)
>   {
> -	ppc_inst_t replaced;
> -
> -	/*
> -	 * Note:
> -	 * We are paranoid about modifying text, as if a bug was to happen, it
> -	 * could cause us to read or write to someplace that could cause harm.
> -	 * Carefully read and modify the code with probe_kernel_*(), and make
> -	 * sure what we read is what we expected it to be before modifying it.
> -	 */
> -
> -	/* read the text we want to modify */
> -	if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
> +	if (copy_inst_from_kernel_nofault(op, (void *)ip)) {
> +		pr_err("0x%lx: fetching instruction failed\n", ip);
>   		return -EFAULT;
> -
> -	/* Make sure it is what we expect it to be */
> -	if (!ppc_inst_equal(replaced, old)) {
> -		pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
> -		       ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
> -		return -EINVAL;
>   	}
>   
> -	/* replace the text with the new text */
> -	return patch_instruction((u32 *)ip, new);
> +	return 0;
> +}
> +
> +static inline int ftrace_validate_inst(unsigned long ip, ppc_inst_t inst)
> +{
> +	ppc_inst_t op;
> +	int ret;
> +
> +	ret = ftrace_read_inst(ip, &op);
> +	if (!ret && !ppc_inst_equal(op, inst)) {
> +		pr_err("0x%lx: expected (%08lx) != found (%08lx)\n",
> +		       ip, ppc_inst_as_ulong(inst), ppc_inst_as_ulong(op));
> +		ret = -EINVAL;
> +	}
> +
> +	return ret;
> +}
> +
> +static inline int ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
> +{
> +	int ret = ftrace_validate_inst(ip, old);
> +
> +	if (!ret)
> +		ret = patch_instruction((u32 *)ip, new);
> +
> +	return ret;
>   }
>   
>   /*

Re: [PATCH 09/17] powerpc/ftrace: Stop re-purposing linker generated long branches for ftrace

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Commit 67361cf8071286 ("powerpc/ftrace: Handle large kernel configs")
> added ftrace support for ppc64 kernel images with a text section larger
> than 32MB. The patch did two things:
> 1. Add stubs at the end of .text to branch into ftrace_[regs_]caller for
>     functions that were out of branch range.
> 2. Re-purpose linker-generated long branches to _mcount to instead branch
>     to ftrace_[regs_]caller.
> 
> Before that, we only supported kernel .text up to ~32MB. With the above,
> we now support up to ~96MB:
> - The first 32MB of kernel text can branch directly into
>    ftrace_[regs_]caller since that symbol is usually at the beginning.
> - The modified long_branch from (2) above is used by the next 32MB of
>    kernel text.
> - The next 32MB of kernel text can use the stub at the end of text to
>    branch back to ftrace_[regs_]caller.
> 
> While re-purposing the long branch works in practice, it still restricts
> ftrace to kernel text up to ~96MB. The stub at the end of kernel text
> from (1) already enables us to extend ftrace support for kernel text
> up to 64MB, which fulfils the original requirement. Further, once we
> switch to -fpatchable-function-entry, there will not be a long branch
> that we can use.
> 
> Stop re-purposing the linker-generated long branches for ftrace to
> simplify the code. If there are good reasons to support ftrace on
> kernels beyond 64MB, we can consider adding support by using
> -fpatchable-function-entry.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 110 +++++------------------------
>   1 file changed, 17 insertions(+), 93 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index ef4e49c2c37781..278bf8e52b6e89 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -28,13 +28,7 @@
>   #include <asm/syscall.h>
>   #include <asm/inst.h>
>   
> -/*
> - * We generally only have a single long_branch tramp and at most 2 or 3 plt
> - * tramps generated. But, we don't use the plt tramps currently. We also allot
> - * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
> - * tramps in total. Set aside 8 just to be sure.
> - */
> -#define	NUM_FTRACE_TRAMPS	8
> +#define	NUM_FTRACE_TRAMPS	2
>   static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
>   
>   static ppc_inst_t
> @@ -100,11 +94,6 @@ static int is_bl_op(ppc_inst_t op)
>   	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
>   }
>   
> -static int is_b_op(ppc_inst_t op)
> -{
> -	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
> -}
> -
>   static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
>   {
>   	int offset;
> @@ -227,11 +216,7 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   {
>   	int i;
>   
> -	/*
> -	 * We have the compiler generated long_branch tramps at the end
> -	 * and we prefer those
> -	 */
> -	for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
> +	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
>   		if (!ftrace_tramps[i])
>   			continue;
>   		else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
> @@ -240,75 +225,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   	return 0;
>   }
>   
> -static int add_ftrace_tramp(unsigned long tramp)
> -{
> -	int i;
> -
> -	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
> -		if (!ftrace_tramps[i]) {
> -			ftrace_tramps[i] = tramp;
> -			return 0;
> -		}
> -
> -	return -1;
> -}
> -
> -/*
> - * If this is a compiler generated long_branch trampoline (essentially, a
> - * trampoline that has a branch to _mcount()), we re-write the branch to
> - * instead go to ftrace_[regs_]caller() and note down the location of this
> - * trampoline.
> - */
> -static int setup_mcount_compiler_tramp(unsigned long tramp)
> -{
> -	int i;
> -	ppc_inst_t op;
> -	unsigned long ptr;
> -
> -	/* Is this a known long jump tramp? */
> -	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
> -		if (ftrace_tramps[i] == tramp)
> -			return 0;
> -
> -	/* New trampoline -- read where this goes */
> -	if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
> -		pr_debug("Fetching opcode failed.\n");
> -		return -1;
> -	}
> -
> -	/* Is this a 24 bit branch? */
> -	if (!is_b_op(op)) {
> -		pr_debug("Trampoline is not a long branch tramp.\n");
> -		return -1;
> -	}
> -
> -	/* lets find where the pointer goes */
> -	ptr = find_bl_target(tramp, op);
> -
> -	if (ptr != ppc_global_function_entry((void *)_mcount)) {
> -		pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
> -		return -1;
> -	}
> -
> -	/* Let's re-write the tramp to go to ftrace_[regs_]caller */
> -	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> -		ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
> -	else
> -		ptr = ppc_global_function_entry((void *)ftrace_caller);
> -
> -	if (patch_branch((u32 *)tramp, ptr, 0)) {
> -		pr_debug("REL24 out of range!\n");
> -		return -1;
> -	}
> -
> -	if (add_ftrace_tramp(tramp)) {
> -		pr_debug("No tramp locations left\n");
> -		return -1;
> -	}
> -
> -	return 0;
> -}
> -
>   static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
>   {
>   	unsigned long tramp, ip = rec->ip;
> @@ -331,13 +247,10 @@ static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
>   
>   	pr_devel("ip:%lx jumps to %lx", ip, tramp);
>   
> -	if (setup_mcount_compiler_tramp(tramp)) {
> -		/* Are other trampolines reachable? */
> -		if (!find_ftrace_tramp(ip)) {
> -			pr_err("No ftrace trampolines reachable from %ps\n",
> -					(void *)ip);
> -			return -EINVAL;
> -		}
> +	/* Are ftrace trampolines reachable? */
> +	if (!find_ftrace_tramp(ip)) {
> +		pr_err("No ftrace trampolines reachable from %ps\n", (void *)ip);
> +		return -EINVAL;
>   	}
>   
>   	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
> @@ -725,6 +638,17 @@ void ftrace_free_init_tramp(void)
>   		}
>   }
>   
> +static void __init add_ftrace_tramp(unsigned long tramp)
> +{
> +	int i;
> +
> +	for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
> +		if (!ftrace_tramps[i]) {
> +			ftrace_tramps[i] = tramp;
> +			return;
> +		}
> +}
> +
>   int __init ftrace_dyn_arch_init(void)
>   {
>   	unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };

Re: [PATCH 10/17] powerpc/ftrace: Add separate ftrace_init_nop() with additional validation

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Currently, we validate instructions around the ftrace location every
> time we have to enable/disable ftrace. Introduce ftrace_init_nop() to
> instead perform all the validation during ftrace initialization. This
> allows us to simply patch the necessary instructions during
> enabling/disabling ftrace.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/include/asm/ftrace.h  |  6 +++
>   arch/powerpc/kernel/trace/ftrace.c | 71 ++++++++++++++++++++++++++++++
>   2 files changed, 77 insertions(+)
> 
> diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
> index 702aaf2efa966c..ef9f0b97670d1c 100644
> --- a/arch/powerpc/include/asm/ftrace.h
> +++ b/arch/powerpc/include/asm/ftrace.h
> @@ -29,11 +29,17 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr)
>   unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
>   				    unsigned long sp);
>   
> +struct module;
> +struct dyn_ftrace;
>   struct dyn_arch_ftrace {
>   	struct module *mod;
>   };
>   
>   #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
> +#define ftrace_need_init_nop()	(true)
> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
> +#define ftrace_init_nop ftrace_init_nop
> +
>   struct ftrace_regs {
>   	struct pt_regs regs;
>   };
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 278bf8e52b6e89..98bd099c428ee0 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -31,6 +31,16 @@
>   #define	NUM_FTRACE_TRAMPS	2
>   static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
>   
> +static ppc_inst_t ftrace_create_branch_inst(unsigned long ip, unsigned long addr, int link)
> +{
> +	ppc_inst_t op;
> +
> +	WARN_ON(!is_offset_in_branch_range(addr - ip));
> +	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
> +
> +	return op;
> +}
> +
>   static ppc_inst_t
>   ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
>   {
> @@ -597,6 +607,67 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>   }
>   #endif
>   
> +int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
> +{
> +	unsigned long addr, ip = rec->ip;
> +	ppc_inst_t old, new;
> +	int ret = 0;
> +
> +	/* Verify instructions surrounding the ftrace location */
> +	if (IS_ENABLED(CONFIG_PPC32)) {
> +		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
> +		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
> +		if (!ret)
> +			ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)));
> +	} else if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
> +		/* Expected sequence: 'mflr r0', ['std r0,16(r1)'], 'bl _mcount' */
> +		ret = ftrace_read_inst(ip - 4, &old);
> +		if (!ret && !ppc_inst_equal(old, ppc_inst(PPC_RAW_MFLR(_R0)))) {
> +			ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
> +			ret |= ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)));
> +		}
> +	} else {
> +		return -EINVAL;
> +	}
> +
> +	if (ret)
> +		return ret;
> +
> +	if (!core_kernel_text(ip)) {
> +		if (!mod) {
> +			pr_err("0x%lx: No module provided for non-kernel address\n", ip);
> +			return -EFAULT;
> +		}
> +		rec->arch.mod = mod;
> +	}
> +
> +	/* Nop-out the ftrace location */
> +	new = ppc_inst(PPC_RAW_NOP());
> +	addr = MCOUNT_ADDR;
> +	if (is_offset_in_branch_range(addr - ip)) {
> +		/* Within range */
> +		old = ftrace_create_branch_inst(ip, addr, 1);
> +		ret = ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip) || (IS_ENABLED(CONFIG_MODULES) && mod)) {
> +		/*
> +		 * We would be branching to a linker-generated stub, or to the module _mcount
> +		 * stub. Let's just confirm we have a 'bl' here.
> +		 */
> +		ret = ftrace_read_inst(ip, &old);
> +		if (ret)
> +			return ret;
> +		if (!is_bl_op(old)) {
> +			pr_err("0x%lx: expected (bl) != found (%08lx)\n", ip, ppc_inst_as_ulong(old));
> +			return -EINVAL;
> +		}
> +		ret = patch_instruction((u32 *)ip, new);
> +	} else {
> +		return -EINVAL;
> +	}
> +
> +	return ret;
> +}
> +
>   int ftrace_update_ftrace_func(ftrace_func_t func)
>   {
>   	unsigned long ip = (unsigned long)(&ftrace_call);

Re: [PATCH 11/17] powerpc/ftrace: Simplify ftrace_make_nop()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Now that we validate the ftrace location during initialization in
> ftrace_init_nop(), we can simplify ftrace_make_nop() to patch-in the nop
> without worrying about the instructions surrounding the ftrace location.
> Note that we continue to ensure that we have a bl to
> ftrace_[regs_]caller at the ftrace location before nop-ing it out.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 220 +++++------------------------
>   1 file changed, 32 insertions(+), 188 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 98bd099c428ee0..05153a1038fdff 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -116,112 +116,6 @@ static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
>   	return ip + (long)offset;
>   }
>   
> -#ifdef CONFIG_MODULES
> -static int
> -__ftrace_make_nop(struct module *mod,
> -		  struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	unsigned long entry, ptr, tramp;
> -	unsigned long ip = rec->ip;
> -	ppc_inst_t op, pop;
> -
> -	/* read where this goes */
> -	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> -		pr_err("Fetching opcode failed.\n");
> -		return -EFAULT;
> -	}
> -
> -	/* Make sure that this is still a 24bit jump */
> -	if (!is_bl_op(op)) {
> -		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> -		return -EINVAL;
> -	}
> -
> -	/* lets find where the pointer goes */
> -	tramp = find_bl_target(ip, op);
> -
> -	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> -
> -	if (module_trampoline_target(mod, tramp, &ptr)) {
> -		pr_err("Failed to get trampoline target\n");
> -		return -EFAULT;
> -	}
> -
> -	pr_devel("trampoline target %lx", ptr);
> -
> -	entry = ppc_global_function_entry((void *)addr);
> -	/* This should match what was called */
> -	if (ptr != entry) {
> -		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> -		return -EINVAL;
> -	}
> -
> -	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
> -		if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
> -			pr_err("Fetching instruction at %lx failed.\n", ip - 4);
> -			return -EFAULT;
> -		}
> -
> -		/* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
> -		if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
> -		    !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
> -			pr_err("Unexpected instruction %08lx around bl _mcount\n",
> -			       ppc_inst_as_ulong(op));
> -			return -EINVAL;
> -		}
> -	} else if (IS_ENABLED(CONFIG_PPC64)) {
> -		/*
> -		 * Check what is in the next instruction. We can see ld r2,40(r1), but
> -		 * on first pass after boot we will see mflr r0.
> -		 */
> -		if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
> -			pr_err("Fetching op failed.\n");
> -			return -EFAULT;
> -		}
> -
> -		if (!ppc_inst_equal(op,  ppc_inst(PPC_INST_LD_TOC))) {
> -			pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
> -			       ppc_inst_as_ulong(op));
> -			return -EINVAL;
> -		}
> -	}
> -
> -	/*
> -	 * When using -mprofile-kernel or PPC32 there is no load to jump over.
> -	 *
> -	 * Otherwise our original call site looks like:
> -	 *
> -	 * bl <tramp>
> -	 * ld r2,XX(r1)
> -	 *
> -	 * Milton Miller pointed out that we can not simply nop the branch.
> -	 * If a task was preempted when calling a trace function, the nops
> -	 * will remove the way to restore the TOC in r2 and the r2 TOC will
> -	 * get corrupted.
> -	 *
> -	 * Use a b +8 to jump over the load.
> -	 * XXX: could make PCREL depend on MPROFILE_KERNEL
> -	 * XXX: check PCREL && MPROFILE_KERNEL calling sequence
> -	 */
> -	if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
> -		pop = ppc_inst(PPC_RAW_NOP());
> -	else
> -		pop = ppc_inst(PPC_RAW_BRANCH(8));	/* b +8 */
> -
> -	if (patch_instruction((u32 *)ip, pop)) {
> -		pr_err("Patching NOP failed.\n");
> -		return -EPERM;
> -	}
> -
> -	return 0;
> -}
> -#else
> -static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	return 0;
> -}
> -#endif /* CONFIG_MODULES */
> -
>   static unsigned long find_ftrace_tramp(unsigned long ip)
>   {
>   	int i;
> @@ -235,88 +129,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   	return 0;
>   }
>   
> -static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	unsigned long tramp, ip = rec->ip;
> -	ppc_inst_t op;
> -
> -	/* Read where this goes */
> -	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> -		pr_err("Fetching opcode failed.\n");
> -		return -EFAULT;
> -	}
> -
> -	/* Make sure that this is still a 24bit jump */
> -	if (!is_bl_op(op)) {
> -		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> -		return -EINVAL;
> -	}
> -
> -	/* Let's find where the pointer goes */
> -	tramp = find_bl_target(ip, op);
> -
> -	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> -
> -	/* Are ftrace trampolines reachable? */
> -	if (!find_ftrace_tramp(ip)) {
> -		pr_err("No ftrace trampolines reachable from %ps\n", (void *)ip);
> -		return -EINVAL;
> -	}
> -
> -	if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
> -		pr_err("Patching NOP failed.\n");
> -		return -EPERM;
> -	}
> -
> -	return 0;
> -}
> -
> -int ftrace_make_nop(struct module *mod,
> -		    struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	unsigned long ip = rec->ip;
> -	ppc_inst_t old, new;
> -
> -	/*
> -	 * If the calling address is more that 24 bits away,
> -	 * then we had to use a trampoline to make the call.
> -	 * Otherwise just update the call site.
> -	 */
> -	if (test_24bit_addr(ip, addr)) {
> -		/* within range */
> -		old = ftrace_call_replace(ip, addr, 1);
> -		new = ppc_inst(PPC_RAW_NOP());
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (core_kernel_text(ip)) {
> -		return __ftrace_make_nop_kernel(rec, addr);
> -	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * Out of range jumps are called from modules.
> -	 * We should either already have a pointer to the module
> -	 * or it has been passed in.
> -	 */
> -	if (!rec->arch.mod) {
> -		if (!mod) {
> -			pr_err("No module loaded addr=%lx\n", addr);
> -			return -EFAULT;
> -		}
> -		rec->arch.mod = mod;
> -	} else if (mod) {
> -		if (mod != rec->arch.mod) {
> -			pr_err("Record mod %p not equal to passed in mod %p\n",
> -			       rec->arch.mod, mod);
> -			return -EINVAL;
> -		}
> -		/* nothing to do if mod == rec->arch.mod */
> -	} else
> -		mod = rec->arch.mod;
> -
> -	return __ftrace_make_nop(mod, rec, addr);
> -}
> -
>   #ifdef CONFIG_MODULES
>   /*
>    * Examine the existing instructions for __ftrace_make_call.
> @@ -607,6 +419,38 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>   }
>   #endif
>   
> +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long tramp, ip = rec->ip;
> +	ppc_inst_t old, new;
> +
> +	/* Nop-out the ftrace location */
> +	new = ppc_inst(PPC_RAW_NOP());
> +	if (is_offset_in_branch_range(addr - ip)) {
> +		/* Within range */
> +		old = ftrace_create_branch_inst(ip, addr, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip)) {
> +		/* We would be branching to one of our ftrace tramps */
> +		tramp = find_ftrace_tramp(ip);
> +		if (!tramp) {
> +			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
> +			return -EINVAL;
> +		}
> +		old = ftrace_create_branch_inst(ip, tramp, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (IS_ENABLED(CONFIG_MODULES)) {
> +		/* Module code would be going to one of the module stubs */
> +		if (!mod)
> +			mod = rec->arch.mod;
> +		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
> +		old = ftrace_create_branch_inst(ip, tramp, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	}
> +
> +	return -EINVAL;
> +}
> +
>   int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
>   {
>   	unsigned long addr, ip = rec->ip;

Re: [PATCH 12/17] powerpc/ftrace: Simplify ftrace_make_call()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Now that we validate the ftrace location during initialization in
> ftrace_init_nop(), we can simplify ftrace_make_call() to replace the nop
> without worrying about the instructions surrounding the ftrace location.
> Note that we continue to ensure that we have a nop at the ftrace
> location before patching it.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 187 +++++------------------------
>   1 file changed, 31 insertions(+), 156 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 05153a1038fdff..6ea8b90246a540 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -129,162 +129,6 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   	return 0;
>   }
>   
> -#ifdef CONFIG_MODULES
> -/*
> - * Examine the existing instructions for __ftrace_make_call.
> - * They should effectively be a NOP, and follow formal constraints,
> - * depending on the ABI. Return false if they don't.
> - */
> -static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
> -{
> -	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> -		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
> -	else
> -		return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
> -		       ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
> -}
> -
> -static int
> -__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	ppc_inst_t op[2];
> -	void *ip = (void *)rec->ip;
> -	unsigned long entry, ptr, tramp;
> -	struct module *mod = rec->arch.mod;
> -
> -	/* read where this goes */
> -	if (copy_inst_from_kernel_nofault(op, ip))
> -		return -EFAULT;
> -
> -	if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
> -	    copy_inst_from_kernel_nofault(op + 1, ip + 4))
> -		return -EFAULT;
> -
> -	if (!expected_nop_sequence(ip, op[0], op[1])) {
> -		pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
> -		       ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
> -		return -EINVAL;
> -	}
> -
> -	/* If we never set up ftrace trampoline(s), then bail */
> -	if (!mod->arch.tramp ||
> -	    (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
> -		pr_err("No ftrace trampoline\n");
> -		return -EINVAL;
> -	}
> -
> -	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
> -		tramp = mod->arch.tramp_regs;
> -	else
> -		tramp = mod->arch.tramp;
> -
> -	if (module_trampoline_target(mod, tramp, &ptr)) {
> -		pr_err("Failed to get trampoline target\n");
> -		return -EFAULT;
> -	}
> -
> -	pr_devel("trampoline target %lx", ptr);
> -
> -	entry = ppc_global_function_entry((void *)addr);
> -	/* This should match what was called */
> -	if (ptr != entry) {
> -		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> -		return -EINVAL;
> -	}
> -
> -	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
> -		pr_err("REL24 out of range!\n");
> -		return -EINVAL;
> -	}
> -
> -	return 0;
> -}
> -#else
> -static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	return 0;
> -}
> -#endif /* CONFIG_MODULES */
> -
> -static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	ppc_inst_t op;
> -	void *ip = (void *)rec->ip;
> -	unsigned long tramp, entry, ptr;
> -
> -	/* Make sure we're being asked to patch branch to a known ftrace addr */
> -	entry = ppc_global_function_entry((void *)ftrace_caller);
> -	ptr = ppc_global_function_entry((void *)addr);
> -
> -	if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
> -		entry = ppc_global_function_entry((void *)ftrace_regs_caller);
> -
> -	if (ptr != entry) {
> -		pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
> -		return -EINVAL;
> -	}
> -
> -	/* Make sure we have a nop */
> -	if (copy_inst_from_kernel_nofault(&op, ip)) {
> -		pr_err("Unable to read ftrace location %p\n", ip);
> -		return -EFAULT;
> -	}
> -
> -	if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
> -		pr_err("Unexpected call sequence at %p: %08lx\n",
> -		       ip, ppc_inst_as_ulong(op));
> -		return -EINVAL;
> -	}
> -
> -	tramp = find_ftrace_tramp((unsigned long)ip);
> -	if (!tramp) {
> -		pr_err("No ftrace trampolines reachable from %ps\n", ip);
> -		return -EINVAL;
> -	}
> -
> -	if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
> -		pr_err("Error patching branch to ftrace tramp!\n");
> -		return -EINVAL;
> -	}
> -
> -	return 0;
> -}
> -
> -int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> -{
> -	unsigned long ip = rec->ip;
> -	ppc_inst_t old, new;
> -
> -	/*
> -	 * If the calling address is more that 24 bits away,
> -	 * then we had to use a trampoline to make the call.
> -	 * Otherwise just update the call site.
> -	 */
> -	if (test_24bit_addr(ip, addr)) {
> -		/* within range */
> -		old = ppc_inst(PPC_RAW_NOP());
> -		new = ftrace_call_replace(ip, addr, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (core_kernel_text(ip)) {
> -		return __ftrace_make_call_kernel(rec, addr);
> -	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> -		/* We should not get here without modules */
> -		return -EINVAL;
> -	}
> -
> -	/*
> -	 * Out of range jumps are called from modules.
> -	 * Being that we are converting from nop, it had better
> -	 * already have a module defined.
> -	 */
> -	if (!rec->arch.mod) {
> -		pr_err("No module loaded\n");
> -		return -EINVAL;
> -	}
> -
> -	return __ftrace_make_call(rec, addr);
> -}
> -
>   #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>   #ifdef CONFIG_MODULES
>   static int
> @@ -419,6 +263,37 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>   }
>   #endif
>   
> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> +	unsigned long tramp, ip = rec->ip;
> +	ppc_inst_t old, new;
> +	struct module *mod;
> +
> +	old = ppc_inst(PPC_RAW_NOP());
> +	if (is_offset_in_branch_range(addr - ip)) {
> +		/* Within range */
> +		new = ftrace_create_branch_inst(ip, addr, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (core_kernel_text(ip)) {
> +		/* We would be branching to one of our ftrace tramps */
> +		tramp = find_ftrace_tramp(ip);
> +		if (!tramp) {
> +			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
> +			return -EINVAL;
> +		}
> +		new = ftrace_create_branch_inst(ip, tramp, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	} else if (IS_ENABLED(CONFIG_MODULES)) {
> +		/* Module code would be going to one of the module stubs */
> +		mod = rec->arch.mod;
> +		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
> +		new = ftrace_create_branch_inst(ip, tramp, 1);
> +		return ftrace_modify_code(ip, old, new);
> +	}
> +
> +	return -EINVAL;
> +}
> +
>   int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
>   {
>   	unsigned long tramp, ip = rec->ip;

Re: [PATCH 13/17] powerpc/ftrace: Simplify ftrace_modify_call()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Now that we validate the ftrace location during initialization in
> ftrace_init_nop(), we can simplify ftrace_modify_call() to patch-in the
> updated branch instruction without worrying about the instructions
> surrounding the ftrace location. Note that we continue to ensure we
> have the expected branch instruction at the ftrace location before
> patching it with the updated branch destination.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 161 ++++-------------------------
>   1 file changed, 21 insertions(+), 140 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 6ea8b90246a540..c37e22c6c26521 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -89,33 +89,11 @@ static inline int ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_
>   	return ret;
>   }
>   
> -/*
> - * Helper functions that are the same for both PPC64 and PPC32.
> - */
> -static int test_24bit_addr(unsigned long ip, unsigned long addr)
> -{
> -	addr = ppc_function_entry((void *)addr);
> -
> -	return is_offset_in_branch_range(addr - ip);
> -}
> -
>   static int is_bl_op(ppc_inst_t op)
>   {
>   	return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
>   }
>   
> -static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
> -{
> -	int offset;
> -
> -	offset = PPC_LI(ppc_inst_val(op));
> -	/* make it signed */
> -	if (offset & 0x02000000)
> -		offset |= 0xfe000000;
> -
> -	return ip + (long)offset;
> -}
> -
>   static unsigned long find_ftrace_tramp(unsigned long ip)
>   {
>   	int i;
> @@ -130,115 +108,16 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   }
>   
>   #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
> -#ifdef CONFIG_MODULES
> -static int
> -__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> -					unsigned long addr)
> +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
>   {
> -	ppc_inst_t op;
> -	unsigned long ip = rec->ip;
> -	unsigned long entry, ptr, tramp;
> -	struct module *mod = rec->arch.mod;
> -
> -	/* If we never set up ftrace trampolines, then bail */
> -	if (!mod->arch.tramp || !mod->arch.tramp_regs) {
> -		pr_err("No ftrace trampoline\n");
> -		return -EINVAL;
> -	}
> -
> -	/* read where this goes */
> -	if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
> -		pr_err("Fetching opcode failed.\n");
> -		return -EFAULT;
> -	}
> -
> -	/* Make sure that this is still a 24bit jump */
> -	if (!is_bl_op(op)) {
> -		pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
> -		return -EINVAL;
> -	}
> -
> -	/* lets find where the pointer goes */
> -	tramp = find_bl_target(ip, op);
> -	entry = ppc_global_function_entry((void *)old_addr);
> -
> -	pr_devel("ip:%lx jumps to %lx", ip, tramp);
> -
> -	if (tramp != entry) {
> -		/* old_addr is not within range, so we must have used a trampoline */
> -		if (module_trampoline_target(mod, tramp, &ptr)) {
> -			pr_err("Failed to get trampoline target\n");
> -			return -EFAULT;
> -		}
> -
> -		pr_devel("trampoline target %lx", ptr);
> -
> -		/* This should match what was called */
> -		if (ptr != entry) {
> -			pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> -			return -EINVAL;
> -		}
> -	}
> -
> -	/* The new target may be within range */
> -	if (test_24bit_addr(ip, addr)) {
> -		/* within range */
> -		if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
> -			pr_err("REL24 out of range!\n");
> -			return -EINVAL;
> -		}
> -
> -		return 0;
> -	}
> -
> -	if (rec->flags & FTRACE_FL_REGS)
> -		tramp = mod->arch.tramp_regs;
> -	else
> -		tramp = mod->arch.tramp;
> -
> -	if (module_trampoline_target(mod, tramp, &ptr)) {
> -		pr_err("Failed to get trampoline target\n");
> -		return -EFAULT;
> -	}
> -
> -	pr_devel("trampoline target %lx", ptr);
> -
> -	entry = ppc_global_function_entry((void *)addr);
> -	/* This should match what was called */
> -	if (ptr != entry) {
> -		pr_err("addr %lx does not match expected %lx\n", ptr, entry);
> -		return -EINVAL;
> -	}
> -
> -	if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
> -		pr_err("REL24 out of range!\n");
> -		return -EINVAL;
> -	}
> -
> -	return 0;
> -}
> -#else
> -static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
> -{
> -	return 0;
> -}
> -#endif
> -
> -int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
> -			unsigned long addr)
> -{
> -	unsigned long ip = rec->ip;
> +	unsigned long tramp, tramp_old, ip = rec->ip;
>   	ppc_inst_t old, new;
> +	struct module *mod;
>   
> -	/*
> -	 * If the calling address is more that 24 bits away,
> -	 * then we had to use a trampoline to make the call.
> -	 * Otherwise just update the call site.
> -	 */
> -	if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
> -		/* within range */
> -		old = ftrace_call_replace(ip, old_addr, 1);
> -		new = ftrace_call_replace(ip, addr, 1);
> +	if (is_offset_in_branch_range(old_addr - ip) && is_offset_in_branch_range(addr - ip)) {
> +		/* Within range */
> +		old = ftrace_create_branch_inst(ip, old_addr, 1);
> +		new = ftrace_create_branch_inst(ip, addr, 1);
>   		return ftrace_modify_code(ip, old, new);
>   	} else if (core_kernel_text(ip)) {
>   		/*
> @@ -246,20 +125,22 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
>   		 * variant, so there is nothing to do here
>   		 */
>   		return 0;
> -	} else if (!IS_ENABLED(CONFIG_MODULES)) {
> -		/* We should not get here without modules */
> -		return -EINVAL;
> +	} else if (IS_ENABLED(CONFIG_MODULES)) {
> +		/* Module code would be going to one of the module stubs */
> +		mod = rec->arch.mod;
> +		if (addr == (unsigned long)ftrace_caller) {
> +			tramp_old = mod->arch.tramp_regs;
> +			tramp = mod->arch.tramp;
> +		} else {
> +			tramp_old = mod->arch.tramp;
> +			tramp = mod->arch.tramp_regs;
> +		}
> +		old = ftrace_create_branch_inst(ip, tramp_old, 1);
> +		new = ftrace_create_branch_inst(ip, tramp, 1);
> +		return ftrace_modify_code(ip, old, new);
>   	}
>   
> -	/*
> -	 * Out of range jumps are called from modules.
> -	 */
> -	if (!rec->arch.mod) {
> -		pr_err("No module loaded\n");
> -		return -EINVAL;
> -	}
> -
> -	return __ftrace_modify_call(rec, old_addr, addr);
> +	return -EINVAL;
>   }
>   #endif
>   

Re: [PATCH 14/17] powerpc/ftrace: Replace use of ftrace_call_replace() with ftrace_create_branch_inst()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> ftrace_create_branch_inst() is clearer about its intent than
> ftrace_call_replace().
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c | 17 ++---------------
>   1 file changed, 2 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index c37e22c6c26521..422dd760fbe013 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -41,19 +41,6 @@ static ppc_inst_t ftrace_create_branch_inst(unsigned long ip, unsigned long addr
>   	return op;
>   }
>   
> -static ppc_inst_t
> -ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
> -{
> -	ppc_inst_t op;
> -
> -	addr = ppc_function_entry((void *)addr);
> -
> -	/* if (link) set op to 'bl' else 'b' */
> -	create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
> -
> -	return op;
> -}
> -
>   static inline int ftrace_read_inst(unsigned long ip, ppc_inst_t *op)
>   {
>   	if (copy_inst_from_kernel_nofault(op, (void *)ip)) {
> @@ -275,14 +262,14 @@ int ftrace_update_ftrace_func(ftrace_func_t func)
>   	int ret;
>   
>   	old = ppc_inst_read((u32 *)&ftrace_call);
> -	new = ftrace_call_replace(ip, (unsigned long)func, 1);
> +	new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
>   	ret = ftrace_modify_code(ip, old, new);
>   
>   	/* Also update the regs callback function */
>   	if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
>   		ip = (unsigned long)(&ftrace_regs_call);
>   		old = ppc_inst_read((u32 *)&ftrace_regs_call);
> -		new = ftrace_call_replace(ip, (unsigned long)func, 1);
> +		new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
>   		ret = ftrace_modify_code(ip, old, new);
>   	}
>   

Re: [PATCH 15/17] powerpc/ftrace: Implement ftrace_replace_code()

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> Implement ftrace_replace_code() to consolidate logic from the different
> ftrace patching routines: ftrace_make_nop(), ftrace_make_call() and
> ftrace_modify_call(). Note that ftrace_make_call() is still required
> primarily to handle patching modules during their load time. The other
> two routines should no longer be called.
> 
> This lays the groundwork to enable better control in patching ftrace
> locations, including the ability to nop-out preceding profiling
> instructions when ftrace is disabled.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>


> ---
>   arch/powerpc/kernel/trace/ftrace.c | 173 ++++++++++++++++-------------
>   1 file changed, 96 insertions(+), 77 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 422dd760fbe013..cf9dce77527920 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -94,104 +94,123 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
>   	return 0;
>   }
>   
> +static int ftrace_get_call_inst(struct dyn_ftrace *rec, unsigned long addr, ppc_inst_t *call_inst)
> +{
> +	unsigned long ip = rec->ip;
> +	unsigned long stub;
> +
> +	if (is_offset_in_branch_range(addr - ip)) {
> +		/* Within range */
> +		stub = addr;
> +#ifdef CONFIG_MODULES
> +	} else if (rec->arch.mod) {
> +		/* Module code would be going to one of the module stubs */
> +		stub = (addr == (unsigned long)ftrace_caller ? rec->arch.mod->arch.tramp :
> +							       rec->arch.mod->arch.tramp_regs);
> +#endif
> +	} else if (core_kernel_text(ip)) {
> +		/* We would be branching to one of our ftrace stubs */
> +		stub = find_ftrace_tramp(ip);
> +		if (!stub) {
> +			pr_err("0x%lx: No ftrace stubs reachable\n", ip);
> +			return -EINVAL;
> +		}
> +	} else {
> +		return -EINVAL;
> +	}
> +
> +	*call_inst = ftrace_create_branch_inst(ip, stub, 1);
> +	return 0;
> +}
> +
>   #ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
>   int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
>   {
> -	unsigned long tramp, tramp_old, ip = rec->ip;
> -	ppc_inst_t old, new;
> -	struct module *mod;
> -
> -	if (is_offset_in_branch_range(old_addr - ip) && is_offset_in_branch_range(addr - ip)) {
> -		/* Within range */
> -		old = ftrace_create_branch_inst(ip, old_addr, 1);
> -		new = ftrace_create_branch_inst(ip, addr, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (core_kernel_text(ip)) {
> -		/*
> -		 * We always patch out of range locations to go to the regs
> -		 * variant, so there is nothing to do here
> -		 */
> -		return 0;
> -	} else if (IS_ENABLED(CONFIG_MODULES)) {
> -		/* Module code would be going to one of the module stubs */
> -		mod = rec->arch.mod;
> -		if (addr == (unsigned long)ftrace_caller) {
> -			tramp_old = mod->arch.tramp_regs;
> -			tramp = mod->arch.tramp;
> -		} else {
> -			tramp_old = mod->arch.tramp;
> -			tramp = mod->arch.tramp_regs;
> -		}
> -		old = ftrace_create_branch_inst(ip, tramp_old, 1);
> -		new = ftrace_create_branch_inst(ip, tramp, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	}
> -
> +	/* This should never be called since we override ftrace_replace_code() */
> +	WARN_ON(1);
>   	return -EINVAL;
>   }
>   #endif
>   
>   int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
>   {
> -	unsigned long tramp, ip = rec->ip;
>   	ppc_inst_t old, new;
> -	struct module *mod;
> +	int ret;
> +
> +	/* This can only ever be called during module load */
> +	if (WARN_ON(!IS_ENABLED(CONFIG_MODULES) || core_kernel_text(rec->ip)))
> +		return -EINVAL;
>   
>   	old = ppc_inst(PPC_RAW_NOP());
> -	if (is_offset_in_branch_range(addr - ip)) {
> -		/* Within range */
> -		new = ftrace_create_branch_inst(ip, addr, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (core_kernel_text(ip)) {
> -		/* We would be branching to one of our ftrace tramps */
> -		tramp = find_ftrace_tramp(ip);
> -		if (!tramp) {
> -			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
> -			return -EINVAL;
> -		}
> -		new = ftrace_create_branch_inst(ip, tramp, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (IS_ENABLED(CONFIG_MODULES)) {
> -		/* Module code would be going to one of the module stubs */
> -		mod = rec->arch.mod;
> -		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
> -		new = ftrace_create_branch_inst(ip, tramp, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	}
> +	ret = ftrace_get_call_inst(rec, addr, &new);
> +	if (ret)
> +		return ret;
>   
> -	return -EINVAL;
> +	return ftrace_modify_code(rec->ip, old, new);
>   }
>   
>   int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
>   {
> -	unsigned long tramp, ip = rec->ip;
> -	ppc_inst_t old, new;
> +	/*
> +	 * This should never be called since we override ftrace_replace_code(),
> +	 * as well as ftrace_init_nop()
> +	 */
> +	WARN_ON(1);
> +	return -EINVAL;
> +}
>   
> -	/* Nop-out the ftrace location */
> -	new = ppc_inst(PPC_RAW_NOP());
> -	if (is_offset_in_branch_range(addr - ip)) {
> -		/* Within range */
> -		old = ftrace_create_branch_inst(ip, addr, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (core_kernel_text(ip)) {
> -		/* We would be branching to one of our ftrace tramps */
> -		tramp = find_ftrace_tramp(ip);
> -		if (!tramp) {
> -			pr_err("0x%lx: No ftrace trampolines reachable\n", ip);
> -			return -EINVAL;
> +void ftrace_replace_code(int enable)
> +{
> +	ppc_inst_t old, new, call_inst, new_call_inst;
> +	ppc_inst_t nop_inst = ppc_inst(PPC_RAW_NOP());
> +	unsigned long ip, new_addr, addr;
> +	struct ftrace_rec_iter *iter;
> +	struct dyn_ftrace *rec;
> +	int ret = 0, update;
> +
> +	for_ftrace_rec_iter(iter) {
> +		rec = ftrace_rec_iter_record(iter);
> +		ip = rec->ip;
> +
> +		if (rec->flags & FTRACE_FL_DISABLED && !(rec->flags & FTRACE_FL_ENABLED))
> +			continue;
> +
> +		addr = ftrace_get_addr_curr(rec);
> +		new_addr = ftrace_get_addr_new(rec);
> +		update = ftrace_update_record(rec, enable);
> +
> +		switch (update) {
> +		case FTRACE_UPDATE_IGNORE:
> +		default:
> +			continue;
> +		case FTRACE_UPDATE_MODIFY_CALL:
> +			ret = ftrace_get_call_inst(rec, new_addr, &new_call_inst);
> +			ret |= ftrace_get_call_inst(rec, addr, &call_inst);
> +			old = call_inst;
> +			new = new_call_inst;
> +			break;
> +		case FTRACE_UPDATE_MAKE_NOP:
> +			ret = ftrace_get_call_inst(rec, addr, &call_inst);
> +			old = call_inst;
> +			new = nop_inst;
> +			break;
> +		case FTRACE_UPDATE_MAKE_CALL:
> +			ret = ftrace_get_call_inst(rec, new_addr, &call_inst);
> +			old = nop_inst;
> +			new = call_inst;
> +			break;
>   		}
> -		old = ftrace_create_branch_inst(ip, tramp, 1);
> -		return ftrace_modify_code(ip, old, new);
> -	} else if (IS_ENABLED(CONFIG_MODULES)) {
> -		/* Module code would be going to one of the module stubs */
> -		if (!mod)
> -			mod = rec->arch.mod;
> -		tramp = (addr == (unsigned long)ftrace_caller ? mod->arch.tramp : mod->arch.tramp_regs);
> -		old = ftrace_create_branch_inst(ip, tramp, 1);
> -		return ftrace_modify_code(ip, old, new);
> +
> +		if (!ret)
> +			ret = ftrace_modify_code(ip, old, new);
> +		if (ret)
> +			goto out;
>   	}
>   
> -	return -EINVAL;
> +out:
> +	if (ret)
> +		ftrace_bug(ret, rec);
> +	return;
>   }
>   
>   int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)

Re: [PATCH 16/17] powerpc/ftrace: Add support for -fpatchable-function-entry

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> GCC v13.1 updated support for -fpatchable-function-entry on ppc64le to
> emit nops after the local entry point, rather than before it. This
> allows us to use this in the kernel for ftrace purposes. A new script is
> added under arch/powerpc/tools/ to help detect if nops are emitted after
> the function local entry point, or before the global entry point.
> 
> With -fpatchable-function-entry, we no longer have the profiling
> instructions generated at function entry, so we only need to validate
> the presence of two nops at the ftrace location in ftrace_init_nop(). We
> patch the preceding instruction with 'mflr r0' to match the
> -mprofile-kernel ABI for subsequent ftrace use.
> 
> This changes the profiling instructions used on ppc32. The default -pg
> option emits an additional 'stw' instruction after 'mflr r0' and before
> the branch to _mcount 'bl _mcount'. This is very similar to the original
> -mprofile-kernel implementation on ppc64le, where an additional 'std'
> instruction was used to save LR to its save location in the caller's
> stackframe. Subsequently, this additional store was removed in later
> compiler versions for performance reasons. The same reasons apply for
> ppc32 so we only patch in a 'mflr r0'.
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

Nit below

> ---
>   arch/powerpc/Kconfig                          | 14 +++++++---
>   arch/powerpc/Makefile                         |  5 ++++
>   arch/powerpc/include/asm/ftrace.h             |  6 +++--
>   arch/powerpc/include/asm/vermagic.h           |  4 ++-
>   arch/powerpc/kernel/module_64.c               |  2 +-
>   arch/powerpc/kernel/trace/ftrace.c            | 14 ++++++++--
>   arch/powerpc/kernel/trace/ftrace_entry.S      |  2 ++
>   .../gcc-check-fpatchable-function-entry.sh    | 26 +++++++++++++++++++
>   8 files changed, 64 insertions(+), 9 deletions(-)
>   create mode 100755 arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
> 
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index bff5820b7cda14..9352d8e68152e1 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -187,6 +187,7 @@ config PPC
>   	select DYNAMIC_FTRACE			if FUNCTION_TRACER
>   	select EDAC_ATOMIC_SCRUB
>   	select EDAC_SUPPORT
> +	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if ARCH_USING_PATCHABLE_FUNCTION_ENTRY
>   	select GENERIC_ATOMIC64			if PPC32
>   	select GENERIC_CLOCKEVENTS_BROADCAST	if SMP
>   	select GENERIC_CMOS_UPDATE
> @@ -227,8 +228,8 @@ config PPC
>   	select HAVE_DEBUG_KMEMLEAK
>   	select HAVE_DEBUG_STACKOVERFLOW
>   	select HAVE_DYNAMIC_FTRACE
> -	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if MPROFILE_KERNEL || PPC32
> -	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if MPROFILE_KERNEL || PPC32
> +	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
> +	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32

ARCH_USING_PATCHABLE_FUNCTION_ENTRY defaults to y if PPC32, so you can 
remove PPC32 from the condition here.

>   	select HAVE_EBPF_JIT
>   	select HAVE_EFFICIENT_UNALIGNED_ACCESS
>   	select HAVE_FAST_GUP
> @@ -256,7 +257,7 @@ config PPC
>   	select HAVE_MOD_ARCH_SPECIFIC
>   	select HAVE_NMI				if PERF_EVENTS || (PPC64 && PPC_BOOK3S)
>   	select HAVE_OPTPROBES
> -	select HAVE_OBJTOOL			if PPC32 || MPROFILE_KERNEL
> +	select HAVE_OBJTOOL			if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32

Same

>   	select HAVE_OBJTOOL_MCOUNT		if HAVE_OBJTOOL
>   	select HAVE_PERF_EVENTS
>   	select HAVE_PERF_EVENTS_NMI		if PPC64
> @@ -550,6 +551,13 @@ config MPROFILE_KERNEL
>   	depends on PPC64 && CPU_LITTLE_ENDIAN && FUNCTION_TRACER
>   	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-mprofile-kernel.sh $(CC) -I$(srctree)/include -D__KERNEL__)
>   
> +config ARCH_USING_PATCHABLE_FUNCTION_ENTRY
> +	depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2)
> +	depends on $(cc-option,-fpatchable-function-entry=2)
> +	def_bool y if PPC32
> +	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN
> +	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN
> +
>   config HOTPLUG_CPU
>   	bool "Support for enabling/disabling CPUs"
>   	depends on SMP && (PPC_PSERIES || \
> diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
> index dca73f673d7046..de39478b1c9e9f 100644
> --- a/arch/powerpc/Makefile
> +++ b/arch/powerpc/Makefile
> @@ -148,11 +148,16 @@ CFLAGS-$(CONFIG_PPC32)	+= $(call cc-option, $(MULTIPLEWORD))
>   CFLAGS-$(CONFIG_PPC32)	+= $(call cc-option,-mno-readonly-in-sdata)
>   
>   ifdef CONFIG_FUNCTION_TRACER
> +ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
> +KBUILD_CPPFLAGS	+= -DCC_USING_PATCHABLE_FUNCTION_ENTRY
> +CC_FLAGS_FTRACE := -fpatchable-function-entry=2
> +else
>   CC_FLAGS_FTRACE := -pg
>   ifdef CONFIG_MPROFILE_KERNEL
>   CC_FLAGS_FTRACE += -mprofile-kernel
>   endif
>   endif
> +endif
>   
>   CFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
>   AFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
> diff --git a/arch/powerpc/include/asm/ftrace.h b/arch/powerpc/include/asm/ftrace.h
> index ef9f0b97670d1c..9e5a39b6a3114b 100644
> --- a/arch/powerpc/include/asm/ftrace.h
> +++ b/arch/powerpc/include/asm/ftrace.h
> @@ -11,7 +11,7 @@
>   #define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
>   
>   /* Ignore unused weak functions which will have larger offsets */
> -#ifdef CONFIG_MPROFILE_KERNEL
> +#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
>   #define FTRACE_MCOUNT_MAX_OFFSET	16
>   #elif defined(CONFIG_PPC32)
>   #define FTRACE_MCOUNT_MAX_OFFSET	8
> @@ -22,7 +22,9 @@ extern void _mcount(void);
>   
>   static inline unsigned long ftrace_call_adjust(unsigned long addr)
>   {
> -       /* relocation of mcount call site is the same as the address */
> +	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY))
> +		addr += MCOUNT_INSN_SIZE;
> +
>          return addr;
>   }
>   
> diff --git a/arch/powerpc/include/asm/vermagic.h b/arch/powerpc/include/asm/vermagic.h
> index b054a8576e5deb..6f250fe506bd1c 100644
> --- a/arch/powerpc/include/asm/vermagic.h
> +++ b/arch/powerpc/include/asm/vermagic.h
> @@ -2,7 +2,9 @@
>   #ifndef _ASM_VERMAGIC_H
>   #define _ASM_VERMAGIC_H
>   
> -#ifdef CONFIG_MPROFILE_KERNEL
> +#ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
> +#define MODULE_ARCH_VERMAGIC_FTRACE	"patchable-function-entry "
> +#elif defined(CONFIG_MPROFILE_KERNEL)
>   #define MODULE_ARCH_VERMAGIC_FTRACE	"mprofile-kernel "
>   #else
>   #define MODULE_ARCH_VERMAGIC_FTRACE	""
> diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
> index 92570289ce08f5..7112adc597a80b 100644
> --- a/arch/powerpc/kernel/module_64.c
> +++ b/arch/powerpc/kernel/module_64.c
> @@ -465,7 +465,7 @@ int module_frob_arch_sections(Elf64_Ehdr *hdr,
>   	return 0;
>   }
>   
> -#ifdef CONFIG_MPROFILE_KERNEL
> +#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
>   
>   static u32 stub_insns[] = {
>   #ifdef CONFIG_PPC_KERNEL_PCREL
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index cf9dce77527920..82010629cf887c 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -220,7 +220,12 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
>   	int ret = 0;
>   
>   	/* Verify instructions surrounding the ftrace location */
> -	if (IS_ENABLED(CONFIG_PPC32)) {
> +	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
> +		/* Expect nops */
> +		ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_NOP()));
> +		if (!ret)
> +			ret = ftrace_validate_inst(ip, ppc_inst(PPC_RAW_NOP()));
> +	} else if (IS_ENABLED(CONFIG_PPC32)) {
>   		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
>   		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
>   		if (!ret)
> @@ -250,7 +255,12 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
>   	/* Nop-out the ftrace location */
>   	new = ppc_inst(PPC_RAW_NOP());
>   	addr = MCOUNT_ADDR;
> -	if (is_offset_in_branch_range(addr - ip)) {
> +	if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
> +		/* we instead patch-in the 'mflr r0' */
> +		old = ppc_inst(PPC_RAW_NOP());
> +		new = ppc_inst(PPC_RAW_MFLR(_R0));
> +		ret = ftrace_modify_code(ip - 4, old, new);
> +	} else if (is_offset_in_branch_range(addr - ip)) {
>   		/* Within range */
>   		old = ftrace_create_branch_inst(ip, addr, 1);
>   		ret = ftrace_modify_code(ip, old, new);
> diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S
> index e8339706e735b1..bab3ab1368a33f 100644
> --- a/arch/powerpc/kernel/trace/ftrace_entry.S
> +++ b/arch/powerpc/kernel/trace/ftrace_entry.S
> @@ -250,6 +250,7 @@ livepatch_handler:
>   	blr
>   #endif /* CONFIG_LIVEPATCH */
>   
> +#ifndef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
>   _GLOBAL(mcount)
>   _GLOBAL(_mcount)
>   EXPORT_SYMBOL(_mcount)
> @@ -257,6 +258,7 @@ EXPORT_SYMBOL(_mcount)
>   	mtctr	r12
>   	mtlr	r0
>   	bctr
> +#endif
>   
>   #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>   _GLOBAL(return_to_handler)
> diff --git a/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh b/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
> new file mode 100755
> index 00000000000000..06706903503b6c
> --- /dev/null
> +++ b/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
> @@ -0,0 +1,26 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +
> +set -e
> +set -o pipefail
> +
> +# To debug, uncomment the following line
> +# set -x
> +
> +# Output from -fpatchable-function-entry can only vary on ppc64 elfv2, so this
> +# should not be invoked for other targets. Therefore we can pass in -m64 and
> +# -mabi explicitly, to take care of toolchains defaulting to other targets.
> +
> +# Test whether the compile option -fpatchable-function-entry exists and
> +# generates appropriate code
> +echo "int func() { return 0; }" | \
> +    $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
> +    grep -q "__patchable_function_entries"
> +
> +# Test whether nops are generated after the local entry point
> +echo "int x; int func() { return x; }" | \
> +    $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
> +    awk 'BEGIN { RS = ";" } /\.localentry.*nop.*\n[[:space:]]*nop/ { print $0 }' | \
> +    grep -q "func:"
> +
> +exit 0

Re: [PATCH 17/17] powerpc/ftrace: Create a dummy stackframe to fix stack unwind

[Christophe Leroy]

Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
> With ppc64 -mprofile-kernel and ppc32 -pg, profiling instructions to
> call into ftrace are emitted right at function entry. The instruction
> sequence used is minimal to reduce overhead. Crucially, a stackframe is
> not created for the function being traced. This breaks stack unwinding
> since the function being traced does not have a stackframe for itself.
> As such, it never shows up in the backtrace:
> 
> /sys/kernel/debug/tracing # echo 1 > /proc/sys/kernel/stack_tracer_enabled
> /sys/kernel/debug/tracing # cat stack_trace
>          Depth    Size   Location    (17 entries)
>          -----    ----   --------
>    0)     4144      32   ftrace_call+0x4/0x44
>    1)     4112     432   get_page_from_freelist+0x26c/0x1ad0
>    2)     3680     496   __alloc_pages+0x290/0x1280
>    3)     3184     336   __folio_alloc+0x34/0x90
>    4)     2848     176   vma_alloc_folio+0xd8/0x540
>    5)     2672     272   __handle_mm_fault+0x700/0x1cc0
>    6)     2400     208   handle_mm_fault+0xf0/0x3f0
>    7)     2192      80   ___do_page_fault+0x3e4/0xbe0
>    8)     2112     160   do_page_fault+0x30/0xc0
>    9)     1952     256   data_access_common_virt+0x210/0x220
>   10)     1696     400   0xc00000000f16b100
>   11)     1296     384   load_elf_binary+0x804/0x1b80
>   12)      912     208   bprm_execve+0x2d8/0x7e0
>   13)      704      64   do_execveat_common+0x1d0/0x2f0
>   14)      640     160   sys_execve+0x54/0x70
>   15)      480      64   system_call_exception+0x138/0x350
>   16)      416     416   system_call_common+0x160/0x2c4
> 
> Fix this by having ftrace create a dummy stackframe for the function
> being traced. Since this is only relevant when ftrace is active, we nop
> out the instruction to store LR in the LR save area in the profiling
> instruction sequence on ppc32 (and in ppc64 with older gcc versions).
> Instead, in ftrace, we store LR in the LR save area of the previous
> stackframe, and create a minimal stackframe to represent the function
> being traced. With this, backtraces now capture the function being
> traced:
> 
> /sys/kernel/debug/tracing # cat stack_trace
>          Depth    Size   Location    (17 entries)
>          -----    ----   --------
>    0)     3888      32   _raw_spin_trylock+0x8/0x70
>    1)     3856     576   get_page_from_freelist+0x26c/0x1ad0
>    2)     3280      64   __alloc_pages+0x290/0x1280
>    3)     3216     336   __folio_alloc+0x34/0x90
>    4)     2880     176   vma_alloc_folio+0xd8/0x540
>    5)     2704     416   __handle_mm_fault+0x700/0x1cc0
>    6)     2288      96   handle_mm_fault+0xf0/0x3f0
>    7)     2192      48   ___do_page_fault+0x3e4/0xbe0
>    8)     2144     192   do_page_fault+0x30/0xc0
>    9)     1952     608   data_access_common_virt+0x210/0x220
>   10)     1344      16   0xc0000000334bbb50
>   11)     1328     416   load_elf_binary+0x804/0x1b80
>   12)      912      64   bprm_execve+0x2d8/0x7e0
>   13)      848     176   do_execveat_common+0x1d0/0x2f0
>   14)      672     192   sys_execve+0x54/0x70
>   15)      480      64   system_call_exception+0x138/0x350
>   16)      416     416   system_call_common+0x160/0x2c4
> 
> This results in two additional stores in the ftrace entry code, but
> produces reliable backtraces. Note that this change now aligns with
> other architectures (arm64, s390, x86).
> 
> Signed-off-by: Naveen N Rao <naveen@kernel.org>

Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

> ---
>   arch/powerpc/kernel/trace/ftrace.c       |  6 ++++--
>   arch/powerpc/kernel/trace/ftrace_entry.S | 11 ++++++++---
>   2 files changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c
> index 82010629cf887c..2956196c98ffdc 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -229,13 +229,15 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
>   		/* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
>   		ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
>   		if (!ret)
> -			ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)));
> +			ret = ftrace_modify_code(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)),
> +						 ppc_inst(PPC_RAW_NOP()));
>   	} else if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
>   		/* Expected sequence: 'mflr r0', ['std r0,16(r1)'], 'bl _mcount' */
>   		ret = ftrace_read_inst(ip - 4, &old);
>   		if (!ret && !ppc_inst_equal(old, ppc_inst(PPC_RAW_MFLR(_R0)))) {
>   			ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
> -			ret |= ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)));
> +			ret |= ftrace_modify_code(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)),
> +						  ppc_inst(PPC_RAW_NOP()));
>   		}
>   	} else {
>   		return -EINVAL;
> diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S
> index bab3ab1368a33f..05e981fb526c2e 100644
> --- a/arch/powerpc/kernel/trace/ftrace_entry.S
> +++ b/arch/powerpc/kernel/trace/ftrace_entry.S
> @@ -33,6 +33,11 @@
>    * and then arrange for the ftrace function to be called.
>    */
>   .macro	ftrace_regs_entry allregs
> +	/* Create a minimal stack frame for representing B */
> +	PPC_STLU	r1, -STACK_FRAME_MIN_SIZE(r1)
> +	/* Save the original return address in A's stack frame */
> +	PPC_STL		r0, LRSAVE+STACK_FRAME_MIN_SIZE(r1)
> +
>   	/* Create our stack frame + pt_regs */
>   	PPC_STLU	r1,-SWITCH_FRAME_SIZE(r1)
>   
> @@ -41,8 +46,6 @@
>   	SAVE_GPRS(3, 10, r1)
>   
>   #ifdef CONFIG_PPC64
> -	/* Save the original return address in A's stack frame */
> -	std	r0, LRSAVE+SWITCH_FRAME_SIZE(r1)
>   	/* Ok to continue? */
>   	lbz	r3, PACA_FTRACE_ENABLED(r13)
>   	cmpdi	r3, 0
> @@ -77,6 +80,8 @@
>   	mflr	r7
>   	/* Save it as pt_regs->nip */
>   	PPC_STL	r7, _NIP(r1)
> +	/* Also save it in B's stackframe header for proper unwind */
> +	PPC_STL	r7, LRSAVE+SWITCH_FRAME_SIZE(r1)
>   	/* Save the read LR in pt_regs->link */
>   	PPC_STL	r0, _LINK(r1)
>   
> @@ -142,7 +147,7 @@
>   #endif
>   
>   	/* Pop our stack frame */
> -	addi r1, r1, SWITCH_FRAME_SIZE
> +	addi r1, r1, SWITCH_FRAME_SIZE+STACK_FRAME_MIN_SIZE
>   
>   #ifdef CONFIG_LIVEPATCH_64
>           /* Based on the cmpd above, if the NIP was altered handle livepatch */

Re: [PATCH 07/17] powerpc/ftrace: Consolidate ftrace support into fewer files

[Naveen N Rao]

Hi Christophe,

Christophe Leroy wrote:
> 
> 
> Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
>> ftrace_low.S has just the _mcount stub and return_to_handler(). Merge
>> this back into ftrace_mprofile.S and ftrace_64_pg.S to keep all ftrace
>> code together, and to allow those to evolve independently.
>> 
>> ftrace_mprofile.S is also not an entirely accurate name since this also
>> holds ppc32 code. This will be all the more incorrect once support for
>> -fpatchable-function-entry is added. Rename files here to more
>> accurately describe the code:
>> - ftrace_mprofile.S is renamed to ftrace_entry.S
>> - ftrace_pg.c is renamed to ftrace_64_pg.c
>> - ftrace_64_pg.S is rename to ftrace_64_pg_entry.S
> 
> The stats from git do not match., it says {ftrace_low.S => 
> ftrace_64_pg_entry.S

I suppose that's git diff view. You can see from the patch below that 
the header of ftrace_64_pg_entry.S carries the line 'Split from 
ftrace_64.S'. In reality, this is still ftrace_low.S being merged into 
ftrace_64_pg.S.

> 
>> 
>> Signed-off-by: Naveen N Rao <naveen@kernel.org>
> 
> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

Thanks for the review.

- Naveen

> 
>> ---
>>   arch/powerpc/kernel/trace/Makefile            | 17 +++--
>>   arch/powerpc/kernel/trace/ftrace_64_pg.S      | 67 -------------------
>>   .../trace/{ftrace_pg.c => ftrace_64_pg.c}     |  0
>>   .../{ftrace_low.S => ftrace_64_pg_entry.S}    | 58 +++++++++++++++-
>>   .../{ftrace_mprofile.S => ftrace_entry.S}     | 65 ++++++++++++++++++
>>   5 files changed, 130 insertions(+), 77 deletions(-)
>>   delete mode 100644 arch/powerpc/kernel/trace/ftrace_64_pg.S
>>   rename arch/powerpc/kernel/trace/{ftrace_pg.c => ftrace_64_pg.c} (100%)
>>   rename arch/powerpc/kernel/trace/{ftrace_low.S => ftrace_64_pg_entry.S} (55%)
>>   rename arch/powerpc/kernel/trace/{ftrace_mprofile.S => ftrace_entry.S} (83%)
>> 
>> diff --git a/arch/powerpc/kernel/trace/Makefile b/arch/powerpc/kernel/trace/Makefile
>> index 342a2d1ae86cd0..125f4ca588b98a 100644
>> --- a/arch/powerpc/kernel/trace/Makefile
>> +++ b/arch/powerpc/kernel/trace/Makefile
>> @@ -6,16 +6,15 @@
>>   ifdef CONFIG_FUNCTION_TRACER
>>   # do not trace tracer code
>>   CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
>> -CFLAGS_REMOVE_ftrace_pg.o = $(CC_FLAGS_FTRACE)
>> +CFLAGS_REMOVE_ftrace_64_pg.o = $(CC_FLAGS_FTRACE)
>>   endif
>>   
>> -obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
>> +obj32-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
>>   ifdef CONFIG_MPROFILE_KERNEL
>> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_mprofile.o ftrace.o
>> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace.o ftrace_entry.o
>>   else
>> -obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_pg.o
>> +obj64-$(CONFIG_FUNCTION_TRACER)		+= ftrace_64_pg.o ftrace_64_pg_entry.o
>>   endif
>> -obj-$(CONFIG_FUNCTION_TRACER)		+= ftrace_low.o
>>   obj-$(CONFIG_TRACING)			+= trace_clock.o
>>   
>>   obj-$(CONFIG_PPC64)			+= $(obj64-y)
>> @@ -26,7 +25,7 @@ GCOV_PROFILE_ftrace.o := n
>>   KCOV_INSTRUMENT_ftrace.o := n
>>   KCSAN_SANITIZE_ftrace.o := n
>>   UBSAN_SANITIZE_ftrace.o := n
>> -GCOV_PROFILE_ftrace_pg.o := n
>> -KCOV_INSTRUMENT_ftrace_pg.o := n
>> -KCSAN_SANITIZE_ftrace_pg.o := n
>> -UBSAN_SANITIZE_ftrace_pg.o := n
>> +GCOV_PROFILE_ftrace_64_pg.o := n
>> +KCOV_INSTRUMENT_ftrace_64_pg.o := n
>> +KCSAN_SANITIZE_ftrace_64_pg.o := n
>> +UBSAN_SANITIZE_ftrace_64_pg.o := n
>> diff --git a/arch/powerpc/kernel/trace/ftrace_64_pg.S b/arch/powerpc/kernel/trace/ftrace_64_pg.S
>> deleted file mode 100644
>> index 6708e24db0aba8..00000000000000
>> --- a/arch/powerpc/kernel/trace/ftrace_64_pg.S
>> +++ /dev/null
>> @@ -1,67 +0,0 @@
>> -/* SPDX-License-Identifier: GPL-2.0-or-later */
>> -/*
>> - * Split from ftrace_64.S
>> - */
>> -
>> -#include <linux/magic.h>
>> -#include <asm/ppc_asm.h>
>> -#include <asm/asm-offsets.h>
>> -#include <asm/ftrace.h>
>> -#include <asm/ppc-opcode.h>
>> -#include <asm/export.h>
>> -
>> -_GLOBAL_TOC(ftrace_caller)
>> -	lbz	r3, PACA_FTRACE_ENABLED(r13)
>> -	cmpdi	r3, 0
>> -	beqlr
>> -
>> -	/* Taken from output of objdump from lib64/glibc */
>> -	mflr	r3
>> -	ld	r11, 0(r1)
>> -	stdu	r1, -112(r1)
>> -	std	r3, 128(r1)
>> -	ld	r4, 16(r11)
>> -	subi	r3, r3, MCOUNT_INSN_SIZE
>> -.globl ftrace_call
>> -ftrace_call:
>> -	bl	ftrace_stub
>> -	nop
>> -#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> -.globl ftrace_graph_call
>> -ftrace_graph_call:
>> -	b	ftrace_graph_stub
>> -_GLOBAL(ftrace_graph_stub)
>> -#endif
>> -	ld	r0, 128(r1)
>> -	mtlr	r0
>> -	addi	r1, r1, 112
>> -
>> -_GLOBAL(ftrace_stub)
>> -	blr
>> -
>> -#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> -_GLOBAL(ftrace_graph_caller)
>> -	addi	r5, r1, 112
>> -	/* load r4 with local address */
>> -	ld	r4, 128(r1)
>> -	subi	r4, r4, MCOUNT_INSN_SIZE
>> -
>> -	/* Grab the LR out of the caller stack frame */
>> -	ld	r11, 112(r1)
>> -	ld	r3, 16(r11)
>> -
>> -	bl	prepare_ftrace_return
>> -	nop
>> -
>> -	/*
>> -	 * prepare_ftrace_return gives us the address we divert to.
>> -	 * Change the LR in the callers stack frame to this.
>> -	 */
>> -	ld	r11, 112(r1)
>> -	std	r3, 16(r11)
>> -
>> -	ld	r0, 128(r1)
>> -	mtlr	r0
>> -	addi	r1, r1, 112
>> -	blr
>> -#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>> diff --git a/arch/powerpc/kernel/trace/ftrace_pg.c b/arch/powerpc/kernel/trace/ftrace_64_pg.c
>> similarity index 100%
>> rename from arch/powerpc/kernel/trace/ftrace_pg.c
>> rename to arch/powerpc/kernel/trace/ftrace_64_pg.c
>> diff --git a/arch/powerpc/kernel/trace/ftrace_low.S b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
>> similarity index 55%
>> rename from arch/powerpc/kernel/trace/ftrace_low.S
>> rename to arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
>> index 2fc7dd0a5ae968..81dbaf70b1513a 100644
>> --- a/arch/powerpc/kernel/trace/ftrace_low.S
>> +++ b/arch/powerpc/kernel/trace/ftrace_64_pg_entry.S
>> @@ -1,6 +1,6 @@
>>   /* SPDX-License-Identifier: GPL-2.0-or-later */
>>   /*
>> - * Split from entry_64.S
>> + * Split from ftrace_64.S
>>    */
>>   
>>   #include <linux/magic.h>
>> @@ -10,6 +10,62 @@
>>   #include <asm/ppc-opcode.h>
>>   #include <asm/export.h>
>>   
>> +_GLOBAL_TOC(ftrace_caller)
>> +	lbz	r3, PACA_FTRACE_ENABLED(r13)
>> +	cmpdi	r3, 0
>> +	beqlr
>> +
>> +	/* Taken from output of objdump from lib64/glibc */
>> +	mflr	r3
>> +	ld	r11, 0(r1)
>> +	stdu	r1, -112(r1)
>> +	std	r3, 128(r1)
>> +	ld	r4, 16(r11)
>> +	subi	r3, r3, MCOUNT_INSN_SIZE
>> +.globl ftrace_call
>> +ftrace_call:
>> +	bl	ftrace_stub
>> +	nop
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> +.globl ftrace_graph_call
>> +ftrace_graph_call:
>> +	b	ftrace_graph_stub
>> +_GLOBAL(ftrace_graph_stub)
>> +#endif
>> +	ld	r0, 128(r1)
>> +	mtlr	r0
>> +	addi	r1, r1, 112
>> +
>> +_GLOBAL(ftrace_stub)
>> +	blr
>> +
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> +_GLOBAL(ftrace_graph_caller)
>> +	addi	r5, r1, 112
>> +	/* load r4 with local address */
>> +	ld	r4, 128(r1)
>> +	subi	r4, r4, MCOUNT_INSN_SIZE
>> +
>> +	/* Grab the LR out of the caller stack frame */
>> +	ld	r11, 112(r1)
>> +	ld	r3, 16(r11)
>> +
>> +	bl	prepare_ftrace_return
>> +	nop
>> +
>> +	/*
>> +	 * prepare_ftrace_return gives us the address we divert to.
>> +	 * Change the LR in the callers stack frame to this.
>> +	 */
>> +	ld	r11, 112(r1)
>> +	std	r3, 16(r11)
>> +
>> +	ld	r0, 128(r1)
>> +	mtlr	r0
>> +	addi	r1, r1, 112
>> +	blr
>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>> +
>>   .pushsection ".tramp.ftrace.text","aw",@progbits;
>>   .globl ftrace_tramp_text
>>   ftrace_tramp_text:
>> diff --git a/arch/powerpc/kernel/trace/ftrace_mprofile.S b/arch/powerpc/kernel/trace/ftrace_entry.S
>> similarity index 83%
>> rename from arch/powerpc/kernel/trace/ftrace_mprofile.S
>> rename to arch/powerpc/kernel/trace/ftrace_entry.S
>> index ffb1db38684998..e8339706e735b1 100644
>> --- a/arch/powerpc/kernel/trace/ftrace_mprofile.S
>> +++ b/arch/powerpc/kernel/trace/ftrace_entry.S
>> @@ -249,3 +249,68 @@ livepatch_handler:
>>   	/* Return to original caller of live patched function */
>>   	blr
>>   #endif /* CONFIG_LIVEPATCH */
>> +
>> +_GLOBAL(mcount)
>> +_GLOBAL(_mcount)
>> +EXPORT_SYMBOL(_mcount)
>> +	mflr	r12
>> +	mtctr	r12
>> +	mtlr	r0
>> +	bctr
>> +
>> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> +_GLOBAL(return_to_handler)
>> +	/* need to save return values */
>> +#ifdef CONFIG_PPC64
>> +	std	r4,  -32(r1)
>> +	std	r3,  -24(r1)
>> +	/* save TOC */
>> +	std	r2,  -16(r1)
>> +	std	r31, -8(r1)
>> +	mr	r31, r1
>> +	stdu	r1, -112(r1)
>> +
>> +	/*
>> +	 * We might be called from a module.
>> +	 * Switch to our TOC to run inside the core kernel.
>> +	 */
>> +	LOAD_PACA_TOC()
>> +#else
>> +	stwu	r1, -16(r1)
>> +	stw	r3, 8(r1)
>> +	stw	r4, 12(r1)
>> +#endif
>> +
>> +	bl	ftrace_return_to_handler
>> +	nop
>> +
>> +	/* return value has real return address */
>> +	mtlr	r3
>> +
>> +#ifdef CONFIG_PPC64
>> +	ld	r1, 0(r1)
>> +	ld	r4,  -32(r1)
>> +	ld	r3,  -24(r1)
>> +	ld	r2,  -16(r1)
>> +	ld	r31, -8(r1)
>> +#else
>> +	lwz	r3, 8(r1)
>> +	lwz	r4, 12(r1)
>> +	addi	r1, r1, 16
>> +#endif
>> +
>> +	/* Jump back to real return address */
>> +	blr
>> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>> +
>> +.pushsection ".tramp.ftrace.text","aw",@progbits;
>> +.globl ftrace_tramp_text
>> +ftrace_tramp_text:
>> +	.space 32
>> +.popsection
>> +
>> +.pushsection ".tramp.ftrace.init","aw",@progbits;
>> +.globl ftrace_tramp_init
>> +ftrace_tramp_init:
>> +	.space 32
>> +.popsection
> 

Re: [PATCH 16/17] powerpc/ftrace: Add support for -fpatchable-function-entry

[Naveen N Rao]

Christophe Leroy wrote:
> 
> 
> Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
>> GCC v13.1 updated support for -fpatchable-function-entry on ppc64le to
>> emit nops after the local entry point, rather than before it. This
>> allows us to use this in the kernel for ftrace purposes. A new script is
>> added under arch/powerpc/tools/ to help detect if nops are emitted after
>> the function local entry point, or before the global entry point.
>> 
>> With -fpatchable-function-entry, we no longer have the profiling
>> instructions generated at function entry, so we only need to validate
>> the presence of two nops at the ftrace location in ftrace_init_nop(). We
>> patch the preceding instruction with 'mflr r0' to match the
>> -mprofile-kernel ABI for subsequent ftrace use.
>> 
>> This changes the profiling instructions used on ppc32. The default -pg
>> option emits an additional 'stw' instruction after 'mflr r0' and before
>> the branch to _mcount 'bl _mcount'. This is very similar to the original
>> -mprofile-kernel implementation on ppc64le, where an additional 'std'
>> instruction was used to save LR to its save location in the caller's
>> stackframe. Subsequently, this additional store was removed in later
>> compiler versions for performance reasons. The same reasons apply for
>> ppc32 so we only patch in a 'mflr r0'.
>> 
>> Signed-off-by: Naveen N Rao <naveen@kernel.org>
> 
> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
> 
> Nit below
> 
>> ---
>>   arch/powerpc/Kconfig                          | 14 +++++++---
>>   arch/powerpc/Makefile                         |  5 ++++
>>   arch/powerpc/include/asm/ftrace.h             |  6 +++--
>>   arch/powerpc/include/asm/vermagic.h           |  4 ++-
>>   arch/powerpc/kernel/module_64.c               |  2 +-
>>   arch/powerpc/kernel/trace/ftrace.c            | 14 ++++++++--
>>   arch/powerpc/kernel/trace/ftrace_entry.S      |  2 ++
>>   .../gcc-check-fpatchable-function-entry.sh    | 26 +++++++++++++++++++
>>   8 files changed, 64 insertions(+), 9 deletions(-)
>>   create mode 100755 arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh
>> 
>> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
>> index bff5820b7cda14..9352d8e68152e1 100644
>> --- a/arch/powerpc/Kconfig
>> +++ b/arch/powerpc/Kconfig
>> @@ -187,6 +187,7 @@ config PPC
>>   	select DYNAMIC_FTRACE			if FUNCTION_TRACER
>>   	select EDAC_ATOMIC_SCRUB
>>   	select EDAC_SUPPORT
>> +	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if ARCH_USING_PATCHABLE_FUNCTION_ENTRY
>>   	select GENERIC_ATOMIC64			if PPC32
>>   	select GENERIC_CLOCKEVENTS_BROADCAST	if SMP
>>   	select GENERIC_CMOS_UPDATE
>> @@ -227,8 +228,8 @@ config PPC
>>   	select HAVE_DEBUG_KMEMLEAK
>>   	select HAVE_DEBUG_STACKOVERFLOW
>>   	select HAVE_DYNAMIC_FTRACE
>> -	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if MPROFILE_KERNEL || PPC32
>> -	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if MPROFILE_KERNEL || PPC32
>> +	select HAVE_DYNAMIC_FTRACE_WITH_ARGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
>> +	select HAVE_DYNAMIC_FTRACE_WITH_REGS	if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
> 
> ARCH_USING_PATCHABLE_FUNCTION_ENTRY defaults to y if PPC32, so you can 
> remove PPC32 from the condition here.

ARCH_USING_PATCHABLE_FUNCTION_ENTRY also depends on compiler support for 
-fpatchable-function-entry, and gcc v5.x doesn't support that. So, we 
need to retain PPC32 here.

- Naveen

> 
>>   	select HAVE_EBPF_JIT
>>   	select HAVE_EFFICIENT_UNALIGNED_ACCESS
>>   	select HAVE_FAST_GUP
>> @@ -256,7 +257,7 @@ config PPC
>>   	select HAVE_MOD_ARCH_SPECIFIC
>>   	select HAVE_NMI				if PERF_EVENTS || (PPC64 && PPC_BOOK3S)
>>   	select HAVE_OPTPROBES
>> -	select HAVE_OBJTOOL			if PPC32 || MPROFILE_KERNEL
>> +	select HAVE_OBJTOOL			if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
> 
> Same
> 
>>   	select HAVE_OBJTOOL_MCOUNT		if HAVE_OBJTOOL
>>   	select HAVE_PERF_EVENTS
>>   	select HAVE_PERF_EVENTS_NMI		if PPC64
>> @@ -550,6 +551,13 @@ config MPROFILE_KERNEL
>>   	depends on PPC64 && CPU_LITTLE_ENDIAN && FUNCTION_TRACER
>>   	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-mprofile-kernel.sh $(CC) -I$(srctree)/include -D__KERNEL__)
>>   
>> +config ARCH_USING_PATCHABLE_FUNCTION_ENTRY
>> +	depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2)
>> +	depends on $(cc-option,-fpatchable-function-entry=2)
>> +	def_bool y if PPC32
>> +	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN
>> +	def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN
>> +
...

Re: [PATCH 17/17] powerpc/ftrace: Create a dummy stackframe to fix stack unwind

[Naveen N Rao]

Christophe Leroy wrote:
> 
> 
> Le 19/06/2023 à 11:47, Naveen N Rao a écrit :
>> With ppc64 -mprofile-kernel and ppc32 -pg, profiling instructions to
>> call into ftrace are emitted right at function entry. The instruction
>> sequence used is minimal to reduce overhead. Crucially, a stackframe is
>> not created for the function being traced. This breaks stack unwinding
>> since the function being traced does not have a stackframe for itself.
>> As such, it never shows up in the backtrace:
>> 
>> /sys/kernel/debug/tracing # echo 1 > /proc/sys/kernel/stack_tracer_enabled
>> /sys/kernel/debug/tracing # cat stack_trace
>>          Depth    Size   Location    (17 entries)
>>          -----    ----   --------
>>    0)     4144      32   ftrace_call+0x4/0x44
>>    1)     4112     432   get_page_from_freelist+0x26c/0x1ad0
>>    2)     3680     496   __alloc_pages+0x290/0x1280
>>    3)     3184     336   __folio_alloc+0x34/0x90
>>    4)     2848     176   vma_alloc_folio+0xd8/0x540
>>    5)     2672     272   __handle_mm_fault+0x700/0x1cc0
>>    6)     2400     208   handle_mm_fault+0xf0/0x3f0
>>    7)     2192      80   ___do_page_fault+0x3e4/0xbe0
>>    8)     2112     160   do_page_fault+0x30/0xc0
>>    9)     1952     256   data_access_common_virt+0x210/0x220
>>   10)     1696     400   0xc00000000f16b100
>>   11)     1296     384   load_elf_binary+0x804/0x1b80
>>   12)      912     208   bprm_execve+0x2d8/0x7e0
>>   13)      704      64   do_execveat_common+0x1d0/0x2f0
>>   14)      640     160   sys_execve+0x54/0x70
>>   15)      480      64   system_call_exception+0x138/0x350
>>   16)      416     416   system_call_common+0x160/0x2c4
>> 
>> Fix this by having ftrace create a dummy stackframe for the function
>> being traced. Since this is only relevant when ftrace is active, we nop
>> out the instruction to store LR in the LR save area in the profiling
>> instruction sequence on ppc32 (and in ppc64 with older gcc versions).
>> Instead, in ftrace, we store LR in the LR save area of the previous
>> stackframe, and create a minimal stackframe to represent the function
>> being traced. With this, backtraces now capture the function being
>> traced:
>> 
>> /sys/kernel/debug/tracing # cat stack_trace
>>          Depth    Size   Location    (17 entries)
>>          -----    ----   --------
>>    0)     3888      32   _raw_spin_trylock+0x8/0x70
>>    1)     3856     576   get_page_from_freelist+0x26c/0x1ad0
>>    2)     3280      64   __alloc_pages+0x290/0x1280
>>    3)     3216     336   __folio_alloc+0x34/0x90
>>    4)     2880     176   vma_alloc_folio+0xd8/0x540
>>    5)     2704     416   __handle_mm_fault+0x700/0x1cc0
>>    6)     2288      96   handle_mm_fault+0xf0/0x3f0
>>    7)     2192      48   ___do_page_fault+0x3e4/0xbe0
>>    8)     2144     192   do_page_fault+0x30/0xc0
>>    9)     1952     608   data_access_common_virt+0x210/0x220
>>   10)     1344      16   0xc0000000334bbb50
>>   11)     1328     416   load_elf_binary+0x804/0x1b80
>>   12)      912      64   bprm_execve+0x2d8/0x7e0
>>   13)      848     176   do_execveat_common+0x1d0/0x2f0
>>   14)      672     192   sys_execve+0x54/0x70
>>   15)      480      64   system_call_exception+0x138/0x350
>>   16)      416     416   system_call_common+0x160/0x2c4
>> 
>> This results in two additional stores in the ftrace entry code, but
>> produces reliable backtraces. Note that this change now aligns with
>> other architectures (arm64, s390, x86).
>> 
>> Signed-off-by: Naveen N Rao <naveen@kernel.org>
> 
> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>

Thanks. This patch can be dropped from this series. I posted a minimal 
fix separately to aid backport:
http://lore.kernel.org/20230621051349.759567-1-naveen@kernel.org

I will post a separate patch to also nop out the duplicate stores in the 
function profiling sequence.


- Naveen

Re: [PATCH 00/17] powerpc/ftrace: refactor and add support for -fpatchable-function-entry

[Michael Ellerman]

On Mon, 19 Jun 2023 15:17:18 +0530, Naveen N Rao wrote:
> Since RFC (*):
> - Patches 1 and 17 have been included in this series due to
>   dependencies. Both had been posted out separately.
> - Patch 10 has a small change to not throw errors when checking
>   instruction sequence generated by older toolchains.
> 
> This has had more testing since and this looks good to me. Christophe
> mentioned that this results in a slowdown with ftrace [de-]activation on
> ppc32, but that isn't performance critical and we can address that
> separately.
> 
> [...]

Patch 1-16 applied to powerpc/next.

[01/17] powerpc/ftrace: Fix dropping weak symbols with older toolchains
        https://git.kernel.org/powerpc/c/f6834c8c59a8e977a6f6e4f96c5d28dfa5db8430
[02/17] powerpc/module: Remove unused .ftrace.tramp section
        https://git.kernel.org/powerpc/c/0240605931ec300ddb698020edff05a4c93edbb2
[03/17] powerpc64/ftrace: Move ELFv1 and -pg support code into a separate file
        https://git.kernel.org/powerpc/c/7f7797b372693ce17223678428490dea2b3e4389
[04/17] powerpc/ftrace: Simplify function_graph support in ftrace.c
        https://git.kernel.org/powerpc/c/96d7a13610abcf6bff9d0d0e195c6d2650310125
[05/17] powerpc/ftrace: Use FTRACE_REGS_ADDR to identify the correct ftrace trampoline
        https://git.kernel.org/powerpc/c/b5efb61c70f8ba9b1e168185530b9c7342184a4c
[06/17] powerpc/ftrace: Extend ftrace support for large kernels to ppc32
        https://git.kernel.org/powerpc/c/f3993a0330e2d11e42c095810c6c33084024df46
[07/17] powerpc/ftrace: Consolidate ftrace support into fewer files
        https://git.kernel.org/powerpc/c/bad90aa52d9a0141c41e00ccd4c40be30a29acc6
[08/17] powerpc/ftrace: Refactor ftrace_modify_code()
        https://git.kernel.org/powerpc/c/f4fcbf2e093e25a7faa8a3c2a5097524114e9547
[09/17] powerpc/ftrace: Stop re-purposing linker generated long branches for ftrace
        https://git.kernel.org/powerpc/c/33bb8a0be9c826fce545ae390ecaf91e96b5db43
[10/17] powerpc/ftrace: Add separate ftrace_init_nop() with additional validation
        https://git.kernel.org/powerpc/c/cc93b9233230312a8a905fabd590c405d60f9edd
[11/17] powerpc/ftrace: Simplify ftrace_make_nop()
        https://git.kernel.org/powerpc/c/562bde0bfc968d212d10ba6bf921a0774feebbac
[12/17] powerpc/ftrace: Simplify ftrace_make_call()
        https://git.kernel.org/powerpc/c/9365e23b15f28b7b3b333a7fc6f4c8e9464ca99f
[13/17] powerpc/ftrace: Simplify ftrace_modify_call()
        https://git.kernel.org/powerpc/c/67385738e3c248673668663ffb434ae4e0abf7f1
[14/17] powerpc/ftrace: Replace use of ftrace_call_replace() with ftrace_create_branch_inst()
        https://git.kernel.org/powerpc/c/a26ce4272eea2b20d4f39b9d7e56daf0c77151d8
[15/17] powerpc/ftrace: Implement ftrace_replace_code()
        https://git.kernel.org/powerpc/c/c91c5a828685563c24ab8879d8386de356d9085a
[16/17] powerpc/ftrace: Add support for -fpatchable-function-entry
        https://git.kernel.org/powerpc/c/0f71dcfb4aef6043da6cc509e7a7f6a3ae87c12d

cheers