From: nstange at suse.de (Nicolai Stange) Subject: [RFC][PATCH v2] ftrace/x86: Emulate call function while updating in breakpoint handler Date: Wed, 01 May 2019 10:26:32 +0200 [thread overview] Message-ID: <87muk6vavb.fsf@suse.de> (raw) In-Reply-To: <20190430175334.423821c0@gandalf.local.home> (Steven Rostedt's message of "Tue, 30 Apr 2019 17:53:34 -0400") Hi Steve, many thanks for moving this forward! Steven Rostedt <rostedt at goodmis.org> writes: > > diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c > index ef49517f6bb2..9160f5cc3b6d 100644 > --- a/arch/x86/kernel/ftrace.c > +++ b/arch/x86/kernel/ftrace.c > @@ -17,6 +17,7 @@ > #include <linux/uaccess.h> > #include <linux/ftrace.h> > #include <linux/percpu.h> > +#include <linux/frame.h> > #include <linux/sched.h> > #include <linux/slab.h> > #include <linux/init.h> > @@ -232,6 +233,9 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, > > static unsigned long ftrace_update_func; > > +/* Used within inline asm below */ > +unsigned long ftrace_update_func_call; > + > static int update_ftrace_func(unsigned long ip, void *new) > { > unsigned char old[MCOUNT_INSN_SIZE]; > @@ -259,6 +263,8 @@ int ftrace_update_ftrace_func(ftrace_func_t func) > unsigned char *new; > int ret; > > + ftrace_update_func_call = (unsigned long)func; > + > new = ftrace_call_replace(ip, (unsigned long)func); > ret = update_ftrace_func(ip, new); > > @@ -280,6 +286,125 @@ static nokprobe_inline int is_ftrace_caller(unsigned long ip) > return 0; > } > > +/* > + * We need to handle the "call func1" -> "call func2" case. > + * Just skipping the call is not sufficient as it will be like > + * changing to "nop" first and then updating the call. But some > + * users of ftrace require calls never to be missed. > + * > + * To emulate the call while converting the call site with a breakpoint, > + * some trampolines are used along with per CPU buffers. > + * There are three trampolines for the call sites and three trampolines > + * for the updating of the call in ftrace trampoline. The three > + * trampolines are: > + * > + * 1) Interrupts are enabled when the breakpoint is hit > + * 2) Interrupts are disabled when the breakpoint is hit > + * 3) The breakpoint was hit in an NMI > + * > + * As per CPU data is used, interrupts must be disabled to prevent them > + * from corrupting the data. A separate NMI trampoline is used for the > + * NMI case. If interrupts are already disabled, then the return path > + * of where the breakpoint was hit (saved in the per CPU data) is pushed > + * on the stack and then a jump to either the ftrace_caller (which will > + * loop through all registered ftrace_ops handlers depending on the ip > + * address), or if its a ftrace trampoline call update, it will call > + * ftrace_update_func_call which will hold the call that should be > + * called. > + */ > +extern asmlinkage void ftrace_emulate_call_irqon(void); > +extern asmlinkage void ftrace_emulate_call_irqoff(void); > +extern asmlinkage void ftrace_emulate_call_nmi(void); > +extern asmlinkage void ftrace_emulate_call_update_irqoff(void); > +extern asmlinkage void ftrace_emulate_call_update_irqon(void); > +extern asmlinkage void ftrace_emulate_call_update_nmi(void); > + > +static DEFINE_PER_CPU(void *, ftrace_bp_call_return); > +static DEFINE_PER_CPU(void *, ftrace_bp_call_nmi_return); Andy mentioned #DB and #MC exceptions here: https://lkml.kernel.org/r/C55DED25-C60D-4731-9A6B-92BDA8771766 at amacapital.net I think that #DB won't be possible, provided the trampolines below get tagged as NOKPROBE (do_int3() and ftrace_int3_handler() already have it). It's highly theoretic, but tracing do_machine_check() could clobber ftrace_bp_call_return or ftrace_bp_call_nmi_return? > +#ifdef CONFIG_SMP > +#ifdef CONFIG_X86_64 > +# define BP_CALL_RETURN "%gs:ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "%gs:ftrace_bp_call_nmi_return" > +#else > +# define BP_CALL_RETURN "%fs:ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "%fs:ftrace_bp_call_nmi_return" > +#endif > +#else /* SMP */ > +# define BP_CALL_RETURN "ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "ftrace_bp_call_nmi_return" > +#endif > + > +/* To hold the ftrace_caller address to push on the stack */ > +void *ftrace_caller_func = (void *)ftrace_caller; The live patching ftrace_ops need ftrace_regs_caller. > + > +asm( > + ".text\n" > + > + /* Trampoline for function update with interrupts enabled */ > + ".global ftrace_emulate_call_irqoff\n" > + ".type ftrace_emulate_call_irqoff, @function\n" > + "ftrace_emulate_call_irqoff:\n\t" > + "push "BP_CALL_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "sti\n\t" > + "ret\n\t" > + ".size ftrace_emulate_call_irqoff, .-ftrace_emulate_call_irqoff\n" > + > + /* Trampoline for function update with interrupts disabled*/ > + ".global ftrace_emulate_call_irqon\n" The naming is perhaps a bit confusing, i.e. "update with interrupts disabled" vs. "irqon"... How about swapping irqoff<->irqon? Thanks, Nicolai > + ".type ftrace_emulate_call_irqon, @function\n" > + "ftrace_emulate_call_irqon:\n\t" > + "push "BP_CALL_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "ret\n\t" > + ".size ftrace_emulate_call_irqon, .-ftrace_emulate_call_irqon\n" > + > + /* Trampoline for function update in an NMI */ > + ".global ftrace_emulate_call_nmi\n" > + ".type ftrace_emulate_call_nmi, @function\n" > + "ftrace_emulate_call_nmi:\n\t" > + "push "BP_CALL_NMI_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "ret\n\t" > + ".size ftrace_emulate_call_nmi, .-ftrace_emulate_call_nmi\n" > + -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
WARNING: multiple messages have this Message-ID (diff)
From: nstange@suse.de (Nicolai Stange) Subject: [RFC][PATCH v2] ftrace/x86: Emulate call function while updating in breakpoint handler Date: Wed, 01 May 2019 10:26:32 +0200 [thread overview] Message-ID: <87muk6vavb.fsf@suse.de> (raw) Message-ID: <20190501082632.MqY4SbeY3n4ArPnsjaguCHjs27T-9MyOAmQKeKtKm7E@z> (raw) In-Reply-To: <20190430175334.423821c0@gandalf.local.home> (Steven Rostedt's message of "Tue, 30 Apr 2019 17:53:34 -0400") Hi Steve, many thanks for moving this forward! Steven Rostedt <rostedt at goodmis.org> writes: > > diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c > index ef49517f6bb2..9160f5cc3b6d 100644 > --- a/arch/x86/kernel/ftrace.c > +++ b/arch/x86/kernel/ftrace.c > @@ -17,6 +17,7 @@ > #include <linux/uaccess.h> > #include <linux/ftrace.h> > #include <linux/percpu.h> > +#include <linux/frame.h> > #include <linux/sched.h> > #include <linux/slab.h> > #include <linux/init.h> > @@ -232,6 +233,9 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, > > static unsigned long ftrace_update_func; > > +/* Used within inline asm below */ > +unsigned long ftrace_update_func_call; > + > static int update_ftrace_func(unsigned long ip, void *new) > { > unsigned char old[MCOUNT_INSN_SIZE]; > @@ -259,6 +263,8 @@ int ftrace_update_ftrace_func(ftrace_func_t func) > unsigned char *new; > int ret; > > + ftrace_update_func_call = (unsigned long)func; > + > new = ftrace_call_replace(ip, (unsigned long)func); > ret = update_ftrace_func(ip, new); > > @@ -280,6 +286,125 @@ static nokprobe_inline int is_ftrace_caller(unsigned long ip) > return 0; > } > > +/* > + * We need to handle the "call func1" -> "call func2" case. > + * Just skipping the call is not sufficient as it will be like > + * changing to "nop" first and then updating the call. But some > + * users of ftrace require calls never to be missed. > + * > + * To emulate the call while converting the call site with a breakpoint, > + * some trampolines are used along with per CPU buffers. > + * There are three trampolines for the call sites and three trampolines > + * for the updating of the call in ftrace trampoline. The three > + * trampolines are: > + * > + * 1) Interrupts are enabled when the breakpoint is hit > + * 2) Interrupts are disabled when the breakpoint is hit > + * 3) The breakpoint was hit in an NMI > + * > + * As per CPU data is used, interrupts must be disabled to prevent them > + * from corrupting the data. A separate NMI trampoline is used for the > + * NMI case. If interrupts are already disabled, then the return path > + * of where the breakpoint was hit (saved in the per CPU data) is pushed > + * on the stack and then a jump to either the ftrace_caller (which will > + * loop through all registered ftrace_ops handlers depending on the ip > + * address), or if its a ftrace trampoline call update, it will call > + * ftrace_update_func_call which will hold the call that should be > + * called. > + */ > +extern asmlinkage void ftrace_emulate_call_irqon(void); > +extern asmlinkage void ftrace_emulate_call_irqoff(void); > +extern asmlinkage void ftrace_emulate_call_nmi(void); > +extern asmlinkage void ftrace_emulate_call_update_irqoff(void); > +extern asmlinkage void ftrace_emulate_call_update_irqon(void); > +extern asmlinkage void ftrace_emulate_call_update_nmi(void); > + > +static DEFINE_PER_CPU(void *, ftrace_bp_call_return); > +static DEFINE_PER_CPU(void *, ftrace_bp_call_nmi_return); Andy mentioned #DB and #MC exceptions here: https://lkml.kernel.org/r/C55DED25-C60D-4731-9A6B-92BDA8771766 at amacapital.net I think that #DB won't be possible, provided the trampolines below get tagged as NOKPROBE (do_int3() and ftrace_int3_handler() already have it). It's highly theoretic, but tracing do_machine_check() could clobber ftrace_bp_call_return or ftrace_bp_call_nmi_return? > +#ifdef CONFIG_SMP > +#ifdef CONFIG_X86_64 > +# define BP_CALL_RETURN "%gs:ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "%gs:ftrace_bp_call_nmi_return" > +#else > +# define BP_CALL_RETURN "%fs:ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "%fs:ftrace_bp_call_nmi_return" > +#endif > +#else /* SMP */ > +# define BP_CALL_RETURN "ftrace_bp_call_return" > +# define BP_CALL_NMI_RETURN "ftrace_bp_call_nmi_return" > +#endif > + > +/* To hold the ftrace_caller address to push on the stack */ > +void *ftrace_caller_func = (void *)ftrace_caller; The live patching ftrace_ops need ftrace_regs_caller. > + > +asm( > + ".text\n" > + > + /* Trampoline for function update with interrupts enabled */ > + ".global ftrace_emulate_call_irqoff\n" > + ".type ftrace_emulate_call_irqoff, @function\n" > + "ftrace_emulate_call_irqoff:\n\t" > + "push "BP_CALL_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "sti\n\t" > + "ret\n\t" > + ".size ftrace_emulate_call_irqoff, .-ftrace_emulate_call_irqoff\n" > + > + /* Trampoline for function update with interrupts disabled*/ > + ".global ftrace_emulate_call_irqon\n" The naming is perhaps a bit confusing, i.e. "update with interrupts disabled" vs. "irqon"... How about swapping irqoff<->irqon? Thanks, Nicolai > + ".type ftrace_emulate_call_irqon, @function\n" > + "ftrace_emulate_call_irqon:\n\t" > + "push "BP_CALL_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "ret\n\t" > + ".size ftrace_emulate_call_irqon, .-ftrace_emulate_call_irqon\n" > + > + /* Trampoline for function update in an NMI */ > + ".global ftrace_emulate_call_nmi\n" > + ".type ftrace_emulate_call_nmi, @function\n" > + "ftrace_emulate_call_nmi:\n\t" > + "push "BP_CALL_NMI_RETURN"\n\t" > + "push ftrace_caller_func\n" > + "ret\n\t" > + ".size ftrace_emulate_call_nmi, .-ftrace_emulate_call_nmi\n" > + -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
next prev parent reply other threads:[~2019-05-01 8:26 UTC|newest] Thread overview: 128+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-27 10:06 [PATCH 0/4] x86/ftrace: make ftrace_int3_handler() not to skip fops invocation nstange 2019-04-27 10:06 ` Nicolai Stange 2019-04-27 10:06 ` [PATCH 1/4] x86/thread_info: introduce ->ftrace_int3_stack member nstange 2019-04-27 10:06 ` Nicolai Stange 2019-04-28 17:41 ` luto 2019-04-28 17:41 ` Andy Lutomirski 2019-04-28 17:51 ` rostedt 2019-04-28 17:51 ` Steven Rostedt 2019-04-28 18:08 ` luto 2019-04-28 18:08 ` Andy Lutomirski 2019-04-28 19:43 ` rostedt 2019-04-28 19:43 ` Steven Rostedt 2019-04-28 20:56 ` luto 2019-04-28 20:56 ` Andy Lutomirski 2019-04-28 21:22 ` nstange 2019-04-28 21:22 ` Nicolai Stange 2019-04-28 23:27 ` luto 2019-04-28 23:27 ` Andy Lutomirski 2019-04-27 10:06 ` [PATCH 2/4] ftrace: drop 'static' qualifier from ftrace_ops_list_func() nstange 2019-04-27 10:06 ` Nicolai Stange 2019-04-27 10:06 ` [PATCH 3/4] x86/ftrace: make ftrace_int3_handler() not to skip fops invocation nstange 2019-04-27 10:06 ` Nicolai Stange 2019-04-27 10:26 ` peterz 2019-04-27 10:26 ` Peter Zijlstra 2019-04-28 17:38 ` rostedt 2019-04-28 17:38 ` Steven Rostedt 2019-04-29 18:06 ` torvalds 2019-04-29 18:06 ` Linus Torvalds 2019-04-29 18:22 ` torvalds 2019-04-29 18:22 ` Linus Torvalds 2019-04-29 18:42 ` luto 2019-04-29 18:42 ` Andy Lutomirski [not found] ` <CAHk-=whtt4K2f0KPtG-4Pykh3FK8UBOjD8jhXCUKB5nWDj_YRA@mail.gmail.com> 2019-04-29 18:56 ` luto 2019-04-29 18:56 ` Andy Lutomirski [not found] ` <CAHk-=wgewK4eFhF3=0RNtk1KQjMANFH6oDE=8m=84RExn2gxhw@mail.gmail.com> [not found] ` <CAHk-=whay7eN6+2gZjY-ybRbkbcqAmgrLwwszzHx8ws3c=S-MA@mail.gmail.com> 2019-04-29 19:24 ` luto 2019-04-29 19:24 ` Andy Lutomirski 2019-04-29 20:07 ` torvalds 2019-04-29 20:07 ` Linus Torvalds 2019-04-30 13:56 ` peterz 2019-04-30 13:56 ` Peter Zijlstra 2019-04-30 16:06 ` torvalds 2019-04-30 16:06 ` Linus Torvalds 2019-04-30 16:33 ` luto 2019-04-30 16:33 ` Andy Lutomirski 2019-04-30 17:03 ` rostedt 2019-04-30 17:03 ` Steven Rostedt 2019-04-30 17:20 ` rostedt 2019-04-30 17:20 ` Steven Rostedt 2019-04-30 17:49 ` [RFC][PATCH] ftrace/x86: Emulate call function while updating in breakpoint handler rostedt 2019-04-30 17:49 ` Steven Rostedt 2019-04-30 18:33 ` torvalds 2019-04-30 18:33 ` Linus Torvalds 2019-04-30 19:00 ` rostedt 2019-04-30 19:00 ` Steven Rostedt 2019-04-30 21:08 ` rostedt 2019-04-30 21:08 ` Steven Rostedt 2019-05-01 13:11 ` peterz 2019-05-01 13:11 ` Peter Zijlstra 2019-05-01 18:58 ` rostedt 2019-05-01 18:58 ` Steven Rostedt 2019-05-01 19:03 ` peterz 2019-05-01 19:03 ` Peter Zijlstra 2019-05-01 19:03 ` torvalds 2019-05-01 19:03 ` Linus Torvalds 2019-05-01 19:13 ` peterz 2019-05-01 19:13 ` Peter Zijlstra 2019-05-01 19:13 ` rostedt 2019-05-01 19:13 ` Steven Rostedt 2019-05-01 19:33 ` jikos 2019-05-01 19:33 ` Jiri Kosina 2019-05-01 19:41 ` peterz 2019-05-01 19:41 ` Peter Zijlstra 2019-04-30 21:53 ` [RFC][PATCH v2] " rostedt 2019-04-30 21:53 ` Steven Rostedt 2019-05-01 1:35 ` rostedt 2019-05-01 1:35 ` Steven Rostedt 2019-05-01 1:58 ` torvalds 2019-05-01 1:58 ` Linus Torvalds 2019-05-01 8:26 ` nstange [this message] 2019-05-01 8:26 ` Nicolai Stange 2019-05-01 13:22 ` rostedt 2019-05-01 13:22 ` Steven Rostedt 2019-04-29 20:16 ` [PATCH 3/4] x86/ftrace: make ftrace_int3_handler() not to skip fops invocation torvalds 2019-04-29 20:16 ` Linus Torvalds 2019-04-29 22:08 ` sean.j.christopherson 2019-04-29 22:08 ` Sean Christopherson 2019-04-29 22:22 ` torvalds 2019-04-29 22:22 ` Linus Torvalds 2019-04-30 0:08 ` sean.j.christopherson 2019-04-30 0:08 ` Sean Christopherson 2019-04-30 0:45 ` sean.j.christopherson 2019-04-30 0:45 ` Sean Christopherson 2019-04-30 2:26 ` torvalds 2019-04-30 2:26 ` Linus Torvalds 2019-04-30 10:40 ` peterz 2019-04-30 10:40 ` Peter Zijlstra 2019-04-30 11:17 ` jikos 2019-04-30 11:17 ` Jiri Kosina 2019-04-29 22:06 ` torvalds 2019-04-29 22:06 ` Linus Torvalds 2019-04-30 11:18 ` peterz 2019-04-30 11:18 ` Peter Zijlstra 2019-04-29 18:52 ` rostedt 2019-04-29 18:52 ` Steven Rostedt [not found] ` <CAHk-=wjm93jLtVxTX4HZs6K4k1Wqh3ujjmapqaYtcibVk_YnzQ@mail.gmail.com> 2019-04-29 19:07 ` rostedt 2019-04-29 19:07 ` Steven Rostedt 2019-04-29 20:06 ` torvalds 2019-04-29 20:06 ` Linus Torvalds 2019-04-29 20:20 ` torvalds 2019-04-29 20:20 ` Linus Torvalds 2019-04-29 20:30 ` rostedt 2019-04-29 20:30 ` Steven Rostedt 2019-04-29 21:38 ` torvalds 2019-04-29 21:38 ` Linus Torvalds 2019-04-29 22:07 ` rostedt 2019-04-29 22:07 ` Steven Rostedt 2019-04-30 9:24 ` nstange 2019-04-30 9:24 ` Nicolai Stange 2019-04-30 10:46 ` peterz 2019-04-30 10:46 ` Peter Zijlstra 2019-04-30 13:44 ` rostedt 2019-04-30 13:44 ` Steven Rostedt 2019-04-30 14:20 ` peterz 2019-04-30 14:20 ` Peter Zijlstra 2019-04-30 14:36 ` rostedt 2019-04-30 14:36 ` Steven Rostedt 2019-04-27 10:06 ` [PATCH 4/4] selftests/livepatch: add "ftrace a live patched function" test nstange 2019-04-27 10:06 ` Nicolai Stange
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=87muk6vavb.fsf@suse.de \ --to=linux-kselftest@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).