From: Thomas Gleixner <tglx@linutronix.de> To: LKML <linux-kernel@vger.kernel.org> Cc: x86@kernel.org, Paul McKenney <paulmck@kernel.org>, Josh Poimboeuf <jpoimboe@redhat.com>, "Joel Fernandes (Google)" <joel@joelfernandes.org>, "Steven Rostedt (VMware)" <rostedt@goodmis.org>, Masami Hiramatsu <mhiramat@kernel.org>, Alexei Starovoitov <ast@kernel.org>, Frederic Weisbecker <frederic@kernel.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Brian Gerst <brgerst@gmail.com>, Juergen Gross <jgross@suse.com>, Alexandre Chartre <alexandre.chartre@oracle.com>, Peter Zijlstra <peterz@infradead.org>, Tom Lendacky <thomas.lendacky@amd.com>, Paolo Bonzini <pbonzini@redhat.com>, kvm@vger.kernel.org Subject: [patch V3 03/23] vmlinux.lds.h: Create section for protection against instrumentation Date: Fri, 20 Mar 2020 18:59:59 +0100 [thread overview] Message-ID: <20200320180032.708673769@linutronix.de> (raw) In-Reply-To: 20200320175956.033706968@linutronix.de Some code pathes, especially the low level entry code, must be protected against instrumentation for various reasons: - Low level entry code can be a fragile beast, especially on x86. - With NO_HZ_FULL RCU state needs to be established before using it. Having a dedicated section for such code allows to validate with tooling that no unsafe functions are invoked. Add the .noinstr.text section and the noinstr attribute to mark functions. noinstr implies notrace. Kprobes will gain a section check later. Provide also two sets of markers: - instr_begin()/end() This is used to mark code inside in a noinstr function which calls into regular instrumentable text section as safe. - noinstr_call_begin()/end() Same as above but used to mark indirect calls which cannot be tracked by tooling and need to be audited manually. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- include/asm-generic/sections.h | 3 +++ include/asm-generic/vmlinux.lds.h | 4 ++++ include/linux/compiler.h | 24 ++++++++++++++++++++++++ include/linux/compiler_types.h | 4 ++++ scripts/mod/modpost.c | 2 +- 5 files changed, 36 insertions(+), 1 deletion(-) --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -53,6 +53,9 @@ extern char __ctors_start[], __ctors_end /* Start and end of .opd section - used for function descriptors. */ extern char __start_opd[], __end_opd[]; +/* Start and end of instrumentation protected text section */ +extern char __noinstr_text_start[], __noinstr_text_end[]; + extern __visible const void __nosave_begin, __nosave_end; /* Function descriptor handling (if any). Override in asm/sections.h */ --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -550,6 +550,10 @@ #define TEXT_TEXT \ ALIGN_FUNCTION(); \ *(.text.hot TEXT_MAIN .text.fixup .text.unlikely) \ + ALIGN_FUNCTION(); \ + __noinstr_text_start = .; \ + *(.noinstr.text) \ + __noinstr_text_end = .; \ *(.text..refcount) \ *(.ref.text) \ MEM_KEEP(init.text*) \ --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -120,12 +120,36 @@ void ftrace_likely_update(struct ftrace_ /* Annotate a C jump table to allow objtool to follow the code flow */ #define __annotate_jump_table __section(.rodata..c_jump_table) +/* Begin/end of an instrumentation safe region */ +#define instr_begin() ({ \ + asm volatile("%c0:\n\t" \ + ".pushsection .discard.instr_begin\n\t" \ + ".long %c0b - .\n\t" \ + ".popsection\n\t" : : "i" (__COUNTER__)); \ +}) + +#define instr_end() ({ \ + asm volatile("%c0:\n\t" \ + ".pushsection .discard.instr_end\n\t" \ + ".long %c0b - .\n\t" \ + ".popsection\n\t" : : "i" (__COUNTER__)); \ +}) + #else #define annotate_reachable() #define annotate_unreachable() #define __annotate_jump_table +#define instr_begin() do { } while(0) +#define instr_end() do { } while(0) #endif +/* + * Annotation for audited indirect calls. Distinct from instr_begin() on + * purpose because the called function has to be noinstr as well. + */ +#define noinstr_call_begin() instr_begin() +#define noinstr_call_end() instr_end() + #ifndef ASM_UNREACHABLE # define ASM_UNREACHABLE #endif --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -118,6 +118,10 @@ struct ftrace_likely_data { #define notrace __attribute__((__no_instrument_function__)) #endif +/* Section for code which can't be instrumented at all */ +#define noinstr \ + noinline notrace __attribute((__section__(".noinstr.text"))) + /* * it doesn't make sense on ARM (currently the only user of __naked) * to trace naked functions because then mcount is called without --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -953,7 +953,7 @@ static void check_section(const char *mo #define DATA_SECTIONS ".data", ".data.rel" #define TEXT_SECTIONS ".text", ".text.unlikely", ".sched.text", \ - ".kprobes.text", ".cpuidle.text" + ".kprobes.text", ".cpuidle.text", ".noinstr.text" #define OTHER_TEXT_SECTIONS ".ref.text", ".head.text", ".spinlock.text", \ ".fixup", ".entry.text", ".exception.text", ".text.*", \ ".coldtext"
WARNING: multiple messages have this Message-ID (diff)
From: Thomas Gleixner <tglx@linutronix.de> To: LKML <linux-kernel@vger.kernel.org> Cc: x86@kernel.org, Paul McKenney <paulmck@kernel.org>, Josh Poimboeuf <jpoimboe@redhat.com>, "Joel Fernandes (Google)" <joel@joelfernandes.org>, "Steven Rostedt (VMware)" <rostedt@goodmis.org>, Masami Hiramatsu <mhiramat@kernel.org>, Alexei Starovoitov <ast@kernel.org>, Frederic Weisbecker <frederic@kernel.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Brian Gerst <brgerst@gmail.com>, Juergen Gross <jgross@suse.com>, Alexandre Chartre <alexandre.chartre@oracle.com>, Peter Zijlstra <peterz@infradead.org>, Tom Lendacky <thomas.lendacky@amd.com>, Paolo Bonzini <pbonzini@redhat.com>, kvm@vger.kernel.org Subject: [RESEND][patch V3 03/23] vmlinux.lds.h: Create section for protection against instrumentation Date: Fri, 20 Mar 2020 18:59:59 +0100 [thread overview] Message-ID: <20200320180032.708673769@linutronix.de> (raw) Message-ID: <20200320175959.cJmUKfkRu8ebp387qX4Bo5PjZeMJ9BzTIqaTcSk2xfs@z> (raw) In-Reply-To: 20200320175956.033706968@linutronix.de Some code pathes, especially the low level entry code, must be protected against instrumentation for various reasons: - Low level entry code can be a fragile beast, especially on x86. - With NO_HZ_FULL RCU state needs to be established before using it. Having a dedicated section for such code allows to validate with tooling that no unsafe functions are invoked. Add the .noinstr.text section and the noinstr attribute to mark functions. noinstr implies notrace. Kprobes will gain a section check later. Provide also two sets of markers: - instr_begin()/end() This is used to mark code inside in a noinstr function which calls into regular instrumentable text section as safe. - noinstr_call_begin()/end() Same as above but used to mark indirect calls which cannot be tracked by tooling and need to be audited manually. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- include/asm-generic/sections.h | 3 +++ include/asm-generic/vmlinux.lds.h | 4 ++++ include/linux/compiler.h | 24 ++++++++++++++++++++++++ include/linux/compiler_types.h | 4 ++++ scripts/mod/modpost.c | 2 +- 5 files changed, 36 insertions(+), 1 deletion(-) --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -53,6 +53,9 @@ extern char __ctors_start[], __ctors_end /* Start and end of .opd section - used for function descriptors. */ extern char __start_opd[], __end_opd[]; +/* Start and end of instrumentation protected text section */ +extern char __noinstr_text_start[], __noinstr_text_end[]; + extern __visible const void __nosave_begin, __nosave_end; /* Function descriptor handling (if any). Override in asm/sections.h */ --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -550,6 +550,10 @@ #define TEXT_TEXT \ ALIGN_FUNCTION(); \ *(.text.hot TEXT_MAIN .text.fixup .text.unlikely) \ + ALIGN_FUNCTION(); \ + __noinstr_text_start = .; \ + *(.noinstr.text) \ + __noinstr_text_end = .; \ *(.text..refcount) \ *(.ref.text) \ MEM_KEEP(init.text*) \ --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -120,12 +120,36 @@ void ftrace_likely_update(struct ftrace_ /* Annotate a C jump table to allow objtool to follow the code flow */ #define __annotate_jump_table __section(.rodata..c_jump_table) +/* Begin/end of an instrumentation safe region */ +#define instr_begin() ({ \ + asm volatile("%c0:\n\t" \ + ".pushsection .discard.instr_begin\n\t" \ + ".long %c0b - .\n\t" \ + ".popsection\n\t" : : "i" (__COUNTER__)); \ +}) + +#define instr_end() ({ \ + asm volatile("%c0:\n\t" \ + ".pushsection .discard.instr_end\n\t" \ + ".long %c0b - .\n\t" \ + ".popsection\n\t" : : "i" (__COUNTER__)); \ +}) + #else #define annotate_reachable() #define annotate_unreachable() #define __annotate_jump_table +#define instr_begin() do { } while(0) +#define instr_end() do { } while(0) #endif +/* + * Annotation for audited indirect calls. Distinct from instr_begin() on + * purpose because the called function has to be noinstr as well. + */ +#define noinstr_call_begin() instr_begin() +#define noinstr_call_end() instr_end() + #ifndef ASM_UNREACHABLE # define ASM_UNREACHABLE #endif --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -118,6 +118,10 @@ struct ftrace_likely_data { #define notrace __attribute__((__no_instrument_function__)) #endif +/* Section for code which can't be instrumented at all */ +#define noinstr \ + noinline notrace __attribute((__section__(".noinstr.text"))) + /* * it doesn't make sense on ARM (currently the only user of __naked) * to trace naked functions because then mcount is called without --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -953,7 +953,7 @@ static void check_section(const char *mo #define DATA_SECTIONS ".data", ".data.rel" #define TEXT_SECTIONS ".text", ".text.unlikely", ".sched.text", \ - ".kprobes.text", ".cpuidle.text" + ".kprobes.text", ".cpuidle.text", ".noinstr.text" #define OTHER_TEXT_SECTIONS ".ref.text", ".head.text", ".spinlock.text", \ ".fixup", ".entry.text", ".exception.text", ".text.*", \ ".coldtext"
next prev parent reply other threads:[~2020-03-20 18:39 UTC|newest] Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-20 17:59 [patch V3 00/23] x86/entry: Consolidation part II (syscalls) Thomas Gleixner 2020-03-20 17:59 ` [RESEND][patch " Thomas Gleixner 2020-03-20 17:59 ` [patch V3 01/23] rcu: Dont acquire lock in NMI handler in rcu_nmi_enter_common() Thomas Gleixner 2020-03-20 17:59 ` [RESEND][patch " Thomas Gleixner 2020-03-24 15:37 ` [patch " Frederic Weisbecker 2020-03-20 17:59 ` [patch V3 02/23] rcu: Add comments marking transitions between RCU watching and not Thomas Gleixner 2020-03-20 17:59 ` [RESEND][patch " Thomas Gleixner 2020-03-24 15:38 ` [patch " Frederic Weisbecker 2020-03-20 17:59 ` Thomas Gleixner [this message] 2020-03-20 17:59 ` [RESEND][patch V3 03/23] vmlinux.lds.h: Create section for protection against instrumentation Thomas Gleixner 2020-03-24 12:26 ` Borislav Petkov 2020-04-03 8:08 ` Alexandre Chartre 2020-03-20 18:00 ` [patch V3 04/23] kprobes: Prevent probes in .noinstr.text section Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-23 14:00 ` [patch " Masami Hiramatsu 2020-03-23 16:03 ` Thomas Gleixner 2020-03-24 5:49 ` Masami Hiramatsu 2020-03-24 9:47 ` Thomas Gleixner 2020-03-25 13:39 ` Masami Hiramatsu 2020-03-20 18:00 ` [patch V3 05/23] tracing: Provide lockdep less trace_hardirqs_on/off() variants Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-04-03 8:34 ` Alexandre Chartre 2020-03-20 18:00 ` [patch V3 06/23] bug: Annotate WARN/BUG/stackfail as noinstr safe Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-04-02 21:01 ` Josh Poimboeuf 2020-04-02 21:34 ` Peter Zijlstra 2020-04-02 21:43 ` Josh Poimboeuf 2020-04-02 21:49 ` Thomas Gleixner 2020-03-20 18:00 ` [patch V3 07/23] lockdep: Prepare for noinstr sections Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 08/23] x86/entry: Mark enter_from_user_mode() noinstr Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 09/23] x86/entry/common: Protect against instrumentation Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 10/23] x86/entry: Move irq tracing on syscall entry to C-code Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 11/23] x86/entry: Move irq flags tracing to prepare_exit_to_usermode() Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 12/23] context_tracking: Ensure that the critical path cannot be instrumented Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 13/23] lib/smp_processor_id: Move it into noinstr section Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 14/23] x86/speculation/mds: Mark mds_user_clear_cpu_buffers() __always_inline Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 15/23] x86/entry/64: Check IF in __preempt_enable_notrace() thunk Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 16/23] x86/entry/64: Mark ___preempt_schedule_notrace() thunk noinstr Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 17/23] rcu/tree: Mark the idle relevant functions noinstr Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-24 16:09 ` Paul E. McKenney 2020-03-24 19:28 ` Thomas Gleixner 2020-03-24 19:58 ` Paul E. McKenney 2020-03-20 18:00 ` [patch V3 18/23] x86/kvm: Move context tracking where it belongs Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 19/23] x86/kvm/vmx: Add hardirq tracing to guest enter/exit Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-24 23:03 ` Peter Zijlstra 2020-03-24 23:21 ` Thomas Gleixner 2020-03-20 18:00 ` [patch V3 20/23] x86/kvm/svm: Handle hardirqs proper on " Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 21/23] context_tracking: Make guest_enter/exit_irqoff() .noinstr ready Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 22/23] x86/kvm/vmx: Move guest enter/exit into .noinstr.text Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner 2020-03-20 18:00 ` [patch V3 23/23] x86/kvm/svm: " Thomas Gleixner 2020-03-20 18:00 ` [RESEND][patch " Thomas Gleixner
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200320180032.708673769@linutronix.de \ --to=tglx@linutronix.de \ --cc=alexandre.chartre@oracle.com \ --cc=ast@kernel.org \ --cc=brgerst@gmail.com \ --cc=frederic@kernel.org \ --cc=jgross@suse.com \ --cc=joel@joelfernandes.org \ --cc=jpoimboe@redhat.com \ --cc=kvm@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mathieu.desnoyers@efficios.com \ --cc=mhiramat@kernel.org \ --cc=paulmck@kernel.org \ --cc=pbonzini@redhat.com \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=thomas.lendacky@amd.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.