From: Nadav Amit <namit@vmware.com> To: Ingo Molnar <mingo@redhat.com> Cc: <linux-kernel@vger.kernel.org>, <x86@kernel.org>, Nadav Amit <namit@vmware.com>, Christopher Li <sparse@chrisli.org>, <linux-sparse@vger.kernel.org> Subject: [PATCH v9 03/10] x86: objtool: use asm macro for better compiler decisions Date: Wed, 3 Oct 2018 14:30:53 -0700 [thread overview] Message-ID: <20181003213100.189959-4-namit@vmware.com> (raw) In-Reply-To: <20181003213100.189959-1-namit@vmware.com> GCC considers the number of statements in inlined assembly blocks, according to new-lines and semicolons, as an indication to the cost of the block in time and space. This data is distorted by the kernel code, which puts information in alternative sections. As a result, the compiler may perform incorrect inlining and branch optimizations. In the case of objtool, this distortion is extreme, since anyhow the annotations of objtool are discarded during linkage. The solution is to set an assembly macro and call it from the inline assembly block. As a result GCC considers the inline assembly block as a single instruction. This patch slightly increases the kernel size. text data bss dec hex filename 18140829 10224724 2957312 31322865 1ddf2f1 ./vmlinux before 18140970 10225412 2957312 31323694 1ddf62e ./vmlinux after (+829) Static text symbols: Before: 40321 After: 40302 (-19) Cc: Christopher Li <sparse@chrisli.org> Cc: linux-sparse@vger.kernel.org Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com> Tested-by: Kees Cook <keescook@chromium.org> Signed-off-by: Nadav Amit <namit@vmware.com> --- arch/x86/kernel/macros.S | 2 ++ include/linux/compiler.h | 56 ++++++++++++++++++++++++++++++---------- 2 files changed, 45 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S index cfc1c7d1a6eb..cee28c3246dc 100644 --- a/arch/x86/kernel/macros.S +++ b/arch/x86/kernel/macros.S @@ -5,3 +5,5 @@ * commonly used. The macros are precompiled into assmebly file which is later * assembled together with each compiled file. */ + +#include <linux/compiler.h> diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 681d866efb1e..1921545c6351 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -99,22 +99,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, * unique, to convince GCC not to merge duplicate inline asm statements. */ #define annotate_reachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.reachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("ANNOTATE_REACHABLE counter=%c0" \ + : : "i" (__COUNTER__)); \ }) #define annotate_unreachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("ANNOTATE_UNREACHABLE counter=%c0" \ + : : "i" (__COUNTER__)); \ }) -#define ASM_UNREACHABLE \ - "999:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long 999b - .\n\t" \ - ".popsection\n\t" #else #define annotate_reachable() #define annotate_unreachable() @@ -299,6 +290,45 @@ static inline void *offset_to_ptr(const int *off) return (void *)((unsigned long)off + *off); } +#else /* __ASSEMBLY__ */ + +#ifdef __KERNEL__ +#ifndef LINKER_SCRIPT + +#ifdef CONFIG_STACK_VALIDATION +.macro ANNOTATE_UNREACHABLE counter:req +\counter: + .pushsection .discard.unreachable + .long \counter\()b -. + .popsection +.endm + +.macro ANNOTATE_REACHABLE counter:req +\counter: + .pushsection .discard.reachable + .long \counter\()b -. + .popsection +.endm + +.macro ASM_UNREACHABLE +999: + .pushsection .discard.unreachable + .long 999b - . + .popsection +.endm +#else /* CONFIG_STACK_VALIDATION */ +.macro ANNOTATE_UNREACHABLE counter:req +.endm + +.macro ANNOTATE_REACHABLE counter:req +.endm + +.macro ASM_UNREACHABLE +.endm +#endif /* CONFIG_STACK_VALIDATION */ + +#endif /* LINKER_SCRIPT */ +#endif /* __KERNEL__ */ #endif /* __ASSEMBLY__ */ #ifndef __optimize -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Nadav Amit <namit@vmware.com> To: Ingo Molnar <mingo@redhat.com> Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Nadav Amit <namit@vmware.com>, Christopher Li <sparse@chrisli.org>, linux-sparse@vger.kernel.org Subject: [PATCH v9 03/10] x86: objtool: use asm macro for better compiler decisions Date: Wed, 3 Oct 2018 14:30:53 -0700 [thread overview] Message-ID: <20181003213100.189959-4-namit@vmware.com> (raw) In-Reply-To: <20181003213100.189959-1-namit@vmware.com> GCC considers the number of statements in inlined assembly blocks, according to new-lines and semicolons, as an indication to the cost of the block in time and space. This data is distorted by the kernel code, which puts information in alternative sections. As a result, the compiler may perform incorrect inlining and branch optimizations. In the case of objtool, this distortion is extreme, since anyhow the annotations of objtool are discarded during linkage. The solution is to set an assembly macro and call it from the inline assembly block. As a result GCC considers the inline assembly block as a single instruction. This patch slightly increases the kernel size. text data bss dec hex filename 18140829 10224724 2957312 31322865 1ddf2f1 ./vmlinux before 18140970 10225412 2957312 31323694 1ddf62e ./vmlinux after (+829) Static text symbols: Before: 40321 After: 40302 (-19) Cc: Christopher Li <sparse@chrisli.org> Cc: linux-sparse@vger.kernel.org Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com> Tested-by: Kees Cook <keescook@chromium.org> Signed-off-by: Nadav Amit <namit@vmware.com> --- arch/x86/kernel/macros.S | 2 ++ include/linux/compiler.h | 56 ++++++++++++++++++++++++++++++---------- 2 files changed, 45 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S index cfc1c7d1a6eb..cee28c3246dc 100644 --- a/arch/x86/kernel/macros.S +++ b/arch/x86/kernel/macros.S @@ -5,3 +5,5 @@ * commonly used. The macros are precompiled into assmebly file which is later * assembled together with each compiled file. */ + +#include <linux/compiler.h> diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 681d866efb1e..1921545c6351 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -99,22 +99,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, * unique, to convince GCC not to merge duplicate inline asm statements. */ #define annotate_reachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.reachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("ANNOTATE_REACHABLE counter=%c0" \ + : : "i" (__COUNTER__)); \ }) #define annotate_unreachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("ANNOTATE_UNREACHABLE counter=%c0" \ + : : "i" (__COUNTER__)); \ }) -#define ASM_UNREACHABLE \ - "999:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long 999b - .\n\t" \ - ".popsection\n\t" #else #define annotate_reachable() #define annotate_unreachable() @@ -299,6 +290,45 @@ static inline void *offset_to_ptr(const int *off) return (void *)((unsigned long)off + *off); } +#else /* __ASSEMBLY__ */ + +#ifdef __KERNEL__ +#ifndef LINKER_SCRIPT + +#ifdef CONFIG_STACK_VALIDATION +.macro ANNOTATE_UNREACHABLE counter:req +\counter: + .pushsection .discard.unreachable + .long \counter\()b -. + .popsection +.endm + +.macro ANNOTATE_REACHABLE counter:req +\counter: + .pushsection .discard.reachable + .long \counter\()b -. + .popsection +.endm + +.macro ASM_UNREACHABLE +999: + .pushsection .discard.unreachable + .long 999b - . + .popsection +.endm +#else /* CONFIG_STACK_VALIDATION */ +.macro ANNOTATE_UNREACHABLE counter:req +.endm + +.macro ANNOTATE_REACHABLE counter:req +.endm + +.macro ASM_UNREACHABLE +.endm +#endif /* CONFIG_STACK_VALIDATION */ + +#endif /* LINKER_SCRIPT */ +#endif /* __KERNEL__ */ #endif /* __ASSEMBLY__ */ #ifndef __optimize -- 2.17.1
next prev parent reply other threads:[~2018-10-03 21:32 UTC|newest] Thread overview: 186+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-03 21:30 [PATCH v9 00/10] x86: macrofying inline asm Nadav Amit 2018-10-03 21:30 ` Nadav Amit 2018-10-03 21:30 ` [PATCH v9 01/10] xtensa: defining LINKER_SCRIPT for the linker script Nadav Amit 2018-10-04 10:00 ` [tip:x86/build] kbuild/arch/xtensa: Define " tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm Nadav Amit 2018-10-03 21:30 ` Nadav Amit 2018-10-04 10:01 ` [tip:x86/build] kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs tip-bot for Nadav Amit 2018-11-06 18:57 ` [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm Logan Gunthorpe 2018-11-06 19:18 ` Nadav Amit 2018-11-06 20:01 ` Logan Gunthorpe 2018-11-07 18:01 ` Nadav Amit 2018-11-07 18:53 ` Logan Gunthorpe 2018-11-07 18:56 ` Nadav Amit 2018-11-07 21:43 ` Logan Gunthorpe 2018-11-07 21:50 ` hpa 2018-11-08 6:18 ` Nadav Amit 2018-11-08 17:14 ` Logan Gunthorpe 2018-11-08 19:54 ` Nadav Amit 2018-11-08 20:00 ` Logan Gunthorpe 2018-11-08 20:18 ` Nadav Amit 2018-11-10 22:04 ` Nadav Amit 2018-11-13 4:56 ` Logan Gunthorpe 2018-10-03 21:30 ` Nadav Amit [this message] 2018-10-03 21:30 ` [PATCH v9 03/10] x86: objtool: use asm macro for better compiler decisions Nadav Amit 2018-10-04 10:02 ` [tip:x86/build] x86/objtool: Use asm macros to work around GCC inlining bugs tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 04/10] x86: refcount: prevent gcc distortions Nadav Amit 2018-10-04 7:57 ` Ingo Molnar 2018-10-04 8:33 ` Ingo Molnar 2018-10-04 8:40 ` hpa 2018-10-04 8:56 ` Ingo Molnar 2018-10-04 8:56 ` Nadav Amit 2018-10-04 9:02 ` hpa 2018-10-04 9:16 ` Ingo Molnar 2018-10-04 19:33 ` H. Peter Anvin 2018-10-04 20:05 ` Nadav Amit 2018-10-04 20:08 ` H. Peter Anvin 2018-10-04 20:29 ` Andy Lutomirski 2018-10-04 23:11 ` H. Peter Anvin 2018-10-06 1:40 ` Rasmus Villemoes 2018-10-04 9:12 ` Ingo Molnar 2018-10-04 9:17 ` hpa 2018-10-04 9:30 ` Nadav Amit 2018-10-04 9:45 ` Ingo Molnar 2018-10-04 10:23 ` Nadav Amit 2018-10-05 9:31 ` Ingo Molnar 2018-10-05 11:20 ` Borislav Petkov 2018-10-05 12:52 ` Ingo Molnar 2018-10-05 20:27 ` [PATCH 0/3] Macrofying inline asm rebased Nadav Amit 2018-10-05 20:27 ` [PATCH 1/3] x86/extable: Macrofy inline assembly code to work around GCC inlining bugs Nadav Amit 2018-10-06 14:42 ` [tip:x86/build] " tip-bot for Nadav Amit 2018-10-05 20:27 ` [PATCH 2/3] x86/cpufeature: " Nadav Amit 2018-10-06 14:43 ` [tip:x86/build] " tip-bot for Nadav Amit 2018-10-05 20:27 ` [PATCH 3/3] x86/jump-labels: " Nadav Amit 2018-10-06 14:44 ` [tip:x86/build] " tip-bot for Nadav Amit 2018-10-08 2:17 ` [PATCH v9 04/10] x86: refcount: prevent gcc distortions Nadav Amit 2018-10-04 8:40 ` Nadav Amit 2018-10-04 9:01 ` Ingo Molnar 2018-10-04 10:02 ` [tip:x86/build] x86/refcount: Work around GCC inlining bug tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 05/10] x86: alternatives: macrofy locks for better inlining Nadav Amit 2018-10-04 10:03 ` [tip:x86/build] x86/alternatives: Macrofy lock prefixes to work around GCC inlining bugs tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 06/10] x86: bug: prevent gcc distortions Nadav Amit 2018-10-04 10:03 ` [tip:x86/build] x86/bug: Macrofy the BUG table section handling, to work around GCC inlining bugs tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 07/10] x86: prevent inline distortion by paravirt ops Nadav Amit 2018-10-04 10:04 ` [tip:x86/build] x86/paravirt: Work around GCC inlining bugs when compiling " tip-bot for Nadav Amit 2018-10-03 21:30 ` [PATCH v9 08/10] x86: extable: use macros instead of inline assembly Nadav Amit 2018-10-03 21:30 ` [PATCH v9 09/10] x86: cpufeature: " Nadav Amit 2018-10-03 21:31 ` [PATCH v9 10/10] x86: jump-labels: " Nadav Amit 2018-10-07 9:18 ` PROPOSAL: Extend inline asm syntax with size spec Borislav Petkov 2018-10-07 9:18 ` Borislav Petkov 2018-10-07 9:18 ` Borislav Petkov 2018-10-07 13:22 ` Segher Boessenkool 2018-10-07 14:13 ` Borislav Petkov 2018-10-07 14:13 ` Borislav Petkov 2018-10-07 15:14 ` Segher Boessenkool 2018-10-07 15:14 ` Segher Boessenkool 2018-10-08 5:58 ` Ingo Molnar 2018-10-08 5:58 ` Ingo Molnar 2018-10-08 5:58 ` Ingo Molnar 2018-10-08 7:53 ` Segher Boessenkool 2018-10-08 7:53 ` Segher Boessenkool 2018-10-07 14:13 ` Borislav Petkov 2018-10-07 15:53 ` Michael Matz 2018-10-07 15:53 ` Michael Matz 2018-10-08 6:13 ` Ingo Molnar 2018-10-08 6:13 ` Ingo Molnar 2018-10-08 8:18 ` Segher Boessenkool 2018-10-08 8:18 ` Segher Boessenkool 2018-10-08 7:31 ` Segher Boessenkool 2018-10-08 7:31 ` Segher Boessenkool 2018-10-08 9:07 ` Richard Biener 2018-10-08 9:07 ` Richard Biener 2018-10-08 10:02 ` Segher Boessenkool 2018-10-08 10:02 ` Segher Boessenkool 2018-10-09 14:53 ` Segher Boessenkool 2018-10-09 14:53 ` Segher Boessenkool 2018-10-10 6:35 ` Ingo Molnar 2018-10-10 6:35 ` Ingo Molnar 2018-10-10 7:12 ` Richard Biener 2018-10-10 7:12 ` Richard Biener 2018-10-10 7:22 ` Ingo Molnar 2018-10-10 7:22 ` Ingo Molnar 2018-10-10 8:03 ` Segher Boessenkool 2018-10-10 8:03 ` Segher Boessenkool 2018-10-10 8:19 ` Borislav Petkov 2018-10-10 8:19 ` Borislav Petkov 2018-10-10 8:19 ` Borislav Petkov 2018-10-10 8:35 ` Richard Biener 2018-10-10 8:35 ` Richard Biener 2018-10-10 18:54 ` Segher Boessenkool 2018-10-10 18:54 ` Segher Boessenkool 2018-10-10 19:14 ` Borislav Petkov 2018-10-10 19:14 ` Borislav Petkov 2018-10-10 19:14 ` Borislav Petkov 2018-10-13 19:33 ` Borislav Petkov 2018-10-13 19:33 ` Borislav Petkov 2018-10-13 21:14 ` Alexander Monakov 2018-10-13 21:14 ` Alexander Monakov 2018-10-13 21:30 ` Borislav Petkov 2018-10-13 21:30 ` Borislav Petkov 2018-10-25 10:24 ` Borislav Petkov 2018-10-25 10:24 ` Borislav Petkov 2018-10-25 10:24 ` Borislav Petkov 2018-10-31 12:55 ` Peter Zijlstra 2018-10-31 12:55 ` Peter Zijlstra 2018-10-31 13:11 ` Peter Zijlstra 2018-10-31 13:11 ` Peter Zijlstra 2018-10-31 16:31 ` Segher Boessenkool 2018-10-31 16:31 ` Segher Boessenkool 2018-11-01 5:20 ` Joe Perches 2018-11-01 5:20 ` Joe Perches 2018-11-01 9:01 ` Peter Zijlstra 2018-11-01 9:01 ` Peter Zijlstra 2018-11-01 9:20 ` Joe Perches 2018-11-01 9:20 ` Joe Perches 2018-11-01 11:15 ` Peter Zijlstra 2018-11-01 11:15 ` Peter Zijlstra 2018-11-01 9:20 ` Joe Perches 2018-11-01 5:20 ` Joe Perches 2018-12-27 4:47 ` Masahiro Yamada 2018-12-27 4:47 ` Masahiro Yamada 2018-10-10 10:29 ` Richard Biener 2018-10-10 10:29 ` Richard Biener 2018-10-10 7:53 ` Segher Boessenkool 2018-10-10 7:53 ` Segher Boessenkool 2018-10-10 16:31 ` Nadav Amit 2018-10-10 16:31 ` Nadav Amit 2018-10-10 19:21 ` Segher Boessenkool 2018-10-10 19:21 ` Segher Boessenkool 2018-10-11 7:04 ` Richard Biener 2018-10-11 7:04 ` Richard Biener 2018-11-29 11:46 ` Masahiro Yamada 2018-11-29 11:46 ` Masahiro Yamada 2018-11-29 11:46 ` Masahiro Yamada 2018-11-29 12:25 ` Segher Boessenkool 2018-11-29 12:25 ` Segher Boessenkool 2018-11-30 9:06 ` Boris Petkov via Virtualization 2018-11-30 9:06 ` Boris Petkov 2018-11-30 9:06 ` Boris Petkov 2018-11-30 13:16 ` Segher Boessenkool 2018-11-30 13:16 ` Segher Boessenkool 2018-12-10 8:16 ` Masahiro Yamada 2018-12-10 8:16 ` Masahiro Yamada 2018-12-10 8:16 ` Masahiro Yamada 2018-11-29 13:07 ` Borislav Petkov 2018-11-29 13:07 ` Borislav Petkov via Virtualization 2018-11-29 13:09 ` Richard Biener 2018-11-29 13:09 ` Richard Biener 2018-11-29 13:16 ` Borislav Petkov via Virtualization 2018-11-29 13:16 ` Borislav Petkov 2018-11-29 13:16 ` Borislav Petkov 2018-11-29 13:24 ` Richard Biener 2018-11-29 13:24 ` Richard Biener 2018-10-08 16:24 ` David Laight 2018-10-08 16:24 ` David Laight 2018-10-08 16:24 ` David Laight 2018-10-07 16:09 ` Nadav Amit 2018-10-07 16:09 ` Nadav Amit 2018-10-07 16:13 ` [RESEND] " Nadav Amit 2018-10-07 16:46 ` Richard Biener 2018-10-07 16:46 ` Richard Biener 2018-10-07 19:06 ` Nadav Amit 2018-10-07 19:06 ` Nadav Amit 2018-10-07 19:52 ` Jeff Law 2018-10-07 19:52 ` Jeff Law 2018-10-08 7:46 ` Richard Biener 2018-10-08 7:46 ` Richard Biener
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181003213100.189959-4-namit@vmware.com \ --to=namit@vmware.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-sparse@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=sparse@chrisli.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.