From: Nadav Amit <namit@vmware.com> To: <linux-kernel@vger.kernel.org> Cc: <nadav.amit@gmail.com>, Nadav Amit <namit@vmware.com>, Christopher Li <sparse@chrisli.org>, <linux-sparse@vger.kernel.org> Subject: [RFC 1/8] x86: objtool: use asm macro for better compiler decisions Date: Tue, 15 May 2018 07:11:17 -0700 [thread overview] Message-ID: <20180515141124.84254-11-namit@vmware.com> (raw) In-Reply-To: <20180515141124.84254-1-namit@vmware.com> GCC considers the number of statements in inlined assembly blocks, according to new-lines and semicolons, as an indication to the cost of the block in time and space. This data is distorted by the kernel code, which puts information in alternative sections. As a result, the compiler may perform incorrect inlining and branch optimizations. In the case of objtool, this distortion is extreme, since anyhow the annotations of objtool are discarded during linkage. The solution is to set an assembly macro and call it from the inlinedv assembly block. As a result GCC considers the inline assembly block as a single instruction. This patch slightly increases the kernel size. text data bss dec hex filename 18126699 10066728 2936832 31130259 1db0293 ./vmlinux before 18126824 10067268 2936832 31130924 1db052c ./vmlinux after (+665) But allows more aggressive inlining. Static text symbols: Before: 40033 After: 40015 (-18) Cc: Christopher Li <sparse@chrisli.org> Cc: linux-sparse@vger.kernel.org Signed-off-by: Nadav Amit <namit@vmware.com> --- include/linux/compiler.h | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/include/linux/compiler.h b/include/linux/compiler.h index ab4711c63601..369753000541 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -98,17 +98,30 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, * The __COUNTER__ based labels are a hack to make each instance of the macros * unique, to convince GCC not to merge duplicate inline asm statements. */ + +asm ("\n" + ".macro __annotate_reachable counter:req\n" + "\\counter:\n\t" + ".pushsection .discard.reachable\n\t" + ".long \\counter\\()b -.\n\t" + ".popsection\n" + ".endm\n"); + #define annotate_reachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.reachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("__annotate_reachable %c0" : : "i" (__COUNTER__)); \ }) + +asm ("\n" + ".macro __annotate_unreachable counter:req\n" + "\\counter:\n\t" + ".pushsection .discard.unreachable\n\t" + ".long \\counter\\()b -.\n\t" + ".popsection\n" + ".endm"); + #define annotate_unreachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("__annotate_unreachable %c0" : : \ + "i" (__COUNTER__)); \ }) #define ASM_UNREACHABLE \ "999:\n\t" \ -- 2.17.0
WARNING: multiple messages have this Message-ID (diff)
From: Nadav Amit <namit@vmware.com> To: linux-kernel@vger.kernel.org Cc: nadav.amit@gmail.com, Nadav Amit <namit@vmware.com>, Christopher Li <sparse@chrisli.org>, linux-sparse@vger.kernel.org Subject: [RFC 1/8] x86: objtool: use asm macro for better compiler decisions Date: Tue, 15 May 2018 07:11:17 -0700 [thread overview] Message-ID: <20180515141124.84254-11-namit@vmware.com> (raw) In-Reply-To: <20180515141124.84254-1-namit@vmware.com> GCC considers the number of statements in inlined assembly blocks, according to new-lines and semicolons, as an indication to the cost of the block in time and space. This data is distorted by the kernel code, which puts information in alternative sections. As a result, the compiler may perform incorrect inlining and branch optimizations. In the case of objtool, this distortion is extreme, since anyhow the annotations of objtool are discarded during linkage. The solution is to set an assembly macro and call it from the inlinedv assembly block. As a result GCC considers the inline assembly block as a single instruction. This patch slightly increases the kernel size. text data bss dec hex filename 18126699 10066728 2936832 31130259 1db0293 ./vmlinux before 18126824 10067268 2936832 31130924 1db052c ./vmlinux after (+665) But allows more aggressive inlining. Static text symbols: Before: 40033 After: 40015 (-18) Cc: Christopher Li <sparse@chrisli.org> Cc: linux-sparse@vger.kernel.org Signed-off-by: Nadav Amit <namit@vmware.com> --- include/linux/compiler.h | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/include/linux/compiler.h b/include/linux/compiler.h index ab4711c63601..369753000541 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -98,17 +98,30 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, * The __COUNTER__ based labels are a hack to make each instance of the macros * unique, to convince GCC not to merge duplicate inline asm statements. */ + +asm ("\n" + ".macro __annotate_reachable counter:req\n" + "\\counter:\n\t" + ".pushsection .discard.reachable\n\t" + ".long \\counter\\()b -.\n\t" + ".popsection\n" + ".endm\n"); + #define annotate_reachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.reachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("__annotate_reachable %c0" : : "i" (__COUNTER__)); \ }) + +asm ("\n" + ".macro __annotate_unreachable counter:req\n" + "\\counter:\n\t" + ".pushsection .discard.unreachable\n\t" + ".long \\counter\\()b -.\n\t" + ".popsection\n" + ".endm"); + #define annotate_unreachable() ({ \ - asm volatile("%c0:\n\t" \ - ".pushsection .discard.unreachable\n\t" \ - ".long %c0b - .\n\t" \ - ".popsection\n\t" : : "i" (__COUNTER__)); \ + asm volatile("__annotate_unreachable %c0" : : \ + "i" (__COUNTER__)); \ }) #define ASM_UNREACHABLE \ "999:\n\t" \ -- 2.17.0
next prev parent reply other threads:[~2018-05-15 21:27 UTC|newest] Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-05-15 14:11 [RFC 0/8] Improving compiler inlining decisions Nadav Amit 2018-05-15 14:11 ` Nadav Amit 2018-05-15 14:11 ` [RFC 1/8] x86: objtool: use asm macro for better compiler decisions Nadav Amit 2018-05-15 14:11 ` Nadav Amit 2018-05-15 21:37 ` Josh Triplett 2018-05-15 21:53 ` Nadav Amit 2018-05-15 21:55 ` Josh Triplett 2018-05-15 14:11 ` [RFC 2/8] x86: bug: prevent gcc distortions Nadav Amit 2018-05-15 14:11 ` [RFC 3/8] x86: alternative: macrofy locks for better inlining Nadav Amit 2018-05-15 14:11 ` [RFC 4/8] x86: prevent inline distortion by paravirt ops Nadav Amit 2018-05-15 14:11 ` [RFC 5/8] x86: refcount: prevent gcc distortions Nadav Amit 2018-05-16 13:59 ` Kees Cook 2018-05-16 16:37 ` Nadav Amit 2018-05-15 14:11 ` [RFC 6/8] x86: removing unneeded new-lines Nadav Amit 2018-05-15 14:11 ` [RFC 7/8] ilog2: preventing compiler distortion due to big condition Nadav Amit 2018-05-15 14:11 ` [RFC 8/8] bitops: prevent compiler inline decision distortion Nadav Amit 2018-05-15 14:11 ` [RFC 0/8] Improving compiler inlining decisions Nadav Amit 2018-05-15 14:11 ` Nadav Amit 2018-05-15 14:11 ` Nadav Amit [this message] 2018-05-15 14:11 ` [RFC 1/8] x86: objtool: use asm macro for better compiler decisions Nadav Amit 2018-05-15 14:11 ` [RFC 2/8] x86: bug: prevent gcc distortions Nadav Amit 2018-05-15 14:11 ` [RFC 3/8] x86: alternative: macrofy locks for better inlining Nadav Amit 2018-05-15 14:11 ` [RFC 4/8] x86: prevent inline distortion by paravirt ops Nadav Amit 2018-05-15 14:11 ` [RFC 5/8] x86: refcount: prevent gcc distortions Nadav Amit 2018-05-16 7:14 ` Jan Beulich 2018-05-16 16:44 ` Nadav Amit 2018-05-17 7:18 ` Jan Beulich 2018-05-15 14:11 ` [RFC 6/8] x86: removing unneeded new-lines Nadav Amit 2018-05-15 14:11 ` [RFC 7/8] ilog2: preventing compiler distortion due to big condition Nadav Amit 2018-05-15 14:11 ` [RFC 8/8] bitops: prevent compiler inline decision distortion Nadav Amit 2018-05-16 14:09 ` Kees Cook 2018-05-15 22:14 ` [RFC 0/8] Improving compiler inlining decisions Nadav Amit 2018-05-16 3:48 ` Josh Poimboeuf 2018-05-16 3:48 ` Josh Poimboeuf 2018-05-16 4:30 ` Nadav Amit
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180515141124.84254-11-namit@vmware.com \ --to=namit@vmware.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-sparse@vger.kernel.org \ --cc=nadav.amit@gmail.com \ --cc=sparse@chrisli.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.