linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nadav Amit <namit@vmware.com>
To: Ingo Molnar <mingo@redhat.com>
Cc: <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
	Nadav Amit <namit@vmware.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Josh Poimboeuf <jpoimboe@redhat.com>
Subject: [PATCH v9 05/10] x86: alternatives: macrofy locks for better inlining
Date: Wed, 3 Oct 2018 14:30:55 -0700	[thread overview]
Message-ID: <20181003213100.189959-6-namit@vmware.com> (raw)
In-Reply-To: <20181003213100.189959-1-namit@vmware.com>

GCC considers the number of statements in inlined assembly blocks,
according to new-lines and semicolons, as an indication to the cost of
the block in time and space. This data is distorted by the kernel code,
which puts information in alternative sections. As a result, the
compiler may perform incorrect inlining and branch optimizations.

The solution is to set an assembly macro and call it from the inlined
assembly block. As a result GCC considers the inline assembly block as
a single instruction.

This patch handles the LOCK prefix, allowing more aggresive inlining.

   text	   data	    bss	    dec	    hex	filename
18140140 10225284 2957312 31322736 1ddf270 ./vmlinux before
18146889 10225380 2957312 31329581 1de0d2d ./vmlinux after (+6845)

Static text symbols:
Before:	40286
After:	40218	(-68)

Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
---
 arch/x86/include/asm/alternative-asm.h | 20 ++++++++++++++------
 arch/x86/include/asm/alternative.h     | 11 ++---------
 arch/x86/kernel/macros.S               |  1 +
 3 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/alternative-asm.h b/arch/x86/include/asm/alternative-asm.h
index 31b627b43a8e..8e4ea39e55d0 100644
--- a/arch/x86/include/asm/alternative-asm.h
+++ b/arch/x86/include/asm/alternative-asm.h
@@ -7,16 +7,24 @@
 #include <asm/asm.h>
 
 #ifdef CONFIG_SMP
-	.macro LOCK_PREFIX
-672:	lock
+.macro LOCK_PREFIX_HERE
 	.pushsection .smp_locks,"a"
 	.balign 4
-	.long 672b - .
+	.long 671f - .		# offset
 	.popsection
-	.endm
+671:
+.endm
+
+.macro LOCK_PREFIX insn:vararg
+	LOCK_PREFIX_HERE
+	lock \insn
+.endm
 #else
-	.macro LOCK_PREFIX
-	.endm
+.macro LOCK_PREFIX_HERE
+.endm
+
+.macro LOCK_PREFIX insn:vararg
+.endm
 #endif
 
 /*
diff --git a/arch/x86/include/asm/alternative.h b/arch/x86/include/asm/alternative.h
index 4cd6a3b71824..d7faa16622d8 100644
--- a/arch/x86/include/asm/alternative.h
+++ b/arch/x86/include/asm/alternative.h
@@ -31,15 +31,8 @@
  */
 
 #ifdef CONFIG_SMP
-#define LOCK_PREFIX_HERE \
-		".pushsection .smp_locks,\"a\"\n"	\
-		".balign 4\n"				\
-		".long 671f - .\n" /* offset */		\
-		".popsection\n"				\
-		"671:"
-
-#define LOCK_PREFIX LOCK_PREFIX_HERE "\n\tlock; "
-
+#define LOCK_PREFIX_HERE "LOCK_PREFIX_HERE\n\t"
+#define LOCK_PREFIX "LOCK_PREFIX "
 #else /* ! CONFIG_SMP */
 #define LOCK_PREFIX_HERE ""
 #define LOCK_PREFIX ""
diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S
index f1fe1d570365..852487a9fc56 100644
--- a/arch/x86/kernel/macros.S
+++ b/arch/x86/kernel/macros.S
@@ -8,3 +8,4 @@
 
 #include <linux/compiler.h>
 #include <asm/refcount.h>
+#include <asm/alternative-asm.h>
-- 
2.17.1


  parent reply	other threads:[~2018-10-03 21:33 UTC|newest]

Thread overview: 116+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-03 21:30 [PATCH v9 00/10] x86: macrofying inline asm Nadav Amit
2018-10-03 21:30 ` [PATCH v9 01/10] xtensa: defining LINKER_SCRIPT for the linker script Nadav Amit
2018-10-04 10:00   ` [tip:x86/build] kbuild/arch/xtensa: Define " tip-bot for Nadav Amit
2018-10-03 21:30 ` [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm Nadav Amit
2018-10-04 10:01   ` [tip:x86/build] kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs tip-bot for Nadav Amit
2018-11-06 18:57   ` [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm Logan Gunthorpe
2018-11-06 19:18     ` Nadav Amit
2018-11-06 20:01       ` Logan Gunthorpe
2018-11-07 18:01         ` Nadav Amit
2018-11-07 18:53           ` Logan Gunthorpe
2018-11-07 18:56             ` Nadav Amit
2018-11-07 21:43               ` Logan Gunthorpe
2018-11-07 21:50                 ` hpa
2018-11-08  6:18                   ` Nadav Amit
2018-11-08 17:14                     ` Logan Gunthorpe
2018-11-08 19:54                       ` Nadav Amit
2018-11-08 20:00                         ` Logan Gunthorpe
2018-11-08 20:18                           ` Nadav Amit
2018-11-10 22:04                             ` Nadav Amit
2018-11-13  4:56                               ` Logan Gunthorpe
2018-10-03 21:30 ` [PATCH v9 03/10] x86: objtool: use asm macro for better compiler decisions Nadav Amit
2018-10-04 10:02   ` [tip:x86/build] x86/objtool: Use asm macros to work around GCC inlining bugs tip-bot for Nadav Amit
2018-10-03 21:30 ` [PATCH v9 04/10] x86: refcount: prevent gcc distortions Nadav Amit
2018-10-04  7:57   ` Ingo Molnar
2018-10-04  8:33     ` Ingo Molnar
2018-10-04  8:40       ` hpa
2018-10-04  8:56         ` Ingo Molnar
2018-10-04  8:56         ` Nadav Amit
2018-10-04  9:02           ` hpa
2018-10-04  9:16             ` Ingo Molnar
2018-10-04 19:33               ` H. Peter Anvin
2018-10-04 20:05                 ` Nadav Amit
2018-10-04 20:08                   ` H. Peter Anvin
2018-10-04 20:29                 ` Andy Lutomirski
2018-10-04 23:11                   ` H. Peter Anvin
2018-10-06  1:40                 ` Rasmus Villemoes
2018-10-04  9:12           ` Ingo Molnar
2018-10-04  9:17             ` hpa
2018-10-04  9:30             ` Nadav Amit
2018-10-04  9:45               ` Ingo Molnar
2018-10-04 10:23                 ` Nadav Amit
2018-10-05  9:31                   ` Ingo Molnar
2018-10-05 11:20                     ` Borislav Petkov
2018-10-05 12:52                       ` Ingo Molnar
2018-10-05 20:27                     ` [PATCH 0/3] Macrofying inline asm rebased Nadav Amit
2018-10-05 20:27                       ` [PATCH 1/3] x86/extable: Macrofy inline assembly code to work around GCC inlining bugs Nadav Amit
2018-10-06 14:42                         ` [tip:x86/build] " tip-bot for Nadav Amit
2018-10-05 20:27                       ` [PATCH 2/3] x86/cpufeature: " Nadav Amit
2018-10-06 14:43                         ` [tip:x86/build] " tip-bot for Nadav Amit
2018-10-05 20:27                       ` [PATCH 3/3] x86/jump-labels: " Nadav Amit
2018-10-06 14:44                         ` [tip:x86/build] " tip-bot for Nadav Amit
2018-10-08  2:17                     ` [PATCH v9 04/10] x86: refcount: prevent gcc distortions Nadav Amit
2018-10-04  8:40     ` Nadav Amit
2018-10-04  9:01       ` Ingo Molnar
2018-10-04 10:02   ` [tip:x86/build] x86/refcount: Work around GCC inlining bug tip-bot for Nadav Amit
2018-10-03 21:30 ` Nadav Amit [this message]
2018-10-04 10:03   ` [tip:x86/build] x86/alternatives: Macrofy lock prefixes to work around GCC inlining bugs tip-bot for Nadav Amit
2018-10-03 21:30 ` [PATCH v9 06/10] x86: bug: prevent gcc distortions Nadav Amit
2018-10-04 10:03   ` [tip:x86/build] x86/bug: Macrofy the BUG table section handling, to work around GCC inlining bugs tip-bot for Nadav Amit
2018-10-03 21:30 ` [PATCH v9 07/10] x86: prevent inline distortion by paravirt ops Nadav Amit
2018-10-04 10:04   ` [tip:x86/build] x86/paravirt: Work around GCC inlining bugs when compiling " tip-bot for Nadav Amit
2018-10-03 21:30 ` [PATCH v9 08/10] x86: extable: use macros instead of inline assembly Nadav Amit
2018-10-03 21:30 ` [PATCH v9 09/10] x86: cpufeature: " Nadav Amit
2018-10-03 21:31 ` [PATCH v9 10/10] x86: jump-labels: " Nadav Amit
2018-10-07  9:18 ` PROPOSAL: Extend inline asm syntax with size spec Borislav Petkov
     [not found]   ` <20181007132228.GJ29268@gate.crashing.org>
2018-10-07 14:13     ` Borislav Petkov
2018-10-07 15:14       ` Segher Boessenkool
2018-10-08  5:58         ` Ingo Molnar
2018-10-08  7:53           ` Segher Boessenkool
2018-10-07 15:53     ` Michael Matz
2018-10-08  6:13       ` Ingo Molnar
2018-10-08  8:18         ` Segher Boessenkool
2018-10-08  7:31       ` Segher Boessenkool
2018-10-08  9:07         ` Richard Biener
2018-10-08 10:02           ` Segher Boessenkool
2018-10-09 14:53           ` Segher Boessenkool
2018-10-10  6:35             ` Ingo Molnar
2018-10-10  7:12             ` Richard Biener
2018-10-10  7:22               ` Ingo Molnar
2018-10-10  8:03                 ` Segher Boessenkool
2018-10-10  8:19                   ` Borislav Petkov
2018-10-10  8:35                     ` Richard Biener
2018-10-10 18:54                     ` Segher Boessenkool
2018-10-10 19:14                       ` Borislav Petkov
2018-10-13 19:33                         ` Borislav Petkov
2018-10-13 21:14                           ` Alexander Monakov
2018-10-13 21:30                             ` Borislav Petkov
2018-10-25 10:24                           ` Borislav Petkov
2018-10-31 12:55                           ` Peter Zijlstra
2018-10-31 13:11                             ` Peter Zijlstra
2018-10-31 16:31                             ` Segher Boessenkool
2018-11-01  5:20                             ` Joe Perches
2018-11-01  9:01                               ` Peter Zijlstra
2018-11-01  9:20                                 ` Joe Perches
2018-11-01 11:15                                   ` Peter Zijlstra
2018-12-27  4:47                             ` Masahiro Yamada
2018-10-10 10:29                   ` Richard Biener
2018-10-10  7:53               ` Segher Boessenkool
2018-10-10 16:31             ` Nadav Amit
2018-10-10 19:21               ` Segher Boessenkool
2018-10-11  7:04               ` Richard Biener
2018-11-29 11:46             ` Masahiro Yamada
2018-11-29 12:25               ` Segher Boessenkool
2018-11-30  9:06                 ` Boris Petkov
2018-11-30 13:16                   ` Segher Boessenkool
2018-12-10  8:16                     ` Masahiro Yamada
2018-11-29 13:07               ` Borislav Petkov
2018-11-29 13:09                 ` Richard Biener
2018-11-29 13:16                   ` Borislav Petkov
2018-11-29 13:24                     ` Richard Biener
2018-10-08 16:24       ` David Laight
2018-10-07 16:09   ` Nadav Amit
2018-10-07 16:46     ` Richard Biener
2018-10-07 19:06       ` Nadav Amit
2018-10-07 19:52         ` Jeff Law
2018-10-08  7:46         ` Richard Biener

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181003213100.189959-6-namit@vmware.com \
    --to=namit@vmware.com \
    --cc=hpa@zytor.com \
    --cc=jpoimboe@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).