linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: x86@kernel.org, jpoimboe@redhat.com, andrew.cooper3@citrix.com
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org,
	alexei.starovoitov@gmail.com, ndesaulniers@google.com,
	bpf@vger.kernel.org
Subject: [PATCH v3 12/16] x86/alternative: Try inline spectre_v2=retpoline,amd
Date: Tue, 26 Oct 2021 14:01:44 +0200	[thread overview]
Message-ID: <20211026120310.359986601@infradead.org> (raw)
In-Reply-To: <20211026120132.613201817@infradead.org>

Try and replace retpoline thunk calls with:

  LFENCE
  CALL    *%\reg

for spectre_v2=retpoline,amd.

Specifically, the sequence above is 5 bytes for the low 8 registers,
but 6 bytes for the high 8 registers. This means that unless the
compilers prefix stuff the call with higher registers this replacement
will fail.

Luckily GCC strongly favours RAX for the indirect calls and most (95%+
for defconfig-x86_64) will be converted. OTOH clang strongly favours
R11 and almost nothing gets converted.

Note: it will also generate a correct replacement for the Jcc.d32
case, except unless the compilers start to prefix stuff that, it'll
never fit. Specifically:

  Jncc.d8 1f
  LFENCE
  JMP     *%\reg
1:

is 7-8 bytes long, where the original instruction in unpadded form is
only 6 bytes.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/kernel/alternative.c |   16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -389,6 +389,7 @@ static int emit_indirect(int op, int reg
  *
  *   CALL *%\reg
  *
+ * It also tries to inline spectre_v2=retpoline,amd when size permits.
  */
 static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
 {
@@ -405,7 +406,8 @@ static int patch_retpoline(void *addr, s
 	/* If anyone ever does: CALL/JMP *%rsp, we're in deep trouble. */
 	BUG_ON(reg == 4);
 
-	if (cpu_feature_enabled(X86_FEATURE_RETPOLINE))
+	if (cpu_feature_enabled(X86_FEATURE_RETPOLINE) &&
+	    !cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD))
 		return -1;
 
 	op = insn->opcode.bytes[0];
@@ -418,8 +420,9 @@ static int patch_retpoline(void *addr, s
 	 * into:
 	 *
 	 *   Jncc.d8 1f
+	 *   [ LFENCE ]
 	 *   JMP *%\reg
-	 *   NOP
+	 *   [ NOP ]
 	 * 1:
 	 */
 	if (op == 0x0f && (insn->opcode.bytes[1] & 0xf0) == 0x80) {
@@ -432,6 +435,15 @@ static int patch_retpoline(void *addr, s
 		op = JMP32_INSN_OPCODE;
 	}
 
+	/*
+	 * For RETPOLINE_AMD: prepend the indirect CALL/JMP with an LFENCE.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD)) {
+		bytes[i++] = 0x0f;
+		bytes[i++] = 0xae;
+		bytes[i++] = 0xe8; /* LFENCE */
+	}
+
 	ret = emit_indirect(op, reg, bytes + i);
 	if (ret < 0)
 		return ret;



  parent reply	other threads:[~2021-10-26 12:05 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-26 12:01 [PATCH v3 00/16] x86: Rewrite the retpoline rewrite logic Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 01/16] objtool: Classify symbols Peter Zijlstra
2021-10-28  3:58   ` Josh Poimboeuf
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 02/16] objtool: Explicitly avoid self modifying code in .altinstr_replacement Peter Zijlstra
2021-10-28  4:01   ` Josh Poimboeuf
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 03/16] objtool: Shrink struct instruction Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 04/16] objtool,x86: Replace alternatives with .retpoline_sites Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 05/16] x86/retpoline: Remove unused replacement symbols Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 06/16] x86/asm: Fix register order Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 07/16] x86/asm: Fixup odd GEN-for-each-reg.h usage Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 08/16] x86/retpoline: Move the retpoline thunk declarations to nospec-branch.h Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 09/16] x86/retpoline: Create a retpoline thunk array Peter Zijlstra
2021-10-29  8:03   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 10/16] x86/alternative: Implement .retpoline_sites support Peter Zijlstra
2021-10-27 17:38   ` Borislav Petkov
2021-10-27 19:10     ` Peter Zijlstra
2021-10-29  8:02   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 11/16] x86/alternative: Handle Jcc __x86_indirect_thunk_\reg Peter Zijlstra
2021-10-28  9:29   ` Borislav Petkov
2021-10-29  8:02   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` Peter Zijlstra [this message]
2021-10-29  8:02   ` [tip: objtool/core] x86/alternative: Try inline spectre_v2=retpoline,amd tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 13/16] x86/alternative: Add debug prints to apply_retpolines() Peter Zijlstra
2021-10-29  8:02   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 14/16] x86,bugs: Unconditionally allow spectre_v2=retpoline,amd Peter Zijlstra
2021-10-29  8:02   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 15/16] bpf, x86: Simplify computing label offsets Peter Zijlstra
2021-10-29  8:02   ` [tip: objtool/core] bpf,x86: " tip-bot2 for Peter Zijlstra
2021-10-26 12:01 ` [PATCH v3 16/16] bpf,x86: Respect X86_FEATURE_RETPOLINE* Peter Zijlstra
2021-10-29  8:02   ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-10-26 18:26 ` [PATCH v3 00/16] x86: Rewrite the retpoline rewrite logic Alexei Starovoitov
2021-10-26 18:45   ` Peter Zijlstra
2021-10-26 20:00     ` Alexei Starovoitov
2021-10-26 21:05       ` Peter Zijlstra
2021-10-26 21:05         ` Alexei Starovoitov
2021-10-27  9:00           ` Peter Zijlstra
2021-10-27 17:32             ` Alexei Starovoitov
2021-10-28  5:17 ` Josh Poimboeuf
2021-10-28 17:43 ` Borislav Petkov
2021-10-29 12:43 ` Miroslav Benes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211026120310.359986601@infradead.org \
    --to=peterz@infradead.org \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=bpf@vger.kernel.org \
    --cc=jpoimboe@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=x86@kernel.org \
    --subject='Re: [PATCH v3 12/16] x86/alternative: Try inline spectre_v2=retpoline,amd' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).