From: Peter Zijlstra <peterz@infradead.org>
To: x86@kernel.org, jpoimboe@redhat.com, andrew.cooper3@citrix.com
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org,
alexei.starovoitov@gmail.com, ndesaulniers@google.com
Subject: [PATCH v2 11/14] x86/alternative: Try inline spectre_v2=retpoline,amd
Date: Wed, 20 Oct 2021 12:44:53 +0200 [thread overview]
Message-ID: <20211020105843.162995272@infradead.org> (raw)
In-Reply-To: 20211020104442.021802560@infradead.org
Try and replace retpoline thunk calls with:
lfence
call *%\reg
for spectre_v2=retpoline,amd.
Specifically, the sequence above is 5 bytes for the low 8 registers,
but 6 bytes for the high 8 registers. This means that unless the
compilers prefix stuff the call with higher registers this replacement
will fail.
Luckily GCC strongly favours RAX for the indirect calls and most (95%+
for defconfig-x86_64) will be converted. OTOH clang strongly favours
R11 and almost nothing gets converted.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
arch/x86/kernel/alternative.c | 20 +++++++++++++++++---
1 file changed, 17 insertions(+), 3 deletions(-)
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -389,12 +389,13 @@ static int emit_indirect(int op, int reg
*
* CALL *%\reg
*
+ * It also tries to inline spectre_v2=retpoline,amd when size permits.
*/
static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
{
+ u8 cc, op = insn->opcode.bytes[0];
void (*target)(void);
int reg, ret, i = 0;
- u8 op, cc;
target = addr + insn->length + insn->immediate.value;
reg = (target - &__x86_indirect_thunk_rax) /
@@ -406,11 +407,23 @@ static int patch_retpoline(void *addr, s
/* If anyone ever does: CALL/JMP *%rsp, we're in deep trouble. */
BUG_ON(reg == 4);
+ if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_AMD)) {
+ /*
+ * Can't do nothing about the Jcc case here.
+ */
+ if (op != JMP32_INSN_OPCODE && op != CALL_INSN_OPCODE)
+ return -1;
+
+ bytes[i++] = 0x0f;
+ bytes[i++] = 0xae;
+ bytes[i++] = 0xe8; /* lfence */
+
+ goto indirect;
+ }
+
if (cpu_feature_enabled(X86_FEATURE_RETPOLINE))
return -1;
- op = insn->opcode.bytes[0];
-
/*
* Convert:
*
@@ -433,6 +446,7 @@ static int patch_retpoline(void *addr, s
op = JMP32_INSN_OPCODE;
}
+indirect:
ret = emit_indirect(op, reg, bytes + i);
if (ret < 0)
return ret;
next prev parent reply other threads:[~2021-10-20 11:04 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-20 10:44 [PATCH v2 00/14] x86: Rewrite the retpoline rewrite logic Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 01/14] objtool: Tag retpoline thunk symbols Peter Zijlstra
2021-10-20 15:17 ` Josh Poimboeuf
2021-10-26 7:55 ` Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 02/14] objtool: Explicitly avoid self modifying code in .altinstr_replacement Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 03/14] objtool: Shrink struct instruction Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 04/14] objtool,x86: Replace alternatives with .retpoline_sites Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 05/14] x86/retpoline: Remove unused replacement symbols Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 06/14] x86/asm: Fix register order Peter Zijlstra
2021-10-22 19:27 ` David Laight
2021-10-25 14:09 ` Borislav Petkov
2021-10-20 10:44 ` [PATCH v2 07/14] x86/asm: Fixup odd GEN-for-each-reg.h usage Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 08/14] x86/retpoline: Create a retpoline thunk array Peter Zijlstra
2021-10-20 15:57 ` Josh Poimboeuf
2021-10-20 16:46 ` Andrew Cooper
2021-10-20 17:09 ` Josh Poimboeuf
2021-10-20 19:22 ` Peter Zijlstra
2021-10-20 19:43 ` Josh Poimboeuf
2021-10-20 19:34 ` Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 09/14] x86/alternative: Implement .retpoline_sites support Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 10/14] x86/alternative: Handle Jcc __x86_indirect_thunk_\reg Peter Zijlstra
2021-10-20 10:44 ` Peter Zijlstra [this message]
2021-10-20 10:44 ` [PATCH v2 12/14] x86/alternative: Add debug prints to apply_retpolines() Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 13/14] x86,bugs: Unconditionally allow spectre_v2=retpoline,amd Peter Zijlstra
2021-10-20 10:44 ` [PATCH v2 14/14] bpf,x86: Respect X86_FEATURE_RETPOLINE* Peter Zijlstra
2021-10-20 11:09 ` Peter Zijlstra
2021-10-20 16:56 ` Josh Poimboeuf
2021-10-20 19:23 ` Peter Zijlstra
2021-10-21 0:05 ` Alexei Starovoitov
2021-10-21 8:47 ` Peter Zijlstra
2021-10-21 18:03 ` Alexei Starovoitov
2021-10-21 22:37 ` Peter Zijlstra
2021-10-21 23:24 ` Alexei Starovoitov
2021-10-21 23:38 ` Josh Poimboeuf
2021-10-21 23:42 ` Alexei Starovoitov
2021-10-22 11:31 ` Peter Zijlstra
2021-10-22 15:22 ` Alexei Starovoitov
2021-10-25 13:44 ` Maciej Fijalkowski
2021-10-25 12:42 ` Peter Zijlstra
2021-10-21 23:51 ` Zvi Effron
2021-10-22 8:33 ` Peter Zijlstra
2021-10-22 21:06 ` Zvi Effron
2021-10-21 0:07 ` Alexei Starovoitov
2021-10-21 0:18 ` Josh Poimboeuf
2021-10-21 8:53 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211020105843.162995272@infradead.org \
--to=peterz@infradead.org \
--cc=alexei.starovoitov@gmail.com \
--cc=andrew.cooper3@citrix.com \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=ndesaulniers@google.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).