bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: bpf@vger.kernel.org
Subject: [PATCH bpf-next 10/17] x86/retpoline: Create a retpoline thunk array
Date: Wed, 27 Oct 2021 10:52:53 +0200	[thread overview]
Message-ID: <20211027085520.684740055@infradead.org> (raw)
In-Reply-To: 20211027085243.008677168@infradead.org

Stick all the retpolines in a single symbol and have the individual
thunks as inner labels, this should guarantee thunk order and layout.

Previously there were 16 (or rather 15 without rsp) separate symbols and
a toolchain might reasonably expect it could displace them however it
liked, with disregard for their relative position.

However, now they're part of a larger symbol. Any change to their
relative position would disrupt this larger _array symbol and thus not
be sound.

This is the same reasoning used for data symbols. On their own there
is no guarantee about their relative position wrt to one aonther, but
we're still able to do arrays because an array as a whole is a single
larger symbol.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/include/asm/nospec-branch.h |    8 +++++++-
 arch/x86/lib/retpoline.S             |   14 +++++++++-----
 2 files changed, 16 insertions(+), 6 deletions(-)

--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -12,6 +12,8 @@
 #include <asm/msr-index.h>
 #include <asm/unwind_hints.h>
 
+#define RETPOLINE_THUNK_SIZE	32
+
 /*
  * Fill the CPU return stack buffer.
  *
@@ -120,11 +122,15 @@
 
 #ifdef CONFIG_RETPOLINE
 
+typedef u8 retpoline_thunk_t[RETPOLINE_THUNK_SIZE];
+
 #define GEN(reg) \
-	extern asmlinkage void __x86_indirect_thunk_ ## reg (void);
+	extern retpoline_thunk_t __x86_indirect_thunk_ ## reg;
 #include <asm/GEN-for-each-reg.h>
 #undef GEN
 
+extern retpoline_thunk_t __x86_indirect_thunk_array[];
+
 #ifdef CONFIG_X86_64
 
 /*
--- a/arch/x86/lib/retpoline.S
+++ b/arch/x86/lib/retpoline.S
@@ -28,16 +28,14 @@
 
 .macro THUNK reg
 
-	.align 32
-
-SYM_FUNC_START(__x86_indirect_thunk_\reg)
+	.align RETPOLINE_THUNK_SIZE
+SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL)
+	UNWIND_HINT_EMPTY
 
 	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
 		      __stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \
 		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_AMD
 
-SYM_FUNC_END(__x86_indirect_thunk_\reg)
-
 .endm
 
 /*
@@ -55,10 +53,16 @@ SYM_FUNC_END(__x86_indirect_thunk_\reg)
 #define __EXPORT_THUNK(sym)	_ASM_NOKPROBE(sym); EXPORT_SYMBOL(sym)
 #define EXPORT_THUNK(reg)	__EXPORT_THUNK(__x86_indirect_thunk_ ## reg)
 
+	.align RETPOLINE_THUNK_SIZE
+SYM_CODE_START(__x86_indirect_thunk_array)
+
 #define GEN(reg) THUNK reg
 #include <asm/GEN-for-each-reg.h>
 #undef GEN
 
+	.align RETPOLINE_THUNK_SIZE
+SYM_CODE_END(__x86_indirect_thunk_array)
+
 #define GEN(reg) EXPORT_THUNK(reg)
 #include <asm/GEN-for-each-reg.h>
 #undef GEN



  parent reply	other threads:[~2021-10-27  8:58 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-27  8:52 [PATCH bpf-next 00/17] x86: Rewrite the retpoline rewrite logic Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 01/17] objtool: Handle __sanitize_cov*() tail calls Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 02/17] objtool: Classify symbols Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 03/17] objtool: Explicitly avoid self modifying code in .altinstr_replacement Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 04/17] objtool: Shrink struct instruction Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 05/17] objtool,x86: Replace alternatives with .retpoline_sites Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 06/17] x86/retpoline: Remove unused replacement symbols Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 07/17] x86/asm: Fix register order Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 08/17] x86/asm: Fixup odd GEN-for-each-reg.h usage Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 09/17] x86/retpoline: Move the retpoline thunk declarations to nospec-branch.h Peter Zijlstra
2021-10-27  8:52 ` Peter Zijlstra [this message]
2021-10-27  8:52 ` [PATCH bpf-next 11/17] x86/alternative: Implement .retpoline_sites support Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 12/17] x86/alternative: Handle Jcc __x86_indirect_thunk_\reg Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 13/17] x86/alternative: Try inline spectre_v2=retpoline,amd Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 14/17] x86/alternative: Add debug prints to apply_retpolines() Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 15/17] x86,bugs: Unconditionally allow spectre_v2=retpoline,amd Peter Zijlstra
2021-10-27  8:52 ` [PATCH bpf-next 16/17] bpf, x86: Simplify computing label offsets Peter Zijlstra
2021-10-27  8:53 ` [PATCH bpf-next 17/17] bpf,x86: Respect X86_FEATURE_RETPOLINE* Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211027085520.684740055@infradead.org \
    --to=peterz@infradead.org \
    --cc=bpf@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).