From: "tip-bot2 for Peter Zijlstra" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: x86/core] x86/returnthunk: Allow different return thunks
Date: Mon, 17 Oct 2022 14:53:42 -0000 [thread overview]
Message-ID: <166601842272.401.18030395061124314174.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20220915111147.680469665@infradead.org>
The following commit has been merged into the x86/core branch of tip:
Commit-ID: 770ae1b709528a6a173b5c7b183818ee9b45e376
Gitweb: https://git.kernel.org/tip/770ae1b709528a6a173b5c7b183818ee9b45e376
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Thu, 15 Sep 2022 13:11:25 +02:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 17 Oct 2022 16:41:14 +02:00
x86/returnthunk: Allow different return thunks
In preparation for call depth tracking on Intel SKL CPUs, make it possible
to patch in a SKL specific return thunk.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20220915111147.680469665@infradead.org
---
arch/x86/include/asm/nospec-branch.h | 6 ++++++
arch/x86/kernel/alternative.c | 17 +++++++++++++----
arch/x86/kernel/ftrace.c | 2 +-
arch/x86/kernel/static_call.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
5 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index c936ce9..f10ca33 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -208,6 +208,12 @@ extern void __x86_return_thunk(void);
extern void zen_untrain_ret(void);
extern void entry_ibpb(void);
+#ifdef CONFIG_CALL_THUNKS
+extern void (*x86_return_thunk)(void);
+#else
+#define x86_return_thunk (&__x86_return_thunk)
+#endif
+
#ifdef CONFIG_RETPOLINE
#define GEN(reg) \
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 963872d..04d1e3d 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -518,6 +518,11 @@ void __init_or_module noinline apply_retpolines(s32 *start, s32 *end)
}
#ifdef CONFIG_RETHUNK
+
+#ifdef CONFIG_CALL_THUNKS
+void (*x86_return_thunk)(void) __ro_after_init = &__x86_return_thunk;
+#endif
+
/*
* Rewrite the compiler generated return thunk tail-calls.
*
@@ -533,14 +538,18 @@ static int patch_return(void *addr, struct insn *insn, u8 *bytes)
{
int i = 0;
- if (cpu_feature_enabled(X86_FEATURE_RETHUNK))
- return -1;
+ if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) {
+ if (x86_return_thunk == __x86_return_thunk)
+ return -1;
- bytes[i++] = RET_INSN_OPCODE;
+ i = JMP32_INSN_SIZE;
+ __text_gen_insn(bytes, JMP32_INSN_OPCODE, addr, x86_return_thunk, i);
+ } else {
+ bytes[i++] = RET_INSN_OPCODE;
+ }
for (; i < insn->length;)
bytes[i++] = INT3_INSN_OPCODE;
-
return i;
}
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 00eac45..4ac6692 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -359,7 +359,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)
ip = trampoline + size;
if (cpu_feature_enabled(X86_FEATURE_RETHUNK))
- __text_gen_insn(ip, JMP32_INSN_OPCODE, ip, &__x86_return_thunk, JMP32_INSN_SIZE);
+ __text_gen_insn(ip, JMP32_INSN_OPCODE, ip, x86_return_thunk, JMP32_INSN_SIZE);
else
memcpy(ip, retq, sizeof(retq));
diff --git a/arch/x86/kernel/static_call.c b/arch/x86/kernel/static_call.c
index aaaba85..5d3844a 100644
--- a/arch/x86/kernel/static_call.c
+++ b/arch/x86/kernel/static_call.c
@@ -52,7 +52,7 @@ static void __ref __static_call_transform(void *insn, enum insn_type type,
case RET:
if (cpu_feature_enabled(X86_FEATURE_RETHUNK))
- code = text_gen_insn(JMP32_INSN_OPCODE, insn, &__x86_return_thunk);
+ code = text_gen_insn(JMP32_INSN_OPCODE, insn, x86_return_thunk);
else
code = &retinsn;
break;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 9962042..0df391e 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -432,7 +432,7 @@ static void emit_return(u8 **pprog, u8 *ip)
u8 *prog = *pprog;
if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) {
- emit_jump(&prog, &__x86_return_thunk, ip);
+ emit_jump(&prog, x86_return_thunk, ip);
} else {
EMIT1(0xC3); /* ret */
if (IS_ENABLED(CONFIG_SLS))
next prev parent reply other threads:[~2022-10-17 14:54 UTC|newest]
Thread overview: 138+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-15 11:10 [PATCH v3 00/59] x86/retbleed: Call depth tracking mitigation Peter Zijlstra
2022-09-15 11:10 ` [PATCH v3 01/59] x86/paravirt: Ensure proper alignment Peter Zijlstra
2022-09-21 11:08 ` [tip: x86/paravirt] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 02/59] x86/cpu: Remove segment load from switch_to_new_gdt() Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 03/59] x86/cpu: Get rid of redundant switch_to_new_gdt() invocations Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 04/59] x86/cpu: Re-enable stackprotector Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 05/59] x86/modules: Set VM_FLUSH_RESET_PERMS in module_alloc() Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 06/59] x86/vdso: Ensure all kernel code is seen by objtool Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 07/59] x86: Sanitize linker script Peter Zijlstra
2022-10-07 16:03 ` Borislav Petkov
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 08/59] arch: Introduce CONFIG_FUNCTION_ALIGNMENT Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:10 ` [PATCH v3 09/59] x86/asm: Differentiate between code and function alignment Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 10/59] x86/error_inject: Align function properly Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:10 ` [PATCH v3 11/59] x86/paravirt: Properly align PV functions Peter Zijlstra
2022-09-15 14:34 ` Juergen Gross
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 12/59] x86/entry: Align SYM_CODE_START() variants Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 13/59] crypto: x86/camellia: Remove redundant alignments Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 14/59] crypto: x86/cast5: " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 15/59] crypto: x86/crct10dif-pcl: " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 16/59] crypto: x86/serpent: " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 17/59] crypto: x86/sha1: Remove custom alignments Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 18/59] crypto: x86/sha256: " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 19/59] crypto: x86/sm[34]: Remove redundant alignments Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:10 ` [PATCH v3 20/59] crypto: twofish: " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 21/59] crypto: x86/poly1305: Remove custom function alignment Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 22/59] x86: Put hot per CPU variables into a struct Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 23/59] x86/percpu: Move preempt_count next to current_task Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 24/59] x86/percpu: Move cpu_number " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 25/59] x86/percpu: Move current_top_of_stack " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 26/59] x86/percpu: Move irq_stack variables " Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 27/59] x86/softirq: Move softirq pending next to current task Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 28/59] objtool: Allow !PC relative relocations Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 29/59] objtool: Track init section Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 30/59] objtool: Add .call_sites section Peter Zijlstra
2022-10-17 14:54 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 31/59] objtool: Add --hacks=skylake Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 32/59] objtool: Allow STT_NOTYPE -> STT_FUNC+0 tail-calls Peter Zijlstra
2022-09-22 5:27 ` Pawan Gupta
2022-09-22 10:29 ` Peter Zijlstra
2022-09-22 10:47 ` Peter Zijlstra
2022-09-22 13:15 ` Peter Zijlstra
2022-09-23 14:35 ` Peter Zijlstra
2022-09-23 17:36 ` Pawan Gupta
2022-09-15 11:11 ` [PATCH v3 33/59] objtool: Fix find_{symbol,func}_containing() Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 34/59] objtool: Allow symbol range comparisons for IBT/ENDBR Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 35/59] x86/entry: Make sync_regs() invocation a tail call Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 36/59] ftrace: Add HAVE_DYNAMIC_FTRACE_NO_PATCHABLE Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 37/59] x86/putuser: Provide room for padding Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 38/59] x86/Kconfig: Add CONFIG_CALL_THUNKS Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 39/59] x86/Kconfig: Introduce function padding Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 40/59] x86/retbleed: Add X86_FEATURE_CALL_DEPTH Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 41/59] x86/alternatives: Provide text_poke_copy_locked() Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 42/59] x86/entry: Make some entry symbols global Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 43/59] x86/paravirt: Make struct paravirt_call_site unconditionally available Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 44/59] x86/callthunks: Add call patching for call depth tracking Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 45/59] x86/modules: Add call patching Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 46/59] x86/returnthunk: Allow different return thunks Peter Zijlstra
2022-10-17 14:53 ` tip-bot2 for Peter Zijlstra [this message]
2022-09-15 11:11 ` [PATCH v3 47/59] x86/asm: Provide ALTERNATIVE_3 Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 48/59] x86/retbleed: Add SKL return thunk Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-10-20 23:10 ` [PATCH v3 48/59] " Nathan Chancellor
2022-10-21 9:53 ` Peter Zijlstra
2022-10-21 15:21 ` Nathan Chancellor
2022-11-03 22:53 ` KVM vs AMD: " Andrew Cooper
2022-11-04 12:44 ` Peter Zijlstra
2022-11-04 15:29 ` Andrew Cooper
2022-11-04 15:32 ` Nathan Chancellor
2022-11-07 9:37 ` Paolo Bonzini
2022-09-15 11:11 ` [PATCH v3 49/59] x86/retpoline: Add SKL retthunk retpolines Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 50/59] x86/retbleed: Add SKL call thunk Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 51/59] x86/calldepth: Add ret/call counting for debug Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2022-09-15 11:11 ` [PATCH v3 52/59] static_call: Add call depth tracking support Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 53/59] kallsyms: Take callthunks into account Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 54/59] x86/orc: Make it callthunk aware Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 55/59] x86/bpf: Emit call depth accounting if required Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
2023-01-05 21:49 ` [PATCH v3 55/59] " Joan Bruguera
2022-09-15 11:11 ` [PATCH v3 56/59] x86/ftrace: Remove ftrace_epilogue() Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-10-20 15:17 ` [tip: x86/urgent] " tip-bot2 for Peter Zijlstra
2022-12-09 15:41 ` [PATCH v3 56/59] " Steven Rostedt
2022-09-15 11:11 ` [PATCH v3 57/59] x86/ftrace: Rebalance RSB Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 58/59] x86/ftrace: Make it call depth tracking aware Peter Zijlstra
2022-09-21 10:19 ` [PATCH v3.1 " Peter Zijlstra
2022-09-21 18:45 ` Pawan Gupta
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2022-09-15 11:11 ` [PATCH v3 59/59] x86/retbleed: Add call depth tracking mitigation Peter Zijlstra
2022-10-17 14:53 ` [tip: x86/core] " tip-bot2 for Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=166601842272.401.18030395061124314174.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).