From: Andi Kleen <andi@firstfloor.org>
To: tglx@linutronix.de
Cc: x86@kernel.org, dwmw@amazon.co.uk, linux-kernel@vger.kernel.org,
pjt@google.com, torvalds@linux-foundation.org,
gregkh@linux-foundation.org, peterz@infradead.org,
luto@amacapital.net, thomas.lendacky@amd.com,
arjan.van.de.ven@intel.com, Andi Kleen <ak@linux.intel.com>
Subject: [PATCH 3/4] x86/retpoline: Fill return buffer after idle
Date: Fri, 12 Jan 2018 10:45:49 -0800 [thread overview]
Message-ID: <20180112184550.6573-4-andi@firstfloor.org> (raw)
In-Reply-To: <20180112184550.6573-1-andi@firstfloor.org>
From: Andi Kleen <ak@linux.intel.com>
When we go into deeper idle states the return buffer could be cleared
in MWAIT, but then another thread which wakes up earlier might
be poisoning the indirect branch predictor. Then when the return
buffer underflows there might an uncontrolled indirect branch.
To guard against this always fill the return buffer when exiting idle.
This is only enabled on Skylake.
v2:
Switch to using inline fill_return_buffer macro
Also handle mwait_idle
Port to new fill_return_buffer infrastructure
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
arch/x86/include/asm/mwait.h | 10 +++++++++-
arch/x86/kernel/process.c | 1 +
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h
index 39a2fb29378a..632b6b39fe01 100644
--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -6,6 +6,7 @@
#include <linux/sched/idle.h>
#include <asm/cpufeature.h>
+#include <asm/nospec-branch.h>
#define MWAIT_SUBSTATE_MASK 0xf
#define MWAIT_CSTATE_MASK 0xf
@@ -107,8 +108,15 @@ static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx)
}
__monitor((void *)¤t_thread_info()->flags, 0, 0);
- if (!need_resched())
+ if (!need_resched()) {
__mwait(eax, ecx);
+ /*
+ * idle could have cleared the return buffer,
+ * so fill it to prevent uncontrolled
+ * speculation.
+ */
+ fill_return_buffer();
+ }
}
current_clr_polling();
}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 3cb2486c47e4..9a7c1bb0e001 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -466,6 +466,7 @@ static __cpuidle void mwait_idle(void)
__sti_mwait(0, 0);
else
local_irq_enable();
+ fill_return_buffer();
trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id());
} else {
local_irq_enable();
--
2.14.3
next prev parent reply other threads:[~2018-01-12 18:46 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-12 18:45 Improve retpoline for Skylake Andi Kleen
2018-01-12 18:45 ` [PATCH 1/4] x86/retpoline: Add new mode RETPOLINE_UNDERFLOW Andi Kleen
2018-01-12 22:13 ` Dominik Brodowski
2018-01-12 22:57 ` Andi Kleen
2018-01-12 18:45 ` [PATCH 2/4] x86/retpoline: Avoid return buffer underflows on context switch Andi Kleen
2018-01-12 18:45 ` Andi Kleen [this message]
2018-01-12 18:45 ` [PATCH 4/4] x86/retpoline: Fill return buffer on interrupt return to kernel Andi Kleen
2018-01-12 19:12 ` Improve retpoline for Skylake David Woodhouse
2018-01-12 19:21 ` Andi Kleen
2018-01-12 22:03 ` Henrique de Moraes Holschuh
2018-01-15 8:26 ` Jon Masters
2018-01-15 9:06 ` David Woodhouse
2018-01-15 10:03 ` Thomas Gleixner
2018-01-15 10:20 ` David Woodhouse
2018-01-15 16:57 ` Andy Lutomirski
2018-01-15 17:38 ` Andrew Cooper
2018-01-15 17:56 ` Van De Ven, Arjan
2018-01-15 18:06 ` Andy Lutomirski
2018-01-15 18:07 ` David Woodhouse
2018-01-15 18:10 ` Andy Lutomirski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180112184550.6573-4-andi@firstfloor.org \
--to=andi@firstfloor.org \
--cc=ak@linux.intel.com \
--cc=arjan.van.de.ven@intel.com \
--cc=dwmw@amazon.co.uk \
--cc=gregkh@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).