LKML Archive on lore.kernel.org
 help / Atom feed
* [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
@ 2018-01-13 23:27 Tom Lendacky
  2018-01-14 15:41 ` Borislav Petkov
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Tom Lendacky @ 2018-01-13 23:27 UTC (permalink / raw)
  To: x86, linux-kernel
  Cc: Rik van Riel, Andi Kleen, Josh Poimboeuf, Peter Zijlstra,
	Linus Torvalds, Jiri Kosina, Dan Williams, Dave Hansen,
	Borislav Petkov, Andy Lutomirski, Kees Cook, Thomas Gleixner,
	Tim Chen, Greg Kroah-Hartman, David Woodhouse, Paul Turner

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
---
 arch/x86/include/asm/nospec-branch.h |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
 	call	772f;				\
 773:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	773b;				\
 772:						\
 	call	774f;				\
 775:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	775b;				\
 774:						\
 	dec	reg;				\
@@ -73,6 +75,7 @@
 	call	.Ldo_rop_\@
 .Lspec_trap_\@:
 	pause
+	lfence
 	jmp	.Lspec_trap_\@
 .Ldo_rop_\@:
 	mov	\reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
 	"       .align 16\n"					\
 	"901:	call   903f;\n"					\
 	"902:	pause;\n"					\
+	"    	lfence;\n"					\
 	"       jmp    902b;\n"					\
 	"       .align 16\n"					\
 	"903:	addl   $4, %%esp;\n"				\

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
  2018-01-13 23:27 [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros Tom Lendacky
@ 2018-01-14 15:41 ` Borislav Petkov
  2018-01-14 17:08   ` Woodhouse, David
  2018-01-14 17:48 ` [tip:x86/pti] " tip-bot for Tom Lendacky
  2018-01-14 23:37 ` tip-bot for Tom Lendacky
  2 siblings, 1 reply; 5+ messages in thread
From: Borislav Petkov @ 2018-01-14 15:41 UTC (permalink / raw)
  To: Tom Lendacky
  Cc: x86, linux-kernel, Rik van Riel, Andi Kleen, Josh Poimboeuf,
	Peter Zijlstra, Linus Torvalds, Jiri Kosina, Dan Williams,
	Dave Hansen, Andy Lutomirski, Kees Cook, Thomas Gleixner,
	Tim Chen, Greg Kroah-Hartman, David Woodhouse, Paul Turner

On Sat, Jan 13, 2018 at 05:27:30PM -0600, Tom Lendacky wrote:
> The PAUSE instruction is currently used in the retpoline and RSB filling
> macros as a speculation trap.  The use of PAUSE was originally suggested
> because it showed a very, very small difference in the amount of
> cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
> the PAUSE instruction is not a serializing instruction, so the pause/jmp
> loop will use excess power as it is speculated over waiting for return
> to mispredict to the correct target.
> 
> The RSB filling macro is applicable to AMD, and, if software is unable to
> verify that LFENCE is serializing on AMD (possible when running under a
> hypervisor), the generic retpoline support will be used and, so, is also
> applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
> LFENCE instruction to the speculation trap for AMD.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
>  arch/x86/include/asm/nospec-branch.h |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)

Reviewed-by: Borislav Petkov <bp@suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
  2018-01-14 15:41 ` Borislav Petkov
@ 2018-01-14 17:08   ` Woodhouse, David
  0 siblings, 0 replies; 5+ messages in thread
From: Woodhouse, David @ 2018-01-14 17:08 UTC (permalink / raw)
  To: Borislav Petkov, Tom Lendacky
  Cc: x86, linux-kernel, Rik van Riel, Andi Kleen, Josh Poimboeuf,
	Peter Zijlstra, Linus Torvalds, Jiri Kosina, Dan Williams,
	Dave Hansen, Andy Lutomirski, Kees Cook, Thomas Gleixner,
	Tim Chen, Greg Kroah-Hartman, Paul Turner

[-- Attachment #1: Type: text/plain, Size: 1350 bytes --]

On Sun, 2018-01-14 at 16:41 +0100, Borislav Petkov wrote:
> On Sat, Jan 13, 2018 at 05:27:30PM -0600, Tom Lendacky wrote:
> > 
> > The PAUSE instruction is currently used in the retpoline and RSB filling
> > macros as a speculation trap.  The use of PAUSE was originally suggested
> > because it showed a very, very small difference in the amount of
> > cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
> > the PAUSE instruction is not a serializing instruction, so the pause/jmp
> > loop will use excess power as it is speculated over waiting for return
> > to mispredict to the correct target.
> > 
> > The RSB filling macro is applicable to AMD, and, if software is unable to
> > verify that LFENCE is serializing on AMD (possible when running under a
> > hypervisor), the generic retpoline support will be used and, so, is also
> > applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
> > LFENCE instruction to the speculation trap for AMD.
> > 
> > Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> > ---
> >  arch/x86/include/asm/nospec-branch.h |    6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
>
> Reviewed-by: Borislav Petkov <bp@suse.de>

Acked-by: Arjan van de Ven <arjan@linux.intel.com>
Acked-by: David Woodhouse <dwmw@amazon.co.uk>

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5210 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
  2018-01-13 23:27 [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros Tom Lendacky
  2018-01-14 15:41 ` Borislav Petkov
@ 2018-01-14 17:48 ` " tip-bot for Tom Lendacky
  2018-01-14 23:37 ` tip-bot for Tom Lendacky
  2 siblings, 0 replies; 5+ messages in thread
From: tip-bot for Tom Lendacky @ 2018-01-14 17:48 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: thomas.lendacky, keescook, jikos, arjan, jpoimboe, riel,
	dave.hansen, mingo, hpa, ak, torvalds, gregkh, luto, pjt, peterz,
	dwmw, tglx, bp, linux-kernel, tim.c.chen, dan.j.williams

Commit-ID:  607d93a2658a0075047e87e47fde5bcc626a1918
Gitweb:     https://git.kernel.org/tip/607d93a2658a0075047e87e47fde5bcc626a1918
Author:     Tom Lendacky <thomas.lendacky@amd.com>
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Sun, 14 Jan 2018 18:22:15 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Borislav Petkov <bp@alien8.de>
Acked-by: David Woodhouse <dwmw@amazon.co.uk>
Acked-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
Cc: Kees Cook <keescook@google.com>
Link: https://lkml.kernel.org/r/20180113232730.31060.36287.stgit@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
 	call	772f;				\
 773:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	773b;				\
 772:						\
 	call	774f;				\
 775:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	775b;				\
 774:						\
 	dec	reg;				\
@@ -73,6 +75,7 @@
 	call	.Ldo_rop_\@
 .Lspec_trap_\@:
 	pause
+	lfence
 	jmp	.Lspec_trap_\@
 .Ldo_rop_\@:
 	mov	\reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
 	"       .align 16\n"					\
 	"901:	call   903f;\n"					\
 	"902:	pause;\n"					\
+	"    	lfence;\n"					\
 	"       jmp    902b;\n"					\
 	"       .align 16\n"					\
 	"903:	addl   $4, %%esp;\n"				\

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [tip:x86/pti] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
  2018-01-13 23:27 [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros Tom Lendacky
  2018-01-14 15:41 ` Borislav Petkov
  2018-01-14 17:48 ` [tip:x86/pti] " tip-bot for Tom Lendacky
@ 2018-01-14 23:37 ` tip-bot for Tom Lendacky
  2 siblings, 0 replies; 5+ messages in thread
From: tip-bot for Tom Lendacky @ 2018-01-14 23:37 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: hpa, bp, mingo, torvalds, ak, dwmw, riel, arjan, jpoimboe, pjt,
	dan.j.williams, tim.c.chen, tglx, dave.hansen, gregkh,
	linux-kernel, luto, thomas.lendacky, keescook, peterz, jikos

Commit-ID:  28d437d550e1e39f805d99f9f8ac399c778827b7
Gitweb:     https://git.kernel.org/tip/28d437d550e1e39f805d99f9f8ac399c778827b7
Author:     Tom Lendacky <thomas.lendacky@amd.com>
AuthorDate: Sat, 13 Jan 2018 17:27:30 -0600
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 15 Jan 2018 00:32:55 +0100

x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros

The PAUSE instruction is currently used in the retpoline and RSB filling
macros as a speculation trap.  The use of PAUSE was originally suggested
because it showed a very, very small difference in the amount of
cycles/time used to execute the retpoline as compared to LFENCE.  On AMD,
the PAUSE instruction is not a serializing instruction, so the pause/jmp
loop will use excess power as it is speculated over waiting for return
to mispredict to the correct target.

The RSB filling macro is applicable to AMD, and, if software is unable to
verify that LFENCE is serializing on AMD (possible when running under a
hypervisor), the generic retpoline support will be used and, so, is also
applicable to AMD.  Keep the current usage of PAUSE for Intel, but add an
LFENCE instruction to the speculation trap for AMD.

The same sequence has been adopted by GCC for the GCC generated retpolines.

Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Borislav Petkov <bp@alien8.de>
Acked-by: David Woodhouse <dwmw@amazon.co.uk>
Acked-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linux-foundation.org>
Cc: Kees Cook <keescook@google.com>
Link: https://lkml.kernel.org/r/20180113232730.31060.36287.stgit@tlendack-t1.amdoffice.net

---
 arch/x86/include/asm/nospec-branch.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 402a11c..7b45d84 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -11,7 +11,7 @@
  * Fill the CPU return stack buffer.
  *
  * Each entry in the RSB, if used for a speculative 'ret', contains an
- * infinite 'pause; jmp' loop to capture speculative execution.
+ * infinite 'pause; lfence; jmp' loop to capture speculative execution.
  *
  * This is required in various cases for retpoline and IBRS-based
  * mitigations for the Spectre variant 2 vulnerability. Sometimes to
@@ -38,11 +38,13 @@
 	call	772f;				\
 773:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	773b;				\
 772:						\
 	call	774f;				\
 775:	/* speculation trap */			\
 	pause;					\
+	lfence;					\
 	jmp	775b;				\
 774:						\
 	dec	reg;				\
@@ -73,6 +75,7 @@
 	call	.Ldo_rop_\@
 .Lspec_trap_\@:
 	pause
+	lfence
 	jmp	.Lspec_trap_\@
 .Ldo_rop_\@:
 	mov	\reg, (%_ASM_SP)
@@ -165,6 +168,7 @@
 	"       .align 16\n"					\
 	"901:	call   903f;\n"					\
 	"902:	pause;\n"					\
+	"    	lfence;\n"					\
 	"       jmp    902b;\n"					\
 	"       .align 16\n"					\
 	"903:	addl   $4, %%esp;\n"				\

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-13 23:27 [PATCH v2] x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros Tom Lendacky
2018-01-14 15:41 ` Borislav Petkov
2018-01-14 17:08   ` Woodhouse, David
2018-01-14 17:48 ` [tip:x86/pti] " tip-bot for Tom Lendacky
2018-01-14 23:37 ` tip-bot for Tom Lendacky

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org linux-kernel@archiver.kernel.org
	public-inbox-index lkml


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/ public-inbox