[PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Zhenzhong Duan]

Lfence is preferred than general retpoline on AMD, add this option
in C / inline asm just as the ASM code does.

For x86_64, it still help to have minimal retpoline for kernel even
if gcc doesn't support it, change the inline asm for x86 so that it
could also be used by x86_64.

Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
"warning: objtool: .altinstr_replacement+0x10: unsupported
intra-function call"
"warning: objtool: If this is a retpoline, please patch it
in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
---
 arch/x86/include/asm/nospec-branch.h |   23 ++++++++++++++++-------
 1 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index fd2a8c1..2d49eab 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -170,21 +170,26 @@
  */
 # define CALL_NOSPEC						\
 	ANNOTATE_NOSPEC_ALTERNATIVE				\
-	ALTERNATIVE(						\
+	ALTERNATIVE_2(						\
 	ANNOTATE_RETPOLINE_SAFE					\
 	"call *%[thunk_target]\n",				\
 	"call __x86_indirect_thunk_%V[thunk_target]\n",		\
-	X86_FEATURE_RETPOLINE)
+	X86_FEATURE_RETPOLINE,					\
+	"lfence;\n"						\
+	ANNOTATE_RETPOLINE_SAFE					\
+	"call *%[thunk_target]\n",				\
+	X86_FEATURE_RETPOLINE_AMD)
 # define THUNK_TARGET(addr) [thunk_target] "r" (addr)
 
-#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
+#elif defined(CONFIG_RETPOLINE)
 /*
  * For i386 we use the original ret-equivalent retpoline, because
  * otherwise we'll run out of registers. We don't care about CET
  * here, anyway.
  */
 # define CALL_NOSPEC						\
-	ALTERNATIVE(						\
+	ANNOTATE_NOSPEC_ALTERNATIVE				\
+	ALTERNATIVE_2(						\
 	ANNOTATE_RETPOLINE_SAFE					\
 	"call *%[thunk_target]\n",				\
 	"       jmp    904f;\n"					\
@@ -194,12 +199,16 @@
 	"    	lfence;\n"					\
 	"       jmp    902b;\n"					\
 	"       .align 16\n"					\
-	"903:	addl   $4, %%esp;\n"				\
-	"       pushl  %[thunk_target];\n"			\
+	"903:	add   $4, %%" _ASM_SP ";\n"			\
+	"       push  %[thunk_target];\n"			\
 	"       ret;\n"						\
 	"       .align 16\n"					\
 	"904:	call   901b;\n",				\
-	X86_FEATURE_RETPOLINE)
+	X86_FEATURE_RETPOLINE,					\
+	"lfence;\n"						\
+	ANNOTATE_RETPOLINE_SAFE					\
+	"call *%[thunk_target]\n",				\
+	X86_FEATURE_RETPOLINE_AMD)
 
 # define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
 #else /* No retpoline for C / inline asm */
-- 
1.7.3

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Peter Zijlstra]

On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> Lfence is preferred than general retpoline on AMD, add this option
> in C / inline asm just as the ASM code does.
> 
> For x86_64, it still help to have minimal retpoline for kernel even
> if gcc doesn't support it, change the inline asm for x86 so that it
> could also be used by x86_64.
> Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
> "warning: objtool: .altinstr_replacement+0x10: unsupported
> intra-function call"
> "warning: objtool: If this is a retpoline, please patch it
> in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."

This Changelog is almost unreadable, please rewrite.

Reverse engineering the patch you add RETPOLINE_AMD support to the
inline-asm CALL_NOSPEC so that they match the asm CALL_NOSPEC.

> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
> ---
>  arch/x86/include/asm/nospec-branch.h |   23 ++++++++++++++++-------
>  1 files changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index fd2a8c1..2d49eab 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -170,21 +170,26 @@
>   */
>  # define CALL_NOSPEC						\
>  	ANNOTATE_NOSPEC_ALTERNATIVE				\
> -	ALTERNATIVE(						\
> +	ALTERNATIVE_2(						\
>  	ANNOTATE_RETPOLINE_SAFE					\
>  	"call *%[thunk_target]\n",				\
>  	"call __x86_indirect_thunk_%V[thunk_target]\n",		\
> -	X86_FEATURE_RETPOLINE)
> +	X86_FEATURE_RETPOLINE,					\
> +	"lfence;\n"						\
> +	ANNOTATE_RETPOLINE_SAFE					\
> +	"call *%[thunk_target]\n",				\
> +	X86_FEATURE_RETPOLINE_AMD)
>  # define THUNK_TARGET(addr) [thunk_target] "r" (addr)

That's OK.

>  
> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> +#elif defined(CONFIG_RETPOLINE)

This doesn't make any sense..

>  /*
>   * For i386 we use the original ret-equivalent retpoline, because
>   * otherwise we'll run out of registers. We don't care about CET
>   * here, anyway.
>   */
>  # define CALL_NOSPEC						\
> -	ALTERNATIVE(						\
> +	ANNOTATE_NOSPEC_ALTERNATIVE				\
> +	ALTERNATIVE_2(						\
>  	ANNOTATE_RETPOLINE_SAFE					\
>  	"call *%[thunk_target]\n",				\
>  	"       jmp    904f;\n"					\
> @@ -194,12 +199,16 @@
>  	"    	lfence;\n"					\
>  	"       jmp    902b;\n"					\
>  	"       .align 16\n"					\
> -	"903:	addl   $4, %%esp;\n"				\
> -	"       pushl  %[thunk_target];\n"			\
> +	"903:	add   $4, %%" _ASM_SP ";\n"			\
> +	"       push  %[thunk_target];\n"			\

Yeah, don't do that.

>  	"       ret;\n"						\
>  	"       .align 16\n"					\
>  	"904:	call   901b;\n",				\
> -	X86_FEATURE_RETPOLINE)
> +	X86_FEATURE_RETPOLINE,					\
> +	"lfence;\n"						\
> +	ANNOTATE_RETPOLINE_SAFE					\
> +	"call *%[thunk_target]\n",				\
> +	X86_FEATURE_RETPOLINE_AMD)

And that's OK again.

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Zhenzhong Duan]

On 2018/9/18 17:50, Peter Zijlstra wrote:
> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>> Lfence is preferred than general retpoline on AMD, add this option
>> in C / inline asm just as the ASM code does.
>>
>> For x86_64, it still help to have minimal retpoline for kernel even
>> if gcc doesn't support it, change the inline asm for x86 so that it
>> could also be used by x86_64.
>> Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
>> "warning: objtool: .altinstr_replacement+0x10: unsupported
>> intra-function call"
>> "warning: objtool: If this is a retpoline, please patch it
>> in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."
> 
> This Changelog is almost unreadable, please rewrite.
Sorry, I'll rewrite it.

> 
> Reverse engineering the patch you add RETPOLINE_AMD support to the
> inline-asm CALL_NOSPEC so that they match the asm CALL_NOSPEC.
> 
>> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
>> ---
>>   arch/x86/include/asm/nospec-branch.h |   23 ++++++++++++++++-------
>>   1 files changed, 16 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
>> index fd2a8c1..2d49eab 100644
>> --- a/arch/x86/include/asm/nospec-branch.h
>> +++ b/arch/x86/include/asm/nospec-branch.h
>> @@ -170,21 +170,26 @@
>>    */
>>   # define CALL_NOSPEC						\
>>   	ANNOTATE_NOSPEC_ALTERNATIVE				\
>> -	ALTERNATIVE(						\
>> +	ALTERNATIVE_2(						\
>>   	ANNOTATE_RETPOLINE_SAFE					\
>>   	"call *%[thunk_target]\n",				\
>>   	"call __x86_indirect_thunk_%V[thunk_target]\n",		\
>> -	X86_FEATURE_RETPOLINE)
>> +	X86_FEATURE_RETPOLINE,					\
>> +	"lfence;\n"						\
>> +	ANNOTATE_RETPOLINE_SAFE					\
>> +	"call *%[thunk_target]\n",				\
>> +	X86_FEATURE_RETPOLINE_AMD)
>>   # define THUNK_TARGET(addr) [thunk_target] "r" (addr)
> 
> That's OK.
> 
>>   
>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>> +#elif defined(CONFIG_RETPOLINE)
> 
> This doesn't make any sense..
This change is used for x86_64 to have minimal Retpoline support when 
CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed 
something?

> 
>>   /*
>>    * For i386 we use the original ret-equivalent retpoline, because
>>    * otherwise we'll run out of registers. We don't care about CET
>>    * here, anyway.
>>    */
>>   # define CALL_NOSPEC						\
>> -	ALTERNATIVE(						\
>> +	ANNOTATE_NOSPEC_ALTERNATIVE				\
>> +	ALTERNATIVE_2(						\
>>   	ANNOTATE_RETPOLINE_SAFE					\
>>   	"call *%[thunk_target]\n",				\
>>   	"       jmp    904f;\n"					\
>> @@ -194,12 +199,16 @@
>>   	"    	lfence;\n"					\
>>   	"       jmp    902b;\n"					\
>>   	"       .align 16\n"					\
>> -	"903:	addl   $4, %%esp;\n"				\
>> -	"       pushl  %[thunk_target];\n"			\
>> +	"903:	add   $4, %%" _ASM_SP ";\n"			\
>> +	"       push  %[thunk_target];\n"			\
> 
> Yeah, don't do that.
This is the change for above reason.

Thanks
Zhenzhong

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Peter Zijlstra]

On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
> On 2018/9/18 17:50, Peter Zijlstra wrote:
> > On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> > > -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> > > +#elif defined(CONFIG_RETPOLINE)
> > 
> > This doesn't make any sense..
> This change is used for x86_64 to have minimal Retpoline support when
> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
> something?

No it doesn't.

#if defined(X86_64) && defined(RETPOLINE)

 /* x86_64 retpoline goes here */

#elif defined(RETPOLINE)

 /* !x86_64 retpoline goes here */

#else

 /* !retpoline goes here

#endif

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Zhenzhong Duan]

On 2018/9/18 18:59, Peter Zijlstra wrote:
> On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
>> On 2018/9/18 17:50, Peter Zijlstra wrote:
>>> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>>>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>>>> +#elif defined(CONFIG_RETPOLINE)
>>>
>>> This doesn't make any sense..
>> This change is used for x86_64 to have minimal Retpoline support when
>> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
>> something?
> 
> No it doesn't.
> 
> #if defined(X86_64) && defined(RETPOLINE)
> 
>   /* x86_64 retpoline goes here */
> 
> #elif defined(RETPOLINE)
> 
>   /* !x86_64 retpoline goes here */
> 
> #else
> 
>   /* !retpoline goes here
> 
> #endif

Sorry, but I am confused.
So where is 'if defined(x86_64) && !defined(RETPOLINE) && 
defined(CONFIG_RETPOLINE)' go?
In original code, it will go to "call *%[thunk_target]\n" while
we have set SPECTRE_V2_RETPOLINE_MINIMAL or 
SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?

Thanks
Zhenzhong

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Peter Zijlstra]

On Tue, Sep 18, 2018 at 08:04:44PM +0800, Zhenzhong Duan wrote:
> On 2018/9/18 18:59, Peter Zijlstra wrote:
> > On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
> > > On 2018/9/18 17:50, Peter Zijlstra wrote:
> > > > On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> > > > > -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> > > > > +#elif defined(CONFIG_RETPOLINE)
> > > > 
> > > > This doesn't make any sense..
> > > This change is used for x86_64 to have minimal Retpoline support when
> > > CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
> > > something?
> > 
> > No it doesn't.
> > 
> > #if defined(X86_64) && defined(RETPOLINE)
> > 
> >   /* x86_64 retpoline goes here */
> > 
> > #elif defined(RETPOLINE)
> > 
> >   /* !x86_64 retpoline goes here */
> > 
> > #else
> > 
> >   /* !retpoline goes here
> > 
> > #endif
> 
> Sorry, but I am confused.
> So where is 'if defined(x86_64) && !defined(RETPOLINE) &&
> defined(CONFIG_RETPOLINE)' go?

Argh, CONFIG_RETPOLINE vs RETPOLINE :/

The thing is, the one you modify has a comment on that explains why it
is i386 only. CET and retpolines don't like one another much.

And the x86_64 version uses %V which requires new GCC.

So I'm all for fixing the RETPOLINE_AMD thing, but at this point nobody
should use the minimal stuff, that's just delusional.

> In original code, it will go to "call *%[thunk_target]\n" while
> we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?

Yes, that is exactly right -- it does that with or without your change
though.

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Peter Zijlstra]

> > In original code, it will go to "call *%[thunk_target]\n" while
> > we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> > SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
> 
> Yes, that is exactly right -- it does that with or without your change
> though.

Soryr, I wrote that before I noticed the CONFIG_RETPOLINE vs RETPOLINE
thing.

In any case, since retpoline capable compilers are widely available, I'm
all for making CONFIG_RETPOLINE hard depend on it. That minimal stuff
really is quite silly.

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 680 bytes --]

On Tue, 2018-09-18 at 15:03 +0200, Peter Zijlstra wrote:
> > > In original code, it will go to "call *%[thunk_target]\n" while
> > > we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> > > SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
> > 
> > Yes, that is exactly right -- it does that with or without your change
> > though.
> 
> Soryr, I wrote that before I noticed the CONFIG_RETPOLINE vs RETPOLINE
> thing.
> 
> In any case, since retpoline capable compilers are widely available, I'm
> all for making CONFIG_RETPOLINE hard depend on it. That minimal stuff
> really is quite silly.

Yep, now the the compiler support is widespread that makes a lot of
sense.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5213 bytes --]

Re: [PATCH] x86/speculation: Use AMD specific retpoline for inline asm on AMD

[Zhenzhong Duan]

On 2018/9/18 21:00, Peter Zijlstra wrote:
> On Tue, Sep 18, 2018 at 08:04:44PM +0800, Zhenzhong Duan wrote:
>> On 2018/9/18 18:59, Peter Zijlstra wrote:
>>> On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
>>>> On 2018/9/18 17:50, Peter Zijlstra wrote:
>>>>> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>>>>>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>>>>>> +#elif defined(CONFIG_RETPOLINE)
>>>>> This doesn't make any sense..
>>>> This change is used for x86_64 to have minimal Retpoline support when
>>>> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
>>>> something?
>>> No it doesn't.
>>>
>>> #if defined(X86_64) && defined(RETPOLINE)
>>>
>>>    /* x86_64 retpoline goes here */
>>>
>>> #elif defined(RETPOLINE)
>>>
>>>    /* !x86_64 retpoline goes here */
>>>
>>> #else
>>>
>>>    /* !retpoline goes here
>>>
>>> #endif
>> Sorry, but I am confused.
>> So where is 'if defined(x86_64) && !defined(RETPOLINE) &&
>> defined(CONFIG_RETPOLINE)' go?
> Argh, CONFIG_RETPOLINE vs RETPOLINE :/
> 
> The thing is, the one you modify has a comment on that explains why it
> is i386 only. CET and retpolines don't like one another much.
> 
> And the x86_64 version uses %V which requires new GCC.
> 
> So I'm all for fixing the RETPOLINE_AMD thing, but at this point nobody
> should use the minimal stuff, that's just delusional.
> 

Clear, thanks for your explanation.

Zhenzhong