[PATCH] objtool: Fix SLS checks

[PATCH] objtool: Fix SLS checks

[Peter Zijlstra]

Fix the SLS validation; not having a next instruction is also a fail
when the next instruction should be INSN_TRAP.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 tools/objtool/check.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 3f6785415894..3354101ffe34 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_RETURN:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after ret",
 					  insn->sec, insn->offset);
 			}
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
 
 		case INSN_JUMP_DYNAMIC:
 			if (sls && !insn->retpoline_safe &&
-			    next_insn && next_insn->type != INSN_TRAP) {
+			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
 				WARN_FUNC("missing int3 after indirect jump",
 					  insn->sec, insn->offset);
 			}

Re: [PATCH] objtool: Fix SLS checks

[Josh Poimboeuf]

On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
> 
> Fix the SLS validation; not having a next instruction is also a fail
> when the next instruction should be INSN_TRAP.
> 
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
>  tools/objtool/check.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index 3f6785415894..3354101ffe34 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
>  
>  		case INSN_RETURN:
>  			if (sls && !insn->retpoline_safe &&
> -			    next_insn && next_insn->type != INSN_TRAP) {
> +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
>  				WARN_FUNC("missing int3 after ret",
>  					  insn->sec, insn->offset);
>  			}
> @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
>  
>  		case INSN_JUMP_DYNAMIC:
>  			if (sls && !insn->retpoline_safe &&
> -			    next_insn && next_insn->type != INSN_TRAP) {
> +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
>  				WARN_FUNC("missing int3 after indirect jump",
>  					  insn->sec, insn->offset);
>  			}

My SLS rewrite in tip/objtool/core already fixed this, FWIW.  But this
could be good for -urgent.

Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>

Here's another SLS improvement I mentioned to you the other day, do you
agree we should do this as well?


From: Josh Poimboeuf <jpoimboe@redhat.com>
Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled

Having disabled retpolines doesn't necessarily mean the user doesn't
care about straight-line speculation.  For example, retpolines are
disabled when eIBRS is used.

If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the
retpolines disabled cases.

Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
---
 arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
 tools/objtool/check.c                |  3 ---
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index acbaeaf83b61..0648746bf60b 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -75,6 +75,13 @@
 	.popsection
 .endm
 
+.macro INDIRECT_JMP reg
+	jmp *%\reg
+#ifdef CONFIG_SLS
+	int3
+#endif
+.endm
+
 /*
  * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
  * indirect jmp/call which may be susceptible to the Spectre variant 2
@@ -82,11 +89,11 @@
  */
 .macro JMP_NOSPEC reg:req
 #ifdef CONFIG_RETPOLINE
-	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
 		      __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
-		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
+		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
 #else
-	jmp	*%\reg
+	INDIRECT_JMP \reg
 #endif
 .endm
 
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index e7983c3e2408..3cf3ad0b5db5 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
 	for_each_insn(file, insn) {
 		next_insn = next_insn_same_sec(file, insn);
 
-		if (insn->retpoline_safe)
-			continue;
-
 		switch (insn->type) {
 		case INSN_RETURN:
 			if (!next_insn || next_insn->type != INSN_TRAP) {
-- 
2.34.1

Re: [PATCH] objtool: Fix SLS checks

[Peter Zijlstra]

On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:
> On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
> > 
> > Fix the SLS validation; not having a next instruction is also a fail
> > when the next instruction should be INSN_TRAP.
> > 
> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > ---
> >  tools/objtool/check.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> > index 3f6785415894..3354101ffe34 100644
> > --- a/tools/objtool/check.c
> > +++ b/tools/objtool/check.c
> > @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> >  
> >  		case INSN_RETURN:
> >  			if (sls && !insn->retpoline_safe &&
> > -			    next_insn && next_insn->type != INSN_TRAP) {
> > +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> >  				WARN_FUNC("missing int3 after ret",
> >  					  insn->sec, insn->offset);
> >  			}
> > @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> >  
> >  		case INSN_JUMP_DYNAMIC:
> >  			if (sls && !insn->retpoline_safe &&
> > -			    next_insn && next_insn->type != INSN_TRAP) {
> > +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> >  				WARN_FUNC("missing int3 after indirect jump",
> >  					  insn->sec, insn->offset);
> >  			}
> 
> My SLS rewrite in tip/objtool/core already fixed this, FWIW.  But this
> could be good for -urgent.

Urgh, I should've looked at that indeed. This didn't find any new sites
though; so I don't think this needs to go through urgent.

Re: [PATCH] objtool: Fix SLS checks

[Josh Poimboeuf]

On Mon, May 02, 2022 at 10:01:53PM +0200, Peter Zijlstra wrote:
> On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:
> > On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
> > > 
> > > Fix the SLS validation; not having a next instruction is also a fail
> > > when the next instruction should be INSN_TRAP.
> > > 
> > > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > > ---
> > >  tools/objtool/check.c | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> > > index 3f6785415894..3354101ffe34 100644
> > > --- a/tools/objtool/check.c
> > > +++ b/tools/objtool/check.c
> > > @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> > >  
> > >  		case INSN_RETURN:
> > >  			if (sls && !insn->retpoline_safe &&
> > > -			    next_insn && next_insn->type != INSN_TRAP) {
> > > +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > >  				WARN_FUNC("missing int3 after ret",
> > >  					  insn->sec, insn->offset);
> > >  			}
> > > @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> > >  
> > >  		case INSN_JUMP_DYNAMIC:
> > >  			if (sls && !insn->retpoline_safe &&
> > > -			    next_insn && next_insn->type != INSN_TRAP) {
> > > +			    (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > >  				WARN_FUNC("missing int3 after indirect jump",
> > >  					  insn->sec, insn->offset);
> > >  			}
> > 
> > My SLS rewrite in tip/objtool/core already fixed this, FWIW.  But this
> > could be good for -urgent.
> 
> Urgh, I should've looked at that indeed. This didn't find any new sites
> though; so I don't think this needs to go through urgent.

Well to be fair, it was easy to miss since I snuck it in with rewrite.

-- 
Josh

Re: [PATCH] objtool: Fix SLS checks

[Peter Zijlstra]

On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:

> From: Josh Poimboeuf <jpoimboe@redhat.com>
> Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled
> 
> Having disabled retpolines doesn't necessarily mean the user doesn't
> care about straight-line speculation.  For example, retpolines are
> disabled when eIBRS is used.
> 
> If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the
> retpolines disabled cases.
> 
> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
> ---
>  arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
>  tools/objtool/check.c                |  3 ---
>  2 files changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index acbaeaf83b61..0648746bf60b 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -75,6 +75,13 @@
>  	.popsection
>  .endm
>  
> +.macro INDIRECT_JMP reg
> +	jmp *%\reg
> +#ifdef CONFIG_SLS
> +	int3
> +#endif
> +.endm
> +
>  /*
>   * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
>   * indirect jmp/call which may be susceptible to the Spectre variant 2
> @@ -82,11 +89,11 @@
>   */
>  .macro JMP_NOSPEC reg:req
>  #ifdef CONFIG_RETPOLINE
> -	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
> +	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
>  		      __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
> -		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
> +		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
>  #else
> -	jmp	*%\reg
> +	INDIRECT_JMP \reg
>  #endif
>  .endm
>  
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index e7983c3e2408..3cf3ad0b5db5 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
>  	for_each_insn(file, insn) {
>  		next_insn = next_insn_same_sec(file, insn);
>  
> -		if (insn->retpoline_safe)
> -			continue;
> -
>  		switch (insn->type) {
>  		case INSN_RETURN:
>  			if (!next_insn || next_insn->type != INSN_TRAP) {

Yes, agreed. But perhaps with something like this on top?

---
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 3c66073e7645..84beeb5297d5 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
 		return ret;
 	i += ret;
 
+#ifdef CONFIG_SLS
+	/*
+	 * Ideally this would be unconditional, except in case of
+	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
+	 * -mharden-sls=all should be extended to emit INT3 after
+	 * direct jumps too, which will then cover that case.
+	 */
+	if (i < insn->length)
+		bytes[i++] = INT3_INSN_OPCODE;
+#endif
+
 	for (; i < insn->length;)
 		bytes[i++] = BYTES_NOP1;
 
diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S
index b2b2366885a2..26e742da3129 100644
--- a/arch/x86/lib/retpoline.S
+++ b/arch/x86/lib/retpoline.S
@@ -33,9 +33,9 @@ SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL)
 	UNWIND_HINT_EMPTY
 	ANNOTATE_NOENDBR
 
-	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+	ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
 		      __stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \
-		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg; int3), X86_FEATURE_RETPOLINE_LFENCE
+		      __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
 
 .endm
 
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 16b6efacf7c6..6929856e7f6d 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -418,6 +418,10 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)
 #endif
 	EMIT2(0xFF, 0xE0 + reg);
 
+#ifdef CONFIG_SLS
+	EMIT1(0xCC);
+#endif
+
 	*pprog = prog;
 }
 

Re: [PATCH] objtool: Fix SLS checks

[Josh Poimboeuf]

On Mon, May 02, 2022 at 10:17:39PM +0200, Peter Zijlstra wrote:
> > +++ b/tools/objtool/check.c
> > @@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
> >  	for_each_insn(file, insn) {
> >  		next_insn = next_insn_same_sec(file, insn);
> >  
> > -		if (insn->retpoline_safe)
> > -			continue;
> > -
> >  		switch (insn->type) {
> >  		case INSN_RETURN:
> >  			if (!next_insn || next_insn->type != INSN_TRAP) {
> 
> Yes, agreed. But perhaps with something like this on top?

Yup, I missed those...  Looks good.  Just one comment:

> +++ b/arch/x86/kernel/alternative.c
> @@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
>  		return ret;
>  	i += ret;
>  
> +#ifdef CONFIG_SLS
> +	/*
> +	 * Ideally this would be unconditional, except in case of
> +	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
> +	 * -mharden-sls=all should be extended to emit INT3 after
> +	 * direct jumps too, which will then cover that case.
> +	 */

I don't quite follow this 2nd sentence and how it's related here, since
this function doesn't actually deal with direct jumps.

Speaking of, I guess we'll eventually need to hack this SLS mess into
jump labels :-/

> +	if (i < insn->length)
> +		bytes[i++] = INT3_INSN_OPCODE;
> +#endif

-- 
Josh

Re: [PATCH] objtool: Fix SLS checks

[Peter Zijlstra]

On Tue, May 03, 2022 at 02:15:10PM -0700, Josh Poimboeuf wrote:
> On Mon, May 02, 2022 at 10:17:39PM +0200, Peter Zijlstra wrote:
> > > +++ b/tools/objtool/check.c
> > > @@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
> > >  	for_each_insn(file, insn) {
> > >  		next_insn = next_insn_same_sec(file, insn);
> > >  
> > > -		if (insn->retpoline_safe)
> > > -			continue;
> > > -
> > >  		switch (insn->type) {
> > >  		case INSN_RETURN:
> > >  			if (!next_insn || next_insn->type != INSN_TRAP) {
> > 
> > Yes, agreed. But perhaps with something like this on top?
> 
> Yup, I missed those...  Looks good.  Just one comment:
> 
> > +++ b/arch/x86/kernel/alternative.c
> > @@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
> >  		return ret;
> >  	i += ret;
> >  
> > +#ifdef CONFIG_SLS
> > +	/*
> > +	 * Ideally this would be unconditional, except in case of
> > +	 * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
> > +	 * -mharden-sls=all should be extended to emit INT3 after
> > +	 * direct jumps too, which will then cover that case.
> > +	 */
> 
> I don't quite follow this 2nd sentence and how it's related here, since
> this function doesn't actually deal with direct jumps.

Ah, my bad. Also, this wrong.

I suppose this wants to be something like:

	if (i < insn->length && op == JMP32_INSN_OPCODE)
		bytes[i++] = INT3_INSN_OPCODE;

So this *can* be a jump, but typically won't be I suppose.

> Speaking of, I guess we'll eventually need to hack this SLS mess into
> jump labels :-/

Urgh... can't we reason that the straight line case is actually expected
to run with the given register state anyway and ignore this?

Re: [PATCH] objtool: Fix SLS checks

[Josh Poimboeuf]

On Wed, May 04, 2022 at 09:26:47AM +0200, Peter Zijlstra wrote:
> > I don't quite follow this 2nd sentence and how it's related here, since
> > this function doesn't actually deal with direct jumps.
> 
> Ah, my bad. Also, this wrong.
> 
> I suppose this wants to be something like:
> 
> 	if (i < insn->length && op == JMP32_INSN_OPCODE)
> 		bytes[i++] = INT3_INSN_OPCODE;
> 
> So this *can* be a jump, but typically won't be I suppose.

Yep.

> > Speaking of, I guess we'll eventually need to hack this SLS mess into
> > jump labels :-/
> 
> Urgh... can't we reason that the straight line case is actually expected
> to run with the given register state anyway and ignore this?

Yeah, that makes sense.

So for jump labels the SLS path would probably not be worse than a
typical v1-style conditional branch misspeculation into the 'else' path,
and we've already given up on worrying about those anyway.

-- 
Josh