[PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Maciej S. Szmigiero]

If we run on a CPU that does not have IBPB support RSB entries from one
userspace process can influence 'ret' target prediction in another
userspace process after a context switch.

Since it is unlikely that existing RSB entries from the previous task match
the new task call stack we can use the existing unconditional
RSB-filling-on-context-switch infrastructure to protect against such
userspace-to-userspace attacks.

This patch brings a change in behavior only for the following CPU types:
* Intel pre-Skylake CPUs without updated microcode,
* AMD Family 15h model >60h, Family 17h CPUs without updated microcode.

Other CPU types either already do the RSB filling on context switch for
other reasons or do support IBPB for more complete userspace-to-userspace
protection.

Signed-off-by: Maciej S. Szmigiero <mail@maciej.szmigiero.name>
---
 arch/x86/kernel/cpu/bugs.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index bfca937bdcc3..777bae86e159 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -280,8 +280,11 @@ static void __init spectre_v2_select_mitigation(void)
 	/*
 	 * If neither SMEP nor PTI are available, there is a risk of
 	 * hitting userspace addresses in the RSB after a context switch
-	 * from a shallow call stack to a deeper one. To prevent this fill
-	 * the entire RSB, even when using IBRS.
+	 * from a shallow call stack to a deeper one.
+	 * Also, if the CPU does not have IBPB support then one userspace
+	 * process can influence 'ret' target prediction for another
+	 * userspace process.
+	 * To prevent this fill the entire RSB, even when using IBRS.
 	 *
 	 * Skylake era CPUs have a separate issue with *underflow* of the
 	 * RSB, when they will predict 'ret' targets from the generic BTB.
@@ -290,7 +293,8 @@ static void __init spectre_v2_select_mitigation(void)
 	 * switch is required.
 	 */
 	if ((!boot_cpu_has(X86_FEATURE_PTI) &&
-	     !boot_cpu_has(X86_FEATURE_SMEP)) || is_skylake_era()) {
+	     !boot_cpu_has(X86_FEATURE_SMEP)) ||
+	     !boot_cpu_has(X86_FEATURE_IBPB) || is_skylake_era()) {
 		setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW);
 		pr_info("Spectre v2 mitigation: Filling RSB on context switch\n");
 	}

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Dave Hansen]

On 03/20/2018 04:17 AM, Maciej S. Szmigiero wrote:
> If we run on a CPU that does not have IBPB support RSB entries from one
> userspace process can influence 'ret' target prediction in another
> userspace process after a context switch.
> 
> Since it is unlikely that existing RSB entries from the previous task match
> the new task call stack we can use the existing unconditional
> RSB-filling-on-context-switch infrastructure to protect against such
> userspace-to-userspace attacks.
> 
> This patch brings a change in behavior only for the following CPU types:
> * Intel pre-Skylake CPUs without updated microcode,

The assumption thus far (good or bad) is that everything will get a
microcode update.  I actually don't know for sure if RSB manipulation is
effective on old microcode before Skylake.  I'm pretty sure it has not
been documented publicly.

How did you decide that this is an effective mitigation?

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Maciej S. Szmigiero]

On 21.03.2018 15:05, Dave Hansen wrote:
> On 03/20/2018 04:17 AM, Maciej S. Szmigiero wrote:
>> If we run on a CPU that does not have IBPB support RSB entries from one
>> userspace process can influence 'ret' target prediction in another
>> userspace process after a context switch.
>>
>> Since it is unlikely that existing RSB entries from the previous task match
>> the new task call stack we can use the existing unconditional
>> RSB-filling-on-context-switch infrastructure to protect against such
>> userspace-to-userspace attacks.
>>
>> This patch brings a change in behavior only for the following CPU types:
>> * Intel pre-Skylake CPUs without updated microcode,
> 
> The assumption thus far (good or bad) is that everything will get a
> microcode update.  I actually don't know for sure if RSB manipulation is
> effective on old microcode before Skylake.  I'm pretty sure it has not
> been documented publicly.
> 
> How did you decide that this is an effective mitigation?
> 
A RSB overwrite is already being done even on pre-Skylake Intel CPUs on
VMEXIT to protect the host from the guest, regardless of the microcode
version.

But I see that an Intel guidance document published last month about
retpolines says that "RET has this [predictable speculative] behavior on
all processors (...) microarchitecture codename Broadwell and earlier
when updated with the latest microcode".
This suggests that updated microcode may be needed for protection anyway
on such CPUs - as you say.  Such update (hopefully) brings IBPB
support, too, so I agree that the change introduced by this patch can be
skipped on Intel CPUs.

Maciej

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Dave Hansen]

On 03/20/2018 04:17 AM, Maciej S. Szmigiero wrote:
> Since it is unlikely that existing RSB entries from the previous task match
> the new task call stack we can use the existing unconditional
> RSB-filling-on-context-switch infrastructure to protect against such
> userspace-to-userspace attacks.
> 
> This patch brings a change in behavior only for the following CPU types:
> * Intel pre-Skylake CPUs without updated microcode,
> * AMD Family 15h model >60h, Family 17h CPUs without updated microcode.
> 
> Other CPU types either already do the RSB filling on context switch for
> other reasons or do support IBPB for more complete userspace-to-userspace
> protection.

I think I misunderstood your reasoning a bit.  Let me see if I can
restate the problem a bit.

IBPB provides provides userspace-to-userspace protection because it
prevents all indirect branch predictions after the barrier from being
controlled by software executed before the barrier.  We only use IBPB
for KVM and when processes clear their dumpable flag.

You're saying that, even if we don't have IBPB, we can do *some*
userspace-to-userspace protection with RSB manipulation.  The RSB
manipulation obviously only helps 'RET' instructions and not JMP/CALL,
but it does do *something* useful.

Is that right?

Do you really want this behavior on all context switches?  We don't do
IBPB on all context switches, only the ones where we are switching *to*
a non-dumpable process.

Do you perhaps want to do RSB manipulation in lieu of IBPB when
switching *to* a non-dumpable process and IBPB is not available?

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Maciej S. Szmigiero]

On 22.03.2018 00:30, Dave Hansen wrote:
> On 03/20/2018 04:17 AM, Maciej S. Szmigiero wrote:
>> Since it is unlikely that existing RSB entries from the previous task match
>> the new task call stack we can use the existing unconditional
>> RSB-filling-on-context-switch infrastructure to protect against such
>> userspace-to-userspace attacks.
>>
>> This patch brings a change in behavior only for the following CPU types:
>> * Intel pre-Skylake CPUs without updated microcode,
>> * AMD Family 15h model >60h, Family 17h CPUs without updated microcode.
>>
>> Other CPU types either already do the RSB filling on context switch for
>> other reasons or do support IBPB for more complete userspace-to-userspace
>> protection.
> 
> I think I misunderstood your reasoning a bit.  Let me see if I can
> restate the problem a bit.
> 
> IBPB provides provides userspace-to-userspace protection because it
> prevents all indirect branch predictions after the barrier from being
> controlled by software executed before the barrier.  We only use IBPB
> for KVM and when processes clear their dumpable flag.
> 
> You're saying that, even if we don't have IBPB, we can do *some*
> userspace-to-userspace protection with RSB manipulation.  The RSB
> manipulation obviously only helps 'RET' instructions and not JMP/CALL,
> but it does do *something* useful.
> 
> Is that right?

Yes.

As far as I understand the issue this should provide a good protection
for userspace processes that were recompiled with retpolines as they
won't have any indirect jumps and calls.

> Do you really want this behavior on all context switches?  We don't do
> IBPB on all context switches, only the ones where we are switching *to*
> a non-dumpable process.
> 
> Do you perhaps want to do RSB manipulation in lieu of IBPB when
> switching *to* a non-dumpable process and IBPB is not available?
> 

Is it worth differentiating such processes in this case?
IBPB is supposed to be very expensive so certainly it is worthwhile
to do it only for high-value processes (=non-dumpable).

However, it is unlikely that existing RSB entries from the previous
task match the new task call stack anyway.
We already do unconditional RSB-filling-on-context-switch in many
cases.

Maciej

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Dave Hansen]

On 03/21/2018 05:09 PM, Maciej S. Szmigiero wrote:
> As far as I understand the issue this should provide a good protection
> for userspace processes that were recompiled with retpolines as they
> won't have any indirect jumps and calls.

Instead of saying "good protection", let's just say that it could
mitigate attacks that require consumption of attacker-placed RSB entries.

>> Do you perhaps want to do RSB manipulation in lieu of IBPB when
>> switching *to* a non-dumpable process and IBPB is not available?
> 
> Is it worth differentiating such processes in this case?
> IBPB is supposed to be very expensive so certainly it is worthwhile
> to do it only for high-value processes (=non-dumpable).
> 
> However, it is unlikely that existing RSB entries from the previous
> task match the new task call stack anyway.
> We already do unconditional RSB-filling-on-context-switch in many
> cases.

I think this case is a bit too obscure and theoretical to complicate the
kernel with it.  You need an unmitigated processor, a
userspace-to-userspace attack that manages to satisfy the five "exploit
composition" steps of Spectre/V2[1], and an application that has been
retpoline-mitigated.

While RSB manipulation is almost certainly less onerous than IBPB, it's
still going to hurt context-switch rates, especially if applied
indiscriminately like this patch does.

So, I totally agree with your analysis about the theoretical potential
for an issue, I'm just not really convinced the fix is worth it.

1.
https://software.intel.com/sites/default/files/managed/1d/46/Retpoline-A-Branch-Target-Injection-Mitigation.pdf

Re: [PATCH] x86/speculation: Fill the RSB on context switch also on non-IBPB CPUs

[Maciej S. Szmigiero]

On 22.03.2018 16:46, Dave Hansen wrote:
> On 03/21/2018 05:09 PM, Maciej S. Szmigiero wrote:
>> As far as I understand the issue this should provide a good protection
>> for userspace processes that were recompiled with retpolines as they
>> won't have any indirect jumps and calls.
> 
> Instead of saying "good protection", let's just say that it could
> mitigate attacks that require consumption of attacker-placed RSB entries.

All right.

>>> Do you perhaps want to do RSB manipulation in lieu of IBPB when
>>> switching *to* a non-dumpable process and IBPB is not available?
>>
>> Is it worth differentiating such processes in this case?
>> IBPB is supposed to be very expensive so certainly it is worthwhile
>> to do it only for high-value processes (=non-dumpable).
>>
>> However, it is unlikely that existing RSB entries from the previous
>> task match the new task call stack anyway.
>> We already do unconditional RSB-filling-on-context-switch in many
>> cases.
> 
> I think this case is a bit too obscure and theoretical to complicate the
> kernel with it.  You need an unmitigated processor, a
> userspace-to-userspace attack that manages to satisfy the five "exploit
> composition" steps of Spectre/V2[1], and an application that has been
> retpoline-mitigated.
> 
> While RSB manipulation is almost certainly less onerous than IBPB, it's
> still going to hurt context-switch rates, especially if applied
> indiscriminately like this patch does.
> 
> So, I totally agree with your analysis about the theoretical potential
> for an issue, I'm just not really convinced the fix is worth it.

Yes, Spectre v2 looks really hard to exploit, but this doesn't mean the
kernel shouldn't do its best to mitigate it.

As I wrote two messages ago, basing on the Intel guidance document you
linked above as "[1]" I think that the mitigation introduced by this
patch should not be done on Intel CPUs, however, since that document
clearly suggests that this may not be enough to cover the issue.
And I think we shouldn't give people a false sense of security.

Maciej