Re: [PATCH 23/35] x86/speculation: Add basic speculation control code

[Andrea Arcangeli <aarcange@redhat.com>]

Hello,

On Thu, Jan 18, 2018 at 03:25:25PM -0800, Andy Lutomirski wrote:
> I read the whitepaper that documented the new MSRs a couple days ago
> and I'm now completely unable to find it.  If anyone could send the
> link, that would be great.

I see Andrew posted a link.

> From memory, however, the docs were quite clear that setting leaving
> IBRS set when entering user mode or guest mode does not guarantee any
> particular protection unless an additional CPUID bit (the name of
> which I forget) is set, and that current CPUs will *not* get that bit

My current understanding is that with SPEC_CTRL alone set in cpuid,
IBRS is meaningful, other bits don't matter.

> set by microcode update.  IOW the protection given is that, if you set
> IBRS bit zero after entry to kernel mode, you are protected until you
> re-enter user mode.  When you're in user mode, you're not protected.

If you leave IBRS set while in user mode, userland is protected as
strong as kernel mode.

Of course kernel can attack usermode through spectre variant#2 if it
wants :), but usermode has to trust the kernel to begin with, just
like guest mode has to trust the host kernel.

If you leave IBRS set, guest mode (including guest usermode) cannot
attack host userland, host userland cannot attack other host userland
and no HT is possible either.

Note guest kernel attack on host userland is not as concerning but
it's possible too, as long as you use host dm-crypt instead of qcow2
encryption on host, it's not a major concern. guest userland attack on
host userland is more of a concern because if it can be mounted
successfully, it would dump all guest memory making it impossible for
the guest to defend itself from its own userland (unless it uses ibpb
2 of course which calls IBPB at guest user to kernel entry instead of
IBRS). IBRS isn't enough in guest because like retpoline it doesn't
guarantee an IBPB.

Because these are only theoretical and there's no PoC IBRS is not
enabled by default in userland of course. However retpolining qemu
would be a low overhead sure solution to this that would avoid having
to set ibrs in userland to achieve the same.

> When you return back to kernel mode, you *still* aren't protected no
> matter what value is in the MSR until you write 1 again.

When you return to kernel mode you've to call IBRS again even if it
was left set, because there's a higher-privilege mode change. That's
equivalent to calling only IBPB and leaving STIBP set (only way to
understand the locations where IBRS has to be set is to imagine IBRS
as a strict "STIBP; IBPB").

specs are very explicit that it's very meaningful to set IBRS even if
already set.

> > That is true no matter if kernel is using retpolines or ibrs.
> >
> > IBRS is semantically equivalent to "STIBP; IBPB", so user_ibrs is
> > always inclusive of user_stibp.
> 
> Are you quite sure?  I had the impression that IBPB was much slower

Nothing in the specs says that IBRS is a "STIBP; IBPB" equivalent, but
if you want to find where to set IBRS, yes, I'm quite sure, you've to
think it like it.

> than writing 1 to IBRS and that writing 1 to IBRS has much more
> limited effects.

I think I already partly answered it already in the sentence that
followed the one you quoted, but I should elaborate it.

The semantics to use depending what you're trying to solve because it
can have both.

"STIBP; IBPB" could be called the coarse semantics and you have to
imagine IBRS as "STIBP; IBPB" whenever you are checking where to write
IBRS, or you risk missing places where you have to write IBRS even if
it is already set.

As opposed when you check when you need to leave IBRS set (note: after
it was already set in all places found with the coarse semantics),
or where you need to call IBPB, you can't rely on the coarse
semantics because of course the IBPB may not have really
happened... and so you've to imagine IBRS with finegrined semantics as
"temporary immunization from the poison including HT/SMT poison and
all poison is still left in the CPU and will return to affect the
runtime as soon as IBRS is cleared".

IBRS Q/A:

1) When to write IBRS in SPEC_CTRL? -> imagine it as "STIBP; IBPB"

2) When to leave IBRS set or when to call IBPB -> imagine the previous
setting of IBRS as temporarily disabling indirect branch prediction
without an IBPB implicit in IBRS

If you think it only like 1) you risk missing some places where you've
to write IBRS even if it was already set.

If you think it only like 2) you risk clearing it too early or you
risk missing a necessary IBPB.

It has to be thought simultaneously in both ways.

The sure thing I get from the specs is IBRS always implies STIBP (even
when STIBP is a noop), specs are pretty explicit about that.

So following the above two rules, assume you've retpolined host kernel
and you want to protect host userland from guest userland, IBRS set in
the host kernel to host user transition (user_ibrs) will achieve it
fine. Setting STIBP in the kernel (user_stibp) to user transition
won't help at all with this scenario.

If the kernel in host was using IBRS instead of retpolines, it already
had to set IBRS in vmexit to satisfy the coarse semantics and it would
just need to leave IBRS set in the kernel to user transition (it would
then need to set IBRS again in the user to kernel transition that
follows even if it was already set).

Thanks,
Andrea