CPU flags disappear on Spectre enabled host

CPU flags disappear on Spectre enabled host

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 808 bytes --]

On a host that is booted with the following command line, certain cpu flags disappear in xen-4.9 and later:

(XEN) Command line: loglvl=all guest_loglvl=all console=com1 com1=57600 cpuid=ibrsb,stibp,ibpb,ssbd spec-ctrl=ibrs,ibpb,ssbd,bti-thunk=retpoline xpti=yes

On my test system the difference in dom0s /proc/cpuinfo is like that:
--- xen-dom0-4.8-spectre.txt
+++ xen-dom0-4.12-spectre.txt
-arat
-arch_perfmon
-dtherm
-epb
-ida
+ss

The concern is that a domU started on a xen-4.8 or earlier host, which is then migrated to xen-4.9 or later may miss these cpu flags at runtime.

Is the loss of cpuflags intentional?

I just noticed that at least in staging a boot with "no-ibrsb no-stibp no-ibpb no-ssbd spec-ctrl=no-xen,rsb=no xpti=no" does still not show these flags in dom0.

Olaf

[-- Attachment #1.2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: CPU flags disappear on Spectre enabled host

[Andrew Cooper]

On 20/07/18 10:21, Olaf Hering wrote:
> On a host that is booted with the following command line, certain cpu flags disappear in xen-4.9 and later:
>
> (XEN) Command line: loglvl=all guest_loglvl=all console=com1 com1=57600 cpuid=ibrsb,stibp,ibpb,ssbd spec-ctrl=ibrs,ibpb,ssbd,bti-thunk=retpoline xpti=yes
>
> On my test system the difference in dom0s /proc/cpuinfo is like that:
> --- xen-dom0-4.8-spectre.txt
> +++ xen-dom0-4.12-spectre.txt
> -arat
> -arch_perfmon
> -dtherm
> -epb
> -ida
> +ss
>
> The concern is that a domU started on a xen-4.8 or earlier host, which is then migrated to xen-4.9 or later may miss these cpu flags at runtime.
>
> Is the loss of cpuflags intentional?

Yes, but they've got nothing to do with Spectre.

ARAT pertains to the Local APIC which doesn't exist for PV. 
ARCH_PERFMON is genuinely unavailable (hidden behind the vpmu Xen
cmdline parameter, and off for security reasons).

DTHERM/EPB/IDA should be (for backwards compatibility, not correctness)
leaked through into the dom0 kernel only, because classic-xen and PVOps
Linux makes some false and broken assumptions.  None of these bits
should be visible even to dom0, because dom0 can't actually use any of
the associated MSRs, but hiding them causes dom0 not to try parsing the
ACPI tables and feeding data to Xen.

SS on the other hand was unilaterally clobbered for reasons I can't work
out, and should have always been available to guests.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: CPU flags disappear on Spectre enabled host

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 595 bytes --]

Am Fri, 20 Jul 2018 10:38:30 +0100
schrieb Andrew Cooper <andrew.cooper3@citrix.com>:

> On 20/07/18 10:21, Olaf Hering wrote:
> > Is the loss of cpuflags intentional?  
> Yes, but they've got nothing to do with Spectre.

Thank you, Andrew. After a few more reboots I figured that booting without any cmdline option makes no difference.

From the original report that I got, these additional flags disappeared:

cqm_llc
cqm_occup_llc
pln
pts

I assume a domU can not make use of them either? They are not listed in CPUID in wikipedia, perhaps just not with those names.

Olaf

[-- Attachment #1.2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: CPU flags disappear on Spectre enabled host

[Andrew Cooper]

On 20/07/18 10:58, Olaf Hering wrote:
> Am Fri, 20 Jul 2018 10:38:30 +0100
> schrieb Andrew Cooper <andrew.cooper3@citrix.com>:
>
>> On 20/07/18 10:21, Olaf Hering wrote:
>>> Is the loss of cpuflags intentional?  
>> Yes, but they've got nothing to do with Spectre.
> Thank you, Andrew. After a few more reboots I figured that booting without any cmdline option makes no difference.
>
> From the original report that I got, these additional flags disappeared:
>
> cqm_llc
> cqm_occup_llc
> pln
> pts
>
> I assume a domU can not make use of them either? They are not listed in CPUID in wikipedia, perhaps just not with those names.

CQM is the quality-of-service measurement infrastructure on newer Intel
hardware.  They are behind the `psr` command line option, and can be
used with `xl psr-*` but only by the privileged domain, as they monitor
system-wide shared resources.

PLN is Power Limit Notification (apparently) which surely means that to
be used, we'd need a Xen driver.  PTS is Package Thermal Status and is
presumably the same.  Neither are things which even dom0 could use.

Having looked over the code, we actually hide all of the Thermal leaf
even from dom0.  It was the monitor leaf we conditionally leak back in. 
For the gory details, read xen/arch/x86/cpuid.c:guest_cpuid()

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel