On 24/10/2019 21:17, speck for Borislav Petkov wrote:
> On Thu, Oct 24, 2019 at 09:07:27PM +0100, speck for Andrew Cooper wrote:
>> On Xen, I've juggled things such that we load microcode, then interpret
>> tsx= if the user has provided it (taking care to always write
>> MSR_TSX_CTRL if it is available, to discard whatever settings firmware
>> or kexec left), before querying CPUID.
> Yap, wanna try the same thing, in that exact order.
>
>> Later, the spec-ctrl= interpretation happens, which might choose to turn
>> off TSX due to TAA, which then has to modify MSR_TSX_CTRL and force
>> clear the bits in the policy.
> Well, the kernel doesn't reeval CPUID feature bits in that case because
> it has gone on booting and enabled all kinds of feature supporting code.
> This is the reason why the whole late microcode loading is such a bad
> thing to do.

:)

I don't necessarily disagree, but the customers (who ultimately pay my
salary) want late microcode loading and livepatching, so we've delivered.

>
>> On Haswell and Broadwell, the microcode which turned HLE/RTM off in the
>> pipeline left the LBR MSRs in a state where you can't context switch the
>> value, because they would yield a value via RDMSR which WRMSR faulted
>> on, because the two operations had an asymmetric view of how the top
>> bits of metadata should be interpreted, given some TSX-related metadata
>> and a sign extended linear address.
>>
>> On Skylake where you can't actually turn RTM off, but we may hide
>> FEATURE_RTM/HLE, the above quirk is probably not true.
> Huh? How is that possible? TSX_CTRL has defined only bit 1 there, the CPUID
> enumeration bit, and bit 0 doesn't do any RTM disabling? Srsly?!

Skylake CPUs aren't getting TSX_CTRL, but force setting/clearing bits at
boot will affect later logic.  (Unless I'm being blind while reading the
patches, which is a distinct possibility).

>
>> On Cascadelake, who knows?  RTM is being turned off in the pipeline, but
>> maybe the HSX/BWX bug has been fixed, or maybe it is being turned off in
>> a different way, or ...
> Right, I guess we'll deal with any perfcounters fallout in public, when
> the stuff releases...

So, I remembered that I had already written a test case for this bug.

Initial experimentation shows that using TSX_CTRL to secure Cascade Lake
doesn't result in Haswell/Broadwell style GP faults, which is good
news.  I will be adjusting Xen's logic not to invoke the quirk on more
modern parts.

~Andrew