Re: XSAVE / RDPKRU on Intel 11th Gen Core CPUs

[Dave Hansen <dave.hansen@intel.com>]

On 11/8/21 5:47 PM, Brian Geffon wrote:
>> One more thing...  Does the protection_keys kernel selftest hit any
>> errors on this same setup?  It does a lot of PKRU sanity checking and
>> I'm a bit surprised it hasn't caught something yet.
> Hi Dave,
> 
> This issue does reproduce with the self tests too, my simple test
> program also fails consistently [1], all it does is spin executing
> RDPKRU waiting for a context switch to clobber the value.
> 
> $ ./test
> unexpected value on iteration 3772082 value:0x55555554 expected:0x55555550

Well, gosh, it's making it back to the software init value.  If you do:

	echo 0x15555554 > /sys/kernel/debug/x86/init_pkru

do you end up with 0x15555554 as the value?

The other thing you can try is to turn on all the
/sys/kernel/debug/tracing/events/x86_fpu tracepoints, pin your test
program to one CPU, then dump the trace buffer out for that CPU.  That
probably won't tell us too much for PKRU since it's generally not ever
in its init state.  But, another test would be to use XRSTOR to *get* it
into its init state then see how long it stays there.

Another thing you could do is figure out if pkeys _ever_ worked on that
hardware.  If so, a (long) bisect could figure out what broke it between
its introduction and the 5.13 kernel that you've been testing.
A random 5.11-based distro kernel that I have running on a Cascade Lake
Xeon doesn't seem to have any issues.

Does your system have any more XSAVE support than mine?

> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
> kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'
> kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
> kernel: [    0.000000] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64
> kernel: [    0.000000] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512
> kernel: [    0.000000] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024
> kernel: [    0.000000] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:    8
> kernel: [    0.000000] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.