From: "Yang, Weijiang" <weijiang.yang@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
Dave Hansen <dave.hansen@intel.com>,
Thomas Gleixner <tglx@linutronix.de>, <peterz@infradead.org>,
"Sean Christopherson" <seanjc@google.com>
Cc: Chao Gao <chao.gao@intel.com>, <john.allen@amd.com>,
<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<rick.p.edgecombe@intel.com>, <binbin.wu@linux.intel.com>
Subject: Re: [PATCH v5 09/19] KVM:x86: Make guest supervisor states as non-XSAVE managed
Date: Fri, 11 Aug 2023 11:03:53 +0800 [thread overview]
Message-ID: <c014fd8a-5456-478f-b643-44cfff27f931@intel.com> (raw)
In-Reply-To: <8396a9f6-fbc4-1e62-b6a9-3df568fd15a2@redhat.com>
On 8/10/2023 11:15 PM, Paolo Bonzini wrote:
> On 8/10/23 16:29, Dave Hansen wrote:
>> On 8/10/23 02:29, Yang, Weijiang wrote:
>> ...
>>> When KVM enumerates shadow stack support for guest in CPUID(0x7,
>>> 0).ECX[bit7], architecturally it claims both SS user and supervisor
>>> mode are supported. Although the latter is not supported in Linux,
>>> but in virtualization world, the guest OS could be non-Linux system,
>>> so KVM supervisor state support is necessary in this case.
>>
>> What actual OSes need this support?
>
> I think Xen could use it when running nested. But KVM cannot expose support for CET in CPUID, and at the same time fake support for MSR_IA32_PL{0,1,2}_SSP (e.g. inject a #GP if it's ever written to a nonzero value).
>
> I suppose we could invent our own paravirtualized CPUID bit for "supervisor IBT works but supervisor SHSTK doesn't". Linux could check that but I don't think it's a good idea.
>
> So... do, or do not. There is no try. :)
>
>>> Two solutions are on the table:
>>> 1) Enable CET supervisor support in Linux kernel like user mode support.
>>
>> We _will_ do this eventually, but not until FRED is merged. The core
>> kernel also probably won't be managing the MSRs on non-FRED hardware.
>>
>> I think what you're really talking about here is that the kernel would
>> enable CET_S XSAVE state management so that CET_S state could be managed
>> by the core kernel's FPU code.
>
> Yes, I understand it that way too.
Sorry for confusion, I missed the word "state" here.
>> That is, frankly, *NOT* like the user mode support at all.
>
> I agree.
>
>>> 2) Enable support in KVM domain.
>>>
>>> Problem:
>>> The Pros/Cons for each solution(my individual thoughts):
>>> In kernel solution:
>>> Pros:
>>> - Avoid saving/restoring 3 supervisor MSRs(PL{0,1,2}_SSP) at vCPU
>>> execution path.
>>> - Easy for KVM to manage guest CET xstate bits for guest.
>>> Cons:
>>> - Unnecessary supervisor state xsaves/xrstors operation for non-vCPU
>>> thread.
>>
>> What operations would be unnecessary exactly?
>
> Saving/restoring PL0/1/2_SSP when switching from one usermode task's fpstate to another.
>
>>> KVM solution:
>>> Pros:
>>> - Not touch current kernel FPU management framework and logic.
>>> - No extra space and operation for non-vCPU thread.
>>> Cons:
>>> - Manually saving/restoring 3 supervisor MSRs is a performance burden to
>>> KVM.
>>> - It looks more like a hack method for KVM, and some handling logic
>>> seems a bit awkward.
>>
>> In a perfect world, we'd just allocate space for CET_S in the KVM
>> fpstates. The core kernel fpstates would have
>> XSTATE_BV[13]==XCOMP_BV[13]==0. An XRSTOR of the core kernel fpstates
>> would just set CET_S to its init state.
>
> Yep. I don't think it's a lot of work to implement. The basic idea as you point out below is something like
>
> #define XFEATURE_MASK_USER_DYNAMIC XFEATURE_MASK_XTILE_DATA
> #define XFEATURE_MASK_USER_OPTIONAL \
> (XFEATURE_MASK_DYNAMIC | XFEATURE_MASK_CET_KERNEL)
>
> where XFEATURE_MASK_USER_DYNAMIC is used for xfd-related tasks (including the ARCH_GET_XCOMP_SUPP arch_prctl) but everything else uses XFEATURE_MASK_USER_OPTIONAL.
>
> KVM would enable the feature by hand when allocating the guest fpstate. Disabled features would be cleared from EDX:EAX when calling XSAVE/XSAVEC/XSAVES.
OK, I'll move ahead in that direction.
>> But I suspect that would be too much work to implement in practice. It
>> would be akin to a new lesser kind of dynamic xstate, one that didn't
>> interact with XFD and *NEVER* gets allocated in the core kernel
>> fpstates, even on demand.
>>
>> I want to hear more about who is going to use CET_S state under KVM in
>> practice. I don't want to touch it if this is some kind of purely
>> academic exercise. But it's also silly to hack some kind of temporary
>> solution into KVM that we'll rip out in a year when real supervisor
>> shadow stack support comes along.
>>
>> If it's actually necessary, we should probably just eat the 24 bytes in
>> the fpstates, flip the bit in IA32_XSS and move on. There shouldn't be
>> any other meaningful impact to the core kernel.
>
> If that's good to you, why not.
Thanks to all of you for quickly helping out!
> Paolo
>
next prev parent reply other threads:[~2023-08-11 3:05 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-03 4:27 [PATCH v5 00/19] Enable CET Virtualization Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 01/19] x86/cpufeatures: Add CPU feature flags for shadow stacks Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 02/19] x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 03/19] KVM:x86: Report XSS as to-be-saved if there are supported features Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 04/19] KVM:x86: Refresh CPUID on write to guest MSR_IA32_XSS Yang Weijiang
2023-08-04 16:02 ` Sean Christopherson
2023-08-04 21:43 ` Paolo Bonzini
2023-08-09 3:11 ` Yang, Weijiang
2023-08-08 14:20 ` Yang, Weijiang
2023-08-04 18:27 ` Sean Christopherson
2023-08-07 6:55 ` Paolo Bonzini
2023-08-09 8:56 ` Yang, Weijiang
2023-08-10 0:01 ` Paolo Bonzini
2023-08-10 1:12 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 05/19] KVM:x86: Initialize kvm_caps.supported_xss Yang Weijiang
2023-08-04 18:45 ` Sean Christopherson
2023-08-08 15:08 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 06/19] KVM:x86: Load guest FPU state when access XSAVE-managed MSRs Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 07/19] KVM:x86: Add fault checks for guest CR4.CET setting Yang Weijiang
2023-08-03 9:07 ` Chao Gao
2023-08-03 4:27 ` [PATCH v5 08/19] KVM:x86: Report KVM supported CET MSRs as to-be-saved Yang Weijiang
2023-08-03 10:39 ` Chao Gao
2023-08-04 3:13 ` Yang, Weijiang
2023-08-04 5:51 ` Chao Gao
2023-08-04 18:51 ` Sean Christopherson
2023-08-04 22:01 ` Paolo Bonzini
2023-08-08 15:16 ` Yang, Weijiang
2023-08-06 8:54 ` Yang, Weijiang
2023-08-04 18:55 ` Sean Christopherson
2023-08-08 15:26 ` Yang, Weijiang
2023-08-04 21:47 ` Paolo Bonzini
2023-08-09 3:14 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 09/19] KVM:x86: Make guest supervisor states as non-XSAVE managed Yang Weijiang
2023-08-03 11:15 ` Chao Gao
2023-08-04 3:26 ` Yang, Weijiang
2023-08-04 20:45 ` Sean Christopherson
2023-08-04 20:59 ` Peter Zijlstra
2023-08-04 21:32 ` Paolo Bonzini
2023-08-09 2:51 ` Yang, Weijiang
2023-08-09 2:39 ` Yang, Weijiang
2023-08-10 9:29 ` Yang, Weijiang
2023-08-10 14:29 ` Dave Hansen
2023-08-10 15:15 ` Paolo Bonzini
2023-08-10 15:37 ` Sean Christopherson
2023-08-11 3:03 ` Yang, Weijiang [this message]
2023-08-28 21:00 ` Dave Hansen
2023-08-29 7:05 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 10/19] KVM:VMX: Introduce CET VMCS fields and control bits Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 11/19] KVM:VMX: Emulate read and write to CET MSRs Yang Weijiang
2023-08-04 5:14 ` Chao Gao
2023-08-04 21:27 ` Sean Christopherson
2023-08-04 21:45 ` Paolo Bonzini
2023-08-04 22:21 ` Sean Christopherson
2023-08-07 7:03 ` Paolo Bonzini
2023-08-06 8:44 ` Yang, Weijiang
2023-08-07 7:00 ` Paolo Bonzini
2023-08-04 8:28 ` Chao Gao
2023-08-09 7:12 ` Yang, Weijiang
2023-08-04 21:40 ` Paolo Bonzini
2023-08-09 3:05 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 12/19] KVM:x86: Save and reload SSP to/from SMRAM Yang Weijiang
2023-08-04 7:53 ` Chao Gao
2023-08-04 15:25 ` Sean Christopherson
2023-08-06 9:14 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 13/19] KVM:VMX: Set up interception for CET MSRs Yang Weijiang
2023-08-04 8:16 ` Chao Gao
2023-08-06 9:22 ` Yang, Weijiang
2023-08-07 1:16 ` Chao Gao
2023-08-09 6:11 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 14/19] KVM:VMX: Set host constant supervisor states to VMCS fields Yang Weijiang
2023-08-04 8:23 ` Chao Gao
2023-08-03 4:27 ` [PATCH v5 15/19] KVM:x86: Optimize CET supervisor SSP save/reload Yang Weijiang
2023-08-04 8:43 ` Chao Gao
2023-08-09 9:00 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 16/19] KVM:x86: Enable CET virtualization for VMX and advertise to userspace Yang Weijiang
2023-08-03 4:27 ` [PATCH v5 17/19] KVM:x86: Enable guest CET supervisor xstate bit support Yang Weijiang
2023-08-04 22:02 ` Paolo Bonzini
2023-08-09 6:07 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 18/19] KVM:nVMX: Refine error code injection to nested VM Yang Weijiang
2023-08-04 21:38 ` Sean Christopherson
2023-08-09 3:00 ` Yang, Weijiang
2023-08-03 4:27 ` [PATCH v5 19/19] KVM:nVMX: Enable CET support for " Yang Weijiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c014fd8a-5456-478f-b643-44cfff27f931@intel.com \
--to=weijiang.yang@intel.com \
--cc=binbin.wu@linux.intel.com \
--cc=chao.gao@intel.com \
--cc=dave.hansen@intel.com \
--cc=john.allen@amd.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rick.p.edgecombe@intel.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).