From: "Kuppuswamy, Sathyanarayanan" <sathyanarayanan.kuppuswamy@linux.intel.com>
To: Dave Hansen <dave.hansen@intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Andy Lutomirski <luto@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>,
Kirill Shutemov <kirill.shutemov@linux.intel.com>,
Kuppuswamy Sathyanarayanan <knsathya@kernel.org>,
Dan Williams <dan.j.williams@intel.com>,
Raj Ashok <ashok.raj@intel.com>,
Sean Christopherson <seanjc@google.com>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 1/1] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Date: Wed, 31 Mar 2021 15:29:44 -0700 [thread overview]
Message-ID: <097b4cad-8f01-40cf-203e-1a9228450c80@linux.intel.com> (raw)
In-Reply-To: <7882ef34-416f-9627-dcbe-7bf88c866dc8@intel.com>
On 3/31/21 2:49 PM, Dave Hansen wrote:
> On 3/31/21 2:09 PM, Kuppuswamy Sathyanarayanan wrote:
>> As per Guest-Host Communication Interface (GHCI) Specification
>> for Intel TDX, sec 2.4.1, TDX architecture does not support
>> MWAIT, MONITOR and WBINVD instructions. So in non-root TDX mode,
>> if MWAIT/MONITOR instructions are executed with CPL != 0 it will
>> trigger #UD, and for CPL = 0 case, virtual exception (#VE) is
>> triggered. WBINVD instruction behavior is also similar to
>> MWAIT/MONITOR, but for CPL != 0 case, it will trigger #GP instead
>> of #UD.
>
> Could we give it a go to try this in plain English before jumping in and
> quoting the exact spec section? Also, the CPL language is nice and
> precise for talking inside Intel, but it's generally easier for me to
> read kernel descriptions when we just talk about the kernel.
>
> When running as a TDX guest, there are a number of existing,
> privileged instructions that do not work. If the guest kernel
> uses these instructions, the hardware generates a #VE.
I will fix it in next version.
>
> Which reminds me... The SDM says: MWAIT will "#UD ... If
> CPUID.01H:ECX.MONITOR[bit 3] = 0". So, is this an architectural change?
> The guest is *supposed* to see that CPUID bit as 0, so shouldn't it
> also get a #UD? Or is this all so that if SEAM *forgets* to clear the
> CPUID bit, the guest gets #VE?
AFAIK, we are only concerned about the case where the instruction support
is not disabled by SEAM. For disabled case, it should get #UD.
Sean, can you confirm it?
>
> What are we *actually* mitigating here?
we add support for #VE, when executing un-supported instruction in TD guest
kernel.
>
> Also, FWIW, MWAIT/MONITOR and WBINVD are pretty different beasts. I
> think this would all have been a lot more clear if this would have been
> two patches instead of shoehorning them into one.
Since all of them are unsupported instructions, I have grouped them
together. Even if we split it, there should be some duplication in commit
log (since handling is similar). But let me know if this is a desired
approach. I can split it in two patches.
>
>> To prevent TD guest from using these unsupported instructions,
>> following measures are adapted:
>>
>> 1. For MWAIT/MONITOR instructions, support for these instructions
>> are already disabled by TDX module (SEAM). So CPUID flags for
>> these instructions should be in disabled state. Also, just to be
>> sure that these instructions are disabled, forcefully unset
>> X86_FEATURE_MWAIT CPU cap in OS.
>>
>> 2. For WBINVD instruction, we use audit to find the code that uses
>> this instruction and disable them for TD.
>
> Really? Where are those patches?
For MWAIT/MONITOR, the change is included in the same patch.
For WBINVD, we have will have some patches included in next
series.
>
>> +static inline bool cpuid_has_mwait(void)
>> +{
>> + if (cpuid_ecx(1) & (1 << (X86_FEATURE_MWAIT % 32)))
>> + return true;
>> +
>> + return false;
>> +}
>> +
>> bool is_tdx_guest(void)
>> {
>> return static_cpu_has(X86_FEATURE_TDX_GUEST);
>> @@ -301,12 +309,25 @@ static int tdg_handle_mmio(struct pt_regs *regs, struct ve_info *ve)
>> return insn.length;
>> }
>>
>> +/* Initialize TDX specific CPU capabilities */
>> +static void __init tdx_cpu_cap_init(void)
>> +{
>> + setup_force_cpu_cap(X86_FEATURE_TDX_GUEST);
>> +
>> + if (cpuid_has_mwait()) {
>> + WARN(1, "TDX Module failed to disable MWAIT\n");
>
> WARN(1, "TDX guest enumerated support for MWAIT, disabling it").
will fix it in next version.
>
>> + /* MWAIT is not supported in TDX platform, so suppress it */
>> + setup_clear_cpu_cap(X86_FEATURE_MWAIT);
>> + }
>> +
>> +}
>
> Extra newline.
>
>> void __init tdx_early_init(void)
>> {
>> if (!cpuid_has_tdx_guest())
>> return;
>>
>> - setup_force_cpu_cap(X86_FEATURE_TDX_GUEST);
>> + tdx_cpu_cap_init();
>>
>> tdg_get_info();
>>
>> @@ -362,6 +383,27 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,
>> case EXIT_REASON_EPT_VIOLATION:
>> ve->instr_len = tdg_handle_mmio(regs, ve);
>> break;
>> + case EXIT_REASON_WBINVD:
>> + /*
>> + * TDX architecture does not support WBINVD instruction.
>> + * Currently, usage of this instruction is prevented by
>> + * disabling the drivers which uses it. So if we still
>> + * reach here, it needs user attention.
>> + */
>
> This comment is awfully vague. "TDX architecture..." what? Any CPUs
> supporting the TDX architecture? TDX VMM's? TDX Guests?
>
> Let's also not waste byte on stating the obvious. If it didn't need
> attention we wouldn't be warning about it, eh?
>
> So, let's halve the size of the comment and say:
>
> /*
> * WBINVD is not supported inside TDX guests. All in-
> * kernel uses should have been disabled.
> */
ok. will fix it next version.
>
>> + pr_err("TD Guest used unsupported WBINVD instruction\n");
>> + BUG();
>> + break;
>> + case EXIT_REASON_MONITOR_INSTRUCTION:
>> + case EXIT_REASON_MWAIT_INSTRUCTION:
>> + /*
>> + * MWAIT/MONITOR features are disabled by TDX Module (SEAM)
>> + * and also re-suppressed in kernel by clearing
>> + * X86_FEATURE_MWAIT CPU feature flag in tdx_early_init(). So
>> + * if TD guest still executes MWAIT/MONITOR instruction with
>> + * above suppression, it needs user attention.
>> + */
>
> Again, let's trim this down:
>
> /*
> * Something in the kernel used MONITOR or MWAIT despite
> * X86_FEATURE_MWAIT being cleared for TDX guests.
> */
will fix it next version.
>
> Rather than naming the function, this makes it quite greppable to find
> where it could have *possibly* been cleared.
>
>> + WARN(1, "TD Guest used unsupported MWAIT/MONITOR instruction\n");
I think WARN_ONCE is good enough for this exception. Do you agree?
>> + break;
>> default:
>> pr_warn("Unexpected #VE: %d\n", ve->exit_reason);
>> return -EFAULT;
>>
>
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
next prev parent reply other threads:[~2021-03-31 22:30 UTC|newest]
Thread overview: 161+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-06 3:02 Test Email sathyanarayanan.kuppuswamy
2021-02-05 23:38 ` [RFC v1 00/26] Add TDX Guest Support Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 01/26] x86/paravirt: Introduce CONFIG_PARAVIRT_XL Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 02/26] x86/cpufeatures: Add TDX Guest CPU feature Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 03/26] x86/cpufeatures: Add is_tdx_guest() interface Kuppuswamy Sathyanarayanan
2021-04-01 21:08 ` Dave Hansen
2021-04-01 21:15 ` Kuppuswamy, Sathyanarayanan
2021-04-01 21:19 ` Dave Hansen
2021-04-01 22:25 ` Kuppuswamy, Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 04/26] x86/tdx: Get TD execution environment information via TDINFO Kuppuswamy Sathyanarayanan
2021-02-08 10:00 ` Peter Zijlstra
2021-02-08 19:10 ` Kuppuswamy, Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 05/26] x86/traps: Add #VE support for TDX guest Kuppuswamy Sathyanarayanan
2021-02-08 10:20 ` Peter Zijlstra
2021-02-08 16:23 ` Andi Kleen
2021-02-08 16:33 ` Peter Zijlstra
2021-02-08 16:46 ` Sean Christopherson
2021-02-08 16:59 ` Peter Zijlstra
2021-02-08 19:05 ` Kuppuswamy, Sathyanarayanan
2021-02-08 16:46 ` Andi Kleen
2021-02-12 19:20 ` Dave Hansen
2021-02-12 19:47 ` Andy Lutomirski
2021-02-12 20:06 ` Sean Christopherson
2021-02-12 20:17 ` Dave Hansen
2021-02-12 20:37 ` Sean Christopherson
2021-02-12 20:46 ` Dave Hansen
2021-02-12 20:54 ` Sean Christopherson
2021-02-12 21:06 ` Dave Hansen
2021-02-12 21:37 ` Sean Christopherson
2021-02-12 21:47 ` Andy Lutomirski
2021-02-12 21:48 ` Dave Hansen
2021-02-14 19:33 ` Andi Kleen
2021-02-14 19:54 ` Andy Lutomirski
2021-02-12 20:20 ` Andy Lutomirski
2021-02-12 20:44 ` Sean Christopherson
2021-02-05 23:38 ` [RFC v1 06/26] x86/tdx: Add HLT " Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 07/26] x86/tdx: Wire up KVM hypercalls Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 08/26] x86/tdx: Add MSR support for TDX guest Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 09/26] x86/tdx: Handle CPUID via #VE Kuppuswamy Sathyanarayanan
2021-02-05 23:42 ` Andy Lutomirski
2021-02-07 14:13 ` Kirill A. Shutemov
2021-02-07 16:01 ` Dave Hansen
2021-02-07 20:29 ` Kirill A. Shutemov
2021-02-07 22:31 ` Dave Hansen
2021-02-07 22:45 ` Andy Lutomirski
2021-02-08 17:10 ` Sean Christopherson
2021-02-08 17:35 ` Andy Lutomirski
2021-02-08 17:47 ` Sean Christopherson
2021-03-18 21:30 ` [PATCH v1 1/1] x86/tdx: Add tdcall() and tdvmcall() helper functions Kuppuswamy Sathyanarayanan
2021-03-19 16:55 ` Sean Christopherson
2021-03-19 17:42 ` Kuppuswamy, Sathyanarayanan
2021-03-19 18:22 ` Dave Hansen
2021-03-19 19:58 ` Kuppuswamy, Sathyanarayanan
2021-03-26 23:38 ` [PATCH v2 1/1] x86/tdx: Add __tdcall() and __tdvmcall() " Kuppuswamy Sathyanarayanan
2021-04-20 17:36 ` Dave Hansen
2021-04-20 19:20 ` Kuppuswamy, Sathyanarayanan
2021-04-20 19:59 ` Dave Hansen
2021-04-20 23:12 ` Kuppuswamy, Sathyanarayanan
2021-04-20 23:42 ` Dave Hansen
2021-04-23 1:09 ` Kuppuswamy, Sathyanarayanan
2021-04-23 1:21 ` Dave Hansen
2021-04-23 1:35 ` Andi Kleen
2021-04-23 15:15 ` Sean Christopherson
2021-04-23 15:28 ` Dan Williams
2021-04-23 15:38 ` Andi Kleen
2021-04-23 15:50 ` Sean Christopherson
2021-04-23 15:47 ` Andi Kleen
2021-04-23 18:18 ` Kuppuswamy, Sathyanarayanan
2021-04-20 23:53 ` Dan Williams
2021-04-20 23:59 ` Kuppuswamy, Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 10/26] x86/io: Allow to override inX() and outX() implementation Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 11/26] x86/tdx: Handle port I/O Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 12/26] x86/tdx: Handle in-kernel MMIO Kuppuswamy Sathyanarayanan
2021-04-01 19:56 ` Dave Hansen
2021-04-01 22:26 ` Sean Christopherson
2021-04-01 22:53 ` Dave Hansen
2021-02-05 23:38 ` [RFC v1 13/26] x86/tdx: Handle MWAIT, MONITOR and WBINVD Kuppuswamy Sathyanarayanan
2021-02-05 23:43 ` Andy Lutomirski
2021-02-05 23:54 ` Kuppuswamy, Sathyanarayanan
2021-02-06 1:05 ` Andy Lutomirski
2021-03-27 0:18 ` [PATCH v1 1/1] " Kuppuswamy Sathyanarayanan
2021-03-27 2:40 ` Andy Lutomirski
2021-03-27 3:40 ` Kuppuswamy, Sathyanarayanan
2021-03-27 16:03 ` Andy Lutomirski
2021-03-27 22:54 ` [PATCH v2 " Kuppuswamy Sathyanarayanan
2021-03-29 17:14 ` Dave Hansen
2021-03-29 21:55 ` Kuppuswamy, Sathyanarayanan
2021-03-29 22:02 ` Dave Hansen
2021-03-29 22:09 ` Kuppuswamy, Sathyanarayanan
2021-03-29 22:12 ` Dave Hansen
2021-03-29 22:42 ` Kuppuswamy, Sathyanarayanan
2021-03-29 23:16 ` [PATCH v3 " Kuppuswamy Sathyanarayanan
2021-03-29 23:23 ` Andy Lutomirski
2021-03-29 23:37 ` Kuppuswamy, Sathyanarayanan
2021-03-29 23:42 ` Sean Christopherson
2021-03-29 23:58 ` Andy Lutomirski
2021-03-30 2:04 ` Andi Kleen
2021-03-30 2:58 ` Andy Lutomirski
2021-03-30 15:14 ` Sean Christopherson
2021-03-30 16:37 ` Andy Lutomirski
2021-03-30 16:57 ` Sean Christopherson
2021-04-07 15:24 ` Andi Kleen
2021-03-31 21:09 ` [PATCH v4 " Kuppuswamy Sathyanarayanan
2021-03-31 21:49 ` Dave Hansen
2021-03-31 22:29 ` Kuppuswamy, Sathyanarayanan [this message]
2021-03-31 21:53 ` Sean Christopherson
2021-03-31 22:00 ` Dave Hansen
2021-03-31 22:06 ` Sean Christopherson
2021-03-31 22:11 ` Dave Hansen
2021-03-31 22:28 ` Kuppuswamy, Sathyanarayanan
2021-03-31 22:32 ` Sean Christopherson
2021-03-31 22:34 ` Dave Hansen
2021-04-01 3:28 ` Andi Kleen
2021-04-01 3:46 ` Dave Hansen
2021-04-01 4:24 ` Andi Kleen
2021-04-01 4:51 ` [PATCH v5 " Kuppuswamy Sathyanarayanan
2021-03-29 23:39 ` [PATCH v3 " Sean Christopherson
2021-03-29 23:38 ` Dave Hansen
2021-03-30 4:56 ` [PATCH v1 " Xiaoyao Li
2021-03-30 15:00 ` Andi Kleen
2021-03-30 15:10 ` Dave Hansen
2021-03-30 17:02 ` Kuppuswamy, Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 14/26] ACPI: tables: Add multiprocessor wake-up support Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 15/26] x86/boot: Add a trampoline for APs booting in 64-bit mode Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 16/26] x86/boot: Avoid #VE during compressed boot for TDX platforms Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 17/26] x86/boot: Avoid unnecessary #VE during boot process Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 18/26] x86/topology: Disable CPU hotplug support for TDX platforms Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 19/26] x86/tdx: Forcefully disable legacy PIC for TDX guests Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 20/26] x86/tdx: Introduce INTEL_TDX_GUEST config option Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 21/26] x86/mm: Move force_dma_unencrypted() to common code Kuppuswamy Sathyanarayanan
2021-04-01 20:06 ` Dave Hansen
2021-04-06 15:37 ` Kirill A. Shutemov
2021-04-06 16:11 ` Dave Hansen
2021-04-06 16:37 ` Kirill A. Shutemov
2021-02-05 23:38 ` [RFC v1 22/26] x86/tdx: Exclude Shared bit from __PHYSICAL_MASK Kuppuswamy Sathyanarayanan
2021-04-01 20:13 ` Dave Hansen
2021-04-06 15:54 ` Kirill A. Shutemov
2021-04-06 16:12 ` Dave Hansen
2021-02-05 23:38 ` [RFC v1 23/26] x86/tdx: Make pages shared in ioremap() Kuppuswamy Sathyanarayanan
2021-04-01 20:26 ` Dave Hansen
2021-04-06 16:00 ` Kirill A. Shutemov
2021-04-06 16:14 ` Dave Hansen
2021-02-05 23:38 ` [RFC v1 24/26] x86/tdx: Add helper to do MapGPA TDVMALL Kuppuswamy Sathyanarayanan
2021-02-05 23:38 ` [RFC v1 25/26] x86/tdx: Make DMA pages shared Kuppuswamy Sathyanarayanan
2021-04-01 21:01 ` Dave Hansen
2021-04-06 16:31 ` Kirill A. Shutemov
2021-04-06 16:38 ` Dave Hansen
2021-04-06 17:16 ` Sean Christopherson
2021-02-05 23:38 ` [RFC v1 26/26] x86/kvm: Use bounce buffers for TD guest Kuppuswamy Sathyanarayanan
2021-04-01 21:17 ` Dave Hansen
2021-02-06 3:04 ` Test Email sathyanarayanan.kuppuswamy
2021-02-06 6:24 ` [RFC v1 00/26] Add TDX Guest Support sathyanarayanan.kuppuswamy
2021-03-31 21:38 ` Kuppuswamy, Sathyanarayanan
2021-04-02 0:02 ` Dave Hansen
2021-04-02 2:48 ` Andi Kleen
2021-04-02 15:27 ` Dave Hansen
2021-04-02 21:32 ` Andi Kleen
2021-04-03 16:26 ` Dave Hansen
2021-04-03 17:28 ` Andi Kleen
2021-04-04 15:02 ` Dave Hansen
2021-04-12 17:24 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=097b4cad-8f01-40cf-203e-1a9228450c80@linux.intel.com \
--to=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=ak@linux.intel.com \
--cc=ashok.raj@intel.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=knsathya@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=peterz@infradead.org \
--cc=seanjc@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).