From: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com>
To: Kai Huang <kai.huang@intel.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com,
len.brown@intel.com, tony.luck@intel.com,
rafael.j.wysocki@intel.com, reinette.chatre@intel.com,
dan.j.williams@intel.com, peterz@infradead.org,
ak@linux.intel.com, kirill.shutemov@linux.intel.com,
isaku.yamahata@intel.com
Subject: Re: [PATCH v3 06/21] x86/virt/tdx: Shut down TDX module in case of error
Date: Mon, 25 Apr 2022 18:48:48 -0700 [thread overview]
Message-ID: <c3619404-e1d7-6745-0ecc-a759d57d60bf@linux.intel.com> (raw)
In-Reply-To: <e14da6ed4520bc2362322434b1b4b336f079f3b7.camel@intel.com>
On 4/25/22 4:41 PM, Kai Huang wrote:
> On Sat, 2022-04-23 at 08:39 -0700, Sathyanarayanan Kuppuswamy wrote:
>>
>> On 4/5/22 9:49 PM, Kai Huang wrote:
>>> TDX supports shutting down the TDX module at any time during its
>>> lifetime. After TDX module is shut down, no further SEAMCALL can be
>>> made on any logical cpu.
>>>
>>> Shut down the TDX module in case of any error happened during the
>>> initialization process. It's pointless to leave the TDX module in some
>>> middle state.
>>>
>>> Shutting down the TDX module requires calling TDH.SYS.LP.SHUTDOWN on all
>>
>> May be adding specification reference will help.
>
> How about adding the reference to the code comment? Here we just need some fact
> description. Adding reference to the code comment also allows people to find
> the relative part in the spec easily when they are looking at the actual code
> (i.e. after the code is merged to upstream). Otherwise people needs to do a git
> blame and find the exact commit message for that.
If it is not a hassle, you can add references both in code and at the
end of the commit log. Adding two more lines to the commit log should
not be difficult.
I think it is fine either way. Your choice.
>
>>
>>> BIOS-enabled cpus, and the SEMACALL can run concurrently on different
>>> cpus. Implement a mechanism to run SEAMCALL concurrently on all online
>>
>> From TDX Module spec, sec 13.4.1 titled "Shutdown Initiated by the Host
>> VMM (as Part of Module Update)",
>>
>> TDH.SYS.LP.SHUTDOWN is designed to set state variables to block all
>> SEAMCALLs on the current LP and all SEAMCALL leaf functions except
>> TDH.SYS.LP.SHUTDOWN on the other LPs.
>>
>> As per above spec reference, executing TDH.SYS.LP.SHUTDOWN in
>> one LP prevent all SEAMCALL leaf function on all other LPs. If so,
>> why execute it on all CPUs?
>
> Prevent all SEAMCALLs on other LPs except TDH.SYS.LP.SHUTDOWN. The spec defnies
> shutting down the TDX module as running this SEAMCALl on all LPs, so why just
> run on a single cpu? What's the benefit?
If executing it in one LP prevents SEAMCALLs on all other LPs, I am
trying to understand why spec recommends running it in all LPs?
But the following explanation answers my query. I recommend making a
note about it in commit log or comments.
>
> Also, the spec also mentions for runtime update, "SEAMLDR can check that
> TDH.SYS.SHUTDOWN has been executed on all LPs". Runtime update isn't supported
> in this series, but it can leverage the existing code if we run SEAMCALL on all
> LPs to shutdown the module as spec suggested. Why just run on a single cpu?
>
>>
>>> cpus. Logical-cpu scope initialization will use it too.
>>
>> Concurrent SEAMCALL support seem to be useful for other SEAMCALL
>> types as well. If you agree, I think it would be better if you move
>> it out to a separate common patch.
>
> There are couple of problems of doing that:
>
> - All the functions are static in this tdx.c. Introducing them separately in
> dedicated patch would result in compile warning about those static functions are
> not used.
> - I have received comments from others I can add those functions when they are
> firstly used. Given those functions is not large, so I prefer this way too.
Ok
>
>>
>>>
>>> Signed-off-by: Kai Huang <kai.huang@intel.com>
>>> ---
>>> arch/x86/virt/vmx/tdx/tdx.c | 40 ++++++++++++++++++++++++++++++++++++-
>>> arch/x86/virt/vmx/tdx/tdx.h | 5 +++++
>>> 2 files changed, 44 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
>>> index 674867bccc14..faf8355965a5 100644
>>> --- a/arch/x86/virt/vmx/tdx/tdx.c
>>> +++ b/arch/x86/virt/vmx/tdx/tdx.c
>>> @@ -11,6 +11,8 @@
>>> #include <linux/cpumask.h>
>>> #include <linux/mutex.h>
>>> #include <linux/cpu.h>
>>> +#include <linux/smp.h>
>>> +#include <linux/atomic.h>
>>> #include <asm/msr-index.h>
>>> #include <asm/msr.h>
>>> #include <asm/cpufeature.h>
>>> @@ -328,6 +330,39 @@ static int seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
>>> return 0;
>>> }
>>>
>>> +/* Data structure to make SEAMCALL on multiple CPUs concurrently */
>>> +struct seamcall_ctx {
>>> + u64 fn;
>>> + u64 rcx;
>>> + u64 rdx;
>>> + u64 r8;
>>> + u64 r9;
>>> + atomic_t err;
>>> + u64 seamcall_ret;
>>> + struct tdx_module_output out;
>>> +};
>>> +
>>> +static void seamcall_smp_call_function(void *data)
>>> +{
>>> + struct seamcall_ctx *sc = data;
>>> + int ret;
>>> +
>>> + ret = seamcall(sc->fn, sc->rcx, sc->rdx, sc->r8, sc->r9,
>>> + &sc->seamcall_ret, &sc->out);
>>> + if (ret)
>>> + atomic_set(&sc->err, ret);
>>> +}
>>> +
>>> +/*
>>> + * Call the SEAMCALL on all online cpus concurrently.
>>> + * Return error if SEAMCALL fails on any cpu.
>>> + */
>>> +static int seamcall_on_each_cpu(struct seamcall_ctx *sc)
>>> +{
>>> + on_each_cpu(seamcall_smp_call_function, sc, true);
>>> + return atomic_read(&sc->err);
>>> +}
>>> +
>>> static inline bool p_seamldr_ready(void)
>>> {
>>> return !!p_seamldr_info.p_seamldr_ready;
>>> @@ -437,7 +472,10 @@ static int init_tdx_module(void)
>>>
>>> static void shutdown_tdx_module(void)
>>> {
>>> - /* TODO: Shut down the TDX module */
>>> + struct seamcall_ctx sc = { .fn = TDH_SYS_LP_SHUTDOWN };
>>> +
>>> + seamcall_on_each_cpu(&sc);
>>
>> May be check the error and WARN_ON on failure?
>
> When SEAMCALL fails, the error code will be printed out actually (please see
> previous patch), so I thought there's no need to WARN_ON() here (and some other
> similar places). I am not sure the additional WARN_ON() will do any help?
OK. I missed that part.
>
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
next prev parent reply other threads:[~2022-04-26 1:48 UTC|newest]
Thread overview: 156+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-06 4:49 [PATCH v3 00/21] TDX host kernel support Kai Huang
2022-04-06 4:49 ` [PATCH v3 01/21] x86/virt/tdx: Detect SEAM Kai Huang
2022-04-18 22:29 ` Sathyanarayanan Kuppuswamy
2022-04-18 22:50 ` Sean Christopherson
2022-04-19 3:38 ` Kai Huang
2022-04-26 20:21 ` Dave Hansen
2022-04-26 23:12 ` Kai Huang
2022-04-26 23:28 ` Dave Hansen
2022-04-26 23:49 ` Kai Huang
2022-04-27 0:22 ` Sean Christopherson
2022-04-27 0:44 ` Kai Huang
2022-04-27 14:22 ` Dave Hansen
2022-04-27 22:39 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 02/21] x86/virt/tdx: Detect TDX private KeyIDs Kai Huang
2022-04-19 5:39 ` Sathyanarayanan Kuppuswamy
2022-04-19 9:41 ` Kai Huang
2022-04-19 5:42 ` Sathyanarayanan Kuppuswamy
2022-04-19 10:07 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 03/21] x86/virt/tdx: Implement the SEAMCALL base function Kai Huang
2022-04-19 14:07 ` Sathyanarayanan Kuppuswamy
2022-04-20 4:16 ` Kai Huang
2022-04-20 7:29 ` Sathyanarayanan Kuppuswamy
2022-04-20 10:39 ` Kai Huang
2022-04-26 20:37 ` Dave Hansen
2022-04-26 23:29 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 04/21] x86/virt/tdx: Add skeleton for detecting and initializing TDX on demand Kai Huang
2022-04-19 14:53 ` Sathyanarayanan Kuppuswamy
2022-04-20 4:37 ` Kai Huang
2022-04-20 5:21 ` Dave Hansen
2022-04-20 14:30 ` Sathyanarayanan Kuppuswamy
2022-04-20 22:35 ` Kai Huang
2022-04-26 20:53 ` Dave Hansen
2022-04-27 0:43 ` Kai Huang
2022-04-27 14:49 ` Dave Hansen
2022-04-28 0:00 ` Kai Huang
2022-04-28 14:27 ` Dave Hansen
2022-04-28 23:44 ` Kai Huang
2022-04-28 23:53 ` Dave Hansen
2022-04-29 0:11 ` Kai Huang
2022-04-29 0:26 ` Dave Hansen
2022-04-29 0:59 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 05/21] x86/virt/tdx: Detect P-SEAMLDR and TDX module Kai Huang
2022-04-26 20:56 ` Dave Hansen
2022-04-27 0:01 ` Kai Huang
2022-04-27 14:24 ` Dave Hansen
2022-04-27 21:30 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 06/21] x86/virt/tdx: Shut down TDX module in case of error Kai Huang
2022-04-23 15:39 ` Sathyanarayanan Kuppuswamy
2022-04-25 23:41 ` Kai Huang
2022-04-26 1:48 ` Sathyanarayanan Kuppuswamy [this message]
2022-04-26 2:12 ` Kai Huang
2022-04-26 20:59 ` Dave Hansen
2022-04-27 0:06 ` Kai Huang
2022-05-18 16:19 ` Sagi Shahar
2022-05-18 23:51 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 07/21] x86/virt/tdx: Do TDX module global initialization Kai Huang
2022-04-20 22:27 ` Sathyanarayanan Kuppuswamy
2022-04-20 22:37 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 08/21] x86/virt/tdx: Do logical-cpu scope TDX module initialization Kai Huang
2022-04-24 1:27 ` Sathyanarayanan Kuppuswamy
2022-04-25 23:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 09/21] x86/virt/tdx: Get information about TDX module and convertible memory Kai Huang
2022-04-25 2:58 ` Sathyanarayanan Kuppuswamy
2022-04-26 0:05 ` Kai Huang
2022-04-27 22:15 ` Dave Hansen
2022-04-28 0:15 ` Kai Huang
2022-04-28 14:06 ` Dave Hansen
2022-04-28 23:14 ` Kai Huang
2022-04-29 17:47 ` Dave Hansen
2022-05-02 5:04 ` Kai Huang
2022-05-25 4:47 ` Kai Huang
2022-05-25 4:57 ` Kai Huang
2022-05-25 16:00 ` Kai Huang
2022-05-18 22:30 ` Sagi Shahar
2022-05-18 23:56 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 10/21] x86/virt/tdx: Add placeholder to coveret all system RAM as TDX memory Kai Huang
2022-04-20 20:48 ` Isaku Yamahata
2022-04-20 22:38 ` Kai Huang
2022-04-27 22:24 ` Dave Hansen
2022-04-28 0:53 ` Kai Huang
2022-04-28 1:07 ` Dave Hansen
2022-04-28 1:35 ` Kai Huang
2022-04-28 3:40 ` Dave Hansen
2022-04-28 3:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 11/21] x86/virt/tdx: Choose to use " Kai Huang
2022-04-20 20:55 ` Isaku Yamahata
2022-04-20 22:39 ` Kai Huang
2022-04-28 15:54 ` Dave Hansen
2022-04-29 7:32 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 12/21] x86/virt/tdx: Create TDMRs to cover all system RAM Kai Huang
2022-04-28 16:22 ` Dave Hansen
2022-04-29 7:24 ` Kai Huang
2022-04-29 13:52 ` Dave Hansen
2022-04-06 4:49 ` [PATCH v3 13/21] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2022-04-28 17:12 ` Dave Hansen
2022-04-29 7:46 ` Kai Huang
2022-04-29 14:20 ` Dave Hansen
2022-04-29 14:30 ` Sean Christopherson
2022-04-29 17:46 ` Dave Hansen
2022-04-29 18:19 ` Sean Christopherson
2022-04-29 18:32 ` Dave Hansen
2022-05-02 5:59 ` Kai Huang
2022-05-02 14:17 ` Dave Hansen
2022-05-02 21:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 14/21] x86/virt/tdx: Set up reserved areas for all TDMRs Kai Huang
2022-04-06 4:49 ` [PATCH v3 15/21] x86/virt/tdx: Reserve TDX module global KeyID Kai Huang
2022-04-06 4:49 ` [PATCH v3 16/21] x86/virt/tdx: Configure TDX module with TDMRs and " Kai Huang
2022-04-06 4:49 ` [PATCH v3 17/21] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2022-04-06 4:49 ` [PATCH v3 18/21] x86/virt/tdx: Initialize all TDMRs Kai Huang
2022-04-06 4:49 ` [PATCH v3 19/21] x86: Flush cache of TDX private memory during kexec() Kai Huang
2022-04-06 4:49 ` [PATCH v3 20/21] x86/virt/tdx: Add kernel command line to opt-in TDX host support Kai Huang
2022-04-28 17:25 ` Dave Hansen
2022-04-06 4:49 ` [PATCH v3 21/21] Documentation/x86: Add documentation for " Kai Huang
2022-04-14 10:19 ` [PATCH v3 00/21] TDX host kernel support Kai Huang
2022-04-26 20:13 ` Dave Hansen
2022-04-27 1:15 ` Kai Huang
2022-04-27 21:59 ` Dave Hansen
2022-04-28 0:37 ` Kai Huang
2022-04-28 0:50 ` Dave Hansen
2022-04-28 0:58 ` Kai Huang
2022-04-29 1:40 ` Kai Huang
2022-04-29 3:04 ` Dan Williams
2022-04-29 5:35 ` Kai Huang
2022-05-03 23:59 ` Kai Huang
2022-05-04 0:25 ` Dave Hansen
2022-05-04 1:15 ` Kai Huang
2022-05-05 9:54 ` Kai Huang
2022-05-05 13:51 ` Dan Williams
2022-05-05 22:14 ` Kai Huang
2022-05-06 0:22 ` Dan Williams
2022-05-06 0:45 ` Kai Huang
2022-05-06 1:15 ` Dan Williams
2022-05-06 1:46 ` Kai Huang
2022-05-06 15:57 ` Dan Williams
2022-05-09 2:46 ` Kai Huang
2022-05-10 10:25 ` Kai Huang
2022-05-07 0:09 ` Mike Rapoport
2022-05-08 10:00 ` Kai Huang
2022-05-09 10:33 ` Mike Rapoport
2022-05-09 23:27 ` Kai Huang
2022-05-04 14:31 ` Dan Williams
2022-05-04 22:50 ` Kai Huang
2022-04-28 1:01 ` Dan Williams
2022-04-28 1:21 ` Kai Huang
2022-04-29 2:58 ` Dan Williams
2022-04-29 5:43 ` Kai Huang
2022-04-29 14:39 ` Dave Hansen
2022-04-29 15:18 ` Dan Williams
2022-04-29 17:18 ` Dave Hansen
2022-04-29 17:48 ` Dan Williams
2022-04-29 18:34 ` Dave Hansen
2022-04-29 18:47 ` Dan Williams
2022-04-29 19:20 ` Dave Hansen
2022-04-29 21:20 ` Dan Williams
2022-04-29 21:27 ` Dave Hansen
2022-05-02 10:18 ` Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c3619404-e1d7-6745-0ecc-a759d57d60bf@linux.intel.com \
--to=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=ak@linux.intel.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=isaku.yamahata@intel.com \
--cc=kai.huang@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=reinette.chatre@intel.com \
--cc=seanjc@google.com \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).