From: Dave Hansen <dave.hansen@intel.com>
To: Kai Huang <kai.huang@intel.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: linux-mm@kvack.org, seanjc@google.com, pbonzini@redhat.com,
dan.j.williams@intel.com, rafael.j.wysocki@intel.com,
kirill.shutemov@linux.intel.com, ying.huang@intel.com,
reinette.chatre@intel.com, len.brown@intel.com,
tony.luck@intel.com, peterz@infradead.org, ak@linux.intel.com,
isaku.yamahata@intel.com, chao.gao@intel.com,
sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com,
sagis@google.com, imammedo@redhat.com
Subject: Re: [PATCH v7 18/20] x86/virt/tdx: Initialize all TDMRs
Date: Wed, 23 Nov 2022 16:42:17 -0800 [thread overview]
Message-ID: <4e19a7e1-9088-21d6-d521-a76c340d8179@intel.com> (raw)
In-Reply-To: <2337c8e9086a006aaa2c4b99caf478420d1fc640.1668988357.git.kai.huang@intel.com>
On 11/20/22 16:26, Kai Huang wrote:
> Initialize TDMRs via TDH.SYS.TDMR.INIT as the last step to complete the
> TDX initialization.
>
> All TDMRs need to be initialized using TDH.SYS.TDMR.INIT SEAMCALL before
> the memory pages can be used by the TDX module. The time to initialize
> TDMR is proportional to the size of the TDMR because TDH.SYS.TDMR.INIT
> internally initializes the PAMT entries using the global KeyID.
>
> To avoid long latency caused in one SEAMCALL, TDH.SYS.TDMR.INIT only
> initializes an (implementation-specific) subset of PAMT entries of one
> TDMR in one invocation. The caller needs to call TDH.SYS.TDMR.INIT
> iteratively until all PAMT entries of the given TDMR are initialized.
>
> TDH.SYS.TDMR.INITs can run concurrently on multiple CPUs as long as they
> are initializing different TDMRs. To keep it simple, just initialize
> all TDMRs one by one. On a 2-socket machine with 2.2G CPUs and 64GB
> memory, each TDH.SYS.TDMR.INIT roughly takes couple of microseconds on
> average, and it takes roughly dozens of milliseconds to complete the
> initialization of all TDMRs while system is idle.
Any chance you could say TDH.SYS.TDMR.INIT a few more times in there? ;)
Seriously, please try to trim that down. If you talk about the
implementation in detail in the code comments, don't cover it in detail
in the changelog too.
Also, this is a momentous patch, right? It's the last piece. Maybe
worth calling that out.
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index 99d1be5941a7..9bcdb30b7a80 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -1066,6 +1066,65 @@ static int config_global_keyid(void)
> return seamcall_on_each_package_serialized(&sc);
> }
>
> +/* Initialize one TDMR */
Does this comment add value? Even if it does, it is better than naming
the dang function init_one_tdmr()?
> +static int init_tdmr(struct tdmr_info *tdmr)
> +{
> + u64 next;
> +
> + /*
> + * Initializing PAMT entries might be time-consuming (in
> + * proportion to the size of the requested TDMR). To avoid long
> + * latency in one SEAMCALL, TDH.SYS.TDMR.INIT only initializes
> + * an (implementation-defined) subset of PAMT entries in one
> + * invocation.
> + *
> + * Call TDH.SYS.TDMR.INIT iteratively until all PAMT entries
> + * of the requested TDMR are initialized (if next-to-initialize
> + * address matches the end address of the TDMR).
> + */
The PAMT discussion here is, IMNHO silly. If this were about
initializing PAMT's then it should be renamed init_pamts() and the
SEAMCALL should be called PAMT_INIT or soemthing. It's not, and the ABI
is built around the TDMR and *its* addresses.
Let's chop this comment down:
/*
* Initializing a TDMR can be time consuming. To avoid long
* SEAMCALLs, the TDX module may only initialize a part of the
* TDMR in each call.
*/
Talk about the looping logic in the loop.
> + do {
> + struct tdx_module_output out;
> + int ret;
> +
> + ret = seamcall(TDH_SYS_TDMR_INIT, tdmr->base, 0, 0, 0, NULL,
> + &out);
> + if (ret)
> + return ret;
> + /*
> + * RDX contains 'next-to-initialize' address if
> + * TDH.SYS.TDMR.INT succeeded.
> + */
> + next = out.rdx;
> + /* Allow scheduling when needed */
That comment is a wee bit superfluous, don't you think?
> + cond_resched();
/* Keep making SEAMCALLs until the TDMR is done */
> + } while (next < tdmr->base + tdmr->size);
> +
> + return 0;
> +}
> +
> +/* Initialize all TDMRs */
Does this comment add value?
> +static int init_tdmrs(struct tdmr_info *tdmr_array, int tdmr_num)
> +{
> + int i;
> +
> + /*
> + * Initialize TDMRs one-by-one for simplicity, though the TDX
> + * architecture does allow different TDMRs to be initialized in
> + * parallel on multiple CPUs. Parallel initialization could
> + * be added later when the time spent in the serialized scheme
> + * becomes a real concern.
> + */
Trim down the comment:
/*
* This operation is costly. It can be parallelized,
* but keep it simple for now.
*/
> + for (i = 0; i < tdmr_num; i++) {
> + int ret;
> +
> + ret = init_tdmr(tdmr_array_entry(tdmr_array, i));
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> /*
> * Detect and initialize the TDX module.
> *
> @@ -1172,11 +1231,11 @@ static int init_tdx_module(void)
> if (ret)
> goto out_free_pamts;
>
> - /*
> - * Return -EINVAL until all steps of TDX module initialization
> - * process are done.
> - */
> - ret = -EINVAL;
> + /* Initialize TDMRs to complete the TDX module initialization */
> + ret = init_tdmrs(tdmr_array, tdmr_num);
> + if (ret)
> + goto out_free_pamts;
> +
> out_free_pamts:
> if (ret) {
> /*
> diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h
> index 768d097412ab..891691b1ea50 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.h
> +++ b/arch/x86/virt/vmx/tdx/tdx.h
> @@ -19,6 +19,7 @@
> #define TDH_SYS_INFO 32
> #define TDH_SYS_INIT 33
> #define TDH_SYS_LP_INIT 35
> +#define TDH_SYS_TDMR_INIT 36
> #define TDH_SYS_LP_SHUTDOWN 44
> #define TDH_SYS_CONFIG 45
>
next prev parent reply other threads:[~2022-11-24 0:42 UTC|newest]
Thread overview: 163+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-21 0:26 [PATCH v7 00/20] TDX host kernel support Kai Huang
2022-11-21 0:26 ` [PATCH v7 01/20] x86/tdx: Define TDX supported page sizes as macros Kai Huang
2022-11-21 2:52 ` Sathyanarayanan Kuppuswamy
2022-11-21 9:15 ` Huang, Kai
2022-11-21 17:23 ` Sathyanarayanan Kuppuswamy
2022-11-21 18:12 ` Dave Hansen
2022-11-21 23:48 ` Dave Hansen
2022-11-22 0:01 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 02/20] x86/virt/tdx: Detect TDX during kernel boot Kai Huang
2022-11-21 3:07 ` Sathyanarayanan Kuppuswamy
2022-11-21 9:37 ` Huang, Kai
2022-11-21 23:57 ` Sathyanarayanan Kuppuswamy
2022-11-22 0:10 ` Dave Hansen
2022-11-22 11:28 ` Huang, Kai
2022-11-22 16:50 ` Dave Hansen
2022-11-22 23:21 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 03/20] x86/virt/tdx: Disable TDX if X2APIC is not enabled Kai Huang
2022-11-21 3:51 ` Sathyanarayanan Kuppuswamy
2022-11-21 9:44 ` Huang, Kai
2022-11-21 22:00 ` Sathyanarayanan Kuppuswamy
2022-11-21 23:40 ` Huang, Kai
2022-11-21 23:46 ` Dave Hansen
2022-11-22 0:30 ` Huang, Kai
2022-11-22 0:44 ` Dave Hansen
2022-11-22 0:58 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 04/20] x86/virt/tdx: Add skeleton to initialize TDX on demand Kai Huang
2022-11-22 9:02 ` Peter Zijlstra
2022-11-22 10:31 ` Thomas Gleixner
2022-11-22 15:35 ` Dave Hansen
2022-11-22 20:03 ` Thomas Gleixner
2022-11-22 20:11 ` Sean Christopherson
2022-11-23 0:30 ` Huang, Kai
2022-11-23 1:12 ` Huang, Kai
2022-11-23 11:05 ` Thomas Gleixner
2022-11-23 12:22 ` Huang, Kai
2022-11-22 18:05 ` Dave Hansen
2022-11-23 10:18 ` Huang, Kai
2022-11-23 16:58 ` Dave Hansen
2022-11-23 21:58 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 05/20] x86/virt/tdx: Implement functions to make SEAMCALL Kai Huang
2022-11-22 9:06 ` Peter Zijlstra
2022-11-23 8:53 ` Huang, Kai
2022-11-22 18:20 ` Dave Hansen
2022-11-23 10:43 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 06/20] x86/virt/tdx: Shut down TDX module in case of error Kai Huang
2022-11-22 9:10 ` Peter Zijlstra
2022-11-22 9:13 ` Peter Zijlstra
2022-11-22 15:14 ` Dave Hansen
2022-11-22 19:13 ` Peter Zijlstra
2022-11-22 19:24 ` Dave Hansen
2022-11-22 19:33 ` Peter Zijlstra
2022-11-23 1:14 ` Huang, Kai
2022-11-29 21:40 ` Dave Hansen
2022-11-30 11:09 ` Thomas Gleixner
2022-11-23 0:58 ` Huang, Kai
2022-11-23 1:04 ` Dave Hansen
2022-11-23 1:22 ` Huang, Kai
2022-11-23 16:20 ` Sean Christopherson
2022-11-23 16:41 ` Dave Hansen
2022-11-23 17:37 ` Sean Christopherson
2022-11-23 18:18 ` Dave Hansen
2022-11-23 19:03 ` Sean Christopherson
2022-11-22 9:20 ` Peter Zijlstra
2022-11-22 15:06 ` Thomas Gleixner
2022-11-22 19:06 ` Peter Zijlstra
2022-11-22 19:31 ` Sean Christopherson
2022-11-23 9:39 ` Huang, Kai
2022-11-22 15:20 ` Dave Hansen
2022-11-22 16:52 ` Thomas Gleixner
2022-11-22 18:57 ` Dave Hansen
2022-11-22 19:14 ` Peter Zijlstra
2022-11-23 1:24 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 07/20] x86/virt/tdx: Do TDX module global initialization Kai Huang
2022-11-22 19:14 ` Dave Hansen
2022-11-23 11:45 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 08/20] x86/virt/tdx: Do logical-cpu scope TDX module initialization Kai Huang
2022-11-21 0:26 ` [PATCH v7 09/20] x86/virt/tdx: Get information about TDX module and TDX-capable memory Kai Huang
2022-11-22 23:39 ` Dave Hansen
2022-11-23 11:40 ` Huang, Kai
2022-11-23 16:44 ` Dave Hansen
2022-11-23 22:53 ` Huang, Kai
2022-12-02 11:19 ` Huang, Kai
2022-12-02 17:25 ` Dave Hansen
2022-12-02 21:57 ` Huang, Kai
2022-12-02 11:11 ` Huang, Kai
2022-12-02 17:06 ` Dave Hansen
2022-12-02 21:56 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 10/20] x86/virt/tdx: Use all system memory when initializing TDX module as TDX memory Kai Huang
2022-11-21 5:37 ` Huang, Ying
2022-11-21 9:09 ` Huang, Kai
2022-11-22 1:54 ` Huang, Ying
2022-11-22 9:16 ` Huang, Kai
2022-11-24 0:47 ` Huang, Ying
2022-11-22 10:10 ` Peter Zijlstra
2022-11-22 11:40 ` Huang, Kai
2022-11-23 0:21 ` Dave Hansen
2022-11-23 9:29 ` Peter Zijlstra
2022-11-24 1:04 ` Huang, Kai
2022-11-24 1:22 ` Dave Hansen
2022-11-24 2:27 ` Huang, Kai
2022-11-24 1:50 ` Dan Williams
2022-11-24 9:06 ` Huang, Kai
2022-11-25 9:28 ` David Hildenbrand
2022-11-28 8:38 ` Huang, Kai
2022-11-28 8:43 ` David Hildenbrand
2022-11-28 9:21 ` Huang, Kai
2022-11-28 9:26 ` David Hildenbrand
2022-11-28 9:50 ` Huang, Kai
2022-11-24 9:26 ` Peter Zijlstra
2022-11-24 10:02 ` Huang, Kai
2022-11-30 22:26 ` Dave Hansen
2022-11-21 0:26 ` [PATCH v7 11/20] x86/virt/tdx: Add placeholder to construct TDMRs to cover all TDX memory regions Kai Huang
2022-11-23 22:17 ` Dave Hansen
2022-11-24 9:51 ` Huang, Kai
2022-11-24 12:02 ` Huang, Kai
2022-11-28 15:59 ` Dave Hansen
2022-11-28 22:13 ` Huang, Kai
2022-11-28 22:19 ` Dave Hansen
2022-11-28 22:50 ` Huang, Kai
2022-12-07 11:47 ` Huang, Kai
2022-12-08 12:56 ` Huang, Kai
2022-12-08 14:58 ` Dave Hansen
2022-12-08 23:29 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 12/20] x86/virt/tdx: Create " Kai Huang
2022-11-23 22:41 ` Dave Hansen
2022-11-24 11:29 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 13/20] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2022-11-23 22:57 ` Dave Hansen
2022-11-24 11:46 ` Huang, Kai
2022-11-28 16:39 ` Dave Hansen
2022-11-28 22:48 ` Huang, Kai
2022-11-28 22:56 ` Dave Hansen
2022-11-28 23:14 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 14/20] x86/virt/tdx: Set up reserved areas for all TDMRs Kai Huang
2022-11-23 23:39 ` Dave Hansen
2022-11-28 9:14 ` Huang, Kai
2022-11-28 13:18 ` Dave Hansen
2022-11-28 22:24 ` Huang, Kai
2022-11-28 22:58 ` Dave Hansen
2022-11-28 23:10 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 15/20] x86/virt/tdx: Reserve TDX module global KeyID Kai Huang
2022-11-23 23:40 ` Dave Hansen
2022-11-24 22:39 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 16/20] x86/virt/tdx: Configure TDX module with TDMRs and " Kai Huang
2022-11-23 23:56 ` Dave Hansen
2022-11-25 0:59 ` Huang, Kai
2022-11-25 1:18 ` Dave Hansen
2022-11-25 1:44 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 17/20] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2022-11-24 0:28 ` Dave Hansen
2022-11-24 22:28 ` Huang, Kai
2022-11-25 0:08 ` Huang, Kai
2022-11-30 3:35 ` Binbin Wu
2022-11-30 8:34 ` Huang, Kai
2022-11-30 14:04 ` kirill.shutemov
2022-11-30 15:13 ` Dave Hansen
2022-11-30 20:17 ` Huang, Kai
2022-11-30 17:37 ` Dave Hansen
2022-11-21 0:26 ` [PATCH v7 18/20] x86/virt/tdx: Initialize all TDMRs Kai Huang
2022-11-24 0:42 ` Dave Hansen [this message]
2022-11-25 2:27 ` Huang, Kai
2022-11-21 0:26 ` [PATCH v7 19/20] x86/virt/tdx: Flush cache in kexec() when TDX is enabled Kai Huang
2022-11-21 0:26 ` [PATCH v7 20/20] Documentation/x86: Add documentation for TDX host support Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4e19a7e1-9088-21d6-d521-a76c340d8179@intel.com \
--to=dave.hansen@intel.com \
--cc=ak@linux.intel.com \
--cc=bagasdotme@gmail.com \
--cc=chao.gao@intel.com \
--cc=dan.j.williams@intel.com \
--cc=imammedo@redhat.com \
--cc=isaku.yamahata@intel.com \
--cc=kai.huang@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=reinette.chatre@intel.com \
--cc=sagis@google.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=tony.luck@intel.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).