From: Kai Huang <kai.huang@intel.com>
To: Dave Hansen <dave.hansen@intel.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: seanjc@google.com, pbonzini@redhat.com, len.brown@intel.com,
tony.luck@intel.com, rafael.j.wysocki@intel.com,
reinette.chatre@intel.com, dan.j.williams@intel.com,
peterz@infradead.org, ak@linux.intel.com,
kirill.shutemov@linux.intel.com,
sathyanarayanan.kuppuswamy@linux.intel.com,
isaku.yamahata@intel.com
Subject: Re: [PATCH v3 12/21] x86/virt/tdx: Create TDMRs to cover all system RAM
Date: Fri, 29 Apr 2022 19:24:40 +1200 [thread overview]
Message-ID: <695f319e637e7afb33f228a230566f0c671e3a03.camel@intel.com> (raw)
In-Reply-To: <fa4d15d5-4690-9e63-f0c9-af4b58e4325c@intel.com>
On Thu, 2022-04-28 at 09:22 -0700, Dave Hansen wrote:
> On 4/5/22 21:49, Kai Huang wrote:
> > The kernel configures TDX usable memory regions to the TDX module via
> > an array of "TD Memory Region" (TDMR).
>
> One bit of language that's repeated in these changelogs that I don't
> like is "configure ... to". I think that's a misuse of the word
> configure. I'd say something more like:
>
> The kernel configures TDX-usable memory regions by passing an
> array of "TD Memory Regions" (TDMRs) to the TDX module.
>
> Could you please take a look over this series and reword those?
Thanks will do.
>
> > Each TDMR entry (TDMR_INFO)
> > contains the information of the base/size of a memory region, the
> > base/size of the associated Physical Address Metadata Table (PAMT) and
> > a list of reserved areas in the region.
> >
> > Create a number of TDMRs according to the verified e820 RAM entries.
> > As the first step only set up the base/size information for each TDMR.
> >
> > TDMR must be 1G aligned and the size must be in 1G granularity. This
>
> ^ Each
OK.
>
> > implies that one TDMR could cover multiple e820 RAM entries. If a RAM
> > entry spans the 1GB boundary and the former part is already covered by
> > the previous TDMR, just create a new TDMR for the latter part.
> >
> > TDX only supports a limited number of TDMRs (currently 64). Abort the
> > TDMR construction process when the number of TDMRs exceeds this
> > limitation.
>
> ... and what does this *MEAN*? Is TDX disabled? Does it throw away the
> RAM? Does it eat puppies?
How about:
TDX only supports a limited number of TDMRs. Simply return error when
the number of TDMRs exceeds the limitation. TDX is disabled in this
case.
>
> > arch/x86/virt/vmx/tdx/tdx.c | 138 ++++++++++++++++++++++++++++++++++++
> > 1 file changed, 138 insertions(+)
> >
> > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> > index 6b0c51aaa7f2..82534e70df96 100644
> > --- a/arch/x86/virt/vmx/tdx/tdx.c
> > +++ b/arch/x86/virt/vmx/tdx/tdx.c
> > @@ -54,6 +54,18 @@
> > ((u32)(((_keyid_part) & 0xffffffffull) + 1))
> > #define TDX_KEYID_NUM(_keyid_part) ((u32)((_keyid_part) >> 32))
> >
> > +/* TDMR must be 1gb aligned */
> > +#define TDMR_ALIGNMENT BIT_ULL(30)
> > +#define TDMR_PFN_ALIGNMENT (TDMR_ALIGNMENT >> PAGE_SHIFT)
> > +
> > +/* Align up and down the address to TDMR boundary */
> > +#define TDMR_ALIGN_DOWN(_addr) ALIGN_DOWN((_addr), TDMR_ALIGNMENT)
> > +#define TDMR_ALIGN_UP(_addr) ALIGN((_addr), TDMR_ALIGNMENT)
> > +
> > +/* TDMR's start and end address */
> > +#define TDMR_START(_tdmr) ((_tdmr)->base)
> > +#define TDMR_END(_tdmr) ((_tdmr)->base + (_tdmr)->size)
>
> Make these 'static inline's please. #defines are only for constants or
> things that can't use real functions.
OK.
>
> > /*
> > * TDX module status during initialization
> > */
> > @@ -813,6 +825,44 @@ static int e820_check_against_cmrs(void)
> > return 0;
> > }
> >
> > +/* The starting offset of reserved areas within TDMR_INFO */
> > +#define TDMR_RSVD_START 64
>
> ^ extra whitespace
Will remove.
>
> > +static struct tdmr_info *__alloc_tdmr(void)
> > +{
> > + int tdmr_sz;
> > +
> > + /*
> > + * TDMR_INFO's actual size depends on maximum number of reserved
> > + * areas that one TDMR supports.
> > + */
> > + tdmr_sz = TDMR_RSVD_START + tdx_sysinfo.max_reserved_per_tdmr *
> > + sizeof(struct tdmr_reserved_area);
>
> You have a structure for this. I know this because it's the return type
> of the function. You have TDMR_RSVD_START available via the structure
> itself. So, derive that 64 either via:
>
> sizeof(struct tdmr_info)
>
> or,
>
> offsetof(struct tdmr_info, reserved_areas);
>
> Which would make things look like this:
>
> tdmr_base_sz = sizeof(struct tdmr_info);
> tdmr_reserved_area_sz = sizeof(struct tdmr_reserved_area) *
> tdx_sysinfo.max_reserved_per_tdmr;
>
> tdmr_sz = tdmr_base_sz + tdmr_reserved_area_sz;
>
> Could you explain why on earth you felt the need for the TDMR_RSVD_START
> #define?
Will use sizeof (struct tdmr_info). Thanks for the tip.
>
> > + /*
> > + * TDX requires TDMR_INFO to be 512 aligned. Always align up
>
> Again, 512 what? 512 pages? 512 hippos?
Will change to 512-byte aligned.
>
> > + * TDMR_INFO size to 512 so the memory allocated via kzalloc()
> > + * can meet the alignment requirement.
> > + */
> > + tdmr_sz = ALIGN(tdmr_sz, TDMR_INFO_ALIGNMENT);
> > +
> > + return kzalloc(tdmr_sz, GFP_KERNEL);
> > +}
> > +
> > +/* Create a new TDMR at given index in the TDMR array */
> > +static struct tdmr_info *alloc_tdmr(struct tdmr_info **tdmr_array, int idx)
> > +{
> > + struct tdmr_info *tdmr;
> > +
> > + if (WARN_ON_ONCE(tdmr_array[idx]))
> > + return NULL;
> > +
> > + tdmr = __alloc_tdmr();
> > + tdmr_array[idx] = tdmr;
> > +
> > + return tdmr;
> > +}
> > +
> > static void free_tdmrs(struct tdmr_info **tdmr_array, int tdmr_num)
> > {
> > int i;
> > @@ -826,6 +876,89 @@ static void free_tdmrs(struct tdmr_info **tdmr_array, int tdmr_num)
> > }
> > }
> >
> > +/*
> > + * Create TDMRs to cover all RAM entries in e820_table. The created
> > + * TDMRs are saved to @tdmr_array and @tdmr_num is set to the actual
> > + * number of TDMRs. All entries in @tdmr_array must be initially NULL.
> > + */
> > +static int create_tdmrs(struct tdmr_info **tdmr_array, int *tdmr_num)
> > +{
> > + struct tdmr_info *tdmr;
> > + u64 start, end;
> > + int i, tdmr_idx;
> > + int ret = 0;
> > +
> > + tdmr_idx = 0;
> > + tdmr = alloc_tdmr(tdmr_array, 0);
> > + if (!tdmr)
> > + return -ENOMEM;
> > + /*
> > + * Loop over all RAM entries in e820 and create TDMRs to cover
> > + * them. To keep it simple, always try to use one TDMR to cover
> > + * one RAM entry.
> > + */
> > + e820_for_each_mem(i, start, end) {
> > + start = TDMR_ALIGN_DOWN(start);
> > + end = TDMR_ALIGN_UP(end);
> ^ vertically align those ='s, please.
OK.
>
>
> > + /*
> > + * If the current TDMR's size hasn't been initialized, it
> > + * is a new allocated TDMR to cover the new RAM entry.
> > + * Otherwise the current TDMR already covers the previous
> > + * RAM entry. In the latter case, check whether the
> > + * current RAM entry has been fully or partially covered
> > + * by the current TDMR, since TDMR is 1G aligned.
> > + */
> > + if (tdmr->size) {
> > + /*
> > + * Loop to next RAM entry if the current entry
> > + * is already fully covered by the current TDMR.
> > + */
> > + if (end <= TDMR_END(tdmr))
> > + continue;
>
> This loop is actually pretty well commented and looks OK. The
> TDMR_END() construct even adds to readability. *BUT*, the
>
> > + /*
> > + * If part of current RAM entry has already been
> > + * covered by current TDMR, skip the already
> > + * covered part.
> > + */
> > + if (start < TDMR_END(tdmr))
> > + start = TDMR_END(tdmr);
> > +
> > + /*
> > + * Create a new TDMR to cover the current RAM
> > + * entry, or the remaining part of it.
> > + */
> > + tdmr_idx++;
> > + if (tdmr_idx >= tdx_sysinfo.max_tdmrs) {
> > + ret = -E2BIG;
> > + goto err;
> > + }
> > + tdmr = alloc_tdmr(tdmr_array, tdmr_idx);
> > + if (!tdmr) {
> > + ret = -ENOMEM;
> > + goto err;
> > + }
>
> This is a bit verbose for this loop. Why not just hide the 'max_tdmrs'
> inside the alloc_tdmr() function? That will make this loop smaller and
> easier to read.
Based on suggestion, I'll change to use alloc_pages_exact() to allocate those
TDMRs at once, so no need to allocate for each TDMR again here. I'll remove the
alloc_tdmr() but keep the max_tdmrs check here.
--
Thanks,
-Kai
next prev parent reply other threads:[~2022-04-29 7:24 UTC|newest]
Thread overview: 156+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-06 4:49 [PATCH v3 00/21] TDX host kernel support Kai Huang
2022-04-06 4:49 ` [PATCH v3 01/21] x86/virt/tdx: Detect SEAM Kai Huang
2022-04-18 22:29 ` Sathyanarayanan Kuppuswamy
2022-04-18 22:50 ` Sean Christopherson
2022-04-19 3:38 ` Kai Huang
2022-04-26 20:21 ` Dave Hansen
2022-04-26 23:12 ` Kai Huang
2022-04-26 23:28 ` Dave Hansen
2022-04-26 23:49 ` Kai Huang
2022-04-27 0:22 ` Sean Christopherson
2022-04-27 0:44 ` Kai Huang
2022-04-27 14:22 ` Dave Hansen
2022-04-27 22:39 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 02/21] x86/virt/tdx: Detect TDX private KeyIDs Kai Huang
2022-04-19 5:39 ` Sathyanarayanan Kuppuswamy
2022-04-19 9:41 ` Kai Huang
2022-04-19 5:42 ` Sathyanarayanan Kuppuswamy
2022-04-19 10:07 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 03/21] x86/virt/tdx: Implement the SEAMCALL base function Kai Huang
2022-04-19 14:07 ` Sathyanarayanan Kuppuswamy
2022-04-20 4:16 ` Kai Huang
2022-04-20 7:29 ` Sathyanarayanan Kuppuswamy
2022-04-20 10:39 ` Kai Huang
2022-04-26 20:37 ` Dave Hansen
2022-04-26 23:29 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 04/21] x86/virt/tdx: Add skeleton for detecting and initializing TDX on demand Kai Huang
2022-04-19 14:53 ` Sathyanarayanan Kuppuswamy
2022-04-20 4:37 ` Kai Huang
2022-04-20 5:21 ` Dave Hansen
2022-04-20 14:30 ` Sathyanarayanan Kuppuswamy
2022-04-20 22:35 ` Kai Huang
2022-04-26 20:53 ` Dave Hansen
2022-04-27 0:43 ` Kai Huang
2022-04-27 14:49 ` Dave Hansen
2022-04-28 0:00 ` Kai Huang
2022-04-28 14:27 ` Dave Hansen
2022-04-28 23:44 ` Kai Huang
2022-04-28 23:53 ` Dave Hansen
2022-04-29 0:11 ` Kai Huang
2022-04-29 0:26 ` Dave Hansen
2022-04-29 0:59 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 05/21] x86/virt/tdx: Detect P-SEAMLDR and TDX module Kai Huang
2022-04-26 20:56 ` Dave Hansen
2022-04-27 0:01 ` Kai Huang
2022-04-27 14:24 ` Dave Hansen
2022-04-27 21:30 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 06/21] x86/virt/tdx: Shut down TDX module in case of error Kai Huang
2022-04-23 15:39 ` Sathyanarayanan Kuppuswamy
2022-04-25 23:41 ` Kai Huang
2022-04-26 1:48 ` Sathyanarayanan Kuppuswamy
2022-04-26 2:12 ` Kai Huang
2022-04-26 20:59 ` Dave Hansen
2022-04-27 0:06 ` Kai Huang
2022-05-18 16:19 ` Sagi Shahar
2022-05-18 23:51 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 07/21] x86/virt/tdx: Do TDX module global initialization Kai Huang
2022-04-20 22:27 ` Sathyanarayanan Kuppuswamy
2022-04-20 22:37 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 08/21] x86/virt/tdx: Do logical-cpu scope TDX module initialization Kai Huang
2022-04-24 1:27 ` Sathyanarayanan Kuppuswamy
2022-04-25 23:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 09/21] x86/virt/tdx: Get information about TDX module and convertible memory Kai Huang
2022-04-25 2:58 ` Sathyanarayanan Kuppuswamy
2022-04-26 0:05 ` Kai Huang
2022-04-27 22:15 ` Dave Hansen
2022-04-28 0:15 ` Kai Huang
2022-04-28 14:06 ` Dave Hansen
2022-04-28 23:14 ` Kai Huang
2022-04-29 17:47 ` Dave Hansen
2022-05-02 5:04 ` Kai Huang
2022-05-25 4:47 ` Kai Huang
2022-05-25 4:57 ` Kai Huang
2022-05-25 16:00 ` Kai Huang
2022-05-18 22:30 ` Sagi Shahar
2022-05-18 23:56 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 10/21] x86/virt/tdx: Add placeholder to coveret all system RAM as TDX memory Kai Huang
2022-04-20 20:48 ` Isaku Yamahata
2022-04-20 22:38 ` Kai Huang
2022-04-27 22:24 ` Dave Hansen
2022-04-28 0:53 ` Kai Huang
2022-04-28 1:07 ` Dave Hansen
2022-04-28 1:35 ` Kai Huang
2022-04-28 3:40 ` Dave Hansen
2022-04-28 3:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 11/21] x86/virt/tdx: Choose to use " Kai Huang
2022-04-20 20:55 ` Isaku Yamahata
2022-04-20 22:39 ` Kai Huang
2022-04-28 15:54 ` Dave Hansen
2022-04-29 7:32 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 12/21] x86/virt/tdx: Create TDMRs to cover all system RAM Kai Huang
2022-04-28 16:22 ` Dave Hansen
2022-04-29 7:24 ` Kai Huang [this message]
2022-04-29 13:52 ` Dave Hansen
2022-04-06 4:49 ` [PATCH v3 13/21] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2022-04-28 17:12 ` Dave Hansen
2022-04-29 7:46 ` Kai Huang
2022-04-29 14:20 ` Dave Hansen
2022-04-29 14:30 ` Sean Christopherson
2022-04-29 17:46 ` Dave Hansen
2022-04-29 18:19 ` Sean Christopherson
2022-04-29 18:32 ` Dave Hansen
2022-05-02 5:59 ` Kai Huang
2022-05-02 14:17 ` Dave Hansen
2022-05-02 21:55 ` Kai Huang
2022-04-06 4:49 ` [PATCH v3 14/21] x86/virt/tdx: Set up reserved areas for all TDMRs Kai Huang
2022-04-06 4:49 ` [PATCH v3 15/21] x86/virt/tdx: Reserve TDX module global KeyID Kai Huang
2022-04-06 4:49 ` [PATCH v3 16/21] x86/virt/tdx: Configure TDX module with TDMRs and " Kai Huang
2022-04-06 4:49 ` [PATCH v3 17/21] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2022-04-06 4:49 ` [PATCH v3 18/21] x86/virt/tdx: Initialize all TDMRs Kai Huang
2022-04-06 4:49 ` [PATCH v3 19/21] x86: Flush cache of TDX private memory during kexec() Kai Huang
2022-04-06 4:49 ` [PATCH v3 20/21] x86/virt/tdx: Add kernel command line to opt-in TDX host support Kai Huang
2022-04-28 17:25 ` Dave Hansen
2022-04-06 4:49 ` [PATCH v3 21/21] Documentation/x86: Add documentation for " Kai Huang
2022-04-14 10:19 ` [PATCH v3 00/21] TDX host kernel support Kai Huang
2022-04-26 20:13 ` Dave Hansen
2022-04-27 1:15 ` Kai Huang
2022-04-27 21:59 ` Dave Hansen
2022-04-28 0:37 ` Kai Huang
2022-04-28 0:50 ` Dave Hansen
2022-04-28 0:58 ` Kai Huang
2022-04-29 1:40 ` Kai Huang
2022-04-29 3:04 ` Dan Williams
2022-04-29 5:35 ` Kai Huang
2022-05-03 23:59 ` Kai Huang
2022-05-04 0:25 ` Dave Hansen
2022-05-04 1:15 ` Kai Huang
2022-05-05 9:54 ` Kai Huang
2022-05-05 13:51 ` Dan Williams
2022-05-05 22:14 ` Kai Huang
2022-05-06 0:22 ` Dan Williams
2022-05-06 0:45 ` Kai Huang
2022-05-06 1:15 ` Dan Williams
2022-05-06 1:46 ` Kai Huang
2022-05-06 15:57 ` Dan Williams
2022-05-09 2:46 ` Kai Huang
2022-05-10 10:25 ` Kai Huang
2022-05-07 0:09 ` Mike Rapoport
2022-05-08 10:00 ` Kai Huang
2022-05-09 10:33 ` Mike Rapoport
2022-05-09 23:27 ` Kai Huang
2022-05-04 14:31 ` Dan Williams
2022-05-04 22:50 ` Kai Huang
2022-04-28 1:01 ` Dan Williams
2022-04-28 1:21 ` Kai Huang
2022-04-29 2:58 ` Dan Williams
2022-04-29 5:43 ` Kai Huang
2022-04-29 14:39 ` Dave Hansen
2022-04-29 15:18 ` Dan Williams
2022-04-29 17:18 ` Dave Hansen
2022-04-29 17:48 ` Dan Williams
2022-04-29 18:34 ` Dave Hansen
2022-04-29 18:47 ` Dan Williams
2022-04-29 19:20 ` Dave Hansen
2022-04-29 21:20 ` Dan Williams
2022-04-29 21:27 ` Dave Hansen
2022-05-02 10:18 ` Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=695f319e637e7afb33f228a230566f0c671e3a03.camel@intel.com \
--to=kai.huang@intel.com \
--cc=ak@linux.intel.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=isaku.yamahata@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=reinette.chatre@intel.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).