From: Dave Hansen <dave.hansen@intel.com>
To: Kai Huang <kai.huang@intel.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: seanjc@google.com, pbonzini@redhat.com, len.brown@intel.com,
tony.luck@intel.com, rafael.j.wysocki@intel.com,
reinette.chatre@intel.com, dan.j.williams@intel.com,
peterz@infradead.org, ak@linux.intel.com,
kirill.shutemov@linux.intel.com,
sathyanarayanan.kuppuswamy@linux.intel.com,
isaku.yamahata@intel.com
Subject: Re: [PATCH v5 15/22] x86/virt/tdx: Allocate and set up PAMTs for TDMRs
Date: Fri, 24 Jun 2022 13:13:18 -0700 [thread overview]
Message-ID: <e72703b0-767a-ec88-7cb6-f95a3564d823@intel.com> (raw)
In-Reply-To: <c504a8acd06dc455050c25e2a4cc70aef5eb9358.1655894131.git.kai.huang@intel.com>
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 4988a91d5283..ec496e96d120 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -1973,6 +1973,7 @@ config INTEL_TDX_HOST
> depends on CPU_SUP_INTEL
> depends on X86_64
> depends on KVM_INTEL
> + depends on CONTIG_ALLOC
> select ARCH_HAS_CC_PLATFORM
> select ARCH_KEEP_MEMBLOCK
> help
> diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
> index fd9f449b5395..36260dd7e69f 100644
> --- a/arch/x86/virt/vmx/tdx/tdx.c
> +++ b/arch/x86/virt/vmx/tdx/tdx.c
> @@ -558,6 +558,196 @@ static int create_tdmrs(struct tdmr_info *tdmr_array, int *tdmr_num)
> return 0;
> }
>
> +/* Page sizes supported by TDX */
> +enum tdx_page_sz {
> + TDX_PG_4K,
> + TDX_PG_2M,
> + TDX_PG_1G,
> + TDX_PG_MAX,
> +};
Are these the same constants as the magic numbers in Kirill's
try_accept_one()?
> +/*
> + * Calculate PAMT size given a TDMR and a page size. The returned
> + * PAMT size is always aligned up to 4K page boundary.
> + */
> +static unsigned long tdmr_get_pamt_sz(struct tdmr_info *tdmr,
> + enum tdx_page_sz pgsz)
> +{
> + unsigned long pamt_sz;
> + int pamt_entry_nr;
'nr_pamt_entries', please.
> + switch (pgsz) {
> + case TDX_PG_4K:
> + pamt_entry_nr = tdmr->size >> PAGE_SHIFT;
> + break;
> + case TDX_PG_2M:
> + pamt_entry_nr = tdmr->size >> PMD_SHIFT;
> + break;
> + case TDX_PG_1G:
> + pamt_entry_nr = tdmr->size >> PUD_SHIFT;
> + break;
> + default:
> + WARN_ON_ONCE(1);
> + return 0;
> + }
> +
> + pamt_sz = pamt_entry_nr * tdx_sysinfo.pamt_entry_size;
> + /* TDX requires PAMT size must be 4K aligned */
> + pamt_sz = ALIGN(pamt_sz, PAGE_SIZE);
> +
> + return pamt_sz;
> +}
> +
> +/*
> + * Pick a NUMA node on which to allocate this TDMR's metadata.
> + *
> + * This is imprecise since TDMRs are 1G aligned and NUMA nodes might
> + * not be. If the TDMR covers more than one node, just use the _first_
> + * one. This can lead to small areas of off-node metadata for some
> + * memory.
> + */
> +static int tdmr_get_nid(struct tdmr_info *tdmr)
> +{
> + unsigned long start_pfn, end_pfn;
> + int i, nid;
> +
> + /* Find the first memory region covered by the TDMR */
> + memblock_for_each_tdx_mem_pfn_range(i, &start_pfn, &end_pfn, &nid) {
> + if (end_pfn > (tdmr_start(tdmr) >> PAGE_SHIFT))
> + return nid;
> + }
> +
> + /*
> + * No memory region found for this TDMR. It cannot happen since
> + * when one TDMR is created, it must cover at least one (or
> + * partial) memory region.
> + */
> + WARN_ON_ONCE(1);
> + return 0;
> +}
You should really describe what you are doing. At first glance "return
0;" looks like "declare success". How about something like this?
/*
* Fall back to allocating the TDMR from node 0 when no memblock
* can be found. This should never happen since TDMRs originate
* from the memblocks.
*/
Does that miss any of the points you were trying to make?
> +static int tdmr_set_up_pamt(struct tdmr_info *tdmr)
> +{
> + unsigned long pamt_base[TDX_PG_MAX];
> + unsigned long pamt_size[TDX_PG_MAX];
> + unsigned long tdmr_pamt_base;
> + unsigned long tdmr_pamt_size;
> + enum tdx_page_sz pgsz;
> + struct page *pamt;
> + int nid;
> +
> + nid = tdmr_get_nid(tdmr);
> +
> + /*
> + * Calculate the PAMT size for each TDX supported page size
> + * and the total PAMT size.
> + */
> + tdmr_pamt_size = 0;
> + for (pgsz = TDX_PG_4K; pgsz < TDX_PG_MAX; pgsz++) {
> + pamt_size[pgsz] = tdmr_get_pamt_sz(tdmr, pgsz);
> + tdmr_pamt_size += pamt_size[pgsz];
> + }
> +
> + /*
> + * Allocate one chunk of physically contiguous memory for all
> + * PAMTs. This helps minimize the PAMT's use of reserved areas
> + * in overlapped TDMRs.
> + */
> + pamt = alloc_contig_pages(tdmr_pamt_size >> PAGE_SHIFT, GFP_KERNEL,
> + nid, &node_online_map);
> + if (!pamt)
> + return -ENOMEM;
I'm not sure it's worth mentioning, but this doesn't really need to be
GFP_KERNEL. __GFP_HIGHMEM would actually be just fine. But,
considering that this is 64-bit only, that's just a technicality.
> + /* Calculate PAMT base and size for all supported page sizes. */
That comment isn't doing much good. If you say anything here it should be:
/*
* Break the contiguous allocation back up into
* the individual PAMTs for each page size:
*/
Also, this is *not* "calculating size". That's done above.
> + tdmr_pamt_base = page_to_pfn(pamt) << PAGE_SHIFT;
> + for (pgsz = TDX_PG_4K; pgsz < TDX_PG_MAX; pgsz++) {
> + pamt_base[pgsz] = tdmr_pamt_base;
> + tdmr_pamt_base += pamt_size[pgsz];
> + }
> +
> + tdmr->pamt_4k_base = pamt_base[TDX_PG_4K];
> + tdmr->pamt_4k_size = pamt_size[TDX_PG_4K];
> + tdmr->pamt_2m_base = pamt_base[TDX_PG_2M];
> + tdmr->pamt_2m_size = pamt_size[TDX_PG_2M];
> + tdmr->pamt_1g_base = pamt_base[TDX_PG_1G];
> + tdmr->pamt_1g_size = pamt_size[TDX_PG_1G];
> +
> + return 0;
> +}
>
> +static void tdmr_get_pamt(struct tdmr_info *tdmr, unsigned long *pamt_pfn,
> + unsigned long *pamt_npages)
> +{
> + unsigned long pamt_base, pamt_sz;
> +
> + /*
> + * The PAMT was allocated in one contiguous unit. The 4K PAMT
> + * should always point to the beginning of that allocation.
> + */
> + pamt_base = tdmr->pamt_4k_base;
> + pamt_sz = tdmr->pamt_4k_size + tdmr->pamt_2m_size + tdmr->pamt_1g_size;
> +
> + *pamt_pfn = pamt_base >> PAGE_SHIFT;
> + *pamt_npages = pamt_sz >> PAGE_SHIFT;
> +}
> +
> +static void tdmr_free_pamt(struct tdmr_info *tdmr)
> +{
> + unsigned long pamt_pfn, pamt_npages;
> +
> + tdmr_get_pamt(tdmr, &pamt_pfn, &pamt_npages);
> +
> + /* Do nothing if PAMT hasn't been allocated for this TDMR */
> + if (!pamt_npages)
> + return;
> +
> + if (WARN_ON_ONCE(!pamt_pfn))
> + return;
> +
> + free_contig_range(pamt_pfn, pamt_npages);
> +}
> +
> +static void tdmrs_free_pamt_all(struct tdmr_info *tdmr_array, int tdmr_num)
> +{
> + int i;
> +
> + for (i = 0; i < tdmr_num; i++)
> + tdmr_free_pamt(tdmr_array_entry(tdmr_array, i));
> +}
> +
> +/* Allocate and set up PAMTs for all TDMRs */
> +static int tdmrs_set_up_pamt_all(struct tdmr_info *tdmr_array, int tdmr_num)
> +{
> + int i, ret = 0;
> +
> + for (i = 0; i < tdmr_num; i++) {
> + ret = tdmr_set_up_pamt(tdmr_array_entry(tdmr_array, i));
> + if (ret)
> + goto err;
> + }
> +
> + return 0;
> +err:
> + tdmrs_free_pamt_all(tdmr_array, tdmr_num);
> + return ret;
> +}
> +
> +static unsigned long tdmrs_get_pamt_pages(struct tdmr_info *tdmr_array,
> + int tdmr_num)
"get" is for refcounting. tdmrs_count_pamt_pages() would be preferable.
> +{
> + unsigned long pamt_npages = 0;
> + int i;
> +
> + for (i = 0; i < tdmr_num; i++) {
> + unsigned long pfn, npages;
> +
> + tdmr_get_pamt(tdmr_array_entry(tdmr_array, i), &pfn, &npages);
> + pamt_npages += npages;
> + }
> +
> + return pamt_npages;
> +}
> +
> /*
> * Construct an array of TDMRs to cover all memory regions in memblock.
> * This makes sure all pages managed by the page allocator are TDX
> @@ -572,8 +762,13 @@ static int construct_tdmrs_memeblock(struct tdmr_info *tdmr_array,
> if (ret)
> goto err;
>
> + ret = tdmrs_set_up_pamt_all(tdmr_array, *tdmr_num);
> + if (ret)
> + goto err;
> +
> /* Return -EINVAL until constructing TDMRs is done */
> ret = -EINVAL;
> + tdmrs_free_pamt_all(tdmr_array, *tdmr_num);
> err:
> return ret;
> }
> @@ -644,6 +839,11 @@ static int init_tdx_module(void)
> * process are done.
> */
> ret = -EINVAL;
> + if (ret)
> + tdmrs_free_pamt_all(tdmr_array, tdmr_num);
> + else
> + pr_info("%lu pages allocated for PAMT.\n",
> + tdmrs_get_pamt_pages(tdmr_array, tdmr_num));
> out_free_tdmrs:
> /*
> * The array of TDMRs is freed no matter the initialization is
The rest looks OK.
next prev parent reply other threads:[~2022-06-24 20:13 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-22 11:15 [PATCH v5 00/22] TDX host kernel support Kai Huang
2022-06-22 11:15 ` [PATCH v5 01/22] x86/virt/tdx: Detect TDX during kernel boot Kai Huang
2022-06-23 5:57 ` Chao Gao
2022-06-23 9:23 ` Kai Huang
2022-08-02 2:01 ` [PATCH v5 1/22] " Wu, Binbin
2022-08-03 9:25 ` Kai Huang
2022-06-22 11:15 ` [PATCH v5 02/22] cc_platform: Add new attribute to prevent ACPI CPU hotplug Kai Huang
2022-06-22 11:42 ` Rafael J. Wysocki
2022-06-23 0:01 ` Kai Huang
2022-06-27 8:01 ` Igor Mammedov
2022-06-28 10:04 ` Kai Huang
2022-06-28 11:52 ` Igor Mammedov
2022-06-28 17:33 ` Rafael J. Wysocki
2022-06-28 23:41 ` Kai Huang
2022-06-24 18:57 ` Dave Hansen
2022-06-27 5:05 ` Kai Huang
2022-07-13 11:09 ` Kai Huang
2022-07-19 17:46 ` Dave Hansen
2022-07-19 23:54 ` Kai Huang
2022-08-03 3:40 ` Binbin Wu
2022-08-03 9:20 ` Kai Huang
2022-06-29 5:33 ` Christoph Hellwig
2022-06-29 9:09 ` Kai Huang
2022-08-03 3:55 ` Binbin Wu
2022-08-03 9:21 ` Kai Huang
2022-06-22 11:15 ` [PATCH v5 03/22] cc_platform: Add new attribute to prevent ACPI memory hotplug Kai Huang
2022-06-22 11:45 ` Rafael J. Wysocki
2022-06-23 0:08 ` Kai Huang
2022-06-28 17:55 ` Rafael J. Wysocki
2022-06-28 12:01 ` Igor Mammedov
2022-06-28 23:49 ` Kai Huang
2022-06-29 8:48 ` Igor Mammedov
2022-06-29 9:13 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 04/22] x86/virt/tdx: Prevent ACPI CPU hotplug and " Kai Huang
2022-06-24 1:41 ` Chao Gao
2022-06-24 11:21 ` Kai Huang
2022-06-29 8:35 ` Yuan Yao
2022-06-29 9:17 ` Kai Huang
2022-06-29 14:22 ` Dave Hansen
2022-06-29 23:02 ` Kai Huang
2022-06-30 15:44 ` Dave Hansen
2022-06-30 22:45 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 05/22] x86/virt/tdx: Prevent hot-add driver managed memory Kai Huang
2022-06-24 2:12 ` Chao Gao
2022-06-24 11:23 ` Kai Huang
2022-06-24 19:01 ` Dave Hansen
2022-06-27 5:27 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 06/22] x86/virt/tdx: Add skeleton to initialize TDX on demand Kai Huang
2022-06-24 2:39 ` Chao Gao
2022-06-24 11:27 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 07/22] x86/virt/tdx: Implement SEAMCALL function Kai Huang
2022-06-24 18:38 ` Dave Hansen
2022-06-27 5:23 ` Kai Huang
2022-06-27 20:58 ` Dave Hansen
2022-06-27 22:10 ` Kai Huang
2022-07-19 19:39 ` Dan Williams
2022-07-19 23:28 ` Kai Huang
2022-07-20 10:18 ` Kai Huang
2022-07-20 16:48 ` Dave Hansen
2022-07-21 1:52 ` Kai Huang
2022-07-27 0:34 ` Kai Huang
2022-07-27 0:50 ` Dave Hansen
2022-07-27 12:46 ` Kai Huang
2022-08-03 2:37 ` Kai Huang
2022-08-03 14:20 ` Dave Hansen
2022-08-03 22:35 ` Kai Huang
2022-08-04 10:06 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 08/22] x86/virt/tdx: Shut down TDX module in case of error Kai Huang
2022-06-24 18:50 ` Dave Hansen
2022-06-27 5:26 ` Kai Huang
2022-06-27 20:46 ` Dave Hansen
2022-06-27 22:34 ` Kai Huang
2022-06-27 22:56 ` Dave Hansen
2022-06-27 23:59 ` Kai Huang
2022-06-28 0:03 ` Dave Hansen
2022-06-28 0:11 ` Kai Huang
2022-06-22 11:16 ` [PATCH v5 09/22] x86/virt/tdx: Detect TDX module by doing module global initialization Kai Huang
2022-06-22 11:16 ` [PATCH v5 10/22] x86/virt/tdx: Do logical-cpu scope TDX module initialization Kai Huang
2022-06-22 11:17 ` [PATCH v5 11/22] x86/virt/tdx: Get information about TDX module and TDX-capable memory Kai Huang
2022-06-22 11:17 ` [PATCH v5 12/22] x86/virt/tdx: Convert all memory regions in memblock to TDX memory Kai Huang
2022-06-24 19:40 ` Dave Hansen
2022-06-27 6:16 ` Kai Huang
2022-07-07 2:37 ` Kai Huang
2022-07-07 14:26 ` Dave Hansen
2022-07-07 14:36 ` Juergen Gross
2022-07-07 23:42 ` Kai Huang
2022-07-07 23:34 ` Kai Huang
2022-08-03 1:30 ` Kai Huang
2022-08-03 14:22 ` Dave Hansen
2022-08-03 22:14 ` Kai Huang
2022-06-22 11:17 ` [PATCH v5 13/22] x86/virt/tdx: Add placeholder to construct TDMRs based on memblock Kai Huang
2022-06-22 11:17 ` [PATCH v5 14/22] x86/virt/tdx: Create TDMRs to cover all memblock memory regions Kai Huang
2022-06-22 11:17 ` [PATCH v5 15/22] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2022-06-24 20:13 ` Dave Hansen [this message]
2022-06-27 10:31 ` Kai Huang
2022-06-27 20:41 ` Dave Hansen
2022-06-27 22:50 ` Kai Huang
2022-06-27 22:57 ` Dave Hansen
2022-06-27 23:05 ` Kai Huang
2022-06-28 0:48 ` Xiaoyao Li
2022-06-28 17:03 ` Dave Hansen
2022-08-17 22:46 ` Sagi Shahar
2022-08-17 23:43 ` Huang, Kai
2022-06-22 11:17 ` [PATCH v5 16/22] x86/virt/tdx: Set up reserved areas for all TDMRs Kai Huang
2022-06-22 11:17 ` [PATCH v5 17/22] x86/virt/tdx: Reserve TDX module global KeyID Kai Huang
2022-06-22 11:17 ` [PATCH v5 18/22] x86/virt/tdx: Configure TDX module with TDMRs and " Kai Huang
2022-06-22 11:17 ` [PATCH v5 19/22] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2022-06-22 11:17 ` [PATCH v5 20/22] x86/virt/tdx: Initialize all TDMRs Kai Huang
2022-06-22 11:17 ` [PATCH v5 21/22] x86/virt/tdx: Support kexec() Kai Huang
2022-06-22 11:17 ` [PATCH v5 22/22] Documentation/x86: Add documentation for TDX host support Kai Huang
2022-08-18 4:07 ` Bagas Sanjaya
2022-08-18 9:33 ` Huang, Kai
2022-06-24 19:47 ` [PATCH v5 00/22] TDX host kernel support Dave Hansen
2022-06-27 4:09 ` Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e72703b0-767a-ec88-7cb6-f95a3564d823@intel.com \
--to=dave.hansen@intel.com \
--cc=ak@linux.intel.com \
--cc=dan.j.williams@intel.com \
--cc=isaku.yamahata@intel.com \
--cc=kai.huang@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=reinette.chatre@intel.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).