kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Kai Huang <kai.huang@intel.com>
To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: linux-mm@kvack.org, dave.hansen@intel.com,
	kirill.shutemov@linux.intel.com, tony.luck@intel.com,
	peterz@infradead.org, tglx@linutronix.de, seanjc@google.com,
	pbonzini@redhat.com, david@redhat.com, dan.j.williams@intel.com,
	rafael.j.wysocki@intel.com, ying.huang@intel.com,
	reinette.chatre@intel.com, len.brown@intel.com,
	ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com,
	sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com,
	sagis@google.com, imammedo@redhat.com, kai.huang@intel.com
Subject: [PATCH v11 11/20] x86/virt/tdx: Fill out TDMRs to cover all TDX memory regions
Date: Mon,  5 Jun 2023 02:27:24 +1200	[thread overview]
Message-ID: <927ec9871721d2a50f1aba7d1cf7c3be50e4f49b.1685887183.git.kai.huang@intel.com> (raw)
In-Reply-To: <cover.1685887183.git.kai.huang@intel.com>

Start to transit out the "multi-steps" to construct a list of "TD Memory
Regions" (TDMRs) to cover all TDX-usable memory regions.

The kernel configures TDX-usable memory regions by passing a list of
TDMRs "TD Memory Regions" (TDMRs) to the TDX module.  Each TDMR contains
the information of the base/size of a memory region, the base/size of the
associated Physical Address Metadata Table (PAMT) and a list of reserved
areas in the region.

Do the first step to fill out a number of TDMRs to cover all TDX memory
regions.  To keep it simple, always try to use one TDMR for each memory
region.  As the first step only set up the base/size for each TDMR.

Each TDMR must be 1G aligned and the size must be in 1G granularity.
This implies that one TDMR could cover multiple memory regions.  If a
memory region spans the 1GB boundary and the former part is already
covered by the previous TDMR, just use a new TDMR for the remaining
part.

TDX only supports a limited number of TDMRs.  Disable TDX if all TDMRs
are consumed but there is more memory region to cover.

There are fancier things that could be done like trying to merge
adjacent TDMRs.  This would allow more pathological memory layouts to be
supported.  But, current systems are not even close to exhausting the
existing TDMR resources in practice.  For now, keep it simple.

Signed-off-by: Kai Huang <kai.huang@intel.com>
---

v10 -> v11:
 - No update

v9 -> v10:
 - No change.

v8 -> v9:

 - Added the last paragraph in the changelog (Dave).
 - Removed unnecessary type cast in tdmr_entry() (Dave).


---
 arch/x86/virt/vmx/tdx/tdx.c | 94 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 93 insertions(+), 1 deletion(-)

diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 7a20c72361e7..fa9fa8bc581a 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -385,6 +385,93 @@ static void free_tdmr_list(struct tdmr_info_list *tdmr_list)
 			tdmr_list->max_tdmrs * tdmr_list->tdmr_sz);
 }
 
+/* Get the TDMR from the list at the given index. */
+static struct tdmr_info *tdmr_entry(struct tdmr_info_list *tdmr_list,
+				    int idx)
+{
+	int tdmr_info_offset = tdmr_list->tdmr_sz * idx;
+
+	return (void *)tdmr_list->tdmrs + tdmr_info_offset;
+}
+
+#define TDMR_ALIGNMENT		BIT_ULL(30)
+#define TDMR_PFN_ALIGNMENT	(TDMR_ALIGNMENT >> PAGE_SHIFT)
+#define TDMR_ALIGN_DOWN(_addr)	ALIGN_DOWN((_addr), TDMR_ALIGNMENT)
+#define TDMR_ALIGN_UP(_addr)	ALIGN((_addr), TDMR_ALIGNMENT)
+
+static inline u64 tdmr_end(struct tdmr_info *tdmr)
+{
+	return tdmr->base + tdmr->size;
+}
+
+/*
+ * Take the memory referenced in @tmb_list and populate the
+ * preallocated @tdmr_list, following all the special alignment
+ * and size rules for TDMR.
+ */
+static int fill_out_tdmrs(struct list_head *tmb_list,
+			  struct tdmr_info_list *tdmr_list)
+{
+	struct tdx_memblock *tmb;
+	int tdmr_idx = 0;
+
+	/*
+	 * Loop over TDX memory regions and fill out TDMRs to cover them.
+	 * To keep it simple, always try to use one TDMR to cover one
+	 * memory region.
+	 *
+	 * In practice TDX1.0 supports 64 TDMRs, which is big enough to
+	 * cover all memory regions in reality if the admin doesn't use
+	 * 'memmap' to create a bunch of discrete memory regions.  When
+	 * there's a real problem, enhancement can be done to merge TDMRs
+	 * to reduce the final number of TDMRs.
+	 */
+	list_for_each_entry(tmb, tmb_list, list) {
+		struct tdmr_info *tdmr = tdmr_entry(tdmr_list, tdmr_idx);
+		u64 start, end;
+
+		start = TDMR_ALIGN_DOWN(PFN_PHYS(tmb->start_pfn));
+		end   = TDMR_ALIGN_UP(PFN_PHYS(tmb->end_pfn));
+
+		/*
+		 * A valid size indicates the current TDMR has already
+		 * been filled out to cover the previous memory region(s).
+		 */
+		if (tdmr->size) {
+			/*
+			 * Loop to the next if the current memory region
+			 * has already been fully covered.
+			 */
+			if (end <= tdmr_end(tdmr))
+				continue;
+
+			/* Otherwise, skip the already covered part. */
+			if (start < tdmr_end(tdmr))
+				start = tdmr_end(tdmr);
+
+			/*
+			 * Create a new TDMR to cover the current memory
+			 * region, or the remaining part of it.
+			 */
+			tdmr_idx++;
+			if (tdmr_idx >= tdmr_list->max_tdmrs) {
+				pr_warn("initialization failed: TDMRs exhausted.\n");
+				return -ENOSPC;
+			}
+
+			tdmr = tdmr_entry(tdmr_list, tdmr_idx);
+		}
+
+		tdmr->base = start;
+		tdmr->size = end - start;
+	}
+
+	/* @tdmr_idx is always the index of last valid TDMR. */
+	tdmr_list->nr_consumed_tdmrs = tdmr_idx + 1;
+
+	return 0;
+}
+
 /*
  * Construct a list of TDMRs on the preallocated space in @tdmr_list
  * to cover all TDX memory regions in @tmb_list based on the TDX module
@@ -394,10 +481,15 @@ static int construct_tdmrs(struct list_head *tmb_list,
 			   struct tdmr_info_list *tdmr_list,
 			   struct tdsysinfo_struct *sysinfo)
 {
+	int ret;
+
+	ret = fill_out_tdmrs(tmb_list, tdmr_list);
+	if (ret)
+		return ret;
+
 	/*
 	 * TODO:
 	 *
-	 *  - Fill out TDMRs to cover all TDX memory regions.
 	 *  - Allocate and set up PAMTs for each TDMR.
 	 *  - Designate reserved areas for each TDMR.
 	 *
-- 
2.40.1


  parent reply	other threads:[~2023-06-04 14:30 UTC|newest]

Thread overview: 174+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-04 14:27 [PATCH v11 00/20] TDX host kernel support Kai Huang
2023-06-04 14:27 ` [PATCH v11 01/20] x86/tdx: Define TDX supported page sizes as macros Kai Huang
2023-06-04 14:27 ` [PATCH v11 02/20] x86/virt/tdx: Detect TDX during kernel boot Kai Huang
2023-06-06 14:00   ` Sathyanarayanan Kuppuswamy
2023-06-06 22:58     ` Huang, Kai
2023-06-06 23:44   ` Isaku Yamahata
2023-06-19 12:12   ` David Hildenbrand
2023-06-19 23:58     ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 03/20] x86/virt/tdx: Make INTEL_TDX_HOST depend on X86_X2APIC Kai Huang
2023-06-08  0:08   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 04/20] x86/cpu: Detect TDX partial write machine check erratum Kai Huang
2023-06-06 12:38   ` kirill.shutemov
2023-06-06 22:58     ` Huang, Kai
2023-06-07 15:06       ` kirill.shutemov
2023-06-07 14:15   ` Dave Hansen
2023-06-07 22:43     ` Huang, Kai
2023-06-19 11:37       ` Huang, Kai
2023-06-20 15:44         ` Dave Hansen
2023-06-20 23:11           ` Huang, Kai
2023-06-19 12:21   ` David Hildenbrand
2023-06-20 10:31     ` Huang, Kai
2023-06-20 15:39     ` Dave Hansen
2023-06-20 16:03       ` David Hildenbrand
2023-06-20 16:21         ` Dave Hansen
2023-06-04 14:27 ` [PATCH v11 05/20] x86/virt/tdx: Add SEAMCALL infrastructure Kai Huang
2023-06-06 23:55   ` Isaku Yamahata
2023-06-07 14:24   ` Dave Hansen
2023-06-07 18:53     ` Isaku Yamahata
2023-06-07 19:27       ` Dave Hansen
2023-06-07 19:47         ` Isaku Yamahata
2023-06-07 20:08           ` Sean Christopherson
2023-06-07 20:22             ` Dave Hansen
2023-06-08  0:51               ` Huang, Kai
2023-06-08 13:50                 ` Dave Hansen
2023-06-07 22:56     ` Huang, Kai
2023-06-08 14:05       ` Dave Hansen
2023-06-19 12:52   ` David Hildenbrand
2023-06-20 10:37     ` Huang, Kai
2023-06-20 12:20       ` kirill.shutemov
2023-06-20 12:39         ` David Hildenbrand
2023-06-20 15:15     ` Dave Hansen
2023-06-04 14:27 ` [PATCH v11 06/20] x86/virt/tdx: Handle SEAMCALL running out of entropy error Kai Huang
2023-06-07  8:19   ` Isaku Yamahata
2023-06-07 15:08   ` Dave Hansen
2023-06-07 23:36     ` Huang, Kai
2023-06-08  0:29       ` Dave Hansen
2023-06-08  0:08   ` kirill.shutemov
2023-06-09 14:42   ` Nikolay Borisov
2023-06-12 11:04     ` Huang, Kai
2023-06-19 13:00   ` David Hildenbrand
2023-06-20 10:39     ` Huang, Kai
2023-06-20 11:14       ` David Hildenbrand
2023-06-04 14:27 ` [PATCH v11 07/20] x86/virt/tdx: Add skeleton to enable TDX on demand Kai Huang
2023-06-05 21:23   ` Isaku Yamahata
2023-06-05 23:04     ` Huang, Kai
2023-06-05 23:08       ` Dave Hansen
2023-06-05 23:24         ` Huang, Kai
2023-06-07 15:22   ` Dave Hansen
2023-06-08  2:10     ` Huang, Kai
2023-06-08 13:43       ` Dave Hansen
2023-06-12 11:21         ` Huang, Kai
2023-06-19 13:16   ` David Hildenbrand
2023-06-19 23:28     ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 08/20] x86/virt/tdx: Get information about TDX module and TDX-capable memory Kai Huang
2023-06-07 15:25   ` Dave Hansen
2023-06-08  0:27   ` kirill.shutemov
2023-06-08  2:40     ` Huang, Kai
2023-06-08 11:41       ` kirill.shutemov
2023-06-08 13:13         ` Dave Hansen
2023-06-12  2:00           ` Huang, Kai
2023-06-08 23:29         ` Isaku Yamahata
2023-06-08 23:54           ` kirill.shutemov
2023-06-09  1:33             ` Isaku Yamahata
2023-06-09 10:02   ` kirill.shutemov
2023-06-12  2:00     ` Huang, Kai
2023-06-19 13:29   ` David Hildenbrand
2023-06-19 23:51     ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 09/20] x86/virt/tdx: Use all system memory when initializing TDX module as TDX memory Kai Huang
2023-06-07 15:48   ` Dave Hansen
2023-06-07 23:22     ` Huang, Kai
2023-06-08 22:40   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 10/20] x86/virt/tdx: Add placeholder to construct TDMRs to cover all TDX memory regions Kai Huang
2023-06-07 15:54   ` Dave Hansen
2023-06-07 15:57   ` Dave Hansen
2023-06-08 10:18     ` Huang, Kai
2023-06-08 22:52   ` kirill.shutemov
2023-06-12  2:21     ` Huang, Kai
2023-06-12  3:01       ` Dave Hansen
2023-06-04 14:27 ` Kai Huang [this message]
2023-06-07 16:05   ` [PATCH v11 11/20] x86/virt/tdx: Fill out " Dave Hansen
2023-06-08 10:48     ` Huang, Kai
2023-06-08 13:11       ` Dave Hansen
2023-06-12  2:33         ` Huang, Kai
2023-06-12 14:33           ` kirill.shutemov
2023-06-12 22:10             ` Huang, Kai
2023-06-13 10:18               ` kirill.shutemov
2023-06-13 23:19                 ` Huang, Kai
2023-06-08 23:02   ` kirill.shutemov
2023-06-12  2:25     ` Huang, Kai
2023-06-09  4:01   ` Sathyanarayanan Kuppuswamy
2023-06-12  2:28     ` Huang, Kai
2023-06-14 12:31   ` Nikolay Borisov
2023-06-14 22:45     ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 12/20] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2023-06-08 23:24   ` kirill.shutemov
2023-06-08 23:43     ` Dave Hansen
2023-06-12  2:52       ` Huang, Kai
2023-06-25 15:38     ` Huang, Kai
2023-06-15  7:48   ` Nikolay Borisov
2023-06-04 14:27 ` [PATCH v11 13/20] x86/virt/tdx: Designate reserved areas for all TDMRs Kai Huang
2023-06-08 23:53   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 14/20] x86/virt/tdx: Configure TDX module with the TDMRs and global KeyID Kai Huang
2023-06-08 23:53   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 15/20] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2023-06-08 23:53   ` kirill.shutemov
2023-06-15  8:12   ` Nikolay Borisov
2023-06-15 22:24     ` Huang, Kai
2023-06-19 14:56       ` kirill.shutemov
2023-06-19 23:38         ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 16/20] x86/virt/tdx: Initialize all TDMRs Kai Huang
2023-06-09 10:03   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 17/20] x86/kexec: Flush cache of TDX private memory Kai Huang
2023-06-09 10:14   ` kirill.shutemov
2023-06-04 14:27 ` [PATCH v11 18/20] x86: Handle TDX erratum to reset TDX private memory during kexec() and reboot Kai Huang
2023-06-09 13:23   ` kirill.shutemov
2023-06-12  3:06     ` Huang, Kai
2023-06-12  7:58       ` kirill.shutemov
2023-06-12 10:27         ` Huang, Kai
2023-06-12 11:48           ` kirill.shutemov
2023-06-12 13:18             ` David Laight
2023-06-12 13:47           ` Dave Hansen
2023-06-13  0:51             ` Huang, Kai
2023-06-13 11:05               ` kirill.shutemov
2023-06-14  0:15                 ` Huang, Kai
2023-06-13 14:25               ` Dave Hansen
2023-06-13 23:18                 ` Huang, Kai
2023-06-14  0:24                   ` Dave Hansen
2023-06-14  0:38                     ` Huang, Kai
2023-06-14  0:42                       ` Huang, Kai
2023-06-19 11:43             ` Huang, Kai
2023-06-19 14:31               ` Dave Hansen
2023-06-19 14:46                 ` kirill.shutemov
2023-06-19 23:35                   ` Huang, Kai
2023-06-19 23:41                   ` Dave Hansen
2023-06-20  0:56                     ` Huang, Kai
2023-06-20  1:06                       ` Dave Hansen
2023-06-20  7:58                         ` Peter Zijlstra
2023-06-25 15:30                         ` Huang, Kai
2023-06-25 23:26                           ` Huang, Kai
2023-06-20  7:48                     ` Peter Zijlstra
2023-06-20  8:11       ` Peter Zijlstra
2023-06-20 10:42         ` Huang, Kai
2023-06-20 10:56           ` Peter Zijlstra
2023-06-14  9:33   ` Huang, Kai
2023-06-14 10:02     ` kirill.shutemov
2023-06-14 10:58       ` Huang, Kai
2023-06-14 11:08         ` kirill.shutemov
2023-06-14 11:17           ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 19/20] x86/mce: Improve error log of kernel space TDX #MC due to erratum Kai Huang
2023-06-05  2:13   ` Xiaoyao Li
2023-06-05 23:05     ` Huang, Kai
2023-06-09 13:17   ` kirill.shutemov
2023-06-12  3:08     ` Huang, Kai
2023-06-12  7:59       ` kirill.shutemov
2023-06-12 13:51         ` Dave Hansen
2023-06-12 23:31           ` Huang, Kai
2023-06-04 14:27 ` [PATCH v11 20/20] Documentation/x86: Add documentation for TDX host support Kai Huang
2023-06-08 23:56   ` Dave Hansen
2023-06-12  3:41     ` Huang, Kai
2023-06-16  9:02   ` Nikolay Borisov
2023-06-16 16:26     ` Dave Hansen
2023-06-06  0:36 ` [PATCH v11 00/20] TDX host kernel support Isaku Yamahata
2023-06-08 21:03 ` Dan Williams
2023-06-12 10:56   ` Huang, Kai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=927ec9871721d2a50f1aba7d1cf7c3be50e4f49b.1685887183.git.kai.huang@intel.com \
    --to=kai.huang@intel.com \
    --cc=ak@linux.intel.com \
    --cc=bagasdotme@gmail.com \
    --cc=chao.gao@intel.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=david@redhat.com \
    --cc=imammedo@redhat.com \
    --cc=isaku.yamahata@intel.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=len.brown@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rafael.j.wysocki@intel.com \
    --cc=reinette.chatre@intel.com \
    --cc=sagis@google.com \
    --cc=sathyanarayanan.kuppuswamy@linux.intel.com \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --cc=ying.huang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).