From: "Huang, Kai" <kai.huang@intel.com>
To: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"Hansen, Dave" <dave.hansen@intel.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc: "Luck, Tony" <tony.luck@intel.com>,
"bagasdotme@gmail.com" <bagasdotme@gmail.com>,
"ak@linux.intel.com" <ak@linux.intel.com>,
"Wysocki, Rafael J" <rafael.j.wysocki@intel.com>,
"kirill.shutemov@linux.intel.com"
<kirill.shutemov@linux.intel.com>, "Christopherson,,
Sean" <seanjc@google.com>,
"Chatre, Reinette" <reinette.chatre@intel.com>,
"pbonzini@redhat.com" <pbonzini@redhat.com>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"Yamahata, Isaku" <isaku.yamahata@intel.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"Shahar, Sagi" <sagis@google.com>,
"imammedo@redhat.com" <imammedo@redhat.com>,
"Gao, Chao" <chao.gao@intel.com>,
"Brown, Len" <len.brown@intel.com>,
"peterz@infradead.org" <peterz@infradead.org>,
"sathyanarayanan.kuppuswamy@linux.intel.com"
<sathyanarayanan.kuppuswamy@linux.intel.com>,
"Huang, Ying" <ying.huang@intel.com>,
"Williams, Dan J" <dan.j.williams@intel.com>
Subject: Re: [PATCH v8 07/16] x86/virt/tdx: Use all system memory when initializing TDX module as TDX memory
Date: Thu, 12 Jan 2023 11:33:06 +0000 [thread overview]
Message-ID: <be85aa450b221bb730013d3b6ec0d4e71b51c228.camel@intel.com> (raw)
In-Reply-To: <e02fd75d-e9d4-15f1-eb9c-31cf3cc9ddc1@intel.com>
On Mon, 2023-01-09 at 08:51 -0800, Dave Hansen wrote:
> > > > > > + list_for_each_entry(tmb, &tdx_memlist, list) {
> > > > > > + /*
> > > > > > + * The new range is TDX memory if it is fully
> > > > > > covered by
> > > > > > + * any TDX memory block.
> > > > > > + *
> > > > > > + * Note TDX memory blocks are originated from
> > > > > > memblock
> > > > > > + * memory regions, which can only be contiguous when
> > > > > > two
> > > > > > + * regions have different NUMA nodes or flags.
> > > > > > Therefore
> > > > > > + * the new range cannot cross multiple TDX memory
> > > > > > blocks.
> > > > > > + */
> > > > > > + if (start_pfn >= tmb->start_pfn && end_pfn <= tmb-
> > > > > > >end_pfn)
> > > > > > + return true;
> > > > > > + }
> > > > > > + return false;
> > > > > > +}
> > > >
> > > > I don't really like that comment. It should first state its behavior
> > > > and assumptions, like:
> > > >
> > > > This check assumes that the start_pfn<->end_pfn range does not
> > > > cross multiple tdx_memlist entries.
> > > >
> > > > Only then should it describe why that is OK:
> > > >
> > > > A single memory hotplug even across mutliple memblocks (from
> > > > which tdx_memlist entries are derived) is impossible. ... then
> > > > actually explain
> > > >
> >
> > How about below?
> >
> > /*
> > * This check assumes that the start_pfn<->end_pfn range does not
> > cross
> > * multiple tdx_memlist entries. A single memory hotplug event
> > across
> > * multiple memblocks (from which tdx_memlist entries are derived)
> > is
> > * impossible. That means start_pfn<->end_pfn range cannot exceed a
> > * tdx_memlist entry, and the new range is TDX memory if it is
> > fully
> > * covered by any tdx_memlist entry.
> > */
>
> I was hoping you would actually explain why it is impossible.
>
> Is there something fundamental that keeps a memory area that spans two
> nodes from being removed and then a new area added that is comprised of
> a single node?
>
> Boot time:
>
> | memblock | memblock |
> <--Node=0--> <--Node=1-->
>
> Funky hotplug... nothing to see here, then:
>
> <--------Node=2-------->
>
> I would believe that there is no current bare-metal TDX system that has
> an implementation like this. But, the comments above speak like it's
> fundamentally impossible. That should be clarified.
>
> In other words, that comment talks about memblock attributes as being
> the core underlying reason that that simplified check is OK. Is that
> it, or is it really the reduced hotplug feature set on TDX systems?
Hi Dave,
I think I have been forgetting that we have switched to reject non-TDX memory in
memory online, but not in memory hot-add.
Memory offline/online is done on granularity of 'struct memory_block', but not
memblock. In fact, the hotpluggable memory region (one memblock) must be
multiple of memory_block, and a "to-be-online" memory_block must be full range
memory (no memory hole).
So if I am not missing something, IIUC that means if the start_pfn<->end_pfn is
TDX memory, it must be fully within some @tdx_memlist entry, but cannot cross
multiple small entries. And the memory hotplug case in your above diagram
actually shouldn't matter.
If above stands, how about below?
/*
* This check assumes that the start_pfn<->end_pfn range does not
* cross multiple @tdx_memlist entries. A single memory online
* event across multiple @tdx_memlist entries (which are derived
* from memblocks at the time of module initialization) is not
* possible.
*
* This is because memory offline/online is done on granularity
* of 'struct memory_block', and the hotpluggable memory region
* (one memblock) must be multiple of memory_block. Also, the
* "to-be-online" memory_block must be full memory (no memory
* hole, i.e. containing multiple small memblocks).
*
* This means if the start_pfn<->end_pfn range is TDX memory, it
* must be fully within one @tdx_memlist entry, but cannot cross
* multiple small entries.
*/
list_for_each_entry(tmb, &tdx_memlist, list) {
if (start_pfn >= tmb->start_pfn && end_pfn <= tmb->end_pfn)
return true;
}
next prev parent reply other threads:[~2023-01-12 11:43 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-09 6:52 [PATCH v8 00/16] TDX host kernel support Kai Huang
2022-12-09 6:52 ` [PATCH v8 01/16] x86/tdx: Define TDX supported page sizes as macros Kai Huang
2023-01-06 19:04 ` Dave Hansen
2022-12-09 6:52 ` [PATCH v8 02/16] x86/virt/tdx: Detect TDX during kernel boot Kai Huang
2023-01-06 17:09 ` Dave Hansen
2023-01-08 22:25 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 03/16] x86/virt/tdx: Make INTEL_TDX_HOST depend on X86_X2APIC Kai Huang
2023-01-06 19:04 ` Dave Hansen
2022-12-09 6:52 ` [PATCH v8 04/16] x86/virt/tdx: Add skeleton to initialize TDX on demand Kai Huang
2023-01-06 17:14 ` Dave Hansen
2023-01-08 22:26 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 05/16] x86/virt/tdx: Implement functions to make SEAMCALL Kai Huang
2023-01-06 17:29 ` Dave Hansen
2023-01-09 10:30 ` Huang, Kai
2023-01-09 19:54 ` Dave Hansen
2023-01-09 22:10 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 06/16] x86/virt/tdx: Get information about TDX module and TDX-capable memory Kai Huang
2023-01-06 17:46 ` Dave Hansen
2023-01-09 10:25 ` Huang, Kai
2023-01-09 19:52 ` Dave Hansen
2023-01-09 22:07 ` Huang, Kai
2023-01-09 22:11 ` Dave Hansen
2022-12-09 6:52 ` [PATCH v8 07/16] x86/virt/tdx: Use all system memory when initializing TDX module as TDX memory Kai Huang
2023-01-06 18:18 ` Dave Hansen
2023-01-09 11:48 ` Huang, Kai
2023-01-09 16:51 ` Dave Hansen
2023-01-10 12:09 ` Huang, Kai
2023-01-10 16:18 ` Dave Hansen
2023-01-11 10:00 ` Huang, Kai
2023-01-12 0:56 ` Huang, Ying
2023-01-12 1:18 ` Huang, Kai
2023-01-12 1:59 ` Huang, Ying
2023-01-12 2:22 ` Huang, Kai
2023-01-12 11:33 ` Huang, Kai [this message]
2023-01-18 11:08 ` Huang, Kai
2023-01-18 13:57 ` David Hildenbrand
2023-01-18 19:38 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 08/16] x86/virt/tdx: Add placeholder to construct TDMRs to cover all TDX memory regions Kai Huang
2023-01-06 19:24 ` Dave Hansen
2023-01-10 0:40 ` Huang, Kai
2023-01-10 0:47 ` Dave Hansen
2023-01-10 2:23 ` Huang, Kai
2023-01-10 19:12 ` Dave Hansen
2023-01-11 9:23 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 09/16] x86/virt/tdx: Fill out " Kai Huang
2023-01-06 19:36 ` Dave Hansen
2023-01-10 0:45 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 10/16] x86/virt/tdx: Allocate and set up PAMTs for TDMRs Kai Huang
2023-01-06 21:53 ` Dave Hansen
2023-01-10 0:49 ` Huang, Kai
2023-01-07 0:47 ` Dave Hansen
2023-01-10 0:47 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 11/16] x86/virt/tdx: Designate reserved areas for all TDMRs Kai Huang
2023-01-06 22:07 ` Dave Hansen
2023-01-10 1:19 ` Huang, Kai
2023-01-10 1:22 ` Dave Hansen
2023-01-10 11:01 ` Huang, Kai
2023-01-10 15:19 ` Dave Hansen
2023-01-11 10:57 ` Huang, Kai
2023-01-11 16:16 ` Dave Hansen
2023-01-11 22:10 ` Huang, Kai
2023-01-10 11:01 ` Huang, Kai
2023-01-10 15:17 ` Dave Hansen
2022-12-09 6:52 ` [PATCH v8 12/16] x86/virt/tdx: Designate the global KeyID and configure the TDX module Kai Huang
2023-01-06 22:21 ` Dave Hansen
2023-01-10 10:48 ` Huang, Kai
2023-01-10 16:25 ` Dave Hansen
2023-01-10 23:33 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 13/16] x86/virt/tdx: Configure global KeyID on all packages Kai Huang
2023-01-06 22:49 ` Dave Hansen
2023-01-10 10:15 ` Huang, Kai
2023-01-10 16:53 ` Dave Hansen
2023-01-11 0:06 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 14/16] x86/virt/tdx: Initialize all TDMRs Kai Huang
2023-01-07 0:17 ` Dave Hansen
2023-01-10 10:23 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 15/16] x86/virt/tdx: Flush cache in kexec() when TDX is enabled Kai Huang
2023-01-07 0:35 ` Dave Hansen
2023-01-10 11:29 ` Huang, Kai
2023-01-10 15:27 ` Dave Hansen
2023-01-11 0:13 ` Huang, Kai
2023-01-11 0:30 ` Dave Hansen
2023-01-11 1:58 ` Huang, Kai
2022-12-09 6:52 ` [PATCH v8 16/16] Documentation/x86: Add documentation for TDX host support Kai Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=be85aa450b221bb730013d3b6ec0d4e71b51c228.camel@intel.com \
--to=kai.huang@intel.com \
--cc=ak@linux.intel.com \
--cc=bagasdotme@gmail.com \
--cc=chao.gao@intel.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=imammedo@redhat.com \
--cc=isaku.yamahata@intel.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm@vger.kernel.org \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=reinette.chatre@intel.com \
--cc=sagis@google.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).