Re: [PATCH v8 11/16] x86/virt/tdx: Designate reserved areas for all TDMRs

From: "Huang, Kai" <kai.huang@intel.com>
To: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Hansen, Dave" <dave.hansen@intel.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc: "Luck, Tony" <tony.luck@intel.com>,
	"bagasdotme@gmail.com" <bagasdotme@gmail.com>,
	"ak@linux.intel.com" <ak@linux.intel.com>,
	"Wysocki, Rafael J" <rafael.j.wysocki@intel.com>,
	"kirill.shutemov@linux.intel.com"
	<kirill.shutemov@linux.intel.com>, "Christopherson,,
	Sean" <seanjc@google.com>,
	"Chatre, Reinette" <reinette.chatre@intel.com>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"Yamahata, Isaku" <isaku.yamahata@intel.com>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"Shahar, Sagi" <sagis@google.com>,
	"imammedo@redhat.com" <imammedo@redhat.com>,
	"Gao, Chao" <chao.gao@intel.com>,
	"Brown, Len" <len.brown@intel.com>,
	"sathyanarayanan.kuppuswamy@linux.intel.com" 
	<sathyanarayanan.kuppuswamy@linux.intel.com>,
	"Huang, Ying" <ying.huang@intel.com>,
	"Williams, Dan J" <dan.j.williams@intel.com>
Subject: Re: [PATCH v8 11/16] x86/virt/tdx: Designate reserved areas for all TDMRs
Date: Tue, 10 Jan 2023 01:19:11 +0000	[thread overview]
Message-ID: <b971cd8b4e6ce9e96c6b4c6192adb74cc6722d54.camel@intel.com> (raw)
In-Reply-To: <2d7d2824-7aa7-5f96-d79b-b44ff7fe2ef9@intel.com>

On Fri, 2023-01-06 at 14:07 -0800, Dave Hansen wrote:
> On 12/8/22 22:52, Kai Huang wrote:
> > +static int tdmr_add_rsvd_area(struct tdmr_info *tdmr, int *p_idx, u64 addr,
> > +			      u64 size, u16 max_reserved_per_tdmr)
> > +{
> > +	struct tdmr_reserved_area *rsvd_areas = tdmr->reserved_areas;
> > +	int idx = *p_idx;
> > +
> > +	/* Reserved area must be 4K aligned in offset and size */
> > +	if (WARN_ON(addr & ~PAGE_MASK || size & ~PAGE_MASK))
> > +		return -EINVAL;
> > +
> > +	if (idx >= max_reserved_per_tdmr)
> > +		return -E2BIG;
> > +
> > +	rsvd_areas[idx].offset = addr - tdmr->base;
> > +	rsvd_areas[idx].size = size;
> > +
> > +	*p_idx = idx + 1;
> > +
> > +	return 0;
> > +}
> 
> It's probably worth at least a comment here to say:
> 
> 	/*
> 	 * Consume one reserved area per call.  Make no effort to
> 	 * optimize or reduce the number of reserved areas which are
> 	 * consumed by contiguous reserved areas, for instance.
> 	 */

I'll add this comment before the code to set up rsvd_areas[idx].

> 
> I think the -E2BIG is also wrong.  It should be ENOSPC.  I'd also add a
> pr_warn() there.  Especially with how lazy this whole thing is, I can
> start to see how the reserved areas might be exhausted.  Let's be kind
> to our future selves and make the error (and the fix) easier to find.

Yes agreed.  Will change to -ENOSPC and add pr_warn().

> It's probably also worth noting *somewhere* that there's a balance to be
> had between TDMRs and reserved areas.  A system that is running out of
> reserved areas in a TDMR could split a TDMR to get more reserved areas.
> A system that has run out of TDMRs could relatively easily coalesce two
> adjacent TDMRs (before the PAMTs are allocated) and use a reserved area
> if there was a gap between them.

We can add above to the changelog of this patch, or the patch 09 ("x86/virt/tdx:
Fill out TDMRs to cover all TDX memory regions").  The latter perhaps is better
since that patch is the first place where the balance of TDMRs and reserved
areas is related.

What is your suggestion?

> 
> I'm *really* close to acking this patch once those are fixed up.