Re: [PATCH RFC] x86/sgx: Allocate form local NUMA node first

[Sean Christopherson <sean.j.christopherson@intel.com>]

On Wed, Jun 24, 2020 at 04:24:25PM -0700, Dave Hansen wrote:
> On 6/22/20 9:39 PM, Jarkko Sakkinen wrote:
> > +static int __init sgx_pfn_to_nid(unsigned long pfn)
> > +{
> > +	pg_data_t *pgdat;
> > +	int nid;
> > +
> > +	for (nid = 0; nid < nr_node_ids; nid++) {
> > +		pgdat = NODE_DATA(nid);
> > +
> > +		if (pfn >= pgdat->node_start_pfn &&
> > +		    pfn < (pgdat->node_start_pfn + pgdat->node_spanned_pages))
> 
> pgdat_end_pfn(), perhaps?
> 
> > +			return nid;
> > +	}
> > +
> > +	return 0;
> > +}
> 
> Does this actually work?
> 
> The node span (->node_start_pfn through start+->node_spanned_pages) only
> contains pages which the OS is actively managing, usually RAM but
> sometimes also persistent memory.  This has some assumption that the SGX
> PFNs are within the node's span.  I would only _expect_ that to happen
> if the node was built like this:
> 
> 	| Node-X RAM | EPC | Node-X RAM |
> 
> If the EPC was on either end:
> 
> 	| Node-X RAM | EPC |
> or
> 	| EPC | Node-X RAM |
> 
> I suspect that the pgdat span wouldn't include EPC.  EPC is, if I
> remember correctly, a E820_RESERVED region.

It is indeed E820_RESERVED, but the BIOS WG for ICX states that EPC regions
should be enumerated in ACPI SRAT along with regular memory.

But, I haven't actually verified that info makes its way into the kernel's
pgdata stuff.