Re: [PATCH 1/2] spapr: number of SMP sockets must be equal to NUMA nodes

From: David Gibson <david@gibson.dropbear.id.au>
To: "Cédric Le Goater" <clg@kaod.org>
Cc: Laurent Vivier <lvivier@redhat.com>,
	Thomas Huth <thuth@redhat.com>,
	Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Daniel Henrique Barboza <danielhb413@gmail.com>,
	qemu-devel@nongnu.org, groug@kaod.org, qemu-ppc@nongnu.org,
	Igor Mammedov <imammedo@redhat.com>
Subject: Re: [PATCH 1/2] spapr: number of SMP sockets must be equal to NUMA nodes
Date: Thu, 1 Apr 2021 13:53:48 +1100	[thread overview]
Message-ID: <YGU1vKCC9ER538dz@yekko.fritz.box> (raw)
In-Reply-To: <041ea566-62cd-0017-fbaa-bb56bd97d381@kaod.org>

[-- Attachment #1: Type: text/plain, Size: 2182 bytes --]

On Wed, Mar 31, 2021 at 05:22:39PM +0200, Cédric Le Goater wrote:
> On 3/31/21 6:58 AM, Michael Ellerman wrote:
> > David Gibson <david@gibson.dropbear.id.au> writes:
> >> On Mon, Mar 29, 2021 at 03:32:37PM -0300, Daniel Henrique Barboza wrote:
> > ...
> >>
> >>> We assign ibm,chip-id=0x0 to CPUs 0-3, but CPUs 2-3 are located in a
> >>> different NUMA node than 0-1. This would mean that the same socket
> >>> would belong to different NUMA nodes at the same time.
> >>
> >> Right... and I'm still not seeing why that's a problem.  AFAICT that's
> >> a possible, if unexpected, situation under real hardware - though
> >> maybe not for POWER9 specifically.
> > 
> > I think I agree.
> > 
> >>> I believe this is what Cedric wants to be addressed. Given that the
> >>> property is called after the OPAL property ibm,chip-id, the kernel
> >>> expects that the property will have the same semantics as in OPAL.
> >>
> >> Even on powernv, I'm not clear why chip-id is tied into the NUMA
> >> configuration, rather than getting all the NUMA info from
> >> associativity properties.
> > 
> > AFAIK we don't use chip-id for anything related to NUMA, if we do I'd
> > consider that a bug.
> 
> Since PAPR only has NUMA nodes, is the use of chip-id in XIVE PAPR 
> considered as a bug ? I would say so.

As noted in another thread, XIVE PAPR *doesn't* actually use chip_id.

And even on PowerNV, I don't think this has any real connection to
NUMA.  For PowerNV we care about whether we're working within a single
XIVE hardware instance, or across multiple.  There's one XIVE
per-chip, hence the relevance of chip-id.  That happens to also match
to NUMA topology on (currently existing) POWER9 chips, but I don't see
that it inherently has to.

> > We do use it for topology_physical_package_id(), but that's almost
> > completely unused.
> 
> In that case, I think it should be fine to return -1 like under PowerVM.
> 
> Thanks,
> 
> C. 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]