Re: RESEND Re: [Patch 2/2]: powerpc/hotplug/mm: Fix hot-add memory node assoc

From: Michael Bringmann <mwb@linux.vnet.ibm.com>
To: Michael Ellerman <mpe@ellerman.id.au>,
	Reza Arbab <arbab@linux.vnet.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>,
	linux-kernel@vger.kernel.org, Paul Mackerras <paulus@samba.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
	Bharata B Rao <bharata@linux.vnet.ibm.com>,
	Shailendra Singh <shailendras@nvidia.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linuxppc-dev@lists.ozlabs.org,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Michael Bringmann from Kernel Team <mbringm@us.ibm.com>
Subject: Re: RESEND Re: [Patch 2/2]: powerpc/hotplug/mm: Fix hot-add memory node assoc
Date: Tue, 13 Jun 2017 17:21:22 -0500	[thread overview]
Message-ID: <ca6ef126-4bc7-5b93-11d1-1342c46fa9b6@linux.vnet.ibm.com> (raw)
In-Reply-To: <87o9tsgo9b.fsf@concordia.ellerman.id.au>

On a related note, we are discussing the addition of 2 new device-tree properties
with Pete Heyrman and his fellows that should simplify the determination of the
set of required nodes.

* One property would provide the total/max number of nodes needed by the kernel
  on the current hardware.
* A second property would provide the total/max number of nodes that the kernel
  could use on any system to which it could be migrated.

These properties aren't available, yet, and it takes time to define new properties
in the PAPR and have them implemented in pHyp and the kernel.  As an intermediary
step, the systems which are doing a lot of dynamic hot-add/hot-remove configuration
could provide equivalent information to the PowerPC kernel with a command line
parameter.  The 'numa.c' code would then read this value and fill in the necessary
entries in the 'node_possible_map'.

Would you foresee any problems with using such a feature?

Thanks.

On 06/13/2017 05:45 AM, Michael Ellerman wrote:
> Michael Bringmann <mwb@linux.vnet.ibm.com> writes:
> 
>> Here is the information from 2 different kernels.  I have not been able to retrieve
>> the information matching yesterday's attachments, yet, as those dumps were
>> acquired in April.
>>  
>> Attached please find 2 dumps of similar material from kernels running with my
>> current patches (Linux 4.4, Linux 4.12).
> 
> OK thanks.
> 
> I'd actually like to see the dmesg output from a kernel *without* your
> patches.
> 
> Looking at the device tree properties:
> 
> ltcalpine2-lp9:/proc/device-tree/ibm,dynamic-reconfiguration-memory # lsprop ibm,associativity-lookup-arrays
> ibm,associativity-lookup-arrays
> 		 00000004 = 4 arrays
>                  00000004 = of 4 entries each
>                  00000000 00000000 00000000 00000000
> 		 00000000 00000000 00000001 00000001
>                  00000000 00000003 00000006 00000006
>                  00000000 00000003 00000007 00000007
> 
> 
> Which does tell us that nodes 0, 1, 6 and 7 exist.
> 
> So your idea of looking at that and setting any node found in there
> online should work.
> 
> My only worry is that behaviour appears to be completely undocumented in
> PAPR, ie. PAPR explicitly says that property only needs to contain
> values for LMBs present at boot.
> 
> But possibly we can talk to the PowerVM/PAPR guys and have that changed
> so that it becomes something we can rely on.
> 
> cheers
> 
> 

-- 
Michael W. Bringmann
Linux Technology Center
IBM Corporation
Tie-Line  363-5196
External: (512) 286-5196
Cell:       (512) 466-0650
mwb@linux.vnet.ibm.com