Re: [PATCH 1/1] powerpc/smp: Set numa node before updating mask

From: Nathan Lynch <nathanl@linux.ibm.com>
To: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Scott Cheloha <cheloha@linux.ibm.com>,
	Geetika Moolchandani <Geetika.Moolchandani1@ibm.com>,
	Valentin Schneider <valentin.schneider@arm.com>,
	Laurent Dufour <ldufour@linux.vnet.ibm.com>,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	Ingo Molnar <mingo@kernel.org>
Subject: Re: [PATCH 1/1] powerpc/smp: Set numa node before updating mask
Date: Wed, 07 Apr 2021 07:19:10 -0500	[thread overview]
Message-ID: <87eefml22p.fsf@linux.ibm.com> (raw)
In-Reply-To: <20210402031815.GI2339179@linux.vnet.ibm.com>

Sorry for the delay in following up here.

Srikar Dronamraju <srikar@linux.vnet.ibm.com> writes:
>> > -	set_numa_node(numa_cpu_lookup_table[cpu]);
>> > -	set_numa_mem(local_memory_node(numa_cpu_lookup_table[cpu]));
>> > -
>> 
>> Regardless of your change: at boot time, this set of calls to
>> set_numa_node() and set_numa_mem() is redundant, right? Because
>> smp_prepare_cpus() has:
>> 
>> 	for_each_possible_cpu(cpu) {
>> 		...
>> 		if (cpu_present(cpu)) {
>> 			set_cpu_numa_node(cpu, numa_cpu_lookup_table[cpu]);
>> 			set_cpu_numa_mem(cpu,
>> 				local_memory_node(numa_cpu_lookup_table[cpu]));
>> 		}
>> 
>> I would rather that, when onlining a CPU that happens to have been
>> dynamically added after boot, we enter start_secondary() with conditions
>> equivalent to those at boot time. Or as close to that as is practical.
>> 
>> So I'd suggest that pseries_add_processor() be made to update
>> these things when the CPUs are marked present, before onlining them.
>
> In pseries_add_processor, we are only marking the cpu as present. i.e
> I believe numa_setup_cpu() would not have been called. So we may not have a
> way to associate the CPU to the node. Otherwise we will have to call
> numa_setup_cpu() or the hcall_vphn.
>
> We could try calling numa_setup_cpu() immediately after we set the
> CPU to be present, but that would be one more extra hcall + I dont know if
> there are any more steps needed before CPU being made present and
> associating the CPU to the node.

An additional hcall in this path doesn't seem too expensive.

> Are we sure the node is already online?

I see that dlpar_online_cpu() calls find_and_online_cpu_nid(), so yes I
think that's covered.

> For the numa_mem, we are better of if the zonelists for the node are
> built.
>
> or the other solution would be to call this in map_cpu_to_node().
> Here also we have to be sure the zonelists for the node are already
> built.