[PATCH] powerpc/numa: Use associativity if VPHN hcall is successful

[PATCH] powerpc/numa: Use associativity if VPHN hcall is successful

[Srikar Dronamraju]

Currently associativity is used to lookup node-id even if the preceding
VPHN hcall failed. However this can cause CPU to be made part of the
wrong node, (most likely to be node 0). This is because VPHN is not
enabled on kvm guests.

With 2ea6263 ("powerpc/topology: Get topology for shared processors at
boot"), associativity is used to set to the wrong node. Hence KVM guest
topology is broken.

For example : A 4 node KVM guest before would have reported.

[root@localhost ~]#  numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3
node 0 size: 1746 MB
node 0 free: 1604 MB
node 1 cpus: 4 5 6 7
node 1 size: 2044 MB
node 1 free: 1765 MB
node 2 cpus: 8 9 10 11
node 2 size: 2044 MB
node 2 free: 1837 MB
node 3 cpus: 12 13 14 15
node 3 size: 2044 MB
node 3 free: 1903 MB
node distances:
node   0   1   2   3
  0:  10  40  40  40
  1:  40  10  40  40
  2:  40  40  10  40
  3:  40  40  40  10

would now report

[root@localhost ~]# numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 2 3 4 5 6 7 8 9 10 11 12 13 14 15
node 0 size: 1746 MB
node 0 free: 1244 MB
node 1 cpus:
node 1 size: 2044 MB
node 1 free: 2032 MB
node 2 cpus: 1
node 2 size: 2044 MB
node 2 free: 2028 MB
node 3 cpus:
node 3 size: 2044 MB
node 3 free: 2032 MB
node distances:
node   0   1   2   3
  0:  10  40  40  40
  1:  40  10  40  40
  2:  40  40  10  40
  3:  40  40  40  10

Fix this by skipping associativity lookup if the VPHN hcall failed.

Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
 arch/powerpc/mm/numa.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index 35ac5422903a..e94148a1d7e4 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -1204,7 +1204,9 @@ int find_and_online_cpu_nid(int cpu)
 	int new_nid;
 
 	/* Use associativity from first thread for all siblings */
-	vphn_get_associativity(cpu, associativity);
+	if (vphn_get_associativity(cpu, associativity))
+		return cpu_to_node(cpu);
+
 	new_nid = associativity_to_nid(associativity);
 	if (new_nid < 0 || !node_possible(new_nid))
 		new_nid = first_online_node;
-- 
2.12.3

Re: powerpc/numa: Use associativity if VPHN hcall is successful

[Michael Ellerman]

On Tue, 2018-09-25 at 12:25:15 UTC, Srikar Dronamraju wrote:
> Currently associativity is used to lookup node-id even if the preceding
> VPHN hcall failed. However this can cause CPU to be made part of the
> wrong node, (most likely to be node 0). This is because VPHN is not
> enabled on kvm guests.
> 
> With 2ea6263 ("powerpc/topology: Get topology for shared processors at
> boot"), associativity is used to set to the wrong node. Hence KVM guest
> topology is broken.
> 
> For example : A 4 node KVM guest before would have reported.
> 
> [root@localhost ~]#  numactl -H
> available: 4 nodes (0-3)
> node 0 cpus: 0 1 2 3
> node 0 size: 1746 MB
> node 0 free: 1604 MB
> node 1 cpus: 4 5 6 7
> node 1 size: 2044 MB
> node 1 free: 1765 MB
> node 2 cpus: 8 9 10 11
> node 2 size: 2044 MB
> node 2 free: 1837 MB
> node 3 cpus: 12 13 14 15
> node 3 size: 2044 MB
> node 3 free: 1903 MB
> node distances:
> node   0   1   2   3
>   0:  10  40  40  40
>   1:  40  10  40  40
>   2:  40  40  10  40
>   3:  40  40  40  10
> 
> would now report
> 
> [root@localhost ~]# numactl -H
> available: 4 nodes (0-3)
> node 0 cpus: 0 2 3 4 5 6 7 8 9 10 11 12 13 14 15
> node 0 size: 1746 MB
> node 0 free: 1244 MB
> node 1 cpus:
> node 1 size: 2044 MB
> node 1 free: 2032 MB
> node 2 cpus: 1
> node 2 size: 2044 MB
> node 2 free: 2028 MB
> node 3 cpus:
> node 3 size: 2044 MB
> node 3 free: 2032 MB
> node distances:
> node   0   1   2   3
>   0:  10  40  40  40
>   1:  40  10  40  40
>   2:  40  40  10  40
>   3:  40  40  40  10
> 
> Fix this by skipping associativity lookup if the VPHN hcall failed.
> 
> Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/2483ef056f6e42f61cd266452e2841

cheers