From: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
To: linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au
Cc: Nathan Lynch <nathanl@linux.ibm.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Daniel Henrique Barboza <danielhb413@gmail.com>,
David Gibson <david@gibson.dropbear.id.au>
Subject: [PATCH v5 0/6] Add support for FORM2 associativity
Date: Mon, 28 Jun 2021 20:41:11 +0530 [thread overview]
Message-ID: <20210628151117.545935-1-aneesh.kumar@linux.ibm.com> (raw)
Form2 associativity adds a much more flexible NUMA topology layout
than what is provided by Form1. More details can be found in patch 7.
$ numactl -H
...
node distances:
node 0 1 2 3
0: 10 11 222 33
1: 44 10 55 66
2: 77 88 10 99
3: 101 121 132 10
$
After DAX kmem memory add
# numactl -H
available: 5 nodes (0-4)
...
node distances:
node 0 1 2 3 4
0: 10 11 222 33 240
1: 44 10 55 66 255
2: 77 88 10 99 255
3: 101 121 132 10 230
4: 255 255 255 230 10
PAPR SCM now use the numa distance details to find the numa_node and target_node
for the device.
kvaneesh@ubuntu-guest:~$ ndctl list -N -v
[
{
"dev":"namespace0.0",
"mode":"devdax",
"map":"dev",
"size":1071644672,
"uuid":"d333d867-3f57-44c8-b386-d4d3abdc2bf2",
"raw_uuid":"915361ad-fe6a-42dd-848f-d6dc9f5af362",
"daxregion":{
"id":0,
"size":1071644672,
"devices":[
{
"chardev":"dax0.0",
"size":1071644672,
"target_node":4,
"mode":"devdax"
}
]
},
"align":2097152,
"numa_node":3
}
]
kvaneesh@ubuntu-guest:~$
The above output is with a Qemu command line
-numa node,nodeid=4 \
-numa dist,src=0,dst=1,val=11 -numa dist,src=0,dst=2,val=222 -numa dist,src=0,dst=3,val=33 -numa dist,src=0,dst=4,val=240 \
-numa dist,src=1,dst=0,val=44 -numa dist,src=1,dst=2,val=55 -numa dist,src=1,dst=3,val=66 -numa dist,src=1,dst=4,val=255 \
-numa dist,src=2,dst=0,val=77 -numa dist,src=2,dst=1,val=88 -numa dist,src=2,dst=3,val=99 -numa dist,src=2,dst=4,val=255 \
-numa dist,src=3,dst=0,val=101 -numa dist,src=3,dst=1,val=121 -numa dist,src=3,dst=2,val=132 -numa dist,src=3,dst=4,val=230 \
-numa dist,src=4,dst=0,val=255 -numa dist,src=4,dst=1,val=255 -numa dist,src=4,dst=2,val=255 -numa dist,src=4,dst=3,val=230 \
-object memory-backend-file,id=memnvdimm1,prealloc=yes,mem-path=$PMEM_DISK,share=yes,size=${PMEM_SIZE} \
-device nvdimm,label-size=128K,memdev=memnvdimm1,id=nvdimm1,slot=4,uuid=72511b67-0b3b-42fd-8d1d-5be3cae8bcaa,node=4
Qemu changes can be found at https://lore.kernel.org/qemu-devel/20210616011944.2996399-1-danielhb413@gmail.com/
Changes from v4:
* Drop DLPAR related device tree property for now because both Qemu nor PowerVM
will provide the distance details of all possible NUMA nodes during boot.
* Rework numa distance code based on review feedback.
Changes from v3:
* Drop PAPR SCM specific changes and depend completely on NUMA distance information.
Changes from v2:
* Add nvdimm list to Cc:
* update PATCH 8 commit message.
Changes from v1:
* Update FORM2 documentation.
* rename max_domain_index to max_associativity_domain_index
Aneesh Kumar K.V (6):
powerpc/pseries: rename min_common_depth to primary_domain_index
powerpc/pseries: rename distance_ref_points_depth to
max_associativity_domain_index
powerpc/pseries: Rename TYPE1_AFFINITY to FORM1_AFFINITY
powerpc/pseries: Consolidate different NUMA distance update code paths
powerpc/pseries: Add a helper for form1 cpu distance
powerpc/pseries: Add support for FORM2 associativity
Documentation/powerpc/associativity.rst | 103 +++++
arch/powerpc/include/asm/firmware.h | 7 +-
arch/powerpc/include/asm/prom.h | 3 +-
arch/powerpc/include/asm/topology.h | 4 +-
arch/powerpc/kernel/prom_init.c | 3 +-
arch/powerpc/mm/numa.c | 415 +++++++++++++-----
arch/powerpc/platforms/pseries/firmware.c | 3 +-
arch/powerpc/platforms/pseries/hotplug-cpu.c | 2 +
.../platforms/pseries/hotplug-memory.c | 2 +
arch/powerpc/platforms/pseries/lpar.c | 4 +-
arch/powerpc/platforms/pseries/pseries.h | 1 +
11 files changed, 432 insertions(+), 115 deletions(-)
create mode 100644 Documentation/powerpc/associativity.rst
--
2.31.1
next reply other threads:[~2021-06-28 15:12 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-28 15:11 Aneesh Kumar K.V [this message]
2021-06-28 15:11 ` [PATCH v5 1/6] powerpc/pseries: rename min_common_depth to primary_domain_index Aneesh Kumar K.V
2021-07-22 1:59 ` David Gibson
2021-07-22 2:36 ` David Gibson
2021-07-22 5:17 ` Aneesh Kumar K.V
2021-07-26 2:28 ` David Gibson
2021-06-28 15:11 ` [PATCH v5 2/6] powerpc/pseries: rename distance_ref_points_depth to max_associativity_domain_index Aneesh Kumar K.V
2021-07-22 0:59 ` David Gibson
2021-07-22 1:19 ` David Gibson
2021-06-28 15:11 ` [PATCH v5 3/6] powerpc/pseries: Rename TYPE1_AFFINITY to FORM1_AFFINITY Aneesh Kumar K.V
2021-06-28 15:11 ` [PATCH v5 4/6] powerpc/pseries: Consolidate different NUMA distance update code paths Aneesh Kumar K.V
2021-06-28 20:21 ` kernel test robot
2021-06-28 20:40 ` kernel test robot
2021-07-22 1:40 ` David Gibson
2021-07-22 7:07 ` Aneesh Kumar K.V
2021-07-26 2:37 ` David Gibson
2021-07-27 3:32 ` Aneesh Kumar K.V
2021-07-27 5:59 ` David Gibson
2021-06-28 15:11 ` [PATCH v5 5/6] powerpc/pseries: Add a helper for form1 cpu distance Aneesh Kumar K.V
2021-07-22 1:42 ` David Gibson
2021-07-22 7:09 ` Aneesh Kumar K.V
2021-07-26 2:38 ` David Gibson
2021-06-28 15:11 ` [PATCH v5 6/6] powerpc/pseries: Add support for FORM2 associativity Aneesh Kumar K.V
2021-07-22 2:28 ` David Gibson
2021-07-22 7:34 ` Aneesh Kumar K.V
2021-07-26 2:41 ` David Gibson
2021-07-13 14:27 ` [PATCH v5 0/6] " Daniel Henrique Barboza
2021-07-13 14:30 ` Aneesh Kumar K.V
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210628151117.545935-1-aneesh.kumar@linux.ibm.com \
--to=aneesh.kumar@linux.ibm.com \
--cc=danielhb413@gmail.com \
--cc=david@gibson.dropbear.id.au \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=nathanl@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).