All of
 help / color / mirror / Atom feed
From: Yanan Wang <>
To: Peter Maydell <>,
	Paolo Bonzini <>,
	Andrew Jones <>,
	"Michael S . Tsirkin" <>,
	Igor Mammedov <>,
	Shannon Zhao <>, <>,
Cc: "Barry Song" <>,
	"Philippe Mathieu-Daudé" <>,
	"Yanan Wang" <>,,,,,
Subject: [RFC PATCH v3 0/4] hw/arm/virt: Introduce cluster cpu topology support
Date: Sun, 16 May 2021 18:32:24 +0800	[thread overview]
Message-ID: <> (raw)


This is v3 of the series [1] that I posted to introduce cluster cpu topology
besides now existing sockets, cores, and threads for ARM platform.

In implementations of ARM architecture, at most there could be a
cpu hierarchy like "sockets/dies/clusters/cores/threads" defined.
For example, ARM64 server chip Kunpeng 920 totally has 2 sockets,
2 NUMA nodes (also means cpu dies) in each socket, 6 clusters in
each NUMA node, 4 cores in each cluster, and doesn't support SMT.
Clusters within the same NUMA share a L3 cache and cores within
the same cluster share a L2 cache.

The cache affinity of ARM cluster has been proved to improve the
kernel scheduling performance and a patchset [2] has been posted,
where a general sched_domain for clusters was added and a cluster
level was added in the arch-neutral cpu topology struct like below.

struct cpu_topology {
    int thread_id;
    int core_id;
    int cluster_id;
    int package_id;
    int llc_id;
    cpumask_t thread_sibling;
    cpumask_t core_sibling;
    cpumask_t cluster_sibling;
    cpumask_t llc_sibling;

In virtuallization, exposing the cluster level topology to guest
kernel may also improve the scheduling performance. So let's add
the -smp, clusters=* command line support for ARM cpu, then users
will be able to define a four-level cpu hierarchy for machines
and it will be sockets/clusters/cores/threads.

In this series, we only add the cluster concept of cpu topology
for ARM platform currently, and only focus on exposure of the
topology to guest through ACPI and DT.


Test results about exposure of topology:
After applying this patch series, launch a guest with virt-6.1.

Cmdline: -smp 96, sockets=2, clusters=12, cores=4, threads=1
linux-atxcNc:~ # lscpu
Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              96
On-line CPU(s) list: 0-95
Thread(s) per core:  1
Core(s) per socket:  48
Socket(s):           2
NUMA node(s):        1
Vendor ID:           0x48

Topology information of clusters can also be got:
cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list: 0-3
cat /sys/devices/system/cpu/cpu4/topology/cluster_cpus_list: 4-7
cat /sys/devices/system/cpu/cpu8/topology/cluster_cpus_list: 8-11

cat /sys/devices/system/cpu/cpu95/topology/cluster_cpus_list: 92-95

1) Run some benchmark to test the scheduling improvement of guest kernel
   introduced by cluster level virtual topology
2) Add some QEMU tests about ARM vcpu topology, ACPI PPTT table, and DT
   cpu nodes. Will post in a separate patchset later.


- Address comments from Philippe, and Andrew. Thanks!
- Rebased the code on v3 of series " hw/arm/virt: Introduce cpu topology support"
- v2:

- Only focus on cluster support for ARM platform
- v1:


Yanan Wang (4):
  vl.c: Add -smp, clusters=* command line support for ARM cpu
  hw/arm/virt: Add cluster level to device tree
  hw/arm/virt-acpi-build: Add cluster level to PPTT table
  hw/arm/virt: Parse -smp cluster parameter in virt_smp_parse

 hw/arm/virt-acpi-build.c | 45 ++++++++++++++++++++++++----------------
 hw/arm/virt.c            | 44 +++++++++++++++++++++++----------------
 include/hw/arm/virt.h    |  1 +
 qemu-options.hx          | 26 +++++++++++++----------
 softmmu/vl.c             |  3 +++
 5 files changed, 72 insertions(+), 47 deletions(-)


             reply	other threads:[~2021-05-16 10:34 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-16 10:32 Yanan Wang [this message]
2021-05-16 10:32 ` [RFC PATCH v3 1/4] vl.c: Add -smp, clusters=* command line support for ARM cpu Yanan Wang
2021-05-17  9:07   ` Andrew Jones
2021-05-17 15:07     ` wangyanan (Y)
2021-05-16 10:32 ` [RFC PATCH v3 2/4] hw/arm/virt: Add cluster level to device tree Yanan Wang
2021-05-16 10:32 ` [RFC PATCH v3 3/4] hw/arm/virt-acpi-build: Add cluster level to PPTT table Yanan Wang
2021-05-16 10:32 ` [RFC PATCH v3 4/4] hw/arm/virt: Parse -smp cluster parameter in virt_smp_parse Yanan Wang
2021-05-17  9:12   ` Andrew Jones
2021-05-17 15:10     ` wangyanan (Y)
2021-05-17 15:17   ` Salil Mehta
2021-05-18  3:48     ` wangyanan (Y)
2021-05-18  6:52       ` Salil Mehta
2021-05-18  6:52         ` Salil Mehta
2021-05-18  8:19       ` Andrew Jones

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.