* [PATCH v6 00/11] s390x: CPU Topology
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Hi,
The goal of this series is to implement CPU topology for S390, it
improves the preceeding series with the implementation of books and
drawers, of non uniform CPU topology and with documentation.
To use these patches, you will need the Linux series version 5.
You find it there:
https://lkml.org/lkml/2022/2/17/253
Currently this code is for KVM only, I have no idea if it is interesting
to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its
implementation in QEMU you can have a look at the documentation in the
last patch or follow the introduction here under.
A short introduction
====================
CPU Topology is described in the S390 POP with essentially the description
of two instructions:
PTF Perform Topology function used to poll for topology change
and used to set the polarization but this part is not part of this item.
STSI Store System Information and the SYSIB 15.1.x providing the Topology
configuration.
S390 Topology is a 6 levels hierarchical topology with up to 5 level
of containers. The last topology level, specifying the CPU cores.
This patch series only uses the two lower levels sockets and cores.
To get the information on the topology, S390 provides the STSI
instruction, which stores a structures providing the list of the
containers used in the Machine topology: the SYSIB.
A selector within the STSI instruction allow to chose how many topology
levels will be provide in the SYSIB.
Using the Topology List Entries (TLE) provided inside the SYSIB we
the Linux kernel is able to compute the information about the cache
distance between two cores and can use this information to take
scheduling decisions.
The design
==========
1) To be ready for hotplug, I chose an Object oriented design
of the topology containers:
- A node is a bridge on the SYSBUS and defines a "node bus"
- A drawer is hotplug on the "node bus"
- A book on the "drawer bus"
- A socket on the "book bus"
- And the CPU Topology List Entry (CPU-TLE)sits on the socket bus.
These objects will be enhanced with the cache information when
NUMA is implemented.
This also allows for easy retrieval when building the different SYSIB
for Store Topology System Information (STSI)
2) Perform Topology Function (PTF) instruction is made available to the
guest with a new KVM capability and intercepted in QEMU, allowing the
guest to pool for topology changes.
Features
========
- There is no direct match between IDs shown by:
- lscpu (unrelated numbered list),
- SYSIB 15.1.x (topology ID)
- The CPU number, left column of lscpu, is used to reference a CPU
by Linux tools
While the CPU address is used by QEMU for hotplug.
- Effect of -smp parsing on the topology with an example:
-smp 9,sockets=4,cores=4,maxcpus=16
We have 4 socket each holding 4 cores so that we have a maximum
of 16 CPU, 9 of them are active on boot. (Should be obvious)
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
#
- To plug a new CPU inside the topology one can simply use the CPU
address like in:
(qemu) device_add host-s390x-cpu,core-id=12
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
9 - - - - - ::: no yes horizontal 12
# chcpu -e 9
CPU 9 enabled
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
9 0 0 0 3 9 9:9:9:9 yes yes horizontal 12
#
It is up to the admin level, Libvirt for example, to pin the righ CPU to the right
vCPU, but as we can see without NUMA, chosing separate sockets for CPUs is not easy
without hotplug because without information the code will assign the vCPU and fill
the sockets one after the other.
Note that this is also the default behavior on the LPAR.
Conclusion
==========
This patch, together with the associated KVM patch allows to provide CPU topology
information to the guest.
Currently, only dedicated vCPU and CPU are supported and a NUMA topology can only
be handled using CPU hotplug inside the guest.
Regards,
Pierre
Pierre Morel (11):
s390x: SCLP: reporting the maximum nested topology entries
s390x: topology: CPU topology objects and structures
s390x: topology: implementating Store Topology System Information
s390x: CPU topology: CPU topology migration
s390x: kvm: topology: interception of PTF instruction
s390x: topology: Adding books to CPU topology
s390x: topology: Adding books to STSI
s390x: topology: Adding drawers to CPU topology
s390x: topology: Adding drawers to STSI
s390x: topology: implementing numa for the s390x topology
s390x: topology: documentation
docs/system/s390x/numa-cpu-topology.rst | 273 +++++++++++
hw/core/machine-smp.c | 48 +-
hw/core/machine.c | 22 +
hw/s390x/cpu-topology.c | 599 ++++++++++++++++++++++++
hw/s390x/meson.build | 1 +
hw/s390x/s390-virtio-ccw.c | 124 ++++-
hw/s390x/sclp.c | 1 +
include/hw/boards.h | 8 +
include/hw/s390x/cpu-topology.h | 97 ++++
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
qapi/machine.json | 12 +-
softmmu/vl.c | 6 +
target/s390x/cpu.h | 50 ++
target/s390x/cpu_features_def.h.inc | 1 +
target/s390x/cpu_models.c | 2 +
target/s390x/cpu_topology.c | 169 +++++++
target/s390x/gen-features.c | 3 +
target/s390x/kvm/kvm.c | 26 +
target/s390x/machine.c | 48 ++
target/s390x/meson.build | 1 +
21 files changed, 1485 insertions(+), 16 deletions(-)
create mode 100644 docs/system/s390x/numa-cpu-topology.rst
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 target/s390x/cpu_topology.c
--
2.27.0
Changelog:
- since v5
- rebasing on newer QEMU version
- reworked most lines above 80 characters.
- since v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 00/11] s390x: CPU Topology
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
Hi,
The goal of this series is to implement CPU topology for S390, it
improves the preceeding series with the implementation of books and
drawers, of non uniform CPU topology and with documentation.
To use these patches, you will need the Linux series version 5.
You find it there:
https://lkml.org/lkml/2022/2/17/253
Currently this code is for KVM only, I have no idea if it is interesting
to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its
implementation in QEMU you can have a look at the documentation in the
last patch or follow the introduction here under.
A short introduction
====================
CPU Topology is described in the S390 POP with essentially the description
of two instructions:
PTF Perform Topology function used to poll for topology change
and used to set the polarization but this part is not part of this item.
STSI Store System Information and the SYSIB 15.1.x providing the Topology
configuration.
S390 Topology is a 6 levels hierarchical topology with up to 5 level
of containers. The last topology level, specifying the CPU cores.
This patch series only uses the two lower levels sockets and cores.
To get the information on the topology, S390 provides the STSI
instruction, which stores a structures providing the list of the
containers used in the Machine topology: the SYSIB.
A selector within the STSI instruction allow to chose how many topology
levels will be provide in the SYSIB.
Using the Topology List Entries (TLE) provided inside the SYSIB we
the Linux kernel is able to compute the information about the cache
distance between two cores and can use this information to take
scheduling decisions.
The design
==========
1) To be ready for hotplug, I chose an Object oriented design
of the topology containers:
- A node is a bridge on the SYSBUS and defines a "node bus"
- A drawer is hotplug on the "node bus"
- A book on the "drawer bus"
- A socket on the "book bus"
- And the CPU Topology List Entry (CPU-TLE)sits on the socket bus.
These objects will be enhanced with the cache information when
NUMA is implemented.
This also allows for easy retrieval when building the different SYSIB
for Store Topology System Information (STSI)
2) Perform Topology Function (PTF) instruction is made available to the
guest with a new KVM capability and intercepted in QEMU, allowing the
guest to pool for topology changes.
Features
========
- There is no direct match between IDs shown by:
- lscpu (unrelated numbered list),
- SYSIB 15.1.x (topology ID)
- The CPU number, left column of lscpu, is used to reference a CPU
by Linux tools
While the CPU address is used by QEMU for hotplug.
- Effect of -smp parsing on the topology with an example:
-smp 9,sockets=4,cores=4,maxcpus=16
We have 4 socket each holding 4 cores so that we have a maximum
of 16 CPU, 9 of them are active on boot. (Should be obvious)
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
#
- To plug a new CPU inside the topology one can simply use the CPU
address like in:
(qemu) device_add host-s390x-cpu,core-id=12
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
9 - - - - - ::: no yes horizontal 12
# chcpu -e 9
CPU 9 enabled
# lscpu -e
CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
2 0 0 0 0 2 2:2:2:2 yes yes horizontal 2
3 0 0 0 0 3 3:3:3:3 yes yes horizontal 3
4 0 0 0 1 4 4:4:4:4 yes yes horizontal 4
5 0 0 0 1 5 5:5:5:5 yes yes horizontal 5
6 0 0 0 1 6 6:6:6:6 yes yes horizontal 6
7 0 0 0 1 7 7:7:7:7 yes yes horizontal 7
8 0 0 0 2 8 8:8:8:8 yes yes horizontal 8
9 0 0 0 3 9 9:9:9:9 yes yes horizontal 12
#
It is up to the admin level, Libvirt for example, to pin the righ CPU to the right
vCPU, but as we can see without NUMA, chosing separate sockets for CPUs is not easy
without hotplug because without information the code will assign the vCPU and fill
the sockets one after the other.
Note that this is also the default behavior on the LPAR.
Conclusion
==========
This patch, together with the associated KVM patch allows to provide CPU topology
information to the guest.
Currently, only dedicated vCPU and CPU are supported and a NUMA topology can only
be handled using CPU hotplug inside the guest.
Regards,
Pierre
Pierre Morel (11):
s390x: SCLP: reporting the maximum nested topology entries
s390x: topology: CPU topology objects and structures
s390x: topology: implementating Store Topology System Information
s390x: CPU topology: CPU topology migration
s390x: kvm: topology: interception of PTF instruction
s390x: topology: Adding books to CPU topology
s390x: topology: Adding books to STSI
s390x: topology: Adding drawers to CPU topology
s390x: topology: Adding drawers to STSI
s390x: topology: implementing numa for the s390x topology
s390x: topology: documentation
docs/system/s390x/numa-cpu-topology.rst | 273 +++++++++++
hw/core/machine-smp.c | 48 +-
hw/core/machine.c | 22 +
hw/s390x/cpu-topology.c | 599 ++++++++++++++++++++++++
hw/s390x/meson.build | 1 +
hw/s390x/s390-virtio-ccw.c | 124 ++++-
hw/s390x/sclp.c | 1 +
include/hw/boards.h | 8 +
include/hw/s390x/cpu-topology.h | 97 ++++
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
qapi/machine.json | 12 +-
softmmu/vl.c | 6 +
target/s390x/cpu.h | 50 ++
target/s390x/cpu_features_def.h.inc | 1 +
target/s390x/cpu_models.c | 2 +
target/s390x/cpu_topology.c | 169 +++++++
target/s390x/gen-features.c | 3 +
target/s390x/kvm/kvm.c | 26 +
target/s390x/machine.c | 48 ++
target/s390x/meson.build | 1 +
21 files changed, 1485 insertions(+), 16 deletions(-)
create mode 100644 docs/system/s390x/numa-cpu-topology.rst
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 target/s390x/cpu_topology.c
--
2.27.0
Changelog:
- since v5
- rebasing on newer QEMU version
- reworked most lines above 80 characters.
- since v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v6 01/11] s390x: SCLP: reporting the maximum nested topology entries
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
The maximum nested topology entries is used by the guest to know
how many nested topology are available on the machine.
Currently, reporting SCLP reports 0, which is the equivalent of
reporting the default value of 2.
Let's use the default SCLP value of 2 and increase this value in the
future patches implementing higher levels.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/sclp.c | 1 +
include/hw/s390x/sclp.h | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 89c30a8a91..7797a45cef 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -125,6 +125,7 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
/* CPU information */
prepare_cpu_entries(machine, entries_start, &cpu_count);
+ read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
read_info->entries_cpu = cpu_to_be16(cpu_count);
read_info->offset_cpu = cpu_to_be16(offset_cpu);
read_info->highest_cpu = cpu_to_be16(machine->smp.max_cpus - 1);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index d3ade40a5a..20dd4bd818 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,12 +111,14 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
+#define SCLP_READ_SCP_INFO_MNEST 2
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
- uint8_t _reserved1[16 - 11]; /* 11-15 */
+ uint8_t _reserved1[15 - 11]; /* 11-14 */
+ uint8_t stsi_parm; /* 15-16 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu; /* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 01/11] s390x: SCLP: reporting the maximum nested topology entries
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
The maximum nested topology entries is used by the guest to know
how many nested topology are available on the machine.
Currently, reporting SCLP reports 0, which is the equivalent of
reporting the default value of 2.
Let's use the default SCLP value of 2 and increase this value in the
future patches implementing higher levels.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/sclp.c | 1 +
include/hw/s390x/sclp.h | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 89c30a8a91..7797a45cef 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -125,6 +125,7 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
/* CPU information */
prepare_cpu_entries(machine, entries_start, &cpu_count);
+ read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
read_info->entries_cpu = cpu_to_be16(cpu_count);
read_info->offset_cpu = cpu_to_be16(offset_cpu);
read_info->highest_cpu = cpu_to_be16(machine->smp.max_cpus - 1);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index d3ade40a5a..20dd4bd818 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,12 +111,14 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
+#define SCLP_READ_SCP_INFO_MNEST 2
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
- uint8_t _reserved1[16 - 11]; /* 11-15 */
+ uint8_t _reserved1[15 - 11]; /* 11-14 */
+ uint8_t stsi_parm; /* 15-16 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu; /* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 02/11] s390x: topology: CPU topology objects and structures
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
We use new objects to have a dynamic administration of the CPU topology.
The highest level object in this implementation is the s390 book and
in this first implementation of CPU topology for S390 we have a single
book.
The book is built as a SYSBUS bridge during the CPU initialization.
Other objects, sockets and core will be built after the parsing
of the QEMU -smp argument.
Every object under this single book will be build dynamically
immediately after a CPU has be realized if it is needed.
The CPU will fill the sockets once after the other, according to the
number of core per socket defined during the smp parsing.
Each CPU inside a socket will be represented by a bit in a 64bit
unsigned long. Set on plug and clear on unplug of a CPU.
For the S390 CPU topology, thread and cores are merged into
topology cores and the number of topology cores is the multiplication
of cores by the numbers of threads.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 361 ++++++++++++++++++++++++++++++++
hw/s390x/meson.build | 1 +
hw/s390x/s390-virtio-ccw.c | 4 +
include/hw/s390x/cpu-topology.h | 74 +++++++
target/s390x/cpu.h | 47 +++++
5 files changed, 487 insertions(+)
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 include/hw/s390x/cpu-topology.h
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 0000000000..b7131b4ac3
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,361 @@
+/*
+ * CPU Topology
+ *
+ * Copyright 2021 IBM Corp.
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+
+static S390TopologyCores *s390_create_cores(S390TopologySocket *socket,
+ int origin)
+{
+ DeviceState *dev;
+ S390TopologyCores *cores;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (socket->bus->num_children >= (ms->smp.cores * ms->smp.threads)) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_CORES);
+ qdev_realize_and_unref(dev, socket->bus, &error_fatal);
+
+ cores = S390_TOPOLOGY_CORES(dev);
+ cores->origin = origin;
+ socket->cnt += 1;
+
+ return cores;
+}
+
+static S390TopologySocket *s390_create_socket(S390TopologyBook *book, int id)
+{
+ DeviceState *dev;
+ S390TopologySocket *socket;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (book->bus->num_children >= ms->smp.sockets) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_SOCKET);
+ qdev_realize_and_unref(dev, book->bus, &error_fatal);
+
+ socket = S390_TOPOLOGY_SOCKET(dev);
+ socket->socket_id = id;
+ book->cnt++;
+
+ return socket;
+}
+
+/*
+ * s390_get_cores:
+ * @socket: the socket to search into
+ * @origin: the origin specified for the S390TopologyCores
+ *
+ * returns a pointer to a S390TopologyCores structure within a socket having
+ * the specified origin.
+ * First search if the socket is already containing the S390TopologyCores
+ * structure and if not create one with this origin.
+ */
+static S390TopologyCores *s390_get_cores(S390TopologySocket *socket, int origin)
+{
+ S390TopologyCores *cores;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &socket->bus->children, sibling) {
+ cores = S390_TOPOLOGY_CORES(kid->child);
+ if (cores->origin == origin) {
+ return cores;
+ }
+ }
+ return s390_create_cores(socket, origin);
+}
+
+/*
+ * s390_get_socket:
+ * @book: The book to search into
+ * @socket_id: the identifier of the socket to search for
+ *
+ * returns a pointer to a S390TopologySocket structure within a book having
+ * the specified socket_id.
+ * First search if the book is already containing the S390TopologySocket
+ * structure and if not create one with this socket_id.
+ */
+static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
+ int socket_id)
+{
+ S390TopologySocket *socket;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &book->bus->children, sibling) {
+ socket = S390_TOPOLOGY_SOCKET(kid->child);
+ if (socket->socket_id == socket_id) {
+ return socket;
+ }
+ }
+ return s390_create_socket(book, socket_id);
+}
+
+/*
+ * s390_topology_new_cpu:
+ * @core_id: the core ID is machine wide
+ *
+ * We have a single book returned by s390_get_topology(),
+ * then we build the hierarchy on demand.
+ * Note that we do not destroy the hierarchy on error creating
+ * an entry in the topology, we just keep it empty.
+ * We do not need to worry about not finding a topology level
+ * entry this would have been caught during smp parsing.
+ */
+void s390_topology_new_cpu(int core_id)
+{
+ const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyBook *book;
+ S390TopologySocket *socket;
+ S390TopologyCores *cores;
+ int cores_per_socket, sock_idx;
+ int origin, bit;
+
+ book = s390_get_topology();
+
+ cores_per_socket = ms->smp.max_cpus / ms->smp.sockets;
+
+ sock_idx = (core_id / cores_per_socket);
+ socket = s390_get_socket(book, sock_idx);
+
+ /*
+ * At the core level, each CPU is represented by a bit in a 64bit
+ * unsigned long. Set on plug and clear on unplug of a CPU.
+ * The firmware assume that all CPU in the core description have the same
+ * type, polarization and are all dedicated or shared.
+ * In the case a socket contains CPU with different type, polarization
+ * or dedication then they will be defined in different CPU containers.
+ * Currently we assume all CPU are identical and the only reason to have
+ * several S390TopologyCores inside a socket is to have more than 64 CPUs
+ * in that case the origin field, representing the offset of the first CPU
+ * in the CPU container allows to represent up to the maximal number of
+ * CPU inside several CPU containers inside the socket container.
+ */
+ origin = 64 * (core_id / 64);
+
+ cores = s390_get_cores(socket, origin);
+
+ bit = 63 - (core_id - origin);
+ set_bit(bit, &cores->mask);
+ cores->origin = origin;
+}
+
+/*
+ * Setting the first topology: 1 book, 1 socket
+ * This is enough for 64 cores if the topology is flat (single socket)
+ */
+void s390_topology_setup(MachineState *ms)
+{
+ DeviceState *dev;
+
+ /* Create BOOK bridge device */
+ dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ object_property_add_child(qdev_get_machine(),
+ TYPE_S390_TOPOLOGY_BOOK, OBJECT(dev));
+ sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+}
+
+S390TopologyBook *s390_get_topology(void)
+{
+ static S390TopologyBook *book;
+
+ if (!book) {
+ book = S390_TOPOLOGY_BOOK(
+ object_resolve_path(TYPE_S390_TOPOLOGY_BOOK, NULL));
+ assert(book != NULL);
+ }
+
+ return book;
+}
+
+/* --- CORES Definitions --- */
+
+static Property s390_topology_cores_properties[] = {
+ DEFINE_PROP_BOOL("dedicated", S390TopologyCores, dedicated, false),
+ DEFINE_PROP_UINT8("polarity", S390TopologyCores, polarity,
+ S390_TOPOLOGY_POLARITY_H),
+ DEFINE_PROP_UINT8("cputype", S390TopologyCores, cputype,
+ S390_TOPOLOGY_CPU_TYPE),
+ DEFINE_PROP_UINT16("origin", S390TopologyCores, origin, 0),
+ DEFINE_PROP_UINT64("mask", S390TopologyCores, mask, 0),
+ DEFINE_PROP_UINT8("id", S390TopologyCores, id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void cpu_cores_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ device_class_set_props(dc, s390_topology_cores_properties);
+ hc->unplug = qdev_simple_device_unplug_cb;
+ dc->bus_type = TYPE_S390_TOPOLOGY_SOCKET_BUS;
+ dc->desc = "topology cpu entry";
+}
+
+static const TypeInfo cpu_cores_info = {
+ .name = TYPE_S390_TOPOLOGY_CORES,
+ .parent = TYPE_DEVICE,
+ .instance_size = sizeof(S390TopologyCores),
+ .class_init = cpu_cores_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+/* --- SOCKETS Definitions --- */
+static Property s390_topology_socket_properties[] = {
+ DEFINE_PROP_UINT8("socket_id", S390TopologySocket, socket_id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static char *socket_bus_get_dev_path(DeviceState *dev)
+{
+ S390TopologySocket *socket = S390_TOPOLOGY_SOCKET(dev);
+ DeviceState *book = dev->parent_bus->parent;
+ char *id = qdev_get_dev_path(book);
+ char *ret;
+
+ if (id) {
+ ret = g_strdup_printf("%s:%02d", id, socket->socket_id);
+ g_free(id);
+ } else {
+ ret = g_strdup_printf("_:%02d", socket->socket_id);
+ }
+
+ return ret;
+}
+
+static void socket_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = socket_bus_get_dev_path;
+ k->max_dev = S390_MAX_SOCKETS;
+}
+
+static const TypeInfo socket_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_SOCKET_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = socket_bus_class_init,
+};
+
+static void s390_socket_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologySocket *socket = S390_TOPOLOGY_SOCKET(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_SOCKET_BUS, dev,
+ TYPE_S390_TOPOLOGY_SOCKET_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ socket->bus = bus;
+}
+
+static void socket_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->bus_type = TYPE_S390_TOPOLOGY_BOOK_BUS;
+ dc->realize = s390_socket_device_realize;
+ device_class_set_props(dc, s390_topology_socket_properties);
+ dc->desc = "topology socket";
+}
+
+static const TypeInfo socket_info = {
+ .name = TYPE_S390_TOPOLOGY_SOCKET,
+ .parent = TYPE_DEVICE,
+ .instance_size = sizeof(S390TopologySocket),
+ .class_init = socket_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+static char *book_bus_get_dev_path(DeviceState *dev)
+{
+ return g_strdup_printf("00");
+}
+
+static void book_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = book_bus_get_dev_path;
+ k->max_dev = S390_MAX_BOOKS;
+}
+
+static const TypeInfo book_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_BOOK_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = book_bus_class_init,
+};
+
+static void s390_book_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyBook *book = S390_TOPOLOGY_BOOK(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_BOOK_BUS, dev,
+ TYPE_S390_TOPOLOGY_BOOK_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ book->bus = bus;
+}
+
+static void book_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_book_device_realize;
+ dc->desc = "topology book";
+}
+
+static const TypeInfo book_info = {
+ .name = TYPE_S390_TOPOLOGY_BOOK,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyBook),
+ .class_init = book_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+static void topology_register(void)
+{
+ type_register_static(&cpu_cores_info);
+ type_register_static(&socket_bus_info);
+ type_register_static(&socket_info);
+ type_register_static(&book_bus_info);
+ type_register_static(&book_info);
+}
+
+type_init(topology_register);
diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
index 28484256ec..74678861cf 100644
--- a/hw/s390x/meson.build
+++ b/hw/s390x/meson.build
@@ -2,6 +2,7 @@ s390x_ss = ss.source_set()
s390x_ss.add(files(
'ap-bridge.c',
'ap-device.c',
+ 'cpu-topology.c',
'ccw-device.c',
'css-bridge.c',
'css.c',
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 84e3e63c43..ae8e99d0b0 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -42,6 +42,7 @@
#include "sysemu/sysemu.h"
#include "hw/s390x/pv.h"
#include "migration/blocker.h"
+#include "hw/s390x/cpu-topology.h"
static Error *pv_mig_blocker;
@@ -88,6 +89,7 @@ static void s390_init_cpus(MachineState *machine)
/* initialize possible_cpus */
mc->possible_cpu_arch_ids(machine);
+ s390_topology_setup(machine);
for (i = 0; i < machine->smp.cpus; i++) {
s390x_new_cpu(machine->cpu_type, i, &error_fatal);
}
@@ -305,6 +307,8 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ s390_topology_new_cpu(cpu->env.core_id);
+
if (dev->hotplugged) {
raise_irq_cpu_hotplug();
}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 0000000000..e6e013a8b8
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,74 @@
+/*
+ * CPU Topology
+ *
+ * Copyright 2021 IBM Corp.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define S390_TOPOLOGY_CPU_TYPE 0x03
+
+#define S390_TOPOLOGY_POLARITY_H 0x00
+#define S390_TOPOLOGY_POLARITY_VL 0x01
+#define S390_TOPOLOGY_POLARITY_VM 0x02
+#define S390_TOPOLOGY_POLARITY_VH 0x03
+
+#define TYPE_S390_TOPOLOGY_CORES "topology cores"
+ /*
+ * Each CPU inside a socket will be represented by a bit in a 64bit
+ * unsigned long. Set on plug and clear on unplug of a CPU.
+ * All CPU inside a mask share the same dedicated, polarity and
+ * cputype values.
+ * The origin is the offset of the first CPU in a mask.
+ */
+struct S390TopologyCores {
+ DeviceState parent_obj;
+ uint8_t id;
+ bool dedicated;
+ uint8_t polarity;
+ uint8_t cputype;
+ uint16_t origin;
+ uint64_t mask;
+ int cnt;
+};
+typedef struct S390TopologyCores S390TopologyCores;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyCores, S390_TOPOLOGY_CORES)
+
+#define TYPE_S390_TOPOLOGY_SOCKET "topology socket"
+#define TYPE_S390_TOPOLOGY_SOCKET_BUS "socket-bus"
+struct S390TopologySocket {
+ DeviceState parent_obj;
+ BusState *bus;
+ uint8_t socket_id;
+ int cnt;
+};
+typedef struct S390TopologySocket S390TopologySocket;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologySocket, S390_TOPOLOGY_SOCKET)
+#define S390_MAX_SOCKETS 4
+
+#define TYPE_S390_TOPOLOGY_BOOK "topology book"
+#define TYPE_S390_TOPOLOGY_BOOK_BUS "book-bus"
+struct S390TopologyBook {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t book_id;
+ int cnt;
+};
+typedef struct S390TopologyBook S390TopologyBook;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
+#define S390_MAX_BOOKS 1
+
+S390TopologyBook *s390_init_topology(void);
+
+S390TopologyBook *s390_get_topology(void);
+void s390_topology_setup(MachineState *ms);
+void s390_topology_new_cpu(int core_id);
+
+#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index a75e559134..e1672158f1 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -564,6 +564,53 @@ typedef union SysIB {
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+ uint8_t nl;
+ uint8_t reserved0[3];
+ uint8_t reserved1:5;
+ uint8_t dedicated:1;
+ uint8_t polarity:2;
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
+} SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
+} QEMU_PACKED SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* Generic Topology List Entry */
+typedef union SysIBTl_entry {
+ uint8_t nl;
+ SysIBTl_container container;
+ SysIBTl_cpu cpu;
+} SysIBTl_entry;
+
+#define TOPOLOGY_NR_MAG 6
+#define TOPOLOGY_NR_MAG6 0
+#define TOPOLOGY_NR_MAG5 1
+#define TOPOLOGY_NR_MAG4 2
+#define TOPOLOGY_NR_MAG3 3
+#define TOPOLOGY_NR_MAG2 4
+#define TOPOLOGY_NR_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+ uint8_t res0[2];
+ uint16_t length;
+ uint8_t mag[TOPOLOGY_NR_MAG];
+ uint8_t res1;
+ uint8_t mnest;
+ uint32_t res2;
+ SysIBTl_entry tle[0];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
#define ASCE_SUBSPACE 0x200 /* subspace group control */
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 02/11] s390x: topology: CPU topology objects and structures
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
We use new objects to have a dynamic administration of the CPU topology.
The highest level object in this implementation is the s390 book and
in this first implementation of CPU topology for S390 we have a single
book.
The book is built as a SYSBUS bridge during the CPU initialization.
Other objects, sockets and core will be built after the parsing
of the QEMU -smp argument.
Every object under this single book will be build dynamically
immediately after a CPU has be realized if it is needed.
The CPU will fill the sockets once after the other, according to the
number of core per socket defined during the smp parsing.
Each CPU inside a socket will be represented by a bit in a 64bit
unsigned long. Set on plug and clear on unplug of a CPU.
For the S390 CPU topology, thread and cores are merged into
topology cores and the number of topology cores is the multiplication
of cores by the numbers of threads.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 361 ++++++++++++++++++++++++++++++++
hw/s390x/meson.build | 1 +
hw/s390x/s390-virtio-ccw.c | 4 +
include/hw/s390x/cpu-topology.h | 74 +++++++
target/s390x/cpu.h | 47 +++++
5 files changed, 487 insertions(+)
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 include/hw/s390x/cpu-topology.h
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 0000000000..b7131b4ac3
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,361 @@
+/*
+ * CPU Topology
+ *
+ * Copyright 2021 IBM Corp.
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+
+static S390TopologyCores *s390_create_cores(S390TopologySocket *socket,
+ int origin)
+{
+ DeviceState *dev;
+ S390TopologyCores *cores;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (socket->bus->num_children >= (ms->smp.cores * ms->smp.threads)) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_CORES);
+ qdev_realize_and_unref(dev, socket->bus, &error_fatal);
+
+ cores = S390_TOPOLOGY_CORES(dev);
+ cores->origin = origin;
+ socket->cnt += 1;
+
+ return cores;
+}
+
+static S390TopologySocket *s390_create_socket(S390TopologyBook *book, int id)
+{
+ DeviceState *dev;
+ S390TopologySocket *socket;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (book->bus->num_children >= ms->smp.sockets) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_SOCKET);
+ qdev_realize_and_unref(dev, book->bus, &error_fatal);
+
+ socket = S390_TOPOLOGY_SOCKET(dev);
+ socket->socket_id = id;
+ book->cnt++;
+
+ return socket;
+}
+
+/*
+ * s390_get_cores:
+ * @socket: the socket to search into
+ * @origin: the origin specified for the S390TopologyCores
+ *
+ * returns a pointer to a S390TopologyCores structure within a socket having
+ * the specified origin.
+ * First search if the socket is already containing the S390TopologyCores
+ * structure and if not create one with this origin.
+ */
+static S390TopologyCores *s390_get_cores(S390TopologySocket *socket, int origin)
+{
+ S390TopologyCores *cores;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &socket->bus->children, sibling) {
+ cores = S390_TOPOLOGY_CORES(kid->child);
+ if (cores->origin == origin) {
+ return cores;
+ }
+ }
+ return s390_create_cores(socket, origin);
+}
+
+/*
+ * s390_get_socket:
+ * @book: The book to search into
+ * @socket_id: the identifier of the socket to search for
+ *
+ * returns a pointer to a S390TopologySocket structure within a book having
+ * the specified socket_id.
+ * First search if the book is already containing the S390TopologySocket
+ * structure and if not create one with this socket_id.
+ */
+static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
+ int socket_id)
+{
+ S390TopologySocket *socket;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &book->bus->children, sibling) {
+ socket = S390_TOPOLOGY_SOCKET(kid->child);
+ if (socket->socket_id == socket_id) {
+ return socket;
+ }
+ }
+ return s390_create_socket(book, socket_id);
+}
+
+/*
+ * s390_topology_new_cpu:
+ * @core_id: the core ID is machine wide
+ *
+ * We have a single book returned by s390_get_topology(),
+ * then we build the hierarchy on demand.
+ * Note that we do not destroy the hierarchy on error creating
+ * an entry in the topology, we just keep it empty.
+ * We do not need to worry about not finding a topology level
+ * entry this would have been caught during smp parsing.
+ */
+void s390_topology_new_cpu(int core_id)
+{
+ const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyBook *book;
+ S390TopologySocket *socket;
+ S390TopologyCores *cores;
+ int cores_per_socket, sock_idx;
+ int origin, bit;
+
+ book = s390_get_topology();
+
+ cores_per_socket = ms->smp.max_cpus / ms->smp.sockets;
+
+ sock_idx = (core_id / cores_per_socket);
+ socket = s390_get_socket(book, sock_idx);
+
+ /*
+ * At the core level, each CPU is represented by a bit in a 64bit
+ * unsigned long. Set on plug and clear on unplug of a CPU.
+ * The firmware assume that all CPU in the core description have the same
+ * type, polarization and are all dedicated or shared.
+ * In the case a socket contains CPU with different type, polarization
+ * or dedication then they will be defined in different CPU containers.
+ * Currently we assume all CPU are identical and the only reason to have
+ * several S390TopologyCores inside a socket is to have more than 64 CPUs
+ * in that case the origin field, representing the offset of the first CPU
+ * in the CPU container allows to represent up to the maximal number of
+ * CPU inside several CPU containers inside the socket container.
+ */
+ origin = 64 * (core_id / 64);
+
+ cores = s390_get_cores(socket, origin);
+
+ bit = 63 - (core_id - origin);
+ set_bit(bit, &cores->mask);
+ cores->origin = origin;
+}
+
+/*
+ * Setting the first topology: 1 book, 1 socket
+ * This is enough for 64 cores if the topology is flat (single socket)
+ */
+void s390_topology_setup(MachineState *ms)
+{
+ DeviceState *dev;
+
+ /* Create BOOK bridge device */
+ dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ object_property_add_child(qdev_get_machine(),
+ TYPE_S390_TOPOLOGY_BOOK, OBJECT(dev));
+ sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+}
+
+S390TopologyBook *s390_get_topology(void)
+{
+ static S390TopologyBook *book;
+
+ if (!book) {
+ book = S390_TOPOLOGY_BOOK(
+ object_resolve_path(TYPE_S390_TOPOLOGY_BOOK, NULL));
+ assert(book != NULL);
+ }
+
+ return book;
+}
+
+/* --- CORES Definitions --- */
+
+static Property s390_topology_cores_properties[] = {
+ DEFINE_PROP_BOOL("dedicated", S390TopologyCores, dedicated, false),
+ DEFINE_PROP_UINT8("polarity", S390TopologyCores, polarity,
+ S390_TOPOLOGY_POLARITY_H),
+ DEFINE_PROP_UINT8("cputype", S390TopologyCores, cputype,
+ S390_TOPOLOGY_CPU_TYPE),
+ DEFINE_PROP_UINT16("origin", S390TopologyCores, origin, 0),
+ DEFINE_PROP_UINT64("mask", S390TopologyCores, mask, 0),
+ DEFINE_PROP_UINT8("id", S390TopologyCores, id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void cpu_cores_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ device_class_set_props(dc, s390_topology_cores_properties);
+ hc->unplug = qdev_simple_device_unplug_cb;
+ dc->bus_type = TYPE_S390_TOPOLOGY_SOCKET_BUS;
+ dc->desc = "topology cpu entry";
+}
+
+static const TypeInfo cpu_cores_info = {
+ .name = TYPE_S390_TOPOLOGY_CORES,
+ .parent = TYPE_DEVICE,
+ .instance_size = sizeof(S390TopologyCores),
+ .class_init = cpu_cores_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+/* --- SOCKETS Definitions --- */
+static Property s390_topology_socket_properties[] = {
+ DEFINE_PROP_UINT8("socket_id", S390TopologySocket, socket_id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static char *socket_bus_get_dev_path(DeviceState *dev)
+{
+ S390TopologySocket *socket = S390_TOPOLOGY_SOCKET(dev);
+ DeviceState *book = dev->parent_bus->parent;
+ char *id = qdev_get_dev_path(book);
+ char *ret;
+
+ if (id) {
+ ret = g_strdup_printf("%s:%02d", id, socket->socket_id);
+ g_free(id);
+ } else {
+ ret = g_strdup_printf("_:%02d", socket->socket_id);
+ }
+
+ return ret;
+}
+
+static void socket_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = socket_bus_get_dev_path;
+ k->max_dev = S390_MAX_SOCKETS;
+}
+
+static const TypeInfo socket_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_SOCKET_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = socket_bus_class_init,
+};
+
+static void s390_socket_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologySocket *socket = S390_TOPOLOGY_SOCKET(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_SOCKET_BUS, dev,
+ TYPE_S390_TOPOLOGY_SOCKET_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ socket->bus = bus;
+}
+
+static void socket_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->bus_type = TYPE_S390_TOPOLOGY_BOOK_BUS;
+ dc->realize = s390_socket_device_realize;
+ device_class_set_props(dc, s390_topology_socket_properties);
+ dc->desc = "topology socket";
+}
+
+static const TypeInfo socket_info = {
+ .name = TYPE_S390_TOPOLOGY_SOCKET,
+ .parent = TYPE_DEVICE,
+ .instance_size = sizeof(S390TopologySocket),
+ .class_init = socket_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+static char *book_bus_get_dev_path(DeviceState *dev)
+{
+ return g_strdup_printf("00");
+}
+
+static void book_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = book_bus_get_dev_path;
+ k->max_dev = S390_MAX_BOOKS;
+}
+
+static const TypeInfo book_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_BOOK_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = book_bus_class_init,
+};
+
+static void s390_book_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyBook *book = S390_TOPOLOGY_BOOK(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_BOOK_BUS, dev,
+ TYPE_S390_TOPOLOGY_BOOK_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ book->bus = bus;
+}
+
+static void book_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_book_device_realize;
+ dc->desc = "topology book";
+}
+
+static const TypeInfo book_info = {
+ .name = TYPE_S390_TOPOLOGY_BOOK,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyBook),
+ .class_init = book_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
+static void topology_register(void)
+{
+ type_register_static(&cpu_cores_info);
+ type_register_static(&socket_bus_info);
+ type_register_static(&socket_info);
+ type_register_static(&book_bus_info);
+ type_register_static(&book_info);
+}
+
+type_init(topology_register);
diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
index 28484256ec..74678861cf 100644
--- a/hw/s390x/meson.build
+++ b/hw/s390x/meson.build
@@ -2,6 +2,7 @@ s390x_ss = ss.source_set()
s390x_ss.add(files(
'ap-bridge.c',
'ap-device.c',
+ 'cpu-topology.c',
'ccw-device.c',
'css-bridge.c',
'css.c',
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 84e3e63c43..ae8e99d0b0 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -42,6 +42,7 @@
#include "sysemu/sysemu.h"
#include "hw/s390x/pv.h"
#include "migration/blocker.h"
+#include "hw/s390x/cpu-topology.h"
static Error *pv_mig_blocker;
@@ -88,6 +89,7 @@ static void s390_init_cpus(MachineState *machine)
/* initialize possible_cpus */
mc->possible_cpu_arch_ids(machine);
+ s390_topology_setup(machine);
for (i = 0; i < machine->smp.cpus; i++) {
s390x_new_cpu(machine->cpu_type, i, &error_fatal);
}
@@ -305,6 +307,8 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ s390_topology_new_cpu(cpu->env.core_id);
+
if (dev->hotplugged) {
raise_irq_cpu_hotplug();
}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 0000000000..e6e013a8b8
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,74 @@
+/*
+ * CPU Topology
+ *
+ * Copyright 2021 IBM Corp.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define S390_TOPOLOGY_CPU_TYPE 0x03
+
+#define S390_TOPOLOGY_POLARITY_H 0x00
+#define S390_TOPOLOGY_POLARITY_VL 0x01
+#define S390_TOPOLOGY_POLARITY_VM 0x02
+#define S390_TOPOLOGY_POLARITY_VH 0x03
+
+#define TYPE_S390_TOPOLOGY_CORES "topology cores"
+ /*
+ * Each CPU inside a socket will be represented by a bit in a 64bit
+ * unsigned long. Set on plug and clear on unplug of a CPU.
+ * All CPU inside a mask share the same dedicated, polarity and
+ * cputype values.
+ * The origin is the offset of the first CPU in a mask.
+ */
+struct S390TopologyCores {
+ DeviceState parent_obj;
+ uint8_t id;
+ bool dedicated;
+ uint8_t polarity;
+ uint8_t cputype;
+ uint16_t origin;
+ uint64_t mask;
+ int cnt;
+};
+typedef struct S390TopologyCores S390TopologyCores;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyCores, S390_TOPOLOGY_CORES)
+
+#define TYPE_S390_TOPOLOGY_SOCKET "topology socket"
+#define TYPE_S390_TOPOLOGY_SOCKET_BUS "socket-bus"
+struct S390TopologySocket {
+ DeviceState parent_obj;
+ BusState *bus;
+ uint8_t socket_id;
+ int cnt;
+};
+typedef struct S390TopologySocket S390TopologySocket;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologySocket, S390_TOPOLOGY_SOCKET)
+#define S390_MAX_SOCKETS 4
+
+#define TYPE_S390_TOPOLOGY_BOOK "topology book"
+#define TYPE_S390_TOPOLOGY_BOOK_BUS "book-bus"
+struct S390TopologyBook {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t book_id;
+ int cnt;
+};
+typedef struct S390TopologyBook S390TopologyBook;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
+#define S390_MAX_BOOKS 1
+
+S390TopologyBook *s390_init_topology(void);
+
+S390TopologyBook *s390_get_topology(void);
+void s390_topology_setup(MachineState *ms);
+void s390_topology_new_cpu(int core_id);
+
+#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index a75e559134..e1672158f1 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -564,6 +564,53 @@ typedef union SysIB {
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+ uint8_t nl;
+ uint8_t reserved0[3];
+ uint8_t reserved1:5;
+ uint8_t dedicated:1;
+ uint8_t polarity:2;
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
+} SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
+} QEMU_PACKED SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* Generic Topology List Entry */
+typedef union SysIBTl_entry {
+ uint8_t nl;
+ SysIBTl_container container;
+ SysIBTl_cpu cpu;
+} SysIBTl_entry;
+
+#define TOPOLOGY_NR_MAG 6
+#define TOPOLOGY_NR_MAG6 0
+#define TOPOLOGY_NR_MAG5 1
+#define TOPOLOGY_NR_MAG4 2
+#define TOPOLOGY_NR_MAG3 3
+#define TOPOLOGY_NR_MAG2 4
+#define TOPOLOGY_NR_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+ uint8_t res0[2];
+ uint16_t length;
+ uint8_t mag[TOPOLOGY_NR_MAG];
+ uint8_t res1;
+ uint8_t mnest;
+ uint32_t res2;
+ SysIBTl_entry tle[0];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
#define ASCE_SUBSPACE 0x200 /* subspace group control */
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 03/11] s390x: topology: implementating Store Topology System Information
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
The handling of STSI is enhanced with the interception of the
function code 15 for storing CPU topology.
Using the objects built during the pluging of CPU, we build the
SYSIB 15_1_x structures.
With this patch the maximum MNEST level is 2, this is also
the only level allowed and only SYSIB 15_1_2 will be built.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 12 ++--
target/s390x/cpu.h | 1 +
target/s390x/cpu_topology.c | 112 ++++++++++++++++++++++++++++++++++++
target/s390x/kvm/kvm.c | 5 ++
target/s390x/meson.build | 1 +
5 files changed, 124 insertions(+), 7 deletions(-)
create mode 100644 target/s390x/cpu_topology.c
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index b7131b4ac3..74e04fd68e 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -127,15 +127,14 @@ void s390_topology_new_cpu(int core_id)
S390TopologyBook *book;
S390TopologySocket *socket;
S390TopologyCores *cores;
- int cores_per_socket, sock_idx;
int origin, bit;
+ int nb_cores_per_socket;
book = s390_get_topology();
- cores_per_socket = ms->smp.max_cpus / ms->smp.sockets;
-
- sock_idx = (core_id / cores_per_socket);
- socket = s390_get_socket(book, sock_idx);
+ /* Cores for the S390 topology are cores and threads of the QEMU topology */
+ nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
+ socket = s390_get_socket(book, core_id / nb_cores_per_socket);
/*
* At the core level, each CPU is represented by a bit in a 64bit
@@ -151,12 +150,11 @@ void s390_topology_new_cpu(int core_id)
* CPU inside several CPU containers inside the socket container.
*/
origin = 64 * (core_id / 64);
-
cores = s390_get_cores(socket, origin);
+ cores->origin = origin;
bit = 63 - (core_id - origin);
set_bit(bit, &cores->mask);
- cores->origin = origin;
}
/*
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index e1672158f1..71ef0b7e95 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -892,4 +892,5 @@ typedef S390CPU ArchCPU;
#include "exec/cpu-all.h"
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
#endif
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
new file mode 100644
index 0000000000..7f6db18829
--- /dev/null
+++ b/target/s390x/cpu_topology.c
@@ -0,0 +1,112 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2021
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+
+static int stsi_15_container(void *p, int nl, int id)
+{
+ SysIBTl_container *tle = (SysIBTl_container *)p;
+
+ tle->nl = nl;
+ tle->id = id;
+
+ return sizeof(*tle);
+}
+
+static int stsi_15_cpus(void *p, S390TopologyCores *cd)
+{
+ SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+
+ tle->nl = 0;
+ tle->dedicated = cd->dedicated;
+ tle->polarity = cd->polarity;
+ tle->type = cd->cputype;
+ tle->origin = be16_to_cpu(cd->origin);
+ tle->mask = be64_to_cpu(cd->mask);
+
+ return sizeof(*tle);
+}
+
+static int set_socket(const MachineState *ms, void *p,
+ S390TopologySocket *socket)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ len += stsi_15_container(p, 1, socket->socket_id);
+ p += len;
+
+ QTAILQ_FOREACH_REVERSE(kid, &socket->bus->children, sibling) {
+ l = stsi_15_cpus(p, S390_TOPOLOGY_CORES(kid->child));
+ p += l;
+ len += l;
+ }
+ return len;
+}
+
+static void setup_stsi(const MachineState *ms, void *p, int level)
+{
+ S390TopologyBook *book;
+ SysIB_151x *sysib;
+ BusChild *kid;
+ int len, l;
+
+ sysib = (SysIB_151x *)p;
+ sysib->mnest = level;
+ sysib->mag[TOPOLOGY_NR_MAG2] = ms->smp.sockets;
+ sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
+
+ book = s390_get_topology();
+ len = sizeof(SysIB_151x);
+ p += len;
+
+ QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
+ l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child));
+ p += l;
+ len += l;
+ }
+
+ sysib->length = be16_to_cpu(len);
+}
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+ const MachineState *machine = MACHINE(qdev_get_machine());
+ void *p;
+ int ret, cc;
+
+ /*
+ * Until the SCLP STSI Facility reporting the MNEST value is used,
+ * a sel2 value of 2 is the only value allowed in STSI 15.1.x.
+ */
+ if (sel2 != 2) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ p = g_malloc0(TARGET_PAGE_SIZE);
+
+ setup_stsi(machine, p, 2);
+
+ if (s390_is_pv()) {
+ ret = s390_cpu_pv_mem_write(cpu, 0, p, TARGET_PAGE_SIZE);
+ } else {
+ ret = s390_cpu_virt_mem_write(cpu, addr, ar, p, TARGET_PAGE_SIZE);
+ }
+ cc = ret ? 3 : 0;
+ setcc(cpu, cc);
+ g_free(p);
+}
+
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 6acf14d5ec..27b3fbfa09 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -52,6 +52,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/s390-virtio-hcall.h"
#include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
#ifndef DEBUG_KVM
#define DEBUG_KVM 0
@@ -1910,6 +1911,10 @@ static int handle_stsi(S390CPU *cpu)
/* Only sysib 3.2.2 needs post-handling for now. */
insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+ run->s390_stsi.ar);
+ return 0;
default:
return 0;
}
diff --git a/target/s390x/meson.build b/target/s390x/meson.build
index 84c1402a6a..890ccfa789 100644
--- a/target/s390x/meson.build
+++ b/target/s390x/meson.build
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
'sigp.c',
'cpu-sysemu.c',
'cpu_models_sysemu.c',
+ 'cpu_topology.c',
))
s390x_user_ss = ss.source_set()
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 03/11] s390x: topology: implementating Store Topology System Information
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
The handling of STSI is enhanced with the interception of the
function code 15 for storing CPU topology.
Using the objects built during the pluging of CPU, we build the
SYSIB 15_1_x structures.
With this patch the maximum MNEST level is 2, this is also
the only level allowed and only SYSIB 15_1_2 will be built.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 12 ++--
target/s390x/cpu.h | 1 +
target/s390x/cpu_topology.c | 112 ++++++++++++++++++++++++++++++++++++
target/s390x/kvm/kvm.c | 5 ++
target/s390x/meson.build | 1 +
5 files changed, 124 insertions(+), 7 deletions(-)
create mode 100644 target/s390x/cpu_topology.c
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index b7131b4ac3..74e04fd68e 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -127,15 +127,14 @@ void s390_topology_new_cpu(int core_id)
S390TopologyBook *book;
S390TopologySocket *socket;
S390TopologyCores *cores;
- int cores_per_socket, sock_idx;
int origin, bit;
+ int nb_cores_per_socket;
book = s390_get_topology();
- cores_per_socket = ms->smp.max_cpus / ms->smp.sockets;
-
- sock_idx = (core_id / cores_per_socket);
- socket = s390_get_socket(book, sock_idx);
+ /* Cores for the S390 topology are cores and threads of the QEMU topology */
+ nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
+ socket = s390_get_socket(book, core_id / nb_cores_per_socket);
/*
* At the core level, each CPU is represented by a bit in a 64bit
@@ -151,12 +150,11 @@ void s390_topology_new_cpu(int core_id)
* CPU inside several CPU containers inside the socket container.
*/
origin = 64 * (core_id / 64);
-
cores = s390_get_cores(socket, origin);
+ cores->origin = origin;
bit = 63 - (core_id - origin);
set_bit(bit, &cores->mask);
- cores->origin = origin;
}
/*
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index e1672158f1..71ef0b7e95 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -892,4 +892,5 @@ typedef S390CPU ArchCPU;
#include "exec/cpu-all.h"
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
#endif
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
new file mode 100644
index 0000000000..7f6db18829
--- /dev/null
+++ b/target/s390x/cpu_topology.c
@@ -0,0 +1,112 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2021
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+
+static int stsi_15_container(void *p, int nl, int id)
+{
+ SysIBTl_container *tle = (SysIBTl_container *)p;
+
+ tle->nl = nl;
+ tle->id = id;
+
+ return sizeof(*tle);
+}
+
+static int stsi_15_cpus(void *p, S390TopologyCores *cd)
+{
+ SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+
+ tle->nl = 0;
+ tle->dedicated = cd->dedicated;
+ tle->polarity = cd->polarity;
+ tle->type = cd->cputype;
+ tle->origin = be16_to_cpu(cd->origin);
+ tle->mask = be64_to_cpu(cd->mask);
+
+ return sizeof(*tle);
+}
+
+static int set_socket(const MachineState *ms, void *p,
+ S390TopologySocket *socket)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ len += stsi_15_container(p, 1, socket->socket_id);
+ p += len;
+
+ QTAILQ_FOREACH_REVERSE(kid, &socket->bus->children, sibling) {
+ l = stsi_15_cpus(p, S390_TOPOLOGY_CORES(kid->child));
+ p += l;
+ len += l;
+ }
+ return len;
+}
+
+static void setup_stsi(const MachineState *ms, void *p, int level)
+{
+ S390TopologyBook *book;
+ SysIB_151x *sysib;
+ BusChild *kid;
+ int len, l;
+
+ sysib = (SysIB_151x *)p;
+ sysib->mnest = level;
+ sysib->mag[TOPOLOGY_NR_MAG2] = ms->smp.sockets;
+ sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
+
+ book = s390_get_topology();
+ len = sizeof(SysIB_151x);
+ p += len;
+
+ QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
+ l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child));
+ p += l;
+ len += l;
+ }
+
+ sysib->length = be16_to_cpu(len);
+}
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+ const MachineState *machine = MACHINE(qdev_get_machine());
+ void *p;
+ int ret, cc;
+
+ /*
+ * Until the SCLP STSI Facility reporting the MNEST value is used,
+ * a sel2 value of 2 is the only value allowed in STSI 15.1.x.
+ */
+ if (sel2 != 2) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ p = g_malloc0(TARGET_PAGE_SIZE);
+
+ setup_stsi(machine, p, 2);
+
+ if (s390_is_pv()) {
+ ret = s390_cpu_pv_mem_write(cpu, 0, p, TARGET_PAGE_SIZE);
+ } else {
+ ret = s390_cpu_virt_mem_write(cpu, addr, ar, p, TARGET_PAGE_SIZE);
+ }
+ cc = ret ? 3 : 0;
+ setcc(cpu, cc);
+ g_free(p);
+}
+
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 6acf14d5ec..27b3fbfa09 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -52,6 +52,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/s390-virtio-hcall.h"
#include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
#ifndef DEBUG_KVM
#define DEBUG_KVM 0
@@ -1910,6 +1911,10 @@ static int handle_stsi(S390CPU *cpu)
/* Only sysib 3.2.2 needs post-handling for now. */
insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+ run->s390_stsi.ar);
+ return 0;
default:
return 0;
}
diff --git a/target/s390x/meson.build b/target/s390x/meson.build
index 84c1402a6a..890ccfa789 100644
--- a/target/s390x/meson.build
+++ b/target/s390x/meson.build
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
'sigp.c',
'cpu-sysemu.c',
'cpu_models_sysemu.c',
+ 'cpu_topology.c',
))
s390x_user_ss = ss.source_set()
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 04/11] s390x: CPU topology: CPU topology migration
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Both source and target must have the same configuration
regarding the activation of Perform Topology Function
and Store Topology System Information.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
target/s390x/cpu.h | 2 ++
target/s390x/cpu_features_def.h.inc | 1 +
target/s390x/cpu_models.c | 2 ++
target/s390x/gen-features.c | 3 ++
target/s390x/kvm/kvm.c | 6 ++++
target/s390x/machine.c | 48 +++++++++++++++++++++++++++++
6 files changed, 62 insertions(+)
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 71ef0b7e95..67933bc6bb 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -150,6 +150,8 @@ struct CPUS390XState {
/* currently processed sigp order */
uint8_t sigp_order;
+ /* Using Perform CPU Topology Function*/
+ bool using_ptf;
};
static inline uint64_t *get_freg(CPUS390XState *cs, int nr)
diff --git a/target/s390x/cpu_features_def.h.inc b/target/s390x/cpu_features_def.h.inc
index e86662bb3b..27e8904978 100644
--- a/target/s390x/cpu_features_def.h.inc
+++ b/target/s390x/cpu_features_def.h.inc
@@ -146,6 +146,7 @@ DEF_FEAT(SIE_CEI, "cei", SCLP_CPU, 43, "SIE: Conditional-external-interception f
DEF_FEAT(DAT_ENH_2, "dateh2", MISC, 0, "DAT-enhancement facility 2")
DEF_FEAT(CMM, "cmm", MISC, 0, "Collaborative-memory-management facility")
DEF_FEAT(AP, "ap", MISC, 0, "AP instructions installed")
+DEF_FEAT(CPU_TOPOLOGY, "cpu_topology", MISC, 0, "CPU Topology available")
/* Features exposed via the PLO instruction. */
DEF_FEAT(PLO_CL, "plo-cl", PLO, 0, "PLO Compare and load (32 bit in general registers)")
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 11e06cc51f..1f9ea6479d 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -238,6 +238,7 @@ bool s390_has_feat(S390Feat feat)
if (s390_is_pv()) {
switch (feat) {
+ case S390_FEAT_CPU_TOPOLOGY:
case S390_FEAT_DIAG_318:
case S390_FEAT_HPMA2:
case S390_FEAT_SIE_F2:
@@ -467,6 +468,7 @@ static void check_consistency(const S390CPUModel *model)
{ S390_FEAT_DIAG_318, S390_FEAT_EXTENDED_LENGTH_SCCB },
{ S390_FEAT_NNPA, S390_FEAT_VECTOR },
{ S390_FEAT_RDP, S390_FEAT_LOCAL_TLB_CLEARING },
+ { S390_FEAT_CPU_TOPOLOGY, S390_FEAT_CONFIGURATION_TOPOLOGY },
};
int i;
diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c
index 7cb1a6ec10..377d8aad90 100644
--- a/target/s390x/gen-features.c
+++ b/target/s390x/gen-features.c
@@ -488,6 +488,7 @@ static uint16_t full_GEN9_GA3[] = {
static uint16_t full_GEN10_GA1[] = {
S390_FEAT_EDAT,
S390_FEAT_CONFIGURATION_TOPOLOGY,
+ S390_FEAT_CPU_TOPOLOGY,
S390_FEAT_GROUP_MSA_EXT_2,
S390_FEAT_ESOP,
S390_FEAT_SIE_PFMFI,
@@ -604,6 +605,8 @@ static uint16_t default_GEN9_GA1[] = {
static uint16_t default_GEN10_GA1[] = {
S390_FEAT_EDAT,
S390_FEAT_GROUP_MSA_EXT_2,
+ S390_FEAT_CONFIGURATION_TOPOLOGY,
+ S390_FEAT_CPU_TOPOLOGY,
};
#define default_GEN10_GA2 EmptyFeat
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 27b3fbfa09..7504cbd21b 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -612,6 +612,7 @@ int kvm_arch_put_registers(CPUState *cs, int level)
} else {
/* prefix is only supported via sync regs */
}
+
return 0;
}
@@ -2431,6 +2432,10 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
clear_bit(S390_FEAT_CMM_NT, model->features);
}
+ if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+ set_bit(S390_FEAT_CPU_TOPOLOGY, model->features);
+ }
+
/* bpb needs kernel support for migration, VSIE and reset */
if (!kvm_check_extension(kvm_state, KVM_CAP_S390_BPB)) {
clear_bit(S390_FEAT_BPB, model->features);
@@ -2539,6 +2544,7 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp)
error_setg(errp, "KVM: Error configuring CPU subfunctions: %d", rc);
return;
}
+
/* enable CMM via CMMA */
if (test_bit(S390_FEAT_CMM, model->features)) {
kvm_s390_enable_cmma();
diff --git a/target/s390x/machine.c b/target/s390x/machine.c
index 37a076858c..6036e8e856 100644
--- a/target/s390x/machine.c
+++ b/target/s390x/machine.c
@@ -250,6 +250,53 @@ const VMStateDescription vmstate_diag318 = {
}
};
+static int cpu_topology_presave(void *opaque)
+{
+ S390CPU *cpu = opaque;
+
+ cpu->env.using_ptf = s390_has_feat(S390_FEAT_CPU_TOPOLOGY);
+ return 0;
+}
+
+static int cpu_topology_postload(void *opaque, int version_id)
+{
+ S390CPU *cpu = opaque;
+
+ if ((cpu->env.using_ptf == s390_has_feat(S390_FEAT_CPU_TOPOLOGY)) &&
+ (cpu->env.using_ptf == s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY))) {
+ return 0;
+ }
+ if (cpu->env.using_ptf) {
+ error_report("Target needs CPU Topology enabled");
+ } else {
+ if (s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ error_report("Target is not allowed to enable configuration Topology");
+ }
+ if (s390_has_feat(S390_FEAT_CPU_TOPOLOGY)) {
+ error_report("Target is not allowed to enable CPU Topology");
+ }
+ }
+ return -EINVAL;
+}
+
+static bool cpu_topology_needed(void *opaque)
+{
+ return 1;
+}
+
+const VMStateDescription vmstate_cpu_topology = {
+ .name = "cpu/cpu_topology",
+ .version_id = 1,
+ .pre_save = cpu_topology_presave,
+ .post_load = cpu_topology_postload,
+ .minimum_version_id = 1,
+ .needed = cpu_topology_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_BOOL(env.using_ptf, S390CPU),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
const VMStateDescription vmstate_s390_cpu = {
.name = "cpu",
.post_load = cpu_post_load,
@@ -287,6 +334,7 @@ const VMStateDescription vmstate_s390_cpu = {
&vmstate_bpbc,
&vmstate_etoken,
&vmstate_diag318,
+ &vmstate_cpu_topology,
NULL
},
};
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 04/11] s390x: CPU topology: CPU topology migration
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
Both source and target must have the same configuration
regarding the activation of Perform Topology Function
and Store Topology System Information.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
target/s390x/cpu.h | 2 ++
target/s390x/cpu_features_def.h.inc | 1 +
target/s390x/cpu_models.c | 2 ++
target/s390x/gen-features.c | 3 ++
target/s390x/kvm/kvm.c | 6 ++++
target/s390x/machine.c | 48 +++++++++++++++++++++++++++++
6 files changed, 62 insertions(+)
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 71ef0b7e95..67933bc6bb 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -150,6 +150,8 @@ struct CPUS390XState {
/* currently processed sigp order */
uint8_t sigp_order;
+ /* Using Perform CPU Topology Function*/
+ bool using_ptf;
};
static inline uint64_t *get_freg(CPUS390XState *cs, int nr)
diff --git a/target/s390x/cpu_features_def.h.inc b/target/s390x/cpu_features_def.h.inc
index e86662bb3b..27e8904978 100644
--- a/target/s390x/cpu_features_def.h.inc
+++ b/target/s390x/cpu_features_def.h.inc
@@ -146,6 +146,7 @@ DEF_FEAT(SIE_CEI, "cei", SCLP_CPU, 43, "SIE: Conditional-external-interception f
DEF_FEAT(DAT_ENH_2, "dateh2", MISC, 0, "DAT-enhancement facility 2")
DEF_FEAT(CMM, "cmm", MISC, 0, "Collaborative-memory-management facility")
DEF_FEAT(AP, "ap", MISC, 0, "AP instructions installed")
+DEF_FEAT(CPU_TOPOLOGY, "cpu_topology", MISC, 0, "CPU Topology available")
/* Features exposed via the PLO instruction. */
DEF_FEAT(PLO_CL, "plo-cl", PLO, 0, "PLO Compare and load (32 bit in general registers)")
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 11e06cc51f..1f9ea6479d 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -238,6 +238,7 @@ bool s390_has_feat(S390Feat feat)
if (s390_is_pv()) {
switch (feat) {
+ case S390_FEAT_CPU_TOPOLOGY:
case S390_FEAT_DIAG_318:
case S390_FEAT_HPMA2:
case S390_FEAT_SIE_F2:
@@ -467,6 +468,7 @@ static void check_consistency(const S390CPUModel *model)
{ S390_FEAT_DIAG_318, S390_FEAT_EXTENDED_LENGTH_SCCB },
{ S390_FEAT_NNPA, S390_FEAT_VECTOR },
{ S390_FEAT_RDP, S390_FEAT_LOCAL_TLB_CLEARING },
+ { S390_FEAT_CPU_TOPOLOGY, S390_FEAT_CONFIGURATION_TOPOLOGY },
};
int i;
diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c
index 7cb1a6ec10..377d8aad90 100644
--- a/target/s390x/gen-features.c
+++ b/target/s390x/gen-features.c
@@ -488,6 +488,7 @@ static uint16_t full_GEN9_GA3[] = {
static uint16_t full_GEN10_GA1[] = {
S390_FEAT_EDAT,
S390_FEAT_CONFIGURATION_TOPOLOGY,
+ S390_FEAT_CPU_TOPOLOGY,
S390_FEAT_GROUP_MSA_EXT_2,
S390_FEAT_ESOP,
S390_FEAT_SIE_PFMFI,
@@ -604,6 +605,8 @@ static uint16_t default_GEN9_GA1[] = {
static uint16_t default_GEN10_GA1[] = {
S390_FEAT_EDAT,
S390_FEAT_GROUP_MSA_EXT_2,
+ S390_FEAT_CONFIGURATION_TOPOLOGY,
+ S390_FEAT_CPU_TOPOLOGY,
};
#define default_GEN10_GA2 EmptyFeat
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 27b3fbfa09..7504cbd21b 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -612,6 +612,7 @@ int kvm_arch_put_registers(CPUState *cs, int level)
} else {
/* prefix is only supported via sync regs */
}
+
return 0;
}
@@ -2431,6 +2432,10 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
clear_bit(S390_FEAT_CMM_NT, model->features);
}
+ if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+ set_bit(S390_FEAT_CPU_TOPOLOGY, model->features);
+ }
+
/* bpb needs kernel support for migration, VSIE and reset */
if (!kvm_check_extension(kvm_state, KVM_CAP_S390_BPB)) {
clear_bit(S390_FEAT_BPB, model->features);
@@ -2539,6 +2544,7 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp)
error_setg(errp, "KVM: Error configuring CPU subfunctions: %d", rc);
return;
}
+
/* enable CMM via CMMA */
if (test_bit(S390_FEAT_CMM, model->features)) {
kvm_s390_enable_cmma();
diff --git a/target/s390x/machine.c b/target/s390x/machine.c
index 37a076858c..6036e8e856 100644
--- a/target/s390x/machine.c
+++ b/target/s390x/machine.c
@@ -250,6 +250,53 @@ const VMStateDescription vmstate_diag318 = {
}
};
+static int cpu_topology_presave(void *opaque)
+{
+ S390CPU *cpu = opaque;
+
+ cpu->env.using_ptf = s390_has_feat(S390_FEAT_CPU_TOPOLOGY);
+ return 0;
+}
+
+static int cpu_topology_postload(void *opaque, int version_id)
+{
+ S390CPU *cpu = opaque;
+
+ if ((cpu->env.using_ptf == s390_has_feat(S390_FEAT_CPU_TOPOLOGY)) &&
+ (cpu->env.using_ptf == s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY))) {
+ return 0;
+ }
+ if (cpu->env.using_ptf) {
+ error_report("Target needs CPU Topology enabled");
+ } else {
+ if (s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ error_report("Target is not allowed to enable configuration Topology");
+ }
+ if (s390_has_feat(S390_FEAT_CPU_TOPOLOGY)) {
+ error_report("Target is not allowed to enable CPU Topology");
+ }
+ }
+ return -EINVAL;
+}
+
+static bool cpu_topology_needed(void *opaque)
+{
+ return 1;
+}
+
+const VMStateDescription vmstate_cpu_topology = {
+ .name = "cpu/cpu_topology",
+ .version_id = 1,
+ .pre_save = cpu_topology_presave,
+ .post_load = cpu_topology_postload,
+ .minimum_version_id = 1,
+ .needed = cpu_topology_needed,
+ .fields = (VMStateField[]) {
+ VMSTATE_BOOL(env.using_ptf, S390CPU),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
const VMStateDescription vmstate_s390_cpu = {
.name = "cpu",
.post_load = cpu_post_load,
@@ -287,6 +334,7 @@ const VMStateDescription vmstate_s390_cpu = {
&vmstate_bpbc,
&vmstate_etoken,
&vmstate_diag318,
+ &vmstate_cpu_topology,
NULL
},
};
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 05/11] s390x: kvm: topology: interception of PTF instruction
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervizor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervizor.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/s390-virtio-ccw.c | 50 ++++++++++++++++++++++++++++++
include/hw/s390x/s390-virtio-ccw.h | 6 ++++
target/s390x/kvm/kvm.c | 15 +++++++++
3 files changed, 71 insertions(+)
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index ae8e99d0b0..12903eb2af 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -408,6 +408,56 @@ static void s390_pv_prepare_reset(S390CcwMachineState *ms)
s390_pv_prep_reset();
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 and 1 handle the CPU polarization.
+ * We assume an horizontal topology, the only one supported currently
+ * by Linux, consequently we answer to function code 0, requesting
+ * horizontal polarization that it is already the current polarization
+ * and reject vertical polarization request without further explanation.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+int s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ uint8_t fc = reg & S390_TOPO_FC_MASK;
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ s390_program_interrupt(env, PGM_OPERATION, ra);
+ return 0;
+ }
+
+ if (env->psw.mask & PSW_MASK_PSTATE) {
+ s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+ return 0;
+ }
+
+ if (reg & ~S390_TOPO_FC_MASK) {
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ return 0;
+ }
+
+ switch (fc) {
+ case 0: /* Horizontal polarization is already set */
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ return 2;
+ case 1: /* Vertical polarization is not supported */
+ env->regs[r1] |= S390_PTF_REASON_NONE;
+ return 2;
+ default:
+ /* Note that fc == 2 is interpreted by the SIE */
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ }
+
+ return 0;
+}
+
static void s390_machine_reset(MachineState *machine)
{
S390CcwMachineState *ms = S390_CCW_MACHINE(machine);
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index 3331990e02..ac4b4a92e7 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+int s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 7504cbd21b..3106654445 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -98,6 +98,7 @@
#define PRIV_B9_EQBS 0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -363,6 +364,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+ kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -1454,6 +1456,16 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
}
}
+static int kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+ uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+ int ret;
+
+ ret = s390_handle_ptf(cpu, r1, RA_IGNORED);
+ setcc(cpu, ret);
+ return 0;
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1471,6 +1483,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+ case PRIV_B9_PTF:
+ r = kvm_handle_ptf(cpu, run);
+ break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 05/11] s390x: kvm: topology: interception of PTF instruction
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervizor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervizor.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/s390-virtio-ccw.c | 50 ++++++++++++++++++++++++++++++
include/hw/s390x/s390-virtio-ccw.h | 6 ++++
target/s390x/kvm/kvm.c | 15 +++++++++
3 files changed, 71 insertions(+)
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index ae8e99d0b0..12903eb2af 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -408,6 +408,56 @@ static void s390_pv_prepare_reset(S390CcwMachineState *ms)
s390_pv_prep_reset();
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 and 1 handle the CPU polarization.
+ * We assume an horizontal topology, the only one supported currently
+ * by Linux, consequently we answer to function code 0, requesting
+ * horizontal polarization that it is already the current polarization
+ * and reject vertical polarization request without further explanation.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+int s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ uint8_t fc = reg & S390_TOPO_FC_MASK;
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ s390_program_interrupt(env, PGM_OPERATION, ra);
+ return 0;
+ }
+
+ if (env->psw.mask & PSW_MASK_PSTATE) {
+ s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+ return 0;
+ }
+
+ if (reg & ~S390_TOPO_FC_MASK) {
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ return 0;
+ }
+
+ switch (fc) {
+ case 0: /* Horizontal polarization is already set */
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ return 2;
+ case 1: /* Vertical polarization is not supported */
+ env->regs[r1] |= S390_PTF_REASON_NONE;
+ return 2;
+ default:
+ /* Note that fc == 2 is interpreted by the SIE */
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ }
+
+ return 0;
+}
+
static void s390_machine_reset(MachineState *machine)
{
S390CcwMachineState *ms = S390_CCW_MACHINE(machine);
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index 3331990e02..ac4b4a92e7 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+int s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 7504cbd21b..3106654445 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -98,6 +98,7 @@
#define PRIV_B9_EQBS 0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -363,6 +364,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+ kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -1454,6 +1456,16 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
}
}
+static int kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+ uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+ int ret;
+
+ ret = s390_handle_ptf(cpu, r1, RA_IGNORED);
+ setcc(cpu, ret);
+ return 0;
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1471,6 +1483,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+ case PRIV_B9_PTF:
+ r = kvm_handle_ptf(cpu, run);
+ break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 06/11] s390x: topology: Adding books to CPU topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
S390 CPU topology may have up to 5 topology containers.
The first container above the cores is level 2, the sockets.
We introduce here the books, book is the level containing sockets.
Let's add books, level3, containers to the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine-smp.c | 29 ++++++++++++++++++++++-------
hw/core/machine.c | 2 ++
hw/s390x/s390-virtio-ccw.c | 1 +
include/hw/boards.h | 4 ++++
qapi/machine.json | 7 ++++++-
softmmu/vl.c | 3 +++
6 files changed, 38 insertions(+), 8 deletions(-)
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index b39ed21e65..d7aa39d540 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.books_supported) {
+ g_string_append_printf(s, " * books (%u)", ms->smp.books);
+ }
+
g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
if (mc->smp_props.dies_supported) {
@@ -73,6 +77,7 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
unsigned clusters = config->has_clusters ? config->clusters : 0;
@@ -85,6 +90,7 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
(config->has_clusters && config->clusters == 0) ||
@@ -111,6 +117,13 @@ void machine_parse_smp_config(MachineState *ms,
dies = dies > 0 ? dies : 1;
clusters = clusters > 0 ? clusters : 1;
+ if (!mc->smp_props.books_supported && books > 1) {
+ error_setg(errp, "books not supported by this machine's CPU topology");
+ return;
+ }
+
+ books = books > 0 ? books : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -124,33 +137,35 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus / (books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus / (books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus / (books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus / (books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (sockets * dies * clusters * cores);
+ threads = maxcpus / (books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
+ maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
+ clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
ms->smp.clusters = clusters;
@@ -159,7 +174,7 @@ void machine_parse_smp_config(MachineState *ms,
ms->smp.max_cpus = maxcpus;
/* sanity-check of the computed topology */
- if (sockets * dies * clusters * cores * threads != maxcpus) {
+ if (books * sockets * dies * clusters * cores * threads != maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index d856485cb4..b8c624d2bf 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
.has_clusters = true, .clusters = ms->smp.clusters,
@@ -935,6 +936,7 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
ms->smp.clusters = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 12903eb2af..193883fba3 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -666,6 +666,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
hc->unplug_request = s390_machine_device_unplug_request;
nc->nmi_monitor_handler = s390_nmi;
mc->default_ram_id = "s390.ram";
+ mc->smp_props.books_supported = true;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index c92ac8815c..bc0f7f22dc 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -130,11 +130,13 @@ typedef struct {
* @prefer_sockets - whether sockets are preferred over cores in smp parsing
* @dies_supported - whether dies are supported by the machine
* @clusters_supported - whether clusters are supported by the machine
+ * @books_supported - whether books are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
+ bool books_supported;
} SMPCompatProps;
/**
@@ -299,6 +301,7 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
+ * @books: the number of books on the machine
* @sockets: the number of sockets on the machine
* @dies: the number of dies in one socket
* @clusters: the number of clusters in one die
@@ -308,6 +311,7 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int books;
unsigned int sockets;
unsigned int dies;
unsigned int clusters;
diff --git a/qapi/machine.json b/qapi/machine.json
index 42fc68403d..73206f811a 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -866,12 +866,13 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
+# @book-id: book number within node/board the CPU belongs to
# @socket-id: socket number within node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @core-id: core number within die the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 5 properties that could be present
+# Note: currently there are 6 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -881,6 +882,7 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*core-id': 'int',
@@ -1400,6 +1402,8 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @books: number of books in the CPU topology
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1416,6 +1420,7 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 5e1b35ba48..a680fb12d4 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "books",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "sockets",
.type = QEMU_OPT_NUMBER,
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 06/11] s390x: topology: Adding books to CPU topology
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
S390 CPU topology may have up to 5 topology containers.
The first container above the cores is level 2, the sockets.
We introduce here the books, book is the level containing sockets.
Let's add books, level3, containers to the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine-smp.c | 29 ++++++++++++++++++++++-------
hw/core/machine.c | 2 ++
hw/s390x/s390-virtio-ccw.c | 1 +
include/hw/boards.h | 4 ++++
qapi/machine.json | 7 ++++++-
softmmu/vl.c | 3 +++
6 files changed, 38 insertions(+), 8 deletions(-)
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index b39ed21e65..d7aa39d540 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.books_supported) {
+ g_string_append_printf(s, " * books (%u)", ms->smp.books);
+ }
+
g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
if (mc->smp_props.dies_supported) {
@@ -73,6 +77,7 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
unsigned clusters = config->has_clusters ? config->clusters : 0;
@@ -85,6 +90,7 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
(config->has_clusters && config->clusters == 0) ||
@@ -111,6 +117,13 @@ void machine_parse_smp_config(MachineState *ms,
dies = dies > 0 ? dies : 1;
clusters = clusters > 0 ? clusters : 1;
+ if (!mc->smp_props.books_supported && books > 1) {
+ error_setg(errp, "books not supported by this machine's CPU topology");
+ return;
+ }
+
+ books = books > 0 ? books : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -124,33 +137,35 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus / (books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus / (books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus / (books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus / (books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (sockets * dies * clusters * cores);
+ threads = maxcpus / (books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
+ maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
+ clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
ms->smp.clusters = clusters;
@@ -159,7 +174,7 @@ void machine_parse_smp_config(MachineState *ms,
ms->smp.max_cpus = maxcpus;
/* sanity-check of the computed topology */
- if (sockets * dies * clusters * cores * threads != maxcpus) {
+ if (books * sockets * dies * clusters * cores * threads != maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index d856485cb4..b8c624d2bf 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
.has_clusters = true, .clusters = ms->smp.clusters,
@@ -935,6 +936,7 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
ms->smp.clusters = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 12903eb2af..193883fba3 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -666,6 +666,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
hc->unplug_request = s390_machine_device_unplug_request;
nc->nmi_monitor_handler = s390_nmi;
mc->default_ram_id = "s390.ram";
+ mc->smp_props.books_supported = true;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index c92ac8815c..bc0f7f22dc 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -130,11 +130,13 @@ typedef struct {
* @prefer_sockets - whether sockets are preferred over cores in smp parsing
* @dies_supported - whether dies are supported by the machine
* @clusters_supported - whether clusters are supported by the machine
+ * @books_supported - whether books are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
+ bool books_supported;
} SMPCompatProps;
/**
@@ -299,6 +301,7 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
+ * @books: the number of books on the machine
* @sockets: the number of sockets on the machine
* @dies: the number of dies in one socket
* @clusters: the number of clusters in one die
@@ -308,6 +311,7 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int books;
unsigned int sockets;
unsigned int dies;
unsigned int clusters;
diff --git a/qapi/machine.json b/qapi/machine.json
index 42fc68403d..73206f811a 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -866,12 +866,13 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
+# @book-id: book number within node/board the CPU belongs to
# @socket-id: socket number within node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @core-id: core number within die the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 5 properties that could be present
+# Note: currently there are 6 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -881,6 +882,7 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*core-id': 'int',
@@ -1400,6 +1402,8 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @books: number of books in the CPU topology
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1416,6 +1420,7 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 5e1b35ba48..a680fb12d4 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "books",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "sockets",
.type = QEMU_OPT_NUMBER,
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 07/11] s390x: topology: Adding books to STSI
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Let's add STSI support for the container level 3, books,
and provide the information back to the guest.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 147 +++++++++++++++++++++++++++++---
include/hw/s390x/cpu-topology.h | 20 ++++-
include/hw/s390x/sclp.h | 2 +-
target/s390x/cpu_topology.c | 53 +++++++++---
4 files changed, 194 insertions(+), 28 deletions(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 74e04fd68e..4705a2af96 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -61,6 +61,26 @@ static S390TopologySocket *s390_create_socket(S390TopologyBook *book, int id)
return socket;
}
+static S390TopologyBook *s390_create_book(S390TopologyDrawer *drawer, int id)
+{
+ DeviceState *dev;
+ S390TopologyBook *book;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (drawer->bus->num_children >= ms->smp.books) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ qdev_realize_and_unref(dev, drawer->bus, &error_fatal);
+
+ book = S390_TOPOLOGY_BOOK(dev);
+ book->book_id = id;
+ drawer->cnt++;
+
+ return book;
+}
+
/*
* s390_get_cores:
* @socket: the socket to search into
@@ -110,6 +130,31 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
return s390_create_socket(book, socket_id);
}
+/*
+ * s390_get_book:
+ * @drawer: The drawer to search into
+ * @book_id: the identifier of the book to search for
+ *
+ * returns a pointer to a S390TopologySocket structure within a drawer having
+ * the specified book_id.
+ * First search if the drawer is already containing the S390TopologySocket
+ * structure and if not create one with this book_id.
+ */
+static S390TopologyBook *s390_get_book(S390TopologyDrawer *drawer,
+ int book_id)
+{
+ S390TopologyBook *book;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &drawer->bus->children, sibling) {
+ book = S390_TOPOLOGY_BOOK(kid->child);
+ if (book->book_id == book_id) {
+ return book;
+ }
+ }
+ return s390_create_book(drawer, book_id);
+}
+
/*
* s390_topology_new_cpu:
* @core_id: the core ID is machine wide
@@ -124,16 +169,21 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
void s390_topology_new_cpu(int core_id)
{
const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyDrawer *drawer;
S390TopologyBook *book;
S390TopologySocket *socket;
S390TopologyCores *cores;
int origin, bit;
int nb_cores_per_socket;
+ int nb_cores_per_book;
- book = s390_get_topology();
+ drawer = s390_get_topology();
/* Cores for the S390 topology are cores and threads of the QEMU topology */
nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
+ nb_cores_per_book = ms->smp.sockets * nb_cores_per_socket;
+
+ book = s390_get_book(drawer, core_id / nb_cores_per_book);
socket = s390_get_socket(book, core_id / nb_cores_per_socket);
/*
@@ -166,23 +216,23 @@ void s390_topology_setup(MachineState *ms)
DeviceState *dev;
/* Create BOOK bridge device */
- dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
object_property_add_child(qdev_get_machine(),
- TYPE_S390_TOPOLOGY_BOOK, OBJECT(dev));
+ TYPE_S390_TOPOLOGY_DRAWER, OBJECT(dev));
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
}
-S390TopologyBook *s390_get_topology(void)
+S390TopologyDrawer *s390_get_topology(void)
{
- static S390TopologyBook *book;
+ static S390TopologyDrawer *drawer;
- if (!book) {
- book = S390_TOPOLOGY_BOOK(
- object_resolve_path(TYPE_S390_TOPOLOGY_BOOK, NULL));
- assert(book != NULL);
+ if (!drawer) {
+ drawer = S390_TOPOLOGY_DRAWER(object_resolve_path(
+ TYPE_S390_TOPOLOGY_DRAWER, NULL));
+ assert(drawer != NULL);
}
- return book;
+ return drawer;
}
/* --- CORES Definitions --- */
@@ -333,12 +383,13 @@ static void book_class_init(ObjectClass *oc, void *data)
hc->unplug = qdev_simple_device_unplug_cb;
set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
dc->realize = s390_book_device_realize;
+ dc->bus_type = TYPE_S390_TOPOLOGY_DRAWER_BUS;
dc->desc = "topology book";
}
static const TypeInfo book_info = {
.name = TYPE_S390_TOPOLOGY_BOOK,
- .parent = TYPE_SYS_BUS_DEVICE,
+ .parent = TYPE_DEVICE,
.instance_size = sizeof(S390TopologyBook),
.class_init = book_class_init,
.interfaces = (InterfaceInfo[]) {
@@ -347,6 +398,78 @@ static const TypeInfo book_info = {
}
};
+/* --- DRAWER Definitions --- */
+static Property s390_topology_drawer_properties[] = {
+ DEFINE_PROP_UINT8("drawer_id", S390TopologyDrawer, drawer_id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static char *drawer_bus_get_dev_path(DeviceState *dev)
+{
+ S390TopologyDrawer *drawer = S390_TOPOLOGY_DRAWER(dev);
+ DeviceState *node = dev->parent_bus->parent;
+ char *id = qdev_get_dev_path(node);
+ char *ret;
+
+ if (id) {
+ ret = g_strdup_printf("%s:%02d", id, drawer->drawer_id);
+ g_free(id);
+ } else {
+ ret = g_malloc(6);
+ snprintf(ret, 6, "_:%02d", drawer->drawer_id);
+ }
+
+ return ret;
+}
+
+static void drawer_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = drawer_bus_get_dev_path;
+ k->max_dev = S390_MAX_DRAWERS;
+}
+
+static const TypeInfo drawer_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_DRAWER_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = drawer_bus_class_init,
+};
+
+static void s390_drawer_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyDrawer *drawer = S390_TOPOLOGY_DRAWER(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_DRAWER_BUS, dev,
+ TYPE_S390_TOPOLOGY_DRAWER_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ drawer->bus = bus;
+}
+
+static void drawer_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_drawer_device_realize;
+ device_class_set_props(dc, s390_topology_drawer_properties);
+ dc->desc = "topology drawer";
+}
+
+static const TypeInfo drawer_info = {
+ .name = TYPE_S390_TOPOLOGY_DRAWER,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyDrawer),
+ .class_init = drawer_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
static void topology_register(void)
{
type_register_static(&cpu_cores_info);
@@ -354,6 +477,8 @@ static void topology_register(void)
type_register_static(&socket_info);
type_register_static(&book_bus_info);
type_register_static(&book_info);
+ type_register_static(&drawer_bus_info);
+ type_register_static(&drawer_info);
}
type_init(topology_register);
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index e6e013a8b8..78017c3d78 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -56,18 +56,30 @@ OBJECT_DECLARE_SIMPLE_TYPE(S390TopologySocket, S390_TOPOLOGY_SOCKET)
#define TYPE_S390_TOPOLOGY_BOOK "topology book"
#define TYPE_S390_TOPOLOGY_BOOK_BUS "book-bus"
struct S390TopologyBook {
- SysBusDevice parent_obj;
+ DeviceState parent_obj;
BusState *bus;
uint8_t book_id;
int cnt;
};
typedef struct S390TopologyBook S390TopologyBook;
OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
-#define S390_MAX_BOOKS 1
+#define S390_MAX_BOOKS 4
+
+#define TYPE_S390_TOPOLOGY_DRAWER "topology drawer"
+#define TYPE_S390_TOPOLOGY_DRAWER_BUS "drawer-bus"
+struct S390TopologyDrawer {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t drawer_id;
+ int cnt;
+};
+typedef struct S390TopologyDrawer S390TopologyDrawer;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyDrawer, S390_TOPOLOGY_DRAWER)
+#define S390_MAX_DRAWERS 1
-S390TopologyBook *s390_init_topology(void);
+S390TopologyDrawer *s390_init_topology(void);
-S390TopologyBook *s390_get_topology(void);
+S390TopologyDrawer *s390_get_topology(void);
void s390_topology_setup(MachineState *ms);
void s390_topology_new_cpu(int core_id);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 20dd4bd818..e8c55486f7 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,7 +111,7 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
-#define SCLP_READ_SCP_INFO_MNEST 2
+#define SCLP_READ_SCP_INFO_MNEST 3
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
index 7f6db18829..08e1fbd13e 100644
--- a/target/s390x/cpu_topology.c
+++ b/target/s390x/cpu_topology.c
@@ -14,6 +14,7 @@
#include "hw/s390x/pv.h"
#include "hw/sysbus.h"
#include "hw/s390x/cpu-topology.h"
+#include "hw/s390x/sclp.h"
static int stsi_15_container(void *p, int nl, int id)
{
@@ -40,7 +41,7 @@ static int stsi_15_cpus(void *p, S390TopologyCores *cd)
}
static int set_socket(const MachineState *ms, void *p,
- S390TopologySocket *socket)
+ S390TopologySocket *socket, int level)
{
BusChild *kid;
int l, len = 0;
@@ -56,24 +57,56 @@ static int set_socket(const MachineState *ms, void *p,
return len;
}
+static int set_book(const MachineState *ms, void *p,
+ S390TopologyBook *book, int level)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ if (level >= 3) {
+ len += stsi_15_container(p, 2, book->book_id);
+ p += len;
+ }
+
+ QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
+ l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child), level);
+ p += l;
+ len += l;
+ }
+
+ return len;
+}
+
static void setup_stsi(const MachineState *ms, void *p, int level)
{
- S390TopologyBook *book;
+ S390TopologyDrawer *drawer;
SysIB_151x *sysib;
BusChild *kid;
+ int nb_sockets, nb_books;
int len, l;
sysib = (SysIB_151x *)p;
sysib->mnest = level;
- sysib->mag[TOPOLOGY_NR_MAG2] = ms->smp.sockets;
+ switch (level) {
+ case 2:
+ nb_books = 0;
+ nb_sockets = ms->smp.sockets * ms->smp.books;
+ break;
+ case 3:
+ nb_books = ms->smp.books;
+ nb_sockets = ms->smp.sockets;
+ break;
+ }
+ sysib->mag[TOPOLOGY_NR_MAG3] = nb_books;
+ sysib->mag[TOPOLOGY_NR_MAG2] = nb_sockets;
sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
- book = s390_get_topology();
+ drawer = s390_get_topology();
len = sizeof(SysIB_151x);
p += len;
- QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
- l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child));
+ QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
+ l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
p += l;
len += l;
}
@@ -87,18 +120,14 @@ void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
void *p;
int ret, cc;
- /*
- * Until the SCLP STSI Facility reporting the MNEST value is used,
- * a sel2 value of 2 is the only value allowed in STSI 15.1.x.
- */
- if (sel2 != 2) {
+ if (sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
setcc(cpu, 3);
return;
}
p = g_malloc0(TARGET_PAGE_SIZE);
- setup_stsi(machine, p, 2);
+ setup_stsi(machine, p, sel2);
if (s390_is_pv()) {
ret = s390_cpu_pv_mem_write(cpu, 0, p, TARGET_PAGE_SIZE);
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 07/11] s390x: topology: Adding books to STSI
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
Let's add STSI support for the container level 3, books,
and provide the information back to the guest.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 147 +++++++++++++++++++++++++++++---
include/hw/s390x/cpu-topology.h | 20 ++++-
include/hw/s390x/sclp.h | 2 +-
target/s390x/cpu_topology.c | 53 +++++++++---
4 files changed, 194 insertions(+), 28 deletions(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 74e04fd68e..4705a2af96 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -61,6 +61,26 @@ static S390TopologySocket *s390_create_socket(S390TopologyBook *book, int id)
return socket;
}
+static S390TopologyBook *s390_create_book(S390TopologyDrawer *drawer, int id)
+{
+ DeviceState *dev;
+ S390TopologyBook *book;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (drawer->bus->num_children >= ms->smp.books) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ qdev_realize_and_unref(dev, drawer->bus, &error_fatal);
+
+ book = S390_TOPOLOGY_BOOK(dev);
+ book->book_id = id;
+ drawer->cnt++;
+
+ return book;
+}
+
/*
* s390_get_cores:
* @socket: the socket to search into
@@ -110,6 +130,31 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
return s390_create_socket(book, socket_id);
}
+/*
+ * s390_get_book:
+ * @drawer: The drawer to search into
+ * @book_id: the identifier of the book to search for
+ *
+ * returns a pointer to a S390TopologySocket structure within a drawer having
+ * the specified book_id.
+ * First search if the drawer is already containing the S390TopologySocket
+ * structure and if not create one with this book_id.
+ */
+static S390TopologyBook *s390_get_book(S390TopologyDrawer *drawer,
+ int book_id)
+{
+ S390TopologyBook *book;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &drawer->bus->children, sibling) {
+ book = S390_TOPOLOGY_BOOK(kid->child);
+ if (book->book_id == book_id) {
+ return book;
+ }
+ }
+ return s390_create_book(drawer, book_id);
+}
+
/*
* s390_topology_new_cpu:
* @core_id: the core ID is machine wide
@@ -124,16 +169,21 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
void s390_topology_new_cpu(int core_id)
{
const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyDrawer *drawer;
S390TopologyBook *book;
S390TopologySocket *socket;
S390TopologyCores *cores;
int origin, bit;
int nb_cores_per_socket;
+ int nb_cores_per_book;
- book = s390_get_topology();
+ drawer = s390_get_topology();
/* Cores for the S390 topology are cores and threads of the QEMU topology */
nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
+ nb_cores_per_book = ms->smp.sockets * nb_cores_per_socket;
+
+ book = s390_get_book(drawer, core_id / nb_cores_per_book);
socket = s390_get_socket(book, core_id / nb_cores_per_socket);
/*
@@ -166,23 +216,23 @@ void s390_topology_setup(MachineState *ms)
DeviceState *dev;
/* Create BOOK bridge device */
- dev = qdev_new(TYPE_S390_TOPOLOGY_BOOK);
+ dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
object_property_add_child(qdev_get_machine(),
- TYPE_S390_TOPOLOGY_BOOK, OBJECT(dev));
+ TYPE_S390_TOPOLOGY_DRAWER, OBJECT(dev));
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
}
-S390TopologyBook *s390_get_topology(void)
+S390TopologyDrawer *s390_get_topology(void)
{
- static S390TopologyBook *book;
+ static S390TopologyDrawer *drawer;
- if (!book) {
- book = S390_TOPOLOGY_BOOK(
- object_resolve_path(TYPE_S390_TOPOLOGY_BOOK, NULL));
- assert(book != NULL);
+ if (!drawer) {
+ drawer = S390_TOPOLOGY_DRAWER(object_resolve_path(
+ TYPE_S390_TOPOLOGY_DRAWER, NULL));
+ assert(drawer != NULL);
}
- return book;
+ return drawer;
}
/* --- CORES Definitions --- */
@@ -333,12 +383,13 @@ static void book_class_init(ObjectClass *oc, void *data)
hc->unplug = qdev_simple_device_unplug_cb;
set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
dc->realize = s390_book_device_realize;
+ dc->bus_type = TYPE_S390_TOPOLOGY_DRAWER_BUS;
dc->desc = "topology book";
}
static const TypeInfo book_info = {
.name = TYPE_S390_TOPOLOGY_BOOK,
- .parent = TYPE_SYS_BUS_DEVICE,
+ .parent = TYPE_DEVICE,
.instance_size = sizeof(S390TopologyBook),
.class_init = book_class_init,
.interfaces = (InterfaceInfo[]) {
@@ -347,6 +398,78 @@ static const TypeInfo book_info = {
}
};
+/* --- DRAWER Definitions --- */
+static Property s390_topology_drawer_properties[] = {
+ DEFINE_PROP_UINT8("drawer_id", S390TopologyDrawer, drawer_id, 0),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static char *drawer_bus_get_dev_path(DeviceState *dev)
+{
+ S390TopologyDrawer *drawer = S390_TOPOLOGY_DRAWER(dev);
+ DeviceState *node = dev->parent_bus->parent;
+ char *id = qdev_get_dev_path(node);
+ char *ret;
+
+ if (id) {
+ ret = g_strdup_printf("%s:%02d", id, drawer->drawer_id);
+ g_free(id);
+ } else {
+ ret = g_malloc(6);
+ snprintf(ret, 6, "_:%02d", drawer->drawer_id);
+ }
+
+ return ret;
+}
+
+static void drawer_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = drawer_bus_get_dev_path;
+ k->max_dev = S390_MAX_DRAWERS;
+}
+
+static const TypeInfo drawer_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_DRAWER_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = drawer_bus_class_init,
+};
+
+static void s390_drawer_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyDrawer *drawer = S390_TOPOLOGY_DRAWER(dev);
+ BusState *bus;
+
+ bus = qbus_new(TYPE_S390_TOPOLOGY_DRAWER_BUS, dev,
+ TYPE_S390_TOPOLOGY_DRAWER_BUS);
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+ drawer->bus = bus;
+}
+
+static void drawer_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_drawer_device_realize;
+ device_class_set_props(dc, s390_topology_drawer_properties);
+ dc->desc = "topology drawer";
+}
+
+static const TypeInfo drawer_info = {
+ .name = TYPE_S390_TOPOLOGY_DRAWER,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyDrawer),
+ .class_init = drawer_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
static void topology_register(void)
{
type_register_static(&cpu_cores_info);
@@ -354,6 +477,8 @@ static void topology_register(void)
type_register_static(&socket_info);
type_register_static(&book_bus_info);
type_register_static(&book_info);
+ type_register_static(&drawer_bus_info);
+ type_register_static(&drawer_info);
}
type_init(topology_register);
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index e6e013a8b8..78017c3d78 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -56,18 +56,30 @@ OBJECT_DECLARE_SIMPLE_TYPE(S390TopologySocket, S390_TOPOLOGY_SOCKET)
#define TYPE_S390_TOPOLOGY_BOOK "topology book"
#define TYPE_S390_TOPOLOGY_BOOK_BUS "book-bus"
struct S390TopologyBook {
- SysBusDevice parent_obj;
+ DeviceState parent_obj;
BusState *bus;
uint8_t book_id;
int cnt;
};
typedef struct S390TopologyBook S390TopologyBook;
OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
-#define S390_MAX_BOOKS 1
+#define S390_MAX_BOOKS 4
+
+#define TYPE_S390_TOPOLOGY_DRAWER "topology drawer"
+#define TYPE_S390_TOPOLOGY_DRAWER_BUS "drawer-bus"
+struct S390TopologyDrawer {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t drawer_id;
+ int cnt;
+};
+typedef struct S390TopologyDrawer S390TopologyDrawer;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyDrawer, S390_TOPOLOGY_DRAWER)
+#define S390_MAX_DRAWERS 1
-S390TopologyBook *s390_init_topology(void);
+S390TopologyDrawer *s390_init_topology(void);
-S390TopologyBook *s390_get_topology(void);
+S390TopologyDrawer *s390_get_topology(void);
void s390_topology_setup(MachineState *ms);
void s390_topology_new_cpu(int core_id);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 20dd4bd818..e8c55486f7 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,7 +111,7 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
-#define SCLP_READ_SCP_INFO_MNEST 2
+#define SCLP_READ_SCP_INFO_MNEST 3
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
index 7f6db18829..08e1fbd13e 100644
--- a/target/s390x/cpu_topology.c
+++ b/target/s390x/cpu_topology.c
@@ -14,6 +14,7 @@
#include "hw/s390x/pv.h"
#include "hw/sysbus.h"
#include "hw/s390x/cpu-topology.h"
+#include "hw/s390x/sclp.h"
static int stsi_15_container(void *p, int nl, int id)
{
@@ -40,7 +41,7 @@ static int stsi_15_cpus(void *p, S390TopologyCores *cd)
}
static int set_socket(const MachineState *ms, void *p,
- S390TopologySocket *socket)
+ S390TopologySocket *socket, int level)
{
BusChild *kid;
int l, len = 0;
@@ -56,24 +57,56 @@ static int set_socket(const MachineState *ms, void *p,
return len;
}
+static int set_book(const MachineState *ms, void *p,
+ S390TopologyBook *book, int level)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ if (level >= 3) {
+ len += stsi_15_container(p, 2, book->book_id);
+ p += len;
+ }
+
+ QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
+ l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child), level);
+ p += l;
+ len += l;
+ }
+
+ return len;
+}
+
static void setup_stsi(const MachineState *ms, void *p, int level)
{
- S390TopologyBook *book;
+ S390TopologyDrawer *drawer;
SysIB_151x *sysib;
BusChild *kid;
+ int nb_sockets, nb_books;
int len, l;
sysib = (SysIB_151x *)p;
sysib->mnest = level;
- sysib->mag[TOPOLOGY_NR_MAG2] = ms->smp.sockets;
+ switch (level) {
+ case 2:
+ nb_books = 0;
+ nb_sockets = ms->smp.sockets * ms->smp.books;
+ break;
+ case 3:
+ nb_books = ms->smp.books;
+ nb_sockets = ms->smp.sockets;
+ break;
+ }
+ sysib->mag[TOPOLOGY_NR_MAG3] = nb_books;
+ sysib->mag[TOPOLOGY_NR_MAG2] = nb_sockets;
sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
- book = s390_get_topology();
+ drawer = s390_get_topology();
len = sizeof(SysIB_151x);
p += len;
- QTAILQ_FOREACH_REVERSE(kid, &book->bus->children, sibling) {
- l = set_socket(ms, p, S390_TOPOLOGY_SOCKET(kid->child));
+ QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
+ l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
p += l;
len += l;
}
@@ -87,18 +120,14 @@ void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
void *p;
int ret, cc;
- /*
- * Until the SCLP STSI Facility reporting the MNEST value is used,
- * a sel2 value of 2 is the only value allowed in STSI 15.1.x.
- */
- if (sel2 != 2) {
+ if (sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
setcc(cpu, 3);
return;
}
p = g_malloc0(TARGET_PAGE_SIZE);
- setup_stsi(machine, p, 2);
+ setup_stsi(machine, p, sel2);
if (s390_is_pv()) {
ret = s390_cpu_pv_mem_write(cpu, 0, p, TARGET_PAGE_SIZE);
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
S390 CPU topology may have up to 5 topology containers.
The first container above the cores is level 2, the sockets,
and the level 3, containing sockets are the books.
We introduce here the drawers, drawers is the level containing books.
Let's add drawers, level4, containers to the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
hw/core/machine.c | 2 ++
hw/s390x/s390-virtio-ccw.c | 1 +
include/hw/boards.h | 4 ++++
qapi/machine.json | 7 ++++++-
softmmu/vl.c | 3 +++
6 files changed, 42 insertions(+), 8 deletions(-)
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index d7aa39d540..26150c748f 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.drawers_supported) {
+ g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
+ }
+
if (mc->smp_props.books_supported) {
g_string_append_printf(s, " * books (%u)", ms->smp.books);
}
@@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned drawers = config->has_drawers ? config->drawers : 0;
unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
@@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_drawers && config->drawers == 0) ||
(config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
@@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
books = books > 0 ? books : 1;
+ if (!mc->smp_props.drawers_supported && drawers > 1) {
+ error_setg(errp, "drawers not supported by this machine's CPU topology");
+ return;
+ }
+
+ drawers = drawers > 0 ? drawers : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (books * dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (books * sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (books * sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (books * dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (books * sockets * dies * clusters * cores);
+ threads = maxcpus /
+ (drawers * books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
+ maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.drawers = drawers;
ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
@@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
ms->smp.max_cpus = maxcpus;
/* sanity-check of the computed topology */
- if (books * sockets * dies * clusters * cores * threads != maxcpus) {
+ if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index b8c624d2bf..1db55e36c8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_drawers = true, .drawers = ms->smp.drawers,
.has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
@@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.drawers = 1;
ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 193883fba3..03829e90b3 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
nc->nmi_monitor_handler = s390_nmi;
mc->default_ram_id = "s390.ram";
mc->smp_props.books_supported = true;
+ mc->smp_props.drawers_supported = true;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index bc0f7f22dc..abc5556c50 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -131,12 +131,14 @@ typedef struct {
* @dies_supported - whether dies are supported by the machine
* @clusters_supported - whether clusters are supported by the machine
* @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool books_supported;
+ bool drawers_supported;
} SMPCompatProps;
/**
@@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
+ * @drawers: the number of drawers on the machine
* @books: the number of books on the machine
* @sockets: the number of sockets on the machine
* @dies: the number of dies in one socket
@@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int drawers;
unsigned int books;
unsigned int sockets;
unsigned int dies;
diff --git a/qapi/machine.json b/qapi/machine.json
index 73206f811a..fa6bde5617 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -866,13 +866,14 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to
# @book-id: book number within node/board the CPU belongs to
# @socket-id: socket number within node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @core-id: core number within die the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present
+# Note: currently there are 7 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -882,6 +883,7 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*drawer-id': 'int',
'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
@@ -1402,6 +1404,8 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology
+#
# @books: number of books in the CPU topology
#
# @sockets: number of sockets in the CPU topology
@@ -1420,6 +1424,7 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
'*books': 'int',
'*sockets': 'int',
'*dies': 'int',
diff --git a/softmmu/vl.c b/softmmu/vl.c
index a680fb12d4..f4c9f3a536 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "drawers",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "books",
.type = QEMU_OPT_NUMBER,
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
S390 CPU topology may have up to 5 topology containers.
The first container above the cores is level 2, the sockets,
and the level 3, containing sockets are the books.
We introduce here the drawers, drawers is the level containing books.
Let's add drawers, level4, containers to the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
hw/core/machine.c | 2 ++
hw/s390x/s390-virtio-ccw.c | 1 +
include/hw/boards.h | 4 ++++
qapi/machine.json | 7 ++++++-
softmmu/vl.c | 3 +++
6 files changed, 42 insertions(+), 8 deletions(-)
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index d7aa39d540..26150c748f 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.drawers_supported) {
+ g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
+ }
+
if (mc->smp_props.books_supported) {
g_string_append_printf(s, " * books (%u)", ms->smp.books);
}
@@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned drawers = config->has_drawers ? config->drawers : 0;
unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
@@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_drawers && config->drawers == 0) ||
(config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
@@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
books = books > 0 ? books : 1;
+ if (!mc->smp_props.drawers_supported && drawers > 1) {
+ error_setg(errp, "drawers not supported by this machine's CPU topology");
+ return;
+ }
+
+ drawers = drawers > 0 ? drawers : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (books * dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (books * sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (books * sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (books * dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (books * sockets * dies * clusters * cores);
+ threads = maxcpus /
+ (drawers * books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
+ maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.drawers = drawers;
ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
@@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
ms->smp.max_cpus = maxcpus;
/* sanity-check of the computed topology */
- if (books * sockets * dies * clusters * cores * threads != maxcpus) {
+ if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index b8c624d2bf..1db55e36c8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_drawers = true, .drawers = ms->smp.drawers,
.has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
@@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.drawers = 1;
ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 193883fba3..03829e90b3 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
nc->nmi_monitor_handler = s390_nmi;
mc->default_ram_id = "s390.ram";
mc->smp_props.books_supported = true;
+ mc->smp_props.drawers_supported = true;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index bc0f7f22dc..abc5556c50 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -131,12 +131,14 @@ typedef struct {
* @dies_supported - whether dies are supported by the machine
* @clusters_supported - whether clusters are supported by the machine
* @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool books_supported;
+ bool drawers_supported;
} SMPCompatProps;
/**
@@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
+ * @drawers: the number of drawers on the machine
* @books: the number of books on the machine
* @sockets: the number of sockets on the machine
* @dies: the number of dies in one socket
@@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int drawers;
unsigned int books;
unsigned int sockets;
unsigned int dies;
diff --git a/qapi/machine.json b/qapi/machine.json
index 73206f811a..fa6bde5617 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -866,13 +866,14 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to
# @book-id: book number within node/board the CPU belongs to
# @socket-id: socket number within node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @core-id: core number within die the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present
+# Note: currently there are 7 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -882,6 +883,7 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*drawer-id': 'int',
'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
@@ -1402,6 +1404,8 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology
+#
# @books: number of books in the CPU topology
#
# @sockets: number of sockets in the CPU topology
@@ -1420,6 +1424,7 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
'*books': 'int',
'*sockets': 'int',
'*dies': 'int',
diff --git a/softmmu/vl.c b/softmmu/vl.c
index a680fb12d4..f4c9f3a536 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "drawers",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "books",
.type = QEMU_OPT_NUMBER,
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 09/11] s390x: topology: Adding drawers to STSI
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Let's add STSI support for the container level 4, drawers,
and provide the information back to the guest.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 137 +++++++++++++++++++++++++++++---
include/hw/s390x/cpu-topology.h | 19 ++++-
include/hw/s390x/sclp.h | 2 +-
target/s390x/cpu_topology.c | 40 ++++++++--
4 files changed, 176 insertions(+), 22 deletions(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 4705a2af96..3ae86f80f1 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -81,6 +81,26 @@ static S390TopologyBook *s390_create_book(S390TopologyDrawer *drawer, int id)
return book;
}
+static S390TopologyDrawer *s390_create_drawer(S390TopologyNode *node, int id)
+{
+ DeviceState *dev;
+ S390TopologyDrawer *drawer;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (node->bus->num_children >= ms->smp.drawers) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
+ qdev_realize_and_unref(dev, node->bus, &error_fatal);
+
+ drawer = S390_TOPOLOGY_DRAWER(dev);
+ drawer->drawer_id = id;
+ node->cnt++;
+
+ return drawer;
+}
+
/*
* s390_get_cores:
* @socket: the socket to search into
@@ -130,6 +150,31 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
return s390_create_socket(book, socket_id);
}
+/*
+ * s390_get_drawer:
+ * @node: The node to search into
+ * @drawer_id: the identifier of the drawer to search for
+ *
+ * returns a pointer to a S390TopologyDrawer structure within a book having
+ * the specified drawer_id.
+ * First search if the book is already containing the S390TopologyDrawer
+ * structure and if not create one with this drawer_id.
+ */
+static S390TopologyDrawer *s390_get_drawer(S390TopologyNode *node,
+ int drawer_id)
+{
+ S390TopologyDrawer *drawer;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &node->bus->children, sibling) {
+ drawer = S390_TOPOLOGY_DRAWER(kid->child);
+ if (drawer->drawer_id == drawer_id) {
+ return drawer;
+ }
+ }
+ return s390_create_drawer(node, drawer_id);
+}
+
/*
* s390_get_book:
* @drawer: The drawer to search into
@@ -169,6 +214,7 @@ static S390TopologyBook *s390_get_book(S390TopologyDrawer *drawer,
void s390_topology_new_cpu(int core_id)
{
const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyNode *node;
S390TopologyDrawer *drawer;
S390TopologyBook *book;
S390TopologySocket *socket;
@@ -176,13 +222,16 @@ void s390_topology_new_cpu(int core_id)
int origin, bit;
int nb_cores_per_socket;
int nb_cores_per_book;
+ int nb_cores_per_drawer;
- drawer = s390_get_topology();
+ node = s390_get_topology();
/* Cores for the S390 topology are cores and threads of the QEMU topology */
nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
nb_cores_per_book = ms->smp.sockets * nb_cores_per_socket;
+ nb_cores_per_drawer = ms->smp.books * nb_cores_per_book;
+ drawer = s390_get_drawer(node, core_id / nb_cores_per_drawer);
book = s390_get_book(drawer, core_id / nb_cores_per_book);
socket = s390_get_socket(book, core_id / nb_cores_per_socket);
@@ -216,23 +265,23 @@ void s390_topology_setup(MachineState *ms)
DeviceState *dev;
/* Create BOOK bridge device */
- dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
+ dev = qdev_new(TYPE_S390_TOPOLOGY_NODE);
object_property_add_child(qdev_get_machine(),
- TYPE_S390_TOPOLOGY_DRAWER, OBJECT(dev));
+ TYPE_S390_TOPOLOGY_NODE, OBJECT(dev));
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
}
-S390TopologyDrawer *s390_get_topology(void)
+S390TopologyNode *s390_get_topology(void)
{
- static S390TopologyDrawer *drawer;
+ static S390TopologyNode *node;
- if (!drawer) {
- drawer = S390_TOPOLOGY_DRAWER(object_resolve_path(
- TYPE_S390_TOPOLOGY_DRAWER, NULL));
- assert(drawer != NULL);
+ if (!node) {
+ node = S390_TOPOLOGY_NODE(object_resolve_path(
+ TYPE_S390_TOPOLOGY_NODE, NULL));
+ assert(node != NULL);
}
- return drawer;
+ return node;
}
/* --- CORES Definitions --- */
@@ -455,6 +504,7 @@ static void drawer_class_init(ObjectClass *oc, void *data)
hc->unplug = qdev_simple_device_unplug_cb;
set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->bus_type = TYPE_S390_TOPOLOGY_NODE_BUS;
dc->realize = s390_drawer_device_realize;
device_class_set_props(dc, s390_topology_drawer_properties);
dc->desc = "topology drawer";
@@ -462,7 +512,7 @@ static void drawer_class_init(ObjectClass *oc, void *data)
static const TypeInfo drawer_info = {
.name = TYPE_S390_TOPOLOGY_DRAWER,
- .parent = TYPE_SYS_BUS_DEVICE,
+ .parent = TYPE_DEVICE,
.instance_size = sizeof(S390TopologyDrawer),
.class_init = drawer_class_init,
.interfaces = (InterfaceInfo[]) {
@@ -470,6 +520,69 @@ static const TypeInfo drawer_info = {
{ }
}
};
+
+/* --- NODE Definitions --- */
+
+/*
+ * Nodes are the first level of CPU topology we support
+ * only one NODE for the moment.
+ */
+static char *node_bus_get_dev_path(DeviceState *dev)
+{
+ return g_strdup_printf("00");
+}
+
+static void node_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = node_bus_get_dev_path;
+ k->max_dev = S390_MAX_NODES;
+}
+
+static const TypeInfo node_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_NODE_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = node_bus_class_init,
+};
+
+static void s390_node_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyNode *node = S390_TOPOLOGY_NODE(dev);
+ BusState *bus;
+
+ /* Create NODE bus on NODE bridge device */
+ bus = qbus_new(TYPE_S390_TOPOLOGY_NODE_BUS, dev,
+ TYPE_S390_TOPOLOGY_NODE_BUS);
+ node->bus = bus;
+
+ /* Enable hotplugging */
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+}
+
+static void node_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_node_device_realize;
+ dc->desc = "topology node";
+}
+
+static const TypeInfo node_info = {
+ .name = TYPE_S390_TOPOLOGY_NODE,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyNode),
+ .class_init = node_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
static void topology_register(void)
{
type_register_static(&cpu_cores_info);
@@ -479,6 +592,8 @@ static void topology_register(void)
type_register_static(&book_info);
type_register_static(&drawer_bus_info);
type_register_static(&drawer_info);
+ type_register_static(&node_bus_info);
+ type_register_static(&node_info);
}
type_init(topology_register);
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 78017c3d78..10e4bd754f 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -68,18 +68,29 @@ OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
#define TYPE_S390_TOPOLOGY_DRAWER "topology drawer"
#define TYPE_S390_TOPOLOGY_DRAWER_BUS "drawer-bus"
struct S390TopologyDrawer {
- SysBusDevice parent_obj;
+ DeviceState parent_obj;
BusState *bus;
uint8_t drawer_id;
int cnt;
};
typedef struct S390TopologyDrawer S390TopologyDrawer;
OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyDrawer, S390_TOPOLOGY_DRAWER)
-#define S390_MAX_DRAWERS 1
+#define S390_MAX_DRAWERS 4
-S390TopologyDrawer *s390_init_topology(void);
+#define TYPE_S390_TOPOLOGY_NODE "topology node"
+#define TYPE_S390_TOPOLOGY_NODE_BUS "node-bus"
+struct S390TopologyNode {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t node_id;
+ int cnt;
+};
+typedef struct S390TopologyNode S390TopologyNode;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyNode, S390_TOPOLOGY_NODE)
+#define S390_MAX_NODES 1
-S390TopologyDrawer *s390_get_topology(void);
+S390TopologyNode *s390_init_topology(void);
+S390TopologyNode *s390_get_topology(void);
void s390_topology_setup(MachineState *ms);
void s390_topology_new_cpu(int core_id);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index e8c55486f7..660ef49ec5 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,7 +111,7 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
-#define SCLP_READ_SCP_INFO_MNEST 3
+#define SCLP_READ_SCP_INFO_MNEST 4
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
index 08e1fbd13e..fd25c56213 100644
--- a/target/s390x/cpu_topology.c
+++ b/target/s390x/cpu_topology.c
@@ -77,36 +77,64 @@ static int set_book(const MachineState *ms, void *p,
return len;
}
+static int set_drawer(const MachineState *ms, void *p,
+ S390TopologyDrawer *drawer, int level)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ if (level >= 4) {
+ len += stsi_15_container(p, 3, drawer->drawer_id);
+ p += len;
+ }
+
+ QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
+ l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
+ p += l;
+ len += l;
+ }
+
+ return len;
+}
+
static void setup_stsi(const MachineState *ms, void *p, int level)
{
- S390TopologyDrawer *drawer;
+ S390TopologyNode *node;
SysIB_151x *sysib;
BusChild *kid;
- int nb_sockets, nb_books;
+ int nb_sockets, nb_books, nb_drawers;
int len, l;
sysib = (SysIB_151x *)p;
sysib->mnest = level;
switch (level) {
case 2:
+ nb_drawers = 0;
nb_books = 0;
- nb_sockets = ms->smp.sockets * ms->smp.books;
+ nb_sockets = ms->smp.sockets * ms->smp.books * ms->smp.drawers;
break;
case 3:
+ nb_drawers = 0;
+ nb_books = ms->smp.books * ms->smp.drawers;
+ nb_sockets = ms->smp.sockets;
+ break;
+ case 4:
+ nb_drawers = ms->smp.drawers;
nb_books = ms->smp.books;
nb_sockets = ms->smp.sockets;
break;
}
+ sysib->mag[TOPOLOGY_NR_MAG4] = nb_drawers;
sysib->mag[TOPOLOGY_NR_MAG3] = nb_books;
sysib->mag[TOPOLOGY_NR_MAG2] = nb_sockets;
sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
- drawer = s390_get_topology();
+ node = s390_get_topology();
len = sizeof(SysIB_151x);
p += len;
- QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
- l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
+ QTAILQ_FOREACH_REVERSE(kid, &node->bus->children, sibling) {
+ l = set_drawer(ms, p, S390_TOPOLOGY_DRAWER(kid->child), level);
p += l;
len += l;
}
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 09/11] s390x: topology: Adding drawers to STSI
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
Let's add STSI support for the container level 4, drawers,
and provide the information back to the guest.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 137 +++++++++++++++++++++++++++++---
include/hw/s390x/cpu-topology.h | 19 ++++-
include/hw/s390x/sclp.h | 2 +-
target/s390x/cpu_topology.c | 40 ++++++++--
4 files changed, 176 insertions(+), 22 deletions(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 4705a2af96..3ae86f80f1 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -81,6 +81,26 @@ static S390TopologyBook *s390_create_book(S390TopologyDrawer *drawer, int id)
return book;
}
+static S390TopologyDrawer *s390_create_drawer(S390TopologyNode *node, int id)
+{
+ DeviceState *dev;
+ S390TopologyDrawer *drawer;
+ const MachineState *ms = MACHINE(qdev_get_machine());
+
+ if (node->bus->num_children >= ms->smp.drawers) {
+ return NULL;
+ }
+
+ dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
+ qdev_realize_and_unref(dev, node->bus, &error_fatal);
+
+ drawer = S390_TOPOLOGY_DRAWER(dev);
+ drawer->drawer_id = id;
+ node->cnt++;
+
+ return drawer;
+}
+
/*
* s390_get_cores:
* @socket: the socket to search into
@@ -130,6 +150,31 @@ static S390TopologySocket *s390_get_socket(S390TopologyBook *book,
return s390_create_socket(book, socket_id);
}
+/*
+ * s390_get_drawer:
+ * @node: The node to search into
+ * @drawer_id: the identifier of the drawer to search for
+ *
+ * returns a pointer to a S390TopologyDrawer structure within a book having
+ * the specified drawer_id.
+ * First search if the book is already containing the S390TopologyDrawer
+ * structure and if not create one with this drawer_id.
+ */
+static S390TopologyDrawer *s390_get_drawer(S390TopologyNode *node,
+ int drawer_id)
+{
+ S390TopologyDrawer *drawer;
+ BusChild *kid;
+
+ QTAILQ_FOREACH(kid, &node->bus->children, sibling) {
+ drawer = S390_TOPOLOGY_DRAWER(kid->child);
+ if (drawer->drawer_id == drawer_id) {
+ return drawer;
+ }
+ }
+ return s390_create_drawer(node, drawer_id);
+}
+
/*
* s390_get_book:
* @drawer: The drawer to search into
@@ -169,6 +214,7 @@ static S390TopologyBook *s390_get_book(S390TopologyDrawer *drawer,
void s390_topology_new_cpu(int core_id)
{
const MachineState *ms = MACHINE(qdev_get_machine());
+ S390TopologyNode *node;
S390TopologyDrawer *drawer;
S390TopologyBook *book;
S390TopologySocket *socket;
@@ -176,13 +222,16 @@ void s390_topology_new_cpu(int core_id)
int origin, bit;
int nb_cores_per_socket;
int nb_cores_per_book;
+ int nb_cores_per_drawer;
- drawer = s390_get_topology();
+ node = s390_get_topology();
/* Cores for the S390 topology are cores and threads of the QEMU topology */
nb_cores_per_socket = ms->smp.cores * ms->smp.threads;
nb_cores_per_book = ms->smp.sockets * nb_cores_per_socket;
+ nb_cores_per_drawer = ms->smp.books * nb_cores_per_book;
+ drawer = s390_get_drawer(node, core_id / nb_cores_per_drawer);
book = s390_get_book(drawer, core_id / nb_cores_per_book);
socket = s390_get_socket(book, core_id / nb_cores_per_socket);
@@ -216,23 +265,23 @@ void s390_topology_setup(MachineState *ms)
DeviceState *dev;
/* Create BOOK bridge device */
- dev = qdev_new(TYPE_S390_TOPOLOGY_DRAWER);
+ dev = qdev_new(TYPE_S390_TOPOLOGY_NODE);
object_property_add_child(qdev_get_machine(),
- TYPE_S390_TOPOLOGY_DRAWER, OBJECT(dev));
+ TYPE_S390_TOPOLOGY_NODE, OBJECT(dev));
sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
}
-S390TopologyDrawer *s390_get_topology(void)
+S390TopologyNode *s390_get_topology(void)
{
- static S390TopologyDrawer *drawer;
+ static S390TopologyNode *node;
- if (!drawer) {
- drawer = S390_TOPOLOGY_DRAWER(object_resolve_path(
- TYPE_S390_TOPOLOGY_DRAWER, NULL));
- assert(drawer != NULL);
+ if (!node) {
+ node = S390_TOPOLOGY_NODE(object_resolve_path(
+ TYPE_S390_TOPOLOGY_NODE, NULL));
+ assert(node != NULL);
}
- return drawer;
+ return node;
}
/* --- CORES Definitions --- */
@@ -455,6 +504,7 @@ static void drawer_class_init(ObjectClass *oc, void *data)
hc->unplug = qdev_simple_device_unplug_cb;
set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->bus_type = TYPE_S390_TOPOLOGY_NODE_BUS;
dc->realize = s390_drawer_device_realize;
device_class_set_props(dc, s390_topology_drawer_properties);
dc->desc = "topology drawer";
@@ -462,7 +512,7 @@ static void drawer_class_init(ObjectClass *oc, void *data)
static const TypeInfo drawer_info = {
.name = TYPE_S390_TOPOLOGY_DRAWER,
- .parent = TYPE_SYS_BUS_DEVICE,
+ .parent = TYPE_DEVICE,
.instance_size = sizeof(S390TopologyDrawer),
.class_init = drawer_class_init,
.interfaces = (InterfaceInfo[]) {
@@ -470,6 +520,69 @@ static const TypeInfo drawer_info = {
{ }
}
};
+
+/* --- NODE Definitions --- */
+
+/*
+ * Nodes are the first level of CPU topology we support
+ * only one NODE for the moment.
+ */
+static char *node_bus_get_dev_path(DeviceState *dev)
+{
+ return g_strdup_printf("00");
+}
+
+static void node_bus_class_init(ObjectClass *oc, void *data)
+{
+ BusClass *k = BUS_CLASS(oc);
+
+ k->get_dev_path = node_bus_get_dev_path;
+ k->max_dev = S390_MAX_NODES;
+}
+
+static const TypeInfo node_bus_info = {
+ .name = TYPE_S390_TOPOLOGY_NODE_BUS,
+ .parent = TYPE_BUS,
+ .instance_size = 0,
+ .class_init = node_bus_class_init,
+};
+
+static void s390_node_device_realize(DeviceState *dev, Error **errp)
+{
+ S390TopologyNode *node = S390_TOPOLOGY_NODE(dev);
+ BusState *bus;
+
+ /* Create NODE bus on NODE bridge device */
+ bus = qbus_new(TYPE_S390_TOPOLOGY_NODE_BUS, dev,
+ TYPE_S390_TOPOLOGY_NODE_BUS);
+ node->bus = bus;
+
+ /* Enable hotplugging */
+ qbus_set_hotplug_handler(bus, OBJECT(dev));
+}
+
+static void node_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+ HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+
+ hc->unplug = qdev_simple_device_unplug_cb;
+ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+ dc->realize = s390_node_device_realize;
+ dc->desc = "topology node";
+}
+
+static const TypeInfo node_info = {
+ .name = TYPE_S390_TOPOLOGY_NODE,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(S390TopologyNode),
+ .class_init = node_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_HOTPLUG_HANDLER },
+ { }
+ }
+};
+
static void topology_register(void)
{
type_register_static(&cpu_cores_info);
@@ -479,6 +592,8 @@ static void topology_register(void)
type_register_static(&book_info);
type_register_static(&drawer_bus_info);
type_register_static(&drawer_info);
+ type_register_static(&node_bus_info);
+ type_register_static(&node_info);
}
type_init(topology_register);
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 78017c3d78..10e4bd754f 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -68,18 +68,29 @@ OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyBook, S390_TOPOLOGY_BOOK)
#define TYPE_S390_TOPOLOGY_DRAWER "topology drawer"
#define TYPE_S390_TOPOLOGY_DRAWER_BUS "drawer-bus"
struct S390TopologyDrawer {
- SysBusDevice parent_obj;
+ DeviceState parent_obj;
BusState *bus;
uint8_t drawer_id;
int cnt;
};
typedef struct S390TopologyDrawer S390TopologyDrawer;
OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyDrawer, S390_TOPOLOGY_DRAWER)
-#define S390_MAX_DRAWERS 1
+#define S390_MAX_DRAWERS 4
-S390TopologyDrawer *s390_init_topology(void);
+#define TYPE_S390_TOPOLOGY_NODE "topology node"
+#define TYPE_S390_TOPOLOGY_NODE_BUS "node-bus"
+struct S390TopologyNode {
+ SysBusDevice parent_obj;
+ BusState *bus;
+ uint8_t node_id;
+ int cnt;
+};
+typedef struct S390TopologyNode S390TopologyNode;
+OBJECT_DECLARE_SIMPLE_TYPE(S390TopologyNode, S390_TOPOLOGY_NODE)
+#define S390_MAX_NODES 1
-S390TopologyDrawer *s390_get_topology(void);
+S390TopologyNode *s390_init_topology(void);
+S390TopologyNode *s390_get_topology(void);
void s390_topology_setup(MachineState *ms);
void s390_topology_new_cpu(int core_id);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index e8c55486f7..660ef49ec5 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -111,7 +111,7 @@ typedef struct CPUEntry {
uint8_t reserved1;
} QEMU_PACKED CPUEntry;
-#define SCLP_READ_SCP_INFO_MNEST 3
+#define SCLP_READ_SCP_INFO_MNEST 4
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
typedef struct ReadInfo {
SCCBHeader h;
diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c
index 08e1fbd13e..fd25c56213 100644
--- a/target/s390x/cpu_topology.c
+++ b/target/s390x/cpu_topology.c
@@ -77,36 +77,64 @@ static int set_book(const MachineState *ms, void *p,
return len;
}
+static int set_drawer(const MachineState *ms, void *p,
+ S390TopologyDrawer *drawer, int level)
+{
+ BusChild *kid;
+ int l, len = 0;
+
+ if (level >= 4) {
+ len += stsi_15_container(p, 3, drawer->drawer_id);
+ p += len;
+ }
+
+ QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
+ l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
+ p += l;
+ len += l;
+ }
+
+ return len;
+}
+
static void setup_stsi(const MachineState *ms, void *p, int level)
{
- S390TopologyDrawer *drawer;
+ S390TopologyNode *node;
SysIB_151x *sysib;
BusChild *kid;
- int nb_sockets, nb_books;
+ int nb_sockets, nb_books, nb_drawers;
int len, l;
sysib = (SysIB_151x *)p;
sysib->mnest = level;
switch (level) {
case 2:
+ nb_drawers = 0;
nb_books = 0;
- nb_sockets = ms->smp.sockets * ms->smp.books;
+ nb_sockets = ms->smp.sockets * ms->smp.books * ms->smp.drawers;
break;
case 3:
+ nb_drawers = 0;
+ nb_books = ms->smp.books * ms->smp.drawers;
+ nb_sockets = ms->smp.sockets;
+ break;
+ case 4:
+ nb_drawers = ms->smp.drawers;
nb_books = ms->smp.books;
nb_sockets = ms->smp.sockets;
break;
}
+ sysib->mag[TOPOLOGY_NR_MAG4] = nb_drawers;
sysib->mag[TOPOLOGY_NR_MAG3] = nb_books;
sysib->mag[TOPOLOGY_NR_MAG2] = nb_sockets;
sysib->mag[TOPOLOGY_NR_MAG1] = ms->smp.cores * ms->smp.threads;
- drawer = s390_get_topology();
+ node = s390_get_topology();
len = sizeof(SysIB_151x);
p += len;
- QTAILQ_FOREACH_REVERSE(kid, &drawer->bus->children, sibling) {
- l = set_book(ms, p, S390_TOPOLOGY_BOOK(kid->child), level);
+ QTAILQ_FOREACH_REVERSE(kid, &node->bus->children, sibling) {
+ l = set_drawer(ms, p, S390_TOPOLOGY_DRAWER(kid->child), level);
p += l;
len += l;
}
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 10/11] s390x: topology: implementing numa for the s390x topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
S390x CPU Topology allows a non uniform repartition of the CPU
inside the topology containers, sockets, books and drawers.
We use numa to place the CPU inside the right topology container
and report the non uniform topology to the guest.
Note that s390x needs CPU0 to belong to the topology and consequently
all topology must include CPU0.
We accept a partial QEMU numa definition, in that case undefined CPUs
are added to free slots in the topology starting with slot 0 and going
up.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine.c | 18 ++++++++++
hw/s390x/s390-virtio-ccw.c | 68 ++++++++++++++++++++++++++++++++++----
2 files changed, 79 insertions(+), 7 deletions(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 1db55e36c8..ee719965f7 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -687,6 +687,16 @@ void machine_set_cpu_numa_node(MachineState *machine,
return;
}
+ if (props->has_book_id && !slot->props.has_book_id) {
+ error_setg(errp, "book-id is not supported");
+ return;
+ }
+
+ if (props->has_drawer_id && !slot->props.has_drawer_id) {
+ error_setg(errp, "drawer-id is not supported");
+ return;
+ }
+
/* skip slots with explicit mismatch */
if (props->has_thread_id && props->thread_id != slot->props.thread_id) {
continue;
@@ -704,6 +714,14 @@ void machine_set_cpu_numa_node(MachineState *machine,
continue;
}
+ if (props->has_book_id && props->book_id != slot->props.book_id) {
+ continue;
+ }
+
+ if (props->has_drawer_id && props->drawer_id != slot->props.drawer_id) {
+ continue;
+ }
+
/* reject assignment if slot is already assigned, for compatibility
* of legacy cpu_index mapping with SPAPR core based mapping do not
* error out if cpu thread and matched core have the same node-id */
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 03829e90b3..b50c304796 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -84,14 +84,34 @@ out:
static void s390_init_cpus(MachineState *machine)
{
MachineClass *mc = MACHINE_GET_CLASS(machine);
- int i;
+ CPUArchId *slot;
+ int i, n = 0;
/* initialize possible_cpus */
mc->possible_cpu_arch_ids(machine);
s390_topology_setup(machine);
- for (i = 0; i < machine->smp.cpus; i++) {
+
+ /* For NUMA configuration create defined nodes */
+ if (machine->numa_state->num_nodes) {
+ for (i = 0; i < machine->smp.max_cpus; i++) {
+ slot = &machine->possible_cpus->cpus[i];
+ if (slot->arch_id != -1 && n < machine->smp.cpus) {
+ s390x_new_cpu(machine->cpu_type, i, &error_fatal);
+ n++;
+ }
+ }
+ }
+
+ /* create all remaining CPUs */
+ for (i = 0; n < machine->smp.cpus && i < machine->smp.max_cpus; i++) {
+ slot = &machine->possible_cpus->cpus[i];
+ /* For NUMA configuration skip defined nodes */
+ if (machine->numa_state->num_nodes && slot->arch_id != -1) {
+ continue;
+ }
s390x_new_cpu(machine->cpu_type, i, &error_fatal);
+ n++;
}
}
@@ -274,6 +294,11 @@ static void ccw_init(MachineState *machine)
/* register hypercalls */
virtio_ccw_register_hcalls();
+ /* CPU0 must exist on S390x */
+ if (!s390_cpu_addr2state(0)) {
+ error_printf("Core_id 0 must be defined in the CPU configuration\n");
+ exit(1);
+ }
s390_enable_css_support(s390_cpu_addr2state(0));
ret = css_create_css_image(VIRTUAL_CSSID, true);
@@ -306,6 +331,7 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ ms->possible_cpus->cpus[cpu->env.core_id].arch_id = cpu->env.core_id;
s390_topology_new_cpu(cpu->env.core_id);
@@ -579,7 +605,9 @@ static CpuInstanceProperties s390_cpu_index_to_props(MachineState *ms,
static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
{
int i;
+ int drawer_id, book_id, socket_id;
unsigned int max_cpus = ms->smp.max_cpus;
+ CPUArchId *slot;
if (ms->possible_cpus) {
g_assert(ms->possible_cpus && ms->possible_cpus->len == max_cpus);
@@ -590,11 +618,25 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
- ms->possible_cpus->cpus[i].type = ms->cpu_type;
- ms->possible_cpus->cpus[i].vcpus_count = 1;
- ms->possible_cpus->cpus[i].arch_id = i;
- ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = i;
+ slot = &ms->possible_cpus->cpus[i];
+
+ slot->type = ms->cpu_type;
+ slot->vcpus_count = 1;
+ slot->arch_id = i;
+ slot->props.has_core_id = true;
+ slot->props.core_id = i;
+
+ socket_id = i / ms->smp.cores;
+ slot->props.socket_id = socket_id;
+ slot->props.has_socket_id = true;
+
+ book_id = socket_id / ms->smp.sockets;
+ slot->props.book_id = book_id;
+ slot->props.has_book_id = true;
+
+ drawer_id = book_id / ms->smp.books;
+ slot->props.drawer_id = drawer_id;
+ slot->props.has_drawer_id = true;
}
return ms->possible_cpus;
@@ -636,6 +678,17 @@ static ram_addr_t s390_fixup_ram_size(ram_addr_t sz)
return newsz;
}
+/*
+ * S390 defines CPU topology level 2 as the level for which a change in topology
+ * is worth being taking care of.
+ * Let use level 2, socket, as the numa node.
+ */
+static int64_t s390_get_default_cpu_node_id(const MachineState *ms, int idx)
+{
+ ms->possible_cpus->cpus[idx].arch_id = -1;
+ return idx / ms->smp.cores;
+}
+
static void ccw_machine_class_init(ObjectClass *oc, void *data)
{
MachineClass *mc = MACHINE_CLASS(oc);
@@ -668,6 +721,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
mc->default_ram_id = "s390.ram";
mc->smp_props.books_supported = true;
mc->smp_props.drawers_supported = true;
+ mc->get_default_cpu_node_id = s390_get_default_cpu_node_id;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 10/11] s390x: topology: implementing numa for the s390x topology
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
S390x CPU Topology allows a non uniform repartition of the CPU
inside the topology containers, sockets, books and drawers.
We use numa to place the CPU inside the right topology container
and report the non uniform topology to the guest.
Note that s390x needs CPU0 to belong to the topology and consequently
all topology must include CPU0.
We accept a partial QEMU numa definition, in that case undefined CPUs
are added to free slots in the topology starting with slot 0 and going
up.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/core/machine.c | 18 ++++++++++
hw/s390x/s390-virtio-ccw.c | 68 ++++++++++++++++++++++++++++++++++----
2 files changed, 79 insertions(+), 7 deletions(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 1db55e36c8..ee719965f7 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -687,6 +687,16 @@ void machine_set_cpu_numa_node(MachineState *machine,
return;
}
+ if (props->has_book_id && !slot->props.has_book_id) {
+ error_setg(errp, "book-id is not supported");
+ return;
+ }
+
+ if (props->has_drawer_id && !slot->props.has_drawer_id) {
+ error_setg(errp, "drawer-id is not supported");
+ return;
+ }
+
/* skip slots with explicit mismatch */
if (props->has_thread_id && props->thread_id != slot->props.thread_id) {
continue;
@@ -704,6 +714,14 @@ void machine_set_cpu_numa_node(MachineState *machine,
continue;
}
+ if (props->has_book_id && props->book_id != slot->props.book_id) {
+ continue;
+ }
+
+ if (props->has_drawer_id && props->drawer_id != slot->props.drawer_id) {
+ continue;
+ }
+
/* reject assignment if slot is already assigned, for compatibility
* of legacy cpu_index mapping with SPAPR core based mapping do not
* error out if cpu thread and matched core have the same node-id */
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 03829e90b3..b50c304796 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -84,14 +84,34 @@ out:
static void s390_init_cpus(MachineState *machine)
{
MachineClass *mc = MACHINE_GET_CLASS(machine);
- int i;
+ CPUArchId *slot;
+ int i, n = 0;
/* initialize possible_cpus */
mc->possible_cpu_arch_ids(machine);
s390_topology_setup(machine);
- for (i = 0; i < machine->smp.cpus; i++) {
+
+ /* For NUMA configuration create defined nodes */
+ if (machine->numa_state->num_nodes) {
+ for (i = 0; i < machine->smp.max_cpus; i++) {
+ slot = &machine->possible_cpus->cpus[i];
+ if (slot->arch_id != -1 && n < machine->smp.cpus) {
+ s390x_new_cpu(machine->cpu_type, i, &error_fatal);
+ n++;
+ }
+ }
+ }
+
+ /* create all remaining CPUs */
+ for (i = 0; n < machine->smp.cpus && i < machine->smp.max_cpus; i++) {
+ slot = &machine->possible_cpus->cpus[i];
+ /* For NUMA configuration skip defined nodes */
+ if (machine->numa_state->num_nodes && slot->arch_id != -1) {
+ continue;
+ }
s390x_new_cpu(machine->cpu_type, i, &error_fatal);
+ n++;
}
}
@@ -274,6 +294,11 @@ static void ccw_init(MachineState *machine)
/* register hypercalls */
virtio_ccw_register_hcalls();
+ /* CPU0 must exist on S390x */
+ if (!s390_cpu_addr2state(0)) {
+ error_printf("Core_id 0 must be defined in the CPU configuration\n");
+ exit(1);
+ }
s390_enable_css_support(s390_cpu_addr2state(0));
ret = css_create_css_image(VIRTUAL_CSSID, true);
@@ -306,6 +331,7 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ ms->possible_cpus->cpus[cpu->env.core_id].arch_id = cpu->env.core_id;
s390_topology_new_cpu(cpu->env.core_id);
@@ -579,7 +605,9 @@ static CpuInstanceProperties s390_cpu_index_to_props(MachineState *ms,
static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
{
int i;
+ int drawer_id, book_id, socket_id;
unsigned int max_cpus = ms->smp.max_cpus;
+ CPUArchId *slot;
if (ms->possible_cpus) {
g_assert(ms->possible_cpus && ms->possible_cpus->len == max_cpus);
@@ -590,11 +618,25 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
- ms->possible_cpus->cpus[i].type = ms->cpu_type;
- ms->possible_cpus->cpus[i].vcpus_count = 1;
- ms->possible_cpus->cpus[i].arch_id = i;
- ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = i;
+ slot = &ms->possible_cpus->cpus[i];
+
+ slot->type = ms->cpu_type;
+ slot->vcpus_count = 1;
+ slot->arch_id = i;
+ slot->props.has_core_id = true;
+ slot->props.core_id = i;
+
+ socket_id = i / ms->smp.cores;
+ slot->props.socket_id = socket_id;
+ slot->props.has_socket_id = true;
+
+ book_id = socket_id / ms->smp.sockets;
+ slot->props.book_id = book_id;
+ slot->props.has_book_id = true;
+
+ drawer_id = book_id / ms->smp.books;
+ slot->props.drawer_id = drawer_id;
+ slot->props.has_drawer_id = true;
}
return ms->possible_cpus;
@@ -636,6 +678,17 @@ static ram_addr_t s390_fixup_ram_size(ram_addr_t sz)
return newsz;
}
+/*
+ * S390 defines CPU topology level 2 as the level for which a change in topology
+ * is worth being taking care of.
+ * Let use level 2, socket, as the numa node.
+ */
+static int64_t s390_get_default_cpu_node_id(const MachineState *ms, int idx)
+{
+ ms->possible_cpus->cpus[idx].arch_id = -1;
+ return idx / ms->smp.cores;
+}
+
static void ccw_machine_class_init(ObjectClass *oc, void *data)
{
MachineClass *mc = MACHINE_CLASS(oc);
@@ -668,6 +721,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
mc->default_ram_id = "s390.ram";
mc->smp_props.books_supported = true;
mc->smp_props.drawers_supported = true;
+ mc->get_default_cpu_node_id = s390_get_default_cpu_node_id;
}
static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 11/11] s390x: topology: documentation
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 13:41 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
The use of the S390x CPU topology is explain in a new documentation
file.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
docs/system/s390x/numa-cpu-topology.rst | 273 ++++++++++++++++++++++++
1 file changed, 273 insertions(+)
create mode 100644 docs/system/s390x/numa-cpu-topology.rst
diff --git a/docs/system/s390x/numa-cpu-topology.rst b/docs/system/s390x/numa-cpu-topology.rst
new file mode 100644
index 0000000000..9ae15f792f
--- /dev/null
+++ b/docs/system/s390x/numa-cpu-topology.rst
@@ -0,0 +1,273 @@
+NUMA CPU Topology on S390x
+==========================
+
+IBM S390 provides a complex CPU architecture with several cache levels.
+Using NUMA with the CPU topology is a way to let the guest optimize his
+accesses to the main memory.
+
+The QEMU smp parameter for S390x allows to specify 4 NUMA levels:
+core, socket, drawer and book and these levels are available for
+the numa parameter too.
+
+
+Prerequisites
+-------------
+
+To take advantage of the CPU topology, KVM must give support for the
+Perform Topology Function and to the Store System Information instructions
+as indicated by the Perform CPU Topology facility (stfle bit 11).
+
+If those requirements are met, the capability ``KVM_CAP_S390_CPU_TOPOLOGY``
+will indicate that KVM can support CPU Topology on that LPAR.
+
+
+Using CPU Topology in QEMU for S390x
+------------------------------------
+
+
+QEMU -smp parameter
+~~~~~~~~~~~~~~~~~~~
+
+With -smp QEMU provides the user with the possibility to define
+a Topology based on ::
+
+ -smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books] \
+ [,sockets=sockets][,cores=cores]
+
+The topology reported to the guest in this situation will provide
+n cpus of a maximum of maxcpus cpus, filling the topology levels one by one
+starting with CPU0 being the first CPU on drawer[0] book[0] socket[0].
+
+For example ``-smp 5,books=2,sockets=2,cores=2`` will provide ::
+
+ drawer[0]--+--book[0]--+--socket[0]--+--core[0]-CPU0
+ | | |
+ | | +--core[1]-CPU1
+ | |
+ | +--socket[1]--+--core[0]-CPU2
+ | |
+ | +--core[1]-CPU3
+ |
+ +--book[1]--+--socket[0]--+--core[0]-CPU4
+
+
+Note that the thread parameter can not be defined on S390 as it
+has no representation on the CPU topology.
+
+
+QEMU -numa parameter
+~~~~~~~~~~~~~~~~~~~
+
+With -numa QEMU provides the user with the possibility to define
+the Topology in a non uniform way ::
+
+ -smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books] \
+ [,sockets=sockets][,cores=cores]
+ -numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]
+ -numa cpu,node-id=node[,drawer-id=x][,book-id=x][,socket-id=x][,core-id=y]
+
+The topology reported to the guest in this situation will provide
+n cpus of a maximum of maxcpus cpus, and the topology entries will be
+
+- if there is less cpus than specified by the -numa arguments
+ the topology will be build by filling the numa definitions
+ starting with the lowest node.
+
+- if there is more cpus than specified by the -numa argument
+ the numa specification will first be fulfilled and the remaining
+ CPU will be assigned to unassigned slots starting with the
+ core 0 on socket 0.
+
+- a CPU declared with -device does not count inside the ncpus parameter
+ of the -smp argument and will be added on the topology based on
+ its core ID.
+
+For example ::
+
+ -smp 3,drawers=8,books=2,sockets=2,cores=2,maxcpus=64
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+ -numa node,nodeid=1
+ -numa node,nodeid=2
+ -numa cpu,node-id=0,drawer-id=0
+ -numa cpu,node-id=1,socket-id=9
+ -device host-s390x-cpu,core-id=19
+
+Will provide the following topology ::
+
+ drawer[0]--+--book[0]--+--socket[0]--+--core[0]-CPU0
+ | |
+ | +--core[1]-CPU1
+ |
+ +--socket[1]--+--core[0]-CPU2
+
+ drawer[2]--+--book[0]--+--socket[1]--+--core[1]-CPU19
+
+
+S390 NUMA specificity
+---------------------
+
+Heterogene Memory Attributes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The S390 topology implementation does not use ACPI HMAT to specify the
+cache size and bandwidth between nodes.
+
+Memory device
+~~~~~~~~~~~~~
+
+When using NUMA S390 needs a memory device to be associated with
+the nodes definitions. As we do not use HMAT, it has little sense
+to assign memory to each node and one should assign all memory to
+a node without CPU and use other nodes to define the CPU Topology.
+
+Exemple ::
+
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+
+
+CPUs
+~~~~
+
+In the S390 topology we do not use threads and the first topology
+level is the core.
+The number of threads can no be defined for S390 and is always equal to 1.
+
+When using NUMA, QEMU issues a warning for CPUS not assigned to nodes.
+The S390 topology will silently assign unassigned CPUs to the topology
+searching for free core starting on the first core of the first socket
+in the first book.
+This is of course advised to assign all possible CPUs to nodes to
+guaranty future compatibility.
+
+
+The topology provided to the guest
+----------------------------------
+
+The guest , when the CPU Topology is available as indicated by the
+Perform CPU Topology facility (stfle bit 11) may use two instructions
+to retrieve the CPU topology and optimize its CPU scheduling:
+
+- PTF (Perform Topology function) which will give information
+ about a change in the CPU Topology, that is a change in the
+ result of the STSI(15,1,2) instruction.
+
+- STSI (Stote System Information) with parameters (15,1,2)
+ to retrieve the CPU Topology.
+
+Exemple ::
+
+ -smp 3,drawers=8,books=2,sockets=2,cores=2,maxcpus=64
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+ -numa node,nodeid=1
+ -numa node,nodeid=2
+ -numa cpu,node-id=1,drawer-id=0
+ -numa cpu,node-id=2,socket-id=9
+ -device host-s390x-cpu,core-id=19
+
+Formated result for STSI(15,1,2) showing the 6 different levels
+with:
+- levels 2 (socket) and 1 (core) used.
+- 3 sockets with a CPU mask for CPU type 3, non dedicated and
+ with horizontal polarization.
+- The first socket contains 2 cores as specified by the -smp argument
+- The second socket contains the 3rd core defined by the -smp argument
+- both these sockets belong to drawer-id=0 and to node-1
+- The third socket hold the CPU with core-id 19 assigned to socket-id 9
+ and to node-2
+
+Here the kernel view ::
+
+ mag[6] = 0
+ mag[5] = 0
+ mag[4] = 0
+ mag[3] = 0
+ mag[2] = 32
+ mag[1] = 2
+ MNest = 2
+ socket: 1 0
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : c000000000000000
+
+ socket: 1 1
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 2000000000000000
+
+ socket: 1 9
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0000100000000000
+
+And the admin view ::
+
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+
+
+Hotplug with NUMA
+-----------------
+
+Using the core-id the topology is automatically calculated to put the core
+inside the right socket.
+
+Example::
+
+ (qemu) device_add host-s390x-cpu,core-id=8
+
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+ 4 - - - - - ::: no yes horizontal 8
+
+ # chcpu -e 4
+ CPU 4 enabled
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+ 4 0 2 2 3 4 4:4:4:4 yes yes horizontal 8
+
+One can see that the userland tool reports serials IDs which do not correspond
+to the firmware IDs but does however report the new CPU on it's own socket.
+
+The result seen by the kernel looks like ::
+
+ mag[6] = 0
+ mag[5] = 0
+ mag[4] = 0
+ mag[3] = 0
+ mag[2] = 32
+ mag[1] = 2
+ MNest = 2
+ 00 - socket: 1 0
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : c000000000000000
+
+ socket: 1 1
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 2000000000000000
+
+ socket: 1 9
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0000100000000000
+
+ socket: 1 4
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0080000000000000
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH v6 11/11] s390x: topology: documentation
@ 2022-02-17 13:41 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 13:41 UTC (permalink / raw)
To: qemu-s390x
Cc: thuth, seiden, nrb, ehabkost, kvm, david, eblake, cohuck,
richard.henderson, qemu-devel, armbru, pasic, borntraeger, mst,
pbonzini, philmd
The use of the S390x CPU topology is explain in a new documentation
file.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
docs/system/s390x/numa-cpu-topology.rst | 273 ++++++++++++++++++++++++
1 file changed, 273 insertions(+)
create mode 100644 docs/system/s390x/numa-cpu-topology.rst
diff --git a/docs/system/s390x/numa-cpu-topology.rst b/docs/system/s390x/numa-cpu-topology.rst
new file mode 100644
index 0000000000..9ae15f792f
--- /dev/null
+++ b/docs/system/s390x/numa-cpu-topology.rst
@@ -0,0 +1,273 @@
+NUMA CPU Topology on S390x
+==========================
+
+IBM S390 provides a complex CPU architecture with several cache levels.
+Using NUMA with the CPU topology is a way to let the guest optimize his
+accesses to the main memory.
+
+The QEMU smp parameter for S390x allows to specify 4 NUMA levels:
+core, socket, drawer and book and these levels are available for
+the numa parameter too.
+
+
+Prerequisites
+-------------
+
+To take advantage of the CPU topology, KVM must give support for the
+Perform Topology Function and to the Store System Information instructions
+as indicated by the Perform CPU Topology facility (stfle bit 11).
+
+If those requirements are met, the capability ``KVM_CAP_S390_CPU_TOPOLOGY``
+will indicate that KVM can support CPU Topology on that LPAR.
+
+
+Using CPU Topology in QEMU for S390x
+------------------------------------
+
+
+QEMU -smp parameter
+~~~~~~~~~~~~~~~~~~~
+
+With -smp QEMU provides the user with the possibility to define
+a Topology based on ::
+
+ -smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books] \
+ [,sockets=sockets][,cores=cores]
+
+The topology reported to the guest in this situation will provide
+n cpus of a maximum of maxcpus cpus, filling the topology levels one by one
+starting with CPU0 being the first CPU on drawer[0] book[0] socket[0].
+
+For example ``-smp 5,books=2,sockets=2,cores=2`` will provide ::
+
+ drawer[0]--+--book[0]--+--socket[0]--+--core[0]-CPU0
+ | | |
+ | | +--core[1]-CPU1
+ | |
+ | +--socket[1]--+--core[0]-CPU2
+ | |
+ | +--core[1]-CPU3
+ |
+ +--book[1]--+--socket[0]--+--core[0]-CPU4
+
+
+Note that the thread parameter can not be defined on S390 as it
+has no representation on the CPU topology.
+
+
+QEMU -numa parameter
+~~~~~~~~~~~~~~~~~~~
+
+With -numa QEMU provides the user with the possibility to define
+the Topology in a non uniform way ::
+
+ -smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books] \
+ [,sockets=sockets][,cores=cores]
+ -numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]
+ -numa cpu,node-id=node[,drawer-id=x][,book-id=x][,socket-id=x][,core-id=y]
+
+The topology reported to the guest in this situation will provide
+n cpus of a maximum of maxcpus cpus, and the topology entries will be
+
+- if there is less cpus than specified by the -numa arguments
+ the topology will be build by filling the numa definitions
+ starting with the lowest node.
+
+- if there is more cpus than specified by the -numa argument
+ the numa specification will first be fulfilled and the remaining
+ CPU will be assigned to unassigned slots starting with the
+ core 0 on socket 0.
+
+- a CPU declared with -device does not count inside the ncpus parameter
+ of the -smp argument and will be added on the topology based on
+ its core ID.
+
+For example ::
+
+ -smp 3,drawers=8,books=2,sockets=2,cores=2,maxcpus=64
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+ -numa node,nodeid=1
+ -numa node,nodeid=2
+ -numa cpu,node-id=0,drawer-id=0
+ -numa cpu,node-id=1,socket-id=9
+ -device host-s390x-cpu,core-id=19
+
+Will provide the following topology ::
+
+ drawer[0]--+--book[0]--+--socket[0]--+--core[0]-CPU0
+ | |
+ | +--core[1]-CPU1
+ |
+ +--socket[1]--+--core[0]-CPU2
+
+ drawer[2]--+--book[0]--+--socket[1]--+--core[1]-CPU19
+
+
+S390 NUMA specificity
+---------------------
+
+Heterogene Memory Attributes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The S390 topology implementation does not use ACPI HMAT to specify the
+cache size and bandwidth between nodes.
+
+Memory device
+~~~~~~~~~~~~~
+
+When using NUMA S390 needs a memory device to be associated with
+the nodes definitions. As we do not use HMAT, it has little sense
+to assign memory to each node and one should assign all memory to
+a node without CPU and use other nodes to define the CPU Topology.
+
+Exemple ::
+
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+
+
+CPUs
+~~~~
+
+In the S390 topology we do not use threads and the first topology
+level is the core.
+The number of threads can no be defined for S390 and is always equal to 1.
+
+When using NUMA, QEMU issues a warning for CPUS not assigned to nodes.
+The S390 topology will silently assign unassigned CPUs to the topology
+searching for free core starting on the first core of the first socket
+in the first book.
+This is of course advised to assign all possible CPUs to nodes to
+guaranty future compatibility.
+
+
+The topology provided to the guest
+----------------------------------
+
+The guest , when the CPU Topology is available as indicated by the
+Perform CPU Topology facility (stfle bit 11) may use two instructions
+to retrieve the CPU topology and optimize its CPU scheduling:
+
+- PTF (Perform Topology function) which will give information
+ about a change in the CPU Topology, that is a change in the
+ result of the STSI(15,1,2) instruction.
+
+- STSI (Stote System Information) with parameters (15,1,2)
+ to retrieve the CPU Topology.
+
+Exemple ::
+
+ -smp 3,drawers=8,books=2,sockets=2,cores=2,maxcpus=64
+ -object memory-backend-ram,id=mem0,size=10G
+ -numa node,nodeid=0,memdev=mem0
+ -numa node,nodeid=1
+ -numa node,nodeid=2
+ -numa cpu,node-id=1,drawer-id=0
+ -numa cpu,node-id=2,socket-id=9
+ -device host-s390x-cpu,core-id=19
+
+Formated result for STSI(15,1,2) showing the 6 different levels
+with:
+- levels 2 (socket) and 1 (core) used.
+- 3 sockets with a CPU mask for CPU type 3, non dedicated and
+ with horizontal polarization.
+- The first socket contains 2 cores as specified by the -smp argument
+- The second socket contains the 3rd core defined by the -smp argument
+- both these sockets belong to drawer-id=0 and to node-1
+- The third socket hold the CPU with core-id 19 assigned to socket-id 9
+ and to node-2
+
+Here the kernel view ::
+
+ mag[6] = 0
+ mag[5] = 0
+ mag[4] = 0
+ mag[3] = 0
+ mag[2] = 32
+ mag[1] = 2
+ MNest = 2
+ socket: 1 0
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : c000000000000000
+
+ socket: 1 1
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 2000000000000000
+
+ socket: 1 9
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0000100000000000
+
+And the admin view ::
+
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+
+
+Hotplug with NUMA
+-----------------
+
+Using the core-id the topology is automatically calculated to put the core
+inside the right socket.
+
+Example::
+
+ (qemu) device_add host-s390x-cpu,core-id=8
+
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+ 4 - - - - - ::: no yes horizontal 8
+
+ # chcpu -e 4
+ CPU 4 enabled
+ # lscpu -e
+ CPU NODE DRAWER BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS
+ 0 0 0 0 0 0 0:0:0:0 yes yes horizontal 0
+ 1 0 0 0 0 1 1:1:1:1 yes yes horizontal 1
+ 2 0 0 0 1 2 2:2:2:2 yes yes horizontal 2
+ 3 0 1 1 2 3 3:3:3:3 yes yes horizontal 19
+ 4 0 2 2 3 4 4:4:4:4 yes yes horizontal 8
+
+One can see that the userland tool reports serials IDs which do not correspond
+to the firmware IDs but does however report the new CPU on it's own socket.
+
+The result seen by the kernel looks like ::
+
+ mag[6] = 0
+ mag[5] = 0
+ mag[4] = 0
+ mag[3] = 0
+ mag[2] = 32
+ mag[1] = 2
+ MNest = 2
+ 00 - socket: 1 0
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : c000000000000000
+
+ socket: 1 1
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 2000000000000000
+
+ socket: 1 9
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0000100000000000
+
+ socket: 1 4
+ cpu type 03 d: 0 pp: 0
+ origin : 0000
+ mask : 0080000000000000
--
2.27.0
^ permalink raw reply related [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-02-17 14:20 ` Daniel P. Berrangé
-1 siblings, 0 replies; 36+ messages in thread
From: Daniel P. Berrangé @ 2022-02-17 14:20 UTC (permalink / raw)
To: Pierre Morel
Cc: qemu-s390x, thuth, seiden, nrb, ehabkost, kvm, david, eblake,
cohuck, richard.henderson, qemu-devel, armbru, pasic,
borntraeger, mst, pbonzini, philmd
On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets,
> and the level 3, containing sockets are the books.
>
> We introduce here the drawers, drawers is the level containing books.
>
> Let's add drawers, level4, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 42 insertions(+), 8 deletions(-)
Needs to update -smp args in qemu-options.hx too.
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index d7aa39d540..26150c748f 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
> + }
> +
> if (mc->smp_props.books_supported) {
> g_string_append_printf(s, " * books (%u)", ms->smp.books);
> }
> @@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> @@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> @@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
>
> books = books > 0 ? books : 1;
>
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp, "drawers not supported by this machine's CPU topology");
> + return;
> + }
> +
> + drawers = drawers > 0 ? drawers : 1;
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (books * sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> @@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index b8c624d2bf..1db55e36c8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> @@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 193883fba3..03829e90b3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index bc0f7f22dc..abc5556c50 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -131,12 +131,14 @@ typedef struct {
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> * @books_supported - whether books are supported by the machine
> + * @drawers_supported - whether drawers are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> bool books_supported;
> + bool drawers_supported;
> } SMPCompatProps;
>
> /**
> @@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @drawers: the number of drawers on the machine
> * @books: the number of books on the machine
> * @sockets: the number of sockets on the machine
> * @dies: the number of dies in one socket
> @@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int drawers;
> unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 73206f811a..fa6bde5617 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,13 +866,14 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to
> # @book-id: book number within node/board the CPU belongs to
> # @socket-id: socket number within node/board the CPU belongs to
So the lack of change here implies that 'socket-id' is unique
across multiple books/drawers. Is that correct, as its differnt
from semantics for die-id/core-id/thread-id which are scoped
to within the next level of the topology ?
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present
> +# Note: currently there are 7 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -882,6 +883,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-02-17 14:20 ` Daniel P. Berrangé
0 siblings, 0 replies; 36+ messages in thread
From: Daniel P. Berrangé @ 2022-02-17 14:20 UTC (permalink / raw)
To: Pierre Morel
Cc: thuth, nrb, ehabkost, kvm, david, philmd, cohuck,
richard.henderson, qemu-devel, armbru, pasic, seiden, qemu-s390x,
mst, pbonzini, eblake, borntraeger
On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets,
> and the level 3, containing sockets are the books.
>
> We introduce here the drawers, drawers is the level containing books.
>
> Let's add drawers, level4, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 42 insertions(+), 8 deletions(-)
Needs to update -smp args in qemu-options.hx too.
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index d7aa39d540..26150c748f 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
> + }
> +
> if (mc->smp_props.books_supported) {
> g_string_append_printf(s, " * books (%u)", ms->smp.books);
> }
> @@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> @@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> @@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
>
> books = books > 0 ? books : 1;
>
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp, "drawers not supported by this machine's CPU topology");
> + return;
> + }
> +
> + drawers = drawers > 0 ? drawers : 1;
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (books * sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> @@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index b8c624d2bf..1db55e36c8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> @@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 193883fba3..03829e90b3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index bc0f7f22dc..abc5556c50 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -131,12 +131,14 @@ typedef struct {
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> * @books_supported - whether books are supported by the machine
> + * @drawers_supported - whether drawers are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> bool books_supported;
> + bool drawers_supported;
> } SMPCompatProps;
>
> /**
> @@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @drawers: the number of drawers on the machine
> * @books: the number of books on the machine
> * @sockets: the number of sockets on the machine
> * @dies: the number of dies in one socket
> @@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int drawers;
> unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 73206f811a..fa6bde5617 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,13 +866,14 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to
> # @book-id: book number within node/board the CPU belongs to
> # @socket-id: socket number within node/board the CPU belongs to
So the lack of change here implies that 'socket-id' is unique
across multiple books/drawers. Is that correct, as its differnt
from semantics for die-id/core-id/thread-id which are scoped
to within the next level of the topology ?
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present
> +# Note: currently there are 7 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -882,6 +883,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 14:20 ` Daniel P. Berrangé
@ 2022-02-17 15:30 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 15:30 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-s390x, thuth, seiden, nrb, ehabkost, kvm, david, eblake,
cohuck, richard.henderson, qemu-devel, armbru, pasic,
borntraeger, mst, pbonzini, philmd
On 2/17/22 15:20, Daniel P. Berrangé wrote:
> On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
>> S390 CPU topology may have up to 5 topology containers.
>> The first container above the cores is level 2, the sockets,
>> and the level 3, containing sockets are the books.
>>
>> We introduce here the drawers, drawers is the level containing books.
>>
>> Let's add drawers, level4, containers to the CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
>> hw/core/machine.c | 2 ++
>> hw/s390x/s390-virtio-ccw.c | 1 +
>> include/hw/boards.h | 4 ++++
>> qapi/machine.json | 7 ++++++-
>> softmmu/vl.c | 3 +++
>> 6 files changed, 42 insertions(+), 8 deletions(-)
>
> Needs to update -smp args in qemu-options.hx too.
>
Oh, right!
Thanks
>>
...snip...
>> index 73206f811a..fa6bde5617 100644
>> --- a/qapi/machine.json
>> +++ b/qapi/machine.json
>> @@ -866,13 +866,14 @@
>> # a CPU is being hotplugged.
>> #
>> # @node-id: NUMA node ID the CPU belongs to
>> +# @drawer-id: drawer number within node/board the CPU belongs to
>> # @book-id: book number within node/board the CPU belongs to
>> # @socket-id: socket number within node/board the CPU belongs to
>
> So the lack of change here implies that 'socket-id' is unique
> across multiple books/drawers. Is that correct, as its differnt
> from semantics for die-id/core-id/thread-id which are scoped
> to within the next level of the topology ?
Hum, no I forgot to update and it needs a change.
What about
# @book-id: book number within node/board/drawer the CPU belongs to
# @socket-id: socket number within node/board/book the CPU belongs to
?
>
>> # @die-id: die number within socket the CPU belongs to (since 4.1)
>> # @core-id: core number within die the CPU belongs to
>> # @thread-id: thread number within core the CPU belongs to
>> #
>> -# Note: currently there are 6 properties that could be present
>> +# Note: currently there are 7 properties that could be present
>> # but management should be prepared to pass through other
>> # properties with device_add command to allow for future
>> # interface extension. This also requires the filed names to be kept in
>> @@ -882,6 +883,7 @@
>> ##
>> { 'struct': 'CpuInstanceProperties',
>> 'data': { '*node-id': 'int',
>> + '*drawer-id': 'int',
>> '*book-id': 'int',
>> '*socket-id': 'int',
>> '*die-id': 'int',
>
> Regards,
> Daniel
>
Thanks a lot,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-02-17 15:30 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 15:30 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: thuth, nrb, ehabkost, kvm, david, philmd, cohuck,
richard.henderson, qemu-devel, armbru, pasic, seiden, qemu-s390x,
mst, pbonzini, eblake, borntraeger
On 2/17/22 15:20, Daniel P. Berrangé wrote:
> On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
>> S390 CPU topology may have up to 5 topology containers.
>> The first container above the cores is level 2, the sockets,
>> and the level 3, containing sockets are the books.
>>
>> We introduce here the drawers, drawers is the level containing books.
>>
>> Let's add drawers, level4, containers to the CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
>> hw/core/machine.c | 2 ++
>> hw/s390x/s390-virtio-ccw.c | 1 +
>> include/hw/boards.h | 4 ++++
>> qapi/machine.json | 7 ++++++-
>> softmmu/vl.c | 3 +++
>> 6 files changed, 42 insertions(+), 8 deletions(-)
>
> Needs to update -smp args in qemu-options.hx too.
>
Oh, right!
Thanks
>>
...snip...
>> index 73206f811a..fa6bde5617 100644
>> --- a/qapi/machine.json
>> +++ b/qapi/machine.json
>> @@ -866,13 +866,14 @@
>> # a CPU is being hotplugged.
>> #
>> # @node-id: NUMA node ID the CPU belongs to
>> +# @drawer-id: drawer number within node/board the CPU belongs to
>> # @book-id: book number within node/board the CPU belongs to
>> # @socket-id: socket number within node/board the CPU belongs to
>
> So the lack of change here implies that 'socket-id' is unique
> across multiple books/drawers. Is that correct, as its differnt
> from semantics for die-id/core-id/thread-id which are scoped
> to within the next level of the topology ?
Hum, no I forgot to update and it needs a change.
What about
# @book-id: book number within node/board/drawer the CPU belongs to
# @socket-id: socket number within node/board/book the CPU belongs to
?
>
>> # @die-id: die number within socket the CPU belongs to (since 4.1)
>> # @core-id: core number within die the CPU belongs to
>> # @thread-id: thread number within core the CPU belongs to
>> #
>> -# Note: currently there are 6 properties that could be present
>> +# Note: currently there are 7 properties that could be present
>> # but management should be prepared to pass through other
>> # properties with device_add command to allow for future
>> # interface extension. This also requires the filed names to be kept in
>> @@ -882,6 +883,7 @@
>> ##
>> { 'struct': 'CpuInstanceProperties',
>> 'data': { '*node-id': 'int',
>> + '*drawer-id': 'int',
>> '*book-id': 'int',
>> '*socket-id': 'int',
>> '*die-id': 'int',
>
> Regards,
> Daniel
>
Thanks a lot,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 15:30 ` Pierre Morel
@ 2022-02-17 15:32 ` Daniel P. Berrangé
-1 siblings, 0 replies; 36+ messages in thread
From: Daniel P. Berrangé @ 2022-02-17 15:32 UTC (permalink / raw)
To: Pierre Morel
Cc: qemu-s390x, thuth, seiden, nrb, ehabkost, kvm, david, eblake,
cohuck, richard.henderson, qemu-devel, armbru, pasic,
borntraeger, mst, pbonzini, philmd
On Thu, Feb 17, 2022 at 04:30:06PM +0100, Pierre Morel wrote:
>
>
> On 2/17/22 15:20, Daniel P. Berrangé wrote:
> > On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
> > > S390 CPU topology may have up to 5 topology containers.
> > > The first container above the cores is level 2, the sockets,
> > > and the level 3, containing sockets are the books.
> > >
> > > We introduce here the drawers, drawers is the level containing books.
> > >
> > > Let's add drawers, level4, containers to the CPU topology.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> > > hw/core/machine.c | 2 ++
> > > hw/s390x/s390-virtio-ccw.c | 1 +
> > > include/hw/boards.h | 4 ++++
> > > qapi/machine.json | 7 ++++++-
> > > softmmu/vl.c | 3 +++
> > > 6 files changed, 42 insertions(+), 8 deletions(-)
> >
> > Needs to update -smp args in qemu-options.hx too.
> >
>
> Oh, right!
>
> Thanks
>
> > >
>
> ...snip...
>
> > > index 73206f811a..fa6bde5617 100644
> > > --- a/qapi/machine.json
> > > +++ b/qapi/machine.json
> > > @@ -866,13 +866,14 @@
> > > # a CPU is being hotplugged.
> > > #
> > > # @node-id: NUMA node ID the CPU belongs to
> > > +# @drawer-id: drawer number within node/board the CPU belongs to
> > > # @book-id: book number within node/board the CPU belongs to
> > > # @socket-id: socket number within node/board the CPU belongs to
> >
> > So the lack of change here implies that 'socket-id' is unique
> > across multiple books/drawers. Is that correct, as its differnt
> > from semantics for die-id/core-id/thread-id which are scoped
> > to within the next level of the topology ?
>
> Hum, no I forgot to update and it needs a change.
> What about
>
> # @book-id: book number within node/board/drawer the CPU belongs to
> # @socket-id: socket number within node/board/book the CPU belongs to
>
> ?
Probably drawer/node/board and book/node/board to keep a
low -> high topology ordering
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-02-17 15:32 ` Daniel P. Berrangé
0 siblings, 0 replies; 36+ messages in thread
From: Daniel P. Berrangé @ 2022-02-17 15:32 UTC (permalink / raw)
To: Pierre Morel
Cc: thuth, nrb, ehabkost, kvm, david, philmd, cohuck,
richard.henderson, qemu-devel, armbru, pasic, seiden, qemu-s390x,
mst, pbonzini, eblake, borntraeger
On Thu, Feb 17, 2022 at 04:30:06PM +0100, Pierre Morel wrote:
>
>
> On 2/17/22 15:20, Daniel P. Berrangé wrote:
> > On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
> > > S390 CPU topology may have up to 5 topology containers.
> > > The first container above the cores is level 2, the sockets,
> > > and the level 3, containing sockets are the books.
> > >
> > > We introduce here the drawers, drawers is the level containing books.
> > >
> > > Let's add drawers, level4, containers to the CPU topology.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> > > hw/core/machine.c | 2 ++
> > > hw/s390x/s390-virtio-ccw.c | 1 +
> > > include/hw/boards.h | 4 ++++
> > > qapi/machine.json | 7 ++++++-
> > > softmmu/vl.c | 3 +++
> > > 6 files changed, 42 insertions(+), 8 deletions(-)
> >
> > Needs to update -smp args in qemu-options.hx too.
> >
>
> Oh, right!
>
> Thanks
>
> > >
>
> ...snip...
>
> > > index 73206f811a..fa6bde5617 100644
> > > --- a/qapi/machine.json
> > > +++ b/qapi/machine.json
> > > @@ -866,13 +866,14 @@
> > > # a CPU is being hotplugged.
> > > #
> > > # @node-id: NUMA node ID the CPU belongs to
> > > +# @drawer-id: drawer number within node/board the CPU belongs to
> > > # @book-id: book number within node/board the CPU belongs to
> > > # @socket-id: socket number within node/board the CPU belongs to
> >
> > So the lack of change here implies that 'socket-id' is unique
> > across multiple books/drawers. Is that correct, as its differnt
> > from semantics for die-id/core-id/thread-id which are scoped
> > to within the next level of the topology ?
>
> Hum, no I forgot to update and it needs a change.
> What about
>
> # @book-id: book number within node/board/drawer the CPU belongs to
> # @socket-id: socket number within node/board/book the CPU belongs to
>
> ?
Probably drawer/node/board and book/node/board to keep a
low -> high topology ordering
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 15:32 ` Daniel P. Berrangé
@ 2022-02-17 15:50 ` Pierre Morel
-1 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 15:50 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-s390x, thuth, seiden, nrb, ehabkost, kvm, david, eblake,
cohuck, richard.henderson, qemu-devel, armbru, pasic,
borntraeger, mst, pbonzini, philmd
On 2/17/22 16:32, Daniel P. Berrangé wrote:
> On Thu, Feb 17, 2022 at 04:30:06PM +0100, Pierre Morel wrote:
>>
>>
>> On 2/17/22 15:20, Daniel P. Berrangé wrote:
>>> On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
>>>> S390 CPU topology may have up to 5 topology containers.
>>>> The first container above the cores is level 2, the sockets,
>>>> and the level 3, containing sockets are the books.
>>>>
>>>> We introduce here the drawers, drawers is the level containing books.
>>>>
>>>> Let's add drawers, level4, containers to the CPU topology.
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> ---
>>>> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
>>>> hw/core/machine.c | 2 ++
>>>> hw/s390x/s390-virtio-ccw.c | 1 +
>>>> include/hw/boards.h | 4 ++++
>>>> qapi/machine.json | 7 ++++++-
>>>> softmmu/vl.c | 3 +++
>>>> 6 files changed, 42 insertions(+), 8 deletions(-)
>>>
>>> Needs to update -smp args in qemu-options.hx too.
>>>
>>
>> Oh, right!
>>
>> Thanks
>>
>>>>
>>
>> ...snip...
>>
>>>> index 73206f811a..fa6bde5617 100644
>>>> --- a/qapi/machine.json
>>>> +++ b/qapi/machine.json
>>>> @@ -866,13 +866,14 @@
>>>> # a CPU is being hotplugged.
>>>> #
>>>> # @node-id: NUMA node ID the CPU belongs to
>>>> +# @drawer-id: drawer number within node/board the CPU belongs to
>>>> # @book-id: book number within node/board the CPU belongs to
>>>> # @socket-id: socket number within node/board the CPU belongs to
>>>
>>> So the lack of change here implies that 'socket-id' is unique
>>> across multiple books/drawers. Is that correct, as its differnt
>>> from semantics for die-id/core-id/thread-id which are scoped
>>> to within the next level of the topology ?
>>
>> Hum, no I forgot to update and it needs a change.
>> What about
>>
>> # @book-id: book number within node/board/drawer the CPU belongs to
>> # @socket-id: socket number within node/board/book the CPU belongs to
>>
>> ?
>
> Probably drawer/node/board and book/node/board to keep a
> low -> high topology ordering
Yes, thanks,
Regards,
Pierre
>
> Regards,
> Daniel
>
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-02-17 15:50 ` Pierre Morel
0 siblings, 0 replies; 36+ messages in thread
From: Pierre Morel @ 2022-02-17 15:50 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: thuth, nrb, ehabkost, kvm, david, philmd, cohuck,
richard.henderson, qemu-devel, armbru, pasic, seiden, qemu-s390x,
mst, pbonzini, eblake, borntraeger
On 2/17/22 16:32, Daniel P. Berrangé wrote:
> On Thu, Feb 17, 2022 at 04:30:06PM +0100, Pierre Morel wrote:
>>
>>
>> On 2/17/22 15:20, Daniel P. Berrangé wrote:
>>> On Thu, Feb 17, 2022 at 02:41:22PM +0100, Pierre Morel wrote:
>>>> S390 CPU topology may have up to 5 topology containers.
>>>> The first container above the cores is level 2, the sockets,
>>>> and the level 3, containing sockets are the books.
>>>>
>>>> We introduce here the drawers, drawers is the level containing books.
>>>>
>>>> Let's add drawers, level4, containers to the CPU topology.
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> ---
>>>> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
>>>> hw/core/machine.c | 2 ++
>>>> hw/s390x/s390-virtio-ccw.c | 1 +
>>>> include/hw/boards.h | 4 ++++
>>>> qapi/machine.json | 7 ++++++-
>>>> softmmu/vl.c | 3 +++
>>>> 6 files changed, 42 insertions(+), 8 deletions(-)
>>>
>>> Needs to update -smp args in qemu-options.hx too.
>>>
>>
>> Oh, right!
>>
>> Thanks
>>
>>>>
>>
>> ...snip...
>>
>>>> index 73206f811a..fa6bde5617 100644
>>>> --- a/qapi/machine.json
>>>> +++ b/qapi/machine.json
>>>> @@ -866,13 +866,14 @@
>>>> # a CPU is being hotplugged.
>>>> #
>>>> # @node-id: NUMA node ID the CPU belongs to
>>>> +# @drawer-id: drawer number within node/board the CPU belongs to
>>>> # @book-id: book number within node/board the CPU belongs to
>>>> # @socket-id: socket number within node/board the CPU belongs to
>>>
>>> So the lack of change here implies that 'socket-id' is unique
>>> across multiple books/drawers. Is that correct, as its differnt
>>> from semantics for die-id/core-id/thread-id which are scoped
>>> to within the next level of the topology ?
>>
>> Hum, no I forgot to update and it needs a change.
>> What about
>>
>> # @book-id: book number within node/board/drawer the CPU belongs to
>> # @socket-id: socket number within node/board/book the CPU belongs to
>>
>> ?
>
> Probably drawer/node/board and book/node/board to keep a
> low -> high topology ordering
Yes, thanks,
Regards,
Pierre
>
> Regards,
> Daniel
>
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 06/11] s390x: topology: Adding books to CPU topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-03-16 3:54 ` wangyanan (Y) via
-1 siblings, 0 replies; 36+ messages in thread
From: wangyanan (Y) @ 2022-03-16 3:54 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Hi Pierre,
On 2022/2/17 21:41, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets.
> We introduce here the books, book is the level containing sockets.
>
> Let's add books, level3, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 29 ++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 38 insertions(+), 8 deletions(-)
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index b39ed21e65..d7aa39d540 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.books_supported) {
> + g_string_append_printf(s, " * books (%u)", ms->smp.books);
> + }
> +
> g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>
Now books become the top level container, string format for sockets should
be tweaked as " * sockets (%u)". Also we need to cut off the " * " at
the head
of the composite topology string.
> if (mc->smp_props.dies_supported) {
> @@ -73,6 +77,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> unsigned clusters = config->has_clusters ? config->clusters : 0;
> @@ -85,6 +90,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> (config->has_clusters && config->clusters == 0) ||
> @@ -111,6 +117,13 @@ void machine_parse_smp_config(MachineState *ms,
> dies = dies > 0 ? dies : 1;
> clusters = clusters > 0 ? clusters : 1;
>
> + if (!mc->smp_props.books_supported && books > 1) {
> + error_setg(errp, "books not supported by this machine's CPU topology");
> + return;
> + }
nit: maybe move above part to the similar sanity checks of "dies and
clusters"?...
> +
> + books = books > 0 ? books : 1;
...and put this line together with the similar operation of "dies and
clusters"
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -124,33 +137,35 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus / (books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus / (books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus / (books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus / (books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (sockets * dies * clusters * cores);
> + threads = maxcpus / (books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
> + maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> ms->smp.clusters = clusters;
> @@ -159,7 +174,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (sockets * dies * clusters * cores * threads != maxcpus) {
> + if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index d856485cb4..b8c624d2bf 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> .has_clusters = true, .clusters = ms->smp.clusters,
> @@ -935,6 +936,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> ms->smp.clusters = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 12903eb2af..193883fba3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -666,6 +666,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> hc->unplug_request = s390_machine_device_unplug_request;
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> + mc->smp_props.books_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index c92ac8815c..bc0f7f22dc 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -130,11 +130,13 @@ typedef struct {
> * @prefer_sockets - whether sockets are preferred over cores in smp parsing
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> + * @books_supported - whether books are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> + bool books_supported;
> } SMPCompatProps;
>
> /**
> @@ -299,6 +301,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @books: the number of books on the machine
> * @sockets: the number of sockets on the machine
I think we may also need to modify the documentation of "sockets"
accordingly, like "sockets: the number of sockets in one book" simply?
We already have some explanation in qemu-options.hx implying that:
machines will have their own meanings of each topology member
if they have different supported topology hierarchies.
> * @dies: the number of dies in one socket
> * @clusters: the number of clusters in one die
> @@ -308,6 +311,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> unsigned int clusters;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 42fc68403d..73206f811a 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,12 +866,13 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @book-id: book number within node/board the CPU belongs to
> # @socket-id: socket number within node/board the CPU belongs to
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 5 properties that could be present
> +# Note: currently there are 6 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -881,6 +882,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> '*core-id': 'int',
> @@ -1400,6 +1402,8 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @books: number of books in the CPU topology
> +#
> # @sockets: number of sockets in the CPU topology
here too: maybe "number of sockets per book in the CPU topology".
> #
> # @dies: number of dies per socket in the CPU topology
> @@ -1416,6 +1420,7 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> '*clusters': 'int',
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 5e1b35ba48..a680fb12d4 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
> {
> .name = "cpus",
> .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "books",
> + .type = QEMU_OPT_NUMBER,
> }, {
> .name = "sockets",
> .type = QEMU_OPT_NUMBER,
Thanks,
Yanan
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 06/11] s390x: topology: Adding books to CPU topology
@ 2022-03-16 3:54 ` wangyanan (Y) via
0 siblings, 0 replies; 36+ messages in thread
From: wangyanan (Y) via @ 2022-03-16 3:54 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Hi Pierre,
On 2022/2/17 21:41, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets.
> We introduce here the books, book is the level containing sockets.
>
> Let's add books, level3, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 29 ++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 38 insertions(+), 8 deletions(-)
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index b39ed21e65..d7aa39d540 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.books_supported) {
> + g_string_append_printf(s, " * books (%u)", ms->smp.books);
> + }
> +
> g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
>
Now books become the top level container, string format for sockets should
be tweaked as " * sockets (%u)". Also we need to cut off the " * " at
the head
of the composite topology string.
> if (mc->smp_props.dies_supported) {
> @@ -73,6 +77,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> unsigned clusters = config->has_clusters ? config->clusters : 0;
> @@ -85,6 +90,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> (config->has_clusters && config->clusters == 0) ||
> @@ -111,6 +117,13 @@ void machine_parse_smp_config(MachineState *ms,
> dies = dies > 0 ? dies : 1;
> clusters = clusters > 0 ? clusters : 1;
>
> + if (!mc->smp_props.books_supported && books > 1) {
> + error_setg(errp, "books not supported by this machine's CPU topology");
> + return;
> + }
nit: maybe move above part to the similar sanity checks of "dies and
clusters"?...
> +
> + books = books > 0 ? books : 1;
...and put this line together with the similar operation of "dies and
clusters"
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -124,33 +137,35 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus / (books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus / (books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (sockets * dies * clusters * threads);
> + cores = maxcpus / (books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (dies * clusters * cores * threads);
> + sockets = maxcpus / (books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (sockets * dies * clusters * cores);
> + threads = maxcpus / (books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
> + maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> ms->smp.clusters = clusters;
> @@ -159,7 +174,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (sockets * dies * clusters * cores * threads != maxcpus) {
> + if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index d856485cb4..b8c624d2bf 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> .has_clusters = true, .clusters = ms->smp.clusters,
> @@ -935,6 +936,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> ms->smp.clusters = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 12903eb2af..193883fba3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -666,6 +666,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> hc->unplug_request = s390_machine_device_unplug_request;
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> + mc->smp_props.books_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index c92ac8815c..bc0f7f22dc 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -130,11 +130,13 @@ typedef struct {
> * @prefer_sockets - whether sockets are preferred over cores in smp parsing
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> + * @books_supported - whether books are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> + bool books_supported;
> } SMPCompatProps;
>
> /**
> @@ -299,6 +301,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @books: the number of books on the machine
> * @sockets: the number of sockets on the machine
I think we may also need to modify the documentation of "sockets"
accordingly, like "sockets: the number of sockets in one book" simply?
We already have some explanation in qemu-options.hx implying that:
machines will have their own meanings of each topology member
if they have different supported topology hierarchies.
> * @dies: the number of dies in one socket
> * @clusters: the number of clusters in one die
> @@ -308,6 +311,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> unsigned int clusters;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 42fc68403d..73206f811a 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,12 +866,13 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @book-id: book number within node/board the CPU belongs to
> # @socket-id: socket number within node/board the CPU belongs to
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 5 properties that could be present
> +# Note: currently there are 6 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -881,6 +882,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> '*core-id': 'int',
> @@ -1400,6 +1402,8 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @books: number of books in the CPU topology
> +#
> # @sockets: number of sockets in the CPU topology
here too: maybe "number of sockets per book in the CPU topology".
> #
> # @dies: number of dies per socket in the CPU topology
> @@ -1416,6 +1420,7 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> '*clusters': 'int',
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 5e1b35ba48..a680fb12d4 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
> {
> .name = "cpus",
> .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "books",
> + .type = QEMU_OPT_NUMBER,
> }, {
> .name = "sockets",
> .type = QEMU_OPT_NUMBER,
Thanks,
Yanan
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
2022-02-17 13:41 ` Pierre Morel
@ 2022-03-16 6:50 ` wangyanan (Y) via
-1 siblings, 0 replies; 36+ messages in thread
From: wangyanan (Y) @ 2022-03-16 6:50 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Hi Pierre,
On 2022/2/17 21:41, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets,
> and the level 3, containing sockets are the books.
>
> We introduce here the drawers, drawers is the level containing books.
>
> Let's add drawers, level4, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 42 insertions(+), 8 deletions(-)
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index d7aa39d540..26150c748f 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
> + }
> +
As mentioned in patch#6, we need to take care of the head of the
composite topology string.
> if (mc->smp_props.books_supported) {
> g_string_append_printf(s, " * books (%u)", ms->smp.books);
> }
> @@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> @@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> @@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
>
> books = books > 0 ? books : 1;
>
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp, "drawers not supported by this machine's CPU topology");
> + return;
> + }
nit: maybe move above part to the similar sanity checks of "dies and
clusters"?...
> +
> + drawers = drawers > 0 ? drawers : 1;
...and put this line together with the similar operation of "dies and
clusters".
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (books * sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> @@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index b8c624d2bf..1db55e36c8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> @@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 193883fba3..03829e90b3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index bc0f7f22dc..abc5556c50 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -131,12 +131,14 @@ typedef struct {
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> * @books_supported - whether books are supported by the machine
> + * @drawers_supported - whether drawers are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> bool books_supported;
> + bool drawers_supported;
> } SMPCompatProps;
>
> /**
> @@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @drawers: the number of drawers on the machine
> * @books: the number of books on the machine
@books: the number of books in one drawer
> * @sockets: the number of sockets on the machine
@sockets: the number of sockets in one book
> * @dies: the number of dies in one socket
> @@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int drawers;
> unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 73206f811a..fa6bde5617 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,13 +866,14 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to
> # @book-id: book number within node/board the CPU belongs to
Since new drawer-id and book-id are introduced, you may need to
check whether to update function machine_set_cpu_numa_node()
and function hmp_hotpluggable_cpus(). We can also add some
testcases for books/drawers in test-smp-parse.c if necessary.
> # @socket-id: socket number within node/board the CPU belongs to
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present
> +# Note: currently there are 7 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -882,6 +883,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> @@ -1402,6 +1404,8 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @drawers: number of drawers in the CPU topology
> +#
> # @books: number of books in the CPU topology
@books: number of books per drawer in the CPU topology
> #
> # @sockets: number of sockets in the CPU topology
@sockets: number of sockets per book in the CPU topology
> @@ -1420,6 +1424,7 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*drawers': 'int',
> '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index a680fb12d4..f4c9f3a536 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
> {
> .name = "cpus",
> .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "drawers",
> + .type = QEMU_OPT_NUMBER,
> }, {
> .name = "books",
> .type = QEMU_OPT_NUMBER,
Thanks,
Yanan
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology
@ 2022-03-16 6:50 ` wangyanan (Y) via
0 siblings, 0 replies; 36+ messages in thread
From: wangyanan (Y) via @ 2022-03-16 6:50 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, philmd,
eblake, armbru, seiden, nrb
Hi Pierre,
On 2022/2/17 21:41, Pierre Morel wrote:
> S390 CPU topology may have up to 5 topology containers.
> The first container above the cores is level 2, the sockets,
> and the level 3, containing sockets are the books.
>
> We introduce here the drawers, drawers is the level containing books.
>
> Let's add drawers, level4, containers to the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/core/machine-smp.c | 33 ++++++++++++++++++++++++++-------
> hw/core/machine.c | 2 ++
> hw/s390x/s390-virtio-ccw.c | 1 +
> include/hw/boards.h | 4 ++++
> qapi/machine.json | 7 ++++++-
> softmmu/vl.c | 3 +++
> 6 files changed, 42 insertions(+), 8 deletions(-)
>
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
> index d7aa39d540..26150c748f 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-smp.c
> @@ -31,6 +31,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> GString *s = g_string_new(NULL);
>
> + if (mc->smp_props.drawers_supported) {
> + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
> + }
> +
As mentioned in patch#6, we need to take care of the head of the
composite topology string.
> if (mc->smp_props.books_supported) {
> g_string_append_printf(s, " * books (%u)", ms->smp.books);
> }
> @@ -77,6 +81,7 @@ void machine_parse_smp_config(MachineState *ms,
> {
> MachineClass *mc = MACHINE_GET_CLASS(ms);
> unsigned cpus = config->has_cpus ? config->cpus : 0;
> + unsigned drawers = config->has_drawers ? config->drawers : 0;
> unsigned books = config->has_books ? config->books : 0;
> unsigned sockets = config->has_sockets ? config->sockets : 0;
> unsigned dies = config->has_dies ? config->dies : 0;
> @@ -90,6 +95,7 @@ void machine_parse_smp_config(MachineState *ms,
> * explicit configuration like "cpus=0" is not allowed.
> */
> if ((config->has_cpus && config->cpus == 0) ||
> + (config->has_drawers && config->drawers == 0) ||
> (config->has_books && config->books == 0) ||
> (config->has_sockets && config->sockets == 0) ||
> (config->has_dies && config->dies == 0) ||
> @@ -124,6 +130,13 @@ void machine_parse_smp_config(MachineState *ms,
>
> books = books > 0 ? books : 1;
>
> + if (!mc->smp_props.drawers_supported && drawers > 1) {
> + error_setg(errp, "drawers not supported by this machine's CPU topology");
> + return;
> + }
nit: maybe move above part to the similar sanity checks of "dies and
clusters"?...
> +
> + drawers = drawers > 0 ? drawers : 1;
...and put this line together with the similar operation of "dies and
clusters".
> +
> /* compute missing values based on the provided ones */
> if (cpus == 0 && maxcpus == 0) {
> sockets = sockets > 0 ? sockets : 1;
> @@ -137,34 +150,40 @@ void machine_parse_smp_config(MachineState *ms,
> if (sockets == 0) {
> cores = cores > 0 ? cores : 1;
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> } else if (cores == 0) {
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> }
> } else {
> /* prefer cores over sockets since 6.2 */
> if (cores == 0) {
> sockets = sockets > 0 ? sockets : 1;
> threads = threads > 0 ? threads : 1;
> - cores = maxcpus / (books * sockets * dies * clusters * threads);
> + cores = maxcpus /
> + (drawers * books * sockets * dies * clusters * threads);
> } else if (sockets == 0) {
> threads = threads > 0 ? threads : 1;
> - sockets = maxcpus / (books * dies * clusters * cores * threads);
> + sockets = maxcpus /
> + (drawers * books * dies * clusters * cores * threads);
> }
> }
>
> /* try to calculate omitted threads at last */
> if (threads == 0) {
> - threads = maxcpus / (books * sockets * dies * clusters * cores);
> + threads = maxcpus /
> + (drawers * books * sockets * dies * clusters * cores);
> }
> }
>
> - maxcpus = maxcpus > 0 ? maxcpus : books * sockets * dies *
> + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
> clusters * cores * threads;
> cpus = cpus > 0 ? cpus : maxcpus;
>
> ms->smp.cpus = cpus;
> + ms->smp.drawers = drawers;
> ms->smp.books = books;
> ms->smp.sockets = sockets;
> ms->smp.dies = dies;
> @@ -174,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms,
> ms->smp.max_cpus = maxcpus;
>
> /* sanity-check of the computed topology */
> - if (books * sockets * dies * clusters * cores * threads != maxcpus) {
> + if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) {
> g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
> error_setg(errp, "Invalid CPU topology: "
> "product of the hierarchy must match maxcpus: "
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index b8c624d2bf..1db55e36c8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -743,6 +743,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
> MachineState *ms = MACHINE(obj);
> SMPConfiguration *config = &(SMPConfiguration){
> .has_cpus = true, .cpus = ms->smp.cpus,
> + .has_drawers = true, .drawers = ms->smp.drawers,
> .has_books = true, .books = ms->smp.books,
> .has_sockets = true, .sockets = ms->smp.sockets,
> .has_dies = true, .dies = ms->smp.dies,
> @@ -936,6 +937,7 @@ static void machine_initfn(Object *obj)
> /* default to mc->default_cpus */
> ms->smp.cpus = mc->default_cpus;
> ms->smp.max_cpus = mc->default_cpus;
> + ms->smp.drawers = 1;
> ms->smp.books = 1;
> ms->smp.sockets = 1;
> ms->smp.dies = 1;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 193883fba3..03829e90b3 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -667,6 +667,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
> nc->nmi_monitor_handler = s390_nmi;
> mc->default_ram_id = "s390.ram";
> mc->smp_props.books_supported = true;
> + mc->smp_props.drawers_supported = true;
> }
>
> static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index bc0f7f22dc..abc5556c50 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -131,12 +131,14 @@ typedef struct {
> * @dies_supported - whether dies are supported by the machine
> * @clusters_supported - whether clusters are supported by the machine
> * @books_supported - whether books are supported by the machine
> + * @drawers_supported - whether drawers are supported by the machine
> */
> typedef struct {
> bool prefer_sockets;
> bool dies_supported;
> bool clusters_supported;
> bool books_supported;
> + bool drawers_supported;
> } SMPCompatProps;
>
> /**
> @@ -301,6 +303,7 @@ typedef struct DeviceMemoryState {
> /**
> * CpuTopology:
> * @cpus: the number of present logical processors on the machine
> + * @drawers: the number of drawers on the machine
> * @books: the number of books on the machine
@books: the number of books in one drawer
> * @sockets: the number of sockets on the machine
@sockets: the number of sockets in one book
> * @dies: the number of dies in one socket
> @@ -311,6 +314,7 @@ typedef struct DeviceMemoryState {
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> + unsigned int drawers;
> unsigned int books;
> unsigned int sockets;
> unsigned int dies;
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 73206f811a..fa6bde5617 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -866,13 +866,14 @@
> # a CPU is being hotplugged.
> #
> # @node-id: NUMA node ID the CPU belongs to
> +# @drawer-id: drawer number within node/board the CPU belongs to
> # @book-id: book number within node/board the CPU belongs to
Since new drawer-id and book-id are introduced, you may need to
check whether to update function machine_set_cpu_numa_node()
and function hmp_hotpluggable_cpus(). We can also add some
testcases for books/drawers in test-smp-parse.c if necessary.
> # @socket-id: socket number within node/board the CPU belongs to
> # @die-id: die number within socket the CPU belongs to (since 4.1)
> # @core-id: core number within die the CPU belongs to
> # @thread-id: thread number within core the CPU belongs to
> #
> -# Note: currently there are 6 properties that could be present
> +# Note: currently there are 7 properties that could be present
> # but management should be prepared to pass through other
> # properties with device_add command to allow for future
> # interface extension. This also requires the filed names to be kept in
> @@ -882,6 +883,7 @@
> ##
> { 'struct': 'CpuInstanceProperties',
> 'data': { '*node-id': 'int',
> + '*drawer-id': 'int',
> '*book-id': 'int',
> '*socket-id': 'int',
> '*die-id': 'int',
> @@ -1402,6 +1404,8 @@
> #
> # @cpus: number of virtual CPUs in the virtual machine
> #
> +# @drawers: number of drawers in the CPU topology
> +#
> # @books: number of books in the CPU topology
@books: number of books per drawer in the CPU topology
> #
> # @sockets: number of sockets in the CPU topology
@sockets: number of sockets per book in the CPU topology
> @@ -1420,6 +1424,7 @@
> ##
> { 'struct': 'SMPConfiguration', 'data': {
> '*cpus': 'int',
> + '*drawers': 'int',
> '*books': 'int',
> '*sockets': 'int',
> '*dies': 'int',
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index a680fb12d4..f4c9f3a536 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -720,6 +720,9 @@ static QemuOptsList qemu_smp_opts = {
> {
> .name = "cpus",
> .type = QEMU_OPT_NUMBER,
> + }, {
> + .name = "drawers",
> + .type = QEMU_OPT_NUMBER,
> }, {
> .name = "books",
> .type = QEMU_OPT_NUMBER,
Thanks,
Yanan
^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2022-03-16 6:53 UTC | newest]
Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-17 13:41 [PATCH v6 00/11] s390x: CPU Topology Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 01/11] s390x: SCLP: reporting the maximum nested topology entries Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 02/11] s390x: topology: CPU topology objects and structures Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 03/11] s390x: topology: implementating Store Topology System Information Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 04/11] s390x: CPU topology: CPU topology migration Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 05/11] s390x: kvm: topology: interception of PTF instruction Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 06/11] s390x: topology: Adding books to CPU topology Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-03-16 3:54 ` wangyanan (Y)
2022-03-16 3:54 ` wangyanan (Y) via
2022-02-17 13:41 ` [PATCH v6 07/11] s390x: topology: Adding books to STSI Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 08/11] s390x: topology: Adding drawers to CPU topology Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 14:20 ` Daniel P. Berrangé
2022-02-17 14:20 ` Daniel P. Berrangé
2022-02-17 15:30 ` Pierre Morel
2022-02-17 15:30 ` Pierre Morel
2022-02-17 15:32 ` Daniel P. Berrangé
2022-02-17 15:32 ` Daniel P. Berrangé
2022-02-17 15:50 ` Pierre Morel
2022-02-17 15:50 ` Pierre Morel
2022-03-16 6:50 ` wangyanan (Y)
2022-03-16 6:50 ` wangyanan (Y) via
2022-02-17 13:41 ` [PATCH v6 09/11] s390x: topology: Adding drawers to STSI Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 10/11] s390x: topology: implementing numa for the s390x topology Pierre Morel
2022-02-17 13:41 ` Pierre Morel
2022-02-17 13:41 ` [PATCH v6 11/11] s390x: topology: documentation Pierre Morel
2022-02-17 13:41 ` Pierre Morel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.