* [PATCH v15 00/11] s390x: CPU Topology
@ 2023-02-01 13:20 Pierre Morel
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
` (11 more replies)
0 siblings, 12 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Hi,
Two important changes in this series:
- The ordering of the Topology List Entries, TLE, is done just
before building the System information Buffer, SYSIB, during
the interception of STSI.
The adventage of this is to do the ordering only if it is needed
whatever the reason for a change of the topology is, hotplug,
qapi, instruction PTF(2) from the guest.
- query-topology is no longer needed, it is replaced by an
extension of query-cpus-fast.
A small change that lead me to remove @Thomas RB:
- Add a trampolin function in cpu-topology code for the
subsystem reset for extending to PTF reset in the following
patch on PTF implementation.
@Nina and @Nico RB have been removed from reset patch too in the last
series because of small changes too.
Implementation discussions
==========================
CPU models
----------
Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already
in the CPU model for old QEMU we could not activate it as usual from
KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
Checking and enabling this capability enables facility 11,
S390_FEAT_CONFIGURATION_TOPOLOGY.
It is the responsibility of the admin to ensure the same CPU
model for source and target host in a migration.
Migration
---------
When the target guest is started, the Multi-processor Topology Change
Report (MTCR) bit is set during the creation of the vCPU by KVM.
We do not need to migrate its state, in the worst case, the target
guest will see the MTCR and actualize its view of the topology
without necessity, but this will be done only one time.
Reset
-----
Reseting the topology is done during subsystem reset, the
polarization is reset to horizontal polarization.
Topology attributes
-------------------
The topology attributes are carried by the CPU object and defined
on object creation.
In the case the new attributes, socket, book, drawer, dedicated,
polarity are not provided QEMU provides defaults values.
- Geometry defaults
The geometry default are based on the core-id of the core to
fill the geometry in a monotone way starting with drawer 0,
book 0, and filling socket 0 with the number of cores per socket,
then filling socket 1, socket 2 ... etc until the book is complete
and all books until the first drawer is complete before starting with
the next drawer.
This allows to keep existing start scripts and Libvirt existing
interface until it is extended.
- Modifiers defaults
Default polarization is horizontal
Default dedication is not dedicated.
Dynamic topology modification
-----------------------------
QAPI interface is extended with:
- a command: 'x-set-cpu-topology'
- a query: extension of 'query-cpus-fast'
- an event: 'CPU_POLARITY_CHANGE'
The admin may use query-cpus-fast to verify the topology provided
to the guest and x-set-cpu-topology to modify it.
The event CPU_POLARITY_CHANGE is sent when the guest successfuly
uses the PTF(2) instruction to request a polarization change.
In that case, the admin is supposed to modify the CPU provisioning
accordingly.
Testing
=======
To use the QEMU patches, you will need Linux V6-rc1 or newer,
or use the following Linux mainline patches:
f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report
24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function
0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..
Currently this code is for KVM only, I have no idea if it is interesting
to provide a TCG patch. If ever it will be done in another series.
Documentation
=============
To have a better understanding of the S390x CPU Topology and its
implementation in QEMU you can have a look at the documentation in the
last patch of this series.
The admin will want to match the host and the guest topology, taking
into account that the guest does not recognize multithreading.
Consequently, two vCPU assigned to threads of the same real CPU should
preferably be assigned to the same socket of the guest machine.
Regards,
Pierre
Pierre Morel (11):
s390x/cpu topology: adding s390 specificities to CPU topology
s390x/cpu topology: add topology entries on CPU hotplug
target/s390x/cpu topology: handle STSI(15) and build the SYSIB
s390x/sclp: reporting the maximum nested topology entries
s390x/cpu topology: resetting the Topology-Change-Report
s390x/cpu topology: interception of PTF instruction
target/s390x/cpu topology: activating CPU topology
qapi/s390x/cpu topology: x-set-cpu-topology monitor command
machine: adding s390 topology to query-cpu-fast
qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
docs/s390x/cpu topology: document s390x cpu topology
docs/system/s390x/cpu-topology.rst | 294 ++++++++++++++++++
docs/system/target-s390x.rst | 1 +
qapi/machine-target.json | 59 ++++
qapi/machine.json | 27 +-
include/hw/boards.h | 10 +-
include/hw/s390x/cpu-topology.h | 71 +++++
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
include/monitor/hmp.h | 1 +
target/s390x/cpu.h | 79 +++++
target/s390x/kvm/kvm_s390x.h | 1 +
hw/core/machine-qmp-cmds.c | 2 +
hw/core/machine-smp.c | 48 ++-
hw/core/machine.c | 4 +
hw/s390x/cpu-topology.c | 471 +++++++++++++++++++++++++++++
hw/s390x/s390-virtio-ccw.c | 28 +-
hw/s390x/sclp.c | 5 +
softmmu/vl.c | 6 +
target/s390x/cpu-sysemu.c | 27 ++
target/s390x/cpu.c | 7 +
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++
target/s390x/kvm/kvm.c | 45 ++-
hmp-commands.hx | 16 +
hw/s390x/meson.build | 1 +
qemu-options.hx | 7 +-
target/s390x/kvm/meson.build | 3 +-
27 files changed, 1539 insertions(+), 20 deletions(-)
create mode 100644 docs/system/s390x/cpu-topology.rst
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 target/s390x/kvm/cpu_topology.c
--
2.31.1
Since v14:
- move the ordering of TLE to just before filling the SYSIB,
optimize TLE ordering to be done on need only.
(Cedric in previous series)
- remove 'query-topology' and instead extend 'query-cpus-fast'
(Daniel)
- rename POLARITY_CHANGE to CPU_POLARITY_CHANGE
(Thomas)
- Divers bugs correction and doc changes
(Thomas, Nina)
- Separate topology and entitlement, simplify pft handling
(Nina)
- add the resetting of all CPU to horizontal polarity
once implementing PTF interpretation
Since v13:
- Suppress the topology device to simplify the code
(Cedric)
- moved reset of MTCR from device reset into subsystem
reset and removed previous reviewed-by from Nico and
Janis
- No need for Migration
- No need for machine dependencies
(Christian, Thomas)
- Adding all features, drawer/book and dynamic
(Cedric)
- since v12
- suppress new CPU flag "disable-topology" just use ctop
- no use of special fields in CCW machine or in CPU
- modifications in documentation
- insert documentation in tree
(Cedric)
- moved cpu-topology.c from target/s390 to target/s390/kvm
to compile smoothly (without topology) for TCG
(Cedric)
- since v11
- new CPU flag "disable-topology"
I would have take "topology" if I was able to have
it false on default.
(Christian, Thomas)
- Build the topology during the interception of the
STSI instruction.
(Cedric)
- return CC3 in case the calculated SYSIB length is
greater than 4096.
(Janis)
- minor corections on documentation
- since v10
- change machine attribute "topology-disable" to "topology"
(Cedric)
- Add preliminary patch for machine properties
(Cedric)
- Use next machine as 7.2
(Cedric / Connie)
- Remove unecessary mutex
(Thomas)
- use ENOTSUP return value for kvm_s390_topology_set_mtcr()
(Cedric)
- Add explanation on container and cpu TLEs
(Thomas)
- use again cpu and socket count in topology structure
(Cedric)
- Suppress the S390TopoTLE structure and integrate
the TLE masks to the socket structure.
(-)
- the STSI instruction now finds the topology from the machine
(Cedric)
- since v9
- remove books and drawers
- remove thread denying and replace with a merge
of cores * threads to specify the CPUs available
to the guest
- add a class option to avoid topology on older
machines
(Cedric)
- Allocate a SYSIB buffer of the maximal length to
avoid overflow.
(Nico, Janis)
- suppress redundancy of smp parameters in topology
and use directly the machine smp structure
- Early check for topology support
(Cedric)
- since v8
- Linux patches are now mainline
- simplification of the implementation
(Janis)
- Migration, new machine definition
(Thomas)
- Documentation
- since v7
- Coherence with the Linux patch series changes for MTCR get
(Pierre)
- check return values during new CPU creation
(Thomas)
- Improving codding style and argument usages
(Thomas)
- since v6
- Changes on smp args in qemu-options
(Daniel)
- changed comments in machine.jason
(Daniel)
- Added reset
(Janosch)
- since v5
- rebasing on newer QEMU version
- reworked most lines above 80 characters.
- since v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
^ permalink raw reply [flat|nested] 76+ messages in thread
* [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-02 10:44 ` Thomas Huth
` (2 more replies)
2023-02-01 13:20 ` [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
` (10 subsequent siblings)
11 siblings, 3 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
S390 adds two new SMP levels, drawers and books to the CPU
topology.
The S390 CPU have specific toplogy features like dedication
and polarity to give to the guest indications on the host
vCPUs scheduling and help the guest take the best decisions
on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and polarity,
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine.json | 14 ++++++++--
include/hw/boards.h | 10 ++++++-
include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
target/s390x/cpu.h | 5 ++++
hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
hw/core/machine.c | 4 +++
hw/s390x/s390-virtio-ccw.c | 2 ++
softmmu/vl.c | 6 +++++
target/s390x/cpu.c | 7 +++++
qemu-options.hx | 7 +++--
10 files changed, 115 insertions(+), 12 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
diff --git a/qapi/machine.json b/qapi/machine.json
index b9228a5e46..3036117059 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -900,13 +900,15 @@
# a CPU is being hotplugged.
#
# @node-id: NUMA node ID the CPU belongs to
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to (since 8.0)
+# @book-id: book number within drawer/node/board the CPU belongs to (since 8.0)
+# @socket-id: socket number within book/node/board the CPU belongs to
# @die-id: die number within socket the CPU belongs to (since 4.1)
# @cluster-id: cluster number within die the CPU belongs to (since 7.1)
# @core-id: core number within cluster the CPU belongs to
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present
+# Note: currently there are 8 properties that could be present
# but management should be prepared to pass through other
# properties with device_add command to allow for future
# interface extension. This also requires the filed names to be kept in
@@ -916,6 +918,8 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+ '*drawer-id': 'int',
+ '*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1465,6 +1469,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.0)
+#
+# @books: number of books in the CPU topology (since 8.0)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1481,6 +1489,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6fbbfd56c8..9ef0bb76cf 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -131,12 +131,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool has_clusters;
+ bool books_supported;
+ bool drawers_supported;
} SMPCompatProps;
/**
@@ -301,7 +305,9 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
- * @sockets: the number of sockets on the machine
+ * @drawers: the number of drawers on the machine
+ * @books: the number of books in one drawer
+ * @sockets: the number of sockets in one book
* @dies: the number of dies in one socket
* @clusters: the number of clusters in one die
* @cores: the number of cores in one cluster
@@ -310,6 +316,8 @@ typedef struct DeviceMemoryState {
*/
typedef struct CpuTopology {
unsigned int cpus;
+ unsigned int drawers;
+ unsigned int books;
unsigned int sockets;
unsigned int dies;
unsigned int clusters;
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 0000000000..7a84b30a21
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,24 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+enum s390_topology_polarity {
+ POLARITY_HORIZONTAL,
+ POLARITY_VERTICAL,
+ POLARITY_VERTICAL_LOW = 1,
+ POLARITY_VERTICAL_MEDIUM,
+ POLARITY_VERTICAL_HIGH,
+ POLARITY_MAX,
+};
+
+#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 7d6d01325b..d654267a71 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -131,6 +131,11 @@ struct CPUArchState {
#if !defined(CONFIG_USER_ONLY)
uint32_t core_id; /* PoP "CPU address", same as cpu_index */
+ int32_t socket_id;
+ int32_t book_id;
+ int32_t drawer_id;
+ bool dedicated;
+ uint8_t entitlement; /* Used only for vertical polarization */
uint64_t cpuid;
#endif
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
index c3dab007da..b8233df5a9 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-smp.c
@@ -31,6 +31,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
MachineClass *mc = MACHINE_GET_CLASS(ms);
GString *s = g_string_new(NULL);
+ if (mc->smp_props.drawers_supported) {
+ g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers);
+ }
+
+ if (mc->smp_props.books_supported) {
+ g_string_append_printf(s, " * books (%u)", ms->smp.books);
+ }
+
g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
if (mc->smp_props.dies_supported) {
@@ -73,6 +81,8 @@ void machine_parse_smp_config(MachineState *ms,
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
unsigned cpus = config->has_cpus ? config->cpus : 0;
+ unsigned drawers = config->has_drawers ? config->drawers : 0;
+ unsigned books = config->has_books ? config->books : 0;
unsigned sockets = config->has_sockets ? config->sockets : 0;
unsigned dies = config->has_dies ? config->dies : 0;
unsigned clusters = config->has_clusters ? config->clusters : 0;
@@ -85,6 +95,8 @@ void machine_parse_smp_config(MachineState *ms,
* explicit configuration like "cpus=0" is not allowed.
*/
if ((config->has_cpus && config->cpus == 0) ||
+ (config->has_drawers && config->drawers == 0) ||
+ (config->has_books && config->books == 0) ||
(config->has_sockets && config->sockets == 0) ||
(config->has_dies && config->dies == 0) ||
(config->has_clusters && config->clusters == 0) ||
@@ -111,6 +123,19 @@ void machine_parse_smp_config(MachineState *ms,
dies = dies > 0 ? dies : 1;
clusters = clusters > 0 ? clusters : 1;
+ if (!mc->smp_props.books_supported && books > 1) {
+ error_setg(errp, "books not supported by this machine's CPU topology");
+ return;
+ }
+ books = books > 0 ? books : 1;
+
+ if (!mc->smp_props.drawers_supported && drawers > 1) {
+ error_setg(errp,
+ "drawers not supported by this machine's CPU topology");
+ return;
+ }
+ drawers = drawers > 0 ? drawers : 1;
+
/* compute missing values based on the provided ones */
if (cpus == 0 && maxcpus == 0) {
sockets = sockets > 0 ? sockets : 1;
@@ -124,33 +149,41 @@ void machine_parse_smp_config(MachineState *ms,
if (sockets == 0) {
cores = cores > 0 ? cores : 1;
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
} else if (cores == 0) {
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
}
} else {
/* prefer cores over sockets since 6.2 */
if (cores == 0) {
sockets = sockets > 0 ? sockets : 1;
threads = threads > 0 ? threads : 1;
- cores = maxcpus / (sockets * dies * clusters * threads);
+ cores = maxcpus /
+ (drawers * books * sockets * dies * clusters * threads);
} else if (sockets == 0) {
threads = threads > 0 ? threads : 1;
- sockets = maxcpus / (dies * clusters * cores * threads);
+ sockets = maxcpus /
+ (drawers * books * dies * clusters * cores * threads);
}
}
/* try to calculate omitted threads at last */
if (threads == 0) {
- threads = maxcpus / (sockets * dies * clusters * cores);
+ threads = maxcpus /
+ (drawers * books * sockets * dies * clusters * cores);
}
}
- maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
+ maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies *
+ clusters * cores * threads;
cpus = cpus > 0 ? cpus : maxcpus;
ms->smp.cpus = cpus;
+ ms->smp.drawers = drawers;
+ ms->smp.books = books;
ms->smp.sockets = sockets;
ms->smp.dies = dies;
ms->smp.clusters = clusters;
@@ -161,7 +194,8 @@ void machine_parse_smp_config(MachineState *ms,
mc->smp_props.has_clusters = config->has_clusters;
/* sanity-check of the computed topology */
- if (sockets * dies * clusters * cores * threads != maxcpus) {
+ if (drawers * books * sockets * dies * clusters * cores * threads !=
+ maxcpus) {
g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);
error_setg(errp, "Invalid CPU topology: "
"product of the hierarchy must match maxcpus: "
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 616f3a207c..e38f99230b 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -828,6 +828,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
MachineState *ms = MACHINE(obj);
SMPConfiguration *config = &(SMPConfiguration){
.has_cpus = true, .cpus = ms->smp.cpus,
+ .has_drawers = true, .drawers = ms->smp.drawers,
+ .has_books = true, .books = ms->smp.books,
.has_sockets = true, .sockets = ms->smp.sockets,
.has_dies = true, .dies = ms->smp.dies,
.has_clusters = true, .clusters = ms->smp.clusters,
@@ -1093,6 +1095,8 @@ static void machine_initfn(Object *obj)
/* default to mc->default_cpus */
ms->smp.cpus = mc->default_cpus;
ms->smp.max_cpus = mc->default_cpus;
+ ms->smp.drawers = 1;
+ ms->smp.books = 1;
ms->smp.sockets = 1;
ms->smp.dies = 1;
ms->smp.clusters = 1;
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index f22f61b8b6..f3cc845d3b 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -733,6 +733,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
mc->no_sdcard = 1;
mc->max_cpus = S390_MAX_CPUS;
mc->has_hotpluggable_cpus = true;
+ mc->smp_props.books_supported = true;
+ mc->smp_props.drawers_supported = true;
assert(!mc->get_hotplug_handler);
mc->get_hotplug_handler = s390_get_hotplug_handler;
mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 9177d95d4e..05e1f33fa5 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -727,6 +727,12 @@ static QemuOptsList qemu_smp_opts = {
{
.name = "cpus",
.type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "drawers",
+ .type = QEMU_OPT_NUMBER,
+ }, {
+ .name = "books",
+ .type = QEMU_OPT_NUMBER,
}, {
.name = "sockets",
.type = QEMU_OPT_NUMBER,
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index b10a8541ff..828e8b8fce 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -37,6 +37,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC2000000UL;
@@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)
static Property s390x_cpu_properties[] = {
#if !defined(CONFIG_USER_ONLY)
DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0),
+ DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1),
+ DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1),
+ DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1),
+ DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false),
+ DEFINE_PROP_UINT8("polarity", S390CPU, env.entitlement,
+ POLARITY_VERTICAL_MEDIUM),
#endif
DEFINE_PROP_END_OF_LIST()
};
diff --git a/qemu-options.hx b/qemu-options.hx
index d59d19704b..131687c6dc 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -250,11 +250,14 @@ SRST
ERST
DEF("smp", HAS_ARG, QEMU_OPTION_smp,
- "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
+ "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"
+ " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n"
" set the number of initial CPUs to 'n' [default=1]\n"
" maxcpus= maximum number of total CPUs, including\n"
" offline CPUs for hotplug, etc\n"
- " sockets= number of sockets on the machine board\n"
+ " drawers= number of drawers on the machine board\n"
+ " books= number of books in one drawer\n"
+ " sockets= number of sockets in one book\n"
" dies= number of dies in one socket\n"
" clusters= number of clusters in one die\n"
" cores= number of cores in one cluster\n"
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-02 16:42 ` Nina Schoetterl-Glausch
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
` (9 subsequent siblings)
11 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
include/hw/s390x/cpu-topology.h | 24 +++
hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
hw/s390x/s390-virtio-ccw.c | 23 ++-
hw/s390x/meson.build | 1 +
4 files changed, 302 insertions(+), 2 deletions(-)
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 7a84b30a21..9b6f889ad4 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -10,6 +10,9 @@
#ifndef HW_S390X_CPU_TOPOLOGY_H
#define HW_S390X_CPU_TOPOLOGY_H
+#include "qemu/queue.h"
+#include "hw/boards.h"
+
#define S390_TOPOLOGY_CPU_IFL 0x03
enum s390_topology_polarity {
@@ -21,4 +24,25 @@ enum s390_topology_polarity {
POLARITY_MAX,
};
+typedef struct S390Topology {
+ uint8_t *cores_per_socket;
+ CpuTopology *smp;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_set_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+int s390_socket_nb(S390CPU *cpu);
+
#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 0000000000..12df4eca6c
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,256 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .list: queue the topology entries inside which
+ * we keep the information on the CPU topology.
+ * .socket: tracks information on the count of cores per socket.
+ * .smp: keeps track of the machine topology.
+ *
+ */
+S390Topology s390_topology = {
+ /* will be initialized after the cpu model is realized */
+ .cores_per_socket = NULL,
+ .smp = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a cpu.
+ */
+int s390_socket_nb(S390CPU *cpu)
+{
+ return (cpu->env.drawer_id * s390_topology.smp->books + cpu->env.book_id) *
+ s390_topology.smp->sockets + cpu->env.socket_id;
+}
+
+/**
+ * s390_has_topology:
+ *
+ * Return value: if the topology is supported by the machine.
+ */
+bool s390_has_topology(void)
+{
+ return false;
+}
+
+/**
+ * s390_topology_init:
+ * @ms: the machine state where the machine topology is defined
+ *
+ * Keep track of the machine topology.
+ *
+ * Allocate an array to keep the count of cores per socket.
+ * The index of the array starts at socket 0 from book 0 and
+ * drawer 0 up to the maximum allowed by the machine topology.
+ */
+static void s390_topology_init(MachineState *ms)
+{
+ CpuTopology *smp = &ms->smp;
+
+ s390_topology.smp = smp;
+ s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
+ smp->books * smp->drawers);
+}
+
+/**
+ * s390_topology_cpu_default:
+ * @cpu: pointer to a S390CPU
+ * @errp: Error pointer
+ *
+ * Setup the default topology for unset attributes.
+ *
+ * The function accept only all all default values or all set values
+ * for the geometry topology.
+ *
+ * The function calculates the (drawer_id, book_id, socket_id)
+ * topology by filling the cores starting from the first socket
+ * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
+ *
+ * CPU type, polarity and dedication have defaults values set in the
+ * s390x_cpu_properties.
+ */
+static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
+{
+ CpuTopology *smp = s390_topology.smp;
+ CPUS390XState *env = &cpu->env;
+
+ /* All geometry topology attributes must be set or all unset */
+ if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
+ (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
+ error_setg(errp,
+ "Please define all or none of the topology geometry attributes");
+ return;
+ }
+
+ /* Check if one of the geometry topology is unset */
+ if (env->socket_id < 0) {
+ /* Calculate default geometry topology attributes */
+ env->socket_id = (env->core_id / smp->cores) % smp->sockets;
+ env->book_id = (env->core_id / (smp->sockets * smp->cores)) %
+ smp->books;
+ env->drawer_id = (env->core_id /
+ (smp->books * smp->sockets * smp->cores)) %
+ smp->drawers;
+ }
+}
+
+/**
+ * s390_topology_check:
+ * @cpu: s390x CPU to be verified
+ * @errp: Error pointer
+ *
+ * The function first setup default values and then checks if the cpu
+ * fits inside the system topology.
+ */
+static void s390_topology_check(S390CPU *cpu, Error **errp)
+{
+ CpuTopology *smp = s390_topology.smp;
+ ERRP_GUARD();
+
+ s390_topology_cpu_default(cpu, errp);
+ if (*errp) {
+ return;
+ }
+
+ if (cpu->env.socket_id > smp->sockets) {
+ error_setg(errp, "Unavailable socket: %d", cpu->env.socket_id);
+ return;
+ }
+ if (cpu->env.book_id > smp->books) {
+ error_setg(errp, "Unavailable book: %d", cpu->env.book_id);
+ return;
+ }
+ if (cpu->env.drawer_id > smp->drawers) {
+ error_setg(errp, "Unavailable drawer: %d", cpu->env.drawer_id);
+ return;
+ }
+ if (cpu->env.entitlement >= POLARITY_MAX) {
+ error_setg(errp, "Unknown polarity: %d", cpu->env.entitlement);
+ return;
+ }
+
+ /* Dedication, boolean, can not be wrong. */
+}
+
+/**
+ * s390_set_core_in_socket:
+ * @cpu: the new S390CPU to insert in the topology structure
+ * @drawer_id: new drawer_id
+ * @book_id: new book_id
+ * @socket_id: new socket_id
+ * @creation: if is true the CPU is a new CPU and there is no old socket
+ * to handle.
+ * if is false, this is a moving the CPU and old socket count
+ * must be decremented.
+ * @errp: the error pointer
+ *
+ */
+static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
+ int socket_id, bool creation, Error **errp)
+{
+ int old_socket = s390_socket_nb(cpu);
+ int new_socket;
+
+ if (creation) {
+ new_socket = old_socket;
+ } else {
+ new_socket = drawer_id * s390_topology.smp->books +
+ book_id * s390_topology.smp->sockets +
+ socket_id;
+ }
+
+ /* Check for space on new socket */
+ if ((new_socket != old_socket) &&
+ (s390_topology.cores_per_socket[new_socket] >=
+ s390_topology.smp->cores)) {
+ error_setg(errp, "No more space on this socket");
+ return;
+ }
+
+ /* Update the count of cores in sockets */
+ s390_topology.cores_per_socket[new_socket] += 1;
+ if (!creation) {
+ s390_topology.cores_per_socket[old_socket] -= 1;
+ }
+}
+
+/**
+ * s390_update_cpu_props:
+ * @ms: the machine state
+ * @cpu: the CPU for which to update the properties from the environment.
+ *
+ */
+static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu)
+{
+ CpuInstanceProperties *props;
+
+ props = &ms->possible_cpus->cpus[cpu->env.core_id].props;
+
+ props->socket_id = cpu->env.socket_id;
+ props->book_id = cpu->env.book_id;
+ props->drawer_id = cpu->env.drawer_id;
+}
+
+/**
+ * s390_topology_set_cpu:
+ * @ms: MachineState used to initialize the topology structure on
+ * first call.
+ * @cpu: the new S390CPU to insert in the topology structure
+ * @errp: the error pointer
+ *
+ * Called from CPU Hotplug to check and setup the CPU attributes
+ * before to insert the CPU in the topology.
+ */
+void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
+{
+ ERRP_GUARD();
+
+ /*
+ * We do not want to initialize the topology if the cpu model
+ * does not support topology consequently, we have to wait for
+ * the first CPU to be realized, which realizes the CPU model
+ * to initialize the topology structures.
+ *
+ * s390_topology_set_cpu() is called from the cpu hotplug.
+ */
+ if (!s390_topology.cores_per_socket) {
+ s390_topology_init(ms);
+ }
+
+ s390_topology_check(cpu, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* Set the CPU inside the socket */
+ s390_set_core_in_socket(cpu, 0, 0, 0, true, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+}
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index f3cc845d3b..9bc51a83f4 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -44,6 +44,7 @@
#include "hw/s390x/pv.h"
#include "migration/blocker.h"
#include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
static Error *pv_mig_blocker;
@@ -310,10 +311,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
{
MachineState *ms = MACHINE(hotplug_dev);
S390CPU *cpu = S390_CPU(dev);
+ ERRP_GUARD();
g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+ if (s390_has_topology()) {
+ s390_topology_set_cpu(ms, cpu, errp);
+ if (*errp) {
+ return;
+ }
+ }
+
if (dev->hotplugged) {
raise_irq_cpu_hotplug();
}
@@ -551,11 +560,21 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
+ CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
+
ms->possible_cpus->cpus[i].type = ms->cpu_type;
ms->possible_cpus->cpus[i].vcpus_count = 1;
ms->possible_cpus->cpus[i].arch_id = i;
- ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = i;
+
+ props->has_core_id = true;
+ props->core_id = i;
+ props->has_socket_id = true;
+ props->socket_id = i / ms->smp.cores;
+ props->has_book_id = true;
+ props->book_id = i / (ms->smp.cores * ms->smp.sockets);
+ props->has_drawer_id = true;
+ props->drawer_id = i /
+ (ms->smp.cores * ms->smp.sockets * ms->smp.books);
}
return ms->possible_cpus;
diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
index f291016fee..58dfbdff4f 100644
--- a/hw/s390x/meson.build
+++ b/hw/s390x/meson.build
@@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
's390-stattrib-kvm.c',
'pv.c',
's390-pci-kvm.c',
+ 'cpu-topology.c',
))
s390x_ss.add(when: 'CONFIG_TCG', if_true: files(
'tod-tcg.c',
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
2023-02-01 13:20 ` [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-03 17:36 ` Nina Schoetterl-Glausch
` (2 more replies)
2023-02-01 13:20 ` [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
` (8 subsequent siblings)
11 siblings, 3 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
include/hw/s390x/cpu-topology.h | 22 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 72 +++++++
hw/s390x/cpu-topology.c | 10 +
target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/meson.build | 3 +-
7 files changed, 446 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/cpu_topology.c
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 9b6f889ad4..1ae7e7c5e3 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -24,9 +24,31 @@ enum s390_topology_polarity {
POLARITY_MAX,
};
+typedef union s390_topology_id {
+ uint64_t id;
+ struct {
+ uint8_t level5;
+ uint8_t drawer;
+ uint8_t book;
+ uint8_t socket;
+ uint8_t dedicated;
+ uint8_t polarity;
+ uint8_t type;
+ uint8_t origin;
+ };
+} s390_topology_id;
+
+typedef struct S390TopologyEntry {
+ QTAILQ_ENTRY(S390TopologyEntry) next;
+ s390_topology_id id;
+ uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+ QTAILQ_HEAD(, S390TopologyEntry) list;
CpuTopology *smp;
+ uint8_t polarity;
} S390Topology;
#ifdef CONFIG_KVM
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index d3ade40a5a..712fd68123 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST 2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index d654267a71..e1f6925856 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -560,6 +560,25 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+ uint8_t reserved0[2];
+ uint16_t length;
+ uint8_t mag[S390_TOPOLOGY_MAG];
+ uint8_t reserved1;
+ uint8_t mnest;
+ uint32_t reserved2;
+ char tle[];
+} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -567,9 +586,62 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+ SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container of with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+ uint8_t nl;
+ uint8_t reserved[6];
+ uint8_t id;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+ uint8_t nl;
+ uint8_t reserved0[3];
+#define SYSIB_TLE_POLARITY_MASK 0x03
+#define SYSIB_TLE_DEDICATED 0x04
+ uint8_t entitlement;
+ uint8_t type;
+ uint16_t origin;
+ uint64_t mask;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
#define ASCE_SUBSPACE 0x200 /* subspace group control */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 12df4eca6c..a80a1ebf22 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -31,6 +31,8 @@ S390Topology s390_topology = {
/* will be initialized after the cpu model is realized */
.cores_per_socket = NULL,
.smp = NULL,
+ .list = QTAILQ_HEAD_INITIALIZER(s390_topology.list),
+ .polarity = POLARITY_HORIZONTAL,
};
/**
@@ -65,14 +67,22 @@ bool s390_has_topology(void)
* Allocate an array to keep the count of cores per socket.
* The index of the array starts at socket 0 from book 0 and
* drawer 0 up to the maximum allowed by the machine topology.
+ *
+ * Insert a sentinel entry using unused level5 with its maximum value.
+ * This entry will never be free.
*/
static void s390_topology_init(MachineState *ms)
{
CpuTopology *smp = &ms->smp;
+ S390TopologyEntry *entry;
s390_topology.smp = smp;
s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets *
smp->books * smp->drawers);
+
+ entry = g_malloc0(sizeof(S390TopologyEntry));
+ entry->id.level5 = 0xff;
+ QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
/**
diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c
new file mode 100644
index 0000000000..aba141fb66
--- /dev/null
+++ b/target/s390x/kvm/cpu_topology.c
@@ -0,0 +1,335 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <pmorel@linux.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/sclp.h"
+#include "hw/s390x/cpu-topology.h"
+
+/**
+ * fill_container:
+ * @p: The address of the container TLE to fill
+ * @level: The level of nesting for this container
+ * @id: The container receives a uniq ID inside its own container
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_container(char *p, int level, int id)
+{
+ SysIBTl_container *tle = (SysIBTl_container *)p;
+
+ tle->nl = level;
+ tle->id = id;
+ return p + sizeof(*tle);
+}
+
+/**
+ * fill_tle_cpu:
+ * @p: The address of the CPU TLE to fill
+ * @entry: a pointer to the S390TopologyEntry defining this
+ * CPU container.
+ *
+ * Returns the next free TLE entry.
+ */
+static char *fill_tle_cpu(char *p, S390TopologyEntry *entry)
+{
+ SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+ s390_topology_id topology_id = entry->id;
+
+ tle->nl = 0;
+ if (topology_id.dedicated) {
+ tle->entitlement = SYSIB_TLE_DEDICATED;
+ }
+ tle->entitlement |= topology_id.polarity;
+ tle->type = topology_id.type;
+ tle->origin = cpu_to_be16(topology_id.origin * 64);
+ tle->mask = cpu_to_be64(entry->mask);
+ return p + sizeof(*tle);
+}
+
+/*
+ * Macro to check that the size of data after increment
+ * will not get bigger than the size of the SysIB.
+ */
+#define SYSIB_GUARD(data, x) do { \
+ data += x; \
+ if (data > sizeof(SysIB)) { \
+ return -ENOSPC; \
+ } \
+ } while (0)
+
+/**
+ * stsi_set_tle:
+ * @p: A pointer to the position of the first TLE
+ * @level: The nested level wanted by the guest
+ *
+ * Loop inside the s390_topology.list until the sentinelle entry
+ * is found and for each entry:
+ * - Check using SYSIB_GUARD() that the size of the SysIB is not
+ * reached.
+ * - Add all the container TLE needed for the level
+ * - Add the CPU TLE.
+ *
+ * Return value:
+ * s390_top_set_level returns the size of the SysIB_15x after being
+ * filled with TLE on success.
+ * It returns -ENOSPC in the case we would overrun the end of the SysIB.
+ */
+static int stsi_set_tle(char *p, int level)
+{
+ S390TopologyEntry *entry;
+ int last_drawer = -1;
+ int last_book = -1;
+ int last_socket = -1;
+ int drawer_id = 0;
+ int book_id = 0;
+ int socket_id = 0;
+ int n = sizeof(SysIB_151x);
+
+ QTAILQ_FOREACH(entry, &s390_topology.list, next) {
+ int current_drawer = entry->id.drawer;
+ int current_book = entry->id.book;
+ int current_socket = entry->id.socket;
+ bool drawer_change = last_drawer != current_drawer;
+ bool book_change = drawer_change || last_book != current_book;
+ bool socket_change = book_change || last_socket != current_socket;
+
+ /* If we reach the guard get out */
+ if (entry->id.level5) {
+ break;
+ }
+
+ if (level > 3 && drawer_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 3, drawer_id++);
+ book_id = 0;
+ }
+ if (level > 2 && book_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 2, book_id++);
+ socket_id = 0;
+ }
+ if (socket_change) {
+ SYSIB_GUARD(n, sizeof(SysIBTl_container));
+ p = fill_container(p, 1, socket_id++);
+ }
+
+ SYSIB_GUARD(n, sizeof(SysIBTl_cpu));
+ p = fill_tle_cpu(p, entry);
+ last_drawer = entry->id.drawer;
+ last_book = entry->id.book;
+ last_socket = entry->id.socket;
+ }
+
+ return n;
+}
+
+/**
+ * setup_stsi:
+ * sysib: pointer to a SysIB to be filled with SysIB_151x data
+ * level: Nested level specified by the guest
+ *
+ * Setup the SysIB_151x header before calling stsi_set_tle with
+ * a pointer to the first TLE entry.
+ */
+static int setup_stsi(SysIB_151x *sysib, int level)
+{
+ sysib->mnest = level;
+ switch (level) {
+ case 4:
+ sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 3:
+ sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
+ current_machine->smp.books;
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ case 2:
+ sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
+ current_machine->smp.books *
+ current_machine->smp.sockets;
+ sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
+ break;
+ }
+
+ return stsi_set_tle(sysib->tle, level);
+}
+
+/**
+ * s390_topology_add_cpu_to_entry:
+ * @entry: Topology entry to setup
+ * @cpu: the S390CPU to add
+ *
+ * Set the core bit inside the topology mask and
+ * increments the number of cores for the socket.
+ */
+static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
+ S390CPU *cpu)
+{
+ set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
+}
+
+/**
+ * s390_topology_new_entry:
+ * @id: s390_topology_id to add
+ * @cpu: the S390CPU to add
+ *
+ * Allocate a new entry and initialize it.
+ *
+ * returns the newly allocated entry.
+ */
+static S390TopologyEntry *s390_topology_new_entry(s390_topology_id id,
+ S390CPU *cpu)
+{
+ S390TopologyEntry *entry;
+
+ entry = g_malloc0(sizeof(S390TopologyEntry));
+ entry->id.id = id.id;
+ s390_topology_add_cpu_to_entry(entry, cpu);
+
+ return entry;
+}
+
+/**
+ * s390_topology_from_cpu:
+ * @cpu: The S390CPU
+ *
+ * Initialize the topology id from the CPU environment.
+ */
+static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
+{
+ s390_topology_id topology_id = {0};
+
+ topology_id.drawer = cpu->env.drawer_id;
+ topology_id.book = cpu->env.book_id;
+ topology_id.socket = cpu->env.socket_id;
+ topology_id.origin = cpu->env.core_id / 64;
+ topology_id.type = S390_TOPOLOGY_CPU_IFL;
+ topology_id.dedicated = cpu->env.dedicated;
+
+ if (s390_topology.polarity == POLARITY_VERTICAL) {
+ /*
+ * Vertical polarity with dedicated CPU implies
+ * vertical high entitlement.
+ */
+ if (topology_id.dedicated) {
+ topology_id.polarity |= POLARITY_VERTICAL_HIGH;
+ } else {
+ topology_id.polarity |= cpu->env.entitlement;
+ }
+ }
+
+ return topology_id;
+}
+
+/**
+ * s390_topology_insert:
+ * @cpu: s390CPU insert.
+ *
+ * Parse the topology list to find if the entry already
+ * exist and add the core in it.
+ * If it does not exist, allocate a new entry and insert
+ * it in the queue from lower id to greater id.
+ */
+static void s390_topology_insert(S390CPU *cpu)
+{
+ s390_topology_id id = s390_topology_from_cpu(cpu);
+ S390TopologyEntry *entry = NULL;
+ S390TopologyEntry *tmp = NULL;
+
+ QTAILQ_FOREACH(tmp, &s390_topology.list, next) {
+ if (id.id == tmp->id.id) {
+ s390_topology_add_cpu_to_entry(tmp, cpu);
+ return;
+ } else if (id.id < tmp->id.id) {
+ entry = s390_topology_new_entry(id, cpu);
+ QTAILQ_INSERT_BEFORE(tmp, entry, next);
+ return;
+ }
+ }
+}
+
+/**
+ * s390_order_tle:
+ *
+ * Loop over all CPU and insert it at the right place
+ * inside the TLE entry list.
+ */
+static void s390_order_tle(void)
+{
+ CPUState *cs;
+
+ CPU_FOREACH(cs) {
+ s390_topology_insert(S390_CPU(cs));
+ }
+}
+
+/**
+ * s390_free_tle:
+ *
+ * Loop over all TLE entries and free them.
+ * Keep the sentinelle which is the only one with level5 != 0
+ */
+static void s390_free_tle(void)
+{
+ S390TopologyEntry *entry = NULL;
+ S390TopologyEntry *tmp = NULL;
+
+ QTAILQ_FOREACH_SAFE(entry, &s390_topology.list, next, tmp) {
+ if (!entry->id.level5) {
+ QTAILQ_REMOVE(&s390_topology.list, entry, next);
+ g_free(entry);
+ }
+ }
+}
+
+/**
+ * insert_stsi_15_1_x:
+ * cpu: the CPU doing the call for which we set CC
+ * sel2: the selector 2, containing the nested level
+ * addr: Guest logical address of the guest SysIB
+ * ar: the access register number
+ *
+ * Reserve a zeroed SysIB, let setup_stsi to fill it and
+ * copy the SysIB to the guest memory.
+ *
+ * In case of overflow set CC(3) and no copy is done.
+ */
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+ SysIB sysib = {0};
+ int len;
+
+ if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ s390_order_tle();
+
+ len = setup_stsi(&sysib.sysib_151x, sel2);
+
+ if (len < 0) {
+ setcc(cpu, 3);
+ return;
+ }
+
+ sysib.sysib_151x.length = cpu_to_be16(len);
+ s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, len);
+ setcc(cpu, 0);
+
+ s390_free_tle();
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 3ac7ec9acf..5ea358cbb0 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu)
if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
return 0;
}
- /* Only sysib 3.2.2 needs post-handling for now. */
insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
return 0;
+ case 15:
+ insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+ run->s390_stsi.ar);
+ return 0;
default:
return 0;
}
diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build
index aef52b6686..5daa5c6033 100644
--- a/target/s390x/kvm/meson.build
+++ b/target/s390x/kvm/meson.build
@@ -1,6 +1,7 @@
s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
- 'kvm.c'
+ 'kvm.c',
+ 'cpu_topology.c'
), if_false: files(
'stubs.c'
))
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (2 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 10:13 ` Thomas Huth
2023-02-01 13:20 ` [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
` (7 subsequent siblings)
11 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 712fd68123..902252b319 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST 2
+#define SCLP_READ_SCP_INFO_MNEST 4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
- uint8_t _reserved1[16 - 11]; /* 11-15 */
+ uint8_t _reserved1[15 - 11]; /* 11-14 */
+ uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu; /* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+ if (s390_has_topology()) {
+ read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+ }
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, &cpu_count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (3 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 11:05 ` Thomas Huth
2023-02-06 17:52 ` Nina Schoetterl-Glausch
2023-02-01 13:20 ` [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Pierre Morel
` (6 subsequent siblings)
11 siblings, 2 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h | 1 +
hw/s390x/cpu-topology.c | 12 ++++++++++++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +++++++++++++
target/s390x/kvm/kvm.c | 17 +++++++++++++++++
7 files changed, 48 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 1ae7e7c5e3..60e0b9fbfa 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
extern S390Topology s390_topology;
int s390_socket_nb(S390CPU *cpu);
+void s390_topology_reset(void);
#endif
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index e1f6925856..848314d2a9 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+void s390_cpu_topology_reset(void);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index a80a1ebf22..cf63f3dd01 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * s390_topology_reset() to reset the kernel Modified Topology
+ * change record.
+ */
+void s390_topology_reset(void)
+{
+ s390_cpu_topology_reset();
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 9bc51a83f4..30fdfe41fa 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -122,6 +122,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+ if (s390_has_topology()) {
+ s390_topology_reset();
+ }
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 948e4bd3e0..e27864c5f5 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_reset(void)
+{
+ int ret;
+
+ if (kvm_enabled()) {
+ ret = kvm_s390_topology_set_mtcr(0);
+ if (ret) {
+ error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+ }
+ }
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 5ea358cbb0..bc953151ce 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+ struct kvm_device_attr attribute = {
+ .group = KVM_S390_VM_CPU_TOPOLOGY,
+ .attr = attr,
+ };
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ return 0;
+ }
+ if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+ return -ENOTSUP;
+ }
+
+ return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute);
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (4 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 11:38 ` Thomas Huth
2023-02-06 18:34 ` Nina Schoetterl-Glausch
2023-02-01 13:20 ` [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology Pierre Morel
` (5 subsequent siblings)
11 siblings, 2 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervizor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervizor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
include/hw/s390x/s390-virtio-ccw.h | 6 ++
target/s390x/cpu.h | 1 +
hw/s390x/cpu-topology.c | 103 +++++++++++++++++++++++++++++
target/s390x/cpu-sysemu.c | 14 ++++
target/s390x/kvm/kvm.c | 11 +++
5 files changed, 135 insertions(+)
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..c1d46e78af 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 848314d2a9..f6e207afde 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -857,6 +857,7 @@ void s390_enable_css_support(S390CPU *cpu);
void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg);
int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id,
int vq, bool assign);
+void s390_cpu_topology_set_modified(void);
#ifndef CONFIG_USER_ONLY
unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu);
#else
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index cf63f3dd01..1028bf4476 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -85,16 +85,104 @@ static void s390_topology_init(MachineState *ms)
QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
}
+/**
+ * s390_topology_set_cpus_polarity:
+ * @polarity: polarity requested by the caller
+ *
+ * Set all CPU entitlement according to polarity and
+ * dedication.
+ * Default vertical entitlement is POLARITY_VERTICAL_MEDIUM as
+ * it does not require host modification of the CPU provisioning
+ * until the host decide to modify individual CPU provisioning
+ * using QAPI interface.
+ * However a dedicated vCPU will have a POLARITY_VERTICAL_HIGH
+ * entitlement.
+ */
+static void s390_topology_set_cpus_polarity(int polarity)
+{
+ CPUState *cs;
+
+ CPU_FOREACH(cs) {
+ if (polarity == POLARITY_HORIZONTAL) {
+ S390_CPU(cs)->env.entitlement = 0;
+ } else if (S390_CPU(cs)->env.dedicated) {
+ S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_HIGH;
+ } else {
+ S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_MEDIUM;
+ }
+ }
+}
+
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Verify that the polarization really need to change and call
+ * s390_topology_set_cpus_polarity() specifying the requested polarity
+ * to set for all CPUs.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+ CPUS390XState *env = &cpu->env;
+ uint64_t reg = env->regs[r1];
+ int fc = reg & S390_TOPO_FC_MASK;
+
+ if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+ s390_program_interrupt(env, PGM_OPERATION, ra);
+ return;
+ }
+
+ if (env->psw.mask & PSW_MASK_PSTATE) {
+ s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+ return;
+ }
+
+ if (reg & ~S390_TOPO_FC_MASK) {
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ return;
+ }
+
+ switch (fc) {
+ case POLARITY_VERTICAL:
+ case POLARITY_HORIZONTAL:
+ if (s390_topology.polarity == fc) {
+ env->regs[r1] |= S390_PTF_REASON_DONE;
+ setcc(cpu, 2);
+ } else {
+ s390_topology.polarity = fc;
+ s390_cpu_topology_set_modified();
+ s390_topology_set_cpus_polarity(fc);
+ setcc(cpu, 0);
+ }
+ break;
+ default:
+ /* Note that fc == 2 is interpreted by the SIE */
+ s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+ }
+}
+
/**
* s390_topology_reset:
*
* Generic reset for CPU topology, calls s390_topology_reset()
* s390_topology_reset() to reset the kernel Modified Topology
* change record.
+ * Then set global and all CPUs polarity to POLARITY_HORIZONTAL.
*/
void s390_topology_reset(void)
{
s390_cpu_topology_reset();
+ /* Set global polarity to POLARITY_HORIZONTAL */
+ s390_topology.polarity = POLARITY_HORIZONTAL;
+ /* Set all CPU polarity to POLARITY_HORIZONTAL */
+ s390_topology_set_cpus_polarity(POLARITY_HORIZONTAL);
}
/**
@@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
(smp->books * smp->sockets * smp->cores)) %
smp->drawers;
}
+
+ /*
+ * Machine polarity is set inside the global s390_topology structure.
+ * In the case the polarity is set as horizontal set the entitlement
+ * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when
+ * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if
+ * the vCPU is dedicated.
+ */
+ if (s390_topology.polarity && !env->entitlement) {
+ if (env->dedicated) {
+ env->entitlement = POLARITY_VERTICAL_HIGH;
+ } else {
+ env->entitlement = POLARITY_VERTICAL_MEDIUM;
+ }
+ }
}
/**
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index e27864c5f5..82e3f3891e 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -37,6 +37,7 @@
#include "sysemu/sysemu.h"
#include "sysemu/tcg.h"
#include "hw/core/sysemu-cpu-ops.h"
+#include "hw/s390x/cpu-topology.h"
/* S390CPUClass::load_normal() */
static void s390_cpu_load_normal(CPUState *s)
@@ -319,3 +320,16 @@ void s390_cpu_topology_reset(void)
}
}
}
+
+void s390_cpu_topology_set_modified(void)
+{
+ int ret;
+
+ if (kvm_enabled()) {
+ ret = kvm_s390_topology_set_mtcr(1);
+ if (ret) {
+ error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+ }
+ }
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index bc953151ce..fb63be41b7 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -96,6 +96,7 @@
#define PRIV_B9_EQBS 0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+ uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+ s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+ case PRIV_B9_PTF:
+ kvm_handle_ptf(cpu, run);
+ break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (5 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 11:57 ` Thomas Huth
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
` (4 subsequent siblings)
11 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 12 ++++++++++++
3 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 1028bf4476..c33378577b 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -55,7 +55,7 @@ int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
- return false;
+ return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 065ec6d66c..aca2c5c96b 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -254,6 +254,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+ case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index fb63be41b7..808e35a7bd 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2470,6 +2470,18 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+ /*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+ if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+ if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) {
+ error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY");
+ return;
+ }
+ set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+ }
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (6 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 12:21 ` Thomas Huth
` (2 more replies)
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
` (3 subsequent siblings)
11 siblings, 3 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimise
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command is made experimental for the moment.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine-target.json | 29 +++++++++++++
include/monitor/hmp.h | 1 +
hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
hmp-commands.hx | 16 ++++++++
4 files changed, 134 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 2e267fa458..58df0f5061 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -342,3 +342,32 @@
'TARGET_S390X',
'TARGET_MIPS',
'TARGET_LOONGARCH64' ] } }
+
+##
+# @x-set-cpu-topology:
+#
+# @core: the vCPU ID to be moved
+# @socket: the destination socket where to move the vCPU
+# @book: the destination book where to move the vCPU
+# @drawer: the destination drawer where to move the vCPU
+# @polarity: optional polarity, default is last polarity set by the guest
+# @dedicated: optional, if the vCPU is dedicated to a real CPU
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: <next qemu stable release, eg. 1.0>
+##
+{ 'command': 'x-set-cpu-topology',
+ 'data': {
+ 'core': 'int',
+ 'socket': 'int',
+ 'book': 'int',
+ 'drawer': 'int',
+ '*polarity': 'int',
+ '*dedicated': 'bool'
+ },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
index 1b3bdcb446..12827479cf 100644
--- a/include/monitor/hmp.h
+++ b/include/monitor/hmp.h
@@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
HumanReadableText *(*qmp_handler)(Error **));
void hmp_info_stats(Monitor *mon, const QDict *qdict);
void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
+void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index c33378577b..6c50050991 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -18,6 +18,10 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qmp/qdict.h"
+#include "monitor/hmp.h"
+#include "monitor/monitor.h"
/*
* s390_topology is used to keep the topology information.
@@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+/*
+ * qmp and hmp implementations
+ */
+
+static void s390_change_topology(int64_t core_id, int64_t socket_id,
+ int64_t book_id, int64_t drawer_id,
+ int64_t polarity, bool dedicated,
+ Error **errp)
+{
+ MachineState *ms = current_machine;
+ S390CPU *cpu;
+ ERRP_GUARD();
+
+ cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
+ if (!cpu) {
+ error_setg(errp, "Core-id %ld does not exist!", core_id);
+ return;
+ }
+
+ /* Verify the new topology */
+ s390_topology_check(cpu, errp);
+ if (*errp) {
+ return;
+ }
+
+ /* Move the CPU into its new socket */
+ s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true, errp);
+
+ /* All checks done, report topology in environment */
+ cpu->env.drawer_id = drawer_id;
+ cpu->env.book_id = book_id;
+ cpu->env.socket_id = socket_id;
+ cpu->env.dedicated = dedicated;
+ cpu->env.entitlement = polarity;
+
+ /* topology tree is reflected in props */
+ s390_update_cpu_props(ms, cpu);
+
+ /* Advertise the topology change */
+ s390_cpu_topology_set_modified();
+}
+
+void qmp_x_set_cpu_topology(int64_t core, int64_t socket,
+ int64_t book, int64_t drawer,
+ bool has_polarity, int64_t polarity,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+ ERRP_GUARD();
+
+ if (!s390_has_topology()) {
+ error_setg(errp, "This machine doesn't support topology");
+ return;
+ }
+ if (!has_polarity) {
+ polarity = POLARITY_VERTICAL_MEDIUM;
+ }
+ if (!has_dedicated) {
+ dedicated = false;
+ }
+ s390_change_topology(core, socket, book, drawer, polarity, dedicated, errp);
+}
+
+void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict)
+{
+ const int64_t core = qdict_get_int(qdict, "core");
+ const int64_t socket = qdict_get_int(qdict, "socket");
+ const int64_t book = qdict_get_int(qdict, "book");
+ const int64_t drawer = qdict_get_int(qdict, "drawer");
+ bool has_polarity = qdict_haskey(qdict, "polarity");
+ const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0);
+ bool has_dedicated = qdict_haskey(qdict, "dedicated");
+ const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false);
+ Error *local_err = NULL;
+
+ qmp_x_set_cpu_topology(core, socket, book, drawer,
+ has_polarity, polarity,
+ has_dedicated, dedicated,
+ &local_err);
+ if (hmp_handle_error(mon, local_err)) {
+ return;
+ }
+}
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 673e39a697..bb3c908356 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1815,3 +1815,19 @@ SRST
Dump the FDT in dtb format to *filename*.
ERST
#endif
+
+#if defined(TARGET_S390X) && defined(CONFIG_KVM)
+ {
+ .name = "x-set-cpu-topology",
+ .args_type = "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?",
+ .params = "core socket book drawer [polarity] [dedicated]",
+ .help = "Move CPU 'core' to 'socket/book/drawer' "
+ "optionaly modifies polarity and dedication",
+ .cmd = hmp_x_set_cpu_topology,
+ },
+
+SRST
+``x-set-cpu-topology`` *core* *socket* *book* *drawer* *polarity* *dedicated*
+ Moves the CPU *core* to *socket* *book* *drawer* with *polarity* *dedicated*.
+ERST
+#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (7 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-06 12:38 ` Thomas Huth
` (2 more replies)
2023-02-01 13:20 ` [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event Pierre Morel
` (2 subsequent siblings)
11 siblings, 3 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
S390x provides two more topology containers above the sockets,
books and drawers.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine.json | 13 ++++++++++---
hw/core/machine-qmp-cmds.c | 2 ++
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 3036117059..e36c39e258 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -53,11 +53,18 @@
#
# Additional information about a virtual S390 CPU
#
-# @cpu-state: the virtual CPU's state
+# @cpu-state: the virtual CPU's state (since 2.12)
+# @dedicated: the virtual CPU's dedication (since 8.0)
+# @polarity: the virtual CPU's polarity (since 8.0)
#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+ 'dedicated': 'bool',
+ 'polarity': 'int'
+ }
+}
##
# @CpuInfoFast:
@@ -70,7 +77,7 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
+# @props: properties describing to which node/drawer/book/socket/core/thread
# virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 80d5e59651..e6d93cf2a0 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -30,6 +30,8 @@ static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu)
CPUS390XState *env = &s390_cpu->env;
info->cpu_state = env->cpu_state;
+ info->dedicated = env->dedicated;
+ info->polarity = env->entitlement;
#else
abort();
#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (8 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-08 17:35 ` Nina Schoetterl-Glausch
2023-02-01 13:20 ` [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
2023-02-09 17:14 ` [PATCH v15 00/11] s390x: CPU Topology Nina Schoetterl-Glausch
11 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
When the guest asks to change the polarity this change
is forwarded to the admin using QAPI.
The admin is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 32 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 58df0f5061..5883c3b020 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -371,3 +371,33 @@
},
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARITY_CHANGE:
+#
+# Emitted when the guest asks to change the polarity.
+#
+# @polarity: polarity specified by the guest
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarity.
+#
+# On horizontal polarity the host is expected to provision all vCPUs
+# equally.
+# On vertical polarity the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# Since: 8.0
+#
+# Example:
+#
+# <- { "event": "CPU_POLARITY_CHANGE",
+# "data": { "polarity": 0 },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+#
+##
+{ 'event': 'CPU_POLARITY_CHANGE',
+ 'data': { 'polarity': 'int' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 6c50050991..2f8e1b60cf 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -19,6 +19,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
#include "qapi/qmp/qdict.h"
#include "monitor/hmp.h"
#include "monitor/monitor.h"
@@ -163,6 +164,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
s390_topology.polarity = fc;
s390_cpu_topology_set_modified();
s390_topology_set_cpus_polarity(fc);
+ qapi_event_send_cpu_polarity_change(fc);
setcc(cpu, 0);
}
break;
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (9 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event Pierre Morel
@ 2023-02-01 13:20 ` Pierre Morel
2023-02-08 16:22 ` Nina Schoetterl-Glausch
2023-02-09 17:14 ` [PATCH v15 00/11] s390x: CPU Topology Nina Schoetterl-Glausch
11 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-01 13:20 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
docs/system/s390x/cpu-topology.rst | 294 +++++++++++++++++++++++++++++
docs/system/target-s390x.rst | 1 +
2 files changed, 295 insertions(+)
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
new file mode 100644
index 0000000000..e2190318c0
--- /dev/null
+++ b/docs/system/s390x/cpu-topology.rst
@@ -0,0 +1,294 @@
+CPU topology on s390x
+=====================
+
+Since QEMU 8.0, CPU topology on s390x provides up to 4 levels of
+topology containers: drawers, books, sockets and cores.
+
+The first three containers define a tree hierarchy, the last one
+provides the placement of the CPUs inside the parent container and
+3 CPU attributes:
+
+- CPU type
+- polarity entitlement
+- dedication
+
+Note also that since 7.2 threads are no longer supported in the topology
+and the ``-smp`` command line argument accepts only ``threads=1``.
+
+Prerequisites
+-------------
+
+To use CPU topology a Linux QEMU/KVM machine providing the CPU topology facility
+(STFLE bit 11) is required.
+
+However, since this facility has been enabled by default in an early version
+of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM
+that QEMU is supporting CPU topology.
+
+Enabling CPU topology
+---------------------
+
+Currently, CPU topology is only enabled in the host model by default.
+
+Enabling CPU topology in a CPU model is done by setting the CPU flag
+``ctop`` to ``on`` like in:
+
+.. code-block:: bash
+
+ -cpu gen16b,ctop=on
+
+Having the topology disabled by default allows migration between
+old and new QEMU without adding new flags.
+
+Default topology usage
+----------------------
+
+The CPU topology, can be specified on the QEMU command line
+with the ``-smp`` or the ``-device`` QEMU command arguments
+without using any new attributes.
+In this case, the topology will be calculated by simply adding
+to the topology the cores based on the core-id starting with
+core-0 at position 0 of socket-0, book-0, drawer-0 with default
+modifier attributes: horizontal polarity and no dedication.
+
+In the following machine we define 8 sockets with 4 cores each.
+Note that s390x QEMU machines do not implement multithreading.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
+ -device host-s390x-cpu,core-id=14 \
+
+New CPUs can be plugged using the device_add hmp command like in:
+
+.. code-block:: bash
+
+ (qemu) device_add gen16b-s390x-cpu,core-id=9
+
+The core-id defines the placement of the core in the topology by
+starting with core 0 in socket 0 up to maxcpus.
+
+In the example above:
+
+* There are 5 CPUs provided to the guest with the ``-smp`` command line
+ They will take the core-ids 0,1,2,3,4
+ As we have 4 cores in a socket, we have 4 CPUs provided
+ to the guest in socket 0, with core-ids 0,1,2,3.
+ The last cpu, with core-id 4, will be on socket 1.
+
+* the core with ID 14 provided by the ``-device`` command line will
+ be placed in socket 3, with core-id 14
+
+* the core with ID 9 provided by the ``device_add`` qmp command will
+ be placed in socket 2, with core-id 9
+
+Polarity and dedication
+-----------------------
+
+Polarity can be of two types: horizontal or vertical.
+
+The horizontal polarization specifies that all guest's vCPUs get
+almost the same amount of provisioning of real CPU by the host.
+
+The vertical polarization specifies that guest's vCPU can get
+different real CPU provisions:
+
+- a vCPU with Vertical high entitlement specifies that this
+ vCPU gets 100% of the real CPU provisioning.
+
+- a vCPU with Vertical medium entitlement specifies that this
+ vCPU shares the real CPU with other vCPUs.
+
+- a vCPU with Vertical low entitlement specifies that this
+ vCPU only get real CPU provisioning when no other vCPU need it.
+
+In the case a vCPU with vertical high entitlement does not use
+the real CPU, the unused "slack" can be dispatched to other vCPU
+with medium or low entitlement.
+
+A subsystem reset puts all vCPU of the configuration into the
+horizontal polarization.
+
+The admin specifies the dedicated bit when the vCPU is dedicated
+to a single real CPU.
+
+As for the Linux admin, the dedicated bit is an indication on the
+affinity of a vCPU for a real CPU while the entitlement indicates the
+sharing or exclusivity of use.
+
+Defining the topology on command line
+-------------------------------------
+
+The topology can be defined entirely during the CPU definition,
+with the exception of CPU 0 which must be defined with the -smp
+argument.
+
+For example, here we set the position of the cores 1,2,3 on
+drawer 1, book 1, socket 2 and cores 0,9 and 14 on drawer 0,
+book 0, socket 0 with all horizontal polarity and not dedicated.
+The core 4, will be set on its default position on socket 1
+(since we have 4 core per socket) and we define it with dedication and
+vertical high entitlement.
+
+.. code-block:: bash
+
+ $ qemu-system-s390x -m 2G \
+ -cpu gen16b,ctop=on \
+ -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
+ -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
+ \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
+ -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
+ \
+ -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarity=3 \
+
+QAPI interface for topology
+---------------------------
+
+Let's start QEMU with the following command:
+
+.. code-block:: bash
+
+ sudo /usr/local/bin/qemu-system-s390x \
+ -enable-kvm \
+ -cpu z14,ctop=on \
+ -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+ \
+ -device z14-s390x-cpu,core-id=19,polarity=3 \
+ -device z14-s390x-cpu,core-id=11,polarity=1 \
+ -device z14-s390x-cpu,core-id=112,polarity=3 \
+ ...
+
+and see the result when using of the QAPI interface.
+
+addons to query-cpus-fast
++++++++++++++++++++++++++
+
+The command query-cpus-fast allows the admin to query the topology
+tree and modifiers for all configured vCPU.
+
+.. code-block:: QMP
+
+ -> { "execute": "query-cpus-fast" }
+ {
+ "return": [
+ {
+ "dedicated": false,
+ "thread-id": 3631238,
+ "props": {
+ "core-id": 0,
+ "socket-id": 0,
+ "drawer-id": 0,
+ "book-id": 0
+ },
+ "cpu-state": "operating",
+ "qom-path": "/machine/unattached/device[0]",
+ "polarity": 2,
+ "cpu-index": 0,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 3631248,
+ "props": {
+ "core-id": 19,
+ "socket-id": 9,
+ "drawer-id": 0,
+ "book-id": 2
+ },
+ "cpu-state": "operating",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "polarity": 3,
+ "cpu-index": 19,
+ "target": "s390x"
+ },
+ {
+ "dedicated": false,
+ "thread-id": 3631249,
+ "props": {
+ "core-id": 11,
+ "socket-id": 5,
+ "drawer-id": 0,
+ "book-id": 1
+ },
+ "cpu-state": "operating",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "polarity": 1,
+ "cpu-index": 11,
+ "target": "s390x"
+ },
+ {
+ "dedicated": true,
+ "thread-id": 3631250,
+ "props": {
+ "core-id": 112,
+ "socket-id": 56,
+ "drawer-id": 3,
+ "book-id": 14
+ },
+ "cpu-state": "operating",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "polarity": 3,
+ "cpu-index": 112,
+ "target": "s390x"
+ }
+ ]
+ }
+
+x-set-cpu-topology
+++++++++++++++++++
+
+The command x-set-cpu-topology allows the admin to modify the topology
+tree or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+ -> { "execute": "x-set-cpu-topology",
+ "arguments": {
+ "core": 11,
+ "socket": 0,
+ "book": 0,
+ "drawer": 0,
+ "polarity": 0,
+ "dedicated": false
+ }
+ }
+ <- {"return": {}}
+
+
+event CPU_POLARITY_CHANGE
++++++++++++++++++++++++++
+
+When a guest is requesting a modification of the polarity,
+QEMU sends a CPU_POLARITY_CHANGE event.
+
+When requesting the change, the guest only specifies horizontal or
+vertical polarity.
+The dedication and fine grain vertical entitlement depends on admin
+to set according to its response to this event.
+
+Note that a vertical polarized dedicated vCPU can only have a high
+entitlement, this gives 6 possibilities for a vCPU polarity:
+
+- Horizontal
+- Horizontal dedicated
+- Vertical low
+- Vertical medium
+- Vertical high
+- Vertical high dedicated
+
+Example of the event received when the guest issues the CPU instruction
+Perform Topology Function PTF(0) to request an horizontal polarity:
+
+.. code-block:: QMP
+
+ <- { "event": "CPU_POLARITY_CHANGE",
+ "data": { "polarity": 0 },
+ "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+
+
diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
index c636f64113..ff0ffe04f3 100644
--- a/docs/system/target-s390x.rst
+++ b/docs/system/target-s390x.rst
@@ -33,3 +33,4 @@ Architectural features
.. toctree::
s390x/bootdevices
s390x/protvirt
+ s390x/cpu-topology
--
2.31.1
^ permalink raw reply related [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
@ 2023-02-02 10:44 ` Thomas Huth
2023-02-02 13:15 ` Pierre Morel
2023-02-02 16:05 ` Nina Schoetterl-Glausch
2023-02-08 17:50 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-02 10:44 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> The S390 CPU have specific toplogy features like dedication
Nit: s/toplogy/topology/
> and polarity to give to the guest indications on the host
> vCPUs scheduling and help the guest take the best decisions
> on the scheduling of threads on the vCPUs.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and polarity,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
Apart from the nit:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-02 10:44 ` Thomas Huth
@ 2023-02-02 13:15 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-02 13:15 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/2/23 11:44, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> S390 adds two new SMP levels, drawers and books to the CPU
>> topology.
>> The S390 CPU have specific toplogy features like dedication
>
> Nit: s/toplogy/topology/
>
>> and polarity to give to the guest indications on the host
>> vCPUs scheduling and help the guest take the best decisions
>> on the scheduling of threads on the vCPUs.
>>
>> Let us provide the SMP properties with books and drawers levels
>> and S390 CPU with dedication and polarity,
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>
> Apart from the nit:
> Reviewed-by: Thomas Huth <thuth@redhat.com>
>
Thanks,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
2023-02-02 10:44 ` Thomas Huth
@ 2023-02-02 16:05 ` Nina Schoetterl-Glausch
2023-02-03 9:39 ` Pierre Morel
2023-02-08 17:50 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-02 16:05 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
Nit patch title: s390x/cpu topology: add s390 specifics to CPU topology ?
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> The S390 CPU have specific toplogy features like dedication
^o
> and polarity to give to the guest indications on the host
> vCPUs scheduling and help the guest take the best decisions
> on the scheduling of threads on the vCPUs.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and polarity,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine.json | 14 ++++++++--
> include/hw/boards.h | 10 ++++++-
> include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
> target/s390x/cpu.h | 5 ++++
> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
> hw/core/machine.c | 4 +++
> hw/s390x/s390-virtio-ccw.c | 2 ++
> softmmu/vl.c | 6 +++++
> target/s390x/cpu.c | 7 +++++
> qemu-options.hx | 7 +++--
> 10 files changed, 115 insertions(+), 12 deletions(-)
> create mode 100644 include/hw/s390x/cpu-topology.h
>
[...]
>
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> new file mode 100644
> index 0000000000..7a84b30a21
> --- /dev/null
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -0,0 +1,24 @@
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
> +#ifndef HW_S390X_CPU_TOPOLOGY_H
> +#define HW_S390X_CPU_TOPOLOGY_H
> +
> +#define S390_TOPOLOGY_CPU_IFL 0x03
> +
> +enum s390_topology_polarity {
> + POLARITY_HORIZONTAL,
> + POLARITY_VERTICAL,
> + POLARITY_VERTICAL_LOW = 1,
> + POLARITY_VERTICAL_MEDIUM,
> + POLARITY_VERTICAL_HIGH,
> + POLARITY_MAX,
> +};
Probably a good idea to keep the S390 prefix.
This works, but aliasing VERTICAL and VERTICAL_LOW is not
entirely straight forward.
Why not have two enum?
enum s390_topology_polarity {
S390_POLARITY_HORIZONTAL,
S390_POLARITY_VERTICAL,
};
enum s390_topology_entitlement {
S390_ENTITLEMENT_LOW = 1,
S390_ENTITLEMENT_MEDIUM,
S390_ENTITLEMENT_HIGH,
S390_ENTITLEMENT_MAX,
};
Maybe add an ENTITLEMENT_INVALID/NONE, if you need that, as first value.
> +#endif
>
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-01 13:20 ` [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
@ 2023-02-02 16:42 ` Nina Schoetterl-Glausch
2023-02-03 9:21 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-02 16:42 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> The topology information are attributes of the CPU and are
> specified during the CPU device creation.
>
> On hot plug we:
> - calculate the default values for the topology for drawers,
> books and sockets in the case they are not specified.
> - verify the CPU attributes
> - check that we have still room on the desired socket
>
> The possibility to insert a CPU in a mask is dependent on the
> number of cores allowed in a socket, a book or a drawer, the
> checking is done during the hot plug of the CPU to have an
> immediate answer.
>
> If the complete topology is not specified, the core is added
> in the physical topology based on its core ID and it gets
> defaults values for the modifier attributes.
>
> This way, starting QEMU without specifying the topology can
> still get some advantage of the CPU topology.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> include/hw/s390x/cpu-topology.h | 24 +++
> hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
> hw/s390x/s390-virtio-ccw.c | 23 ++-
> hw/s390x/meson.build | 1 +
> 4 files changed, 302 insertions(+), 2 deletions(-)
> create mode 100644 hw/s390x/cpu-topology.c
>
[...]
>
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> new file mode 100644
> index 0000000000..12df4eca6c
> --- /dev/null
> +++ b/hw/s390x/cpu-topology.c
> @@ -0,0 +1,256 @@
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022
> + * Author(s): Pierre Morel <pmorel@linux.ibm.com>
> +
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qemu/error-report.h"
> +#include "hw/qdev-properties.h"
> +#include "hw/boards.h"
> +#include "qemu/typedefs.h"
> +#include "target/s390x/cpu.h"
> +#include "hw/s390x/s390-virtio-ccw.h"
> +#include "hw/s390x/cpu-topology.h"
> +
> +/*
> + * s390_topology is used to keep the topology information.
> + * .list: queue the topology entries inside which
> + * we keep the information on the CPU topology.
.list doesn't exist yet.
> + * .socket: tracks information on the count of cores per socket.
> + * .smp: keeps track of the machine topology.
> + *
> + */
> +S390Topology s390_topology = {
> + /* will be initialized after the cpu model is realized */
> + .cores_per_socket = NULL,
> + .smp = NULL,
> +};
>
[...]
> +
> +/**
> + * s390_topology_cpu_default:
> + * @cpu: pointer to a S390CPU
> + * @errp: Error pointer
> + *
> + * Setup the default topology for unset attributes.
My suggestion:
Setup the default topology if no attributes are already set.
Passing a CPU with some, but not all, attributes set is considered an error.
> + *
> + * The function accept only all all default values or all set values
> + * for the geometry topology.
acceptS, all all
If you take my suggestion, you can just drop this sentence.
> + *
> + * The function calculates the (drawer_id, book_id, socket_id)
> + * topology by filling the cores starting from the first socket
> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
> + *
> + * CPU type, polarity and dedication have defaults values set in the
> + * s390x_cpu_properties.
> + */
> +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> +{
> + CpuTopology *smp = s390_topology.smp;
> + CPUS390XState *env = &cpu->env;
> +
> + /* All geometry topology attributes must be set or all unset */
> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
> + error_setg(errp,
> + "Please define all or none of the topology geometry attributes");
> + return;
> + }
> +
> + /* Check if one of the geometry topology is unset */
> + if (env->socket_id < 0) {
> + /* Calculate default geometry topology attributes */
> + env->socket_id = (env->core_id / smp->cores) % smp->sockets;
> + env->book_id = (env->core_id / (smp->sockets * smp->cores)) %
> + smp->books;
> + env->drawer_id = (env->core_id /
> + (smp->books * smp->sockets * smp->cores)) %
> + smp->drawers;
> + }
> +}
>
[...]
> +
> +/**
> + * s390_set_core_in_socket:
> + * @cpu: the new S390CPU to insert in the topology structure
> + * @drawer_id: new drawer_id
> + * @book_id: new book_id
> + * @socket_id: new socket_id
> + * @creation: if is true the CPU is a new CPU and there is no old socket
> + * to handle.
> + * if is false, this is a moving the CPU and old socket count
> + * must be decremented.
> + * @errp: the error pointer
> + *
> + */
> +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
Maybe name it s390_(topology_)?add_core_to_socket instead.
> + int socket_id, bool creation, Error **errp)
> +{
> + int old_socket = s390_socket_nb(cpu);
> + int new_socket;
> +
> + if (creation) {
> + new_socket = old_socket;
> + } else {
You need parentheses here.
> + new_socket = drawer_id * s390_topology.smp->books +
(
> + book_id * s390_topology.smp->sockets +
)
> + socket_id;
> + }
> +
> + /* Check for space on new socket */
> + if ((new_socket != old_socket) &&
> + (s390_topology.cores_per_socket[new_socket] >=
> + s390_topology.smp->cores)) {
> + error_setg(errp, "No more space on this socket");
> + return;
> + }
> +
> + /* Update the count of cores in sockets */
> + s390_topology.cores_per_socket[new_socket] += 1;
> + if (!creation) {
> + s390_topology.cores_per_socket[old_socket] -= 1;
> + }
> +}
>
[...]
> +/**
> + * s390_topology_set_cpu:
> + * @ms: MachineState used to initialize the topology structure on
> + * first call.
> + * @cpu: the new S390CPU to insert in the topology structure
> + * @errp: the error pointer
> + *
> + * Called from CPU Hotplug to check and setup the CPU attributes
> + * before to insert the CPU in the topology.
> + */
> +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
The name is rather non informative.
s390_topology_setup_cpu ?
> +{
> + ERRP_GUARD();
> +
> + /*
> + * We do not want to initialize the topology if the cpu model
> + * does not support topology consequently, we have to wait for
^
Still think there should be a comma here.
> + * the first CPU to be realized, which realizes the CPU model
> + * to initialize the topology structures.
> + *
> + * s390_topology_set_cpu() is called from the cpu hotplug.
> + */
> + if (!s390_topology.cores_per_socket) {
> + s390_topology_init(ms);
> + }
> +
> + s390_topology_check(cpu, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* Set the CPU inside the socket */
> + s390_set_core_in_socket(cpu, 0, 0, 0, true, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +}
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index f3cc845d3b..9bc51a83f4 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -44,6 +44,7 @@
> #include "hw/s390x/pv.h"
> #include "migration/blocker.h"
> #include "qapi/visitor.h"
> +#include "hw/s390x/cpu-topology.h"
>
> static Error *pv_mig_blocker;
>
> @@ -310,10 +311,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
> {
> MachineState *ms = MACHINE(hotplug_dev);
> S390CPU *cpu = S390_CPU(dev);
> + ERRP_GUARD();
>
> g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
> ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
>
> + if (s390_has_topology()) {
> + s390_topology_set_cpu(ms, cpu, errp);
> + if (*errp) {
> + return;
> + }
> + }
> +
> if (dev->hotplugged) {
> raise_irq_cpu_hotplug();
> }
> @@ -551,11 +560,21 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
> sizeof(CPUArchId) * max_cpus);
> ms->possible_cpus->len = max_cpus;
> for (i = 0; i < ms->possible_cpus->len; i++) {
> + CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
> +
> ms->possible_cpus->cpus[i].type = ms->cpu_type;
> ms->possible_cpus->cpus[i].vcpus_count = 1;
> ms->possible_cpus->cpus[i].arch_id = i;
> - ms->possible_cpus->cpus[i].props.has_core_id = true;
> - ms->possible_cpus->cpus[i].props.core_id = i;
> +
> + props->has_core_id = true;
> + props->core_id = i;
> + props->has_socket_id = true;
> + props->socket_id = i / ms->smp.cores;
> + props->has_book_id = true;
> + props->book_id = i / (ms->smp.cores * ms->smp.sockets);
> + props->has_drawer_id = true;
> + props->drawer_id = i /
> + (ms->smp.cores * ms->smp.sockets * ms->smp.books);
You need to calculate the modulus like in s390_topology_cpu_default, right?
> }
>
> return ms->possible_cpus;
> diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build
> index f291016fee..58dfbdff4f 100644
> --- a/hw/s390x/meson.build
> +++ b/hw/s390x/meson.build
> @@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
> 's390-stattrib-kvm.c',
> 'pv.c',
> 's390-pci-kvm.c',
> + 'cpu-topology.c',
> ))
> s390x_ss.add(when: 'CONFIG_TCG', if_true: files(
> 'tod-tcg.c',
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-02 16:42 ` Nina Schoetterl-Glausch
@ 2023-02-03 9:21 ` Pierre Morel
2023-02-03 13:22 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-03 9:21 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/2/23 17:42, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> The topology information are attributes of the CPU and are
>> specified during the CPU device creation.
>>
>> On hot plug we:
>> - calculate the default values for the topology for drawers,
>> books and sockets in the case they are not specified.
>> - verify the CPU attributes
>> - check that we have still room on the desired socket
>>
>> The possibility to insert a CPU in a mask is dependent on the
>> number of cores allowed in a socket, a book or a drawer, the
>> checking is done during the hot plug of the CPU to have an
>> immediate answer.
>>
>> If the complete topology is not specified, the core is added
>> in the physical topology based on its core ID and it gets
>> defaults values for the modifier attributes.
>>
>> This way, starting QEMU without specifying the topology can
>> still get some advantage of the CPU topology.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> include/hw/s390x/cpu-topology.h | 24 +++
>> hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
>> hw/s390x/s390-virtio-ccw.c | 23 ++-
>> hw/s390x/meson.build | 1 +
>> 4 files changed, 302 insertions(+), 2 deletions(-)
>> create mode 100644 hw/s390x/cpu-topology.c
>>
> [...]
>>
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> new file mode 100644
>> index 0000000000..12df4eca6c
>> --- /dev/null
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -0,0 +1,256 @@
>> +/*
>> + * CPU Topology
>> + *
>> + * Copyright IBM Corp. 2022
>> + * Author(s): Pierre Morel <pmorel@linux.ibm.com>
>> +
>> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
>> + * your option) any later version. See the COPYING file in the top-level
>> + * directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/error.h"
>> +#include "qemu/error-report.h"
>> +#include "hw/qdev-properties.h"
>> +#include "hw/boards.h"
>> +#include "qemu/typedefs.h"
>> +#include "target/s390x/cpu.h"
>> +#include "hw/s390x/s390-virtio-ccw.h"
>> +#include "hw/s390x/cpu-topology.h"
>> +
>> +/*
>> + * s390_topology is used to keep the topology information.
>> + * .list: queue the topology entries inside which
>> + * we keep the information on the CPU topology.
>
> .list doesn't exist yet.
right, I move it to the corresponding patch.
>
>> + * .socket: tracks information on the count of cores per socket.
>> + * .smp: keeps track of the machine topology.
>> + *
>> + */
>> +S390Topology s390_topology = {
>> + /* will be initialized after the cpu model is realized */
>> + .cores_per_socket = NULL,
>> + .smp = NULL,
>> +};
>>
> [...]
>> +
>> +/**
>> + * s390_topology_cpu_default:
>> + * @cpu: pointer to a S390CPU
>> + * @errp: Error pointer
>> + *
>> + * Setup the default topology for unset attributes.
>
> My suggestion:
> Setup the default topology if no attributes are already set.
> Passing a CPU with some, but not all, attributes set is considered an error.
Thanks, I take your suggestion.
>
>> + *
>> + * The function accept only all all default values or all set values
>> + * for the geometry topology.
>
> acceptS, all all
> If you take my suggestion, you can just drop this sentence.
>
>> + *
>> + * The function calculates the (drawer_id, book_id, socket_id)
>> + * topology by filling the cores starting from the first socket
>> + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets).
>> + *
>> + * CPU type, polarity and dedication have defaults values set in the
>> + * s390x_cpu_properties.
>> + */
>> +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
>> +{
>> + CpuTopology *smp = s390_topology.smp;
>> + CPUS390XState *env = &cpu->env;
>> +
>> + /* All geometry topology attributes must be set or all unset */
>> + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) &&
>> + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) {
>> + error_setg(errp,
>> + "Please define all or none of the topology geometry attributes");
>> + return;
>> + }
>> +
>> + /* Check if one of the geometry topology is unset */
>> + if (env->socket_id < 0) {
>> + /* Calculate default geometry topology attributes */
>> + env->socket_id = (env->core_id / smp->cores) % smp->sockets;
>> + env->book_id = (env->core_id / (smp->sockets * smp->cores)) %
>> + smp->books;
>> + env->drawer_id = (env->core_id /
>> + (smp->books * smp->sockets * smp->cores)) %
>> + smp->drawers;
>> + }
>> +}
>>
> [...]
>> +
>> +/**
>> + * s390_set_core_in_socket:
>> + * @cpu: the new S390CPU to insert in the topology structure
>> + * @drawer_id: new drawer_id
>> + * @book_id: new book_id
>> + * @socket_id: new socket_id
>> + * @creation: if is true the CPU is a new CPU and there is no old socket
>> + * to handle.
>> + * if is false, this is a moving the CPU and old socket count
>> + * must be decremented.
>> + * @errp: the error pointer
>> + *
>> + */
>> +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
>
> Maybe name it s390_(topology_)?add_core_to_socket instead.
OK, it is better
>
>> + int socket_id, bool creation, Error **errp)
>> +{
>> + int old_socket = s390_socket_nb(cpu);
>> + int new_socket;
>> +
>> + if (creation) {
>> + new_socket = old_socket;
>> + } else {
>
> You need parentheses here.
>
>> + new_socket = drawer_id * s390_topology.smp->books +
> (
>> + book_id * s390_topology.smp->sockets +
> )
>> + socket_id;
If you prefer I can us parentheses.
>> + }
>> +
>> + /* Check for space on new socket */
>> + if ((new_socket != old_socket) &&
>> + (s390_topology.cores_per_socket[new_socket] >=
>> + s390_topology.smp->cores)) {
>> + error_setg(errp, "No more space on this socket");
>> + return;
>> + }
>> +
>> + /* Update the count of cores in sockets */
>> + s390_topology.cores_per_socket[new_socket] += 1;
>> + if (!creation) {
>> + s390_topology.cores_per_socket[old_socket] -= 1;
>> + }
>> +}
>>
> [...]
>
>> +/**
>> + * s390_topology_set_cpu:
>> + * @ms: MachineState used to initialize the topology structure on
>> + * first call.
>> + * @cpu: the new S390CPU to insert in the topology structure
>> + * @errp: the error pointer
>> + *
>> + * Called from CPU Hotplug to check and setup the CPU attributes
>> + * before to insert the CPU in the topology.
>> + */
>> +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
>
> The name is rather non informative.
> s390_topology_setup_cpu ?
OK
>
>> +{
>> + ERRP_GUARD();
>> +
>> + /*
>> + * We do not want to initialize the topology if the cpu model
>> + * does not support topology consequently, we have to wait for
> ^
> Still think there should be a comma here.
OK
>
>> + * the first CPU to be realized, which realizes the CPU model
>> + * to initialize the topology structures.
>> + *
>> + * s390_topology_set_cpu() is called from the cpu hotplug.
>> + */
>> + if (!s390_topology.cores_per_socket) {
>> + s390_topology_init(ms);
>> + }
>> +
>> + s390_topology_check(cpu, errp);
>> + if (*errp) {
>> + return;
>> + }
>> +
>> + /* Set the CPU inside the socket */
>> + s390_set_core_in_socket(cpu, 0, 0, 0, true, errp);
>> + if (*errp) {
>> + return;
>> + }
>> +
>> + /* topology tree is reflected in props */
>> + s390_update_cpu_props(ms, cpu);
>> +}
>> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
>> index f3cc845d3b..9bc51a83f4 100644
>> --- a/hw/s390x/s390-virtio-ccw.c
>> +++ b/hw/s390x/s390-virtio-ccw.c
>> @@ -44,6 +44,7 @@
>> #include "hw/s390x/pv.h"
>> #include "migration/blocker.h"
>> #include "qapi/visitor.h"
>> +#include "hw/s390x/cpu-topology.h"
>>
>> static Error *pv_mig_blocker;
>>
>> @@ -310,10 +311,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
>> {
>> MachineState *ms = MACHINE(hotplug_dev);
>> S390CPU *cpu = S390_CPU(dev);
>> + ERRP_GUARD();
>>
>> g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
>> ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
>>
>> + if (s390_has_topology()) {
>> + s390_topology_set_cpu(ms, cpu, errp);
>> + if (*errp) {
>> + return;
>> + }
>> + }
>> +
>> if (dev->hotplugged) {
>> raise_irq_cpu_hotplug();
>> }
>> @@ -551,11 +560,21 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
>> sizeof(CPUArchId) * max_cpus);
>> ms->possible_cpus->len = max_cpus;
>> for (i = 0; i < ms->possible_cpus->len; i++) {
>> + CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
>> +
>> ms->possible_cpus->cpus[i].type = ms->cpu_type;
>> ms->possible_cpus->cpus[i].vcpus_count = 1;
>> ms->possible_cpus->cpus[i].arch_id = i;
>> - ms->possible_cpus->cpus[i].props.has_core_id = true;
>> - ms->possible_cpus->cpus[i].props.core_id = i;
>> +
>> + props->has_core_id = true;
>> + props->core_id = i;
>> + props->has_socket_id = true;
>> + props->socket_id = i / ms->smp.cores;
>> + props->has_book_id = true;
>> + props->book_id = i / (ms->smp.cores * ms->smp.sockets);
>> + props->has_drawer_id = true;
>> + props->drawer_id = i /
>> + (ms->smp.cores * ms->smp.sockets * ms->smp.books);
>
> You need to calculate the modulus like in s390_topology_cpu_default, right?
!!! yes of course, good catch, I forgot that.
Since there are two uses of this calculation, what about using inlines?
like:
static inline int s390_std_socket(int n, CpuTopology *smp)
{
return (n / smp->cores) % smp->sockets;
}
static inline int s390_std_book(int n, CpuTopology *smp)
{
return (n / (smp->cores * smp->sockets)) % smp->books;
}
static inline int s390_std_drawer(int n, CpuTopology *smp)
{
return (n / (smp->cores * smp->sockets * smp->books)) % smp->books;
}
Thanks for the comments.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-02 16:05 ` Nina Schoetterl-Glausch
@ 2023-02-03 9:39 ` Pierre Morel
2023-02-03 11:21 ` Thomas Huth
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-03 9:39 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
@Thomas, these changes look good to me.
What do you think, do I make the change and keep your RB ?
On 2/2/23 17:05, Nina Schoetterl-Glausch wrote:
> Nit patch title: s390x/cpu topology: add s390 specifics to CPU topology ?
>
OK
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> S390 adds two new SMP levels, drawers and books to the CPU
>> topology.
>> The S390 CPU have specific toplogy features like dedication
> ^o
Yes thx
>> and polarity to give to the guest indications on the host
>> vCPUs scheduling and help the guest take the best decisions
>> on the scheduling of threads on the vCPUs.
>>
>> Let us provide the SMP properties with books and drawers levels
>> and S390 CPU with dedication and polarity,
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine.json | 14 ++++++++--
>> include/hw/boards.h | 10 ++++++-
>> include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
>> target/s390x/cpu.h | 5 ++++
>> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
>> hw/core/machine.c | 4 +++
>> hw/s390x/s390-virtio-ccw.c | 2 ++
>> softmmu/vl.c | 6 +++++
>> target/s390x/cpu.c | 7 +++++
>> qemu-options.hx | 7 +++--
>> 10 files changed, 115 insertions(+), 12 deletions(-)
>> create mode 100644 include/hw/s390x/cpu-topology.h
>>
> [...]
>>
>> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
>> new file mode 100644
>> index 0000000000..7a84b30a21
>> --- /dev/null
>> +++ b/include/hw/s390x/cpu-topology.h
>> @@ -0,0 +1,24 @@
>> +/*
>> + * CPU Topology
>> + *
>> + * Copyright IBM Corp. 2022
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
>> + * your option) any later version. See the COPYING file in the top-level
>> + * directory.
>> + */
>> +#ifndef HW_S390X_CPU_TOPOLOGY_H
>> +#define HW_S390X_CPU_TOPOLOGY_H
>> +
>> +#define S390_TOPOLOGY_CPU_IFL 0x03
>> +
>> +enum s390_topology_polarity {
>> + POLARITY_HORIZONTAL,
>> + POLARITY_VERTICAL,
>> + POLARITY_VERTICAL_LOW = 1,
>> + POLARITY_VERTICAL_MEDIUM,
>> + POLARITY_VERTICAL_HIGH,
>> + POLARITY_MAX,
>> +};
>
> Probably a good idea to keep the S390 prefix.
> This works, but aliasing VERTICAL and VERTICAL_LOW is not
> entirely straight forward.
>
> Why not have two enum?
> enum s390_topology_polarity {
> S390_POLARITY_HORIZONTAL,
> S390_POLARITY_VERTICAL,
> };
>
> enum s390_topology_entitlement {
> S390_ENTITLEMENT_LOW = 1,
> S390_ENTITLEMENT_MEDIUM,
> S390_ENTITLEMENT_HIGH,
> S390_ENTITLEMENT_MAX,
> };
> Maybe add an ENTITLEMENT_INVALID/NONE, if you need that, as first value.
>
If Thomas agree, I do the changes.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-03 9:39 ` Pierre Morel
@ 2023-02-03 11:21 ` Thomas Huth
0 siblings, 0 replies; 76+ messages in thread
From: Thomas Huth @ 2023-02-03 11:21 UTC (permalink / raw)
To: Pierre Morel, Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, frankja, berrange, clg
On 03/02/2023 10.39, Pierre Morel wrote:
>
> @Thomas, these changes look good to me.
> What do you think, do I make the change and keep your RB ?
Yes, splitting that enum sounds cleaner, indeed.
Thomas
> On 2/2/23 17:05, Nina Schoetterl-Glausch wrote:
>> Nit patch title: s390x/cpu topology: add s390 specifics to CPU topology ?
>>
>
> OK
>
>> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>>> S390 adds two new SMP levels, drawers and books to the CPU
>>> topology.
>>> The S390 CPU have specific toplogy features like dedication
>> ^o
>
> Yes thx
>
>>> and polarity to give to the guest indications on the host
>>> vCPUs scheduling and help the guest take the best decisions
>>> on the scheduling of threads on the vCPUs.
>>>
>>> Let us provide the SMP properties with books and drawers levels
>>> and S390 CPU with dedication and polarity,
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> ---
>>> qapi/machine.json | 14 ++++++++--
>>> include/hw/boards.h | 10 ++++++-
>>> include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
>>> target/s390x/cpu.h | 5 ++++
>>> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
>>> hw/core/machine.c | 4 +++
>>> hw/s390x/s390-virtio-ccw.c | 2 ++
>>> softmmu/vl.c | 6 +++++
>>> target/s390x/cpu.c | 7 +++++
>>> qemu-options.hx | 7 +++--
>>> 10 files changed, 115 insertions(+), 12 deletions(-)
>>> create mode 100644 include/hw/s390x/cpu-topology.h
>>>
>> [...]
>>>
>>> diff --git a/include/hw/s390x/cpu-topology.h
>>> b/include/hw/s390x/cpu-topology.h
>>> new file mode 100644
>>> index 0000000000..7a84b30a21
>>> --- /dev/null
>>> +++ b/include/hw/s390x/cpu-topology.h
>>> @@ -0,0 +1,24 @@
>>> +/*
>>> + * CPU Topology
>>> + *
>>> + * Copyright IBM Corp. 2022
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
>>> + * your option) any later version. See the COPYING file in the top-level
>>> + * directory.
>>> + */
>>> +#ifndef HW_S390X_CPU_TOPOLOGY_H
>>> +#define HW_S390X_CPU_TOPOLOGY_H
>>> +
>>> +#define S390_TOPOLOGY_CPU_IFL 0x03
>>> +
>>> +enum s390_topology_polarity {
>>> + POLARITY_HORIZONTAL,
>>> + POLARITY_VERTICAL,
>>> + POLARITY_VERTICAL_LOW = 1,
>>> + POLARITY_VERTICAL_MEDIUM,
>>> + POLARITY_VERTICAL_HIGH,
>>> + POLARITY_MAX,
>>> +};
>>
>> Probably a good idea to keep the S390 prefix.
>> This works, but aliasing VERTICAL and VERTICAL_LOW is not
>> entirely straight forward.
>>
>> Why not have two enum?
>> enum s390_topology_polarity {
>> S390_POLARITY_HORIZONTAL,
>> S390_POLARITY_VERTICAL,
>> };
>>
>> enum s390_topology_entitlement {
>> S390_ENTITLEMENT_LOW = 1,
>> S390_ENTITLEMENT_MEDIUM,
>> S390_ENTITLEMENT_HIGH,
>> S390_ENTITLEMENT_MAX,
>> };
>> Maybe add an ENTITLEMENT_INVALID/NONE, if you need that, as first value.
>>
>
> If Thomas agree, I do the changes.
>
> Regards,
> Pierre
>
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-03 9:21 ` Pierre Morel
@ 2023-02-03 13:22 ` Nina Schoetterl-Glausch
2023-02-03 14:40 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-03 13:22 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Fri, 2023-02-03 at 10:21 +0100, Pierre Morel wrote:
>
> On 2/2/23 17:42, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > The topology information are attributes of the CPU and are
> > > specified during the CPU device creation.
> > >
> > > On hot plug we:
> > > - calculate the default values for the topology for drawers,
> > > books and sockets in the case they are not specified.
> > > - verify the CPU attributes
> > > - check that we have still room on the desired socket
> > >
> > > The possibility to insert a CPU in a mask is dependent on the
> > > number of cores allowed in a socket, a book or a drawer, the
> > > checking is done during the hot plug of the CPU to have an
> > > immediate answer.
> > >
> > > If the complete topology is not specified, the core is added
> > > in the physical topology based on its core ID and it gets
> > > defaults values for the modifier attributes.
> > >
> > > This way, starting QEMU without specifying the topology can
> > > still get some advantage of the CPU topology.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > include/hw/s390x/cpu-topology.h | 24 +++
> > > hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
> > > hw/s390x/s390-virtio-ccw.c | 23 ++-
> > > hw/s390x/meson.build | 1 +
> > > 4 files changed, 302 insertions(+), 2 deletions(-)
> > > create mode 100644 hw/s390x/cpu-topology.c
> > >
[...]
> >
> > > +/**
> > > + * s390_set_core_in_socket:
> > > + * @cpu: the new S390CPU to insert in the topology structure
> > > + * @drawer_id: new drawer_id
> > > + * @book_id: new book_id
> > > + * @socket_id: new socket_id
> > > + * @creation: if is true the CPU is a new CPU and there is no old socket
> > > + * to handle.
> > > + * if is false, this is a moving the CPU and old socket count
> > > + * must be decremented.
> > > + * @errp: the error pointer
> > > + *
> > > + */
> > > +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
> >
> > Maybe name it s390_(topology_)?add_core_to_socket instead.
>
> OK, it is better
>
> >
> > > + int socket_id, bool creation, Error **errp)
> > > +{
> > > + int old_socket = s390_socket_nb(cpu);
> > > + int new_socket;
> > > +
> > > + if (creation) {
> > > + new_socket = old_socket;
> > > + } else {
> >
> > You need parentheses here.
> >
> > > + new_socket = drawer_id * s390_topology.smp->books +
> > (
> > > + book_id * s390_topology.smp->sockets +
> > )
> > > + socket_id;
>
> If you prefer I can us parentheses.
It's necessary, otherwise the multiplication of book_id and smp->sockets takes precedence.
>
>
> > > + }
> > > +
> > > + /* Check for space on new socket */
> > > + if ((new_socket != old_socket) &&
> > > + (s390_topology.cores_per_socket[new_socket] >=
> > > + s390_topology.smp->cores)) {
> > > + error_setg(errp, "No more space on this socket");
> > > + return;
> > > + }
> > > +
> > > + /* Update the count of cores in sockets */
> > > + s390_topology.cores_per_socket[new_socket] += 1;
> > > + if (!creation) {
> > > + s390_topology.cores_per_socket[old_socket] -= 1;
> > > + }
> > > +}
> > >
[...]
> >
> > > diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> > > index f3cc845d3b..9bc51a83f4 100644
> > > --- a/hw/s390x/s390-virtio-ccw.c
> > > +++ b/hw/s390x/s390-virtio-ccw.c
> > > @@ -44,6 +44,7 @@
> > > #include "hw/s390x/pv.h"
> > > #include "migration/blocker.h"
> > > #include "qapi/visitor.h"
> > > +#include "hw/s390x/cpu-topology.h"
> > >
> > > static Error *pv_mig_blocker;
> > >
> > > @@ -310,10 +311,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
> > > {
> > > MachineState *ms = MACHINE(hotplug_dev);
> > > S390CPU *cpu = S390_CPU(dev);
> > > + ERRP_GUARD();
> > >
> > > g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
> > > ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
> > >
> > > + if (s390_has_topology()) {
> > > + s390_topology_set_cpu(ms, cpu, errp);
> > > + if (*errp) {
> > > + return;
> > > + }
> > > + }
> > > +
> > > if (dev->hotplugged) {
> > > raise_irq_cpu_hotplug();
> > > }
> > > @@ -551,11 +560,21 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
> > > sizeof(CPUArchId) * max_cpus);
> > > ms->possible_cpus->len = max_cpus;
> > > for (i = 0; i < ms->possible_cpus->len; i++) {
> > > + CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props;
> > > +
> > > ms->possible_cpus->cpus[i].type = ms->cpu_type;
> > > ms->possible_cpus->cpus[i].vcpus_count = 1;
> > > ms->possible_cpus->cpus[i].arch_id = i;
> > > - ms->possible_cpus->cpus[i].props.has_core_id = true;
> > > - ms->possible_cpus->cpus[i].props.core_id = i;
> > > +
> > > + props->has_core_id = true;
> > > + props->core_id = i;
> > > + props->has_socket_id = true;
> > > + props->socket_id = i / ms->smp.cores;
> > > + props->has_book_id = true;
> > > + props->book_id = i / (ms->smp.cores * ms->smp.sockets);
> > > + props->has_drawer_id = true;
> > > + props->drawer_id = i /
> > > + (ms->smp.cores * ms->smp.sockets * ms->smp.books);
> >
> > You need to calculate the modulus like in s390_topology_cpu_default, right?
>
> !!! yes of course, good catch, I forgot that.
>
> Since there are two uses of this calculation, what about using inlines?
> like:
Fine by me, I just have no idea what std is short for.
Since this would be used in s390_topology_cpu_default and calculates the default assignment,
I would call it s390_topology_cpu_default_socket, book, etc. or similar
>
> static inline int s390_std_socket(int n, CpuTopology *smp)
> {
> return (n / smp->cores) % smp->sockets;
> }
>
> static inline int s390_std_book(int n, CpuTopology *smp)
> {
> return (n / (smp->cores * smp->sockets)) % smp->books;
> }
>
> static inline int s390_std_drawer(int n, CpuTopology *smp)
> {
> return (n / (smp->cores * smp->sockets * smp->books)) % smp->books;
> }
>
>
> Thanks for the comments.
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-03 13:22 ` Nina Schoetterl-Glausch
@ 2023-02-03 14:40 ` Pierre Morel
2023-02-03 15:38 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-03 14:40 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/3/23 14:22, Nina Schoetterl-Glausch wrote:
> On Fri, 2023-02-03 at 10:21 +0100, Pierre Morel wrote:
>>
>> On 2/2/23 17:42, Nina Schoetterl-Glausch wrote:
>>> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>>>> The topology information are attributes of the CPU and are
>>>> specified during the CPU device creation.
>>>>
>>>> On hot plug we:
>>>> - calculate the default values for the topology for drawers,
>>>> books and sockets in the case they are not specified.
>>>> - verify the CPU attributes
>>>> - check that we have still room on the desired socket
>>>>
>>>> The possibility to insert a CPU in a mask is dependent on the
>>>> number of cores allowed in a socket, a book or a drawer, the
>>>> checking is done during the hot plug of the CPU to have an
>>>> immediate answer.
>>>>
>>>> If the complete topology is not specified, the core is added
>>>> in the physical topology based on its core ID and it gets
>>>> defaults values for the modifier attributes.
>>>>
>>>> This way, starting QEMU without specifying the topology can
>>>> still get some advantage of the CPU topology.
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> ---
>>>> include/hw/s390x/cpu-topology.h | 24 +++
>>>> hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
>>>> hw/s390x/s390-virtio-ccw.c | 23 ++-
>>>> hw/s390x/meson.build | 1 +
>>>> 4 files changed, 302 insertions(+), 2 deletions(-)
>>>> create mode 100644 hw/s390x/cpu-topology.c
>>>>
> [...]
>>>
>>>> +/**
>>>> + * s390_set_core_in_socket:
>>>> + * @cpu: the new S390CPU to insert in the topology structure
>>>> + * @drawer_id: new drawer_id
>>>> + * @book_id: new book_id
>>>> + * @socket_id: new socket_id
>>>> + * @creation: if is true the CPU is a new CPU and there is no old socket
>>>> + * to handle.
>>>> + * if is false, this is a moving the CPU and old socket count
>>>> + * must be decremented.
>>>> + * @errp: the error pointer
>>>> + *
>>>> + */
>>>> +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
>>>
>>> Maybe name it s390_(topology_)?add_core_to_socket instead.
>>
>> OK, it is better
>>
>>>
>>>> + int socket_id, bool creation, Error **errp)
>>>> +{
>>>> + int old_socket = s390_socket_nb(cpu);
>>>> + int new_socket;
>>>> +
>>>> + if (creation) {
>>>> + new_socket = old_socket;
>>>> + } else {
>>>
>>> You need parentheses here.
>>>
>>>> + new_socket = drawer_id * s390_topology.smp->books +
>>> (
>>>> + book_id * s390_topology.smp->sockets +
>>> )
>>>> + socket_id;
>>
>> If you prefer I can us parentheses.
>
> It's necessary, otherwise the multiplication of book_id and smp->sockets takes precedence.
>>
>>
Right, I did not understand where you want the parenthesis.
I think you mean:
new_socket = (drawer_id * s390_topology.smp->books + book_id) *
s390_topology.smp->sockets + socket_id;
thanks,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug
2023-02-03 14:40 ` Pierre Morel
@ 2023-02-03 15:38 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-03 15:38 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Fri, 2023-02-03 at 15:40 +0100, Pierre Morel wrote:
>
> On 2/3/23 14:22, Nina Schoetterl-Glausch wrote:
> > On Fri, 2023-02-03 at 10:21 +0100, Pierre Morel wrote:
> > >
> > > On 2/2/23 17:42, Nina Schoetterl-Glausch wrote:
> > > > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > > > The topology information are attributes of the CPU and are
> > > > > specified during the CPU device creation.
> > > > >
> > > > > On hot plug we:
> > > > > - calculate the default values for the topology for drawers,
> > > > > books and sockets in the case they are not specified.
> > > > > - verify the CPU attributes
> > > > > - check that we have still room on the desired socket
> > > > >
> > > > > The possibility to insert a CPU in a mask is dependent on the
> > > > > number of cores allowed in a socket, a book or a drawer, the
> > > > > checking is done during the hot plug of the CPU to have an
> > > > > immediate answer.
> > > > >
> > > > > If the complete topology is not specified, the core is added
> > > > > in the physical topology based on its core ID and it gets
> > > > > defaults values for the modifier attributes.
> > > > >
> > > > > This way, starting QEMU without specifying the topology can
> > > > > still get some advantage of the CPU topology.
> > > > >
> > > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > > ---
> > > > > include/hw/s390x/cpu-topology.h | 24 +++
> > > > > hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++
> > > > > hw/s390x/s390-virtio-ccw.c | 23 ++-
> > > > > hw/s390x/meson.build | 1 +
> > > > > 4 files changed, 302 insertions(+), 2 deletions(-)
> > > > > create mode 100644 hw/s390x/cpu-topology.c
> > > > >
> > [...]
> > > >
> > > > > +/**
> > > > > + * s390_set_core_in_socket:
> > > > > + * @cpu: the new S390CPU to insert in the topology structure
> > > > > + * @drawer_id: new drawer_id
> > > > > + * @book_id: new book_id
> > > > > + * @socket_id: new socket_id
> > > > > + * @creation: if is true the CPU is a new CPU and there is no old socket
> > > > > + * to handle.
> > > > > + * if is false, this is a moving the CPU and old socket count
> > > > > + * must be decremented.
> > > > > + * @errp: the error pointer
> > > > > + *
> > > > > + */
> > > > > +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id,
> > > >
> > > > Maybe name it s390_(topology_)?add_core_to_socket instead.
> > >
> > > OK, it is better
> > >
> > > >
> > > > > + int socket_id, bool creation, Error **errp)
> > > > > +{
> > > > > + int old_socket = s390_socket_nb(cpu);
> > > > > + int new_socket;
> > > > > +
> > > > > + if (creation) {
> > > > > + new_socket = old_socket;
> > > > > + } else {
> > > >
> > > > You need parentheses here.
> > > >
> > > > > + new_socket = drawer_id * s390_topology.smp->books +
> > > > (
> > > > > + book_id * s390_topology.smp->sockets +
> > > > )
> > > > > + socket_id;
> > >
> > > If you prefer I can us parentheses.
> >
> > It's necessary, otherwise the multiplication of book_id and smp->sockets takes precedence.
> > >
> > >
>
> Right, I did not understand where you want the parenthesis.
> I think you mean:
>
> new_socket = (drawer_id * s390_topology.smp->books + book_id) *
> s390_topology.smp->sockets + socket_id;
Yes, exactly. I see how it was ambiguous.
>
> thanks,
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
@ 2023-02-03 17:36 ` Nina Schoetterl-Glausch
2023-02-06 10:06 ` Pierre Morel
2023-02-06 11:24 ` Thomas Huth
2023-02-09 16:39 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-03 17:36 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > On interception of STSI(15.1.x) the System Information Block
> > (SYSIB) is built from the list of pre-ordered topology entries.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > ---
> > include/hw/s390x/cpu-topology.h | 22 +++
> > include/hw/s390x/sclp.h | 1 +
> > target/s390x/cpu.h | 72 +++++++
> > hw/s390x/cpu-topology.c | 10 +
> > target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
> > target/s390x/kvm/kvm.c | 5 +-
> > target/s390x/kvm/meson.build | 3 +-
> > 7 files changed, 446 insertions(+), 2 deletions(-)
> > create mode 100644 target/s390x/kvm/cpu_topology.c
> >
[...]
> >
> > diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> > index d654267a71..e1f6925856 100644
> > --- a/target/s390x/cpu.h
> > +++ b/target/s390x/cpu.h
[...]
> > +
> > +/* CPU type Topology List Entry */
> > +typedef struct SysIBTl_cpu {
> > + uint8_t nl;
> > + uint8_t reserved0[3];
> > +#define SYSIB_TLE_POLARITY_MASK 0x03
> > +#define SYSIB_TLE_DEDICATED 0x04
> > + uint8_t entitlement;
I would just call this flags, since it's multiple fields.
> > + uint8_t type;
> > + uint16_t origin;
> > + uint64_t mask;
> > +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
> > +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
> >
> > +
> >
[...]
> > /**
> > diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c
> > new file mode 100644
> > index 0000000000..aba141fb66
> > --- /dev/null
> > +++ b/target/s390x/kvm/cpu_topology.c
> >
[...]
> > +
> > +/*
> > + * Macro to check that the size of data after increment
> > + * will not get bigger than the size of the SysIB.
> > + */
> > +#define SYSIB_GUARD(data, x) do { \
> > + data += x; \
> > + if (data > sizeof(SysIB)) { \
> > + return -ENOSPC; \
I would go with ENOMEM here.
> > + } \
> > + } while (0)
> > +
> > +/**
> > + * stsi_set_tle:
> > + * @p: A pointer to the position of the first TLE
> > + * @level: The nested level wanted by the guest
> > + *
> > + * Loop inside the s390_topology.list until the sentinelle entry
s/sentinelle/sentinel/
> > + * is found and for each entry:
> > + * - Check using SYSIB_GUARD() that the size of the SysIB is not
> > + * reached.
> > + * - Add all the container TLE needed for the level
> > + * - Add the CPU TLE.
I'd focus more on *what* the function does instead of *how*.
Fill the SYSIB with the topology information as described in the PoP,
nesting containers as appropriate, with the maximum nesting limited by @level.
Or something similar.
> > + *
> > + * Return value:
> > + * s390_top_set_level returns the size of the SysIB_15x after being
You forgot to rename the function here, right?
How about stsi_fill_topology_sysib or stsi_topology_fill_sysib, instead?
> > + * filled with TLE on success.
> > + * It returns -ENOSPC in the case we would overrun the end of the SysIB.
You would have to change to ENOMEM here than also.
> > + */
> > +static int stsi_set_tle(char *p, int level)
> > +{
> > + S390TopologyEntry *entry;
> > + int last_drawer = -1;
> > + int last_book = -1;
> > + int last_socket = -1;
> > + int drawer_id = 0;
> > + int book_id = 0;
> > + int socket_id = 0;
> > + int n = sizeof(SysIB_151x);
> > +
> > + QTAILQ_FOREACH(entry, &s390_topology.list, next) {
> > + int current_drawer = entry->id.drawer;
> > + int current_book = entry->id.book;
> > + int current_socket = entry->id.socket;
This only saves two characters, so you could just use entry->id. ...
> > + bool drawer_change = last_drawer != current_drawer;
> > + bool book_change = drawer_change || last_book != current_book;
> > + bool socket_change = book_change || last_socket != current_socket;
... but keep it if it would make this line too long.
You could also rename entry, to current or cur, if you want to emphasize that.
> > +
> > + /* If we reach the guard get out */
> > + if (entry->id.level5) {
> > + break;
> > + }
> > +
> > + if (level > 3 && drawer_change) {
> > + SYSIB_GUARD(n, sizeof(SysIBTl_container));
> > + p = fill_container(p, 3, drawer_id++);
> > + book_id = 0;
> > + }
> > + if (level > 2 && book_change) {
> > + SYSIB_GUARD(n, sizeof(SysIBTl_container));
> > + p = fill_container(p, 2, book_id++);
> > + socket_id = 0;
> > + }
> > + if (socket_change) {
> > + SYSIB_GUARD(n, sizeof(SysIBTl_container));
> > + p = fill_container(p, 1, socket_id++);
> > + }
> > +
> > + SYSIB_GUARD(n, sizeof(SysIBTl_cpu));
> > + p = fill_tle_cpu(p, entry);
> > + last_drawer = entry->id.drawer;
> > + last_book = entry->id.book;
> > + last_socket = entry->id.socket;
> > + }
> > +
> > + return n;
> > +}
> > +
> > +/**
> > + * setup_stsi:
> > + * sysib: pointer to a SysIB to be filled with SysIB_151x data
> > + * level: Nested level specified by the guest
> > + *
> > + * Setup the SysIB_151x header before calling stsi_set_tle with
> > + * a pointer to the first TLE entry.
Same thing here with regards to describing the what.
Setup the SYSIB for STSI 15.1, the header as well as the description
of the topology.
> > + */
> > +static int setup_stsi(SysIB_151x *sysib, int level)
> > +{
> > + sysib->mnest = level;
> > + switch (level) {
> > + case 4:
> > + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
> > + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
> > + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> > + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> > + break;
> > + case 3:
> > + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
> > + current_machine->smp.books;
> > + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
> > + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> > + break;
> > + case 2:
> > + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
> > + current_machine->smp.books *
> > + current_machine->smp.sockets;
> > + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
> > + break;
> > + }
> > +
> > + return stsi_set_tle(sysib->tle, level);
> > +}
> > +
> > +/**
> > + * s390_topology_add_cpu_to_entry:
> > + * @entry: Topology entry to setup
> > + * @cpu: the S390CPU to add
> > + *
> > + * Set the core bit inside the topology mask and
> > + * increments the number of cores for the socket.
> > + */
> > +static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
> > + S390CPU *cpu)
> > +{
> > + set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
> > +}
> > +
> > +/**
> > + * s390_topology_new_entry:
> > + * @id: s390_topology_id to add
> > + * @cpu: the S390CPU to add
> > + *
> > + * Allocate a new entry and initialize it.
> > + *
> > + * returns the newly allocated entry.
> > + */
> > +static S390TopologyEntry *s390_topology_new_entry(s390_topology_id id,
> > + S390CPU *cpu)
This is used only once, right?
I think I'd go ahead and inline it into s390_topology_insert, since I had
to go back and check if new_entry calls add_cpu when reading s390_topology_insert.
> > +{
> > + S390TopologyEntry *entry;
> > +
> > + entry = g_malloc0(sizeof(S390TopologyEntry));
> > + entry->id.id = id.id;
> > + s390_topology_add_cpu_to_entry(entry, cpu);
> > +
> > + return entry;
> > +}
> > +
> > +/**
> > + * s390_topology_from_cpu:
> > + * @cpu: The S390CPU
> > + *
> > + * Initialize the topology id from the CPU environment.
> > + */
> > +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
> > +{
> > + s390_topology_id topology_id = {0};
> > +
> > + topology_id.drawer = cpu->env.drawer_id;
> > + topology_id.book = cpu->env.book_id;
> > + topology_id.socket = cpu->env.socket_id;
> > + topology_id.origin = cpu->env.core_id / 64;
> > + topology_id.type = S390_TOPOLOGY_CPU_IFL;
> > + topology_id.dedicated = cpu->env.dedicated;
> > +
> > + if (s390_topology.polarity == POLARITY_VERTICAL) {
> > + /*
> > + * Vertical polarity with dedicated CPU implies
> > + * vertical high entitlement.
> > + */
> > + if (topology_id.dedicated) {
> > + topology_id.polarity |= POLARITY_VERTICAL_HIGH;
> > + } else {
> > + topology_id.polarity |= cpu->env.entitlement;
> > + }
> > + }
> > +
> > + return topology_id;
> > +}
> > +
> > +/**
> > + * s390_topology_insert:
> > + * @cpu: s390CPU insert.
> > + *
> > + * Parse the topology list to find if the entry already
> > + * exist and add the core in it.
> > + * If it does not exist, allocate a new entry and insert
> > + * it in the queue from lower id to greater id.
> > + */
> > +static void s390_topology_insert(S390CPU *cpu)
> > +{
> > + s390_topology_id id = s390_topology_from_cpu(cpu);
> > + S390TopologyEntry *entry = NULL;
> > + S390TopologyEntry *tmp = NULL;
> > +
> > + QTAILQ_FOREACH(tmp, &s390_topology.list, next) {
> > + if (id.id == tmp->id.id) {
> > + s390_topology_add_cpu_to_entry(tmp, cpu);
> > + return;
> > + } else if (id.id < tmp->id.id) {
> > + entry = s390_topology_new_entry(id, cpu);
> > + QTAILQ_INSERT_BEFORE(tmp, entry, next);
> > + return;
> > + }
> > + }
> > +}
> > +
> > +/**
> > + * s390_order_tle:
> > + *
> > + * Loop over all CPU and insert it at the right place
> > + * inside the TLE entry list.
> > + */
Suggestion:
s390_topology_fill_list_sorted
Fill the S390Topology list with entries according to the order specified
by the PoP.
> > +static void s390_order_tle(void)
> > +{
> > + CPUState *cs;
> > +
> > + CPU_FOREACH(cs) {
> > + s390_topology_insert(S390_CPU(cs));
> > + }
> > +}
> > +
> > +/**
> > + * s390_free_tle:
> > + *
> > + * Loop over all TLE entries and free them.
> > + * Keep the sentinelle which is the only one with level5 != 0
s/sentinelle/sentinel/
> > + */
Suggestion:
s390_topology_empty_list
Clear all entries in the S390Topology list except the sentinel.
> > +static void s390_free_tle(void)
> > +{
> > + S390TopologyEntry *entry = NULL;
> > + S390TopologyEntry *tmp = NULL;
> > +
> > + QTAILQ_FOREACH_SAFE(entry, &s390_topology.list, next, tmp) {
> > + if (!entry->id.level5) {
> > + QTAILQ_REMOVE(&s390_topology.list, entry, next);
> > + g_free(entry);
> > + }
> > + }
> > +}
> > +
> > +/**
> > + * insert_stsi_15_1_x:
> > + * cpu: the CPU doing the call for which we set CC
> > + * sel2: the selector 2, containing the nested level
> > + * addr: Guest logical address of the guest SysIB
> > + * ar: the access register number
> > + *
> > + * Reserve a zeroed SysIB, let setup_stsi to fill it and
> > + * copy the SysIB to the guest memory.
> > + *
> > + * In case of overflow set CC(3) and no copy is done.
Suggestion:
Emulate STSI 15.1.x, that is, perform all necessary checks and fill the SYSIB.
In case the topology description is too long to fit into the SYSIB,
set CC=3 and abort without writing the SYSIB.
> > + */
> > +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
> > +{
> > + SysIB sysib = {0};
> > + int len;
> > +
> > + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
> > + setcc(cpu, 3);
> > + return;
> > + }
> > +
> > + s390_order_tle();
> > +
> > + len = setup_stsi(&sysib.sysib_151x, sel2);
> > +
> > + if (len < 0) {
I stumbled a bit over this, maybe rename len to r.
> > + setcc(cpu, 3);
> > + return;
> > + }
> > +
> > + sysib.sysib_151x.length = cpu_to_be16(len);
> > + s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, len);
> > + setcc(cpu, 0);
> > +
> > + s390_free_tle();
> > +}
> > diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> > index 3ac7ec9acf..5ea358cbb0 100644
> > --- a/target/s390x/kvm/kvm.c
> > +++ b/target/s390x/kvm/kvm.c
> > @@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu)
> > if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
> > return 0;
> > }
> > - /* Only sysib 3.2.2 needs post-handling for now. */
> > insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
> > return 0;
> > + case 15:
> > + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
> > + run->s390_stsi.ar);
> > + return 0;
> > default:
> > return 0;
> > }
> > diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build
> > index aef52b6686..5daa5c6033 100644
> > --- a/target/s390x/kvm/meson.build
> > +++ b/target/s390x/kvm/meson.build
> > @@ -1,6 +1,7 @@
> >
> > s390x_ss.add(when: 'CONFIG_KVM', if_true: files(
> > - 'kvm.c'
> > + 'kvm.c',
> > + 'cpu_topology.c'
> > ), if_false: files(
> > 'stubs.c'
> > ))
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-03 17:36 ` Nina Schoetterl-Glausch
@ 2023-02-06 10:06 ` Pierre Morel
2023-02-06 10:32 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 10:06 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/3/23 18:36, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>>> On interception of STSI(15.1.x) the System Information Block
>>> (SYSIB) is built from the list of pre-ordered topology entries.
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> ---
>>> include/hw/s390x/cpu-topology.h | 22 +++
>>> include/hw/s390x/sclp.h | 1 +
>>> target/s390x/cpu.h | 72 +++++++
>>> hw/s390x/cpu-topology.c | 10 +
>>> target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
>>> target/s390x/kvm/kvm.c | 5 +-
>>> target/s390x/kvm/meson.build | 3 +-
>>> 7 files changed, 446 insertions(+), 2 deletions(-)
>>> create mode 100644 target/s390x/kvm/cpu_topology.c
>>>
> [...]
>>>
>>> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
>>> index d654267a71..e1f6925856 100644
>>> --- a/target/s390x/cpu.h
>>> +++ b/target/s390x/cpu.h
> [...]
>>> +
>>> +/* CPU type Topology List Entry */
>>> +typedef struct SysIBTl_cpu {
>>> + uint8_t nl;
>>> + uint8_t reserved0[3];
>>> +#define SYSIB_TLE_POLARITY_MASK 0x03
>>> +#define SYSIB_TLE_DEDICATED 0x04
>>> + uint8_t entitlement;
>
> I would just call this flags, since it's multiple fields.
OK
>
>>> + uint8_t type;
>>> + uint16_t origin;
>>> + uint64_t mask;
>>> +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
>>> +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
>>>
>>> +
>>>
> [...]
>>> /**
>>> diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c
>>> new file mode 100644
>>> index 0000000000..aba141fb66
>>> --- /dev/null
>>> +++ b/target/s390x/kvm/cpu_topology.c
>>>
> [...]
>>> +
>>> +/*
>>> + * Macro to check that the size of data after increment
>>> + * will not get bigger than the size of the SysIB.
>>> + */
>>> +#define SYSIB_GUARD(data, x) do { \
>>> + data += x; \
>>> + if (data > sizeof(SysIB)) { \
>>> + return -ENOSPC; \
>
> I would go with ENOMEM here.
Considering your comment of the length in insert_stsi_15_1_x(), I think
the best would be to return 0.
Because:
- We do not need to report any error reason.
- The value will be forwarded to insert_stsi_15_1_x() as the length of
the SYSIB to write
- On error we do not write the SYSIB (len = 0)
- In normal case the return value is always non zero and positive.
>
>>> + } \
>>> + } while (0)
>>> +
>>> +/**
>>> + * stsi_set_tle:
>>> + * @p: A pointer to the position of the first TLE
>>> + * @level: The nested level wanted by the guest
>>> + *
>>> + * Loop inside the s390_topology.list until the sentinelle entry
>
> s/sentinelle/sentinel/
OK, thx,
>
>>> + * is found and for each entry:
>>> + * - Check using SYSIB_GUARD() that the size of the SysIB is not
>>> + * reached.
>>> + * - Add all the container TLE needed for the level
>>> + * - Add the CPU TLE.
>
> I'd focus more on *what* the function does instead of *how*.
Right.
>
> Fill the SYSIB with the topology information as described in the PoP,
> nesting containers as appropriate, with the maximum nesting limited by @level.
>
> Or something similar.
Thanks, looks good to me .
>
>>> + *
>>> + * Return value:
>>> + * s390_top_set_level returns the size of the SysIB_15x after being
>
> You forgot to rename the function here, right?
> How about stsi_fill_topology_sysib or stsi_topology_fill_sysib, instead?
OK with stsi_topology_fill_sysib()
>
>>> + * filled with TLE on success.
>>> + * It returns -ENOSPC in the case we would overrun the end of the SysIB.
>
> You would have to change to ENOMEM here than also.
As discussed above, it seems to me that return 0 is even better.
>
>>> + */
>>> +static int stsi_set_tle(char *p, int level)
>>> +{
>>> + S390TopologyEntry *entry;
>>> + int last_drawer = -1;
>>> + int last_book = -1;
>>> + int last_socket = -1;
>>> + int drawer_id = 0;
>>> + int book_id = 0;
>>> + int socket_id = 0;
>>> + int n = sizeof(SysIB_151x);
>>> +
>>> + QTAILQ_FOREACH(entry, &s390_topology.list, next) {
>>> + int current_drawer = entry->id.drawer;
>>> + int current_book = entry->id.book;
>>> + int current_socket = entry->id.socket;
>
> This only saves two characters, so you could just use entry->id. ...
OK
>
>>> + bool drawer_change = last_drawer != current_drawer;
>>> + bool book_change = drawer_change || last_book != current_book;
>>> + bool socket_change = book_change || last_socket != current_socket;
>
> ... but keep it if it would make this line too long.
it is OK
> You could also rename entry, to current or cur, if you want to emphasize that.
>
>>> +
>>> + /* If we reach the guard get out */
>>> + if (entry->id.level5) {
>>> + break;
>>> + }
>>> +
>>> + if (level > 3 && drawer_change) {
>>> + SYSIB_GUARD(n, sizeof(SysIBTl_container));
>>> + p = fill_container(p, 3, drawer_id++);
>>> + book_id = 0;
>>> + }
>>> + if (level > 2 && book_change) {
>>> + SYSIB_GUARD(n, sizeof(SysIBTl_container));
>>> + p = fill_container(p, 2, book_id++);
>>> + socket_id = 0;
>>> + }
>>> + if (socket_change) {
>>> + SYSIB_GUARD(n, sizeof(SysIBTl_container));
>>> + p = fill_container(p, 1, socket_id++);
>>> + }
>>> +
>>> + SYSIB_GUARD(n, sizeof(SysIBTl_cpu));
>>> + p = fill_tle_cpu(p, entry);
>>> + last_drawer = entry->id.drawer;
>>> + last_book = entry->id.book;
>>> + last_socket = entry->id.socket;
>>> + }
>>> +
>>> + return n;
>>> +}
>>> +
>>> +/**
>>> + * setup_stsi:
>>> + * sysib: pointer to a SysIB to be filled with SysIB_151x data
>>> + * level: Nested level specified by the guest
>>> + *
>>> + * Setup the SysIB_151x header before calling stsi_set_tle with
>>> + * a pointer to the first TLE entry.
>
> Same thing here with regards to describing the what.
>
> Setup the SYSIB for STSI 15.1, the header as well as the description
> of the topology.
OK, thx
>
>>> + */
>>> +static int setup_stsi(SysIB_151x *sysib, int level)
>>> +{
>>> + sysib->mnest = level;
>>> + switch (level) {
>>> + case 4:
>>> + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers;
>>> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books;
>>> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
>>> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
>>> + break;
>>> + case 3:
>>> + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers *
>>> + current_machine->smp.books;
>>> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets;
>>> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
>>> + break;
>>> + case 2:
>>> + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers *
>>> + current_machine->smp.books *
>>> + current_machine->smp.sockets;
>>> + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores;
>>> + break;
>>> + }
>>> +
>>> + return stsi_set_tle(sysib->tle, level);
>>> +}
>>> +
>>> +/**
>>> + * s390_topology_add_cpu_to_entry:
>>> + * @entry: Topology entry to setup
>>> + * @cpu: the S390CPU to add
>>> + *
>>> + * Set the core bit inside the topology mask and
>>> + * increments the number of cores for the socket.
>>> + */
>>> +static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry,
>>> + S390CPU *cpu)
>>> +{
>>> + set_bit(63 - (cpu->env.core_id % 64), &entry->mask);
>>> +}
>>> +
>>> +/**
>>> + * s390_topology_new_entry:
>>> + * @id: s390_topology_id to add
>>> + * @cpu: the S390CPU to add
>>> + *
>>> + * Allocate a new entry and initialize it.
>>> + *
>>> + * returns the newly allocated entry.
>>> + */
>>> +static S390TopologyEntry *s390_topology_new_entry(s390_topology_id id,
>>> + S390CPU *cpu)
>
> This is used only once, right?
> I think I'd go ahead and inline it into s390_topology_insert, since I had
> to go back and check if new_entry calls add_cpu when reading s390_topology_insert.
OK
>
>>> +{
>>> + S390TopologyEntry *entry;
>>> +
>>> + entry = g_malloc0(sizeof(S390TopologyEntry));
>>> + entry->id.id = id.id;
>>> + s390_topology_add_cpu_to_entry(entry, cpu);
>>> +
>>> + return entry;
>>> +}
>>> +
>>> +/**
>>> + * s390_topology_from_cpu:
>>> + * @cpu: The S390CPU
>>> + *
>>> + * Initialize the topology id from the CPU environment.
>>> + */
>>> +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
>>> +{
>>> + s390_topology_id topology_id = {0};
>>> +
>>> + topology_id.drawer = cpu->env.drawer_id;
>>> + topology_id.book = cpu->env.book_id;
>>> + topology_id.socket = cpu->env.socket_id;
>>> + topology_id.origin = cpu->env.core_id / 64;
>>> + topology_id.type = S390_TOPOLOGY_CPU_IFL;
>>> + topology_id.dedicated = cpu->env.dedicated;
>>> +
>>> + if (s390_topology.polarity == POLARITY_VERTICAL) {
>>> + /*
>>> + * Vertical polarity with dedicated CPU implies
>>> + * vertical high entitlement.
>>> + */
>>> + if (topology_id.dedicated) {
>>> + topology_id.polarity |= POLARITY_VERTICAL_HIGH;
>>> + } else {
>>> + topology_id.polarity |= cpu->env.entitlement;
>>> + }
>>> + }
>>> +
>>> + return topology_id;
>>> +}
>>> +
>>> +/**
>>> + * s390_topology_insert:
>>> + * @cpu: s390CPU insert.
>>> + *
>>> + * Parse the topology list to find if the entry already
>>> + * exist and add the core in it.
>>> + * If it does not exist, allocate a new entry and insert
>>> + * it in the queue from lower id to greater id.
>>> + */
>>> +static void s390_topology_insert(S390CPU *cpu)
>>> +{
>>> + s390_topology_id id = s390_topology_from_cpu(cpu);
>>> + S390TopologyEntry *entry = NULL;
>>> + S390TopologyEntry *tmp = NULL;
>>> +
>>> + QTAILQ_FOREACH(tmp, &s390_topology.list, next) {
>>> + if (id.id == tmp->id.id) {
>>> + s390_topology_add_cpu_to_entry(tmp, cpu);
>>> + return;
>>> + } else if (id.id < tmp->id.id) {
>>> + entry = s390_topology_new_entry(id, cpu);
>>> + QTAILQ_INSERT_BEFORE(tmp, entry, next);
>>> + return;
>>> + }
>>> + }
>>> +}
>>> +
>>> +/**
>>> + * s390_order_tle:
>>> + *
>>> + * Loop over all CPU and insert it at the right place
>>> + * inside the TLE entry list.
>>> + */
>
> Suggestion:
>
> s390_topology_fill_list_sorted
>
> Fill the S390Topology list with entries according to the order specified
> by the PoP.
OK
>
>>> +static void s390_order_tle(void)
>>> +{
>>> + CPUState *cs;
>>> +
>>> + CPU_FOREACH(cs) {
>>> + s390_topology_insert(S390_CPU(cs));
>>> + }
>>> +}
>>> +
>>> +/**
>>> + * s390_free_tle:
>>> + *
>>> + * Loop over all TLE entries and free them.
>>> + * Keep the sentinelle which is the only one with level5 != 0
>
> s/sentinelle/sentinel/
yes, thx
>
>>> + */
>
> Suggestion:
> s390_topology_empty_list
>
> Clear all entries in the S390Topology list except the sentinel.
>
>>> +static void s390_free_tle(void)
>>> +{
>>> + S390TopologyEntry *entry = NULL;
>>> + S390TopologyEntry *tmp = NULL;
>>> +
>>> + QTAILQ_FOREACH_SAFE(entry, &s390_topology.list, next, tmp) {
>>> + if (!entry->id.level5) {
>>> + QTAILQ_REMOVE(&s390_topology.list, entry, next);
>>> + g_free(entry);
>>> + }
>>> + }
>>> +}
>>> +
>>> +/**
>>> + * insert_stsi_15_1_x:
>>> + * cpu: the CPU doing the call for which we set CC
>>> + * sel2: the selector 2, containing the nested level
>>> + * addr: Guest logical address of the guest SysIB
>>> + * ar: the access register number
>>> + *
>>> + * Reserve a zeroed SysIB, let setup_stsi to fill it and
>>> + * copy the SysIB to the guest memory.
>>> + *
>>> + * In case of overflow set CC(3) and no copy is done.
>
> Suggestion:
>
> Emulate STSI 15.1.x, that is, perform all necessary checks and fill the SYSIB.
> In case the topology description is too long to fit into the SYSIB,
> set CC=3 and abort without writing the SYSIB.
OK, better thanks.
>
>>> + */
>>> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
>>> +{
>>> + SysIB sysib = {0};
>>> + int len;
>>> +
>>> + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
>>> + setcc(cpu, 3);
>>> + return;
>>> + }
>>> +
>>> + s390_order_tle();
>>> +
>>> + len = setup_stsi(&sysib.sysib_151x, sel2);
>>> +
>>> + if (len < 0) {
>
> I stumbled a bit over this, maybe rename len to r.
Why ? it is the length used to fill the length field of the SYSIB.
May be it would be clearer if we give back the length to write and 0 on
error then we have here:
if (!len) {
setcc(cpu, 3);
return;
}
>
>>> + setcc(cpu, 3);
>>> + return;
>>> + }
>>> +
>>> + sysib.sysib_151x.length = cpu_to_be16(len);
>>> + s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, len);
>>> + setcc(cpu, 0);
>>> +
>>> + s390_free_tle();
>>> +}
Thanks for the comments.
If there are only comments and cosmetic changes will I get your RB if I
change them according to your wishes?
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries
2023-02-01 13:20 ` [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
@ 2023-02-06 10:13 ` Thomas Huth
2023-02-06 10:19 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 10:13 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> The maximum nested topology entries is used by the guest to
> know how many nested topology are available on the machine.
>
> Let change the MNEST value from 2 to 4 in the SCLP READ INFO
> structure now that we support books and drawers.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
> ---
> include/hw/s390x/sclp.h | 5 +++--
> hw/s390x/sclp.c | 5 +++++
> 2 files changed, 8 insertions(+), 2 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries
2023-02-06 10:13 ` Thomas Huth
@ 2023-02-06 10:19 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 10:19 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 11:13, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> The maximum nested topology entries is used by the guest to
>> know how many nested topology are available on the machine.
>>
>> Let change the MNEST value from 2 to 4 in the SCLP READ INFO
>> structure now that we support books and drawers.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> Reviewed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
>> ---
>> include/hw/s390x/sclp.h | 5 +++--
>> hw/s390x/sclp.c | 5 +++++
>> 2 files changed, 8 insertions(+), 2 deletions(-)
>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
>
Thanks,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-06 10:06 ` Pierre Morel
@ 2023-02-06 10:32 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-06 10:32 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Mon, 2023-02-06 at 11:06 +0100, Pierre Morel wrote:
>
> On 2/3/23 18:36, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > > On interception of STSI(15.1.x) the System Information Block
> > > > (SYSIB) is built from the list of pre-ordered topology entries.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > ---
> > > > include/hw/s390x/cpu-topology.h | 22 +++
> > > > include/hw/s390x/sclp.h | 1 +
> > > > target/s390x/cpu.h | 72 +++++++
> > > > hw/s390x/cpu-topology.c | 10 +
> > > > target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
> > > > target/s390x/kvm/kvm.c | 5 +-
> > > > target/s390x/kvm/meson.build | 3 +-
> > > > 7 files changed, 446 insertions(+), 2 deletions(-)
> > > > create mode 100644 target/s390x/kvm/cpu_topology.c
> > > >
[...]
> >
> > > > + */
> > > > +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
> > > > +{
> > > > + SysIB sysib = {0};
> > > > + int len;
> > > > +
> > > > + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) {
> > > > + setcc(cpu, 3);
> > > > + return;
> > > > + }
> > > > +
> > > > + s390_order_tle();
> > > > +
> > > > + len = setup_stsi(&sysib.sysib_151x, sel2);
> > > > +
> > > > + if (len < 0) {
> >
> > I stumbled a bit over this, maybe rename len to r.
>
> Why ? it is the length used to fill the length field of the SYSIB.
Well a negative length doesn't make sense, that's what confused me for a bit.
It's the error value of course, I proposed renaming it to the more generic r return value,
to signify that the return value isn't exactly the length.
You're solution of using 0 is fine with me, too.
>
> May be it would be clearer if we give back the length to write and 0 on
> error then we have here:
>
> if (!len) {
> setcc(cpu, 3);
> return;
> }
>
> >
> > > > + setcc(cpu, 3);
> > > > + return;
> > > > + }
> > > > +
> > > > + sysib.sysib_151x.length = cpu_to_be16(len);
> > > > + s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, len);
> > > > + setcc(cpu, 0);
> > > > +
> > > > + s390_free_tle();
> > > > +}
>
> Thanks for the comments.
> If there are only comments and cosmetic changes will I get your RB if I
> change them according to your wishes?
Usually I review the whole series and then comment, this time sent feedback early,
so the review isn't as deep. Probably easiest to give you the R-b for v16.
My impression is that the series is close to final.
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-01 13:20 ` [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
@ 2023-02-06 11:05 ` Thomas Huth
2023-02-06 12:50 ` Pierre Morel
2023-02-06 17:52 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 11:05 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> During a subsystem reset the Topology-Change-Report is cleared
> by the machine.
> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
> bit of the SCA in the case of a subsystem reset.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
...
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index a80a1ebf22..cf63f3dd01 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> }
>
> +/**
> + * s390_topology_reset:
> + *
> + * Generic reset for CPU topology, calls s390_topology_reset()
> + * s390_topology_reset() to reset the kernel Modified Topology
Duplicated s390_topology_reset() in the comment.
> + * change record.
> + */
> +void s390_topology_reset(void)
> +{
> + s390_cpu_topology_reset();
> +}
With the nit fixed:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
2023-02-03 17:36 ` Nina Schoetterl-Glausch
@ 2023-02-06 11:24 ` Thomas Huth
2023-02-06 12:57 ` Pierre Morel
2023-02-09 16:39 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 11:24 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
...
> diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
> index d3ade40a5a..712fd68123 100644
> --- a/include/hw/s390x/sclp.h
> +++ b/include/hw/s390x/sclp.h
> @@ -112,6 +112,7 @@ typedef struct CPUEntry {
> } QEMU_PACKED CPUEntry;
>
> #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
> +#define SCLP_READ_SCP_INFO_MNEST 2
> typedef struct ReadInfo {
> SCCBHeader h;
> uint16_t rnmax;
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index d654267a71..e1f6925856 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -560,6 +560,25 @@ typedef struct SysIB_322 {
> } SysIB_322;
> QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
>
> +#define S390_TOPOLOGY_MAG 6
> +#define S390_TOPOLOGY_MAG6 0
> +#define S390_TOPOLOGY_MAG5 1
> +#define S390_TOPOLOGY_MAG4 2
> +#define S390_TOPOLOGY_MAG3 3
> +#define S390_TOPOLOGY_MAG2 4
> +#define S390_TOPOLOGY_MAG1 5
> +/* Configuration topology */
> +typedef struct SysIB_151x {
> + uint8_t reserved0[2];
> + uint16_t length;
> + uint8_t mag[S390_TOPOLOGY_MAG];
> + uint8_t reserved1;
> + uint8_t mnest;
> + uint32_t reserved2;
> + char tle[];
> +} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
> +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
> +
> typedef union SysIB {
> SysIB_111 sysib_111;
> SysIB_121 sysib_121;
> @@ -567,9 +586,62 @@ typedef union SysIB {
> SysIB_221 sysib_221;
> SysIB_222 sysib_222;
> SysIB_322 sysib_322;
> + SysIB_151x sysib_151x;
> } SysIB;
> QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
>
> +/*
> + * CPU Topology List provided by STSI with fc=15 provides a list
> + * of two different Topology List Entries (TLE) types to specify
> + * the topology hierarchy.
> + *
> + * - Container Topology List Entry
> + * Defines a container to contain other Topology List Entries
> + * of any type, nested containers or CPU.
> + * - CPU Topology List Entry
> + * Specifies the CPUs position, type, entitlement and polarization
> + * of the CPUs contained in the last Container TLE.
> + *
> + * There can be theoretically up to five levels of containers, QEMU
> + * uses only three levels, the drawer's, book's and socket's level.
> + *
> + * A container of with a nesting level (NL) greater than 1 can only
> + * contain another container of nesting level NL-1.
> + *
> + * A container of nesting level 1 (socket), contains as many CPU TLE
> + * as needed to describe the position and qualities of all CPUs inside
> + * the container.
> + * The qualities of a CPU are polarization, entitlement and type.
> + *
> + * The CPU TLE defines the position of the CPUs of identical qualities
> + * using a 64bits mask which first bit has its offset defined by
> + * the CPU address orgin field of the CPU TLE like in:
> + * CPU address = origin * 64 + bit position within the mask
> + *
> + */
> +/* Container type Topology List Entry */
> +typedef struct SysIBTl_container {
> + uint8_t nl;
> + uint8_t reserved[6];
> + uint8_t id;
> +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
> +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
> +
> +/* CPU type Topology List Entry */
> +typedef struct SysIBTl_cpu {
> + uint8_t nl;
> + uint8_t reserved0[3];
> +#define SYSIB_TLE_POLARITY_MASK 0x03
> +#define SYSIB_TLE_DEDICATED 0x04
> + uint8_t entitlement;
> + uint8_t type;
> + uint16_t origin;
> + uint64_t mask;
> +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
> +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
> +
> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
__u64 is not a portable type (e.g. fails when doing "make
vm-build-openbsd"), please replace with uint64_t.
Thanks,
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-01 13:20 ` [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Pierre Morel
@ 2023-02-06 11:38 ` Thomas Huth
2023-02-06 13:02 ` Pierre Morel
2023-02-06 18:34 ` Nina Schoetterl-Glausch
1 sibling, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 11:38 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> When the host supports the CPU topology facility, the PTF
> instruction with function code 2 is interpreted by the SIE,
> provided that the userland hypervizor activates the interpretation
s/hypervizor/hypervisor/
> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>
> The PTF instructions with function code 0 and 1 are intercepted
> and must be emulated by the userland hypervizor.
dito
> During RESET all CPU of the configuration are placed in
> horizontal polarity.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
...
> /**
> * s390_topology_reset:
> *
> * Generic reset for CPU topology, calls s390_topology_reset()
> * s390_topology_reset() to reset the kernel Modified Topology
> * change record.
> + * Then set global and all CPUs polarity to POLARITY_HORIZONTAL.
You describe here already what's going to happen...
> */
> void s390_topology_reset(void)
> {
> s390_cpu_topology_reset();
> + /* Set global polarity to POLARITY_HORIZONTAL */
... then here again ...
> + s390_topology.polarity = POLARITY_HORIZONTAL;
... and the code is (fortunately) also very self-exaplaining...
> + /* Set all CPU polarity to POLARITY_HORIZONTAL */
> + s390_topology_set_cpus_polarity(POLARITY_HORIZONTAL);
... so I'd rather drop the two comments within the function body.
> }
(rest of the patch looks fine to me)
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology
2023-02-01 13:20 ` [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology Pierre Morel
@ 2023-02-06 11:57 ` Thomas Huth
2023-02-06 13:19 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 11:57 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
> activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
> the topology facility in the host CPU model for the guest
> in the case the topology is available in QEMU and in KVM.
>
> The feature is disabled by default and fenced for SE
> (secure execution).
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> hw/s390x/cpu-topology.c | 2 +-
> target/s390x/cpu_models.c | 1 +
> target/s390x/kvm/kvm.c | 12 ++++++++++++
> 3 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 1028bf4476..c33378577b 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -55,7 +55,7 @@ int s390_socket_nb(S390CPU *cpu)
> */
> bool s390_has_topology(void)
> {
> - return false;
> + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
> }
>
> /**
> diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
> index 065ec6d66c..aca2c5c96b 100644
> --- a/target/s390x/cpu_models.c
> +++ b/target/s390x/cpu_models.c
> @@ -254,6 +254,7 @@ bool s390_has_feat(S390Feat feat)
> case S390_FEAT_SIE_CMMA:
> case S390_FEAT_SIE_PFMFI:
> case S390_FEAT_SIE_IBS:
> + case S390_FEAT_CONFIGURATION_TOPOLOGY:
> return false;
> break;
> default:
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index fb63be41b7..808e35a7bd 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -2470,6 +2470,18 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
> set_bit(S390_FEAT_UNPACK, model->features);
> }
>
> + /*
> + * If we have kernel support for CPU Topology indicate the
> + * configuration-topology facility.
> + */
> + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
> + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) {
> + error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY");
> + return;
> + }
> + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
> + }
Not sure, but for the other capabilities, the kvm_vm_enable_cap() is rather
done in kvm_arch_init() instead ... likely that it is properly available in
case you don't run with the "host" cpu model? So should the
kvm_vm_enable_cap(KVM_CAP_S390_CPU_TOPOLOGY) also be moved there (but of
course keep the set_bit() here in kvm_s390_get_host_cpu_model())?
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
@ 2023-02-06 12:21 ` Thomas Huth
2023-02-06 14:03 ` Pierre Morel
2023-02-07 14:59 ` Pierre Morel
2023-02-08 18:40 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 12:21 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> The modification of the CPU attributes are done through a monitor
> command.
>
> It allows to move the core inside the topology tree to optimise
I'm not a native speaker, but "optimize" is more common, I think?
> the cache usage in the case the host's hypervisor previously
> moved the CPU.
>
> The same command allows to modify the CPU attributes modifiers
> like polarization entitlement and the dedicated attribute to notify
> the guest if the host admin modified scheduling or dedication of a vCPU.
>
> With this knowledge the guest has the possibility to optimize the
> usage of the vCPUs.
>
> The command is made experimental for the moment.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine-target.json | 29 +++++++++++++
> include/monitor/hmp.h | 1 +
> hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
> hmp-commands.hx | 16 ++++++++
> 4 files changed, 134 insertions(+)
>
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index 2e267fa458..58df0f5061 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -342,3 +342,32 @@
> 'TARGET_S390X',
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64' ] } }
> +
> +##
> +# @x-set-cpu-topology:
> +#
> +# @core: the vCPU ID to be moved
> +# @socket: the destination socket where to move the vCPU
> +# @book: the destination book where to move the vCPU
> +# @drawer: the destination drawer where to move the vCPU
> +# @polarity: optional polarity, default is last polarity set by the guest
Hmm, below you do something like that:
if (!has_polarity) {
polarity = POLARITY_VERTICAL_MEDIUM;
}
... so it rather seems like medium polarity is the default? ... that looks
weird. I think you should maybe rather pass the has_polarity and
has_dedication flags to the s390_change_topology() function and only change
the value if they have really been provided?
> +# @dedicated: optional, if the vCPU is dedicated to a real CPU
> +#
> +# Modifies the topology by moving the CPU inside the topology
> +# tree or by changing a modifier attribute of a CPU.
> +#
> +# Returns: Nothing on success, the reason on failure.
> +#
> +# Since: <next qemu stable release, eg. 1.0>
Please replace with "Since: 8.0" ... I hope we'll get it merged during this
cycle!
> +{ 'command': 'x-set-cpu-topology',
> + 'data': {
> + 'core': 'int',
> + 'socket': 'int',
> + 'book': 'int',
> + 'drawer': 'int',
> + '*polarity': 'int',
> + '*dedicated': 'bool'
> + },
> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> +}
> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
> index 1b3bdcb446..12827479cf 100644
> --- a/include/monitor/hmp.h
> +++ b/include/monitor/hmp.h
> @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
> HumanReadableText *(*qmp_handler)(Error **));
> void hmp_info_stats(Monitor *mon, const QDict *qdict);
> void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
>
> #endif
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index c33378577b..6c50050991 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -18,6 +18,10 @@
> #include "target/s390x/cpu.h"
> #include "hw/s390x/s390-virtio-ccw.h"
> #include "hw/s390x/cpu-topology.h"
> +#include "qapi/qapi-commands-machine-target.h"
> +#include "qapi/qmp/qdict.h"
> +#include "monitor/hmp.h"
> +#include "monitor/monitor.h"
>
> /*
> * s390_topology is used to keep the topology information.
> @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> /* topology tree is reflected in props */
> s390_update_cpu_props(ms, cpu);
> }
> +
> +/*
> + * qmp and hmp implementations
> + */
> +
> +static void s390_change_topology(int64_t core_id, int64_t socket_id,
> + int64_t book_id, int64_t drawer_id,
> + int64_t polarity, bool dedicated,
> + Error **errp)
> +{
> + MachineState *ms = current_machine;
> + S390CPU *cpu;
> + ERRP_GUARD();
> +
> + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
I think you should add a sanity check for core_id being in a valid range ...
otherwise this will cause an access beyond the end of the array if core_id
is too big or negative.
> + if (!cpu) {
> + error_setg(errp, "Core-id %ld does not exist!", core_id);
> + return;
> + }
> +
> + /* Verify the new topology */
> + s390_topology_check(cpu, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* Move the CPU into its new socket */
> + s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true, errp);
> +
> + /* All checks done, report topology in environment */
> + cpu->env.drawer_id = drawer_id;
> + cpu->env.book_id = book_id;
> + cpu->env.socket_id = socket_id;
> + cpu->env.dedicated = dedicated;
> + cpu->env.entitlement = polarity;
As mentioned above, I think dedicated and polarity should only be changed if
they have really been provided?
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +
> + /* Advertise the topology change */
> + s390_cpu_topology_set_modified();
> +}
> +
> +void qmp_x_set_cpu_topology(int64_t core, int64_t socket,
> + int64_t book, int64_t drawer,
> + bool has_polarity, int64_t polarity,
> + bool has_dedicated, bool dedicated,
> + Error **errp)
> +{
> + ERRP_GUARD();
> +
> + if (!s390_has_topology()) {
> + error_setg(errp, "This machine doesn't support topology");
> + return;
> + }
> + if (!has_polarity) {
> + polarity = POLARITY_VERTICAL_MEDIUM;
> + }
> + if (!has_dedicated) {
> + dedicated = false;
> + }
> + s390_change_topology(core, socket, book, drawer, polarity, dedicated, errp);
> +}
> +
> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict)
> +{
> + const int64_t core = qdict_get_int(qdict, "core");
> + const int64_t socket = qdict_get_int(qdict, "socket");
> + const int64_t book = qdict_get_int(qdict, "book");
> + const int64_t drawer = qdict_get_int(qdict, "drawer");
> + bool has_polarity = qdict_haskey(qdict, "polarity");
> + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0);
> + bool has_dedicated = qdict_haskey(qdict, "dedicated");
> + const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false);
> + Error *local_err = NULL;
> +
> + qmp_x_set_cpu_topology(core, socket, book, drawer,
> + has_polarity, polarity,
> + has_dedicated, dedicated,
> + &local_err);
> + if (hmp_handle_error(mon, local_err)) {
> + return;
> + }
> +}
> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index 673e39a697..bb3c908356 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -1815,3 +1815,19 @@ SRST
> Dump the FDT in dtb format to *filename*.
> ERST
> #endif
> +
> +#if defined(TARGET_S390X) && defined(CONFIG_KVM)
> + {
> + .name = "x-set-cpu-topology",
> + .args_type = "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?",
> + .params = "core socket book drawer [polarity] [dedicated]",
> + .help = "Move CPU 'core' to 'socket/book/drawer' "
> + "optionaly modifies polarity and dedication",
optionaly ==> optionally
> + .cmd = hmp_x_set_cpu_topology,
> + },
> +
> +SRST
> +``x-set-cpu-topology`` *core* *socket* *book* *drawer* *polarity* *dedicated*
> + Moves the CPU *core* to *socket* *book* *drawer* with *polarity* *dedicated*.
> +ERST
> +#endif
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
@ 2023-02-06 12:38 ` Thomas Huth
2023-02-06 14:12 ` Pierre Morel
2023-02-06 12:41 ` Thomas Huth
2023-02-07 18:26 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 12:38 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> S390x provides two more topology containers above the sockets,
> books and drawers.
books and drawers are already handled via the entries in
CpuInstanceProperties, so this sentence looks like a wrong leftover now?
I'd suggest talking about "dedication" and "polarity" instead?
> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine.json | 13 ++++++++++---
> hw/core/machine-qmp-cmds.c | 2 ++
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 3036117059..e36c39e258 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -53,11 +53,18 @@
> #
> # Additional information about a virtual S390 CPU
> #
> -# @cpu-state: the virtual CPU's state
> +# @cpu-state: the virtual CPU's state (since 2.12)
> +# @dedicated: the virtual CPU's dedication (since 8.0)
> +# @polarity: the virtual CPU's polarity (since 8.0)
> #
> # Since: 2.12
> ##
> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
> +{ 'struct': 'CpuInfoS390',
> + 'data': { 'cpu-state': 'CpuS390State',
> + 'dedicated': 'bool',
> + 'polarity': 'int'
> + }
> +}
>
> ##
> # @CpuInfoFast:
> @@ -70,7 +77,7 @@
> #
> # @thread-id: ID of the underlying host thread
> #
> -# @props: properties describing to which node/socket/core/thread
> +# @props: properties describing to which node/drawer/book/socket/core/thread
I think this hunk should rather be moved to patch 1 now.
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
2023-02-06 12:38 ` Thomas Huth
@ 2023-02-06 12:41 ` Thomas Huth
2023-02-06 12:49 ` Daniel P. Berrangé
2023-02-07 18:26 ` Nina Schoetterl-Glausch
2 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 12:41 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 01/02/2023 14.20, Pierre Morel wrote:
> S390x provides two more topology containers above the sockets,
> books and drawers.
>
> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine.json | 13 ++++++++++---
> hw/core/machine-qmp-cmds.c | 2 ++
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 3036117059..e36c39e258 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -53,11 +53,18 @@
> #
> # Additional information about a virtual S390 CPU
> #
> -# @cpu-state: the virtual CPU's state
> +# @cpu-state: the virtual CPU's state (since 2.12)
> +# @dedicated: the virtual CPU's dedication (since 8.0)
> +# @polarity: the virtual CPU's polarity (since 8.0)
> #
> # Since: 2.12
> ##
> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
> +{ 'struct': 'CpuInfoS390',
> + 'data': { 'cpu-state': 'CpuS390State',
> + 'dedicated': 'bool',
> + 'polarity': 'int'
I think it would also be better to mark the new fields as optional and only
return them if the guest has the topology enabled, to avoid confusing
clients that were written before this change.
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 12:41 ` Thomas Huth
@ 2023-02-06 12:49 ` Daniel P. Berrangé
2023-02-06 13:09 ` Thomas Huth
2023-02-06 14:16 ` Pierre Morel
0 siblings, 2 replies; 76+ messages in thread
From: Daniel P. Berrangé @ 2023-02-06 12:49 UTC (permalink / raw)
To: Thomas Huth
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, cohuck, mst, pbonzini, kvm, ehabkost,
marcel.apfelbaum, eblake, armbru, seiden, nrb, nsg, frankja, clg
On Mon, Feb 06, 2023 at 01:41:44PM +0100, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
> > S390x provides two more topology containers above the sockets,
> > books and drawers.
> >
> > Let's add these CPU attributes to the QAPI command query-cpu-fast.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > ---
> > qapi/machine.json | 13 ++++++++++---
> > hw/core/machine-qmp-cmds.c | 2 ++
> > 2 files changed, 12 insertions(+), 3 deletions(-)
> >
> > diff --git a/qapi/machine.json b/qapi/machine.json
> > index 3036117059..e36c39e258 100644
> > --- a/qapi/machine.json
> > +++ b/qapi/machine.json
> > @@ -53,11 +53,18 @@
> > #
> > # Additional information about a virtual S390 CPU
> > #
> > -# @cpu-state: the virtual CPU's state
> > +# @cpu-state: the virtual CPU's state (since 2.12)
> > +# @dedicated: the virtual CPU's dedication (since 8.0)
> > +# @polarity: the virtual CPU's polarity (since 8.0)
> > #
> > # Since: 2.12
> > ##
> > -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
> > +{ 'struct': 'CpuInfoS390',
> > + 'data': { 'cpu-state': 'CpuS390State',
> > + 'dedicated': 'bool',
> > + 'polarity': 'int'
>
> I think it would also be better to mark the new fields as optional and only
> return them if the guest has the topology enabled, to avoid confusing
> clients that were written before this change.
FWIW, I would say that the general expectation of QMP clients is that
they must *always* expect new fields to appear in dicts that are
returned in QMP replies. We add new fields at will on a frequent basis.
So personally I'd keep life simple and unconditionally report the new
fields.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-06 11:05 ` Thomas Huth
@ 2023-02-06 12:50 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 12:50 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 12:05, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> During a subsystem reset the Topology-Change-Report is cleared
>> by the machine.
>> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
>> bit of the SCA in the case of a subsystem reset.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
> ...
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index a80a1ebf22..cf63f3dd01 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>> }
>> +/**
>> + * s390_topology_reset:
>> + *
>> + * Generic reset for CPU topology, calls s390_topology_reset()
>> + * s390_topology_reset() to reset the kernel Modified Topology
>
> Duplicated s390_topology_reset() in the comment.
right, thx.
>
>> + * change record.
>> + */
>> +void s390_topology_reset(void)
>> +{
>> + s390_cpu_topology_reset();
>> +}
>
> With the nit fixed:
> Reviewed-by: Thomas Huth <thuth@redhat.com>
>
Thanks!
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-06 11:24 ` Thomas Huth
@ 2023-02-06 12:57 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 12:57 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 12:24, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> On interception of STSI(15.1.x) the System Information Block
>> (SYSIB) is built from the list of pre-ordered topology entries.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
> ...
...
>> --- a/target/s390x/cpu.h
>> +++ b/target/s390x/cpu.h
...
>> +
>> +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
>
> __u64 is not a portable type (e.g. fails when doing "make
> vm-build-openbsd"), please replace with uint64_t.
done.
Thanks
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-06 11:38 ` Thomas Huth
@ 2023-02-06 13:02 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 13:02 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 12:38, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> When the host supports the CPU topology facility, the PTF
>> instruction with function code 2 is interpreted by the SIE,
>> provided that the userland hypervizor activates the interpretation
>
> s/hypervizor/hypervisor/
grr, I was pretty sure you already said that to me and I did change
it... seems I did not.
done now.
>
>> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>>
>> The PTF instructions with function code 0 and 1 are intercepted
>> and must be emulated by the userland hypervizor.
>
> dito
yes, thx.
>
>> During RESET all CPU of the configuration are placed in
>> horizontal polarity.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
> ...
>> /**
>> * s390_topology_reset:
>> *
>> * Generic reset for CPU topology, calls s390_topology_reset()
>> * s390_topology_reset() to reset the kernel Modified Topology
>> * change record.
>> + * Then set global and all CPUs polarity to POLARITY_HORIZONTAL.
>
> You describe here already what's going to happen...
>
>> */
>> void s390_topology_reset(void)
>> {
>> s390_cpu_topology_reset();
>> + /* Set global polarity to POLARITY_HORIZONTAL */
>
> ... then here again ...
>
>> + s390_topology.polarity = POLARITY_HORIZONTAL;
>
> ... and the code is (fortunately) also very self-exaplaining...
>
>> + /* Set all CPU polarity to POLARITY_HORIZONTAL */
>> + s390_topology_set_cpus_polarity(POLARITY_HORIZONTAL);
>
> ... so I'd rather drop the two comments within the function body.
OK
>
>> }
>
> (rest of the patch looks fine to me)
>
> Thomas
>
Thanks.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 12:49 ` Daniel P. Berrangé
@ 2023-02-06 13:09 ` Thomas Huth
2023-02-06 14:50 ` Daniel P. Berrangé
2023-02-06 14:16 ` Pierre Morel
1 sibling, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2023-02-06 13:09 UTC (permalink / raw)
To: Daniel P. Berrangé, armbru, Michael Roth, Pierre Morel
Cc: qemu-s390x, qemu-devel, borntraeger, pasic, richard.henderson,
david, cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum,
eblake, seiden, nrb, nsg, frankja, clg
On 06/02/2023 13.49, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 01:41:44PM +0100, Thomas Huth wrote:
>> On 01/02/2023 14.20, Pierre Morel wrote:
>>> S390x provides two more topology containers above the sockets,
>>> books and drawers.
>>>
>>> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> ---
>>> qapi/machine.json | 13 ++++++++++---
>>> hw/core/machine-qmp-cmds.c | 2 ++
>>> 2 files changed, 12 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/qapi/machine.json b/qapi/machine.json
>>> index 3036117059..e36c39e258 100644
>>> --- a/qapi/machine.json
>>> +++ b/qapi/machine.json
>>> @@ -53,11 +53,18 @@
>>> #
>>> # Additional information about a virtual S390 CPU
>>> #
>>> -# @cpu-state: the virtual CPU's state
>>> +# @cpu-state: the virtual CPU's state (since 2.12)
>>> +# @dedicated: the virtual CPU's dedication (since 8.0)
>>> +# @polarity: the virtual CPU's polarity (since 8.0)
>>> #
>>> # Since: 2.12
>>> ##
>>> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
>>> +{ 'struct': 'CpuInfoS390',
>>> + 'data': { 'cpu-state': 'CpuS390State',
>>> + 'dedicated': 'bool',
>>> + 'polarity': 'int'
>>
>> I think it would also be better to mark the new fields as optional and only
>> return them if the guest has the topology enabled, to avoid confusing
>> clients that were written before this change.
>
> FWIW, I would say that the general expectation of QMP clients is that
> they must *always* expect new fields to appear in dicts that are
> returned in QMP replies. We add new fields at will on a frequent basis.
Did we change our policy here? I slightly remember I've been told
differently in the past ... but I can't recall where this was, it's
certainly been a while.
So a question to the QAPI maintainers: What's the preferred handling for new
fields nowadays in such situations?
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology
2023-02-06 11:57 ` Thomas Huth
@ 2023-02-06 13:19 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 13:19 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 12:57, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
>> activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
>> the topology facility in the host CPU model for the guest
>> in the case the topology is available in QEMU and in KVM.
>>
>> The feature is disabled by default and fenced for SE
>> (secure execution).
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> hw/s390x/cpu-topology.c | 2 +-
>> target/s390x/cpu_models.c | 1 +
>> target/s390x/kvm/kvm.c | 12 ++++++++++++
>> 3 files changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index 1028bf4476..c33378577b 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -55,7 +55,7 @@ int s390_socket_nb(S390CPU *cpu)
>> */
>> bool s390_has_topology(void)
>> {
>> - return false;
>> + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
>> }
>> /**
>> diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
>> index 065ec6d66c..aca2c5c96b 100644
>> --- a/target/s390x/cpu_models.c
>> +++ b/target/s390x/cpu_models.c
>> @@ -254,6 +254,7 @@ bool s390_has_feat(S390Feat feat)
>> case S390_FEAT_SIE_CMMA:
>> case S390_FEAT_SIE_PFMFI:
>> case S390_FEAT_SIE_IBS:
>> + case S390_FEAT_CONFIGURATION_TOPOLOGY:
>> return false;
>> break;
>> default:
>> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
>> index fb63be41b7..808e35a7bd 100644
>> --- a/target/s390x/kvm/kvm.c
>> +++ b/target/s390x/kvm/kvm.c
>> @@ -2470,6 +2470,18 @@ void kvm_s390_get_host_cpu_model(S390CPUModel
>> *model, Error **errp)
>> set_bit(S390_FEAT_UNPACK, model->features);
>> }
>> + /*
>> + * If we have kernel support for CPU Topology indicate the
>> + * configuration-topology facility.
>> + */
>> + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
>> + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY,
>> 0) < 0) {
>> + error_setg(errp, "KVM: Error enabling
>> KVM_CAP_S390_CPU_TOPOLOGY");
>> + return;
>> + }
>> + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
>> + }
>
> Not sure, but for the other capabilities, the kvm_vm_enable_cap() is
> rather done in kvm_arch_init() instead ... likely that it is properly
> available in case you don't run with the "host" cpu model? So should the
> kvm_vm_enable_cap(KVM_CAP_S390_CPU_TOPOLOGY) also be moved there (but of
> course keep the set_bit() here in kvm_s390_get_host_cpu_model())?
I think you are right, I do this.
Thanks.
regards,
PIerre
>
> Thomas
>
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-06 12:21 ` Thomas Huth
@ 2023-02-06 14:03 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 14:03 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 13:21, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> The modification of the CPU attributes are done through a monitor
>> command.
>>
>> It allows to move the core inside the topology tree to optimise
>
> I'm not a native speaker, but "optimize" is more common, I think?
Yes. I will stick on it.
>
>> the cache usage in the case the host's hypervisor previously
>> moved the CPU.
>>
>> The same command allows to modify the CPU attributes modifiers
>> like polarization entitlement and the dedicated attribute to notify
>> the guest if the host admin modified scheduling or dedication of a vCPU.
>>
>> With this knowledge the guest has the possibility to optimize the
>> usage of the vCPUs.
>>
>> The command is made experimental for the moment.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine-target.json | 29 +++++++++++++
>> include/monitor/hmp.h | 1 +
>> hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
>> hmp-commands.hx | 16 ++++++++
>> 4 files changed, 134 insertions(+)
>>
>> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
>> index 2e267fa458..58df0f5061 100644
>> --- a/qapi/machine-target.json
>> +++ b/qapi/machine-target.json
>> @@ -342,3 +342,32 @@
>> 'TARGET_S390X',
>> 'TARGET_MIPS',
>> 'TARGET_LOONGARCH64' ] } }
>> +
>> +##
>> +# @x-set-cpu-topology:
>> +#
>> +# @core: the vCPU ID to be moved
>> +# @socket: the destination socket where to move the vCPU
>> +# @book: the destination book where to move the vCPU
>> +# @drawer: the destination drawer where to move the vCPU
>> +# @polarity: optional polarity, default is last polarity set by the
>> guest
>
> Hmm, below you do something like that:
>
> if (!has_polarity) {
> polarity = POLARITY_VERTICAL_MEDIUM;
> } >
> ... so it rather seems like medium polarity is the default? ... that
> looks weird. I think you should maybe rather pass the has_polarity and
> has_dedication flags to the s390_change_topology() function and only
> change the value if they have really been provided?
OK, I will change this incoherence.
>
>> +# @dedicated: optional, if the vCPU is dedicated to a real CPU
>> +#
>> +# Modifies the topology by moving the CPU inside the topology
>> +# tree or by changing a modifier attribute of a CPU.
>> +#
>> +# Returns: Nothing on success, the reason on failure.
>> +#
>> +# Since: <next qemu stable release, eg. 1.0>
>
> Please replace with "Since: 8.0" ... I hope we'll get it merged during
> this cycle!
OK. :)
>
>> +{ 'command': 'x-set-cpu-topology',
>> + 'data': {
>> + 'core': 'int',
>> + 'socket': 'int',
>> + 'book': 'int',
>> + 'drawer': 'int',
>> + '*polarity': 'int',
>> + '*dedicated': 'bool'
>> + },
>> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
>> +}
>> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
>> index 1b3bdcb446..12827479cf 100644
>> --- a/include/monitor/hmp.h
>> +++ b/include/monitor/hmp.h
>> @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
>> HumanReadableText
>> *(*qmp_handler)(Error **));
>> void hmp_info_stats(Monitor *mon, const QDict *qdict);
>> void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
>> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
>> #endif
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index c33378577b..6c50050991 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -18,6 +18,10 @@
>> #include "target/s390x/cpu.h"
>> #include "hw/s390x/s390-virtio-ccw.h"
>> #include "hw/s390x/cpu-topology.h"
>> +#include "qapi/qapi-commands-machine-target.h"
>> +#include "qapi/qmp/qdict.h"
>> +#include "monitor/hmp.h"
>> +#include "monitor/monitor.h"
>> /*
>> * s390_topology is used to keep the topology information.
>> @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms,
>> S390CPU *cpu, Error **errp)
>> /* topology tree is reflected in props */
>> s390_update_cpu_props(ms, cpu);
>> }
>> +
>> +/*
>> + * qmp and hmp implementations
>> + */
>> +
>> +static void s390_change_topology(int64_t core_id, int64_t socket_id,
>> + int64_t book_id, int64_t drawer_id,
>> + int64_t polarity, bool dedicated,
>> + Error **errp)
>> +{
>> + MachineState *ms = current_machine;
>> + S390CPU *cpu;
>> + ERRP_GUARD();
>> +
>> + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
>
> I think you should add a sanity check for core_id being in a valid range
> ... otherwise this will cause an access beyond the end of the array if
> core_id is too big or negative.
Right, I will also change the int64_t for the declaration of the core,
socket etc to uint16_t so avoiding a check for negative values.
>
>> + if (!cpu) {
>> + error_setg(errp, "Core-id %ld does not exist!", core_id);
>> + return;
>> + }
>> +
>> + /* Verify the new topology */
>> + s390_topology_check(cpu, errp);
>> + if (*errp) {
>> + return;
>> + }
>> +
>> + /* Move the CPU into its new socket */
>> + s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true,
>> errp);
>> +
>> + /* All checks done, report topology in environment */
>> + cpu->env.drawer_id = drawer_id;
>> + cpu->env.book_id = book_id;
>> + cpu->env.socket_id = socket_id;
>> + cpu->env.dedicated = dedicated;
>> + cpu->env.entitlement = polarity;
>
> As mentioned above, I think dedicated and polarity should only be
> changed if they have really been provided?
right, I do this.
>
>> + /* topology tree is reflected in props */
>> + s390_update_cpu_props(ms, cpu);
>> +
>> + /* Advertise the topology change */
>> + s390_cpu_topology_set_modified();
>> +}
>> +
>> +void qmp_x_set_cpu_topology(int64_t core, int64_t socket,
>> + int64_t book, int64_t drawer,
>> + bool has_polarity, int64_t polarity,
>> + bool has_dedicated, bool dedicated,
>> + Error **errp)
>> +{
>> + ERRP_GUARD();
>> +
>> + if (!s390_has_topology()) {
>> + error_setg(errp, "This machine doesn't support topology");
>> + return;
>> + }
>> + if (!has_polarity) {
>> + polarity = POLARITY_VERTICAL_MEDIUM;
>> + }
>> + if (!has_dedicated) {
>> + dedicated = false;
>> + }
>> + s390_change_topology(core, socket, book, drawer, polarity,
>> dedicated, errp);
>> +}
>> +
>> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict)
>> +{
>> + const int64_t core = qdict_get_int(qdict, "core");
>> + const int64_t socket = qdict_get_int(qdict, "socket");
>> + const int64_t book = qdict_get_int(qdict, "book");
>> + const int64_t drawer = qdict_get_int(qdict, "drawer");
>> + bool has_polarity = qdict_haskey(qdict, "polarity");
>> + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0);
>> + bool has_dedicated = qdict_haskey(qdict, "dedicated");
>> + const bool dedicated = qdict_get_try_bool(qdict, "dedicated",
>> false);
>> + Error *local_err = NULL;
>> +
>> + qmp_x_set_cpu_topology(core, socket, book, drawer,
>> + has_polarity, polarity,
>> + has_dedicated, dedicated,
>> + &local_err);
>> + if (hmp_handle_error(mon, local_err)) {
>> + return;
>> + }
>> +}
>> diff --git a/hmp-commands.hx b/hmp-commands.hx
>> index 673e39a697..bb3c908356 100644
>> --- a/hmp-commands.hx
>> +++ b/hmp-commands.hx
>> @@ -1815,3 +1815,19 @@ SRST
>> Dump the FDT in dtb format to *filename*.
>> ERST
>> #endif
>> +
>> +#if defined(TARGET_S390X) && defined(CONFIG_KVM)
>> + {
>> + .name = "x-set-cpu-topology",
>> + .args_type =
>> "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?",
>> + .params = "core socket book drawer [polarity] [dedicated]",
>> + .help = "Move CPU 'core' to 'socket/book/drawer' "
>> + "optionaly modifies polarity and dedication",
>
> optionaly ==> optionally
OK
>
>> + .cmd = hmp_x_set_cpu_topology,
>> + },
>> +
>> +SRST
>> +``x-set-cpu-topology`` *core* *socket* *book* *drawer* *polarity*
>> *dedicated*
>> + Moves the CPU *core* to *socket* *book* *drawer* with *polarity*
>> *dedicated*.
>> +ERST
>> +#endif
>
> Thomas
>
Thanks,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 12:38 ` Thomas Huth
@ 2023-02-06 14:12 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 14:12 UTC (permalink / raw)
To: Thomas Huth, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, cohuck,
mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, nsg, frankja, berrange, clg
On 2/6/23 13:38, Thomas Huth wrote:
> On 01/02/2023 14.20, Pierre Morel wrote:
>> S390x provides two more topology containers above the sockets,
>> books and drawers.
>
> books and drawers are already handled via the entries in
> CpuInstanceProperties, so this sentence looks like a wrong leftover now?
yes it is
>
> I'd suggest talking about "dedication" and "polarity" instead?
OK.
>
>> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine.json | 13 ++++++++++---
>> hw/core/machine-qmp-cmds.c | 2 ++
>> 2 files changed, 12 insertions(+), 3 deletions(-)
>>
>> diff --git a/qapi/machine.json b/qapi/machine.json
>> index 3036117059..e36c39e258 100644
>> --- a/qapi/machine.json
>> +++ b/qapi/machine.json
>> @@ -53,11 +53,18 @@
>> #
>> # Additional information about a virtual S390 CPU
>> #
>> -# @cpu-state: the virtual CPU's state
>> +# @cpu-state: the virtual CPU's state (since 2.12)
>> +# @dedicated: the virtual CPU's dedication (since 8.0)
>> +# @polarity: the virtual CPU's polarity (since 8.0)
>> #
>> # Since: 2.12
>> ##
>> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
>> +{ 'struct': 'CpuInfoS390',
>> + 'data': { 'cpu-state': 'CpuS390State',
>> + 'dedicated': 'bool',
>> + 'polarity': 'int'
>> + }
>> +}
>> ##
>> # @CpuInfoFast:
>> @@ -70,7 +77,7 @@
>> #
>> # @thread-id: ID of the underlying host thread
>> #
>> -# @props: properties describing to which node/socket/core/thread
>> +# @props: properties describing to which
>> node/drawer/book/socket/core/thread
>
> I think this hunk should rather be moved to patch 1 now.
yes it should.
Thanks.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 12:49 ` Daniel P. Berrangé
2023-02-06 13:09 ` Thomas Huth
@ 2023-02-06 14:16 ` Pierre Morel
1 sibling, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-06 14:16 UTC (permalink / raw)
To: Daniel P. Berrangé, Thomas Huth
Cc: qemu-s390x, qemu-devel, borntraeger, pasic, richard.henderson,
david, cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum,
eblake, armbru, seiden, nrb, nsg, frankja, clg
On 2/6/23 13:49, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 01:41:44PM +0100, Thomas Huth wrote:
>> On 01/02/2023 14.20, Pierre Morel wrote:
>>> S390x provides two more topology containers above the sockets,
>>> books and drawers.
>>>
>>> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>>>
>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>> ---
>>> qapi/machine.json | 13 ++++++++++---
>>> hw/core/machine-qmp-cmds.c | 2 ++
>>> 2 files changed, 12 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/qapi/machine.json b/qapi/machine.json
>>> index 3036117059..e36c39e258 100644
>>> --- a/qapi/machine.json
>>> +++ b/qapi/machine.json
>>> @@ -53,11 +53,18 @@
>>> #
>>> # Additional information about a virtual S390 CPU
>>> #
>>> -# @cpu-state: the virtual CPU's state
>>> +# @cpu-state: the virtual CPU's state (since 2.12)
>>> +# @dedicated: the virtual CPU's dedication (since 8.0)
>>> +# @polarity: the virtual CPU's polarity (since 8.0)
>>> #
>>> # Since: 2.12
>>> ##
>>> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
>>> +{ 'struct': 'CpuInfoS390',
>>> + 'data': { 'cpu-state': 'CpuS390State',
>>> + 'dedicated': 'bool',
>>> + 'polarity': 'int'
>>
>> I think it would also be better to mark the new fields as optional and only
>> return them if the guest has the topology enabled, to avoid confusing
>> clients that were written before this change.
>
> FWIW, I would say that the general expectation of QMP clients is that
> they must *always* expect new fields to appear in dicts that are
> returned in QMP replies. We add new fields at will on a frequent basis.
>
> So personally I'd keep life simple and unconditionally report the new
> fields.
>
> With regards,
> Daniel
OK, thanks both of you.
I will then keep the simple way.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 13:09 ` Thomas Huth
@ 2023-02-06 14:50 ` Daniel P. Berrangé
2023-02-07 10:10 ` Thomas Huth
0 siblings, 1 reply; 76+ messages in thread
From: Daniel P. Berrangé @ 2023-02-06 14:50 UTC (permalink / raw)
To: Thomas Huth
Cc: armbru, Michael Roth, Pierre Morel, qemu-s390x, qemu-devel,
borntraeger, pasic, richard.henderson, david, cohuck, mst,
pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, seiden, nrb,
nsg, frankja, clg
On Mon, Feb 06, 2023 at 02:09:57PM +0100, Thomas Huth wrote:
> On 06/02/2023 13.49, Daniel P. Berrangé wrote:
> > On Mon, Feb 06, 2023 at 01:41:44PM +0100, Thomas Huth wrote:
> > > On 01/02/2023 14.20, Pierre Morel wrote:
> > > > S390x provides two more topology containers above the sockets,
> > > > books and drawers.
> > > >
> > > > Let's add these CPU attributes to the QAPI command query-cpu-fast.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > ---
> > > > qapi/machine.json | 13 ++++++++++---
> > > > hw/core/machine-qmp-cmds.c | 2 ++
> > > > 2 files changed, 12 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/qapi/machine.json b/qapi/machine.json
> > > > index 3036117059..e36c39e258 100644
> > > > --- a/qapi/machine.json
> > > > +++ b/qapi/machine.json
> > > > @@ -53,11 +53,18 @@
> > > > #
> > > > # Additional information about a virtual S390 CPU
> > > > #
> > > > -# @cpu-state: the virtual CPU's state
> > > > +# @cpu-state: the virtual CPU's state (since 2.12)
> > > > +# @dedicated: the virtual CPU's dedication (since 8.0)
> > > > +# @polarity: the virtual CPU's polarity (since 8.0)
> > > > #
> > > > # Since: 2.12
> > > > ##
> > > > -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
> > > > +{ 'struct': 'CpuInfoS390',
> > > > + 'data': { 'cpu-state': 'CpuS390State',
> > > > + 'dedicated': 'bool',
> > > > + 'polarity': 'int'
> > >
> > > I think it would also be better to mark the new fields as optional and only
> > > return them if the guest has the topology enabled, to avoid confusing
> > > clients that were written before this change.
> >
> > FWIW, I would say that the general expectation of QMP clients is that
> > they must *always* expect new fields to appear in dicts that are
> > returned in QMP replies. We add new fields at will on a frequent basis.
>
> Did we change our policy here? I slightly remember I've been told
> differently in the past ... but I can't recall where this was, it's
> certainly been a while.
>
> So a question to the QAPI maintainers: What's the preferred handling for new
> fields nowadays in such situations?
I think you're mixing it up with policy for adding new fields to *input*
parameters. You can't add a new mandatory field to input params of a
command because no existing client will be sending that. Only optional
params are viable, without a deprecation cycle. For output parameters
such as this case, there's no compatibilty problem.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-01 13:20 ` [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
2023-02-06 11:05 ` Thomas Huth
@ 2023-02-06 17:52 ` Nina Schoetterl-Glausch
2023-02-07 9:24 ` Pierre Morel
1 sibling, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-06 17:52 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> During a subsystem reset the Topology-Change-Report is cleared
> by the machine.
> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
> bit of the SCA in the case of a subsystem reset.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> include/hw/s390x/cpu-topology.h | 1 +
> target/s390x/cpu.h | 1 +
> target/s390x/kvm/kvm_s390x.h | 1 +
> hw/s390x/cpu-topology.c | 12 ++++++++++++
> hw/s390x/s390-virtio-ccw.c | 3 +++
> target/s390x/cpu-sysemu.c | 13 +++++++++++++
> target/s390x/kvm/kvm.c | 17 +++++++++++++++++
> 7 files changed, 48 insertions(+)
>
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> index 1ae7e7c5e3..60e0b9fbfa 100644
> --- a/include/hw/s390x/cpu-topology.h
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
>
> extern S390Topology s390_topology;
> int s390_socket_nb(S390CPU *cpu);
> +void s390_topology_reset(void);
>
> #endif
> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> index e1f6925856..848314d2a9 100644
> --- a/target/s390x/cpu.h
> +++ b/target/s390x/cpu.h
> @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
> QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
>
> void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
> +void s390_cpu_topology_reset(void);
How about you call this s390_cpu_topology_reset_modified, so it's symmetric
with the function you define in the next patch. You could also drop the "cpu"
from the name.
Or maybe even better, you only define a function for setting the modified state,
but make it take a bool argument. This way you also get rid of some code duplication
and it wouldn't harm readability IMO.
>
> /* MMU defines */
> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
> diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
> index f9785564d0..649dae5948 100644
> --- a/target/s390x/kvm/kvm_s390x.h
> +++ b/target/s390x/kvm/kvm_s390x.h
> @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
> void kvm_s390_restart_interrupt(S390CPU *cpu);
> void kvm_s390_stop_interrupt(S390CPU *cpu);
> void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
> +int kvm_s390_topology_set_mtcr(uint64_t attr);
>
> #endif /* KVM_S390X_H */
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index a80a1ebf22..cf63f3dd01 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> }
>
> +/**
> + * s390_topology_reset:
> + *
> + * Generic reset for CPU topology, calls s390_topology_reset()
> + * s390_topology_reset() to reset the kernel Modified Topology
> + * change record.
> + */
> +void s390_topology_reset(void)
> +{
I'm wondering if you shouldn't move the reset changes you do in the next patch
into this one. I don't see what they have to do with PTF emulation.
> + s390_cpu_topology_reset();
> +}
> +
> /**
> * s390_topology_cpu_default:
> * @cpu: pointer to a S390CPU
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 9bc51a83f4..30fdfe41fa 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -122,6 +122,9 @@ static void subsystem_reset(void)
> device_cold_reset(dev);
> }
> }
> + if (s390_has_topology()) {
> + s390_topology_reset();
> + }
> }
>
> static int virtio_ccw_hcall_notify(const uint64_t *args)
> diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
> index 948e4bd3e0..e27864c5f5 100644
> --- a/target/s390x/cpu-sysemu.c
> +++ b/target/s390x/cpu-sysemu.c
> @@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)
> kvm_s390_set_diag318(cs, arg.host_ulong);
> }
> }
> +
> +void s390_cpu_topology_reset(void)
> +{
> + int ret;
> +
> + if (kvm_enabled()) {
> + ret = kvm_s390_topology_set_mtcr(0);
> + if (ret) {
> + error_report("Failed to set Modified Topology Change Report: %s",
> + strerror(-ret));
> + }
> + }
> +}
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index 5ea358cbb0..bc953151ce 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void)
> return cap_zpci_op;
> }
>
> +int kvm_s390_topology_set_mtcr(uint64_t attr)
> +{
> + struct kvm_device_attr attribute = {
> + .group = KVM_S390_VM_CPU_TOPOLOGY,
> + .attr = attr,
> + };
> +
> + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
> + return 0;
> + }
> + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
> + return -ENOTSUP;
> + }
> +
> + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute);
> +}
> +
> void kvm_arch_accel_class_init(ObjectClass *oc)
> {
> }
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-01 13:20 ` [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Pierre Morel
2023-02-06 11:38 ` Thomas Huth
@ 2023-02-06 18:34 ` Nina Schoetterl-Glausch
2023-02-07 9:59 ` Pierre Morel
1 sibling, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-06 18:34 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> When the host supports the CPU topology facility, the PTF
> instruction with function code 2 is interpreted by the SIE,
> provided that the userland hypervizor activates the interpretation
> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>
> The PTF instructions with function code 0 and 1 are intercepted
> and must be emulated by the userland hypervizor.
>
> During RESET all CPU of the configuration are placed in
> horizontal polarity.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> include/hw/s390x/s390-virtio-ccw.h | 6 ++
> target/s390x/cpu.h | 1 +
> hw/s390x/cpu-topology.c | 103 +++++++++++++++++++++++++++++
> target/s390x/cpu-sysemu.c | 14 ++++
> target/s390x/kvm/kvm.c | 11 +++
> 5 files changed, 135 insertions(+)
>
[...]
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index cf63f3dd01..1028bf4476 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -85,16 +85,104 @@ static void s390_topology_init(MachineState *ms)
> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> }
>
> +/**
> + * s390_topology_set_cpus_polarity:
> + * @polarity: polarity requested by the caller
> + *
> + * Set all CPU entitlement according to polarity and
> + * dedication.
> + * Default vertical entitlement is POLARITY_VERTICAL_MEDIUM as
> + * it does not require host modification of the CPU provisioning
> + * until the host decide to modify individual CPU provisioning
> + * using QAPI interface.
> + * However a dedicated vCPU will have a POLARITY_VERTICAL_HIGH
> + * entitlement.
> + */
> +static void s390_topology_set_cpus_polarity(int polarity)
Since you set the entitlement field I'd prefer _set_cpus_entitlement or similar.
> +{
> + CPUState *cs;
> +
> + CPU_FOREACH(cs) {
> + if (polarity == POLARITY_HORIZONTAL) {
> + S390_CPU(cs)->env.entitlement = 0;
> + } else if (S390_CPU(cs)->env.dedicated) {
> + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_HIGH;
> + } else {
> + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_MEDIUM;
> + }
> + }
> +}
> +
[...]
>
> /**
> @@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> (smp->books * smp->sockets * smp->cores)) %
> smp->drawers;
> }
Why are the changes below in this patch?
> +
> + /*
> + * Machine polarity is set inside the global s390_topology structure.
> + * In the case the polarity is set as horizontal set the entitlement
> + * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when
> + * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if
> + * the vCPU is dedicated.
> + */
> + if (s390_topology.polarity && !env->entitlement) {
It'd be more readable if you compared against enum values by name.
I don't see why you check s390_topology.polarity. If it is horizontal
then the value of the entitlement doesn't matter at all, so you can set it
to whatever.
All you want to do is enforce dedicated -> VERTICAL_HIGH, right?
So why don't you just add
+ if (cpu->env.dedicated && cpu->env.entitlement != POLARITY_VERTICAL_HIGH) {
+ error_setg(errp, "A dedicated cpu implies high entitlement");
+ return;
+ }
to s390_topology_check?
> + if (env->dedicated) {
> + env->entitlement = POLARITY_VERTICAL_HIGH;
> + } else {
> + env->entitlement = POLARITY_VERTICAL_MEDIUM;
> + }
If it is horizontal, then setting the entitlement is pointless as it will be
reset to medium on PTF.
So the current polarization is vertical and a cpu is being hotplugged,
but setting the entitlement of the cpu being added is also pointless, because
it's determined by the dedication. That seems weird.
> + }
> }
>
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-06 17:52 ` Nina Schoetterl-Glausch
@ 2023-02-07 9:24 ` Pierre Morel
2023-02-07 10:50 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 9:24 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/6/23 18:52, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> During a subsystem reset the Topology-Change-Report is cleared
>> by the machine.
>> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
>> bit of the SCA in the case of a subsystem reset.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> include/hw/s390x/cpu-topology.h | 1 +
>> target/s390x/cpu.h | 1 +
>> target/s390x/kvm/kvm_s390x.h | 1 +
>> hw/s390x/cpu-topology.c | 12 ++++++++++++
>> hw/s390x/s390-virtio-ccw.c | 3 +++
>> target/s390x/cpu-sysemu.c | 13 +++++++++++++
>> target/s390x/kvm/kvm.c | 17 +++++++++++++++++
>> 7 files changed, 48 insertions(+)
>>
>> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
>> index 1ae7e7c5e3..60e0b9fbfa 100644
>> --- a/include/hw/s390x/cpu-topology.h
>> +++ b/include/hw/s390x/cpu-topology.h
>> @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
>>
>> extern S390Topology s390_topology;
>> int s390_socket_nb(S390CPU *cpu);
>> +void s390_topology_reset(void);
>>
>> #endif
>> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
>> index e1f6925856..848314d2a9 100644
>> --- a/target/s390x/cpu.h
>> +++ b/target/s390x/cpu.h
>> @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
>> QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
>>
>> void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
>> +void s390_cpu_topology_reset(void);
>
> How about you call this s390_cpu_topology_reset_modified, so it's symmetric
> with the function you define in the next patch. You could also drop the "cpu"
> from the name.
I am not sure about this, Thomas already gave his R-B on this patch so I
prefer to stay on the original name, unless he says it is a good idea to
change.
Also in cpu-sysemu.c most of the function are tagged with _cpu_
>
> Or maybe even better, you only define a function for setting the modified state,
> but make it take a bool argument. This way you also get rid of some code duplication
> and it wouldn't harm readability IMO.
There is already a single function kvm_s390_topology_set_mtcr(attr) to
set the "modified state"
>
>>
>> /* MMU defines */
>> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
>> diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
>> index f9785564d0..649dae5948 100644
>> --- a/target/s390x/kvm/kvm_s390x.h
>> +++ b/target/s390x/kvm/kvm_s390x.h
>> @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
>> void kvm_s390_restart_interrupt(S390CPU *cpu);
>> void kvm_s390_stop_interrupt(S390CPU *cpu);
>> void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
>> +int kvm_s390_topology_set_mtcr(uint64_t attr);
>>
>> #endif /* KVM_S390X_H */
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index a80a1ebf22..cf63f3dd01 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>> }
>>
>> +/**
>> + * s390_topology_reset:
>> + *
>> + * Generic reset for CPU topology, calls s390_topology_reset()
>> + * s390_topology_reset() to reset the kernel Modified Topology
>> + * change record.
>> + */
>> +void s390_topology_reset(void)
>> +{
>
> I'm wondering if you shouldn't move the reset changes you do in the next patch
> into this one. I don't see what they have to do with PTF emulation.
Here in this patch we do not intercept PTF and we have only an
horizontal polarity.
So we do not need to reset the polarity for all the vCPUs, we only need
it when we have vertical polarity.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-06 18:34 ` Nina Schoetterl-Glausch
@ 2023-02-07 9:59 ` Pierre Morel
2023-02-07 11:27 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 9:59 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/6/23 19:34, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> When the host supports the CPU topology facility, the PTF
>> instruction with function code 2 is interpreted by the SIE,
>> provided that the userland hypervizor activates the interpretation
>> by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
>>
>> The PTF instructions with function code 0 and 1 are intercepted
>> and must be emulated by the userland hypervizor.
>>
>> During RESET all CPU of the configuration are placed in
>> horizontal polarity.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> include/hw/s390x/s390-virtio-ccw.h | 6 ++
>> target/s390x/cpu.h | 1 +
>> hw/s390x/cpu-topology.c | 103 +++++++++++++++++++++++++++++
>> target/s390x/cpu-sysemu.c | 14 ++++
>> target/s390x/kvm/kvm.c | 11 +++
>> 5 files changed, 135 insertions(+)
>>
> [...]
>
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index cf63f3dd01..1028bf4476 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -85,16 +85,104 @@ static void s390_topology_init(MachineState *ms)
>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>> }
>>
>> +/**
>> + * s390_topology_set_cpus_polarity:
>> + * @polarity: polarity requested by the caller
>> + *
>> + * Set all CPU entitlement according to polarity and
>> + * dedication.
>> + * Default vertical entitlement is POLARITY_VERTICAL_MEDIUM as
>> + * it does not require host modification of the CPU provisioning
>> + * until the host decide to modify individual CPU provisioning
>> + * using QAPI interface.
>> + * However a dedicated vCPU will have a POLARITY_VERTICAL_HIGH
>> + * entitlement.
>> + */
>> +static void s390_topology_set_cpus_polarity(int polarity)
>
> Since you set the entitlement field I'd prefer _set_cpus_entitlement or similar.
OK if you prefer.
>
>> +{
>> + CPUState *cs;
>> +
>> + CPU_FOREACH(cs) {
>> + if (polarity == POLARITY_HORIZONTAL) {
>> + S390_CPU(cs)->env.entitlement = 0;
>> + } else if (S390_CPU(cs)->env.dedicated) {
>> + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_HIGH;
>> + } else {
>> + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_MEDIUM;
>> + }
>> + }
>> +}
>> +
> [...]
>>
>> /**
>> @@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
>> (smp->books * smp->sockets * smp->cores)) %
>> smp->drawers;
>> }
>
> Why are the changes below in this patch?
Because before thos patch we have only horizontal polarization.
>
>> +
>> + /*
>> + * Machine polarity is set inside the global s390_topology structure.
>> + * In the case the polarity is set as horizontal set the entitlement
Sorry here an error in the comment should be :
"In the case the polarity is NOT set as horizontal..."
>> + * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when
>> + * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if
>> + * the vCPU is dedicated.
>> + */
>> + if (s390_topology.polarity && !env->entitlement) {
>
> It'd be more readable if you compared against enum values by name.
Right, I will change this to
if (s390_topology.polarity != S390_POLARITY_HORIZONTAL &&
env->entitlement == S390_ENTITLEMENT_UNSET) {
>
> I don't see why you check s390_topology.polarity. If it is horizontal
> then the value of the entitlement doesn't matter at all, so you can set it
> to whatever.
Right, that is why it is done only for vertical polarization (sorry for
the wrong comment)
> All you want to do is enforce dedicated -> VERTICAL_HIGH, right?
> So why don't you just add
>
> + if (cpu->env.dedicated && cpu->env.entitlement != POLARITY_VERTICAL_HIGH) {
> + error_setg(errp, "A dedicated cpu implies high entitlement");
> + return;
> + } >
> to s390_topology_check?
Here it is to set the default in the case the values are not provided.
But where you are right is that I should add a verification to the check
function.
>
>> + if (env->dedicated) {
>> + env->entitlement = POLARITY_VERTICAL_HIGH;
>> + } else {
>> + env->entitlement = POLARITY_VERTICAL_MEDIUM;
>> + }
>
> If it is horizontal, then setting the entitlement is pointless as it will be
> reset to medium on PTF.
That is why the polarity is tested (sorry for the bad comment)
> So the current polarization is vertical and a cpu is being hotplugged,
> but setting the entitlement of the cpu being added is also pointless, because
> it's determined by the dedication. That seems weird.
No it is not determined by the dedication, if there is no dedication the
3 vertical values are possible.
Regards,
Pierre
>
>> + }
>> }
>>
>
> [...]
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-06 14:50 ` Daniel P. Berrangé
@ 2023-02-07 10:10 ` Thomas Huth
0 siblings, 0 replies; 76+ messages in thread
From: Thomas Huth @ 2023-02-07 10:10 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: armbru, Michael Roth, Pierre Morel, qemu-s390x, qemu-devel,
borntraeger, pasic, richard.henderson, david, cohuck, mst,
pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, seiden, nrb,
nsg, frankja, clg
On 06/02/2023 15.50, Daniel P. Berrangé wrote:
> On Mon, Feb 06, 2023 at 02:09:57PM +0100, Thomas Huth wrote:
>> On 06/02/2023 13.49, Daniel P. Berrangé wrote:
>>> On Mon, Feb 06, 2023 at 01:41:44PM +0100, Thomas Huth wrote:
>>>> On 01/02/2023 14.20, Pierre Morel wrote:
>>>>> S390x provides two more topology containers above the sockets,
>>>>> books and drawers.
>>>>>
>>>>> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>>>>>
>>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>>> ---
>>>>> qapi/machine.json | 13 ++++++++++---
>>>>> hw/core/machine-qmp-cmds.c | 2 ++
>>>>> 2 files changed, 12 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/qapi/machine.json b/qapi/machine.json
>>>>> index 3036117059..e36c39e258 100644
>>>>> --- a/qapi/machine.json
>>>>> +++ b/qapi/machine.json
>>>>> @@ -53,11 +53,18 @@
>>>>> #
>>>>> # Additional information about a virtual S390 CPU
>>>>> #
>>>>> -# @cpu-state: the virtual CPU's state
>>>>> +# @cpu-state: the virtual CPU's state (since 2.12)
>>>>> +# @dedicated: the virtual CPU's dedication (since 8.0)
>>>>> +# @polarity: the virtual CPU's polarity (since 8.0)
>>>>> #
>>>>> # Since: 2.12
>>>>> ##
>>>>> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
>>>>> +{ 'struct': 'CpuInfoS390',
>>>>> + 'data': { 'cpu-state': 'CpuS390State',
>>>>> + 'dedicated': 'bool',
>>>>> + 'polarity': 'int'
>>>>
>>>> I think it would also be better to mark the new fields as optional and only
>>>> return them if the guest has the topology enabled, to avoid confusing
>>>> clients that were written before this change.
>>>
>>> FWIW, I would say that the general expectation of QMP clients is that
>>> they must *always* expect new fields to appear in dicts that are
>>> returned in QMP replies. We add new fields at will on a frequent basis.
>>
>> Did we change our policy here? I slightly remember I've been told
>> differently in the past ... but I can't recall where this was, it's
>> certainly been a while.
>>
>> So a question to the QAPI maintainers: What's the preferred handling for new
>> fields nowadays in such situations?
>
> I think you're mixing it up with policy for adding new fields to *input*
> parameters. You can't add a new mandatory field to input params of a
> command because no existing client will be sending that. Only optional
> params are viable, without a deprecation cycle. For output parameters
> such as this case, there's no compatibilty problem.
Ah, right, I likely mixed it up with that. Thanks for the clarification!
Thomas
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-07 9:24 ` Pierre Morel
@ 2023-02-07 10:50 ` Nina Schoetterl-Glausch
2023-02-07 12:19 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-07 10:50 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Tue, 2023-02-07 at 10:24 +0100, Pierre Morel wrote:
>
> On 2/6/23 18:52, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > During a subsystem reset the Topology-Change-Report is cleared
> > > by the machine.
> > > Let's ask KVM to clear the Modified Topology Change Report (MTCR)
> > > bit of the SCA in the case of a subsystem reset.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > include/hw/s390x/cpu-topology.h | 1 +
> > > target/s390x/cpu.h | 1 +
> > > target/s390x/kvm/kvm_s390x.h | 1 +
> > > hw/s390x/cpu-topology.c | 12 ++++++++++++
> > > hw/s390x/s390-virtio-ccw.c | 3 +++
> > > target/s390x/cpu-sysemu.c | 13 +++++++++++++
> > > target/s390x/kvm/kvm.c | 17 +++++++++++++++++
> > > 7 files changed, 48 insertions(+)
> > >
> > > diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> > > index 1ae7e7c5e3..60e0b9fbfa 100644
> > > --- a/include/hw/s390x/cpu-topology.h
> > > +++ b/include/hw/s390x/cpu-topology.h
> > > @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
> > >
> > > extern S390Topology s390_topology;
> > > int s390_socket_nb(S390CPU *cpu);
> > > +void s390_topology_reset(void);
> > >
> > > #endif
> > > diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> > > index e1f6925856..848314d2a9 100644
> > > --- a/target/s390x/cpu.h
> > > +++ b/target/s390x/cpu.h
> > > @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
> > > QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
> > >
> > > void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
> > > +void s390_cpu_topology_reset(void);
> >
> > How about you call this s390_cpu_topology_reset_modified, so it's symmetric
> > with the function you define in the next patch. You could also drop the "cpu"
> > from the name.
>
> I am not sure about this, Thomas already gave his R-B on this patch so I
> prefer to stay on the original name, unless he says it is a good idea to
> change.
> Also in cpu-sysemu.c most of the function are tagged with _cpu_
IMO, renaming a function would be a small enough change to keep an R-b.
>
> >
> > Or maybe even better, you only define a function for setting the modified state,
> > but make it take a bool argument. This way you also get rid of some code duplication
> > and it wouldn't harm readability IMO.
>
> There is already a single function kvm_s390_topology_set_mtcr(attr) to
> set the "modified state"
Yes, but that is for KVM only and doesn't do error handling.
So you need at least one function on top of that. What I'm suggesting is to
only have one function instead of two because it gets rid of some code.
>
> >
> > >
> > > /* MMU defines */
> > > #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
> > > diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
> > > index f9785564d0..649dae5948 100644
> > > --- a/target/s390x/kvm/kvm_s390x.h
> > > +++ b/target/s390x/kvm/kvm_s390x.h
> > > @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
> > > void kvm_s390_restart_interrupt(S390CPU *cpu);
> > > void kvm_s390_stop_interrupt(S390CPU *cpu);
> > > void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
> > > +int kvm_s390_topology_set_mtcr(uint64_t attr);
> > >
> > > #endif /* KVM_S390X_H */
> > > diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> > > index a80a1ebf22..cf63f3dd01 100644
> > > --- a/hw/s390x/cpu-topology.c
> > > +++ b/hw/s390x/cpu-topology.c
> > > @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
> > > QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> > > }
> > >
> > > +/**
> > > + * s390_topology_reset:
> > > + *
> > > + * Generic reset for CPU topology, calls s390_topology_reset()
> > > + * s390_topology_reset() to reset the kernel Modified Topology
> > > + * change record.
> > > + */
> > > +void s390_topology_reset(void)
> > > +{
> >
> > I'm wondering if you shouldn't move the reset changes you do in the next patch
> > into this one. I don't see what they have to do with PTF emulation.
>
> Here in this patch we do not intercept PTF and we have only an
> horizontal polarity.
> So we do not need to reset the polarity for all the vCPUs, we only need
> it when we have vertical polarity.
Well, with the PTF patch you don't get vertical polarity either, because you
only enable the topology with patch 7.
And since it's about resetting, it fits better in this patch IMO.
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-07 9:59 ` Pierre Morel
@ 2023-02-07 11:27 ` Nina Schoetterl-Glausch
2023-02-07 13:03 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-07 11:27 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Tue, 2023-02-07 at 10:59 +0100, Pierre Morel wrote:
>
> On 2/6/23 19:34, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > When the host supports the CPU topology facility, the PTF
> > > instruction with function code 2 is interpreted by the SIE,
> > > provided that the userland hypervizor activates the interpretation
> > > by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
> > >
> > > The PTF instructions with function code 0 and 1 are intercepted
> > > and must be emulated by the userland hypervizor.
> > >
> > > During RESET all CPU of the configuration are placed in
> > > horizontal polarity.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > include/hw/s390x/s390-virtio-ccw.h | 6 ++
> > > target/s390x/cpu.h | 1 +
> > > hw/s390x/cpu-topology.c | 103 +++++++++++++++++++++++++++++
> > > target/s390x/cpu-sysemu.c | 14 ++++
> > > target/s390x/kvm/kvm.c | 11 +++
> > > 5 files changed, 135 insertions(+)
> > >
> > [...]
> >
> > > diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> > > index cf63f3dd01..1028bf4476 100644
> > > --- a/hw/s390x/cpu-topology.c
> > > +++ b/hw/s390x/cpu-topology.c
> > > @@ -85,16 +85,104 @@ static void s390_topology_init(MachineState *ms)
> > > QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> > > }
> > >
> > > +/**
> > > + * s390_topology_set_cpus_polarity:
> > > + * @polarity: polarity requested by the caller
> > > + *
> > > + * Set all CPU entitlement according to polarity and
> > > + * dedication.
> > > + * Default vertical entitlement is POLARITY_VERTICAL_MEDIUM as
> > > + * it does not require host modification of the CPU provisioning
> > > + * until the host decide to modify individual CPU provisioning
> > > + * using QAPI interface.
> > > + * However a dedicated vCPU will have a POLARITY_VERTICAL_HIGH
> > > + * entitlement.
> > > + */
> > > +static void s390_topology_set_cpus_polarity(int polarity)
> >
> > Since you set the entitlement field I'd prefer _set_cpus_entitlement or similar.
>
> OK if you prefer.
>
> >
> > > +{
> > > + CPUState *cs;
> > > +
> > > + CPU_FOREACH(cs) {
> > > + if (polarity == POLARITY_HORIZONTAL) {
> > > + S390_CPU(cs)->env.entitlement = 0;
> > > + } else if (S390_CPU(cs)->env.dedicated) {
> > > + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_HIGH;
> > > + } else {
> > > + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_MEDIUM;
> > > + }
> > > + }
> > > +}
> > > +
> > [...]
> > >
> > > /**
> > > @@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
> > > (smp->books * smp->sockets * smp->cores)) %
> > > smp->drawers;
> > > }
> >
> > Why are the changes below in this patch?
>
> Because before thos patch we have only horizontal polarization.
Not really since you only enable topology in the next patch.
>
> >
> > > +
> > > + /*
> > > + * Machine polarity is set inside the global s390_topology structure.
> > > + * In the case the polarity is set as horizontal set the entitlement
>
> Sorry here an error in the comment should be :
> "In the case the polarity is NOT set as horizontal..."
>
> > > + * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when
> > > + * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if
> > > + * the vCPU is dedicated.
> > > + */
> > > + if (s390_topology.polarity && !env->entitlement) {
> >
> > It'd be more readable if you compared against enum values by name.
>
> Right, I will change this to
>
> if (s390_topology.polarity != S390_POLARITY_HORIZONTAL &&
> env->entitlement == S390_ENTITLEMENT_UNSET) {
>
> >
> > I don't see why you check s390_topology.polarity. If it is horizontal
> > then the value of the entitlement doesn't matter at all, so you can set it
> > to whatever.
>
> Right, that is why it is done only for vertical polarization (sorry for
> the wrong comment)
I'm saying you don't need to check it at all. You adjust the values for
vertical polarization, but you could just always do that since the values
don't matter at all if the polarization is horizontal.
>
> > All you want to do is enforce dedicated -> VERTICAL_HIGH, right?
> > So why don't you just add
> >
> > + if (cpu->env.dedicated && cpu->env.entitlement != POLARITY_VERTICAL_HIGH) {
> > + error_setg(errp, "A dedicated cpu implies high entitlement");
> > + return;
> > + } >
> > to s390_topology_check?
>
> Here it is to set the default in the case the values are not provided.
If no values are provided, they default to dedication=false and entitlement=medium,
as defined by patch 1, which are fine and don't need to be adjusted.
>
> But where you are right is that I should add a verification to the check
> function.
>
> >
> > > + if (env->dedicated) {
> > > + env->entitlement = POLARITY_VERTICAL_HIGH;
> > > + } else {
> > > + env->entitlement = POLARITY_VERTICAL_MEDIUM;
> > > + }
> >
> > If it is horizontal, then setting the entitlement is pointless as it will be
> > reset to medium on PTF.
>
> That is why the polarity is tested (sorry for the bad comment)
I said this because I'm fine with setting it pointlessly.
> > So the current polarization is vertical and a cpu is being hotplugged,
> > but setting the entitlement of the cpu being added is also pointless, because
> > it's determined by the dedication. That seems weird.
>
> No it is not determined by the dedication, if there is no dedication the
> 3 vertical values are possible.
You set it to either high or medium based on the dedication. And for horizontal
polarization it obviously doesn't matter.
As far as I understand you don't need this because the default values are fine.
You should add the check that if a dedicated cpu is hotplugged, then the entitlement
must be high, to patch 2 and that's it, no additional changes necessary.
>
>
> Regards,
> Pierre
>
> >
> > > + }
> > > }
> > >
> >
> > [...]
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-07 10:50 ` Nina Schoetterl-Glausch
@ 2023-02-07 12:19 ` Pierre Morel
2023-02-07 13:37 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 12:19 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/7/23 11:50, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-02-07 at 10:24 +0100, Pierre Morel wrote:
>>
>> On 2/6/23 18:52, Nina Schoetterl-Glausch wrote:
>>> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>>>> During a subsystem reset the Topology-Change-Report is cleared
>>>> by the machine.
>>>> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
>>>> bit of the SCA in the case of a subsystem reset.
>>>>
>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>> ---
>>>> include/hw/s390x/cpu-topology.h | 1 +
>>>> target/s390x/cpu.h | 1 +
>>>> target/s390x/kvm/kvm_s390x.h | 1 +
>>>> hw/s390x/cpu-topology.c | 12 ++++++++++++
>>>> hw/s390x/s390-virtio-ccw.c | 3 +++
>>>> target/s390x/cpu-sysemu.c | 13 +++++++++++++
>>>> target/s390x/kvm/kvm.c | 17 +++++++++++++++++
>>>> 7 files changed, 48 insertions(+)
>>>>
>>>> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
>>>> index 1ae7e7c5e3..60e0b9fbfa 100644
>>>> --- a/include/hw/s390x/cpu-topology.h
>>>> +++ b/include/hw/s390x/cpu-topology.h
>>>> @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
>>>>
>>>> extern S390Topology s390_topology;
>>>> int s390_socket_nb(S390CPU *cpu);
>>>> +void s390_topology_reset(void);
>>>>
>>>> #endif
>>>> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
>>>> index e1f6925856..848314d2a9 100644
>>>> --- a/target/s390x/cpu.h
>>>> +++ b/target/s390x/cpu.h
>>>> @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
>>>> QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
>>>>
>>>> void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
>>>> +void s390_cpu_topology_reset(void);
>>>
>>> How about you call this s390_cpu_topology_reset_modified, so it's symmetric
>>> with the function you define in the next patch. You could also drop the "cpu"
>>> from the name.
>>
>> I am not sure about this, Thomas already gave his R-B on this patch so I
>> prefer to stay on the original name, unless he says it is a good idea to
>> change.
>> Also in cpu-sysemu.c most of the function are tagged with _cpu_
>
> IMO, renaming a function would be a small enough change to keep an R-b.
>>
>>>
>>> Or maybe even better, you only define a function for setting the modified state,
>>> but make it take a bool argument. This way you also get rid of some code duplication
>>> and it wouldn't harm readability IMO.
>>
>> There is already a single function kvm_s390_topology_set_mtcr(attr) to
>> set the "modified state"
>
> Yes, but that is for KVM only and doesn't do error handling.
> So you need at least one function on top of that. What I'm suggesting is to
> only have one function instead of two because it gets rid of some code.
OK this is right.
I rename
void s390_cpu_topology_reset(void);
to
void s390_cpu_topology_set_mtcr(int value);
and then:
- s390_cpu_topology_reset();
+ s390_cpu_topology_set_mtcr(0);
>>
>>>
>>>>
>>>> /* MMU defines */
>>>> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
>>>> diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
>>>> index f9785564d0..649dae5948 100644
>>>> --- a/target/s390x/kvm/kvm_s390x.h
>>>> +++ b/target/s390x/kvm/kvm_s390x.h
>>>> @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
>>>> void kvm_s390_restart_interrupt(S390CPU *cpu);
>>>> void kvm_s390_stop_interrupt(S390CPU *cpu);
>>>> void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
>>>> +int kvm_s390_topology_set_mtcr(uint64_t attr);
>>>>
>>>> #endif /* KVM_S390X_H */
>>>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>>>> index a80a1ebf22..cf63f3dd01 100644
>>>> --- a/hw/s390x/cpu-topology.c
>>>> +++ b/hw/s390x/cpu-topology.c
>>>> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
>>>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>>>> }
>>>>
>>>> +/**
>>>> + * s390_topology_reset:
>>>> + *
>>>> + * Generic reset for CPU topology, calls s390_topology_reset()
>>>> + * s390_topology_reset() to reset the kernel Modified Topology
>>>> + * change record.
>>>> + */
>>>> +void s390_topology_reset(void)
>>>> +{
>>>
>>> I'm wondering if you shouldn't move the reset changes you do in the next patch
>>> into this one. I don't see what they have to do with PTF emulation.
>>
>> Here in this patch we do not intercept PTF and we have only an
>> horizontal polarity.
>> So we do not need to reset the polarity for all the vCPUs, we only need
>> it when we have vertical polarity.
>
> Well, with the PTF patch you don't get vertical polarity either, because you
> only enable the topology with patch 7.
> And since it's about resetting, it fits better in this patch IMO.
Not in my opinion, suppose the next patch never get included it has no
sense.
However if there is a majority in favor of this change I will do it.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction
2023-02-07 11:27 ` Nina Schoetterl-Glausch
@ 2023-02-07 13:03 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 13:03 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/7/23 12:27, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-02-07 at 10:59 +0100, Pierre Morel wrote:
>>
>> On 2/6/23 19:34, Nina Schoetterl-Glausch wrote:
>>> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
...
>>> [...]
>>>>
>>>> /**
>>>> @@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp)
>>>> (smp->books * smp->sockets * smp->cores)) %
>>>> smp->drawers;
>>>> }
>>>
>>> Why are the changes below in this patch?
>>
>> Because before thos patch we have only horizontal polarization.
>
> Not really since you only enable topology in the next patch.
>>
>>>
>>>> +
>>>> + /*
>>>> + * Machine polarity is set inside the global s390_topology structure.
>>>> + * In the case the polarity is set as horizontal set the entitlement
>>
>> Sorry here an error in the comment should be :
>> "In the case the polarity is NOT set as horizontal..."
>>
>>>> + * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when
>>>> + * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if
>>>> + * the vCPU is dedicated.
>>>> + */
>>>> + if (s390_topology.polarity && !env->entitlement) {
>>>
>>> It'd be more readable if you compared against enum values by name.
>>
>> Right, I will change this to
>>
>> if (s390_topology.polarity != S390_POLARITY_HORIZONTAL &&
>> env->entitlement == S390_ENTITLEMENT_UNSET) {
>>
>>>
>>> I don't see why you check s390_topology.polarity. If it is horizontal
>>> then the value of the entitlement doesn't matter at all, so you can set it
>>> to whatever.
>>
>> Right, that is why it is done only for vertical polarization (sorry for
>> the wrong comment)
>
> I'm saying you don't need to check it at all. You adjust the values for
> vertical polarization, but you could just always do that since the values
> don't matter at all if the polarization is horizontal.
OK right
>>
>>> All you want to do is enforce dedicated -> VERTICAL_HIGH, right?
>>> So why don't you just add
>>>
>>> + if (cpu->env.dedicated && cpu->env.entitlement != POLARITY_VERTICAL_HIGH) {
>>> + error_setg(errp, "A dedicated cpu implies high entitlement");
>>> + return;
>>> + } >
>>> to s390_topology_check?
>>
>> Here it is to set the default in the case the values are not provided.
>
> If no values are provided, they default to dedication=false and entitlement=medium,
> as defined by patch 1, which are fine and don't need to be adjusted.
Right, I think I added this when working on modifying the attributes in
QAPI and rebased it in the wrong patch.
Here it has no sense.
>
>>
>> But where you are right is that I should add a verification to the check
>> function.
>>
>>>
>>>> + if (env->dedicated) {
>>>> + env->entitlement = POLARITY_VERTICAL_HIGH;
>>>> + } else {
>>>> + env->entitlement = POLARITY_VERTICAL_MEDIUM;
>>>> + }
>>>
>>> If it is horizontal, then setting the entitlement is pointless as it will be
>>> reset to medium on PTF.
>>
>> That is why the polarity is tested (sorry for the bad comment)
>
> I said this because I'm fine with setting it pointlessly.
>
>>> So the current polarization is vertical and a cpu is being hotplugged,
>>> but setting the entitlement of the cpu being added is also pointless, because
>>> it's determined by the dedication. That seems weird.
>>
>> No it is not determined by the dedication, if there is no dedication the
>> 3 vertical values are possible.
>
> You set it to either high or medium based on the dedication. And for horizontal
> polarization it obviously doesn't matter.
on PTF yes but not on hotplug, on hotplug all 3 values are possible.
>
> As far as I understand you don't need this because the default values are fine.
> You should add the check that if a dedicated cpu is hotplugged, then the entitlement
> must be high, to patch 2 and that's it, no additional changes necessary.
Yes, right.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-07 12:19 ` Pierre Morel
@ 2023-02-07 13:37 ` Nina Schoetterl-Glausch
2023-02-07 14:08 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-07 13:37 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Tue, 2023-02-07 at 13:19 +0100, Pierre Morel wrote:
>
> On 2/7/23 11:50, Nina Schoetterl-Glausch wrote:
> > On Tue, 2023-02-07 at 10:24 +0100, Pierre Morel wrote:
> > >
> > > On 2/6/23 18:52, Nina Schoetterl-Glausch wrote:
> > > > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > > > During a subsystem reset the Topology-Change-Report is cleared
> > > > > by the machine.
> > > > > Let's ask KVM to clear the Modified Topology Change Report (MTCR)
> > > > > bit of the SCA in the case of a subsystem reset.
> > > > >
> > > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > > ---
> > > > > include/hw/s390x/cpu-topology.h | 1 +
> > > > > target/s390x/cpu.h | 1 +
> > > > > target/s390x/kvm/kvm_s390x.h | 1 +
> > > > > hw/s390x/cpu-topology.c | 12 ++++++++++++
> > > > > hw/s390x/s390-virtio-ccw.c | 3 +++
> > > > > target/s390x/cpu-sysemu.c | 13 +++++++++++++
> > > > > target/s390x/kvm/kvm.c | 17 +++++++++++++++++
> > > > > 7 files changed, 48 insertions(+)
> > > > >
> > > > > diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> > > > > index 1ae7e7c5e3..60e0b9fbfa 100644
> > > > > --- a/include/hw/s390x/cpu-topology.h
> > > > > +++ b/include/hw/s390x/cpu-topology.h
> > > > > @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
> > > > >
> > > > > extern S390Topology s390_topology;
> > > > > int s390_socket_nb(S390CPU *cpu);
> > > > > +void s390_topology_reset(void);
> > > > >
> > > > > #endif
> > > > > diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
> > > > > index e1f6925856..848314d2a9 100644
> > > > > --- a/target/s390x/cpu.h
> > > > > +++ b/target/s390x/cpu.h
> > > > > @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
> > > > > QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
> > > > >
> > > > > void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
> > > > > +void s390_cpu_topology_reset(void);
> > > >
> > > > How about you call this s390_cpu_topology_reset_modified, so it's symmetric
> > > > with the function you define in the next patch. You could also drop the "cpu"
> > > > from the name.
> > >
> > > I am not sure about this, Thomas already gave his R-B on this patch so I
> > > prefer to stay on the original name, unless he says it is a good idea to
> > > change.
> > > Also in cpu-sysemu.c most of the function are tagged with _cpu_
> >
> > IMO, renaming a function would be a small enough change to keep an R-b.
> > >
> > > >
> > > > Or maybe even better, you only define a function for setting the modified state,
> > > > but make it take a bool argument. This way you also get rid of some code duplication
> > > > and it wouldn't harm readability IMO.
> > >
> > > There is already a single function kvm_s390_topology_set_mtcr(attr) to
> > > set the "modified state"
> >
> > Yes, but that is for KVM only and doesn't do error handling.
> > So you need at least one function on top of that. What I'm suggesting is to
> > only have one function instead of two because it gets rid of some code.
>
> OK this is right.
> I rename
> void s390_cpu_topology_reset(void);
> to
> void s390_cpu_topology_set_mtcr(int value);
I don't find mtcr very descriptive and a bit of a SIE/KVM name, it might not
fit a possible future tcg implementation.
I'd just call it s390_cpu_topology_set_changed/modified, and have it take a bool,
because I cannot imagine other int values to make sense.
>
> and then:
>
> - s390_cpu_topology_reset();
> + s390_cpu_topology_set_mtcr(0);
>
>
> > >
> > > >
> > > > >
> > > > > /* MMU defines */
> > > > > #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
> > > > > diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
> > > > > index f9785564d0..649dae5948 100644
> > > > > --- a/target/s390x/kvm/kvm_s390x.h
> > > > > +++ b/target/s390x/kvm/kvm_s390x.h
> > > > > @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
> > > > > void kvm_s390_restart_interrupt(S390CPU *cpu);
> > > > > void kvm_s390_stop_interrupt(S390CPU *cpu);
> > > > > void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
> > > > > +int kvm_s390_topology_set_mtcr(uint64_t attr);
> > > > >
> > > > > #endif /* KVM_S390X_H */
> > > > > diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> > > > > index a80a1ebf22..cf63f3dd01 100644
> > > > > --- a/hw/s390x/cpu-topology.c
> > > > > +++ b/hw/s390x/cpu-topology.c
> > > > > @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
> > > > > QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
> > > > > }
> > > > >
> > > > > +/**
> > > > > + * s390_topology_reset:
> > > > > + *
> > > > > + * Generic reset for CPU topology, calls s390_topology_reset()
> > > > > + * s390_topology_reset() to reset the kernel Modified Topology
> > > > > + * change record.
> > > > > + */
> > > > > +void s390_topology_reset(void)
> > > > > +{
> > > >
> > > > I'm wondering if you shouldn't move the reset changes you do in the next patch
> > > > into this one. I don't see what they have to do with PTF emulation.
> > >
> > > Here in this patch we do not intercept PTF and we have only an
> > > horizontal polarity.
> > > So we do not need to reset the polarity for all the vCPUs, we only need
> > > it when we have vertical polarity.
> >
> > Well, with the PTF patch you don't get vertical polarity either, because you
> > only enable the topology with patch 7.
> > And since it's about resetting, it fits better in this patch IMO.
>
> Not in my opinion, suppose the next patch never get included it has no
> sense.
Well, yes, but then the series would be broken, since the facility requires PTF to work.
> However if there is a majority in favor of this change I will do it.
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report
2023-02-07 13:37 ` Nina Schoetterl-Glausch
@ 2023-02-07 14:08 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 14:08 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/7/23 14:37, Nina Schoetterl-Glausch wrote:
> On Tue, 2023-02-07 at 13:19 +0100, Pierre Morel wrote:
>>
>> On 2/7/23 11:50, Nina Schoetterl-Glausch wrote:
>>> On Tue, 2023-02-07 at 10:24 +0100, Pierre Morel wrote:
>>>>
>>>> On 2/6/23 18:52, Nina Schoetterl-Glausch wrote:
>>>>> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>>>>>> During a subsystem reset the Topology-Change-Report is cleared
>>>>>> by the machine.
>>>>>> Let's ask KVM to clear the Modified Topology Change Report (MTCR)
>>>>>> bit of the SCA in the case of a subsystem reset.
>>>>>>
>>>>>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>>>>>> ---
>>>>>> include/hw/s390x/cpu-topology.h | 1 +
>>>>>> target/s390x/cpu.h | 1 +
>>>>>> target/s390x/kvm/kvm_s390x.h | 1 +
>>>>>> hw/s390x/cpu-topology.c | 12 ++++++++++++
>>>>>> hw/s390x/s390-virtio-ccw.c | 3 +++
>>>>>> target/s390x/cpu-sysemu.c | 13 +++++++++++++
>>>>>> target/s390x/kvm/kvm.c | 17 +++++++++++++++++
>>>>>> 7 files changed, 48 insertions(+)
>>>>>>
>>>>>> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
>>>>>> index 1ae7e7c5e3..60e0b9fbfa 100644
>>>>>> --- a/include/hw/s390x/cpu-topology.h
>>>>>> +++ b/include/hw/s390x/cpu-topology.h
>>>>>> @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms,
>>>>>>
>>>>>> extern S390Topology s390_topology;
>>>>>> int s390_socket_nb(S390CPU *cpu);
>>>>>> +void s390_topology_reset(void);
>>>>>>
>>>>>> #endif
>>>>>> diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
>>>>>> index e1f6925856..848314d2a9 100644
>>>>>> --- a/target/s390x/cpu.h
>>>>>> +++ b/target/s390x/cpu.h
>>>>>> @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu {
>>>>>> QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
>>>>>>
>>>>>> void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
>>>>>> +void s390_cpu_topology_reset(void);
>>>>>
>>>>> How about you call this s390_cpu_topology_reset_modified, so it's symmetric
>>>>> with the function you define in the next patch. You could also drop the "cpu"
>>>>> from the name.
>>>>
>>>> I am not sure about this, Thomas already gave his R-B on this patch so I
>>>> prefer to stay on the original name, unless he says it is a good idea to
>>>> change.
>>>> Also in cpu-sysemu.c most of the function are tagged with _cpu_
>>>
>>> IMO, renaming a function would be a small enough change to keep an R-b.
>>>>
>>>>>
>>>>> Or maybe even better, you only define a function for setting the modified state,
>>>>> but make it take a bool argument. This way you also get rid of some code duplication
>>>>> and it wouldn't harm readability IMO.
>>>>
>>>> There is already a single function kvm_s390_topology_set_mtcr(attr) to
>>>> set the "modified state"
>>>
>>> Yes, but that is for KVM only and doesn't do error handling.
>>> So you need at least one function on top of that. What I'm suggesting is to
>>> only have one function instead of two because it gets rid of some code.
>>
>> OK this is right.
>> I rename
>> void s390_cpu_topology_reset(void);
>> to
>> void s390_cpu_topology_set_mtcr(int value);
>
> I don't find mtcr very descriptive and a bit of a SIE/KVM name, it might not
> fit a possible future tcg implementation.
> I'd just call it s390_cpu_topology_set_changed/modified, and have it take a bool,
> because I cannot imagine other int values to make sense.
OK
>
>>
>> and then:
>>
>> - s390_cpu_topology_reset();
>> + s390_cpu_topology_set_mtcr(0);
>>
>>
>>>>
>>>>>
>>>>>>
>>>>>> /* MMU defines */
>>>>>> #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */
>>>>>> diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
>>>>>> index f9785564d0..649dae5948 100644
>>>>>> --- a/target/s390x/kvm/kvm_s390x.h
>>>>>> +++ b/target/s390x/kvm/kvm_s390x.h
>>>>>> @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
>>>>>> void kvm_s390_restart_interrupt(S390CPU *cpu);
>>>>>> void kvm_s390_stop_interrupt(S390CPU *cpu);
>>>>>> void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
>>>>>> +int kvm_s390_topology_set_mtcr(uint64_t attr);
>>>>>>
>>>>>> #endif /* KVM_S390X_H */
>>>>>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>>>>>> index a80a1ebf22..cf63f3dd01 100644
>>>>>> --- a/hw/s390x/cpu-topology.c
>>>>>> +++ b/hw/s390x/cpu-topology.c
>>>>>> @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms)
>>>>>> QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next);
>>>>>> }
>>>>>>
>>>>>> +/**
>>>>>> + * s390_topology_reset:
>>>>>> + *
>>>>>> + * Generic reset for CPU topology, calls s390_topology_reset()
>>>>>> + * s390_topology_reset() to reset the kernel Modified Topology
>>>>>> + * change record.
>>>>>> + */
>>>>>> +void s390_topology_reset(void)
>>>>>> +{
>>>>>
>>>>> I'm wondering if you shouldn't move the reset changes you do in the next patch
>>>>> into this one. I don't see what they have to do with PTF emulation.
>>>>
>>>> Here in this patch we do not intercept PTF and we have only an
>>>> horizontal polarity.
>>>> So we do not need to reset the polarity for all the vCPUs, we only need
>>>> it when we have vertical polarity.
>>>
>>> Well, with the PTF patch you don't get vertical polarity either, because you
>>> only enable the topology with patch 7.
>>> And since it's about resetting, it fits better in this patch IMO.
>>
>> Not in my opinion, suppose the next patch never get included it has no
>> sense.
>
> Well, yes, but then the series would be broken, since the facility requires PTF to work.
Yes, same if the activation of the facility is not included.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
2023-02-06 12:21 ` Thomas Huth
@ 2023-02-07 14:59 ` Pierre Morel
2023-02-08 18:40 ` Nina Schoetterl-Glausch
2 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-07 14:59 UTC (permalink / raw)
To: qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, nsg, frankja, berrange, clg
On 2/1/23 14:20, Pierre Morel wrote:
> The modification of the CPU attributes are done through a monitor
> command.
>
> It allows to move the core inside the topology tree to optimise
> the cache usage in the case the host's hypervisor previously
> moved the CPU.
>
> The same command allows to modify the CPU attributes modifiers
> like polarization entitlement and the dedicated attribute to notify
> the guest if the host admin modified scheduling or dedication of a vCPU.
>
> With this knowledge the guest has the possibility to optimize the
> usage of the vCPUs.
>
> The command is made experimental for the moment.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine-target.json | 29 +++++++++++++
> include/monitor/hmp.h | 1 +
> hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
> hmp-commands.hx | 16 ++++++++
> 4 files changed, 134 insertions(+)
>
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index 2e267fa458..58df0f5061 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -342,3 +342,32 @@
> 'TARGET_S390X',
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64' ] } }
> +
> +##
> +# @x-set-cpu-topology:
> +#
> +# @core: the vCPU ID to be moved
> +# @socket: the destination socket where to move the vCPU
> +# @book: the destination book where to move the vCPU
> +# @drawer: the destination drawer where to move the vCPU
> +# @polarity: optional polarity, default is last polarity set by the guest
> +# @dedicated: optional, if the vCPU is dedicated to a real CPU
> +#
> +# Modifies the topology by moving the CPU inside the topology
> +# tree or by changing a modifier attribute of a CPU.
> +#
> +# Returns: Nothing on success, the reason on failure.
> +#
> +# Since: <next qemu stable release, eg. 1.0>
> +##
> +{ 'command': 'x-set-cpu-topology',
> + 'data': {
> + 'core': 'int',
> + 'socket': 'int',
> + 'book': 'int',
> + 'drawer': 'int',
> + '*polarity': 'int',
> + '*dedicated': 'bool'
> + },
> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> +}
> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
> index 1b3bdcb446..12827479cf 100644
> --- a/include/monitor/hmp.h
> +++ b/include/monitor/hmp.h
> @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
> HumanReadableText *(*qmp_handler)(Error **));
> void hmp_info_stats(Monitor *mon, const QDict *qdict);
> void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
>
> #endif
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index c33378577b..6c50050991 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -18,6 +18,10 @@
> #include "target/s390x/cpu.h"
> #include "hw/s390x/s390-virtio-ccw.h"
> #include "hw/s390x/cpu-topology.h"
> +#include "qapi/qapi-commands-machine-target.h"
> +#include "qapi/qmp/qdict.h"
> +#include "monitor/hmp.h"
> +#include "monitor/monitor.h"
>
> /*
> * s390_topology is used to keep the topology information.
> @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> /* topology tree is reflected in props */
> s390_update_cpu_props(ms, cpu);
> }
> +
> +/*
> + * qmp and hmp implementations
> + */
> +
> +static void s390_change_topology(int64_t core_id, int64_t socket_id,
> + int64_t book_id, int64_t drawer_id,
> + int64_t polarity, bool dedicated,
> + Error **errp)
> +{
> + MachineState *ms = current_machine;
> + S390CPU *cpu;
> + ERRP_GUARD();
> +
> + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
> + if (!cpu) {
> + error_setg(errp, "Core-id %ld does not exist!", core_id);
> + return;
> + }
> +
> + /* Verify the new topology */
!! Should verify the new topology but ...
> + s390_topology_check(cpu, errp);
... verifies the old one => I will have to change this.
Sorry, this will impact the patch 2 because I think I should modify the
s390_topology_check() function to take as arguments the individual
attributes.
Like:
static void s390_topology_check( uint16_t core_id, uint16_t socket_id,
uint16_t book_id, uint16_t drawer_id,
uint16_t entitlement, bool dedicated,
Error **errp)
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
2023-02-06 12:38 ` Thomas Huth
2023-02-06 12:41 ` Thomas Huth
@ 2023-02-07 18:26 ` Nina Schoetterl-Glausch
2023-02-08 9:11 ` Pierre Morel
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-07 18:26 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> S390x provides two more topology containers above the sockets,
> books and drawers.
>
> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine.json | 13 ++++++++++---
> hw/core/machine-qmp-cmds.c | 2 ++
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 3036117059..e36c39e258 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -53,11 +53,18 @@
> #
> # Additional information about a virtual S390 CPU
> #
> -# @cpu-state: the virtual CPU's state
> +# @cpu-state: the virtual CPU's state (since 2.12)
> +# @dedicated: the virtual CPU's dedication (since 8.0)
> +# @polarity: the virtual CPU's polarity (since 8.0)
> #
> # Since: 2.12
> ##
> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
> +{ 'struct': 'CpuInfoS390',
> + 'data': { 'cpu-state': 'CpuS390State',
> + 'dedicated': 'bool',
> + 'polarity': 'int'
> + }
> +}
>
> ##
> # @CpuInfoFast:
> @@ -70,7 +77,7 @@
> #
> # @thread-id: ID of the underlying host thread
> #
> -# @props: properties describing to which node/socket/core/thread
> +# @props: properties describing to which node/drawer/book/socket/core/thread
> # virtual CPU belongs to, provided if supported by board
> #
> # @target: the QEMU system emulation target, which determines which
> diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
> index 80d5e59651..e6d93cf2a0 100644
> --- a/hw/core/machine-qmp-cmds.c
> +++ b/hw/core/machine-qmp-cmds.c
> @@ -30,6 +30,8 @@ static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu)
> CPUS390XState *env = &s390_cpu->env;
>
> info->cpu_state = env->cpu_state;
> + info->dedicated = env->dedicated;
> + info->polarity = env->entitlement;
Might want to do s/polarity/entitlement on the whole patch to make this more coherent.
Reviewed-by: Nina Schoetterl-Glausch if you fix the issues in Thomas' first reply.
> #else
> abort();
> #endif
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast
2023-02-07 18:26 ` Nina Schoetterl-Glausch
@ 2023-02-08 9:11 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-08 9:11 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/7/23 19:26, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> S390x provides two more topology containers above the sockets,
>> books and drawers.
>>
>> Let's add these CPU attributes to the QAPI command query-cpu-fast.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine.json | 13 ++++++++++---
>> hw/core/machine-qmp-cmds.c | 2 ++
>> 2 files changed, 12 insertions(+), 3 deletions(-)
>>
>> diff --git a/qapi/machine.json b/qapi/machine.json
>> index 3036117059..e36c39e258 100644
>> --- a/qapi/machine.json
>> +++ b/qapi/machine.json
>> @@ -53,11 +53,18 @@
>> #
>> # Additional information about a virtual S390 CPU
>> #
>> -# @cpu-state: the virtual CPU's state
>> +# @cpu-state: the virtual CPU's state (since 2.12)
>> +# @dedicated: the virtual CPU's dedication (since 8.0)
>> +# @polarity: the virtual CPU's polarity (since 8.0)
>> #
>> # Since: 2.12
>> ##
>> -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
>> +{ 'struct': 'CpuInfoS390',
>> + 'data': { 'cpu-state': 'CpuS390State',
>> + 'dedicated': 'bool',
>> + 'polarity': 'int'
>> + }
>> +}
>>
>> ##
>> # @CpuInfoFast:
>> @@ -70,7 +77,7 @@
>> #
>> # @thread-id: ID of the underlying host thread
>> #
>> -# @props: properties describing to which node/socket/core/thread
>> +# @props: properties describing to which node/drawer/book/socket/core/thread
>> # virtual CPU belongs to, provided if supported by board
>> #
>> # @target: the QEMU system emulation target, which determines which
>> diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
>> index 80d5e59651..e6d93cf2a0 100644
>> --- a/hw/core/machine-qmp-cmds.c
>> +++ b/hw/core/machine-qmp-cmds.c
>> @@ -30,6 +30,8 @@ static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu)
>> CPUS390XState *env = &s390_cpu->env;
>>
>> info->cpu_state = env->cpu_state;
>> + info->dedicated = env->dedicated;
>> + info->polarity = env->entitlement;
>
> Might want to do s/polarity/entitlement on the whole patch to make this more coherent.
> Reviewed-by: Nina Schoetterl-Glausch if you fix the issues in Thomas' first reply.
>
OK, substitution done and Thomas changes already done.
Thanks.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology
2023-02-01 13:20 ` [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
@ 2023-02-08 16:22 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-08 16:22 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > > Add some basic examples for the definition of cpu topology
> > > > in s390x.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > ---
> > > > docs/system/s390x/cpu-topology.rst | 294 +++++++++++++++++++++++++++++
> > > > docs/system/target-s390x.rst | 1 +
> > > > 2 files changed, 295 insertions(+)
> > > > create mode 100644 docs/system/s390x/cpu-topology.rst
> > > >
> > > > diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst
> > > > new file mode 100644
> > > > index 0000000000..e2190318c0
> > > > --- /dev/null
> > > > +++ b/docs/system/s390x/cpu-topology.rst
> > > > @@ -0,0 +1,294 @@
> > > > +CPU topology on s390x
> > > > +=====================
> > > > +
> > > > +Since QEMU 8.0, CPU topology on s390x provides up to 4 levels of
> > > > +topology containers: drawers, books, sockets and cores.
Cores isn't a container itself, at least as defined by STSI.
Maybe use "nodes" instead?
> > > > +
> > > > +The first three containers define a tree hierarchy, the last one
> > > > +provides the placement of the CPUs inside the parent container and
> > > > +3 CPU attributes:
The latter half of the sentence is a bit confusing, it's too informed
by the output of STSI.
Suggestion:
The first three are containers, defining a tree shaped hierarchy.
There are three attributes associated with each core:
> > > > +
> > > > +- CPU type
> > > > +- polarity entitlement
> > > > +- dedication
> > > > +
> > > > +Note also that since 7.2 threads are no longer supported in the topology
> > > > +and the ``-smp`` command line argument accepts only ``threads=1``.
> > > > +
> > > > +Prerequisites
> > > > +-------------
> > > > +
> > > > +To use CPU topology a Linux QEMU/KVM machine providing the CPU topology facility
> > > > +(STFLE bit 11) is required.
> > > > +
> > > > +However, since this facility has been enabled by default in an early version
> > > > +of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM
> > > > +that QEMU is supporting CPU topology.
supports
> > > > +
> > > > +Enabling CPU topology
> > > > +---------------------
> > > > +
> > > > +Currently, CPU topology is only enabled in the host model by default.
> > > > +
> > > > +Enabling CPU topology in a CPU model is done by setting the CPU flag
> > > > +``ctop`` to ``on`` like in:
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + -cpu gen16b,ctop=on
> > > > +
> > > > +Having the topology disabled by default allows migration between
> > > > +old and new QEMU without adding new flags.
> > > > +
> > > > +Default topology usage
> > > > +----------------------
> > > > +
> > > > +The CPU topology, can be specified on the QEMU command line
Drop the comma.
> > > > +with the ``-smp`` or the ``-device`` QEMU command arguments
> > > > +without using any new attributes.
Might want to split this up for -smp and -device.
If none of the new attributes (drawers, books) are specified for the
-smp flag, <description of default>
If none of the new attributes (drawer, book, dedicated, entitlement)
are specified for the -device cpu flag, the topology will be ...
> > > > +In this case, the topology will be calculated by simply adding
> > > > +to the topology the cores based on the core-id starting with
> > > > +core-0 at position 0 of socket-0, book-0, drawer-0 with default
> > > > +modifier attributes: horizontal polarity and no dedication.
> > > > +
> > > > +In the following machine we define 8 sockets with 4 cores each.
> > > > +Note that s390x QEMU machines do not implement multithreading.
You could drop that since you already said it earlier.
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + $ qemu-system-s390x -m 2G \
> > > > + -cpu gen16b,ctop=on \
> > > > + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \
> > > > + -device host-s390x-cpu,core-id=14 \
> > > > +
> > > > +New CPUs can be plugged using the device_add hmp command like in:
"as in" or "like so" sounds better to my non-native ears.
Applies to other instances, too.
>
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + (qemu) device_add gen16b-s390x-cpu,core-id=9
> > > > +
> > > > +The core-id defines the placement of the core in the topology by
> > > > +starting with core 0 in socket 0 up to maxcpus.
Suggestion:
The same placement of the CPU is derived from the core-id as described above.
> > > > +
> > > > +In the example above:
> > > > +
> > > > +* There are 5 CPUs provided to the guest with the ``-smp`` command line
> > > > + They will take the core-ids 0,1,2,3,4
> > > > + As we have 4 cores in a socket, we have 4 CPUs provided
> > > > + to the guest in socket 0, with core-ids 0,1,2,3.
> > > > + The last cpu, with core-id 4, will be on socket 1.
> > > > +
> > > > +* the core with ID 14 provided by the ``-device`` command line will
> > > > + be placed in socket 3, with core-id 14
> > > > +
> > > > +* the core with ID 9 provided by the ``device_add`` qmp command will
You said it's a hmp command earlier.
> > > > + be placed in socket 2, with core-id 9
> > > > +
> > > > +Polarity and dedication
s/Polarity/Polarization, same below
> > > > +-----------------------
> > > > +
> > > > +Polarity can be of two types: horizontal or vertical.
> > > > +
> > > > +The horizontal polarization specifies that all guest's vCPUs get
> > > > +almost the same amount of provisioning of real CPU by the host.
> > > > +
> > > > +The vertical polarization specifies that guest's vCPU can get
> > > > +different real CPU provisions:
> > > > +
No reason to capitalize vertical below, is there?
> > > > +- a vCPU with Vertical high entitlement specifies that this
> > > > + vCPU gets 100% of the real CPU provisioning.
> > > > +
> > > > +- a vCPU with Vertical medium entitlement specifies that this
> > > > + vCPU shares the real CPU with other vCPUs.
> > > > +
> > > > +- a vCPU with Vertical low entitlement specifies that this
> > > > + vCPU only get real CPU provisioning when no other vCPU need it.
getS, vCPUs or needS
> > > > +
> > > > +In the case a vCPU with vertical high entitlement does not use
In case
> > > > +the real CPU, the unused "slack" can be dispatched to other vCPU
vCPUs
> > > > +with medium or low entitlement.
> > > > +
> > > > +A subsystem reset puts all vCPU of the configuration into the
vCPUs
> > > > +horizontal polarization.
> > > > +
> > > > +The admin specifies the dedicated bit when the vCPU is dedicated
> > > > +to a single real CPU.
> > > > +
> > > > +As for the Linux admin, the dedicated bit is an indication on the
Drop everything before the comma. indication of
> > > > +affinity of a vCPU for a real CPU while the entitlement indicates the
affinity of a vCPU to a real CPU, while
> > > > +sharing or exclusivity of use.
> > > > +
> > > > +Defining the topology on command line
on the command line
> > > > +-------------------------------------
> > > > +
> > > > +The topology can be defined entirely during the CPU definition,
The topology can entirely be defined using -device cpu statements,
> > > > +with the exception of CPU 0 which must be defined with the -smp
> > > > +argument.
> > > > +
> > > > +For example, here we set the position of the cores 1,2,3 on
position ... to
> > > > +drawer 1, book 1, socket 2 and cores 0,9 and 14 on drawer 0,
same here, s/on/to
> > > > +book 0, socket 0 with all horizontal polarity and not dedicated.
> > > > +The core 4, will be set on its default position on socket 1
set to ... in socket 1
> > > > +(since we have 4 core per socket) and we define it with dedication and
> > > > +vertical high entitlement.
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + $ qemu-system-s390x -m 2G \
> > > > + -cpu gen16b,ctop=on \
> > > > + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \
> > > > + \
> > > > + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \
> > > > + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \
> > > > + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \
> > > > + \
> > > > + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \
> > > > + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \
> > > > + \
> > > > + -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarity=3 \
> > > > +
> > > > +QAPI interface for topology
> > > > +---------------------------
> > > > +
> > > > +Let's start QEMU with the following command:
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + sudo /usr/local/bin/qemu-system-s390x \
> > > > + -enable-kvm \
> > > > + -cpu z14,ctop=on \
> > > > + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
> > > > + \
> > > > + -device z14-s390x-cpu,core-id=19,polarity=3 \
> > > > + -device z14-s390x-cpu,core-id=11,polarity=1 \
> > > > + -device z14-s390x-cpu,core-id=112,polarity=3 \
> > > > + ...
> > > > +
> > > > +and see the result when using of the QAPI interface.
s/of//
> > > > +
> > > > +addons to query-cpus-fast
> > > > ++++++++++++++++++++++++++
> > > > +
> > > > +The command query-cpus-fast allows the admin to query the topology
> > > > +tree and modifiers for all configured vCPU.
vCPUs
> > > > +
> > > > +.. code-block:: QMP
> > > > +
> > > > + -> { "execute": "query-cpus-fast" }
> > > > + {
> > > > + "return": [
> > > > + {
> > > > + "dedicated": false,
> > > > + "thread-id": 3631238,
> > > > + "props": {
> > > > + "core-id": 0,
> > > > + "socket-id": 0,
> > > > + "drawer-id": 0,
> > > > + "book-id": 0
> > > > + },
> > > > + "cpu-state": "operating",
> > > > + "qom-path": "/machine/unattached/device[0]",
> > > > + "polarity": 2,
> > > > + "cpu-index": 0,
> > > > + "target": "s390x"
> > > > + },
> > > > + {
> > > > + "dedicated": false,
> > > > + "thread-id": 3631248,
> > > > + "props": {
> > > > + "core-id": 19,
> > > > + "socket-id": 9,
> > > > + "drawer-id": 0,
> > > > + "book-id": 2
> > > > + },
> > > > + "cpu-state": "operating",
> > > > + "qom-path": "/machine/peripheral-anon/device[0]",
> > > > + "polarity": 3,
> > > > + "cpu-index": 19,
> > > > + "target": "s390x"
> > > > + },
> > > > + {
> > > > + "dedicated": false,
> > > > + "thread-id": 3631249,
> > > > + "props": {
> > > > + "core-id": 11,
> > > > + "socket-id": 5,
> > > > + "drawer-id": 0,
> > > > + "book-id": 1
> > > > + },
> > > > + "cpu-state": "operating",
> > > > + "qom-path": "/machine/peripheral-anon/device[1]",
> > > > + "polarity": 1,
> > > > + "cpu-index": 11,
> > > > + "target": "s390x"
> > > > + },
> > > > + {
> > > > + "dedicated": true,
> > > > + "thread-id": 3631250,
> > > > + "props": {
> > > > + "core-id": 112,
> > > > + "socket-id": 56,
> > > > + "drawer-id": 3,
> > > > + "book-id": 14
> > > > + },
> > > > + "cpu-state": "operating",
> > > > + "qom-path": "/machine/peripheral-anon/device[2]",
> > > > + "polarity": 3,
> > > > + "cpu-index": 112,
> > > > + "target": "s390x"
> > > > + }
> > > > + ]
> > > > + }
> > > > +
> > > > +x-set-cpu-topology
> > > > +++++++++++++++++++
> > > > +
> > > > +The command x-set-cpu-topology allows the admin to modify the topology
> > > > +tree or the topology modifiers of a vCPU in the configuration.
> > > > +
> > > > +.. code-block:: QMP
> > > > +
> > > > + -> { "execute": "x-set-cpu-topology",
> > > > + "arguments": {
> > > > + "core": 11,
> > > > + "socket": 0,
> > > > + "book": 0,
> > > > + "drawer": 0,
> > > > + "polarity": 0,
> > > > + "dedicated": false
> > > > + }
> > > > + }
> > > > + <- {"return": {}}
> > > > +
> > > > +
> > > > +event CPU_POLARITY_CHANGE
> > > > ++++++++++++++++++++++++++
> > > > +
> > > > +When a guest is requesting a modification of the polarity,
requests
> > > > +QEMU sends a CPU_POLARITY_CHANGE event.
> > > > +
> > > > +When requesting the change, the guest only specifies horizontal or
> > > > +vertical polarity.
> > > > +The dedication and fine grain vertical entitlement depends on admin
> > > > +to set according to its response to this event.
It is the job of the admin to set the dedication and fine grained vertical entitlement
in response to this event.
> > > > +
> > > > +Note that a vertical polarized dedicated vCPU can only have a high
> > > > +entitlement, this gives 6 possibilities for a vCPU polarity:
for vCPU polarization.
> > > > +
> > > > +- Horizontal
> > > > +- Horizontal dedicated
> > > > +- Vertical low
> > > > +- Vertical medium
> > > > +- Vertical high
> > > > +- Vertical high dedicated
> > > > +
> > > > +Example of the event received when the guest issues the CPU instruction
> > > > +Perform Topology Function PTF(0) to request an horizontal polarity:
> > > > +
> > > > +.. code-block:: QMP
> > > > +
> > > > + <- { "event": "CPU_POLARITY_CHANGE",
> > > > + "data": { "polarity": 0 },
> > > > + "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> > > > +
> > > > +
> > > > diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst
> > > > index c636f64113..ff0ffe04f3 100644
> > > > --- a/docs/system/target-s390x.rst
> > > > +++ b/docs/system/target-s390x.rst
> > > > @@ -33,3 +33,4 @@ Architectural features
> > > > .. toctree::
> > > > s390x/bootdevices
> > > > s390x/protvirt
> > > > + s390x/cpu-topology
I'm not too big a fan of using "admin", I guess in reality it's mostly going to be libvirt
adjusting things. Maybe reformulate sentences to get rid of it or use
"outside administrative entity" or something similar. But in the end it's not too important.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-01 13:20 ` [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event Pierre Morel
@ 2023-02-08 17:35 ` Nina Schoetterl-Glausch
2023-02-08 19:23 ` Markus Armbruster
2023-02-09 10:04 ` Daniel P. Berrangé
0 siblings, 2 replies; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-08 17:35 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> When the guest asks to change the polarity this change
> is forwarded to the admin using QAPI.
> The admin is supposed to take according decisions concerning
> CPU provisioning.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
> hw/s390x/cpu-topology.c | 2 ++
> 2 files changed, 32 insertions(+)
>
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index 58df0f5061..5883c3b020 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -371,3 +371,33 @@
> },
> 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> }
> +
> +##
> +# @CPU_POLARITY_CHANGE:
> +#
> +# Emitted when the guest asks to change the polarity.
> +#
> +# @polarity: polarity specified by the guest
> +#
> +# The guest can tell the host (via the PTF instruction) whether the
> +# CPUs should be provisioned using horizontal or vertical polarity.
> +#
> +# On horizontal polarity the host is expected to provision all vCPUs
> +# equally.
> +# On vertical polarity the host can provision each vCPU differently.
> +# The guest will get information on the details of the provisioning
> +# the next time it uses the STSI(15) instruction.
> +#
> +# Since: 8.0
> +#
> +# Example:
> +#
> +# <- { "event": "CPU_POLARITY_CHANGE",
> +# "data": { "polarity": 0 },
> +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> +#
> +##
> +{ 'event': 'CPU_POLARITY_CHANGE',
> + 'data': { 'polarity': 'int' },
> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
support it will use the same event and right now it would just never be emitted.
On the other hand it's more conservative this way.
I also wonder if you should add 'feature' : [ 'unstable' ].
On the upside, it would mark the event as unstable, but I don't know what the
consequences are exactly.
Also I guess one can remove qemu events without breaking backwards compatibility,
since they just won't be emitted? Unless I guess you specify that a event must
occur under certain situations and the client waits on it?
Patch looks good.
> +}
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index 6c50050991..2f8e1b60cf 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -19,6 +19,7 @@
> #include "hw/s390x/s390-virtio-ccw.h"
> #include "hw/s390x/cpu-topology.h"
> #include "qapi/qapi-commands-machine-target.h"
> +#include "qapi/qapi-events-machine-target.h"
> #include "qapi/qmp/qdict.h"
> #include "monitor/hmp.h"
> #include "monitor/monitor.h"
> @@ -163,6 +164,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
> s390_topology.polarity = fc;
> s390_cpu_topology_set_modified();
> s390_topology_set_cpus_polarity(fc);
> + qapi_event_send_cpu_polarity_change(fc);
> setcc(cpu, 0);
> }
> break;
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
2023-02-02 10:44 ` Thomas Huth
2023-02-02 16:05 ` Nina Schoetterl-Glausch
@ 2023-02-08 17:50 ` Nina Schoetterl-Glausch
2023-02-10 14:19 ` Pierre Morel
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-08 17:50 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> S390 adds two new SMP levels, drawers and books to the CPU
> topology.
> The S390 CPU have specific toplogy features like dedication
> and polarity to give to the guest indications on the host
> vCPUs scheduling and help the guest take the best decisions
> on the scheduling of threads on the vCPUs.
>
> Let us provide the SMP properties with books and drawers levels
> and S390 CPU with dedication and polarity,
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine.json | 14 ++++++++--
> include/hw/boards.h | 10 ++++++-
> include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
> target/s390x/cpu.h | 5 ++++
> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
> hw/core/machine.c | 4 +++
> hw/s390x/s390-virtio-ccw.c | 2 ++
> softmmu/vl.c | 6 +++++
> target/s390x/cpu.c | 7 +++++
> qemu-options.hx | 7 +++--
> 10 files changed, 115 insertions(+), 12 deletions(-)
> create mode 100644 include/hw/s390x/cpu-topology.h
>
[...]
>
> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
> new file mode 100644
> index 0000000000..7a84b30a21
> --- /dev/null
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -0,0 +1,24 @@
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
> +#ifndef HW_S390X_CPU_TOPOLOGY_H
> +#define HW_S390X_CPU_TOPOLOGY_H
> +
> +#define S390_TOPOLOGY_CPU_IFL 0x03
> +
> +enum s390_topology_polarity {
> + POLARITY_HORIZONTAL,
> + POLARITY_VERTICAL,
> + POLARITY_VERTICAL_LOW = 1,
> + POLARITY_VERTICAL_MEDIUM,
> + POLARITY_VERTICAL_HIGH,
> + POLARITY_MAX,
> +};
IMO you should define the polarization and entitlement enums as
qapi enums.
The polarization is passed as data in the qapi event when the polarization
changes.
And the entitlement is passed to qemu when modifying the topology.
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
2023-02-06 12:21 ` Thomas Huth
2023-02-07 14:59 ` Pierre Morel
@ 2023-02-08 18:40 ` Nina Schoetterl-Glausch
2023-02-09 13:14 ` Pierre Morel
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-08 18:40 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> The modification of the CPU attributes are done through a monitor
> command.
>
> It allows to move the core inside the topology tree to optimise
> the cache usage in the case the host's hypervisor previously
> moved the CPU.
>
> The same command allows to modify the CPU attributes modifiers
> like polarization entitlement and the dedicated attribute to notify
> the guest if the host admin modified scheduling or dedication of a vCPU.
>
> With this knowledge the guest has the possibility to optimize the
> usage of the vCPUs.
>
> The command is made experimental for the moment.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> qapi/machine-target.json | 29 +++++++++++++
> include/monitor/hmp.h | 1 +
> hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
> hmp-commands.hx | 16 ++++++++
> 4 files changed, 134 insertions(+)
>
> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> index 2e267fa458..58df0f5061 100644
> --- a/qapi/machine-target.json
> +++ b/qapi/machine-target.json
> @@ -342,3 +342,32 @@
> 'TARGET_S390X',
> 'TARGET_MIPS',
> 'TARGET_LOONGARCH64' ] } }
> +
> +##
> +# @x-set-cpu-topology:
> +#
> +# @core: the vCPU ID to be moved
> +# @socket: the destination socket where to move the vCPU
> +# @book: the destination book where to move the vCPU
> +# @drawer: the destination drawer where to move the vCPU
I wonder if it wouldn't be more convenient for the caller if everything is optional.
> +# @polarity: optional polarity, default is last polarity set by the guest
> +# @dedicated: optional, if the vCPU is dedicated to a real CPU
> +#
> +# Modifies the topology by moving the CPU inside the topology
> +# tree or by changing a modifier attribute of a CPU.
> +#
> +# Returns: Nothing on success, the reason on failure.
> +#
> +# Since: <next qemu stable release, eg. 1.0>
> +##
> +{ 'command': 'x-set-cpu-topology',
> + 'data': {
> + 'core': 'int',
> + 'socket': 'int',
> + 'book': 'int',
> + 'drawer': 'int',
Did you consider naming those core-id, etc.? It would be consistent with
query-cpus-fast/CpuInstanceProperties. Also all your variables end with _id.
I don't care really just wanted to point it out.
> + '*polarity': 'int',
> + '*dedicated': 'bool'
> + },
> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> +}
So apparently this is the old way of doing an experimental api.
> Names beginning with ``x-`` used to signify "experimental". This
> convention has been replaced by special feature "unstable".
> Feature "unstable" marks a command, event, enum value, or struct
> member as unstable. It is not supported elsewhere so far. Interfaces
> so marked may be withdrawn or changed incompatibly in future releases.
> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
> index 1b3bdcb446..12827479cf 100644
> --- a/include/monitor/hmp.h
> +++ b/include/monitor/hmp.h
> @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
> HumanReadableText *(*qmp_handler)(Error **));
> void hmp_info_stats(Monitor *mon, const QDict *qdict);
> void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
>
> #endif
> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
> index c33378577b..6c50050991 100644
> --- a/hw/s390x/cpu-topology.c
> +++ b/hw/s390x/cpu-topology.c
> @@ -18,6 +18,10 @@
> #include "target/s390x/cpu.h"
> #include "hw/s390x/s390-virtio-ccw.h"
> #include "hw/s390x/cpu-topology.h"
> +#include "qapi/qapi-commands-machine-target.h"
> +#include "qapi/qmp/qdict.h"
> +#include "monitor/hmp.h"
> +#include "monitor/monitor.h"
>
> /*
> * s390_topology is used to keep the topology information.
> @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
> /* topology tree is reflected in props */
> s390_update_cpu_props(ms, cpu);
> }
> +
> +/*
> + * qmp and hmp implementations
> + */
> +
> +static void s390_change_topology(int64_t core_id, int64_t socket_id,
> + int64_t book_id, int64_t drawer_id,
> + int64_t polarity, bool dedicated,
> + Error **errp)
> +{
> + MachineState *ms = current_machine;
> + S390CPU *cpu;
> + ERRP_GUARD();
> +
> + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
> + if (!cpu) {
> + error_setg(errp, "Core-id %ld does not exist!", core_id);
> + return;
> + }
> +
> + /* Verify the new topology */
> + s390_topology_check(cpu, errp);
> + if (*errp) {
> + return;
> + }
> +
> + /* Move the CPU into its new socket */
> + s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true, errp);
The cpu isn't being created, so that should be false instead of true, right?
> +
> + /* All checks done, report topology in environment */
> + cpu->env.drawer_id = drawer_id;
> + cpu->env.book_id = book_id;
> + cpu->env.socket_id = socket_id;
> + cpu->env.dedicated = dedicated;
> + cpu->env.entitlement = polarity;
> +
> + /* topology tree is reflected in props */
> + s390_update_cpu_props(ms, cpu);
> +
> + /* Advertise the topology change */
> + s390_cpu_topology_set_modified();
> +}
> +
> +void qmp_x_set_cpu_topology(int64_t core, int64_t socket,
> + int64_t book, int64_t drawer,
> + bool has_polarity, int64_t polarity,
> + bool has_dedicated, bool dedicated,
> + Error **errp)
> +{
> + ERRP_GUARD();
> +
> + if (!s390_has_topology()) {
> + error_setg(errp, "This machine doesn't support topology");
> + return;
> + }
> + if (!has_polarity) {
> + polarity = POLARITY_VERTICAL_MEDIUM;
> + }
> + if (!has_dedicated) {
> + dedicated = false;
> + }
> + s390_change_topology(core, socket, book, drawer, polarity, dedicated, errp);
> +}
> +
> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict)
> +{
> + const int64_t core = qdict_get_int(qdict, "core");
> + const int64_t socket = qdict_get_int(qdict, "socket");
> + const int64_t book = qdict_get_int(qdict, "book");
> + const int64_t drawer = qdict_get_int(qdict, "drawer");
> + bool has_polarity = qdict_haskey(qdict, "polarity");
> + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0);
> + bool has_dedicated = qdict_haskey(qdict, "dedicated");
> + const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false);
> + Error *local_err = NULL;
> +
> + qmp_x_set_cpu_topology(core, socket, book, drawer,
> + has_polarity, polarity,
> + has_dedicated, dedicated,
> + &local_err);
> + if (hmp_handle_error(mon, local_err)) {
> + return;
> + }
What is the if for? The function ends anyway.
> +}
> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index 673e39a697..bb3c908356 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -1815,3 +1815,19 @@ SRST
> Dump the FDT in dtb format to *filename*.
> ERST
> #endif
> +
> +#if defined(TARGET_S390X) && defined(CONFIG_KVM)
> + {
> + .name = "x-set-cpu-topology",
> + .args_type = "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?",
> + .params = "core socket book drawer [polarity] [dedicated]",
> + .help = "Move CPU 'core' to 'socket/book/drawer' "
> + "optionaly modifies polarity and dedication",
> + .cmd = hmp_x_set_cpu_topology,
> + },
> +
> +SRST
> +``x-set-cpu-topology`` *core* *socket* *book* *drawer* *polarity* *dedicated*
> + Moves the CPU *core* to *socket* *book* *drawer* with *polarity* *dedicated*.
> +ERST
> +#endif
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-08 17:35 ` Nina Schoetterl-Glausch
@ 2023-02-08 19:23 ` Markus Armbruster
2023-02-09 12:28 ` Nina Schoetterl-Glausch
2023-02-09 10:04 ` Daniel P. Berrangé
1 sibling, 1 reply; 76+ messages in thread
From: Markus Armbruster @ 2023-02-08 19:23 UTC (permalink / raw)
To: Nina Schoetterl-Glausch
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, seiden, nrb, frankja,
berrange, clg
Nina Schoetterl-Glausch <nsg@linux.ibm.com> writes:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> When the guest asks to change the polarity this change
>> is forwarded to the admin using QAPI.
>> The admin is supposed to take according decisions concerning
>> CPU provisioning.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
>> hw/s390x/cpu-topology.c | 2 ++
>> 2 files changed, 32 insertions(+)
>>
>> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
>> index 58df0f5061..5883c3b020 100644
>> --- a/qapi/machine-target.json
>> +++ b/qapi/machine-target.json
>> @@ -371,3 +371,33 @@
>> },
>> 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
>> }
>> +
>> +##
>> +# @CPU_POLARITY_CHANGE:
>> +#
>> +# Emitted when the guest asks to change the polarity.
>> +#
>> +# @polarity: polarity specified by the guest
>> +#
>> +# The guest can tell the host (via the PTF instruction) whether the
>> +# CPUs should be provisioned using horizontal or vertical polarity.
>> +#
>> +# On horizontal polarity the host is expected to provision all vCPUs
>> +# equally.
>> +# On vertical polarity the host can provision each vCPU differently.
>> +# The guest will get information on the details of the provisioning
>> +# the next time it uses the STSI(15) instruction.
>> +#
>> +# Since: 8.0
>> +#
>> +# Example:
>> +#
>> +# <- { "event": "CPU_POLARITY_CHANGE",
>> +# "data": { "polarity": 0 },
>> +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
>> +#
>> +##
>> +{ 'event': 'CPU_POLARITY_CHANGE',
>> + 'data': { 'polarity': 'int' },
>> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
>
> I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
> support it will use the same event and right now it would just never be emitted.
> On the other hand it's more conservative this way.
TCG vs. KVM should be as transparent as we can make it.
If only KVM can get into the state where the event is emitted, say
because the state is only possible with features only KVM supports, then
making the event conditional on KVM makes sense. Of course, when
another accelerator acquires these features, we need to emit the event
there as well, which will involve adjusting the condition.
> I also wonder if you should add 'feature' : [ 'unstable' ].
> On the upside, it would mark the event as unstable, but I don't know what the
> consequences are exactly.
docs/devel/qapi-code-gen.rst:
Special features
~~~~~~~~~~~~~~~~
Feature "deprecated" marks a command, event, enum value, or struct
member as deprecated. It is not supported elsewhere so far.
Interfaces so marked may be withdrawn in future releases in accordance
with QEMU's deprecation policy.
Feature "unstable" marks a command, event, enum value, or struct
member as unstable. It is not supported elsewhere so far. Interfaces
so marked may be withdrawn or changed incompatibly in future releases.
See also -compat parameters unstable-input, unstable-output, both
intended for "testing the future".
> Also I guess one can remove qemu events without breaking backwards compatibility,
> since they just won't be emitted? Unless I guess you specify that a event must
> occur under certain situations and the client waits on it?
Events are part of the interface just like command returns are. Not
emitting an event in a situation where it was emitted before can easily
break things. Only when the situation is no longer possible, the event
can be removed safely.
Questions?
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-08 17:35 ` Nina Schoetterl-Glausch
2023-02-08 19:23 ` Markus Armbruster
@ 2023-02-09 10:04 ` Daniel P. Berrangé
2023-02-09 11:01 ` Markus Armbruster
2023-02-09 12:12 ` Nina Schoetterl-Glausch
1 sibling, 2 replies; 76+ messages in thread
From: Daniel P. Berrangé @ 2023-02-09 10:04 UTC (permalink / raw)
To: Nina Schoetterl-Glausch
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, armbru, seiden, nrb, frankja,
clg
On Wed, Feb 08, 2023 at 06:35:39PM +0100, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > When the guest asks to change the polarity this change
> > is forwarded to the admin using QAPI.
> > The admin is supposed to take according decisions concerning
> > CPU provisioning.
> >
> > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > ---
> > qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
> > hw/s390x/cpu-topology.c | 2 ++
> > 2 files changed, 32 insertions(+)
> >
> > diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> > index 58df0f5061..5883c3b020 100644
> > --- a/qapi/machine-target.json
> > +++ b/qapi/machine-target.json
> > @@ -371,3 +371,33 @@
> > },
> > 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> > }
> > +
> > +##
> > +# @CPU_POLARITY_CHANGE:
> > +#
> > +# Emitted when the guest asks to change the polarity.
> > +#
> > +# @polarity: polarity specified by the guest
> > +#
> > +# The guest can tell the host (via the PTF instruction) whether the
> > +# CPUs should be provisioned using horizontal or vertical polarity.
> > +#
> > +# On horizontal polarity the host is expected to provision all vCPUs
> > +# equally.
> > +# On vertical polarity the host can provision each vCPU differently.
> > +# The guest will get information on the details of the provisioning
> > +# the next time it uses the STSI(15) instruction.
> > +#
> > +# Since: 8.0
> > +#
> > +# Example:
> > +#
> > +# <- { "event": "CPU_POLARITY_CHANGE",
> > +# "data": { "polarity": 0 },
> > +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> > +#
> > +##
> > +{ 'event': 'CPU_POLARITY_CHANGE',
> > + 'data': { 'polarity': 'int' },
> > + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
>
> I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
> support it will use the same event and right now it would just never be emitted.
> On the other hand it's more conservative this way.
>
> I also wonder if you should add 'feature' : [ 'unstable' ].
> On the upside, it would mark the event as unstable, but I don't know what the
> consequences are exactly.
The intention of this flag is to allow mgmt apps to make a usage policy
decision.
Libvirt's policy is that we'll never use features marked unstable.
IOW, the consequence of marking it unstable is that it'll likely
go unused until the unstable marker gets removed.
Using 'unstable' is useful if you want to get complex code merged
before you're quite happy with the design, and then iterate on the
impl in-tree. This is OK if there's no urgent need for apps to
consume the feature. If you want the feature to be used for real
though, the unstable flag is not desirable and you need to finalize
the design.
> Also I guess one can remove qemu events without breaking backwards compatibility,
> since they just won't be emitted? Unless I guess you specify that a event must
> occur under certain situations and the client waits on it?
As Markus says, that's not a safe assumption. If a mgmt app is expecting
to receive an event, ceasing to emit it would likely be considered a
regression.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-09 10:04 ` Daniel P. Berrangé
@ 2023-02-09 11:01 ` Markus Armbruster
2023-02-09 12:12 ` Nina Schoetterl-Glausch
1 sibling, 0 replies; 76+ messages in thread
From: Markus Armbruster @ 2023-02-09 11:01 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Nina Schoetterl-Glausch, Pierre Morel, qemu-s390x, qemu-devel,
borntraeger, pasic, richard.henderson, david, thuth, cohuck, mst,
pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake, armbru,
seiden, nrb, frankja, clg
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Wed, Feb 08, 2023 at 06:35:39PM +0100, Nina Schoetterl-Glausch wrote:
[...]
>> I also wonder if you should add 'feature' : [ 'unstable' ].
>> On the upside, it would mark the event as unstable, but I don't know what the
>> consequences are exactly.
>
> The intention of this flag is to allow mgmt apps to make a usage policy
> decision.
>
> Libvirt's policy is that we'll never use features marked unstable.
>
> IOW, the consequence of marking it unstable is that it'll likely
> go unused until the unstable marker gets removed.
>
> Using 'unstable' is useful if you want to get complex code merged
> before you're quite happy with the design, and then iterate on the
> impl in-tree. This is OK if there's no urgent need for apps to
> consume the feature. If you want the feature to be used for real
> though, the unstable flag is not desirable and you need to finalize
> the design.
Another use of 'unstable' is debugging aids. Making these stable can be
plenty of pain for precious little gain.
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-09 10:04 ` Daniel P. Berrangé
2023-02-09 11:01 ` Markus Armbruster
@ 2023-02-09 12:12 ` Nina Schoetterl-Glausch
2023-02-09 12:15 ` Daniel P. Berrangé
1 sibling, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-09 12:12 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, armbru, seiden, nrb, frankja,
clg
On Thu, 2023-02-09 at 10:04 +0000, Daniel P. Berrangé wrote:
> On Wed, Feb 08, 2023 at 06:35:39PM +0100, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > When the guest asks to change the polarity this change
> > > is forwarded to the admin using QAPI.
> > > The admin is supposed to take according decisions concerning
> > > CPU provisioning.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
> > > hw/s390x/cpu-topology.c | 2 ++
> > > 2 files changed, 32 insertions(+)
> > >
> > > diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> > > index 58df0f5061..5883c3b020 100644
> > > --- a/qapi/machine-target.json
> > > +++ b/qapi/machine-target.json
> > > @@ -371,3 +371,33 @@
> > > },
> > > 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> > > }
> > > +
> > > +##
> > > +# @CPU_POLARITY_CHANGE:
> > > +#
> > > +# Emitted when the guest asks to change the polarity.
> > > +#
> > > +# @polarity: polarity specified by the guest
> > > +#
> > > +# The guest can tell the host (via the PTF instruction) whether the
> > > +# CPUs should be provisioned using horizontal or vertical polarity.
> > > +#
> > > +# On horizontal polarity the host is expected to provision all vCPUs
> > > +# equally.
> > > +# On vertical polarity the host can provision each vCPU differently.
> > > +# The guest will get information on the details of the provisioning
> > > +# the next time it uses the STSI(15) instruction.
> > > +#
> > > +# Since: 8.0
> > > +#
> > > +# Example:
> > > +#
> > > +# <- { "event": "CPU_POLARITY_CHANGE",
> > > +# "data": { "polarity": 0 },
> > > +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> > > +#
> > > +##
> > > +{ 'event': 'CPU_POLARITY_CHANGE',
> > > + 'data': { 'polarity': 'int' },
> > > + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
> >
> > I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
> > support it will use the same event and right now it would just never be emitted.
> > On the other hand it's more conservative this way.
> >
> > I also wonder if you should add 'feature' : [ 'unstable' ].
> > On the upside, it would mark the event as unstable, but I don't know what the
> > consequences are exactly.
>
> The intention of this flag is to allow mgmt apps to make a usage policy
> decision.
>
> Libvirt's policy is that we'll never use features marked unstable.
Does it enforce that, e.g via compat policies?
If so, I assume there is some way to allow use of unstable features in libvirt for development?
If for example you're prototyping a new mgmt feature that uses unstable commands.
>
> IOW, the consequence of marking it unstable is that it'll likely
> go unused until the unstable marker gets removed.
>
> Using 'unstable' is useful if you want to get complex code merged
> before you're quite happy with the design, and then iterate on the
> impl in-tree. This is OK if there's no urgent need for apps to
> consume the feature. If you want the feature to be used for real
> though, the unstable flag is not desirable and you need to finalize
> the design.
>
> > Also I guess one can remove qemu events without breaking backwards compatibility,
> > since they just won't be emitted? Unless I guess you specify that a event must
> > occur under certain situations and the client waits on it?
>
> As Markus says, that's not a safe assumption. If a mgmt app is expecting
> to receive an event, ceasing to emit it would likely be considered a
> regression.
>
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-09 12:12 ` Nina Schoetterl-Glausch
@ 2023-02-09 12:15 ` Daniel P. Berrangé
0 siblings, 0 replies; 76+ messages in thread
From: Daniel P. Berrangé @ 2023-02-09 12:15 UTC (permalink / raw)
To: Nina Schoetterl-Glausch
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, armbru, seiden, nrb, frankja,
clg
On Thu, Feb 09, 2023 at 01:12:17PM +0100, Nina Schoetterl-Glausch wrote:
> On Thu, 2023-02-09 at 10:04 +0000, Daniel P. Berrangé wrote:
> > On Wed, Feb 08, 2023 at 06:35:39PM +0100, Nina Schoetterl-Glausch wrote:
> > > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > > When the guest asks to change the polarity this change
> > > > is forwarded to the admin using QAPI.
> > > > The admin is supposed to take according decisions concerning
> > > > CPU provisioning.
> > > >
> > > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > > ---
> > > > qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
> > > > hw/s390x/cpu-topology.c | 2 ++
> > > > 2 files changed, 32 insertions(+)
> > > >
> > > > diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> > > > index 58df0f5061..5883c3b020 100644
> > > > --- a/qapi/machine-target.json
> > > > +++ b/qapi/machine-target.json
> > > > @@ -371,3 +371,33 @@
> > > > },
> > > > 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> > > > }
> > > > +
> > > > +##
> > > > +# @CPU_POLARITY_CHANGE:
> > > > +#
> > > > +# Emitted when the guest asks to change the polarity.
> > > > +#
> > > > +# @polarity: polarity specified by the guest
> > > > +#
> > > > +# The guest can tell the host (via the PTF instruction) whether the
> > > > +# CPUs should be provisioned using horizontal or vertical polarity.
> > > > +#
> > > > +# On horizontal polarity the host is expected to provision all vCPUs
> > > > +# equally.
> > > > +# On vertical polarity the host can provision each vCPU differently.
> > > > +# The guest will get information on the details of the provisioning
> > > > +# the next time it uses the STSI(15) instruction.
> > > > +#
> > > > +# Since: 8.0
> > > > +#
> > > > +# Example:
> > > > +#
> > > > +# <- { "event": "CPU_POLARITY_CHANGE",
> > > > +# "data": { "polarity": 0 },
> > > > +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> > > > +#
> > > > +##
> > > > +{ 'event': 'CPU_POLARITY_CHANGE',
> > > > + 'data': { 'polarity': 'int' },
> > > > + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
> > >
> > > I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
> > > support it will use the same event and right now it would just never be emitted.
> > > On the other hand it's more conservative this way.
> > >
> > > I also wonder if you should add 'feature' : [ 'unstable' ].
> > > On the upside, it would mark the event as unstable, but I don't know what the
> > > consequences are exactly.
> >
> > The intention of this flag is to allow mgmt apps to make a usage policy
> > decision.
> >
> > Libvirt's policy is that we'll never use features marked unstable.
>
> Does it enforce that, e.g via compat policies?
The policy is applied at time of code review, in that we'll not
merge patches that use features marked unstable.
> If so, I assume there is some way to allow use of unstable features in libvirt for development?
> If for example you're prototyping a new mgmt feature that uses unstable commands.
You can prototype usage in libvirt in a fork of course, but we
won't take patches into the libvirt upstream repo.
Alternatively in some cases the the libvirt QMP passthrough can
be used for experiemnts (eg virsh qemu-monitor-command ) in a
non-production envionrment.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-08 19:23 ` Markus Armbruster
@ 2023-02-09 12:28 ` Nina Schoetterl-Glausch
2023-02-09 13:00 ` Pierre Morel
0 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-09 12:28 UTC (permalink / raw)
To: Markus Armbruster
Cc: Pierre Morel, qemu-s390x, qemu-devel, borntraeger, pasic,
richard.henderson, david, thuth, cohuck, mst, pbonzini, kvm,
ehabkost, marcel.apfelbaum, eblake, seiden, nrb, frankja,
berrange, clg
On Wed, 2023-02-08 at 20:23 +0100, Markus Armbruster wrote:
> Nina Schoetterl-Glausch <nsg@linux.ibm.com> writes:
>
> > On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> > > When the guest asks to change the polarity this change
> > > is forwarded to the admin using QAPI.
> > > The admin is supposed to take according decisions concerning
> > > CPU provisioning.
> > >
> > > Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> > > ---
> > > qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++
> > > hw/s390x/cpu-topology.c | 2 ++
> > > 2 files changed, 32 insertions(+)
> > >
> > > diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> > > index 58df0f5061..5883c3b020 100644
> > > --- a/qapi/machine-target.json
> > > +++ b/qapi/machine-target.json
> > > @@ -371,3 +371,33 @@
> > > },
> > > 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
> > > }
> > > +
> > > +##
> > > +# @CPU_POLARITY_CHANGE:
> > > +#
> > > +# Emitted when the guest asks to change the polarity.
> > > +#
> > > +# @polarity: polarity specified by the guest
> > > +#
> > > +# The guest can tell the host (via the PTF instruction) whether the
> > > +# CPUs should be provisioned using horizontal or vertical polarity.
> > > +#
> > > +# On horizontal polarity the host is expected to provision all vCPUs
> > > +# equally.
> > > +# On vertical polarity the host can provision each vCPU differently.
> > > +# The guest will get information on the details of the provisioning
> > > +# the next time it uses the STSI(15) instruction.
> > > +#
> > > +# Since: 8.0
> > > +#
> > > +# Example:
> > > +#
> > > +# <- { "event": "CPU_POLARITY_CHANGE",
> > > +# "data": { "polarity": 0 },
> > > +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
> > > +#
> > > +##
> > > +{ 'event': 'CPU_POLARITY_CHANGE',
> > > + 'data': { 'polarity': 'int' },
> > > + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] }
> >
> > I wonder if you should depend on CONFIG_KVM or not. If tcg gets topology
> > support it will use the same event and right now it would just never be emitted.
> > On the other hand it's more conservative this way.
>
> TCG vs. KVM should be as transparent as we can make it.
>
> If only KVM can get into the state where the event is emitted, say
> because the state is only possible with features only KVM supports, then
> making the event conditional on KVM makes sense. Of course, when
> another accelerator acquires these features, we need to emit the event
> there as well, which will involve adjusting the condition.
That's the case here, KVM supports the feature, TCG doesn't, although there is no
reason it couldn't in the future.
>
> > I also wonder if you should add 'feature' : [ 'unstable' ].
> > On the upside, it would mark the event as unstable, but I don't know what the
> > consequences are exactly.
>
> docs/devel/qapi-code-gen.rst:
>
> Special features
> ~~~~~~~~~~~~~~~~
>
> Feature "deprecated" marks a command, event, enum value, or struct
> member as deprecated. It is not supported elsewhere so far.
> Interfaces so marked may be withdrawn in future releases in accordance
> with QEMU's deprecation policy.
>
> Feature "unstable" marks a command, event, enum value, or struct
> member as unstable. It is not supported elsewhere so far. Interfaces
> so marked may be withdrawn or changed incompatibly in future releases.
Yeah, I saw that, but wasn't aware of -compat, thanks.
>
> See also -compat parameters unstable-input, unstable-output, both
> intended for "testing the future".
>
> > Also I guess one can remove qemu events without breaking backwards compatibility,
> > since they just won't be emitted? Unless I guess you specify that a event must
> > occur under certain situations and the client waits on it?
>
> Events are part of the interface just like command returns are. Not
> emitting an event in a situation where it was emitted before can easily
> break things. Only when the situation is no longer possible, the event
> can be removed safely.
@Pierre, seems it would be a good idea to mark all changes to qmp unstable, not just
set-cpu-topology, can just remove it later after all.
>
> Questions?
>
> [...]
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-09 12:28 ` Nina Schoetterl-Glausch
@ 2023-02-09 13:00 ` Pierre Morel
2023-02-09 14:50 ` Nina Schoetterl-Glausch
0 siblings, 1 reply; 76+ messages in thread
From: Pierre Morel @ 2023-02-09 13:00 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, Markus Armbruster
Cc: qemu-s390x, qemu-devel, borntraeger, pasic, richard.henderson,
david, thuth, cohuck, mst, pbonzini, kvm, ehabkost,
marcel.apfelbaum, eblake, seiden, nrb, frankja, berrange, clg
On 2/9/23 13:28, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-08 at 20:23 +0100, Markus Armbruster wrote:
>> Nina Schoetterl-Glausch <nsg@linux.ibm.com> writes:
>>
...
>
>>
>>> I also wonder if you should add 'feature' : [ 'unstable' ].
>>> On the upside, it would mark the event as unstable, but I don't know what the
>>> consequences are exactly.
>>
>> docs/devel/qapi-code-gen.rst:
>>
>> Special features
>> ~~~~~~~~~~~~~~~~
>>
>> Feature "deprecated" marks a command, event, enum value, or struct
>> member as deprecated. It is not supported elsewhere so far.
>> Interfaces so marked may be withdrawn in future releases in accordance
>> with QEMU's deprecation policy.
>>
>> Feature "unstable" marks a command, event, enum value, or struct
>> member as unstable. It is not supported elsewhere so far. Interfaces
>> so marked may be withdrawn or changed incompatibly in future releases.
>
> Yeah, I saw that, but wasn't aware of -compat, thanks.
>
>>
>> See also -compat parameters unstable-input, unstable-output, both
>> intended for "testing the future".
>>
>>> Also I guess one can remove qemu events without breaking backwards compatibility,
>>> since they just won't be emitted? Unless I guess you specify that a event must
>>> occur under certain situations and the client waits on it?
>>
>> Events are part of the interface just like command returns are. Not
>> emitting an event in a situation where it was emitted before can easily
>> break things. Only when the situation is no longer possible, the event
>> can be removed safely.
>
> @Pierre, seems it would be a good idea to mark all changes to qmp unstable, not just
> set-cpu-topology, can just remove it later after all.
OK.
Just curious: how do you think this simple event matching the guest
request 1 on 1 may evolve?
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command
2023-02-08 18:40 ` Nina Schoetterl-Glausch
@ 2023-02-09 13:14 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-09 13:14 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/8/23 19:40, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> The modification of the CPU attributes are done through a monitor
>> command.
>>
>> It allows to move the core inside the topology tree to optimise
>> the cache usage in the case the host's hypervisor previously
>> moved the CPU.
>>
>> The same command allows to modify the CPU attributes modifiers
>> like polarization entitlement and the dedicated attribute to notify
>> the guest if the host admin modified scheduling or dedication of a vCPU.
>>
>> With this knowledge the guest has the possibility to optimize the
>> usage of the vCPUs.
>>
>> The command is made experimental for the moment.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine-target.json | 29 +++++++++++++
>> include/monitor/hmp.h | 1 +
>> hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++
>> hmp-commands.hx | 16 ++++++++
>> 4 files changed, 134 insertions(+)
>>
>> diff --git a/qapi/machine-target.json b/qapi/machine-target.json
>> index 2e267fa458..58df0f5061 100644
>> --- a/qapi/machine-target.json
>> +++ b/qapi/machine-target.json
>> @@ -342,3 +342,32 @@
>> 'TARGET_S390X',
>> 'TARGET_MIPS',
>> 'TARGET_LOONGARCH64' ] } }
>> +
>> +##
>> +# @x-set-cpu-topology:
>> +#
>> +# @core: the vCPU ID to be moved
>> +# @socket: the destination socket where to move the vCPU
>> +# @book: the destination book where to move the vCPU
>> +# @drawer: the destination drawer where to move the vCPU
>
> I wonder if it wouldn't be more convenient for the caller if everything is optional.
Yes, it is a good point.
>
>> +# @polarity: optional polarity, default is last polarity set by the guest
>> +# @dedicated: optional, if the vCPU is dedicated to a real CPU
>> +#
>> +# Modifies the topology by moving the CPU inside the topology
>> +# tree or by changing a modifier attribute of a CPU.
>> +#
>> +# Returns: Nothing on success, the reason on failure.
>> +#
>> +# Since: <next qemu stable release, eg. 1.0>
>> +##
>> +{ 'command': 'x-set-cpu-topology',
>> + 'data': {
>> + 'core': 'int',
>> + 'socket': 'int',
>> + 'book': 'int',
>> + 'drawer': 'int',
>
> Did you consider naming those core-id, etc.? It would be consistent with
> query-cpus-fast/CpuInstanceProperties. Also all your variables end with _id.
> I don't care really just wanted to point it out.
OK, core-id etc. looks better
>
>> + '*polarity': 'int',
>> + '*dedicated': 'bool'
>> + },
>> + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
>> +}
>
> So apparently this is the old way of doing an experimental api.
>
>> Names beginning with ``x-`` used to signify "experimental". This
>> convention has been replaced by special feature "unstable".
>
>> Feature "unstable" marks a command, event, enum value, or struct
>> member as unstable. It is not supported elsewhere so far. Interfaces
>> so marked may be withdrawn or changed incompatibly in future releases.
OK
>
>> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
>> index 1b3bdcb446..12827479cf 100644
>> --- a/include/monitor/hmp.h
>> +++ b/include/monitor/hmp.h
>> @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon,
>> HumanReadableText *(*qmp_handler)(Error **));
>> void hmp_info_stats(Monitor *mon, const QDict *qdict);
>> void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict);
>> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict);
>>
>> #endif
>> diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
>> index c33378577b..6c50050991 100644
>> --- a/hw/s390x/cpu-topology.c
>> +++ b/hw/s390x/cpu-topology.c
>> @@ -18,6 +18,10 @@
>> #include "target/s390x/cpu.h"
>> #include "hw/s390x/s390-virtio-ccw.h"
>> #include "hw/s390x/cpu-topology.h"
>> +#include "qapi/qapi-commands-machine-target.h"
>> +#include "qapi/qmp/qdict.h"
>> +#include "monitor/hmp.h"
>> +#include "monitor/monitor.h"
>>
>> /*
>> * s390_topology is used to keep the topology information.
>> @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp)
>> /* topology tree is reflected in props */
>> s390_update_cpu_props(ms, cpu);
>> }
>> +
>> +/*
>> + * qmp and hmp implementations
>> + */
>> +
>> +static void s390_change_topology(int64_t core_id, int64_t socket_id,
>> + int64_t book_id, int64_t drawer_id,
>> + int64_t polarity, bool dedicated,
>> + Error **errp)
>> +{
>> + MachineState *ms = current_machine;
>> + S390CPU *cpu;
>> + ERRP_GUARD();
>> +
>> + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu;
>> + if (!cpu) {
>> + error_setg(errp, "Core-id %ld does not exist!", core_id);
>> + return;
>> + }
>> +
>> + /* Verify the new topology */
>> + s390_topology_check(cpu, errp);
>> + if (*errp) {
>> + return;
>> + }
>> +
>> + /* Move the CPU into its new socket */
>> + s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true, errp);
>
> The cpu isn't being created, so that should be false instead of true, right?
Yes right, should be "false"
>
>> +
>> + /* All checks done, report topology in environment */
>> + cpu->env.drawer_id = drawer_id;
>> + cpu->env.book_id = book_id;
>> + cpu->env.socket_id = socket_id;
>> + cpu->env.dedicated = dedicated;
>> + cpu->env.entitlement = polarity;
>> +
>> + /* topology tree is reflected in props */
>> + s390_update_cpu_props(ms, cpu);
>> +
>> + /* Advertise the topology change */
>> + s390_cpu_topology_set_modified();
>> +}
>> +
>> +void qmp_x_set_cpu_topology(int64_t core, int64_t socket,
>> + int64_t book, int64_t drawer,
>> + bool has_polarity, int64_t polarity,
>> + bool has_dedicated, bool dedicated,
>> + Error **errp)
>> +{
>> + ERRP_GUARD();
>> +
>> + if (!s390_has_topology()) {
>> + error_setg(errp, "This machine doesn't support topology");
>> + return;
>> + }
>> + if (!has_polarity) {
>> + polarity = POLARITY_VERTICAL_MEDIUM;
>> + }
>> + if (!has_dedicated) {
>> + dedicated = false;
>> + }
>> + s390_change_topology(core, socket, book, drawer, polarity, dedicated, errp);
>> +}
>> +
>> +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict)
>> +{
>> + const int64_t core = qdict_get_int(qdict, "core");
>> + const int64_t socket = qdict_get_int(qdict, "socket");
>> + const int64_t book = qdict_get_int(qdict, "book");
>> + const int64_t drawer = qdict_get_int(qdict, "drawer");
>> + bool has_polarity = qdict_haskey(qdict, "polarity");
>> + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0);
>> + bool has_dedicated = qdict_haskey(qdict, "dedicated");
>> + const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false);
>> + Error *local_err = NULL;
>> +
>> + qmp_x_set_cpu_topology(core, socket, book, drawer,
>> + has_polarity, polarity,
>> + has_dedicated, dedicated,
>> + &local_err);
>> + if (hmp_handle_error(mon, local_err)) {
>> + return;
>> + }
>
> What is the if for? The function ends anyway.
Right, I take it away.
Thanks.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event
2023-02-09 13:00 ` Pierre Morel
@ 2023-02-09 14:50 ` Nina Schoetterl-Glausch
0 siblings, 0 replies; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-09 14:50 UTC (permalink / raw)
To: Pierre Morel, Markus Armbruster
Cc: qemu-s390x, qemu-devel, borntraeger, pasic, richard.henderson,
david, thuth, cohuck, mst, pbonzini, kvm, ehabkost,
marcel.apfelbaum, eblake, seiden, nrb, frankja, berrange, clg
On Thu, 2023-02-09 at 14:00 +0100, Pierre Morel wrote:
>
> On 2/9/23 13:28, Nina Schoetterl-Glausch wrote:
> > On Wed, 2023-02-08 at 20:23 +0100, Markus Armbruster wrote:
> > > Nina Schoetterl-Glausch <nsg@linux.ibm.com> writes:
> > >
>
> ...
>
> >
> > >
> > > > I also wonder if you should add 'feature' : [ 'unstable' ].
> > > > On the upside, it would mark the event as unstable, but I don't know what the
> > > > consequences are exactly.
> > >
> > > docs/devel/qapi-code-gen.rst:
> > >
> > > Special features
> > > ~~~~~~~~~~~~~~~~
> > >
> > > Feature "deprecated" marks a command, event, enum value, or struct
> > > member as deprecated. It is not supported elsewhere so far.
> > > Interfaces so marked may be withdrawn in future releases in accordance
> > > with QEMU's deprecation policy.
> > >
> > > Feature "unstable" marks a command, event, enum value, or struct
> > > member as unstable. It is not supported elsewhere so far. Interfaces
> > > so marked may be withdrawn or changed incompatibly in future releases.
> >
> > Yeah, I saw that, but wasn't aware of -compat, thanks.
> >
> > >
> > > See also -compat parameters unstable-input, unstable-output, both
> > > intended for "testing the future".
> > >
> > > > Also I guess one can remove qemu events without breaking backwards compatibility,
> > > > since they just won't be emitted? Unless I guess you specify that a event must
> > > > occur under certain situations and the client waits on it?
> > >
> > > Events are part of the interface just like command returns are. Not
> > > emitting an event in a situation where it was emitted before can easily
> > > break things. Only when the situation is no longer possible, the event
> > > can be removed safely.
> >
> > @Pierre, seems it would be a good idea to mark all changes to qmp unstable, not just
> > set-cpu-topology, can just remove it later after all.
>
> OK.
>
> Just curious: how do you think this simple event matching the guest
> request 1 on 1 may evolve?
Well, I don't know really, making it unstable is just more conservative for now.
But this way you can prototype/implement support in libvirt for topology and then
make the interface stable one you've confirmed that everything works as intended.
Here is something to think about: The architecture allows rejection of the PTF
polarization change request, if a request is currently in process. A possible design
to allow the same semantics for qemu/libvirt would be to set a polarization_change_in_progess
bit when a PTF request occurs and refuse subsequent requests until this bit was reset via qmp.
This wouldn't result in the event data structure being changed, but its semantics, since it
isn't fired for every request anymore.
>
> Regards,
> Pierre
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
2023-02-03 17:36 ` Nina Schoetterl-Glausch
2023-02-06 11:24 ` Thomas Huth
@ 2023-02-09 16:39 ` Nina Schoetterl-Glausch
2023-02-10 14:16 ` Pierre Morel
2 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-09 16:39 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
> include/hw/s390x/cpu-topology.h | 22 +++
> include/hw/s390x/sclp.h | 1 +
> target/s390x/cpu.h | 72 +++++++
> hw/s390x/cpu-topology.c | 10 +
> target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
> target/s390x/kvm/kvm.c | 5 +-
> target/s390x/kvm/meson.build | 3 +-
> 7 files changed, 446 insertions(+), 2 deletions(-)
> create mode 100644 target/s390x/kvm/cpu_topology.c
>
[...]
> +
> +/**
> + * s390_topology_from_cpu:
> + * @cpu: The S390CPU
> + *
> + * Initialize the topology id from the CPU environment.
> + */
> +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
> +{
> + s390_topology_id topology_id = {0};
> +
> + topology_id.drawer = cpu->env.drawer_id;
> + topology_id.book = cpu->env.book_id;
> + topology_id.socket = cpu->env.socket_id;
> + topology_id.origin = cpu->env.core_id / 64;
> + topology_id.type = S390_TOPOLOGY_CPU_IFL;
> + topology_id.dedicated = cpu->env.dedicated;
> +
> + if (s390_topology.polarity == POLARITY_VERTICAL) {
> + /*
> + * Vertical polarity with dedicated CPU implies
> + * vertical high entitlement.
> + */
> + if (topology_id.dedicated) {
> + topology_id.polarity |= POLARITY_VERTICAL_HIGH;
> + } else {
> + topology_id.polarity |= cpu->env.entitlement;
> + }
Why |= instead of an assignment?
Anyway, I think you can get rid of this in the next version.
If you define the entitlement via qapi you can just put a little switch
here and convert it to the hardware definition of polarization.
(Or you just do +1, but I think the switch is easier to understand)
> + }
> +
> + return topology_id;
> +}
> +
[...]
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 00/11] s390x: CPU Topology
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
` (10 preceding siblings ...)
2023-02-01 13:20 ` [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
@ 2023-02-09 17:14 ` Nina Schoetterl-Glausch
2023-02-10 13:23 ` Pierre Morel
11 siblings, 1 reply; 76+ messages in thread
From: Nina Schoetterl-Glausch @ 2023-02-09 17:14 UTC (permalink / raw)
To: Pierre Morel, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
IMO this series looks good overall and like it's nearing the final stages.
You use "polarity" instead of "polarization" a lot.
Since the PoP uses polarization I think that term would be preferred.
With the series as it is, one cannot set the polarization via qmp,
only the entitlement of individual cpus. So only the VM can change
the polarization.
Would it be desirable to also be able to set the polarization from the outside?
Like I said in one response, it would be good to consider if we need an
polarization_change_in_progress state that refuses requests.
I'm guessing not, if a request is always completed before the next is handled
and there is no way to lose requests.
I don't know how long it would take to change the CPU assignment and if there
is a chance that could be overwhelmed by too many requests.
Probably not but something worth thinking about.
Might be a good idea to add a test case that performs test via qmp.
So starts an instance with some cpu topology assignments, checks that
qmp returns the correct topology, hot plugs a cpu and does another check,
Changes topology attributes, etc.
I guess this would be an avocado test.
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 00/11] s390x: CPU Topology
2023-02-09 17:14 ` [PATCH v15 00/11] s390x: CPU Topology Nina Schoetterl-Glausch
@ 2023-02-10 13:23 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-10 13:23 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/9/23 18:14, Nina Schoetterl-Glausch wrote:
> IMO this series looks good overall and like it's nearing the final stages.
Thank you for your helping this.
>
> You use "polarity" instead of "polarization" a lot.
> Since the PoP uses polarization I think that term would be preferred.
OK
>
> With the series as it is, one cannot set the polarization via qmp,
> only the entitlement of individual cpus. So only the VM can change
> the polarization.
> Would it be desirable to also be able to set the polarization from the outside?
I do not think so, AFAIK this is not foreseen by the architecture.
My point of view on this is that the application running on the guest is
the one knowing if it can get use of the heterogeneous CPU provisioning
provided by the vertical polarization or not.
Horizontal polarization being the default with an homogeneous, or
considered as default, provisioning.
>
> Like I said in one response, it would be good to consider if we need an
> polarization_change_in_progress state that refuses requests.
> I'm guessing not, if a request is always completed before the next is handled
> and there is no way to lose requests.
> I don't know how long it would take to change the CPU assignment and if there
> is a chance that could be overwhelmed by too many requests.
> Probably not but something worth thinking about.
Currently, guest request for a topology change is done via the sysfs
interface by a userland process.
The value returned by the kernel to userland is -BUSY, for both DONE and
IN_PROGRESS.
So at first sight I do not see a overwhelming problem but having a
completion indication looks like a good thing to have in a future
extension in both QEMU and Kernel
>
> Might be a good idea to add a test case that performs test via qmp.
> So starts an instance with some cpu topology assignments, checks that
> qmp returns the correct topology, hot plugs a cpu and does another check,
> Changes topology attributes, etc.
> I guess this would be an avocado test.
Yes you are right there is a lot to test.
There is already a test for kvm_unit_tests in review to test PTF and
STSI instruction's interception.
We do not use avocado as far as I know but our hades tests framework for
the kind of tests you propose.
I never used avocado for now but at first sight, avocado and hades look
similar.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
2023-02-09 16:39 ` Nina Schoetterl-Glausch
@ 2023-02-10 14:16 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-10 14:16 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/9/23 17:39, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> On interception of STSI(15.1.x) the System Information Block
>> (SYSIB) is built from the list of pre-ordered topology entries.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> include/hw/s390x/cpu-topology.h | 22 +++
>> include/hw/s390x/sclp.h | 1 +
>> target/s390x/cpu.h | 72 +++++++
>> hw/s390x/cpu-topology.c | 10 +
>> target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++
>> target/s390x/kvm/kvm.c | 5 +-
>> target/s390x/kvm/meson.build | 3 +-
>> 7 files changed, 446 insertions(+), 2 deletions(-)
>> create mode 100644 target/s390x/kvm/cpu_topology.c
>>
> [...]
>> +
>> +/**
>> + * s390_topology_from_cpu:
>> + * @cpu: The S390CPU
>> + *
>> + * Initialize the topology id from the CPU environment.
>> + */
>> +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu)
>> +{
>> + s390_topology_id topology_id = {0};
>> +
>> + topology_id.drawer = cpu->env.drawer_id;
>> + topology_id.book = cpu->env.book_id;
>> + topology_id.socket = cpu->env.socket_id;
>> + topology_id.origin = cpu->env.core_id / 64;
>> + topology_id.type = S390_TOPOLOGY_CPU_IFL;
>> + topology_id.dedicated = cpu->env.dedicated;
>> +
>> + if (s390_topology.polarity == POLARITY_VERTICAL) {
>> + /*
>> + * Vertical polarity with dedicated CPU implies
>> + * vertical high entitlement.
>> + */
>> + if (topology_id.dedicated) {
>> + topology_id.polarity |= POLARITY_VERTICAL_HIGH;
>> + } else {
>> + topology_id.polarity |= cpu->env.entitlement;
>> + }
>
> Why |= instead of an assignment?
> Anyway, I think you can get rid of this in the next version.
> If you define the entitlement via qapi you can just put a little switch
> here and convert it to the hardware definition of polarization.
> (Or you just do +1, but I think the switch is easier to understand)
Oh! right thanks, it is a leftover from when dedication and polarity
were in the same variable.
I change it with the QAPI enum.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology
2023-02-08 17:50 ` Nina Schoetterl-Glausch
@ 2023-02-10 14:19 ` Pierre Morel
0 siblings, 0 replies; 76+ messages in thread
From: Pierre Morel @ 2023-02-10 14:19 UTC (permalink / raw)
To: Nina Schoetterl-Glausch, qemu-s390x
Cc: qemu-devel, borntraeger, pasic, richard.henderson, david, thuth,
cohuck, mst, pbonzini, kvm, ehabkost, marcel.apfelbaum, eblake,
armbru, seiden, nrb, frankja, berrange, clg
On 2/8/23 18:50, Nina Schoetterl-Glausch wrote:
> On Wed, 2023-02-01 at 14:20 +0100, Pierre Morel wrote:
>> S390 adds two new SMP levels, drawers and books to the CPU
>> topology.
>> The S390 CPU have specific toplogy features like dedication
>> and polarity to give to the guest indications on the host
>> vCPUs scheduling and help the guest take the best decisions
>> on the scheduling of threads on the vCPUs.
>>
>> Let us provide the SMP properties with books and drawers levels
>> and S390 CPU with dedication and polarity,
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>> qapi/machine.json | 14 ++++++++--
>> include/hw/boards.h | 10 ++++++-
>> include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++
>> target/s390x/cpu.h | 5 ++++
>> hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++-----
>> hw/core/machine.c | 4 +++
>> hw/s390x/s390-virtio-ccw.c | 2 ++
>> softmmu/vl.c | 6 +++++
>> target/s390x/cpu.c | 7 +++++
>> qemu-options.hx | 7 +++--
>> 10 files changed, 115 insertions(+), 12 deletions(-)
>> create mode 100644 include/hw/s390x/cpu-topology.h
>>
> [...]
>>
>> diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
>> new file mode 100644
>> index 0000000000..7a84b30a21
>> --- /dev/null
>> +++ b/include/hw/s390x/cpu-topology.h
>> @@ -0,0 +1,24 @@
>> +/*
>> + * CPU Topology
>> + *
>> + * Copyright IBM Corp. 2022
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
>> + * your option) any later version. See the COPYING file in the top-level
>> + * directory.
>> + */
>> +#ifndef HW_S390X_CPU_TOPOLOGY_H
>> +#define HW_S390X_CPU_TOPOLOGY_H
>> +
>> +#define S390_TOPOLOGY_CPU_IFL 0x03
>> +
>> +enum s390_topology_polarity {
>> + POLARITY_HORIZONTAL,
>> + POLARITY_VERTICAL,
>> + POLARITY_VERTICAL_LOW = 1,
>> + POLARITY_VERTICAL_MEDIUM,
>> + POLARITY_VERTICAL_HIGH,
>> + POLARITY_MAX,
>> +};
>
> IMO you should define the polarization and entitlement enums as
> qapi enums.
> The polarization is passed as data in the qapi event when the polarization
> changes.
> And the entitlement is passed to qemu when modifying the topology.
(Thought I already answered but... seems no.)
looks good, thanks, I change this.
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
^ permalink raw reply [flat|nested] 76+ messages in thread
end of thread, other threads:[~2023-02-10 14:19 UTC | newest]
Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-01 13:20 [PATCH v15 00/11] s390x: CPU Topology Pierre Morel
2023-02-01 13:20 ` [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Pierre Morel
2023-02-02 10:44 ` Thomas Huth
2023-02-02 13:15 ` Pierre Morel
2023-02-02 16:05 ` Nina Schoetterl-Glausch
2023-02-03 9:39 ` Pierre Morel
2023-02-03 11:21 ` Thomas Huth
2023-02-08 17:50 ` Nina Schoetterl-Glausch
2023-02-10 14:19 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug Pierre Morel
2023-02-02 16:42 ` Nina Schoetterl-Glausch
2023-02-03 9:21 ` Pierre Morel
2023-02-03 13:22 ` Nina Schoetterl-Glausch
2023-02-03 14:40 ` Pierre Morel
2023-02-03 15:38 ` Nina Schoetterl-Glausch
2023-02-01 13:20 ` [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Pierre Morel
2023-02-03 17:36 ` Nina Schoetterl-Glausch
2023-02-06 10:06 ` Pierre Morel
2023-02-06 10:32 ` Nina Schoetterl-Glausch
2023-02-06 11:24 ` Thomas Huth
2023-02-06 12:57 ` Pierre Morel
2023-02-09 16:39 ` Nina Schoetterl-Glausch
2023-02-10 14:16 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries Pierre Morel
2023-02-06 10:13 ` Thomas Huth
2023-02-06 10:19 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Pierre Morel
2023-02-06 11:05 ` Thomas Huth
2023-02-06 12:50 ` Pierre Morel
2023-02-06 17:52 ` Nina Schoetterl-Glausch
2023-02-07 9:24 ` Pierre Morel
2023-02-07 10:50 ` Nina Schoetterl-Glausch
2023-02-07 12:19 ` Pierre Morel
2023-02-07 13:37 ` Nina Schoetterl-Glausch
2023-02-07 14:08 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Pierre Morel
2023-02-06 11:38 ` Thomas Huth
2023-02-06 13:02 ` Pierre Morel
2023-02-06 18:34 ` Nina Schoetterl-Glausch
2023-02-07 9:59 ` Pierre Morel
2023-02-07 11:27 ` Nina Schoetterl-Glausch
2023-02-07 13:03 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology Pierre Morel
2023-02-06 11:57 ` Thomas Huth
2023-02-06 13:19 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Pierre Morel
2023-02-06 12:21 ` Thomas Huth
2023-02-06 14:03 ` Pierre Morel
2023-02-07 14:59 ` Pierre Morel
2023-02-08 18:40 ` Nina Schoetterl-Glausch
2023-02-09 13:14 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Pierre Morel
2023-02-06 12:38 ` Thomas Huth
2023-02-06 14:12 ` Pierre Morel
2023-02-06 12:41 ` Thomas Huth
2023-02-06 12:49 ` Daniel P. Berrangé
2023-02-06 13:09 ` Thomas Huth
2023-02-06 14:50 ` Daniel P. Berrangé
2023-02-07 10:10 ` Thomas Huth
2023-02-06 14:16 ` Pierre Morel
2023-02-07 18:26 ` Nina Schoetterl-Glausch
2023-02-08 9:11 ` Pierre Morel
2023-02-01 13:20 ` [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event Pierre Morel
2023-02-08 17:35 ` Nina Schoetterl-Glausch
2023-02-08 19:23 ` Markus Armbruster
2023-02-09 12:28 ` Nina Schoetterl-Glausch
2023-02-09 13:00 ` Pierre Morel
2023-02-09 14:50 ` Nina Schoetterl-Glausch
2023-02-09 10:04 ` Daniel P. Berrangé
2023-02-09 11:01 ` Markus Armbruster
2023-02-09 12:12 ` Nina Schoetterl-Glausch
2023-02-09 12:15 ` Daniel P. Berrangé
2023-02-01 13:20 ` [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology Pierre Morel
2023-02-08 16:22 ` Nina Schoetterl-Glausch
2023-02-09 17:14 ` [PATCH v15 00/11] s390x: CPU Topology Nina Schoetterl-Glausch
2023-02-10 13:23 ` Pierre Morel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.