* [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR
@ 2016-02-25 16:22 Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 1/6] cpu: Abstract CPU core type Bharata B Rao
` (6 more replies)
0 siblings, 7 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Hi,
This is an attempt to implement CPU hotplug for PowerPC sPAPR based on
the approach suggested by Andreas. While I say that, I should also explicitly
add that I have tried to follow Andreas' suggestions to the best of my
understanding and hence there could be bits which are still not
as per expectations.
I have tried to model this similarly to what Andreas did for x86 an year back at
https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04858.html
- Base type to represent CPU core
static const TypeInfo cpu_core_type_info = {
.name = TYPE_CPU_CORE,
.parent = TYPE_DEVICE,
.abstract = true,
};
This is similar to the abstract socket device type that Andreas created
- sPAPR specific CPU core device
static const TypeInfo spapr_cpu_core_type_info = {
.name = TYPE_SPAPR_CPU_CORE,
.parent = TYPE_CPU_CORE,
.instance_init = spapr_cpu_core_instance_init,
.instance_size = sizeof(sPAPRCPUCore),
.class_init = spapr_cpu_core_class_init,
};
This is similar to TYPE_X86_CPU_SOCKET that Andreas created to represent
the x86 specifc CPU socket.
- QOM links from machine object to CPU objects
Target machine determines the hotplug granularity. Granularity is "core"
for PowerPC.
MacineClass:init() would create as many "core" links as necessary to
span the max_cpus. The links are set for all the boot CPU cores from
machine init. For rest of the cores, the links are left dangling and
will be set during hotplug.
The link essentially indicates or represents the CPU slots present on
the board. The link name is used to identify the slot. In case of PowerPC,
standard link names like core[0], core[1] etc will be used. Archs that
work at socket granularity could use link names like socket[0] etc.
sPAPR core device will have a property called "slot" which will be used
to determine to which slot the core will get plugged into. Users are
expected to use existing link names like core[0] etc as the values for
slot property of the core device. Command to hotplug will look like this:
(qemu) device_add spapr_cpu_core,id=core2,slot=core[2]
- Creation of thread objects from core.
It is ideal to create the thread objects from core object's instance_init.
However thread object creation needs two inputs: number of threads and
cpu_model. If we want to stick to the global values for these (smp_threads and
MachineState.cpu_model), then we already know how many threads of what
type to create in core's instance_init.
However, if we want to be flexible and support heterogenous configuration
in future, nr_threads and cpu_model can be made properties of core device
and the CPU threads can be created from core object's property setters.
While this implementation defines these two properties to obtain nr_threads
and cpu_model, heterogenous configurations aren't allowed.
sPAPR core device looks like this:
typedef struct sPAPRCPUCore {
/*< private >*/
DeviceState parent_obj;
/*< public >*/
int nr_threads;
char *cpu_model;
char *slot;
PowerPCCPU *threads;
} sPAPRCPUCore;
@nr_threads, @cpu_model and @slot get set as properties during device_add
of spapr_cpu_core device.
(qemu) device_add spapr_cpu_core,id=core2,nr_threads=8,cpu_model=host,slot=core[2]
This will result in a core with 8 threads of cpu_model "host" getting
created and the resulting core will populate the pre-existing slot "core[2].
@threads will be g_malloc'ed to contain @nr_threads from property setter.
@cpu_model along with the base CPU type of powerpc64-cpu will be used
to create the actual PowerPCCPU threads. So we will have CPU types
like host-powerpc64-cpu or POWER8-powerpc64-cpu etc.
- QMP interface
The layout that I have in this implementation is more or less what Igor has
implemented/suggested.
MachineClass:cpu_slots() will be main interface to collect CPU slot
information. Target machine can implement them to provide information
about slots. What sits in the slot (core, socket or thread) is determined
by the machine.
{ 'struct': 'CPUInfo',
'data': { 'arch-id': 'int',
'type': 'str',
'*thread': 'int',
'*core': 'int',
'*socket' : 'int',
'*node' : 'int',
'*qom-path': 'str'
}
}
{ 'struct': 'CPUSlotInfo',
'data': { '*id': 'str',
'type': 'str',
'hotplug-granularity' : 'str',
'slot-id' : 'str',
'*qom-path': 'str',
'realized': 'bool',
'*nr-cpus': 'int',
'*cpus' : ['CPUInfo']
}
}
The slot links that we created from machine object will serve as the
starting point to get the information about available CPU slots and
their occupancy status. We can walk the link properties of the machine
object, and extract information about "core" links for PowerPC.
Each CPU slot will provide the following information:
Slot name/ID: core[0] etc for PowerPC
hotplug_granularity: "core" for PowerPC (Probably not required)
type: "spapr-cpu-core" for sPAPR
realized: true/false indicating if the slot is populated or not
I feel, the above information should be enough for the management to
come up with an appropriate device_add command to create and fill a core
into an empty slot.
If the slot is populated, the following additional information is provided:
qom_path: QOM path of the core (PowerPC) device
nr_cpus: Number of CPUs that are part of this slot
Each CPU will then have: Type, Arch ID, Thread ID, Core ID, Socket ID,
NUMA node.
This patchset is present at:
https://github.com/bharata/qemu/commits/spapr-cpu-core
Bharata B Rao (6):
cpu: Abstract CPU core type
spapr: CPU core device
spapr: Represent boot CPUs as spapr-cpu-core devices
spapr: CPU hotplug support
qmp,spapr: Show hot-plugged/pluggable CPU slots in the Machine
hmp: Implement 'info cpu-slots'
hmp-commands-info.hx | 14 ++
hmp.c | 56 ++++++++
hmp.h | 1 +
hw/core/machine.c | 19 +++
hw/cpu/Makefile.objs | 1 +
hw/cpu/core.c | 22 +++
hw/ppc/Makefile.objs | 1 +
hw/ppc/spapr.c | 311 ++++++++++++++++++++++++++++++++++++++--
hw/ppc/spapr_cpu_core.c | 210 +++++++++++++++++++++++++++
hw/ppc/spapr_events.c | 3 +
hw/ppc/spapr_rtas.c | 24 ++++
include/hw/boards.h | 4 +
include/hw/cpu/core.h | 17 +++
include/hw/ppc/spapr.h | 4 +
include/hw/ppc/spapr_cpu_core.h | 32 +++++
qapi-schema.json | 85 +++++++++++
qmp-commands.hx | 47 ++++++
17 files changed, 843 insertions(+), 8 deletions(-)
create mode 100644 hw/cpu/core.c
create mode 100644 hw/ppc/spapr_cpu_core.c
create mode 100644 include/hw/cpu/core.h
create mode 100644 include/hw/ppc/spapr_cpu_core.h
--
2.1.0
^ permalink raw reply [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 1/6] cpu: Abstract CPU core type
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
` (5 subsequent siblings)
6 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Add an abstract CPU core type that could be used by machines that want
to define and hotplug CPUs in core granularity.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/cpu/Makefile.objs | 1 +
hw/cpu/core.c | 22 ++++++++++++++++++++++
include/hw/cpu/core.h | 17 +++++++++++++++++
3 files changed, 40 insertions(+)
create mode 100644 hw/cpu/core.c
create mode 100644 include/hw/cpu/core.h
diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
index 0954a18..942a4bb 100644
--- a/hw/cpu/Makefile.objs
+++ b/hw/cpu/Makefile.objs
@@ -2,4 +2,5 @@ obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
obj-$(CONFIG_REALVIEW) += realview_mpcore.o
obj-$(CONFIG_A9MPCORE) += a9mpcore.o
obj-$(CONFIG_A15MPCORE) += a15mpcore.o
+obj-y += core.o
diff --git a/hw/cpu/core.c b/hw/cpu/core.c
new file mode 100644
index 0000000..3c42236
--- /dev/null
+++ b/hw/cpu/core.c
@@ -0,0 +1,22 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+
+static const TypeInfo cpu_core_type_info = {
+ .name = TYPE_CPU_CORE,
+ .parent = TYPE_DEVICE,
+ .abstract = true,
+};
+
+static void cpu_core_register_types(void)
+{
+ type_register_static(&cpu_core_type_info);
+}
+
+type_init(cpu_core_register_types)
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
new file mode 100644
index 0000000..71d4724
--- /dev/null
+++ b/include/hw/cpu/core.h
@@ -0,0 +1,17 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_CPU_CORE_H
+#define HW_CPU_CORE_H
+
+#include "qemu/osdep.h"
+#include "hw/qdev.h"
+
+#define TYPE_CPU_CORE "cpu-core"
+
+#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 1/6] cpu: Abstract CPU core type Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-02-26 2:57 ` David Gibson
` (2 more replies)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices Bharata B Rao
` (4 subsequent siblings)
6 siblings, 3 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Add sPAPR specific CPU core device that is based on generic CPU core device.
Creating this core device will result in creation of all the CPU thread
devices that are part of this core.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/Makefile.objs | 1 +
hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
include/hw/ppc/spapr_cpu_core.h | 32 ++++++
3 files changed, 243 insertions(+)
create mode 100644 hw/ppc/spapr_cpu_core.c
create mode 100644 include/hw/ppc/spapr_cpu_core.h
diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
index c1ffc77..5cc6608 100644
--- a/hw/ppc/Makefile.objs
+++ b/hw/ppc/Makefile.objs
@@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
+obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
obj-y += spapr_pci_vfio.o
endif
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
new file mode 100644
index 0000000..c44eb61
--- /dev/null
+++ b/hw/ppc/spapr_cpu_core.c
@@ -0,0 +1,210 @@
+/*
+ * sPAPR CPU core device, acts as container of CPU thread devices.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+#include "hw/ppc/spapr_cpu_core.h"
+#include "hw/ppc/spapr.h"
+#include "hw/boards.h"
+#include "qemu/error-report.h"
+#include "qapi/visitor.h"
+#include <sysemu/cpus.h>
+
+static int spapr_cpu_core_realize_child(Object *child, void *opaque)
+{
+ Error **errp = opaque;
+
+ object_property_set_bool(child, true, "realized", errp);
+ if (*errp) {
+ return 1;
+ }
+ return 0;
+}
+
+static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+ Error *local_err = NULL;
+
+ if (!core->nr_threads) {
+ error_setg(errp, "nr_threads property can't be 0");
+ return;
+ }
+
+ if (!core->cpu_model) {
+ error_setg(errp, "cpu_model property isn't set");
+ return;
+ }
+
+ /*
+ * TODO: If slot isn't specified, plug this core into
+ * an existing empty slot.
+ */
+ if (!core->slot) {
+ error_setg(errp, "slot property isn't set");
+ return;
+ }
+
+ object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
+ &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
+}
+
+/*
+ * This creates the CPU threads for a given @core.
+ *
+ * In order to create the threads, we need two inputs - number of
+ * threads and the cpu_model. These are set as core object's properties.
+ * When both of them become available/set, this routine will be called from
+ * either property's set handler to create the threads.
+ *
+ * TODO: Dependence of threads creation on two properties is resulting
+ * in this not-so-clean way of creating threads from either of the
+ * property setters based on the order in which they get set. Check if
+ * this can be handled in a better manner.
+ */
+static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
+{
+ int i;
+
+ for (i = 0; i < core->nr_threads; i++) {
+ char id[32];
+ char type[32];
+
+ snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
+ TYPE_POWERPC_CPU);
+ object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
+
+ snprintf(id, sizeof(id), "thread[%d]", i);
+ object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
+ errp);
+ }
+}
+
+static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+
+ return core->slot;
+}
+
+static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
+ Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+
+ core->slot = g_strdup(val);
+}
+
+static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+
+ return core->cpu_model;
+}
+
+static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
+ Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+ MachineState *machine = MACHINE(qdev_get_machine());
+
+ /*
+ * cpu_model can't be different from what is specified with -cpu
+ */
+ if (strcmp(val, machine->cpu_model)) {
+ error_setg(errp, "cpu_model should be %s", machine->cpu_model);
+ return;
+ }
+
+ core->cpu_model = g_strdup(val);
+ if (core->nr_threads && core->cpu_model) {
+ spapr_cpu_core_create_threads(core, errp);
+ }
+}
+
+static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+ int64_t value = core->nr_threads;
+
+ visit_type_int(v, name, &value, errp);
+}
+
+static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
+ Error *local_err = NULL;
+ int64_t value;
+
+ visit_type_int(v, name, &value, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ /* Allow only homogeneous configuration */
+ if (value != smp_threads) {
+ error_setg(errp, "nr_threads should be %d", smp_threads);
+ return;
+ }
+
+ core->nr_threads = value;
+ core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
+
+ if (core->nr_threads && core->cpu_model) {
+ spapr_cpu_core_create_threads(core, errp);
+ }
+}
+
+static void spapr_cpu_core_instance_init(Object *obj)
+{
+ object_property_add(obj, "nr_threads", "int",
+ spapr_cpu_core_prop_get_nr_threads,
+ spapr_cpu_core_prop_set_nr_threads,
+ NULL, NULL, NULL);
+ object_property_add_str(obj, "cpu_model",
+ spapr_cpu_core_prop_get_cpu_model,
+ spapr_cpu_core_prop_set_cpu_model,
+ NULL);
+ object_property_add_str(obj, "slot",
+ spapr_cpu_core_prop_get_slot,
+ spapr_cpu_core_prop_set_slot,
+ NULL);
+}
+
+static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(oc);
+
+ dc->realize = spapr_cpu_core_realize;
+}
+
+static const TypeInfo spapr_cpu_core_type_info = {
+ .name = TYPE_SPAPR_CPU_CORE,
+ .parent = TYPE_CPU_CORE,
+ .instance_init = spapr_cpu_core_instance_init,
+ .instance_size = sizeof(sPAPRCPUCore),
+ .class_init = spapr_cpu_core_class_init,
+};
+
+static void spapr_cpu_core_register_types(void)
+{
+ type_register_static(&spapr_cpu_core_type_info);
+}
+
+type_init(spapr_cpu_core_register_types)
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
new file mode 100644
index 0000000..ed9bc7f
--- /dev/null
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -0,0 +1,32 @@
+/*
+ * sPAPR CPU core device.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_SPAPR_CPU_CORE_H
+#define HW_SPAPR_CPU_CORE_H
+
+#include "hw/qdev.h"
+#include "hw/cpu/core.h"
+
+#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
+#define SPAPR_CPU_CORE(obj) \
+ OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
+
+typedef struct sPAPRCPUCore {
+ /*< private >*/
+ DeviceState parent_obj;
+
+ /*< public >*/
+ int nr_threads;
+ char *cpu_model;
+ char *slot;
+ PowerPCCPU *threads;
+} sPAPRCPUCore;
+
+#define SPAPR_CPU_CORE_SLOT_PROP "slot"
+
+#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 1/6] cpu: Abstract CPU core type Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-02-26 3:45 ` David Gibson
2016-02-26 15:18 ` Igor Mammedov
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support Bharata B Rao
` (3 subsequent siblings)
6 siblings, 2 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Initialize boot CPUs as spapr-cpu-core devices and create links from
machine object to these core devices. These links can be considered
as CPU slots in which core devices will get hot-plugged. spapr-cpu-core
device's slot property indicates the slot where it is plugged. Information
about all the CPU slots can be obtained by walking these links.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 65 +++++++++++++++++++++++++++++++++++++++++++-------
include/hw/ppc/spapr.h | 3 +++
2 files changed, 60 insertions(+), 8 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e214a34..1f0d232 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -64,6 +64,7 @@
#include "hw/compat.h"
#include "qemu-common.h"
+#include "hw/ppc/spapr_cpu_core.h"
#include <libfdt.h>
@@ -1720,6 +1721,21 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
}
}
+/*
+ * Check to see if core is being hot-plugged into an already populated slot.
+ */
+static void spapr_cpu_core_allow_set_link(Object *obj, const char *name,
+ Object *val, Error **errp)
+{
+ Object *core = object_property_get_link(qdev_get_machine(), name, NULL);
+
+ if (core) {
+ char *path = object_get_canonical_path(core);
+ error_setg(errp, "Slot %s already populated with %s", name, path);
+ g_free(path);
+ }
+}
+
/* pSeries LPAR / sPAPR hardware init */
static void ppc_spapr_init(MachineState *machine)
{
@@ -1728,7 +1744,6 @@ static void ppc_spapr_init(MachineState *machine)
const char *kernel_filename = machine->kernel_filename;
const char *kernel_cmdline = machine->kernel_cmdline;
const char *initrd_filename = machine->initrd_filename;
- PowerPCCPU *cpu;
PCIHostState *phb;
int i;
MemoryRegion *sysmem = get_system_memory();
@@ -1742,6 +1757,8 @@ static void ppc_spapr_init(MachineState *machine)
long load_limit, fw_size;
bool kernel_le = false;
char *filename;
+ int spapr_cores = smp_cpus / smp_threads;
+ int spapr_max_cores = max_cpus / smp_threads;
msi_supported = true;
@@ -1800,13 +1817,38 @@ static void ppc_spapr_init(MachineState *machine)
if (machine->cpu_model == NULL) {
machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
}
- for (i = 0; i < smp_cpus; i++) {
- cpu = cpu_ppc_init(machine->cpu_model);
- if (cpu == NULL) {
- error_report("Unable to find PowerPC CPU definition");
- exit(1);
+
+ spapr->cores = g_malloc0(spapr_max_cores * sizeof(Object));
+
+ for (i = 0; i < spapr_max_cores; i++) {
+ Object *spapr_cpu_core = object_new(TYPE_SPAPR_CPU_CORE);
+ char name[32];
+
+ object_property_set_str(spapr_cpu_core, machine->cpu_model, "cpu_model",
+ &error_fatal);
+ object_property_set_int(spapr_cpu_core, smp_threads, "nr_threads",
+ &error_fatal);
+ /*
+ * Create links from machine objects to all possible cores.
+ */
+ snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
+ object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
+ (Object **)&spapr->cores[i],
+ spapr_cpu_core_allow_set_link, 0,
+ &error_fatal);
+
+ /*
+ * Set the link from machine object to core object for all
+ * boot time CPUs specified with -smp and realize them.
+ * For rest of the hotpluggable cores this is happens from
+ * the core hotplug/realization path.
+ */
+ if (i < spapr_cores) {
+ object_property_set_str(spapr_cpu_core, name,
+ SPAPR_CPU_CORE_SLOT_PROP, &error_fatal);
+ object_property_set_bool(spapr_cpu_core, true, "realized",
+ &error_fatal);
}
- spapr_cpu_init(spapr, cpu, &error_fatal);
}
if (kvm_enabled()) {
@@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
DeviceState *dev, Error **errp)
{
sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+ sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
int node;
@@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
}
spapr_memory_plug(hotplug_dev, dev, node, errp);
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+ CPUState *cs = CPU(dev);
+ PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+ spapr_cpu_init(ms, cpu, errp);
}
}
@@ -2259,7 +2307,8 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
DeviceState *dev)
{
- if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
+ if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
+ object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
return HOTPLUG_HANDLER(machine);
}
return NULL;
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 098d85d..20b3417 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -79,8 +79,11 @@ struct sPAPRMachineState {
/*< public >*/
char *kvm_type;
MemoryHotplugState hotplug_memory;
+ Object *cores;
};
+#define SPAPR_MACHINE_CPU_CORE_PROP "core"
+
#define H_SUCCESS 0
#define H_BUSY 1 /* Hardware busy -- retry later */
#define H_CLOSED 2 /* Resource closed */
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (2 preceding siblings ...)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-02-26 3:51 ` David Gibson
2016-02-26 13:03 ` Thomas Huth
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
` (2 subsequent siblings)
6 siblings, 2 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Set up device tree entries for the hotplugged CPU core and use the
exising EPOW event infrastructure to send CPU hotplug notification to
the guest.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++-
hw/ppc/spapr_events.c | 3 ++
hw/ppc/spapr_rtas.c | 24 +++++++++
include/hw/ppc/spapr.h | 1 +
4 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1f0d232..780cd00 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
size_t page_sizes_prop_size;
uint32_t vcpus_per_socket = smp_threads * smp_cores;
uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
+ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+ sPAPRDRConnector *drc;
+ sPAPRDRConnectorClass *drck;
+ int drc_index;
+
+ if (smc->dr_cpu_enabled) {
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
+ g_assert(drc);
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drc_index = drck->get_index(drc);
+ _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
+ }
/* Note: we keep CI large pages off for now because a 64K capable guest
* provisioned with large pages might otherwise try to map a qemu
@@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
_FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
}
+ if (smc->dr_cpu_enabled) {
+ int offset = fdt_path_offset(fdt, "/cpus");
+ ret = spapr_drc_populate_dt(fdt, offset, NULL,
+ SPAPR_DR_CONNECTOR_TYPE_CPU);
+ if (ret < 0) {
+ fprintf(stderr, "Couldn't set up CPU DR device tree properties\n");
+ exit(1);
+ }
+ }
+
_FDT((fdt_pack(fdt)));
if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
@@ -1759,6 +1781,7 @@ static void ppc_spapr_init(MachineState *machine)
char *filename;
int spapr_cores = smp_cpus / smp_threads;
int spapr_max_cores = max_cpus / smp_threads;
+ int smt = kvmppc_smt_threads();
msi_supported = true;
@@ -1813,6 +1836,15 @@ static void ppc_spapr_init(MachineState *machine)
spapr_validate_node_memory(machine, &error_fatal);
}
+ if (smc->dr_cpu_enabled) {
+ for (i = 0; i < spapr_max_cores; i++) {
+ sPAPRDRConnector *drc =
+ spapr_dr_connector_new(OBJECT(spapr),
+ SPAPR_DR_CONNECTOR_TYPE_CPU, i * smt);
+ qemu_register_reset(spapr_drc_reset, drc);
+ }
+ }
+
/* init CPUs */
if (machine->cpu_model == NULL) {
machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
@@ -2247,6 +2279,88 @@ out:
error_propagate(errp, local_err);
}
+static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs,
+ int *fdt_offset,
+ sPAPRMachineState *spapr)
+{
+ PowerPCCPU *cpu = POWERPC_CPU(cs);
+ DeviceClass *dc = DEVICE_GET_CLASS(cs);
+ int id = ppc_get_vcpu_dt_id(cpu);
+ void *fdt;
+ int offset, fdt_size;
+ char *nodename;
+
+ fdt = create_device_tree(&fdt_size);
+ nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
+ offset = fdt_add_subnode(fdt, 0, nodename);
+
+ spapr_populate_cpu_dt(cs, fdt, offset, spapr);
+ g_free(nodename);
+
+ *fdt_offset = offset;
+ return fdt;
+}
+
+static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp)
+{
+ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+ sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine());
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+ PowerPCCPU *cpu = &core->threads[0];
+ CPUState *cs = CPU(cpu);
+ int id = ppc_get_vcpu_dt_id(cpu);
+ sPAPRDRConnector *drc =
+ spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
+ sPAPRDRConnectorClass *drck;
+ Error *local_err = NULL;
+ void *fdt = NULL;
+ int fdt_offset = 0;
+
+ if (!smc->dr_cpu_enabled) {
+ /*
+ * This is a cold plugged CPU core but the machine doesn't support
+ * DR. So skip the hotplug path ensuring that the core is brought
+ * up online with out an associated DR connector.
+ */
+ return;
+ }
+
+ g_assert(drc);
+
+ /*
+ * Setup CPU DT entries only for hotplugged CPUs. For boot time or
+ * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
+ */
+ if (dev->hotplugged) {
+ fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms);
+ dev->hotplugged = true;
+ }
+
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
+ if (local_err) {
+ g_free(fdt);
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ if (dev->hotplugged) {
+ /*
+ * Send hotplug notification interrupt to the guest only in case
+ * of hotplugged CPUs.
+ */
+ spapr_hotplug_req_add_by_index(drc);
+ } else {
+ /*
+ * Set the right DRC states for cold plugged CPU.
+ */
+ drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
+ drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
+ }
+ return;
+}
+
static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
DeviceState *dev, Error **errp)
{
@@ -2291,8 +2405,25 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
} else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
CPUState *cs = CPU(dev);
PowerPCCPU *cpu = POWERPC_CPU(cs);
+ int i;
+
+ if (!smc->dr_cpu_enabled && dev->hotplugged) {
+ error_setg(errp, "CPU hotplug not supported for this machine");
+ return;
+ }
+
+ /* Set NUMA node for the added CPUs */
+ for (i = 0; i < nb_numa_nodes; i++) {
+ if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
+ cs->numa_node = i;
+ break;
+ }
+ }
spapr_cpu_init(ms, cpu, errp);
+ spapr_cpu_reset(cpu);
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+ spapr_core_plug(hotplug_dev, dev, errp);
}
}
@@ -2308,7 +2439,8 @@ static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
DeviceState *dev)
{
if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
- object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+ object_dynamic_cast(OBJECT(dev), TYPE_CPU) ||
+ object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
return HOTPLUG_HANDLER(machine);
}
return NULL;
@@ -2352,6 +2484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
smc->dr_lmb_enabled = true;
+ smc->dr_cpu_enabled = true;
fwc->get_dev_path = spapr_get_fw_dev_path;
nc->nmi_monitor_handler = spapr_nmi;
}
@@ -2431,6 +2564,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
spapr_machine_2_6_class_options(mc);
smc->use_ohci_by_default = true;
+ smc->dr_cpu_enabled = false;
SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
}
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index f5eac4b..e50bb16 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
case SPAPR_DR_CONNECTOR_TYPE_LMB:
hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
break;
+ case SPAPR_DR_CONNECTOR_TYPE_CPU:
+ hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
+ break;
default:
/* we shouldn't be signaling hotplug events for resources
* that don't support them
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index b7c5ebd..cc0369e 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -34,6 +34,7 @@
#include "hw/ppc/spapr.h"
#include "hw/ppc/spapr_vio.h"
+#include "hw/ppc/ppc.h"
#include "qapi-event.h"
#include "hw/boards.h"
@@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
}
+/*
+ * Set the timebase offset of the CPU to that of first CPU.
+ * This helps hotplugged CPU to have the correct timebase offset.
+ */
+static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
+{
+ PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+
+ cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
+}
+
+static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
+{
+ PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+ PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
+
+ if (!pcc->interrupts_big_endian(fcpu)) {
+ cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
+ }
+}
+
static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
uint32_t token, uint32_t nargs,
target_ulong args,
@@ -197,6 +219,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
env->nip = start;
env->gpr[3] = r3;
cs->halted = 0;
+ spapr_cpu_set_endianness(cpu);
+ spapr_cpu_update_tb_offset(cpu);
qemu_cpu_kick(cs);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 20b3417..eb4ccd1 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -36,6 +36,7 @@ struct sPAPRMachineClass {
/*< public >*/
bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */
+ bool dr_cpu_enabled; /* enable dynamic-reconfig/hotplug of CPUs */
bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */
};
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (3 preceding siblings ...)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-02-26 4:03 ` David Gibson
` (3 more replies)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 6/6] hmp: Implement 'info cpu-slots' Bharata B Rao
2016-03-01 10:00 ` [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
6 siblings, 4 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Implement query cpu-slots that provides information about hot-plugged
as well as hot-pluggable CPU slots that the machine supports.
TODO: As Eric suggested use enum for type instead of str.
TODO: @hotplug-granularity probably isn't required.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/core/machine.c | 19 +++++++++
hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
include/hw/boards.h | 4 ++
qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
qmp-commands.hx | 47 ++++++++++++++++++++++
5 files changed, 267 insertions(+)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 6d1a0d8..3055ef8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -17,6 +17,25 @@
#include "hw/sysbus.h"
#include "sysemu/sysemu.h"
#include "qemu/error-report.h"
+#include "qmp-commands.h"
+
+/*
+ * QMP: query-cpu-slots
+ *
+ * TODO: Ascertain if this is the right place to for this arch-neutral routine.
+ */
+CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
+{
+ MachineState *ms = MACHINE(qdev_get_machine());
+ MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+ if (!mc->cpu_slots) {
+ error_setg(errp, QERR_UNSUPPORTED);
+ return NULL;
+ }
+
+ return mc->cpu_slots(ms);
+}
static char *machine_get_accel(Object *obj, Error **errp)
{
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 780cd00..b76ed85 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
return cpu_index / smp_threads / smp_cores;
}
+static int spapr_cpuinfo_list(Object *obj, void *opaque)
+{
+ MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
+ CPUInfoList ***prev = opaque;
+
+ if (object_dynamic_cast(obj, TYPE_CPU)) {
+ CPUInfoList *elem = g_new0(CPUInfoList, 1);
+ CPUInfo *s = g_new0(CPUInfo, 1);
+ CPUState *cpu = CPU(obj);
+ PowerPCCPU *pcpu = POWERPC_CPU(cpu);
+
+ s->arch_id = ppc_get_vcpu_dt_id(pcpu);
+ s->type = g_strdup(object_get_typename(obj));
+ s->thread = cpu->cpu_index;
+ s->has_thread = true;
+ s->core = cpu->cpu_index / smp_threads;
+ s->has_core = true;
+ if (mc->cpu_index_to_socket_id) {
+ s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
+ } else {
+ s->socket = cpu->cpu_index / smp_threads / smp_cores;
+ }
+ s->has_socket = true;
+ s->node = cpu->numa_node;
+ s->has_node = true;
+ s->qom_path = object_get_canonical_path(obj);
+ s->has_qom_path = true;
+
+ elem->value = s;
+ elem->next = NULL;
+ **prev = elem;
+ *prev = &elem->next;
+ }
+ object_child_foreach(obj, spapr_cpuinfo_list, opaque);
+ return 0;
+}
+
+static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
+{
+ CPUSlotInfoList *head = NULL;
+ CPUSlotInfoList **prev = &head;
+ Object *root_container;
+ ObjectProperty *prop;
+ ObjectPropertyIterator iter;
+
+ /*
+ * TODO: There surely must be a better/easier way to walk all
+ * the link properties of an object ?
+ */
+ root_container = container_get(object_get_root(), "/machine");
+ object_property_iter_init(&iter, root_container);
+
+ while ((prop = object_property_iter_next(&iter))) {
+ Object *obj;
+ DeviceState *dev;
+ CPUSlotInfoList *elem;
+ CPUSlotInfo *s;
+ CPUInfoList *cpu_head = NULL;
+ CPUInfoList **cpu_prev = &cpu_head;
+ sPAPRCPUCore *core;
+
+ if (!strstart(prop->type, "link<", NULL)) {
+ continue;
+ }
+
+ if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
+ continue;
+ }
+
+ elem = g_new0(CPUSlotInfoList, 1);
+ s = g_new0(CPUSlotInfo, 1);
+
+ obj = object_property_get_link(root_container, prop->name, NULL);
+ if (obj) {
+ /* Slot populated */
+ dev = DEVICE(obj);
+ core = SPAPR_CPU_CORE(obj);
+
+ if (dev->id) {
+ s->has_id = true;
+ s->id = g_strdup(dev->id);
+ }
+ s->realized = object_property_get_bool(obj, "realized", NULL);
+ s->nr_cpus = core->nr_threads;
+ s->has_nr_cpus = true;
+ s->qom_path = object_get_canonical_path(obj);
+ s->has_qom_path = true;
+ if (s->realized) {
+ spapr_cpuinfo_list(obj, &cpu_prev);
+ }
+ s->has_cpus = true;
+ } else {
+ /* Slot empty */
+ s->has_id = false;
+ s->has_nr_cpus = false;
+ s->has_qom_path = false;
+ s->has_cpus = false;
+ s->realized = false;
+ }
+ s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
+ s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
+ s->slot_id = g_strdup(prop->name);
+ s->cpus = cpu_head;
+ elem->value = s;
+ elem->next = NULL;
+ *prev = elem;
+ prev = &elem->next;
+ }
+ return head;
+}
+
static void spapr_machine_class_init(ObjectClass *oc, void *data)
{
MachineClass *mc = MACHINE_CLASS(oc);
@@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
hc->plug = spapr_machine_device_plug;
hc->unplug = spapr_machine_device_unplug;
mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
+ mc->cpu_slots = spapr_cpu_slots;
smc->dr_lmb_enabled = true;
smc->dr_cpu_enabled = true;
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 0f30959..d888a02 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
* Set only by old machines because they need to keep
* compatibility on code that exposed QEMU_VERSION to guests in
* the past (and now use qemu_hw_version()).
+ * @cpu_slots:
+ * Provides information about populated and yet-to-be populated
+ * CPU slots in the machine. Used by QMP query-cpu-slots.
*/
struct MachineClass {
/*< private >*/
@@ -99,6 +102,7 @@ struct MachineClass {
HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
DeviceState *dev);
unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
+ CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
};
/**
diff --git a/qapi-schema.json b/qapi-schema.json
index 8d04897..e9a52a2 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4083,3 +4083,88 @@
##
{ 'enum': 'ReplayMode',
'data': [ 'none', 'record', 'play' ] }
+
+##
+# @CPUInfo:
+#
+# Information about CPUs
+#
+# @arch-id: Arch-specific ID for the CPU.
+#
+# @type: QOM type of the CPU.
+#
+# @thread: Thread ID of the CPU.
+#
+# @core: Core ID of the CPU.
+#
+# @socket: Socket ID of the CPU.
+#
+# @node: NUMA node to which the CPU belongs.
+#
+# @qom-path: QOM path of the CPU object
+#
+# Since: 2.6
+##
+
+{ 'struct': 'CPUInfo',
+ 'data': { 'arch-id': 'int',
+ 'type': 'str',
+ '*thread': 'int',
+ '*core': 'int',
+ '*socket' : 'int',
+ '*node' : 'int',
+ '*qom-path': 'str'
+ }
+}
+
+##
+# @CPUSlotInfo:
+#
+# Information about CPU Slots
+#
+# @id: Device ID of the CPU composite object that occupies the slot.
+#
+# @type: QOM type of the CPU composite object that occupies the slot.
+#
+# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
+# can be thread, core or socket.
+#
+# @slot-id: Slot's identifier.
+#
+# @qom-path: QOM path of the CPU composite object that occupies the slot.
+#
+# @realized: A boolean that indicates whether the slot is filled or empty.
+#
+# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
+# this slot.
+#
+# @cpus: An array of @CPUInfo elements where each element describes one
+# CPU that is part of this slot's CPU composite object.
+#
+# @type: QOM type
+#
+# Since: 2.6
+##
+
+{ 'struct': 'CPUSlotInfo',
+ 'data': { '*id': 'str',
+ 'type': 'str',
+ 'hotplug-granularity' : 'str',
+ 'slot-id' : 'str',
+ '*qom-path': 'str',
+ 'realized': 'bool',
+ '*nr-cpus': 'int',
+ '*cpus' : ['CPUInfo']
+ }
+}
+
+##
+# @query-cpu-slots:
+#
+# Returns information for all CPU slots
+#
+# Returns: a list of @CPUSlotInfo
+#
+# Since: 2.6
+##
+{ 'command': 'query-cpu-slots', 'returns': ['CPUSlotInfo'] }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 085dc7d..185aa13 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -4825,3 +4825,50 @@ Example:
{"type": 0, "out-pport": 0, "pport": 0, "vlan-id": 3840,
"pop-vlan": 1, "id": 251658240}
]}
+
+EQMP
+
+ {
+ .name = "query-cpu-slots",
+ .args_type = "",
+ .mhandler.cmd_new = qmp_marshal_query_cpu_slots,
+ },
+
+SQMP
+@query-cpu-slots
+--------------------
+
+Show CPU slots information
+
+Example:
+-> { "execute": "query-cpu-slots" }
+<- {"return": [{
+ "slot-id": "core[2]",
+ "hotplug-granularity": "core",
+ "realized": false,
+ "type": "spapr-cpu-core"
+ },
+ {
+ "slot-id": "core[1]",
+ "qom-path": "/machine/unattached/device[2]",
+ "hotplug-granularity": "core",
+ "realized": true,
+ "type": "spapr-cpu-core"
+ "nr-cpus": 2,
+ "cpus": [
+ {"thread": 8,
+ "socket": 0,
+ "core": 1,
+ "arch-id": 8,
+ "qom-path": "/machine/unattached/device[2]/thread[0]",
+ "node": 0,
+ "type": "host-powerpc64-cpu"},
+ {"thread": 9,
+ "socket": 0,
+ "core": 1,
+ "arch-id": 9,
+ "qom-path": "/machine/unattached/device[2]/thread[2]",
+ "node": 0,
+ "type": "host-powerpc64-cpu"}]
+ }
+ ]}
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [Qemu-devel] [RFC PATCH v0 6/6] hmp: Implement 'info cpu-slots'
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (4 preceding siblings ...)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
@ 2016-02-25 16:22 ` Bharata B Rao
2016-03-01 10:00 ` [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
6 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-25 16:22 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hmp-commands-info.hx | 14 +++++++++++++
hmp.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++
hmp.h | 1 +
3 files changed, 71 insertions(+)
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 9b71351..b67399c 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -786,6 +786,20 @@ STEXI
Display the value of a storage key (s390 only)
ETEXI
+ {
+ .name = "cpu-slots",
+ .args_type = "",
+ .params = "",
+ .help = "show CPU slots",
+ .mhandler.cmd = hmp_info_cpu_slots,
+ },
+
+STEXI
+@item info cpu-slots
+@findex cpu-slots
+Show CPU slots
+ETEXI
+
STEXI
@end table
ETEXI
diff --git a/hmp.c b/hmp.c
index d00c2d4..4bcaa58 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2358,3 +2358,59 @@ void hmp_rocker_of_dpa_groups(Monitor *mon, const QDict *qdict)
qapi_free_RockerOfDpaGroupList(list);
}
+
+void hmp_info_cpu_slots(Monitor *mon, const QDict *qdict)
+{
+ Error *err = NULL;
+ CPUSlotInfoList *cpu_slotinfo_list = qmp_query_cpu_slots(&err);
+ CPUSlotInfoList *s = cpu_slotinfo_list;
+ CPUInfoList *cpu;
+ int i;
+
+ while (s) {
+ monitor_printf(mon, "CPU Slot: \"%s\"\n", s->value->slot_id);
+ monitor_printf(mon, " hotplug granularity: \"%s\"\n",
+ s->value->hotplug_granularity);
+ if (s->value->has_id) {
+ monitor_printf(mon, " dev id: \"%s\"\n", s->value->id);
+ }
+ monitor_printf(mon, " type: \"%s\"\n", s->value->type);
+ monitor_printf(mon, " realized: %s\n",
+ s->value->realized ? "true" : "false");
+ if (s->value->has_qom_path) {
+ monitor_printf(mon, " qom_path: \"%s\"\n", s->value->qom_path);
+ }
+ if (s->value->has_nr_cpus) {
+ monitor_printf(mon, " nr_cpus: %" PRId64 "\n", s->value->nr_cpus);
+ for (i = 0, cpu = s->value->cpus; cpu; cpu = cpu->next, i++) {
+ monitor_printf(mon, " CPU: %" PRId32 "\n", i);
+ monitor_printf(mon, " Type: \"%s\"\n", cpu->value->type);
+ monitor_printf(mon, " Arch ID: %" PRId64 "\n",
+ cpu->value->arch_id);
+ if (cpu->value->has_thread) {
+ monitor_printf(mon, " Thread: %" PRId64 "\n",
+ cpu->value->thread);
+ }
+ if (cpu->value->has_core) {
+ monitor_printf(mon, " Core: %" PRId64 "\n",
+ cpu->value->core);
+ }
+ if (cpu->value->has_core) {
+ monitor_printf(mon, " Socket: %" PRId64 "\n",
+ cpu->value->socket);
+ }
+ if (cpu->value->has_core) {
+ monitor_printf(mon, " Node: %" PRId64 "\n",
+ cpu->value->node);
+ }
+ if (cpu->value->has_qom_path) {
+ monitor_printf(mon, " qom_path: \"%s\"\n",
+ cpu->value->qom_path);
+ }
+ }
+ }
+ s = s->next;
+ }
+
+ qapi_free_CPUSlotInfoList(cpu_slotinfo_list);
+}
diff --git a/hmp.h b/hmp.h
index a8c5b5a..9316201 100644
--- a/hmp.h
+++ b/hmp.h
@@ -131,5 +131,6 @@ void hmp_rocker(Monitor *mon, const QDict *qdict);
void hmp_rocker_ports(Monitor *mon, const QDict *qdict);
void hmp_rocker_of_dpa_flows(Monitor *mon, const QDict *qdict);
void hmp_rocker_of_dpa_groups(Monitor *mon, const QDict *qdict);
+void hmp_info_cpu_slots(Monitor *mon, const QDict *qdict);
#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
@ 2016-02-26 2:57 ` David Gibson
2016-02-26 5:39 ` Bharata B Rao
2016-02-26 10:46 ` Thomas Huth
2016-02-26 18:13 ` Michael Roth
2 siblings, 1 reply; 45+ messages in thread
From: David Gibson @ 2016-02-26 2:57 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 10236 bytes --]
On Thu, Feb 25, 2016 at 09:52:38PM +0530, Bharata B Rao wrote:
> Add sPAPR specific CPU core device that is based on generic CPU core device.
> Creating this core device will result in creation of all the CPU thread
> devices that are part of this core.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Looks good overall, a few minor points below.
> ---
> hw/ppc/Makefile.objs | 1 +
> hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> 3 files changed, 243 insertions(+)
> create mode 100644 hw/ppc/spapr_cpu_core.c
> create mode 100644 include/hw/ppc/spapr_cpu_core.h
>
> diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> index c1ffc77..5cc6608 100644
> --- a/hw/ppc/Makefile.objs
> +++ b/hw/ppc/Makefile.objs
> @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> obj-y += spapr_pci_vfio.o
> endif
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> new file mode 100644
> index 0000000..c44eb61
> --- /dev/null
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -0,0 +1,210 @@
> +/*
> + * sPAPR CPU core device, acts as container of CPU thread devices.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#include "hw/cpu/core.h"
> +#include "hw/ppc/spapr_cpu_core.h"
> +#include "hw/ppc/spapr.h"
> +#include "hw/boards.h"
> +#include "qemu/error-report.h"
> +#include "qapi/visitor.h"
> +#include <sysemu/cpus.h>
> +
> +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> +{
> + Error **errp = opaque;
> +
> + object_property_set_bool(child, true, "realized", errp);
> + if (*errp) {
> + return 1;
> + }
> + return 0;
> +}
> +
> +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + Error *local_err = NULL;
> +
> + if (!core->nr_threads) {
> + error_setg(errp, "nr_threads property can't be 0");
> + return;
> + }
> +
> + if (!core->cpu_model) {
> + error_setg(errp, "cpu_model property isn't set");
> + return;
> + }
> +
> + /*
> + * TODO: If slot isn't specified, plug this core into
> + * an existing empty slot.
> + */
> + if (!core->slot) {
> + error_setg(errp, "slot property isn't set");
> + return;
> + }
> +
> + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> + &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> +}
> +
> +/*
> + * This creates the CPU threads for a given @core.
> + *
> + * In order to create the threads, we need two inputs - number of
> + * threads and the cpu_model. These are set as core object's properties.
> + * When both of them become available/set, this routine will be called from
> + * either property's set handler to create the threads.
> + *
> + * TODO: Dependence of threads creation on two properties is resulting
> + * in this not-so-clean way of creating threads from either of the
> + * property setters based on the order in which they get set. Check if
> + * this can be handled in a better manner.
> + */
> +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> +{
> + int i;
> +
> + for (i = 0; i < core->nr_threads; i++) {
> + char id[32];
> + char type[32];
> +
> + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> + TYPE_POWERPC_CPU);
I think you want to use cpu_class_by_name() rather than explicitly
constructing the class name here.
> + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> +
> + snprintf(id, sizeof(id), "thread[%d]", i);
> + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> + errp);
> + }
> +}
> +
> +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->slot;
> +}
> +
> +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + core->slot = g_strdup(val);
> +}
> +
> +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->cpu_model;
> +}
> +
> +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + MachineState *machine = MACHINE(qdev_get_machine());
> +
> + /*
> + * cpu_model can't be different from what is specified with -cpu
> + */
> + if (strcmp(val, machine->cpu_model)) {
> + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> + return;
> + }
I'm not sure if this is the right place to test this. Possibly it
should be tested in the machine hotplug handler instead.
> + core->cpu_model = g_strdup(val);
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + int64_t value = core->nr_threads;
> +
> + visit_type_int(v, name, &value, errp);
> +}
> +
> +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + Error *local_err = NULL;
> + int64_t value;
> +
> + visit_type_int(v, name, &value, &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + /* Allow only homogeneous configuration */
> + if (value != smp_threads) {
> + error_setg(errp, "nr_threads should be %d", smp_threads);
> + return;
> + }
Likewise, not sure if this is the right place for this check.
> + core->nr_threads = value;
> + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> +
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_instance_init(Object *obj)
> +{
> + object_property_add(obj, "nr_threads", "int",
> + spapr_cpu_core_prop_get_nr_threads,
> + spapr_cpu_core_prop_set_nr_threads,
> + NULL, NULL, NULL);
> + object_property_add_str(obj, "cpu_model",
> + spapr_cpu_core_prop_get_cpu_model,
> + spapr_cpu_core_prop_set_cpu_model,
> + NULL);
> + object_property_add_str(obj, "slot",
> + spapr_cpu_core_prop_get_slot,
> + spapr_cpu_core_prop_set_slot,
> + NULL);
> +}
> +
> +static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
> +{
> + DeviceClass *dc = DEVICE_CLASS(oc);
> +
> + dc->realize = spapr_cpu_core_realize;
> +}
> +
> +static const TypeInfo spapr_cpu_core_type_info = {
> + .name = TYPE_SPAPR_CPU_CORE,
> + .parent = TYPE_CPU_CORE,
> + .instance_init = spapr_cpu_core_instance_init,
> + .instance_size = sizeof(sPAPRCPUCore),
> + .class_init = spapr_cpu_core_class_init,
> +};
> +
> +static void spapr_cpu_core_register_types(void)
> +{
> + type_register_static(&spapr_cpu_core_type_info);
> +}
> +
> +type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> new file mode 100644
> index 0000000..ed9bc7f
> --- /dev/null
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -0,0 +1,32 @@
> +/*
> + * sPAPR CPU core device.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#ifndef HW_SPAPR_CPU_CORE_H
> +#define HW_SPAPR_CPU_CORE_H
> +
> +#include "hw/qdev.h"
> +#include "hw/cpu/core.h"
> +
> +#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
> +#define SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
> +
> +typedef struct sPAPRCPUCore {
> + /*< private >*/
> + DeviceState parent_obj;
> +
> + /*< public >*/
> + int nr_threads;
> + char *cpu_model;
I think this might be better as an ObjectClass *, rather than a
string. It avoids strdup()ing above, and works a bit more nicely with
cpu_class_by_name().
> + char *slot;
> + PowerPCCPU *threads;
> +} sPAPRCPUCore;
> +
> +#define SPAPR_CPU_CORE_SLOT_PROP "slot"
> +
> +#endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices Bharata B Rao
@ 2016-02-26 3:45 ` David Gibson
2016-02-26 4:02 ` Bharata B Rao
2016-02-26 15:18 ` Igor Mammedov
1 sibling, 1 reply; 45+ messages in thread
From: David Gibson @ 2016-02-26 3:45 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 7183 bytes --]
On Thu, Feb 25, 2016 at 09:52:39PM +0530, Bharata B Rao wrote:
> Initialize boot CPUs as spapr-cpu-core devices and create links from
> machine object to these core devices. These links can be considered
> as CPU slots in which core devices will get hot-plugged. spapr-cpu-core
> device's slot property indicates the slot where it is plugged. Information
> about all the CPU slots can be obtained by walking these links.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 65 +++++++++++++++++++++++++++++++++++++++++++-------
> include/hw/ppc/spapr.h | 3 +++
> 2 files changed, 60 insertions(+), 8 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index e214a34..1f0d232 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -64,6 +64,7 @@
>
> #include "hw/compat.h"
> #include "qemu-common.h"
> +#include "hw/ppc/spapr_cpu_core.h"
>
> #include <libfdt.h>
>
> @@ -1720,6 +1721,21 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
> }
> }
>
> +/*
> + * Check to see if core is being hot-plugged into an already populated slot.
> + */
> +static void spapr_cpu_core_allow_set_link(Object *obj, const char *name,
> + Object *val, Error **errp)
> +{
> + Object *core = object_property_get_link(qdev_get_machine(), name, NULL);
> +
> + if (core) {
> + char *path = object_get_canonical_path(core);
> + error_setg(errp, "Slot %s already populated with %s", name, path);
> + g_free(path);
> + }
> +}
> +
> /* pSeries LPAR / sPAPR hardware init */
> static void ppc_spapr_init(MachineState *machine)
> {
> @@ -1728,7 +1744,6 @@ static void ppc_spapr_init(MachineState *machine)
> const char *kernel_filename = machine->kernel_filename;
> const char *kernel_cmdline = machine->kernel_cmdline;
> const char *initrd_filename = machine->initrd_filename;
> - PowerPCCPU *cpu;
> PCIHostState *phb;
> int i;
> MemoryRegion *sysmem = get_system_memory();
> @@ -1742,6 +1757,8 @@ static void ppc_spapr_init(MachineState *machine)
> long load_limit, fw_size;
> bool kernel_le = false;
> char *filename;
> + int spapr_cores = smp_cpus / smp_threads;
> + int spapr_max_cores = max_cpus / smp_threads;
>
> msi_supported = true;
>
> @@ -1800,13 +1817,38 @@ static void ppc_spapr_init(MachineState *machine)
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> }
> - for (i = 0; i < smp_cpus; i++) {
> - cpu = cpu_ppc_init(machine->cpu_model);
> - if (cpu == NULL) {
> - error_report("Unable to find PowerPC CPU definition");
> - exit(1);
> +
> + spapr->cores = g_malloc0(spapr_max_cores * sizeof(Object));
> +
> + for (i = 0; i < spapr_max_cores; i++) {
> + Object *spapr_cpu_core = object_new(TYPE_SPAPR_CPU_CORE);
So.. if I understand correctly this will always construct maxcpus
threads at startup. I's just that the ones beyond initial cpus won't
be realized. Was that your intention? I thought the plan was to only
construct the hotplugged cpus when they were hotplugged (in contrast
to my earlier proposal).
> + char name[32];
> +
> + object_property_set_str(spapr_cpu_core, machine->cpu_model, "cpu_model",
> + &error_fatal);
> + object_property_set_int(spapr_cpu_core, smp_threads, "nr_threads",
> + &error_fatal);
> + /*
> + * Create links from machine objects to all possible cores.
> + */
> + snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
Why do SPAPR_MACHINE_CPU_CORE_PROP and SPAPR_CPU_CORE_SLOT_PROP get
#defines, but the other properties get bare strings? I find the bare
strings are usually easier to follow.
> + object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
> + (Object **)&spapr->cores[i],
> + spapr_cpu_core_allow_set_link, 0,
> + &error_fatal);
> +
> + /*
> + * Set the link from machine object to core object for all
> + * boot time CPUs specified with -smp and realize them.
> + * For rest of the hotpluggable cores this is happens from
> + * the core hotplug/realization path.
> + */
> + if (i < spapr_cores) {
> + object_property_set_str(spapr_cpu_core, name,
> + SPAPR_CPU_CORE_SLOT_PROP, &error_fatal);
> + object_property_set_bool(spapr_cpu_core, true, "realized",
> + &error_fatal);
> }
> - spapr_cpu_init(spapr, cpu, &error_fatal);
> }
>
> if (kvm_enabled()) {
> @@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
>
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> int node;
> @@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
>
> spapr_memory_plug(hotplug_dev, dev, node, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + CPUState *cs = CPU(dev);
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> +
> + spapr_cpu_init(ms, cpu, errp);
I tend to thin the spapr_cpu_init() should be done from the core's
create_threads() function, but I'm willing to be persuaded otherwise.
> }
> }
>
> @@ -2259,7 +2307,8 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> - if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 098d85d..20b3417 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -79,8 +79,11 @@ struct sPAPRMachineState {
> /*< public >*/
> char *kvm_type;
> MemoryHotplugState hotplug_memory;
> + Object *cores;
> };
>
> +#define SPAPR_MACHINE_CPU_CORE_PROP "core"
> +
> #define H_SUCCESS 0
> #define H_BUSY 1 /* Hardware busy -- retry later */
> #define H_CLOSED 2 /* Resource closed */
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support Bharata B Rao
@ 2016-02-26 3:51 ` David Gibson
2016-02-29 4:42 ` Bharata B Rao
2016-02-26 13:03 ` Thomas Huth
1 sibling, 1 reply; 45+ messages in thread
From: David Gibson @ 2016-02-26 3:51 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 11567 bytes --]
On Thu, Feb 25, 2016 at 09:52:40PM +0530, Bharata B Rao wrote:
> Set up device tree entries for the hotplugged CPU core and use the
> exising EPOW event infrastructure to send CPU hotplug notification to
> the guest.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++-
> hw/ppc/spapr_events.c | 3 ++
> hw/ppc/spapr_rtas.c | 24 +++++++++
> include/hw/ppc/spapr.h | 1 +
> 4 files changed, 163 insertions(+), 1 deletion(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1f0d232..780cd00 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> size_t page_sizes_prop_size;
> uint32_t vcpus_per_socket = smp_threads * smp_cores;
> uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + int drc_index;
> +
> + if (smc->dr_cpu_enabled) {
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> + g_assert(drc);
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drc_index = drck->get_index(drc);
> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> + }
>
> /* Note: we keep CI large pages off for now because a 64K capable guest
> * provisioned with large pages might otherwise try to map a qemu
> @@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> }
>
> + if (smc->dr_cpu_enabled) {
> + int offset = fdt_path_offset(fdt, "/cpus");
> + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> + SPAPR_DR_CONNECTOR_TYPE_CPU);
> + if (ret < 0) {
> + fprintf(stderr, "Couldn't set up CPU DR device tree properties\n");
> + exit(1);
> + }
> + }
> +
> _FDT((fdt_pack(fdt)));
>
> if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> @@ -1759,6 +1781,7 @@ static void ppc_spapr_init(MachineState *machine)
> char *filename;
> int spapr_cores = smp_cpus / smp_threads;
> int spapr_max_cores = max_cpus / smp_threads;
> + int smt = kvmppc_smt_threads();
>
> msi_supported = true;
>
> @@ -1813,6 +1836,15 @@ static void ppc_spapr_init(MachineState *machine)
> spapr_validate_node_memory(machine, &error_fatal);
> }
>
> + if (smc->dr_cpu_enabled) {
> + for (i = 0; i < spapr_max_cores; i++) {
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_new(OBJECT(spapr),
> + SPAPR_DR_CONNECTOR_TYPE_CPU, i * smt);
> + qemu_register_reset(spapr_drc_reset, drc);
> + }
> + }
> +
> /* init CPUs */
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> @@ -2247,6 +2279,88 @@ out:
> error_propagate(errp, local_err);
> }
>
> +static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs,
> + int *fdt_offset,
> + sPAPRMachineState *spapr)
> +{
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + void *fdt;
> + int offset, fdt_size;
> + char *nodename;
> +
> + fdt = create_device_tree(&fdt_size);
> + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> + offset = fdt_add_subnode(fdt, 0, nodename);
> +
> + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> + g_free(nodename);
> +
> + *fdt_offset = offset;
> + return fdt;
> +}
> +
> +static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + PowerPCCPU *cpu = &core->threads[0];
> + CPUState *cs = CPU(cpu);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> + void *fdt = NULL;
> + int fdt_offset = 0;
> +
> + if (!smc->dr_cpu_enabled) {
> + /*
> + * This is a cold plugged CPU core but the machine doesn't support
> + * DR. So skip the hotplug path ensuring that the core is brought
> + * up online with out an associated DR connector.
> + */
> + return;
> + }
> +
> + g_assert(drc);
> +
> + /*
> + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> + */
> + if (dev->hotplugged) {
> + fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms);
> + dev->hotplugged = true;
> + }
> +
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
> + if (local_err) {
> + g_free(fdt);
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + if (dev->hotplugged) {
> + /*
> + * Send hotplug notification interrupt to the guest only in case
> + * of hotplugged CPUs.
> + */
> + spapr_hotplug_req_add_by_index(drc);
> + } else {
> + /*
> + * Set the right DRC states for cold plugged CPU.
> + */
> + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
> + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
> + }
> + return;
> +}
> +
> static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> @@ -2291,8 +2405,25 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> CPUState *cs = CPU(dev);
> PowerPCCPU *cpu = POWERPC_CPU(cs);
> + int i;
> +
> + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> + error_setg(errp, "CPU hotplug not supported for this machine");
> + return;
> + }
> +
> + /* Set NUMA node for the added CPUs */
> + for (i = 0; i < nb_numa_nodes; i++) {
> + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> + cs->numa_node = i;
> + break;
> + }
> + }
>
> spapr_cpu_init(ms, cpu, errp);
> + spapr_cpu_reset(cpu);
It looks to me like all of this setup should be either in the last
patch, or in this one, not split between them.
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_plug(hotplug_dev, dev, errp);
> }
> }
>
> @@ -2308,7 +2439,8 @@ static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> - object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + object_dynamic_cast(OBJECT(dev), TYPE_CPU) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> @@ -2352,6 +2484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
>
> smc->dr_lmb_enabled = true;
> + smc->dr_cpu_enabled = true;
> fwc->get_dev_path = spapr_get_fw_dev_path;
> nc->nmi_monitor_handler = spapr_nmi;
> }
> @@ -2431,6 +2564,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
>
> spapr_machine_2_6_class_options(mc);
> smc->use_ohci_by_default = true;
> + smc->dr_cpu_enabled = false;
> SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
> }
>
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index f5eac4b..e50bb16 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
> case SPAPR_DR_CONNECTOR_TYPE_LMB:
> hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
> break;
> + case SPAPR_DR_CONNECTOR_TYPE_CPU:
> + hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
> + break;
> default:
> /* we shouldn't be signaling hotplug events for resources
> * that don't support them
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index b7c5ebd..cc0369e 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -34,6 +34,7 @@
>
> #include "hw/ppc/spapr.h"
> #include "hw/ppc/spapr_vio.h"
> +#include "hw/ppc/ppc.h"
> #include "qapi-event.h"
> #include "hw/boards.h"
>
> @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> }
>
> +/*
> + * Set the timebase offset of the CPU to that of first CPU.
> + * This helps hotplugged CPU to have the correct timebase offset.
> + */
> +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> +
> + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> +}
> +
> +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> +
> + if (!pcc->interrupts_big_endian(fcpu)) {
> + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> + }
> +}
> +
Any particular reason for doing these things at rtas_start_cpu() time,
but other initialization at plug time? Could you consolidate it to
one place or the other?
> static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> uint32_t token, uint32_t nargs,
> target_ulong args,
> @@ -197,6 +219,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> env->nip = start;
> env->gpr[3] = r3;
> cs->halted = 0;
> + spapr_cpu_set_endianness(cpu);
> + spapr_cpu_update_tb_offset(cpu);
>
> qemu_cpu_kick(cs);
>
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 20b3417..eb4ccd1 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -36,6 +36,7 @@ struct sPAPRMachineClass {
>
> /*< public >*/
> bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */
> + bool dr_cpu_enabled; /* enable dynamic-reconfig/hotplug of CPUs */
> bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */
> };
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-26 3:45 ` David Gibson
@ 2016-02-26 4:02 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-26 4:02 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
On Fri, Feb 26, 2016 at 02:45:35PM +1100, David Gibson wrote:
> On Thu, Feb 25, 2016 at 09:52:39PM +0530, Bharata B Rao wrote:
> > Initialize boot CPUs as spapr-cpu-core devices and create links from
> > machine object to these core devices. These links can be considered
> > as CPU slots in which core devices will get hot-plugged. spapr-cpu-core
> > device's slot property indicates the slot where it is plugged. Information
> > about all the CPU slots can be obtained by walking these links.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 65 +++++++++++++++++++++++++++++++++++++++++++-------
> > include/hw/ppc/spapr.h | 3 +++
> > 2 files changed, 60 insertions(+), 8 deletions(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index e214a34..1f0d232 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -64,6 +64,7 @@
> >
> > #include "hw/compat.h"
> > #include "qemu-common.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> >
> > #include <libfdt.h>
> >
> > @@ -1720,6 +1721,21 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
> > }
> > }
> >
> > +/*
> > + * Check to see if core is being hot-plugged into an already populated slot.
> > + */
> > +static void spapr_cpu_core_allow_set_link(Object *obj, const char *name,
> > + Object *val, Error **errp)
> > +{
> > + Object *core = object_property_get_link(qdev_get_machine(), name, NULL);
> > +
> > + if (core) {
> > + char *path = object_get_canonical_path(core);
> > + error_setg(errp, "Slot %s already populated with %s", name, path);
> > + g_free(path);
> > + }
> > +}
> > +
> > /* pSeries LPAR / sPAPR hardware init */
> > static void ppc_spapr_init(MachineState *machine)
> > {
> > @@ -1728,7 +1744,6 @@ static void ppc_spapr_init(MachineState *machine)
> > const char *kernel_filename = machine->kernel_filename;
> > const char *kernel_cmdline = machine->kernel_cmdline;
> > const char *initrd_filename = machine->initrd_filename;
> > - PowerPCCPU *cpu;
> > PCIHostState *phb;
> > int i;
> > MemoryRegion *sysmem = get_system_memory();
> > @@ -1742,6 +1757,8 @@ static void ppc_spapr_init(MachineState *machine)
> > long load_limit, fw_size;
> > bool kernel_le = false;
> > char *filename;
> > + int spapr_cores = smp_cpus / smp_threads;
> > + int spapr_max_cores = max_cpus / smp_threads;
> >
> > msi_supported = true;
> >
> > @@ -1800,13 +1817,38 @@ static void ppc_spapr_init(MachineState *machine)
> > if (machine->cpu_model == NULL) {
> > machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> > }
> > - for (i = 0; i < smp_cpus; i++) {
> > - cpu = cpu_ppc_init(machine->cpu_model);
> > - if (cpu == NULL) {
> > - error_report("Unable to find PowerPC CPU definition");
> > - exit(1);
> > +
> > + spapr->cores = g_malloc0(spapr_max_cores * sizeof(Object));
> > +
> > + for (i = 0; i < spapr_max_cores; i++) {
> > + Object *spapr_cpu_core = object_new(TYPE_SPAPR_CPU_CORE);
>
> So.. if I understand correctly this will always construct maxcpus
> threads at startup. I's just that the ones beyond initial cpus won't
> be realized. Was that your intention? I thought the plan was to only
> construct the hotplugged cpus when they were hotplugged (in contrast
> to my earlier proposal).
Oops! The intention was to to create only cores for smp_cpus
and let device_add create the rest. Will fix.
>
> > + char name[32];
> > +
> > + object_property_set_str(spapr_cpu_core, machine->cpu_model, "cpu_model",
> > + &error_fatal);
> > + object_property_set_int(spapr_cpu_core, smp_threads, "nr_threads",
> > + &error_fatal);
> > + /*
> > + * Create links from machine objects to all possible cores.
> > + */
> > + snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
>
> Why do SPAPR_MACHINE_CPU_CORE_PROP and SPAPR_CPU_CORE_SLOT_PROP get
> #defines, but the other properties get bare strings? I find the bare
> strings are usually easier to follow.
Let me check the convention regarding properties and stick to the
common usage.
>
> > + object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
> > + (Object **)&spapr->cores[i],
> > + spapr_cpu_core_allow_set_link, 0,
> > + &error_fatal);
> > +
> > + /*
> > + * Set the link from machine object to core object for all
> > + * boot time CPUs specified with -smp and realize them.
> > + * For rest of the hotpluggable cores this is happens from
> > + * the core hotplug/realization path.
> > + */
> > + if (i < spapr_cores) {
> > + object_property_set_str(spapr_cpu_core, name,
> > + SPAPR_CPU_CORE_SLOT_PROP, &error_fatal);
> > + object_property_set_bool(spapr_cpu_core, true, "realized",
> > + &error_fatal);
> > }
> > - spapr_cpu_init(spapr, cpu, &error_fatal);
> > }
> >
> > if (kvm_enabled()) {
> > @@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > DeviceState *dev, Error **errp)
> > {
> > sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
> >
> > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> > int node;
> > @@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > }
> >
> > spapr_memory_plug(hotplug_dev, dev, node, errp);
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > + CPUState *cs = CPU(dev);
> > + PowerPCCPU *cpu = POWERPC_CPU(cs);
> > +
> > + spapr_cpu_init(ms, cpu, errp);
>
> I tend to thin the spapr_cpu_init() should be done from the core's
> create_threads() function, but I'm willing to be persuaded otherwise.
In addition to core device, every cpu thread device will go through
->plug() invocation via its realization path. And spapr_cpu_init()
is needed for every CPU thread. Let me check if it is possible to
invoke this from core's routines.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
@ 2016-02-26 4:03 ` David Gibson
2016-02-26 9:40 ` Bharata B Rao
2016-02-26 15:58 ` Eric Blake
` (2 subsequent siblings)
3 siblings, 1 reply; 45+ messages in thread
From: David Gibson @ 2016-02-26 4:03 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 12654 bytes --]
On Thu, Feb 25, 2016 at 09:52:41PM +0530, Bharata B Rao wrote:
> Implement query cpu-slots that provides information about hot-plugged
> as well as hot-pluggable CPU slots that the machine supports.
>
> TODO: As Eric suggested use enum for type instead of str.
> TODO: @hotplug-granularity probably isn't required.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/core/machine.c | 19 +++++++++
> hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> include/hw/boards.h | 4 ++
> qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> qmp-commands.hx | 47 ++++++++++++++++++++++
> 5 files changed, 267 insertions(+)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 6d1a0d8..3055ef8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -17,6 +17,25 @@
> #include "hw/sysbus.h"
> #include "sysemu/sysemu.h"
> #include "qemu/error-report.h"
> +#include "qmp-commands.h"
> +
> +/*
> + * QMP: query-cpu-slots
> + *
> + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> + */
> +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> +{
> + MachineState *ms = MACHINE(qdev_get_machine());
> + MachineClass *mc = MACHINE_GET_CLASS(ms);
> +
> + if (!mc->cpu_slots) {
> + error_setg(errp, QERR_UNSUPPORTED);
> + return NULL;
> + }
> +
> + return mc->cpu_slots(ms);
> +}
>
> static char *machine_get_accel(Object *obj, Error **errp)
> {
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 780cd00..b76ed85 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> return cpu_index / smp_threads / smp_cores;
> }
>
> +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> +{
> + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> + CPUInfoList ***prev = opaque;
> +
> + if (object_dynamic_cast(obj, TYPE_CPU)) {
> + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> + CPUInfo *s = g_new0(CPUInfo, 1);
> + CPUState *cpu = CPU(obj);
> + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
Since you're assuming a POWERPC_CPU here, you should probably check
that in the object_dynamic_cast() above.
> +
> + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> + s->type = g_strdup(object_get_typename(obj));
> + s->thread = cpu->cpu_index;
> + s->has_thread = true;
Unlike Igor's patch, you are attaching this thread information to,
well, a thread, not a cpu slot. So I don't think the has_thread,
has_core etc. booleans are useful any more.
> + s->core = cpu->cpu_index / smp_threads;
> + s->has_core = true;
> + if (mc->cpu_index_to_socket_id) {
> + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> + } else {
> + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> + }
> + s->has_socket = true;
> + s->node = cpu->numa_node;
> + s->has_node = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> +
> + elem->value = s;
> + elem->next = NULL;
> + **prev = elem;
> + *prev = &elem->next;
Hmm.. the dt_id is the only power specific thing, here, wonder if
there's a way we could make this generic.
> + }
> + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
Hmm.. IIUC this can be called either on a core, or on a thread, with
somewhat different behaviour. That seems dangerously subtle.
> + return 0;
> +}
> +
> +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> +{
> + CPUSlotInfoList *head = NULL;
> + CPUSlotInfoList **prev = &head;
> + Object *root_container;
> + ObjectProperty *prop;
> + ObjectPropertyIterator iter;
> +
> + /*
> + * TODO: There surely must be a better/easier way to walk all
> + * the link properties of an object ?
> + */
> + root_container = container_get(object_get_root(), "/machine");
> + object_property_iter_init(&iter, root_container);
> +
> + while ((prop = object_property_iter_next(&iter))) {
> + Object *obj;
> + DeviceState *dev;
> + CPUSlotInfoList *elem;
> + CPUSlotInfo *s;
> + CPUInfoList *cpu_head = NULL;
> + CPUInfoList **cpu_prev = &cpu_head;
> + sPAPRCPUCore *core;
> +
> + if (!strstart(prop->type, "link<", NULL)) {
> + continue;
> + }
Hmm.. we really out to have an is_link() helper, but that's not your
problem.
> + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> + continue;
> + }
> +
> + elem = g_new0(CPUSlotInfoList, 1);
> + s = g_new0(CPUSlotInfo, 1);
> +
> + obj = object_property_get_link(root_container, prop->name, NULL);
> + if (obj) {
> + /* Slot populated */
> + dev = DEVICE(obj);
> + core = SPAPR_CPU_CORE(obj);
> +
> + if (dev->id) {
> + s->has_id = true;
> + s->id = g_strdup(dev->id);
> + }
> + s->realized = object_property_get_bool(obj, "realized", NULL);
> + s->nr_cpus = core->nr_threads;
> + s->has_nr_cpus = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> + if (s->realized) {
> + spapr_cpuinfo_list(obj, &cpu_prev);
> + }
> + s->has_cpus = true;
> + } else {
> + /* Slot empty */
> + s->has_id = false;
> + s->has_nr_cpus = false;
> + s->has_qom_path = false;
> + s->has_cpus = false;
> + s->realized = false;
Um.. doesn't this leave out the cpu model and nr_threads for a
non-populated slot? Which is crucial information for mgmt to populate
the slot.
I can't actually see that this interface gives you any information you
can't already see from the qom tree.
> + }
> + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> + s->slot_id = g_strdup(prop->name);
> + s->cpus = cpu_head;
> + elem->value = s;
> + elem->next = NULL;
> + *prev = elem;
> + prev = &elem->next;
I also wonder if there's a way to use the visit_* interfaces to avoid
having to fully construct this list-of-lists.
I can't see any code that frees the info data structure either...
> + }
> + return head;
> +}
> +
> static void spapr_machine_class_init(ObjectClass *oc, void *data)
> {
> MachineClass *mc = MACHINE_CLASS(oc);
> @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> hc->plug = spapr_machine_device_plug;
> hc->unplug = spapr_machine_device_unplug;
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> + mc->cpu_slots = spapr_cpu_slots;
>
> smc->dr_lmb_enabled = true;
> smc->dr_cpu_enabled = true;
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 0f30959..d888a02 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> * Set only by old machines because they need to keep
> * compatibility on code that exposed QEMU_VERSION to guests in
> * the past (and now use qemu_hw_version()).
> + * @cpu_slots:
> + * Provides information about populated and yet-to-be populated
> + * CPU slots in the machine. Used by QMP query-cpu-slots.
> */
> struct MachineClass {
> /*< private >*/
> @@ -99,6 +102,7 @@ struct MachineClass {
> HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> DeviceState *dev);
> unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> };
>
> /**
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 8d04897..e9a52a2 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -4083,3 +4083,88 @@
> ##
> { 'enum': 'ReplayMode',
> 'data': [ 'none', 'record', 'play' ] }
> +
> +##
> +# @CPUInfo:
> +#
> +# Information about CPUs
> +#
> +# @arch-id: Arch-specific ID for the CPU.
> +#
> +# @type: QOM type of the CPU.
> +#
> +# @thread: Thread ID of the CPU.
> +#
> +# @core: Core ID of the CPU.
> +#
> +# @socket: Socket ID of the CPU.
> +#
> +# @node: NUMA node to which the CPU belongs.
> +#
> +# @qom-path: QOM path of the CPU object
> +#
> +# Since: 2.6
> +##
> +
> +{ 'struct': 'CPUInfo',
> + 'data': { 'arch-id': 'int',
> + 'type': 'str',
> + '*thread': 'int',
> + '*core': 'int',
> + '*socket' : 'int',
> + '*node' : 'int',
> + '*qom-path': 'str'
> + }
> +}
> +
> +##
> +# @CPUSlotInfo:
> +#
> +# Information about CPU Slots
> +#
> +# @id: Device ID of the CPU composite object that occupies the slot.
> +#
> +# @type: QOM type of the CPU composite object that occupies the slot.
> +#
> +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> +# can be thread, core or socket.
> +#
> +# @slot-id: Slot's identifier.
> +#
> +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> +#
> +# @realized: A boolean that indicates whether the slot is filled or empty.
> +#
> +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> +# this slot.
> +#
> +# @cpus: An array of @CPUInfo elements where each element describes one
> +# CPU that is part of this slot's CPU composite object.
> +#
> +# @type: QOM type
> +#
> +# Since: 2.6
> +##
> +
> +{ 'struct': 'CPUSlotInfo',
> + 'data': { '*id': 'str',
> + 'type': 'str',
> + 'hotplug-granularity' : 'str',
> + 'slot-id' : 'str',
> + '*qom-path': 'str',
> + 'realized': 'bool',
> + '*nr-cpus': 'int',
> + '*cpus' : ['CPUInfo']
> + }
> +}
> +
> +##
> +# @query-cpu-slots:
> +#
> +# Returns information for all CPU slots
> +#
> +# Returns: a list of @CPUSlotInfo
> +#
> +# Since: 2.6
> +##
> +{ 'command': 'query-cpu-slots', 'returns': ['CPUSlotInfo'] }
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index 085dc7d..185aa13 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -4825,3 +4825,50 @@ Example:
> {"type": 0, "out-pport": 0, "pport": 0, "vlan-id": 3840,
> "pop-vlan": 1, "id": 251658240}
> ]}
> +
> +EQMP
> +
> + {
> + .name = "query-cpu-slots",
> + .args_type = "",
> + .mhandler.cmd_new = qmp_marshal_query_cpu_slots,
> + },
> +
> +SQMP
> +@query-cpu-slots
> +--------------------
> +
> +Show CPU slots information
> +
> +Example:
> +-> { "execute": "query-cpu-slots" }
> +<- {"return": [{
> + "slot-id": "core[2]",
> + "hotplug-granularity": "core",
> + "realized": false,
> + "type": "spapr-cpu-core"
> + },
> + {
> + "slot-id": "core[1]",
> + "qom-path": "/machine/unattached/device[2]",
> + "hotplug-granularity": "core",
> + "realized": true,
> + "type": "spapr-cpu-core"
> + "nr-cpus": 2,
> + "cpus": [
> + {"thread": 8,
> + "socket": 0,
> + "core": 1,
> + "arch-id": 8,
> + "qom-path": "/machine/unattached/device[2]/thread[0]",
> + "node": 0,
> + "type": "host-powerpc64-cpu"},
> + {"thread": 9,
> + "socket": 0,
> + "core": 1,
> + "arch-id": 9,
> + "qom-path": "/machine/unattached/device[2]/thread[2]",
> + "node": 0,
> + "type": "host-powerpc64-cpu"}]
> + }
> + ]}
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-26 2:57 ` David Gibson
@ 2016-02-26 5:39 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-26 5:39 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
On Fri, Feb 26, 2016 at 01:57:44PM +1100, David Gibson wrote:
> On Thu, Feb 25, 2016 at 09:52:38PM +0530, Bharata B Rao wrote:
> > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > Creating this core device will result in creation of all the CPU thread
> > devices that are part of this core.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>
> Looks good overall, a few minor points below.
>
> > ---
> > hw/ppc/Makefile.objs | 1 +
> > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > 3 files changed, 243 insertions(+)
> > create mode 100644 hw/ppc/spapr_cpu_core.c
> > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> >
> > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > index c1ffc77..5cc6608 100644
> > --- a/hw/ppc/Makefile.objs
> > +++ b/hw/ppc/Makefile.objs
> > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > obj-y += spapr_pci_vfio.o
> > endif
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > new file mode 100644
> > index 0000000..c44eb61
> > --- /dev/null
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -0,0 +1,210 @@
> > +/*
> > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > + *
> > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#include "hw/cpu/core.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> > +#include "hw/ppc/spapr.h"
> > +#include "hw/boards.h"
> > +#include "qemu/error-report.h"
> > +#include "qapi/visitor.h"
> > +#include <sysemu/cpus.h>
> > +
> > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > +{
> > + Error **errp = opaque;
> > +
> > + object_property_set_bool(child, true, "realized", errp);
> > + if (*errp) {
> > + return 1;
> > + }
> > + return 0;
> > +}
> > +
> > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > + Error *local_err = NULL;
> > +
> > + if (!core->nr_threads) {
> > + error_setg(errp, "nr_threads property can't be 0");
> > + return;
> > + }
> > +
> > + if (!core->cpu_model) {
> > + error_setg(errp, "cpu_model property isn't set");
> > + return;
> > + }
> > +
> > + /*
> > + * TODO: If slot isn't specified, plug this core into
> > + * an existing empty slot.
> > + */
> > + if (!core->slot) {
> > + error_setg(errp, "slot property isn't set");
> > + return;
> > + }
> > +
> > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > + &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > +}
> > +
> > +/*
> > + * This creates the CPU threads for a given @core.
> > + *
> > + * In order to create the threads, we need two inputs - number of
> > + * threads and the cpu_model. These are set as core object's properties.
> > + * When both of them become available/set, this routine will be called from
> > + * either property's set handler to create the threads.
> > + *
> > + * TODO: Dependence of threads creation on two properties is resulting
> > + * in this not-so-clean way of creating threads from either of the
> > + * property setters based on the order in which they get set. Check if
> > + * this can be handled in a better manner.
> > + */
> > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < core->nr_threads; i++) {
> > + char id[32];
> > + char type[32];
> > +
> > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > + TYPE_POWERPC_CPU);
>
> I think you want to use cpu_class_by_name() rather than explicitly
> constructing the class name here.
Good point, started using ObjectClass.
>
> > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > +
> > + snprintf(id, sizeof(id), "thread[%d]", i);
> > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > + errp);
> > + }
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->slot;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + core->slot = g_strdup(val);
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->cpu_model;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + MachineState *machine = MACHINE(qdev_get_machine());
> > +
> > + /*
> > + * cpu_model can't be different from what is specified with -cpu
> > + */
> > + if (strcmp(val, machine->cpu_model)) {
> > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > + return;
> > + }
>
> I'm not sure if this is the right place to test this. Possibly it
> should be tested in the machine hotplug handler instead.
Not sure about the right place to validate this and nr_threads property. But
it essentially boils down to backing out from property setter after
object init vs backing out during realization.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-26 4:03 ` David Gibson
@ 2016-02-26 9:40 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-26 9:40 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
David, I am responding to just one comment here and will respond to others
in detail later.
On Fri, Feb 26, 2016 at 03:03:17PM +1100, David Gibson wrote:
>
> > + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> > + continue;
> > + }
> > +
> > + elem = g_new0(CPUSlotInfoList, 1);
> > + s = g_new0(CPUSlotInfo, 1);
> > +
> > + obj = object_property_get_link(root_container, prop->name, NULL);
> > + if (obj) {
> > + /* Slot populated */
> > + dev = DEVICE(obj);
> > + core = SPAPR_CPU_CORE(obj);
> > +
> > + if (dev->id) {
> > + s->has_id = true;
> > + s->id = g_strdup(dev->id);
> > + }
> > + s->realized = object_property_get_bool(obj, "realized", NULL);
> > + s->nr_cpus = core->nr_threads;
> > + s->has_nr_cpus = true;
> > + s->qom_path = object_get_canonical_path(obj);
> > + s->has_qom_path = true;
> > + if (s->realized) {
> > + spapr_cpuinfo_list(obj, &cpu_prev);
> > + }
> > + s->has_cpus = true;
> > + } else {
> > + /* Slot empty */
> > + s->has_id = false;
> > + s->has_nr_cpus = false;
> > + s->has_qom_path = false;
> > + s->has_cpus = false;
> > + s->realized = false;
>
> Um.. doesn't this leave out the cpu model and nr_threads for a
> non-populated slot? Which is crucial information for mgmt to populate
> the slot.
Actually if we don't allow heterogenous configurations, management currently
knows how to get model and threads that have already been specified with
-cpu and -smp global options respectively.
If we want to support heterogenous configurations, should we then have
supported model types and thread values listed here ? At the moment I
am not sure what those supported types and thread values could be.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
2016-02-26 2:57 ` David Gibson
@ 2016-02-26 10:46 ` Thomas Huth
2016-02-29 5:39 ` Bharata B Rao
2016-02-26 18:13 ` Michael Roth
2 siblings, 1 reply; 45+ messages in thread
From: Thomas Huth @ 2016-02-26 10:46 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, pkrempa, ehabkost, aik, armbru, agraf, borntraeger,
qemu-ppc, pbonzini, imammedo, mdroth, afaerber, david
On 25.02.2016 17:22, Bharata B Rao wrote:
> Add sPAPR specific CPU core device that is based on generic CPU core device.
> Creating this core device will result in creation of all the CPU thread
> devices that are part of this core.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
...
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> new file mode 100644
> index 0000000..c44eb61
> --- /dev/null
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -0,0 +1,210 @@
> +/*
> + * sPAPR CPU core device, acts as container of CPU thread devices.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#include "hw/cpu/core.h"
> +#include "hw/ppc/spapr_cpu_core.h"
> +#include "hw/ppc/spapr.h"
> +#include "hw/boards.h"
> +#include "qemu/error-report.h"
> +#include "qapi/visitor.h"
> +#include <sysemu/cpus.h>
> +
> +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> +{
> + Error **errp = opaque;
> +
> + object_property_set_bool(child, true, "realized", errp);
> + if (*errp) {
> + return 1;
> + }
> + return 0;
> +}
> +
> +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + Error *local_err = NULL;
> +
> + if (!core->nr_threads) {
> + error_setg(errp, "nr_threads property can't be 0");
> + return;
> + }
> +
> + if (!core->cpu_model) {
> + error_setg(errp, "cpu_model property isn't set");
> + return;
> + }
> +
> + /*
> + * TODO: If slot isn't specified, plug this core into
> + * an existing empty slot.
> + */
> + if (!core->slot) {
> + error_setg(errp, "slot property isn't set");
> + return;
> + }
> +
> + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> + &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> +}
> +
> +/*
> + * This creates the CPU threads for a given @core.
> + *
> + * In order to create the threads, we need two inputs - number of
> + * threads and the cpu_model. These are set as core object's properties.
> + * When both of them become available/set, this routine will be called from
> + * either property's set handler to create the threads.
> + *
> + * TODO: Dependence of threads creation on two properties is resulting
> + * in this not-so-clean way of creating threads from either of the
> + * property setters based on the order in which they get set. Check if
> + * this can be handled in a better manner.
> + */
> +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> +{
> + int i;
> +
> + for (i = 0; i < core->nr_threads; i++) {
> + char id[32];
> + char type[32];
> +
> + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> + TYPE_POWERPC_CPU);
> + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> +
> + snprintf(id, sizeof(id), "thread[%d]", i);
> + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> + errp);
Need to check errp here to see whether something went wrong?
> + }
> +}
> +
> +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->slot;
> +}
> +
> +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + core->slot = g_strdup(val);
> +}
> +
> +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->cpu_model;
> +}
> +
> +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + MachineState *machine = MACHINE(qdev_get_machine());
> +
> + /*
> + * cpu_model can't be different from what is specified with -cpu
> + */
> + if (strcmp(val, machine->cpu_model)) {
> + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
s/should/must/
> + return;
> + }
> +
> + core->cpu_model = g_strdup(val);
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + int64_t value = core->nr_threads;
> +
> + visit_type_int(v, name, &value, errp);
> +}
> +
> +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + Error *local_err = NULL;
> + int64_t value;
> +
> + visit_type_int(v, name, &value, &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + /* Allow only homogeneous configuration */
> + if (value != smp_threads) {
> + error_setg(errp, "nr_threads should be %d", smp_threads);
s/should/must/
> + return;
> + }
> +
> + core->nr_threads = value;
> + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
I think it's preferable to use g_new0 for such allocations instead.
Also, should this memory maybe be freed during instance_finalize again,
so that there is no memory leak here in case the cores are deleted again
one day?
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_instance_init(Object *obj)
> +{
> + object_property_add(obj, "nr_threads", "int",
> + spapr_cpu_core_prop_get_nr_threads,
> + spapr_cpu_core_prop_set_nr_threads,
> + NULL, NULL, NULL);
> + object_property_add_str(obj, "cpu_model",
> + spapr_cpu_core_prop_get_cpu_model,
> + spapr_cpu_core_prop_set_cpu_model,
> + NULL);
> + object_property_add_str(obj, "slot",
> + spapr_cpu_core_prop_get_slot,
> + spapr_cpu_core_prop_set_slot,
> + NULL);
> +}
> +
> +static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
> +{
> + DeviceClass *dc = DEVICE_CLASS(oc);
> +
> + dc->realize = spapr_cpu_core_realize;
> +}
> +
> +static const TypeInfo spapr_cpu_core_type_info = {
> + .name = TYPE_SPAPR_CPU_CORE,
> + .parent = TYPE_CPU_CORE,
> + .instance_init = spapr_cpu_core_instance_init,
> + .instance_size = sizeof(sPAPRCPUCore),
> + .class_init = spapr_cpu_core_class_init,
> +};
> +
> +static void spapr_cpu_core_register_types(void)
> +{
> + type_register_static(&spapr_cpu_core_type_info);
> +}
> +
> +type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> new file mode 100644
> index 0000000..ed9bc7f
> --- /dev/null
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -0,0 +1,32 @@
> +/*
> + * sPAPR CPU core device.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#ifndef HW_SPAPR_CPU_CORE_H
> +#define HW_SPAPR_CPU_CORE_H
> +
> +#include "hw/qdev.h"
> +#include "hw/cpu/core.h"
> +
> +#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
> +#define SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
> +
> +typedef struct sPAPRCPUCore {
> + /*< private >*/
> + DeviceState parent_obj;
> +
> + /*< public >*/
> + int nr_threads;
> + char *cpu_model;
> + char *slot;
<bikeshedpainting>
I'd maybe call that "slot_name" instead ... if you only call it "slot",
I'd rather think of an integer value than a string here.
</bikeshedpainting>
> + PowerPCCPU *threads;
> +} sPAPRCPUCore;
> +
> +#define SPAPR_CPU_CORE_SLOT_PROP "slot"
> +
> +#endif
>
Thomas
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support Bharata B Rao
2016-02-26 3:51 ` David Gibson
@ 2016-02-26 13:03 ` Thomas Huth
2016-02-26 14:54 ` Bharata B Rao
1 sibling, 1 reply; 45+ messages in thread
From: Thomas Huth @ 2016-02-26 13:03 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, pkrempa, ehabkost, aik, armbru, agraf, borntraeger,
qemu-ppc, pbonzini, imammedo, mdroth, afaerber, david
On 25.02.2016 17:22, Bharata B Rao wrote:
> Set up device tree entries for the hotplugged CPU core and use the
> exising EPOW event infrastructure to send CPU hotplug notification to
> the guest.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++-
> hw/ppc/spapr_events.c | 3 ++
> hw/ppc/spapr_rtas.c | 24 +++++++++
> include/hw/ppc/spapr.h | 1 +
> 4 files changed, 163 insertions(+), 1 deletion(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1f0d232..780cd00 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> size_t page_sizes_prop_size;
> uint32_t vcpus_per_socket = smp_threads * smp_cores;
> uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + int drc_index;
> +
> + if (smc->dr_cpu_enabled) {
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> + g_assert(drc);
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drc_index = drck->get_index(drc);
> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> + }
>
> /* Note: we keep CI large pages off for now because a 64K capable guest
> * provisioned with large pages might otherwise try to map a qemu
> @@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> }
>
> + if (smc->dr_cpu_enabled) {
> + int offset = fdt_path_offset(fdt, "/cpus");
> + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> + SPAPR_DR_CONNECTOR_TYPE_CPU);
> + if (ret < 0) {
> + fprintf(stderr, "Couldn't set up CPU DR device tree properties\n");
I think it's better to use error_report() nowadays instead.
> + exit(1);
> + }
> + }
> +
> _FDT((fdt_pack(fdt)));
>
> if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> @@ -1759,6 +1781,7 @@ static void ppc_spapr_init(MachineState *machine)
> char *filename;
> int spapr_cores = smp_cpus / smp_threads;
> int spapr_max_cores = max_cpus / smp_threads;
> + int smt = kvmppc_smt_threads();
>
> msi_supported = true;
>
> @@ -1813,6 +1836,15 @@ static void ppc_spapr_init(MachineState *machine)
> spapr_validate_node_memory(machine, &error_fatal);
> }
>
> + if (smc->dr_cpu_enabled) {
> + for (i = 0; i < spapr_max_cores; i++) {
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_new(OBJECT(spapr),
> + SPAPR_DR_CONNECTOR_TYPE_CPU, i * smt);
> + qemu_register_reset(spapr_drc_reset, drc);
> + }
> + }
> +
> /* init CPUs */
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> @@ -2247,6 +2279,88 @@ out:
> error_propagate(errp, local_err);
> }
>
> +static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs,
> + int *fdt_offset,
> + sPAPRMachineState *spapr)
> +{
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + void *fdt;
> + int offset, fdt_size;
> + char *nodename;
> +
> + fdt = create_device_tree(&fdt_size);
> + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> + offset = fdt_add_subnode(fdt, 0, nodename);
> +
> + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> + g_free(nodename);
> +
> + *fdt_offset = offset;
> + return fdt;
> +}
> +
> +static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + PowerPCCPU *cpu = &core->threads[0];
> + CPUState *cs = CPU(cpu);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> + void *fdt = NULL;
> + int fdt_offset = 0;
> +
> + if (!smc->dr_cpu_enabled) {
> + /*
> + * This is a cold plugged CPU core but the machine doesn't support
> + * DR. So skip the hotplug path ensuring that the core is brought
> + * up online with out an associated DR connector.
> + */
> + return;
> + }
> +
> + g_assert(drc);
> +
> + /*
> + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> + */
> + if (dev->hotplugged) {
> + fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms);
> + dev->hotplugged = true;
That looks strange ... why "dev->hotplugged = true" after you've already
checked that with the above if-statement?
> + }
> +
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
> + if (local_err) {
> + g_free(fdt);
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + if (dev->hotplugged) {
> + /*
> + * Send hotplug notification interrupt to the guest only in case
> + * of hotplugged CPUs.
> + */
> + spapr_hotplug_req_add_by_index(drc);
> + } else {
> + /*
> + * Set the right DRC states for cold plugged CPU.
> + */
> + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
> + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
> + }
> + return;
Superfluous return statement.
> +}
> +
> static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> @@ -2291,8 +2405,25 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> CPUState *cs = CPU(dev);
> PowerPCCPU *cpu = POWERPC_CPU(cs);
> + int i;
> +
> + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> + error_setg(errp, "CPU hotplug not supported for this machine");
> + return;
> + }
> +
> + /* Set NUMA node for the added CPUs */
> + for (i = 0; i < nb_numa_nodes; i++) {
> + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> + cs->numa_node = i;
> + break;
> + }
> + }
>
> spapr_cpu_init(ms, cpu, errp);
> + spapr_cpu_reset(cpu);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_plug(hotplug_dev, dev, errp);
> }
> }
>
> @@ -2308,7 +2439,8 @@ static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> - object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + object_dynamic_cast(OBJECT(dev), TYPE_CPU) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> @@ -2352,6 +2484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
>
> smc->dr_lmb_enabled = true;
> + smc->dr_cpu_enabled = true;
> fwc->get_dev_path = spapr_get_fw_dev_path;
> nc->nmi_monitor_handler = spapr_nmi;
> }
> @@ -2431,6 +2564,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
>
> spapr_machine_2_6_class_options(mc);
> smc->use_ohci_by_default = true;
> + smc->dr_cpu_enabled = false;
> SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
> }
>
Thomas
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-26 13:03 ` Thomas Huth
@ 2016-02-26 14:54 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-26 14:54 UTC (permalink / raw)
To: Thomas Huth
Cc: mjrosato, agraf, pkrempa, ehabkost, aik, qemu-devel, armbru,
borntraeger, qemu-ppc, pbonzini, imammedo, mdroth, afaerber,
david
On Fri, Feb 26, 2016 at 02:03:57PM +0100, Thomas Huth wrote:
> On 25.02.2016 17:22, Bharata B Rao wrote:
> > Set up device tree entries for the hotplugged CPU core and use the
> > exising EPOW event infrastructure to send CPU hotplug notification to
> > the guest.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++-
> > hw/ppc/spapr_events.c | 3 ++
> > hw/ppc/spapr_rtas.c | 24 +++++++++
> > include/hw/ppc/spapr.h | 1 +
> > 4 files changed, 163 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 1f0d232..780cd00 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> > size_t page_sizes_prop_size;
> > uint32_t vcpus_per_socket = smp_threads * smp_cores;
> > uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRDRConnector *drc;
> > + sPAPRDRConnectorClass *drck;
> > + int drc_index;
> > +
> > + if (smc->dr_cpu_enabled) {
> > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> > + g_assert(drc);
> > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > + drc_index = drck->get_index(drc);
> > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> > + }
> >
> > /* Note: we keep CI large pages off for now because a 64K capable guest
> > * provisioned with large pages might otherwise try to map a qemu
> > @@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> > _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> > }
> >
> > + if (smc->dr_cpu_enabled) {
> > + int offset = fdt_path_offset(fdt, "/cpus");
> > + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > + SPAPR_DR_CONNECTOR_TYPE_CPU);
> > + if (ret < 0) {
> > + fprintf(stderr, "Couldn't set up CPU DR device tree properties\n");
>
> I think it's better to use error_report() nowadays instead.
Yeah, have been carrying this old hunk, will fix.
>
> > + exit(1);
> > + }
> > + }
> > +
> > _FDT((fdt_pack(fdt)));
> >
> > if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> > @@ -1759,6 +1781,7 @@ static void ppc_spapr_init(MachineState *machine)
> > char *filename;
> > int spapr_cores = smp_cpus / smp_threads;
> > int spapr_max_cores = max_cpus / smp_threads;
> > + int smt = kvmppc_smt_threads();
> >
> > msi_supported = true;
> >
> > @@ -1813,6 +1836,15 @@ static void ppc_spapr_init(MachineState *machine)
> > spapr_validate_node_memory(machine, &error_fatal);
> > }
> >
> > + if (smc->dr_cpu_enabled) {
> > + for (i = 0; i < spapr_max_cores; i++) {
> > + sPAPRDRConnector *drc =
> > + spapr_dr_connector_new(OBJECT(spapr),
> > + SPAPR_DR_CONNECTOR_TYPE_CPU, i * smt);
> > + qemu_register_reset(spapr_drc_reset, drc);
> > + }
> > + }
> > +
> > /* init CPUs */
> > if (machine->cpu_model == NULL) {
> > machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> > @@ -2247,6 +2279,88 @@ out:
> > error_propagate(errp, local_err);
> > }
> >
> > +static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs,
> > + int *fdt_offset,
> > + sPAPRMachineState *spapr)
> > +{
> > + PowerPCCPU *cpu = POWERPC_CPU(cs);
> > + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> > + int id = ppc_get_vcpu_dt_id(cpu);
> > + void *fdt;
> > + int offset, fdt_size;
> > + char *nodename;
> > +
> > + fdt = create_device_tree(&fdt_size);
> > + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> > + offset = fdt_add_subnode(fdt, 0, nodename);
> > +
> > + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> > + g_free(nodename);
> > +
> > + *fdt_offset = offset;
> > + return fdt;
> > +}
> > +
> > +static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > + Error **errp)
> > +{
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine());
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + PowerPCCPU *cpu = &core->threads[0];
> > + CPUState *cs = CPU(cpu);
> > + int id = ppc_get_vcpu_dt_id(cpu);
> > + sPAPRDRConnector *drc =
> > + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> > + sPAPRDRConnectorClass *drck;
> > + Error *local_err = NULL;
> > + void *fdt = NULL;
> > + int fdt_offset = 0;
> > +
> > + if (!smc->dr_cpu_enabled) {
> > + /*
> > + * This is a cold plugged CPU core but the machine doesn't support
> > + * DR. So skip the hotplug path ensuring that the core is brought
> > + * up online with out an associated DR connector.
> > + */
> > + return;
> > + }
> > +
> > + g_assert(drc);
> > +
> > + /*
> > + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> > + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> > + */
> > + if (dev->hotplugged) {
> > + fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms);
> > + dev->hotplugged = true;
>
> That looks strange ... why "dev->hotplugged = true" after you've already
> checked that with the above if-statement?
Yeah this is a remnant of the implmentation that was based on David's proposal
where all the devices were pre-created and I had to set dev->hotplugged
manually here. Will remove as it is not needed here in this device_add
based implementation.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices Bharata B Rao
2016-02-26 3:45 ` David Gibson
@ 2016-02-26 15:18 ` Igor Mammedov
2016-02-29 5:35 ` Bharata B Rao
1 sibling, 1 reply; 45+ messages in thread
From: Igor Mammedov @ 2016-02-26 15:18 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber,
david
On Thu, 25 Feb 2016 21:52:39 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> Initialize boot CPUs as spapr-cpu-core devices and create links from
> machine object to these core devices. These links can be considered
> as CPU slots in which core devices will get hot-plugged. spapr-cpu-core
> device's slot property indicates the slot where it is plugged. Information
> about all the CPU slots can be obtained by walking these links.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 65 +++++++++++++++++++++++++++++++++++++++++++-------
> include/hw/ppc/spapr.h | 3 +++
> 2 files changed, 60 insertions(+), 8 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index e214a34..1f0d232 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -64,6 +64,7 @@
>
> #include "hw/compat.h"
> #include "qemu-common.h"
> +#include "hw/ppc/spapr_cpu_core.h"
>
> #include <libfdt.h>
>
> @@ -1720,6 +1721,21 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
> }
> }
>
> +/*
> + * Check to see if core is being hot-plugged into an already populated slot.
> + */
> +static void spapr_cpu_core_allow_set_link(Object *obj, const char *name,
> + Object *val, Error **errp)
> +{
> + Object *core = object_property_get_link(qdev_get_machine(), name, NULL);
> +
> + if (core) {
> + char *path = object_get_canonical_path(core);
> + error_setg(errp, "Slot %s already populated with %s", name, path);
> + g_free(path);
> + }
> +}
> +
> /* pSeries LPAR / sPAPR hardware init */
> static void ppc_spapr_init(MachineState *machine)
> {
> @@ -1728,7 +1744,6 @@ static void ppc_spapr_init(MachineState *machine)
> const char *kernel_filename = machine->kernel_filename;
> const char *kernel_cmdline = machine->kernel_cmdline;
> const char *initrd_filename = machine->initrd_filename;
> - PowerPCCPU *cpu;
> PCIHostState *phb;
> int i;
> MemoryRegion *sysmem = get_system_memory();
> @@ -1742,6 +1757,8 @@ static void ppc_spapr_init(MachineState *machine)
> long load_limit, fw_size;
> bool kernel_le = false;
> char *filename;
> + int spapr_cores = smp_cpus / smp_threads;
> + int spapr_max_cores = max_cpus / smp_threads;
>
> msi_supported = true;
>
> @@ -1800,13 +1817,38 @@ static void ppc_spapr_init(MachineState *machine)
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> }
> - for (i = 0; i < smp_cpus; i++) {
> - cpu = cpu_ppc_init(machine->cpu_model);
> - if (cpu == NULL) {
> - error_report("Unable to find PowerPC CPU definition");
> - exit(1);
> +
> + spapr->cores = g_malloc0(spapr_max_cores * sizeof(Object));
souldn't it be sizeof(Object *)
> +
> + for (i = 0; i < spapr_max_cores; i++) {
> + Object *spapr_cpu_core = object_new(TYPE_SPAPR_CPU_CORE);
you allocate spapr_max_cores cores but set links to only to spapr_cores only,
it looks like all spapr_cpu_core objects are leaked for range spapr_cores..spapr_max_cores
> + char name[32];
> +
> + object_property_set_str(spapr_cpu_core, machine->cpu_model, "cpu_model",
> + &error_fatal);
> + object_property_set_int(spapr_cpu_core, smp_threads, "nr_threads",
> + &error_fatal);
> + /*
> + * Create links from machine objects to all possible cores.
> + */
> + snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
> + object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
> + (Object **)&spapr->cores[i],
> + spapr_cpu_core_allow_set_link, 0,
> + &error_fatal);
> +
> + /*
> + * Set the link from machine object to core object for all
> + * boot time CPUs specified with -smp and realize them.
> + * For rest of the hotpluggable cores this is happens from
> + * the core hotplug/realization path.
> + */
> + if (i < spapr_cores) {
> + object_property_set_str(spapr_cpu_core, name,
> + SPAPR_CPU_CORE_SLOT_PROP, &error_fatal);
> + object_property_set_bool(spapr_cpu_core, true, "realized",
> + &error_fatal);
> }
> - spapr_cpu_init(spapr, cpu, &error_fatal);
> }
>
> if (kvm_enabled()) {
> @@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
>
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> int node;
> @@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
>
> spapr_memory_plug(hotplug_dev, dev, node, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
here probably should be CORE and not TYPE_CPU,
then if board needs to set some state for child threads it
could ennumerate child of core here and do the required job.
> + CPUState *cs = CPU(dev);
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> +
> + spapr_cpu_init(ms, cpu, errp);
> }
> }
>
> @@ -2259,7 +2307,8 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> - if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 098d85d..20b3417 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -79,8 +79,11 @@ struct sPAPRMachineState {
> /*< public >*/
> char *kvm_type;
> MemoryHotplugState hotplug_memory;
> + Object *cores;
> };
>
> +#define SPAPR_MACHINE_CPU_CORE_PROP "core"
> +
> #define H_SUCCESS 0
> #define H_BUSY 1 /* Hardware busy -- retry later */
> #define H_CLOSED 2 /* Resource closed */
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
2016-02-26 4:03 ` David Gibson
@ 2016-02-26 15:58 ` Eric Blake
2016-02-29 5:43 ` Bharata B Rao
2016-02-26 16:33 ` Thomas Huth
2016-02-29 10:46 ` Igor Mammedov
3 siblings, 1 reply; 45+ messages in thread
From: Eric Blake @ 2016-02-26 15:58 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, mdroth, afaerber,
david
[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]
On 02/25/2016 09:22 AM, Bharata B Rao wrote:
> Implement query cpu-slots that provides information about hot-plugged
> as well as hot-pluggable CPU slots that the machine supports.
>
> TODO: As Eric suggested use enum for type instead of str.
> TODO: @hotplug-granularity probably isn't required.
I guess this is still marked TODO because the series is still RFC?
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> +++ b/qapi-schema.json
> @@ -4083,3 +4083,88 @@
> ##
> { 'enum': 'ReplayMode',
> 'data': [ 'none', 'record', 'play' ] }
> +
> +##
> +# @CPUInfo:
> +#
> +# Information about CPUs
> +#
> +# @arch-id: Arch-specific ID for the CPU.
> +#
> +# @type: QOM type of the CPU.
> +#
> +# @thread: Thread ID of the CPU.
> +#
> +# @core: Core ID of the CPU.
> +#
> +# @socket: Socket ID of the CPU.
> +#
> +# @node: NUMA node to which the CPU belongs.
Please add the '#optional' tag to the fields which are not always present.
> +#
> +# @qom-path: QOM path of the CPU object
> +#
> +# Since: 2.6
> +##
> +
> +{ 'struct': 'CPUInfo',
> + 'data': { 'arch-id': 'int',
> + 'type': 'str',
The TODO in the commit message mentions that this should be converted to
an enum.
> + '*thread': 'int',
> + '*core': 'int',
> + '*socket' : 'int',
> + '*node' : 'int',
> + '*qom-path': 'str'
> + }
But looking better than the previous round.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
2016-02-26 4:03 ` David Gibson
2016-02-26 15:58 ` Eric Blake
@ 2016-02-26 16:33 ` Thomas Huth
2016-02-29 10:46 ` Igor Mammedov
3 siblings, 0 replies; 45+ messages in thread
From: Thomas Huth @ 2016-02-26 16:33 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, pkrempa, ehabkost, aik, armbru, agraf, borntraeger,
qemu-ppc, pbonzini, imammedo, mdroth, afaerber, david
On 25.02.2016 17:22, Bharata B Rao wrote:
> Implement query cpu-slots that provides information about hot-plugged
> as well as hot-pluggable CPU slots that the machine supports.
>
> TODO: As Eric suggested use enum for type instead of str.
> TODO: @hotplug-granularity probably isn't required.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/core/machine.c | 19 +++++++++
> hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> include/hw/boards.h | 4 ++
> qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> qmp-commands.hx | 47 ++++++++++++++++++++++
> 5 files changed, 267 insertions(+)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 6d1a0d8..3055ef8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -17,6 +17,25 @@
> #include "hw/sysbus.h"
> #include "sysemu/sysemu.h"
> #include "qemu/error-report.h"
> +#include "qmp-commands.h"
> +
> +/*
> + * QMP: query-cpu-slots
> + *
> + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> + */
> +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> +{
> + MachineState *ms = MACHINE(qdev_get_machine());
> + MachineClass *mc = MACHINE_GET_CLASS(ms);
> +
> + if (!mc->cpu_slots) {
> + error_setg(errp, QERR_UNSUPPORTED);
> + return NULL;
> + }
> +
> + return mc->cpu_slots(ms);
> +}
>
> static char *machine_get_accel(Object *obj, Error **errp)
> {
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 780cd00..b76ed85 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> return cpu_index / smp_threads / smp_cores;
> }
>
> +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> +{
> + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> + CPUInfoList ***prev = opaque;
> +
> + if (object_dynamic_cast(obj, TYPE_CPU)) {
> + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> + CPUInfo *s = g_new0(CPUInfo, 1);
> + CPUState *cpu = CPU(obj);
> + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> +
> + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> + s->type = g_strdup(object_get_typename(obj));
> + s->thread = cpu->cpu_index;
> + s->has_thread = true;
> + s->core = cpu->cpu_index / smp_threads;
> + s->has_core = true;
> + if (mc->cpu_index_to_socket_id) {
> + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> + } else {
> + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> + }
> + s->has_socket = true;
> + s->node = cpu->numa_node;
> + s->has_node = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> +
> + elem->value = s;
> + elem->next = NULL;
> + **prev = elem;
> + *prev = &elem->next;
> + }
> + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> + return 0;
> +}
> +
> +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> +{
> + CPUSlotInfoList *head = NULL;
> + CPUSlotInfoList **prev = &head;
> + Object *root_container;
> + ObjectProperty *prop;
> + ObjectPropertyIterator iter;
> +
> + /*
> + * TODO: There surely must be a better/easier way to walk all
> + * the link properties of an object ?
> + */
> + root_container = container_get(object_get_root(), "/machine");
> + object_property_iter_init(&iter, root_container);
> +
> + while ((prop = object_property_iter_next(&iter))) {
> + Object *obj;
> + DeviceState *dev;
> + CPUSlotInfoList *elem;
> + CPUSlotInfo *s;
> + CPUInfoList *cpu_head = NULL;
> + CPUInfoList **cpu_prev = &cpu_head;
> + sPAPRCPUCore *core;
> +
> + if (!strstart(prop->type, "link<", NULL)) {
> + continue;
> + }
> +
> + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> + continue;
> + }
> +
> + elem = g_new0(CPUSlotInfoList, 1);
> + s = g_new0(CPUSlotInfo, 1);
> +
> + obj = object_property_get_link(root_container, prop->name, NULL);
> + if (obj) {
> + /* Slot populated */
> + dev = DEVICE(obj);
> + core = SPAPR_CPU_CORE(obj);
> +
> + if (dev->id) {
> + s->has_id = true;
> + s->id = g_strdup(dev->id);
> + }
> + s->realized = object_property_get_bool(obj, "realized", NULL);
> + s->nr_cpus = core->nr_threads;
> + s->has_nr_cpus = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> + if (s->realized) {
> + spapr_cpuinfo_list(obj, &cpu_prev);
> + }
> + s->has_cpus = true;
> + } else {
> + /* Slot empty */
> + s->has_id = false;
> + s->has_nr_cpus = false;
> + s->has_qom_path = false;
> + s->has_cpus = false;
> + s->realized = false;
> + }
I think you could drop that whole else-path since you've already used
g_new0 to make sure that the memory of the new CPUSlotInfo is cleared.
> + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> + s->slot_id = g_strdup(prop->name);
> + s->cpus = cpu_head;
> + elem->value = s;
> + elem->next = NULL;
> + *prev = elem;
> + prev = &elem->next;
> + }
> + return head;
> +}
Thomas
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
2016-02-26 2:57 ` David Gibson
2016-02-26 10:46 ` Thomas Huth
@ 2016-02-26 18:13 ` Michael Roth
2016-02-29 3:44 ` David Gibson
2016-02-29 5:50 ` Bharata B Rao
2 siblings, 2 replies; 45+ messages in thread
From: Michael Roth @ 2016-02-26 18:13 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, afaerber, david
Quoting Bharata B Rao (2016-02-25 10:22:38)
> Add sPAPR specific CPU core device that is based on generic CPU core device.
> Creating this core device will result in creation of all the CPU thread
> devices that are part of this core.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/Makefile.objs | 1 +
> hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> 3 files changed, 243 insertions(+)
> create mode 100644 hw/ppc/spapr_cpu_core.c
> create mode 100644 include/hw/ppc/spapr_cpu_core.h
>
> diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> index c1ffc77..5cc6608 100644
> --- a/hw/ppc/Makefile.objs
> +++ b/hw/ppc/Makefile.objs
> @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> obj-y += spapr_pci_vfio.o
> endif
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> new file mode 100644
> index 0000000..c44eb61
> --- /dev/null
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -0,0 +1,210 @@
> +/*
> + * sPAPR CPU core device, acts as container of CPU thread devices.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#include "hw/cpu/core.h"
> +#include "hw/ppc/spapr_cpu_core.h"
> +#include "hw/ppc/spapr.h"
> +#include "hw/boards.h"
> +#include "qemu/error-report.h"
> +#include "qapi/visitor.h"
> +#include <sysemu/cpus.h>
> +
> +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> +{
> + Error **errp = opaque;
> +
> + object_property_set_bool(child, true, "realized", errp);
> + if (*errp) {
> + return 1;
> + }
> + return 0;
> +}
> +
> +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + Error *local_err = NULL;
> +
> + if (!core->nr_threads) {
> + error_setg(errp, "nr_threads property can't be 0");
> + return;
> + }
> +
> + if (!core->cpu_model) {
> + error_setg(errp, "cpu_model property isn't set");
> + return;
> + }
> +
> + /*
> + * TODO: If slot isn't specified, plug this core into
> + * an existing empty slot.
> + */
> + if (!core->slot) {
> + error_setg(errp, "slot property isn't set");
> + return;
> + }
> +
> + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> + &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> +}
> +
> +/*
> + * This creates the CPU threads for a given @core.
> + *
> + * In order to create the threads, we need two inputs - number of
> + * threads and the cpu_model. These are set as core object's properties.
> + * When both of them become available/set, this routine will be called from
> + * either property's set handler to create the threads.
> + *
> + * TODO: Dependence of threads creation on two properties is resulting
> + * in this not-so-clean way of creating threads from either of the
> + * property setters based on the order in which they get set. Check if
> + * this can be handled in a better manner.
> + */
> +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> +{
> + int i;
> +
> + for (i = 0; i < core->nr_threads; i++) {
> + char id[32];
> + char type[32];
> +
> + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> + TYPE_POWERPC_CPU);
> + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> +
> + snprintf(id, sizeof(id), "thread[%d]", i);
> + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> + errp);
> + }
> +}
> +
> +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->slot;
> +}
> +
> +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + core->slot = g_strdup(val);
> +}
> +
> +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> +
> + return core->cpu_model;
> +}
> +
> +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + MachineState *machine = MACHINE(qdev_get_machine());
> +
> + /*
> + * cpu_model can't be different from what is specified with -cpu
> + */
> + if (strcmp(val, machine->cpu_model)) {
> + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> + return;
> + }
> +
> + core->cpu_model = g_strdup(val);
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + int64_t value = core->nr_threads;
> +
> + visit_type_int(v, name, &value, errp);
> +}
> +
> +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> + Error *local_err = NULL;
> + int64_t value;
> +
> + visit_type_int(v, name, &value, &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + /* Allow only homogeneous configuration */
> + if (value != smp_threads) {
> + error_setg(errp, "nr_threads should be %d", smp_threads);
> + return;
> + }
> +
> + core->nr_threads = value;
> + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> +
> + if (core->nr_threads && core->cpu_model) {
> + spapr_cpu_core_create_threads(core, errp);
> + }
> +}
> +
> +static void spapr_cpu_core_instance_init(Object *obj)
> +{
> + object_property_add(obj, "nr_threads", "int",
> + spapr_cpu_core_prop_get_nr_threads,
> + spapr_cpu_core_prop_set_nr_threads,
> + NULL, NULL, NULL);
> + object_property_add_str(obj, "cpu_model",
> + spapr_cpu_core_prop_get_cpu_model,
> + spapr_cpu_core_prop_set_cpu_model,
> + NULL);
> + object_property_add_str(obj, "slot",
> + spapr_cpu_core_prop_get_slot,
> + spapr_cpu_core_prop_set_slot,
> + NULL);
"slot" seems intended to be a machine-agnostic of mapping device
types discovered from qmp_query_cpu_slots() to an appropriate
"bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
It seems like maybe TYPE_CPU_CORE is a better place, but then on
x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
right approach, but I don't know how we could expose that as
a property. I guess it's somewhat implied that this "interface"
exists if qmp_query_cpu_slots() returns the type, but I wonder
if something a bit more formal should be modeled to make the
implementation requirements a bit clearer.
Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
class method, expose them via "slot" property, then have the
defaults generate "not implemented" errors?
> +}
> +
> +static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
> +{
> + DeviceClass *dc = DEVICE_CLASS(oc);
> +
> + dc->realize = spapr_cpu_core_realize;
> +}
> +
> +static const TypeInfo spapr_cpu_core_type_info = {
> + .name = TYPE_SPAPR_CPU_CORE,
> + .parent = TYPE_CPU_CORE,
> + .instance_init = spapr_cpu_core_instance_init,
> + .instance_size = sizeof(sPAPRCPUCore),
> + .class_init = spapr_cpu_core_class_init,
> +};
> +
> +static void spapr_cpu_core_register_types(void)
> +{
> + type_register_static(&spapr_cpu_core_type_info);
> +}
> +
> +type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> new file mode 100644
> index 0000000..ed9bc7f
> --- /dev/null
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -0,0 +1,32 @@
> +/*
> + * sPAPR CPU core device.
> + *
> + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +#ifndef HW_SPAPR_CPU_CORE_H
> +#define HW_SPAPR_CPU_CORE_H
> +
> +#include "hw/qdev.h"
> +#include "hw/cpu/core.h"
> +
> +#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
> +#define SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
> +
> +typedef struct sPAPRCPUCore {
> + /*< private >*/
> + DeviceState parent_obj;
> +
> + /*< public >*/
> + int nr_threads;
> + char *cpu_model;
> + char *slot;
> + PowerPCCPU *threads;
> +} sPAPRCPUCore;
> +
> +#define SPAPR_CPU_CORE_SLOT_PROP "slot"
> +
> +#endif
> --
> 2.1.0
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-26 18:13 ` Michael Roth
@ 2016-02-29 3:44 ` David Gibson
2016-02-29 5:50 ` Bharata B Rao
1 sibling, 0 replies; 45+ messages in thread
From: David Gibson @ 2016-02-29 3:44 UTC (permalink / raw)
To: Michael Roth
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, pbonzini, qemu-ppc, Bharata B Rao,
imammedo, afaerber
[-- Attachment #1: Type: text/plain, Size: 9559 bytes --]
On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> Quoting Bharata B Rao (2016-02-25 10:22:38)
> > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > Creating this core device will result in creation of all the CPU thread
> > devices that are part of this core.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/Makefile.objs | 1 +
> > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > 3 files changed, 243 insertions(+)
> > create mode 100644 hw/ppc/spapr_cpu_core.c
> > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> >
> > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > index c1ffc77..5cc6608 100644
> > --- a/hw/ppc/Makefile.objs
> > +++ b/hw/ppc/Makefile.objs
> > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > obj-y += spapr_pci_vfio.o
> > endif
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > new file mode 100644
> > index 0000000..c44eb61
> > --- /dev/null
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -0,0 +1,210 @@
> > +/*
> > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > + *
> > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#include "hw/cpu/core.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> > +#include "hw/ppc/spapr.h"
> > +#include "hw/boards.h"
> > +#include "qemu/error-report.h"
> > +#include "qapi/visitor.h"
> > +#include <sysemu/cpus.h>
> > +
> > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > +{
> > + Error **errp = opaque;
> > +
> > + object_property_set_bool(child, true, "realized", errp);
> > + if (*errp) {
> > + return 1;
> > + }
> > + return 0;
> > +}
> > +
> > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > + Error *local_err = NULL;
> > +
> > + if (!core->nr_threads) {
> > + error_setg(errp, "nr_threads property can't be 0");
> > + return;
> > + }
> > +
> > + if (!core->cpu_model) {
> > + error_setg(errp, "cpu_model property isn't set");
> > + return;
> > + }
> > +
> > + /*
> > + * TODO: If slot isn't specified, plug this core into
> > + * an existing empty slot.
> > + */
> > + if (!core->slot) {
> > + error_setg(errp, "slot property isn't set");
> > + return;
> > + }
> > +
> > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > + &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > +}
> > +
> > +/*
> > + * This creates the CPU threads for a given @core.
> > + *
> > + * In order to create the threads, we need two inputs - number of
> > + * threads and the cpu_model. These are set as core object's properties.
> > + * When both of them become available/set, this routine will be called from
> > + * either property's set handler to create the threads.
> > + *
> > + * TODO: Dependence of threads creation on two properties is resulting
> > + * in this not-so-clean way of creating threads from either of the
> > + * property setters based on the order in which they get set. Check if
> > + * this can be handled in a better manner.
> > + */
> > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < core->nr_threads; i++) {
> > + char id[32];
> > + char type[32];
> > +
> > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > + TYPE_POWERPC_CPU);
> > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > +
> > + snprintf(id, sizeof(id), "thread[%d]", i);
> > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > + errp);
> > + }
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->slot;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + core->slot = g_strdup(val);
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->cpu_model;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + MachineState *machine = MACHINE(qdev_get_machine());
> > +
> > + /*
> > + * cpu_model can't be different from what is specified with -cpu
> > + */
> > + if (strcmp(val, machine->cpu_model)) {
> > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > + return;
> > + }
> > +
> > + core->cpu_model = g_strdup(val);
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + int64_t value = core->nr_threads;
> > +
> > + visit_type_int(v, name, &value, errp);
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + Error *local_err = NULL;
> > + int64_t value;
> > +
> > + visit_type_int(v, name, &value, &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + /* Allow only homogeneous configuration */
> > + if (value != smp_threads) {
> > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > + return;
> > + }
> > +
> > + core->nr_threads = value;
> > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > +
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_instance_init(Object *obj)
> > +{
> > + object_property_add(obj, "nr_threads", "int",
> > + spapr_cpu_core_prop_get_nr_threads,
> > + spapr_cpu_core_prop_set_nr_threads,
> > + NULL, NULL, NULL);
> > + object_property_add_str(obj, "cpu_model",
> > + spapr_cpu_core_prop_get_cpu_model,
> > + spapr_cpu_core_prop_set_cpu_model,
> > + NULL);
> > + object_property_add_str(obj, "slot",
> > + spapr_cpu_core_prop_get_slot,
> > + spapr_cpu_core_prop_set_slot,
> > + NULL);
>
> "slot" seems intended to be a machine-agnostic of mapping device
> types discovered from qmp_query_cpu_slots() to an appropriate
> "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> It seems like maybe TYPE_CPU_CORE is a better place, but then on
> x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
>
> It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> right approach, but I don't know how we could expose that as
> a property. I guess it's somewhat implied that this "interface"
> exists if qmp_query_cpu_slots() returns the type, but I wonder
> if something a bit more formal should be modeled to make the
> implementation requirements a bit clearer.
>
> Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> class method, expose them via "slot" property, then have the
> defaults generate "not implemented" errors?
Yeah, TBH the "slot" stuff has me a bit confused still. I'm intending
to have a closer look at it once the more obvious cleanups are
addressed.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-26 3:51 ` David Gibson
@ 2016-02-29 4:42 ` Bharata B Rao
2016-03-01 7:58 ` Bharata B Rao
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 4:42 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
On Fri, Feb 26, 2016 at 02:51:41PM +1100, David Gibson wrote:
> On Thu, Feb 25, 2016 at 09:52:40PM +0530, Bharata B Rao wrote:
> > Set up device tree entries for the hotplugged CPU core and use the
> > exising EPOW event infrastructure to send CPU hotplug notification to
> > the guest.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++-
> > hw/ppc/spapr_events.c | 3 ++
> > hw/ppc/spapr_rtas.c | 24 +++++++++
> > include/hw/ppc/spapr.h | 1 +
> > 4 files changed, 163 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 1f0d232..780cd00 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> > size_t page_sizes_prop_size;
> > uint32_t vcpus_per_socket = smp_threads * smp_cores;
> > uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRDRConnector *drc;
> > + sPAPRDRConnectorClass *drck;
> > + int drc_index;
> > +
> > + if (smc->dr_cpu_enabled) {
> > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> > + g_assert(drc);
> > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > + drc_index = drck->get_index(drc);
> > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> > + }
> >
> > /* Note: we keep CI large pages off for now because a 64K capable guest
> > * provisioned with large pages might otherwise try to map a qemu
> > @@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> > _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> > }
> >
> > + if (smc->dr_cpu_enabled) {
> > + int offset = fdt_path_offset(fdt, "/cpus");
> > + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > + SPAPR_DR_CONNECTOR_TYPE_CPU);
> > + if (ret < 0) {
> > + fprintf(stderr, "Couldn't set up CPU DR device tree properties\n");
> > + exit(1);
> > + }
> > + }
> > +
> > _FDT((fdt_pack(fdt)));
> >
> > if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> > @@ -1759,6 +1781,7 @@ static void ppc_spapr_init(MachineState *machine)
> > char *filename;
> > int spapr_cores = smp_cpus / smp_threads;
> > int spapr_max_cores = max_cpus / smp_threads;
> > + int smt = kvmppc_smt_threads();
> >
> > msi_supported = true;
> >
> > @@ -1813,6 +1836,15 @@ static void ppc_spapr_init(MachineState *machine)
> > spapr_validate_node_memory(machine, &error_fatal);
> > }
> >
> > + if (smc->dr_cpu_enabled) {
> > + for (i = 0; i < spapr_max_cores; i++) {
> > + sPAPRDRConnector *drc =
> > + spapr_dr_connector_new(OBJECT(spapr),
> > + SPAPR_DR_CONNECTOR_TYPE_CPU, i * smt);
> > + qemu_register_reset(spapr_drc_reset, drc);
> > + }
> > + }
> > +
> > /* init CPUs */
> > if (machine->cpu_model == NULL) {
> > machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> > @@ -2247,6 +2279,88 @@ out:
> > error_propagate(errp, local_err);
> > }
> >
> > +static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs,
> > + int *fdt_offset,
> > + sPAPRMachineState *spapr)
> > +{
> > + PowerPCCPU *cpu = POWERPC_CPU(cs);
> > + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> > + int id = ppc_get_vcpu_dt_id(cpu);
> > + void *fdt;
> > + int offset, fdt_size;
> > + char *nodename;
> > +
> > + fdt = create_device_tree(&fdt_size);
> > + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> > + offset = fdt_add_subnode(fdt, 0, nodename);
> > +
> > + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> > + g_free(nodename);
> > +
> > + *fdt_offset = offset;
> > + return fdt;
> > +}
> > +
> > +static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > + Error **errp)
> > +{
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine());
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + PowerPCCPU *cpu = &core->threads[0];
> > + CPUState *cs = CPU(cpu);
> > + int id = ppc_get_vcpu_dt_id(cpu);
> > + sPAPRDRConnector *drc =
> > + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> > + sPAPRDRConnectorClass *drck;
> > + Error *local_err = NULL;
> > + void *fdt = NULL;
> > + int fdt_offset = 0;
> > +
> > + if (!smc->dr_cpu_enabled) {
> > + /*
> > + * This is a cold plugged CPU core but the machine doesn't support
> > + * DR. So skip the hotplug path ensuring that the core is brought
> > + * up online with out an associated DR connector.
> > + */
> > + return;
> > + }
> > +
> > + g_assert(drc);
> > +
> > + /*
> > + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> > + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> > + */
> > + if (dev->hotplugged) {
> > + fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms);
> > + dev->hotplugged = true;
> > + }
> > +
> > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
> > + if (local_err) {
> > + g_free(fdt);
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + if (dev->hotplugged) {
> > + /*
> > + * Send hotplug notification interrupt to the guest only in case
> > + * of hotplugged CPUs.
> > + */
> > + spapr_hotplug_req_add_by_index(drc);
> > + } else {
> > + /*
> > + * Set the right DRC states for cold plugged CPU.
> > + */
> > + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
> > + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
> > + }
> > + return;
> > +}
> > +
> > static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > DeviceState *dev, Error **errp)
> > {
> > @@ -2291,8 +2405,25 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > CPUState *cs = CPU(dev);
> > PowerPCCPU *cpu = POWERPC_CPU(cs);
> > + int i;
> > +
> > + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> > + error_setg(errp, "CPU hotplug not supported for this machine");
> > + return;
> > + }
> > +
> > + /* Set NUMA node for the added CPUs */
> > + for (i = 0; i < nb_numa_nodes; i++) {
> > + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> > + cs->numa_node = i;
> > + break;
> > + }
> > + }
> >
> > spapr_cpu_init(ms, cpu, errp);
> > + spapr_cpu_reset(cpu);
>
> It looks to me like all of this setup should be either in the last
> patch, or in this one, not split between them.
The way I have done the functional split is like this:
- Introduce the spapr-cpu-core device as the 2nd patch in the series
- The previous patch converts the boot CPU initialization into
spapr-cpu-core creation. So until this point, we can just boot a guest
with this CPUs created as core devices.
- Then this patch adds the bits needed to actually perform a device_add
operation of core device.
But I am open to re-arrange things if you feel it is necessary.
>
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > + spapr_core_plug(hotplug_dev, dev, errp);
> > }
> > }
> >
> > @@ -2308,7 +2439,8 @@ static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> > DeviceState *dev)
> > {
> > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> > - object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > + object_dynamic_cast(OBJECT(dev), TYPE_CPU) ||
> > + object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > return HOTPLUG_HANDLER(machine);
> > }
> > return NULL;
> > @@ -2352,6 +2484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> >
> > smc->dr_lmb_enabled = true;
> > + smc->dr_cpu_enabled = true;
> > fwc->get_dev_path = spapr_get_fw_dev_path;
> > nc->nmi_monitor_handler = spapr_nmi;
> > }
> > @@ -2431,6 +2564,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
> >
> > spapr_machine_2_6_class_options(mc);
> > smc->use_ohci_by_default = true;
> > + smc->dr_cpu_enabled = false;
> > SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
> > }
> >
> > diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> > index f5eac4b..e50bb16 100644
> > --- a/hw/ppc/spapr_events.c
> > +++ b/hw/ppc/spapr_events.c
> > @@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
> > case SPAPR_DR_CONNECTOR_TYPE_LMB:
> > hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
> > break;
> > + case SPAPR_DR_CONNECTOR_TYPE_CPU:
> > + hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
> > + break;
> > default:
> > /* we shouldn't be signaling hotplug events for resources
> > * that don't support them
> > diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> > index b7c5ebd..cc0369e 100644
> > --- a/hw/ppc/spapr_rtas.c
> > +++ b/hw/ppc/spapr_rtas.c
> > @@ -34,6 +34,7 @@
> >
> > #include "hw/ppc/spapr.h"
> > #include "hw/ppc/spapr_vio.h"
> > +#include "hw/ppc/ppc.h"
> > #include "qapi-event.h"
> > #include "hw/boards.h"
> >
> > @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> > rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> > }
> >
> > +/*
> > + * Set the timebase offset of the CPU to that of first CPU.
> > + * This helps hotplugged CPU to have the correct timebase offset.
> > + */
> > +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> > +{
> > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > +
> > + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> > +}
> > +
> > +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> > +{
> > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> > +
> > + if (!pcc->interrupts_big_endian(fcpu)) {
> > + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> > + }
> > +}
> > +
>
> Any particular reason for doing these things at rtas_start_cpu() time,
> but other initialization at plug time? Could you consolidate it to
> one place or the other?
Those board specific things that are needed to be done have been consolidated
into spapr_cpu_init() which will be called from the plug handler. We have
discussed this earlier at:
https://lists.nongnu.org/archive/html/qemu-devel/2015-02/msg04399.html
It has been a while but there was a good reason why setting endianness
here rather than in plug handler is necessary. W/o this LE hotplug on guests
wouldn't work, I will dig up and come back on what exactly necessiated
this change.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-26 15:18 ` Igor Mammedov
@ 2016-02-29 5:35 ` Bharata B Rao
2016-02-29 7:11 ` David Gibson
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 5:35 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber,
david
On Fri, Feb 26, 2016 at 04:18:57PM +0100, Igor Mammedov wrote:
> On Thu, 25 Feb 2016 21:52:39 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > Initialize boot CPUs as spapr-cpu-core devices and create links from
> > machine object to these core devices. These links can be considered
> > as CPU slots in which core devices will get hot-plugged. spapr-cpu-core
> > device's slot property indicates the slot where it is plugged. Information
> > about all the CPU slots can be obtained by walking these links.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 65 +++++++++++++++++++++++++++++++++++++++++++-------
> > include/hw/ppc/spapr.h | 3 +++
> > 2 files changed, 60 insertions(+), 8 deletions(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index e214a34..1f0d232 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -64,6 +64,7 @@
> >
> > #include "hw/compat.h"
> > #include "qemu-common.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> >
> > #include <libfdt.h>
> >
> > @@ -1720,6 +1721,21 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
> > }
> > }
> >
> > +/*
> > + * Check to see if core is being hot-plugged into an already populated slot.
> > + */
> > +static void spapr_cpu_core_allow_set_link(Object *obj, const char *name,
> > + Object *val, Error **errp)
> > +{
> > + Object *core = object_property_get_link(qdev_get_machine(), name, NULL);
> > +
> > + if (core) {
> > + char *path = object_get_canonical_path(core);
> > + error_setg(errp, "Slot %s already populated with %s", name, path);
> > + g_free(path);
> > + }
> > +}
> > +
> > /* pSeries LPAR / sPAPR hardware init */
> > static void ppc_spapr_init(MachineState *machine)
> > {
> > @@ -1728,7 +1744,6 @@ static void ppc_spapr_init(MachineState *machine)
> > const char *kernel_filename = machine->kernel_filename;
> > const char *kernel_cmdline = machine->kernel_cmdline;
> > const char *initrd_filename = machine->initrd_filename;
> > - PowerPCCPU *cpu;
> > PCIHostState *phb;
> > int i;
> > MemoryRegion *sysmem = get_system_memory();
> > @@ -1742,6 +1757,8 @@ static void ppc_spapr_init(MachineState *machine)
> > long load_limit, fw_size;
> > bool kernel_le = false;
> > char *filename;
> > + int spapr_cores = smp_cpus / smp_threads;
> > + int spapr_max_cores = max_cpus / smp_threads;
> >
> > msi_supported = true;
> >
> > @@ -1800,13 +1817,38 @@ static void ppc_spapr_init(MachineState *machine)
> > if (machine->cpu_model == NULL) {
> > machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> > }
> > - for (i = 0; i < smp_cpus; i++) {
> > - cpu = cpu_ppc_init(machine->cpu_model);
> > - if (cpu == NULL) {
> > - error_report("Unable to find PowerPC CPU definition");
> > - exit(1);
> > +
> > + spapr->cores = g_malloc0(spapr_max_cores * sizeof(Object));
> souldn't it be sizeof(Object *)
Yes, I meant to store the pointers to Object here, will change.
>
> > +
> > + for (i = 0; i < spapr_max_cores; i++) {
> > + Object *spapr_cpu_core = object_new(TYPE_SPAPR_CPU_CORE);
> you allocate spapr_max_cores cores but set links to only to spapr_cores only,
> it looks like all spapr_cpu_core objects are leaked for range spapr_cores..spapr_max_cores
>
Yes, as I replied in an ealier thread, the intention is to create links
for all possible cores, but create only those many cores required for CPUs
specified with -smp from machine init. The links will be set only for those
cores. Hotplugged cores will get created and the links will be set for them
during device_add. So the changed code now looks like this:
for (i = 0; i < spapr_max_cores; i++) {
char name[32];
/*
* Create links from machine objects to all possible cores.
*/
snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
(Object **)&spapr->cores[i],
spapr_cpu_core_allow_set_link, 0,
&error_fatal);
/*
* Create cores and set link from machine object to core object for
* boot time CPUs and realize them.
*/
if (i < spapr_cores) {
Object *core = object_new(TYPE_SPAPR_CPU_CORE);
object_property_set_str(core, machine->cpu_model, "cpu_model",
&error_fatal);
object_property_set_int(core, smp_threads, "nr_threads",
&error_fatal);
object_property_set_str(core, name, SPAPR_CPU_CORE_SLOT_PROP,
&error_fatal);
object_property_set_bool(core, true, "realized", &error_fatal);
}
}
>
> > + char name[32];
> > +
> > + object_property_set_str(spapr_cpu_core, machine->cpu_model, "cpu_model",
> > + &error_fatal);
> > + object_property_set_int(spapr_cpu_core, smp_threads, "nr_threads",
> > + &error_fatal);
> > + /*
> > + * Create links from machine objects to all possible cores.
> > + */
> > + snprintf(name, sizeof(name), "%s[%d]", SPAPR_MACHINE_CPU_CORE_PROP, i);
> > + object_property_add_link(OBJECT(spapr), name, TYPE_SPAPR_CPU_CORE,
> > + (Object **)&spapr->cores[i],
> > + spapr_cpu_core_allow_set_link, 0,
> > + &error_fatal);
> > +
> > + /*
> > + * Set the link from machine object to core object for all
> > + * boot time CPUs specified with -smp and realize them.
> > + * For rest of the hotpluggable cores this is happens from
> > + * the core hotplug/realization path.
> > + */
> > + if (i < spapr_cores) {
> > + object_property_set_str(spapr_cpu_core, name,
> > + SPAPR_CPU_CORE_SLOT_PROP, &error_fatal);
> > + object_property_set_bool(spapr_cpu_core, true, "realized",
> > + &error_fatal);
> > }
> > - spapr_cpu_init(spapr, cpu, &error_fatal);
> > }
> >
> > if (kvm_enabled()) {
> > @@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > DeviceState *dev, Error **errp)
> > {
> > sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
> >
> > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> > int node;
> > @@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > }
> >
> > spapr_memory_plug(hotplug_dev, dev, node, errp);
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> here probably should be CORE and not TYPE_CPU,
> then if board needs to set some state for child threads it
> could ennumerate child of core here and do the required job.
Hmm, we have things to set for CPUs and cores as well from hotplug
handler. So the above code is for CPUs and I have spapr_core_plug()
which is called conditionally when device is of type TYPE_SPAPR_CPU_CORE.
Given that ->plug() is called during realization of both individual CPU
threads as well as their parent core, I thought handling the setup for
them separately like the above is simpler, no ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-26 10:46 ` Thomas Huth
@ 2016-02-29 5:39 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 5:39 UTC (permalink / raw)
To: Thomas Huth
Cc: mjrosato, agraf, pkrempa, ehabkost, aik, qemu-devel, armbru,
borntraeger, qemu-ppc, pbonzini, imammedo, mdroth, afaerber,
david
On Fri, Feb 26, 2016 at 11:46:19AM +0100, Thomas Huth wrote:
> On 25.02.2016 17:22, Bharata B Rao wrote:
> > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > Creating this core device will result in creation of all the CPU thread
> > devices that are part of this core.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> ...
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > new file mode 100644
> > index 0000000..c44eb61
> > --- /dev/null
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -0,0 +1,210 @@
> > +/*
> > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > + *
> > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#include "hw/cpu/core.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> > +#include "hw/ppc/spapr.h"
> > +#include "hw/boards.h"
> > +#include "qemu/error-report.h"
> > +#include "qapi/visitor.h"
> > +#include <sysemu/cpus.h>
> > +
> > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > +{
> > + Error **errp = opaque;
> > +
> > + object_property_set_bool(child, true, "realized", errp);
> > + if (*errp) {
> > + return 1;
> > + }
> > + return 0;
> > +}
> > +
> > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > + Error *local_err = NULL;
> > +
> > + if (!core->nr_threads) {
> > + error_setg(errp, "nr_threads property can't be 0");
> > + return;
> > + }
> > +
> > + if (!core->cpu_model) {
> > + error_setg(errp, "cpu_model property isn't set");
> > + return;
> > + }
> > +
> > + /*
> > + * TODO: If slot isn't specified, plug this core into
> > + * an existing empty slot.
> > + */
> > + if (!core->slot) {
> > + error_setg(errp, "slot property isn't set");
> > + return;
> > + }
> > +
> > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > + &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > +}
> > +
> > +/*
> > + * This creates the CPU threads for a given @core.
> > + *
> > + * In order to create the threads, we need two inputs - number of
> > + * threads and the cpu_model. These are set as core object's properties.
> > + * When both of them become available/set, this routine will be called from
> > + * either property's set handler to create the threads.
> > + *
> > + * TODO: Dependence of threads creation on two properties is resulting
> > + * in this not-so-clean way of creating threads from either of the
> > + * property setters based on the order in which they get set. Check if
> > + * this can be handled in a better manner.
> > + */
> > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < core->nr_threads; i++) {
> > + char id[32];
> > + char type[32];
> > +
> > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > + TYPE_POWERPC_CPU);
> > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > +
> > + snprintf(id, sizeof(id), "thread[%d]", i);
> > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > + errp);
>
> Need to check errp here to see whether something went wrong?
Yes, I will use local_err and them propagate.
>
> > + }
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->slot;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + core->slot = g_strdup(val);
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->cpu_model;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + MachineState *machine = MACHINE(qdev_get_machine());
> > +
> > + /*
> > + * cpu_model can't be different from what is specified with -cpu
> > + */
> > + if (strcmp(val, machine->cpu_model)) {
> > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
>
> s/should/must/
sure :)
>
> > + return;
> > + }
> > +
> > + core->cpu_model = g_strdup(val);
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + int64_t value = core->nr_threads;
> > +
> > + visit_type_int(v, name, &value, errp);
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + Error *local_err = NULL;
> > + int64_t value;
> > +
> > + visit_type_int(v, name, &value, &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + /* Allow only homogeneous configuration */
> > + if (value != smp_threads) {
> > + error_setg(errp, "nr_threads should be %d", smp_threads);
>
> s/should/must/
>
> > + return;
> > + }
> > +
> > + core->nr_threads = value;
> > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
>
> I think it's preferable to use g_new0 for such allocations instead.
Ok.
>
> Also, should this memory maybe be freed during instance_finalize again,
> so that there is no memory leak here in case the cores are deleted again
> one day?
I will be forced to care of this when I add hot removal in the next version.
>
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_instance_init(Object *obj)
> > +{
> > + object_property_add(obj, "nr_threads", "int",
> > + spapr_cpu_core_prop_get_nr_threads,
> > + spapr_cpu_core_prop_set_nr_threads,
> > + NULL, NULL, NULL);
> > + object_property_add_str(obj, "cpu_model",
> > + spapr_cpu_core_prop_get_cpu_model,
> > + spapr_cpu_core_prop_set_cpu_model,
> > + NULL);
> > + object_property_add_str(obj, "slot",
> > + spapr_cpu_core_prop_get_slot,
> > + spapr_cpu_core_prop_set_slot,
> > + NULL);
> > +}
> > +
> > +static void spapr_cpu_core_class_init(ObjectClass *oc, void *data)
> > +{
> > + DeviceClass *dc = DEVICE_CLASS(oc);
> > +
> > + dc->realize = spapr_cpu_core_realize;
> > +}
> > +
> > +static const TypeInfo spapr_cpu_core_type_info = {
> > + .name = TYPE_SPAPR_CPU_CORE,
> > + .parent = TYPE_CPU_CORE,
> > + .instance_init = spapr_cpu_core_instance_init,
> > + .instance_size = sizeof(sPAPRCPUCore),
> > + .class_init = spapr_cpu_core_class_init,
> > +};
> > +
> > +static void spapr_cpu_core_register_types(void)
> > +{
> > + type_register_static(&spapr_cpu_core_type_info);
> > +}
> > +
> > +type_init(spapr_cpu_core_register_types)
> > diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> > new file mode 100644
> > index 0000000..ed9bc7f
> > --- /dev/null
> > +++ b/include/hw/ppc/spapr_cpu_core.h
> > @@ -0,0 +1,32 @@
> > +/*
> > + * sPAPR CPU core device.
> > + *
> > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#ifndef HW_SPAPR_CPU_CORE_H
> > +#define HW_SPAPR_CPU_CORE_H
> > +
> > +#include "hw/qdev.h"
> > +#include "hw/cpu/core.h"
> > +
> > +#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
> > +#define SPAPR_CPU_CORE(obj) \
> > + OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
> > +
> > +typedef struct sPAPRCPUCore {
> > + /*< private >*/
> > + DeviceState parent_obj;
> > +
> > + /*< public >*/
> > + int nr_threads;
> > + char *cpu_model;
> > + char *slot;
>
> <bikeshedpainting>
> I'd maybe call that "slot_name" instead ... if you only call it "slot",
> I'd rather think of an integer value than a string here.
> </bikeshedpainting>
Point taken, pc-dimm has slot which is a number :)
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-26 15:58 ` Eric Blake
@ 2016-02-29 5:43 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 5:43 UTC (permalink / raw)
To: Eric Blake
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
On Fri, Feb 26, 2016 at 08:58:05AM -0700, Eric Blake wrote:
> On 02/25/2016 09:22 AM, Bharata B Rao wrote:
> > Implement query cpu-slots that provides information about hot-plugged
> > as well as hot-pluggable CPU slots that the machine supports.
> >
> > TODO: As Eric suggested use enum for type instead of str.
> > TODO: @hotplug-granularity probably isn't required.
>
> I guess this is still marked TODO because the series is still RFC?
Yes.
>
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
>
> > +++ b/qapi-schema.json
> > @@ -4083,3 +4083,88 @@
> > ##
> > { 'enum': 'ReplayMode',
> > 'data': [ 'none', 'record', 'play' ] }
> > +
> > +##
> > +# @CPUInfo:
> > +#
> > +# Information about CPUs
> > +#
> > +# @arch-id: Arch-specific ID for the CPU.
> > +#
> > +# @type: QOM type of the CPU.
> > +#
> > +# @thread: Thread ID of the CPU.
> > +#
> > +# @core: Core ID of the CPU.
> > +#
> > +# @socket: Socket ID of the CPU.
> > +#
> > +# @node: NUMA node to which the CPU belongs.
>
> Please add the '#optional' tag to the fields which are not always present.
>
Sure.
> > +#
> > +# @qom-path: QOM path of the CPU object
> > +#
> > +# Since: 2.6
> > +##
> > +
> > +{ 'struct': 'CPUInfo',
> > + 'data': { 'arch-id': 'int',
> > + 'type': 'str',
>
> The TODO in the commit message mentions that this should be converted to
> an enum.
>
> > + '*thread': 'int',
> > + '*core': 'int',
> > + '*socket' : 'int',
> > + '*node' : 'int',
> > + '*qom-path': 'str'
> > + }
>
> But looking better than the previous round.
Thanks for the review. Do you have any comments on the applicability/suitability
of this interface from libvirt point of view for performing device_add based
CPU hotplug ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-26 18:13 ` Michael Roth
2016-02-29 3:44 ` David Gibson
@ 2016-02-29 5:50 ` Bharata B Rao
2016-02-29 10:03 ` Igor Mammedov
1 sibling, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 5:50 UTC (permalink / raw)
To: Michael Roth
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
david
On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> Quoting Bharata B Rao (2016-02-25 10:22:38)
> > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > Creating this core device will result in creation of all the CPU thread
> > devices that are part of this core.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/Makefile.objs | 1 +
> > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > 3 files changed, 243 insertions(+)
> > create mode 100644 hw/ppc/spapr_cpu_core.c
> > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> >
> > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > index c1ffc77..5cc6608 100644
> > --- a/hw/ppc/Makefile.objs
> > +++ b/hw/ppc/Makefile.objs
> > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > obj-y += spapr_pci_vfio.o
> > endif
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > new file mode 100644
> > index 0000000..c44eb61
> > --- /dev/null
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -0,0 +1,210 @@
> > +/*
> > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > + *
> > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#include "hw/cpu/core.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> > +#include "hw/ppc/spapr.h"
> > +#include "hw/boards.h"
> > +#include "qemu/error-report.h"
> > +#include "qapi/visitor.h"
> > +#include <sysemu/cpus.h>
> > +
> > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > +{
> > + Error **errp = opaque;
> > +
> > + object_property_set_bool(child, true, "realized", errp);
> > + if (*errp) {
> > + return 1;
> > + }
> > + return 0;
> > +}
> > +
> > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > + Error *local_err = NULL;
> > +
> > + if (!core->nr_threads) {
> > + error_setg(errp, "nr_threads property can't be 0");
> > + return;
> > + }
> > +
> > + if (!core->cpu_model) {
> > + error_setg(errp, "cpu_model property isn't set");
> > + return;
> > + }
> > +
> > + /*
> > + * TODO: If slot isn't specified, plug this core into
> > + * an existing empty slot.
> > + */
> > + if (!core->slot) {
> > + error_setg(errp, "slot property isn't set");
> > + return;
> > + }
> > +
> > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > + &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > +}
> > +
> > +/*
> > + * This creates the CPU threads for a given @core.
> > + *
> > + * In order to create the threads, we need two inputs - number of
> > + * threads and the cpu_model. These are set as core object's properties.
> > + * When both of them become available/set, this routine will be called from
> > + * either property's set handler to create the threads.
> > + *
> > + * TODO: Dependence of threads creation on two properties is resulting
> > + * in this not-so-clean way of creating threads from either of the
> > + * property setters based on the order in which they get set. Check if
> > + * this can be handled in a better manner.
> > + */
> > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < core->nr_threads; i++) {
> > + char id[32];
> > + char type[32];
> > +
> > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > + TYPE_POWERPC_CPU);
> > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > +
> > + snprintf(id, sizeof(id), "thread[%d]", i);
> > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > + errp);
> > + }
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->slot;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + core->slot = g_strdup(val);
> > +}
> > +
> > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > +
> > + return core->cpu_model;
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + MachineState *machine = MACHINE(qdev_get_machine());
> > +
> > + /*
> > + * cpu_model can't be different from what is specified with -cpu
> > + */
> > + if (strcmp(val, machine->cpu_model)) {
> > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > + return;
> > + }
> > +
> > + core->cpu_model = g_strdup(val);
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + int64_t value = core->nr_threads;
> > +
> > + visit_type_int(v, name, &value, errp);
> > +}
> > +
> > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > + const char *name, void *opaque,
> > + Error **errp)
> > +{
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > + Error *local_err = NULL;
> > + int64_t value;
> > +
> > + visit_type_int(v, name, &value, &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return;
> > + }
> > +
> > + /* Allow only homogeneous configuration */
> > + if (value != smp_threads) {
> > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > + return;
> > + }
> > +
> > + core->nr_threads = value;
> > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > +
> > + if (core->nr_threads && core->cpu_model) {
> > + spapr_cpu_core_create_threads(core, errp);
> > + }
> > +}
> > +
> > +static void spapr_cpu_core_instance_init(Object *obj)
> > +{
> > + object_property_add(obj, "nr_threads", "int",
> > + spapr_cpu_core_prop_get_nr_threads,
> > + spapr_cpu_core_prop_set_nr_threads,
> > + NULL, NULL, NULL);
> > + object_property_add_str(obj, "cpu_model",
> > + spapr_cpu_core_prop_get_cpu_model,
> > + spapr_cpu_core_prop_set_cpu_model,
> > + NULL);
> > + object_property_add_str(obj, "slot",
> > + spapr_cpu_core_prop_get_slot,
> > + spapr_cpu_core_prop_set_slot,
> > + NULL);
>
> "slot" seems intended to be a machine-agnostic of mapping device
> types discovered from qmp_query_cpu_slots() to an appropriate
> "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> It seems like maybe TYPE_CPU_CORE is a better place, but then on
> x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
Correct.
>
> It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> right approach, but I don't know how we could expose that as
> a property. I guess it's somewhat implied that this "interface"
> exists if qmp_query_cpu_slots() returns the type, but I wonder
> if something a bit more formal should be modeled to make the
> implementation requirements a bit clearer.
>
> Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> class method, expose them via "slot" property, then have the
> defaults generate "not implemented" errors?
Yes makes sense. In fact David has often times said that generic
properties/routines should be pushed to base class wherever possible.
I didn't do that in this first iteration to keep the generic changes
as minimum as possible, but yes slot should be a property of the
base class of core or socket.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices
2016-02-29 5:35 ` Bharata B Rao
@ 2016-02-29 7:11 ` David Gibson
0 siblings, 0 replies; 45+ messages in thread
From: David Gibson @ 2016-02-29 7:11 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, Igor Mammedov, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]
On Mon, Feb 29, 2016 at 11:05:32AM +0530, Bharata B Rao wrote:
> On Fri, Feb 26, 2016 at 04:18:57PM +0100, Igor Mammedov wrote:
> > On Thu, 25 Feb 2016 21:52:39 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
[snip]
> > > @@ -2209,6 +2251,7 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > > DeviceState *dev, Error **errp)
> > > {
> > > sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > > + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
> > >
> > > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> > > int node;
> > > @@ -2245,6 +2288,11 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > > }
> > >
> > > spapr_memory_plug(hotplug_dev, dev, node, errp);
> > > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > here probably should be CORE and not TYPE_CPU,
> > then if board needs to set some state for child threads it
> > could ennumerate child of core here and do the required job.
>
> Hmm, we have things to set for CPUs and cores as well from hotplug
> handler. So the above code is for CPUs and I have spapr_core_plug()
> which is called conditionally when device is of type TYPE_SPAPR_CPU_CORE.
>
> Given that ->plug() is called during realization of both individual CPU
> threads as well as their parent core, I thought handling the setup for
> them separately like the above is simpler, no ?
So, this bothered me a bit as well. I think having the hotplug
handler for the threads is misleading, since it suggests you can
individually hotplug them, when in fact the intention is just that
this is part of the hotplug code path for a whole core.
My suggestion earlier was to have the core realize code itself perform
this configuration on the sub-threads. However, Igor's suggestion of,
essentially, an explicit loop over the threads in the core hotplug
handler would be fine by me also.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-29 5:50 ` Bharata B Rao
@ 2016-02-29 10:03 ` Igor Mammedov
2016-02-29 12:55 ` Bharata B Rao
0 siblings, 1 reply; 45+ messages in thread
From: Igor Mammedov @ 2016-02-29 10:03 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Mon, 29 Feb 2016 11:20:19 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > Creating this core device will result in creation of all the CPU thread
> > > devices that are part of this core.
> > >
> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > ---
> > > hw/ppc/Makefile.objs | 1 +
> > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > 3 files changed, 243 insertions(+)
> > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > >
> > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > index c1ffc77..5cc6608 100644
> > > --- a/hw/ppc/Makefile.objs
> > > +++ b/hw/ppc/Makefile.objs
> > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > obj-y += spapr_pci_vfio.o
> > > endif
> > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > new file mode 100644
> > > index 0000000..c44eb61
> > > --- /dev/null
> > > +++ b/hw/ppc/spapr_cpu_core.c
> > > @@ -0,0 +1,210 @@
> > > +/*
> > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > + *
> > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > + *
> > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > + * See the COPYING file in the top-level directory.
> > > + */
> > > +#include "hw/cpu/core.h"
> > > +#include "hw/ppc/spapr_cpu_core.h"
> > > +#include "hw/ppc/spapr.h"
> > > +#include "hw/boards.h"
> > > +#include "qemu/error-report.h"
> > > +#include "qapi/visitor.h"
> > > +#include <sysemu/cpus.h>
> > > +
> > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > +{
> > > + Error **errp = opaque;
> > > +
> > > + object_property_set_bool(child, true, "realized", errp);
> > > + if (*errp) {
> > > + return 1;
> > > + }
> > > + return 0;
> > > +}
> > > +
> > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > + Error *local_err = NULL;
> > > +
> > > + if (!core->nr_threads) {
> > > + error_setg(errp, "nr_threads property can't be 0");
> > > + return;
> > > + }
> > > +
> > > + if (!core->cpu_model) {
> > > + error_setg(errp, "cpu_model property isn't set");
> > > + return;
> > > + }
> > > +
> > > + /*
> > > + * TODO: If slot isn't specified, plug this core into
> > > + * an existing empty slot.
> > > + */
> > > + if (!core->slot) {
> > > + error_setg(errp, "slot property isn't set");
> > > + return;
> > > + }
> > > +
> > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > + &local_err);
> > > + if (local_err) {
> > > + error_propagate(errp, local_err);
> > > + return;
> > > + }
> > > +
> > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > +}
> > > +
> > > +/*
> > > + * This creates the CPU threads for a given @core.
> > > + *
> > > + * In order to create the threads, we need two inputs - number of
> > > + * threads and the cpu_model. These are set as core object's properties.
> > > + * When both of them become available/set, this routine will be called from
> > > + * either property's set handler to create the threads.
> > > + *
> > > + * TODO: Dependence of threads creation on two properties is resulting
> > > + * in this not-so-clean way of creating threads from either of the
> > > + * property setters based on the order in which they get set. Check if
> > > + * this can be handled in a better manner.
> > > + */
> > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > +{
> > > + int i;
> > > +
> > > + for (i = 0; i < core->nr_threads; i++) {
> > > + char id[32];
> > > + char type[32];
> > > +
> > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > + TYPE_POWERPC_CPU);
> > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > +
> > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > + errp);
> > > + }
> > > +}
> > > +
> > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > +
> > > + return core->slot;
> > > +}
> > > +
> > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > + Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > +
> > > + core->slot = g_strdup(val);
> > > +}
> > > +
> > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > +
> > > + return core->cpu_model;
> > > +}
> > > +
> > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > + Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > +
> > > + /*
> > > + * cpu_model can't be different from what is specified with -cpu
> > > + */
> > > + if (strcmp(val, machine->cpu_model)) {
> > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > + return;
> > > + }
> > > +
> > > + core->cpu_model = g_strdup(val);
> > > + if (core->nr_threads && core->cpu_model) {
> > > + spapr_cpu_core_create_threads(core, errp);
> > > + }
> > > +}
> > > +
> > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > + const char *name, void *opaque,
> > > + Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > + int64_t value = core->nr_threads;
> > > +
> > > + visit_type_int(v, name, &value, errp);
> > > +}
> > > +
> > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > + const char *name, void *opaque,
> > > + Error **errp)
> > > +{
> > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > + Error *local_err = NULL;
> > > + int64_t value;
> > > +
> > > + visit_type_int(v, name, &value, &local_err);
> > > + if (local_err) {
> > > + error_propagate(errp, local_err);
> > > + return;
> > > + }
> > > +
> > > + /* Allow only homogeneous configuration */
> > > + if (value != smp_threads) {
> > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > + return;
> > > + }
> > > +
> > > + core->nr_threads = value;
> > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > +
> > > + if (core->nr_threads && core->cpu_model) {
> > > + spapr_cpu_core_create_threads(core, errp);
> > > + }
> > > +}
> > > +
> > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > +{
> > > + object_property_add(obj, "nr_threads", "int",
> > > + spapr_cpu_core_prop_get_nr_threads,
> > > + spapr_cpu_core_prop_set_nr_threads,
> > > + NULL, NULL, NULL);
> > > + object_property_add_str(obj, "cpu_model",
> > > + spapr_cpu_core_prop_get_cpu_model,
> > > + spapr_cpu_core_prop_set_cpu_model,
> > > + NULL);
> > > + object_property_add_str(obj, "slot",
> > > + spapr_cpu_core_prop_get_slot,
> > > + spapr_cpu_core_prop_set_slot,
> > > + NULL);
> >
> > "slot" seems intended to be a machine-agnostic of mapping device
> > types discovered from qmp_query_cpu_slots() to an appropriate
> > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
>
> Correct.
>
> >
> > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > right approach, but I don't know how we could expose that as
> > a property. I guess it's somewhat implied that this "interface"
> > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > if something a bit more formal should be modeled to make the
> > implementation requirements a bit clearer.
> >
> > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > class method, expose them via "slot" property, then have the
> > defaults generate "not implemented" errors?
>
> Yes makes sense. In fact David has often times said that generic
> properties/routines should be pushed to base class wherever possible.
>
> I didn't do that in this first iteration to keep the generic changes
> as minimum as possible, but yes slot should be a property of the
> base class of core or socket.
Then what will happen to slot if there isn't any core/socket device
to query it, i.e. cpu hasn't been plugged in yet?
To me slot looks like a machine belonged feature.
>
> Regards,
> Bharata.
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
` (2 preceding siblings ...)
2016-02-26 16:33 ` Thomas Huth
@ 2016-02-29 10:46 ` Igor Mammedov
2016-03-01 9:09 ` Bharata B Rao
3 siblings, 1 reply; 45+ messages in thread
From: Igor Mammedov @ 2016-02-29 10:46 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Thu, 25 Feb 2016 21:52:41 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> Implement query cpu-slots that provides information about hot-plugged
> as well as hot-pluggable CPU slots that the machine supports.
>
> TODO: As Eric suggested use enum for type instead of str.
> TODO: @hotplug-granularity probably isn't required.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/core/machine.c | 19 +++++++++
> hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> include/hw/boards.h | 4 ++
> qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> qmp-commands.hx | 47 ++++++++++++++++++++++
> 5 files changed, 267 insertions(+)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 6d1a0d8..3055ef8 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -17,6 +17,25 @@
> #include "hw/sysbus.h"
> #include "sysemu/sysemu.h"
> #include "qemu/error-report.h"
> +#include "qmp-commands.h"
> +
> +/*
> + * QMP: query-cpu-slots
> + *
> + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> + */
> +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> +{
> + MachineState *ms = MACHINE(qdev_get_machine());
> + MachineClass *mc = MACHINE_GET_CLASS(ms);
> +
> + if (!mc->cpu_slots) {
> + error_setg(errp, QERR_UNSUPPORTED);
> + return NULL;
> + }
> +
> + return mc->cpu_slots(ms);
> +}
>
> static char *machine_get_accel(Object *obj, Error **errp)
> {
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 780cd00..b76ed85 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> return cpu_index / smp_threads / smp_cores;
> }
>
> +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> +{
> + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> + CPUInfoList ***prev = opaque;
> +
> + if (object_dynamic_cast(obj, TYPE_CPU)) {
> + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> + CPUInfo *s = g_new0(CPUInfo, 1);
> + CPUState *cpu = CPU(obj);
> + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> +
> + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> + s->type = g_strdup(object_get_typename(obj));
> + s->thread = cpu->cpu_index;
> + s->has_thread = true;
> + s->core = cpu->cpu_index / smp_threads;
> + s->has_core = true;
> + if (mc->cpu_index_to_socket_id) {
> + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> + } else {
> + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> + }
> + s->has_socket = true;
> + s->node = cpu->numa_node;
> + s->has_node = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> +
> + elem->value = s;
> + elem->next = NULL;
> + **prev = elem;
> + *prev = &elem->next;
> + }
> + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> + return 0;
> +}
> +
> +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> +{
> + CPUSlotInfoList *head = NULL;
> + CPUSlotInfoList **prev = &head;
> + Object *root_container;
> + ObjectProperty *prop;
> + ObjectPropertyIterator iter;
> +
> + /*
> + * TODO: There surely must be a better/easier way to walk all
> + * the link properties of an object ?
> + */
> + root_container = container_get(object_get_root(), "/machine");
> + object_property_iter_init(&iter, root_container);
> +
> + while ((prop = object_property_iter_next(&iter))) {
> + Object *obj;
> + DeviceState *dev;
> + CPUSlotInfoList *elem;
> + CPUSlotInfo *s;
> + CPUInfoList *cpu_head = NULL;
> + CPUInfoList **cpu_prev = &cpu_head;
> + sPAPRCPUCore *core;
> +
> + if (!strstart(prop->type, "link<", NULL)) {
> + continue;
> + }
> +
> + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> + continue;
> + }
> +
> + elem = g_new0(CPUSlotInfoList, 1);
> + s = g_new0(CPUSlotInfo, 1);
> +
> + obj = object_property_get_link(root_container, prop->name, NULL);
> + if (obj) {
> + /* Slot populated */
> + dev = DEVICE(obj);
> + core = SPAPR_CPU_CORE(obj);
> +
> + if (dev->id) {
> + s->has_id = true;
> + s->id = g_strdup(dev->id);
> + }
> + s->realized = object_property_get_bool(obj, "realized", NULL);
> + s->nr_cpus = core->nr_threads;
> + s->has_nr_cpus = true;
> + s->qom_path = object_get_canonical_path(obj);
> + s->has_qom_path = true;
> + if (s->realized) {
> + spapr_cpuinfo_list(obj, &cpu_prev);
> + }
> + s->has_cpus = true;
> + } else {
> + /* Slot empty */
> + s->has_id = false;
> + s->has_nr_cpus = false;
> + s->has_qom_path = false;
> + s->has_cpus = false;
> + s->realized = false;
> + }
> + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> + s->slot_id = g_strdup(prop->name);
> + s->cpus = cpu_head;
> + elem->value = s;
> + elem->next = NULL;
> + *prev = elem;
> + prev = &elem->next;
> + }
> + return head;
> +}
> +
> static void spapr_machine_class_init(ObjectClass *oc, void *data)
> {
> MachineClass *mc = MACHINE_CLASS(oc);
> @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> hc->plug = spapr_machine_device_plug;
> hc->unplug = spapr_machine_device_unplug;
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> + mc->cpu_slots = spapr_cpu_slots;
>
> smc->dr_lmb_enabled = true;
> smc->dr_cpu_enabled = true;
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 0f30959..d888a02 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> * Set only by old machines because they need to keep
> * compatibility on code that exposed QEMU_VERSION to guests in
> * the past (and now use qemu_hw_version()).
> + * @cpu_slots:
> + * Provides information about populated and yet-to-be populated
> + * CPU slots in the machine. Used by QMP query-cpu-slots.
> */
> struct MachineClass {
> /*< private >*/
> @@ -99,6 +102,7 @@ struct MachineClass {
> HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> DeviceState *dev);
> unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> };
>
> /**
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 8d04897..e9a52a2 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -4083,3 +4083,88 @@
> ##
> { 'enum': 'ReplayMode',
> 'data': [ 'none', 'record', 'play' ] }
> +
> +##
> +# @CPUInfo:
> +#
> +# Information about CPUs
> +#
> +# @arch-id: Arch-specific ID for the CPU.
> +#
> +# @type: QOM type of the CPU.
> +#
> +# @thread: Thread ID of the CPU.
> +#
> +# @core: Core ID of the CPU.
> +#
> +# @socket: Socket ID of the CPU.
> +#
> +# @node: NUMA node to which the CPU belongs.
> +#
> +# @qom-path: QOM path of the CPU object
> +#
> +# Since: 2.6
> +##
> +
> +{ 'struct': 'CPUInfo',
> + 'data': { 'arch-id': 'int',
> + 'type': 'str',
> + '*thread': 'int',
> + '*core': 'int',
> + '*socket' : 'int',
> + '*node' : 'int',
> + '*qom-path': 'str'
> + }
> +}
> +
> +##
> +# @CPUSlotInfo:
> +#
> +# Information about CPU Slots
> +#
> +# @id: Device ID of the CPU composite object that occupies the slot.
> +#
> +# @type: QOM type of the CPU composite object that occupies the slot.
> +#
> +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> +# can be thread, core or socket.
> +#
> +# @slot-id: Slot's identifier.
> +#
> +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> +#
> +# @realized: A boolean that indicates whether the slot is filled or empty.
> +#
> +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> +# this slot.
> +#
> +# @cpus: An array of @CPUInfo elements where each element describes one
> +# CPU that is part of this slot's CPU composite object.
> +#
> +# @type: QOM type
> +#
> +# Since: 2.6
> +##
> +
> +{ 'struct': 'CPUSlotInfo',
> + 'data': { '*id': 'str',
> + 'type': 'str',
> + 'hotplug-granularity' : 'str',
Does it convey any useful info, if yes how it will be used by mgmt?
> + 'slot-id' : 'str',
> + '*qom-path': 'str',
> + 'realized': 'bool',
field's redundant, presence of qom-path answers this question
> + '*nr-cpus': 'int',
> + '*cpus' : ['CPUInfo']
I'd suggest to drop above 2 fields as it's impl dependent,
qom-path already point's to socket/core/thread object and
its internal composition can be explored by other means if needed.
Moreover 'CPUInfo' is confusing and sort of conflicts with existing
'CpuInfo'.
I'd drop CPUInfo altogether and introduce only 'CPUSlotInfo' here,
existing thread enumeration's already implemented query-cpus.
> + }
> +}
What I miss here is that CPUSlotInfo doesn't provide any
information to about where CPU might be hotplugged to.
Maybe use following tuple instead of slot-id?
{ 'struct': 'CPUSlotProperties',
'data': { '*node': 'int',
'*socket': 'int',
'*core': 'int',
'*thread': 'int'
}
}
it's generic for the most targets, which should work for spapr, s390, x86, ARM
and could be extended for other cases adding other board specific
properties if it's needed.
The all mgmt would have to do for hotplug is to execute:
device_add ${type},${CPUSlotProperties}
> +
> +##
> +# @query-cpu-slots:
> +#
> +# Returns information for all CPU slots
> +#
> +# Returns: a list of @CPUSlotInfo
> +#
> +# Since: 2.6
> +##
> +{ 'command': 'query-cpu-slots', 'returns': ['CPUSlotInfo'] }
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index 085dc7d..185aa13 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -4825,3 +4825,50 @@ Example:
> {"type": 0, "out-pport": 0, "pport": 0, "vlan-id": 3840,
> "pop-vlan": 1, "id": 251658240}
> ]}
> +
> +EQMP
> +
> + {
> + .name = "query-cpu-slots",
> + .args_type = "",
> + .mhandler.cmd_new = qmp_marshal_query_cpu_slots,
> + },
> +
> +SQMP
> +@query-cpu-slots
> +--------------------
> +
> +Show CPU slots information
> +
> +Example:
> +-> { "execute": "query-cpu-slots" }
> +<- {"return": [{
> + "slot-id": "core[2]",
> + "hotplug-granularity": "core",
> + "realized": false,
> + "type": "spapr-cpu-core"
> + },
> + {
> + "slot-id": "core[1]",
> + "qom-path": "/machine/unattached/device[2]",
> + "hotplug-granularity": "core",
> + "realized": true,
> + "type": "spapr-cpu-core"
> + "nr-cpus": 2,
> + "cpus": [
> + {"thread": 8,
> + "socket": 0,
> + "core": 1,
> + "arch-id": 8,
> + "qom-path": "/machine/unattached/device[2]/thread[0]",
> + "node": 0,
> + "type": "host-powerpc64-cpu"},
> + {"thread": 9,
> + "socket": 0,
> + "core": 1,
> + "arch-id": 9,
> + "qom-path": "/machine/unattached/device[2]/thread[2]",
> + "node": 0,
> + "type": "host-powerpc64-cpu"}]
> + }
> + ]}
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-29 10:03 ` Igor Mammedov
@ 2016-02-29 12:55 ` Bharata B Rao
2016-02-29 15:15 ` Igor Mammedov
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-02-29 12:55 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> On Mon, 29 Feb 2016 11:20:19 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > > Creating this core device will result in creation of all the CPU thread
> > > > devices that are part of this core.
> > > >
> > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > ---
> > > > hw/ppc/Makefile.objs | 1 +
> > > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > > 3 files changed, 243 insertions(+)
> > > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > > >
> > > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > > index c1ffc77..5cc6608 100644
> > > > --- a/hw/ppc/Makefile.objs
> > > > +++ b/hw/ppc/Makefile.objs
> > > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > > obj-y += spapr_pci_vfio.o
> > > > endif
> > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > new file mode 100644
> > > > index 0000000..c44eb61
> > > > --- /dev/null
> > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > @@ -0,0 +1,210 @@
> > > > +/*
> > > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > > + *
> > > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > + *
> > > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > + * See the COPYING file in the top-level directory.
> > > > + */
> > > > +#include "hw/cpu/core.h"
> > > > +#include "hw/ppc/spapr_cpu_core.h"
> > > > +#include "hw/ppc/spapr.h"
> > > > +#include "hw/boards.h"
> > > > +#include "qemu/error-report.h"
> > > > +#include "qapi/visitor.h"
> > > > +#include <sysemu/cpus.h>
> > > > +
> > > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > > +{
> > > > + Error **errp = opaque;
> > > > +
> > > > + object_property_set_bool(child, true, "realized", errp);
> > > > + if (*errp) {
> > > > + return 1;
> > > > + }
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > + Error *local_err = NULL;
> > > > +
> > > > + if (!core->nr_threads) {
> > > > + error_setg(errp, "nr_threads property can't be 0");
> > > > + return;
> > > > + }
> > > > +
> > > > + if (!core->cpu_model) {
> > > > + error_setg(errp, "cpu_model property isn't set");
> > > > + return;
> > > > + }
> > > > +
> > > > + /*
> > > > + * TODO: If slot isn't specified, plug this core into
> > > > + * an existing empty slot.
> > > > + */
> > > > + if (!core->slot) {
> > > > + error_setg(errp, "slot property isn't set");
> > > > + return;
> > > > + }
> > > > +
> > > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > > + &local_err);
> > > > + if (local_err) {
> > > > + error_propagate(errp, local_err);
> > > > + return;
> > > > + }
> > > > +
> > > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > > +}
> > > > +
> > > > +/*
> > > > + * This creates the CPU threads for a given @core.
> > > > + *
> > > > + * In order to create the threads, we need two inputs - number of
> > > > + * threads and the cpu_model. These are set as core object's properties.
> > > > + * When both of them become available/set, this routine will be called from
> > > > + * either property's set handler to create the threads.
> > > > + *
> > > > + * TODO: Dependence of threads creation on two properties is resulting
> > > > + * in this not-so-clean way of creating threads from either of the
> > > > + * property setters based on the order in which they get set. Check if
> > > > + * this can be handled in a better manner.
> > > > + */
> > > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > > +{
> > > > + int i;
> > > > +
> > > > + for (i = 0; i < core->nr_threads; i++) {
> > > > + char id[32];
> > > > + char type[32];
> > > > +
> > > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > > + TYPE_POWERPC_CPU);
> > > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > > +
> > > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > > + errp);
> > > > + }
> > > > +}
> > > > +
> > > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > +
> > > > + return core->slot;
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > > + Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > +
> > > > + core->slot = g_strdup(val);
> > > > +}
> > > > +
> > > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > +
> > > > + return core->cpu_model;
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > > + Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > > +
> > > > + /*
> > > > + * cpu_model can't be different from what is specified with -cpu
> > > > + */
> > > > + if (strcmp(val, machine->cpu_model)) {
> > > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > > + return;
> > > > + }
> > > > +
> > > > + core->cpu_model = g_strdup(val);
> > > > + if (core->nr_threads && core->cpu_model) {
> > > > + spapr_cpu_core_create_threads(core, errp);
> > > > + }
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > > + const char *name, void *opaque,
> > > > + Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > + int64_t value = core->nr_threads;
> > > > +
> > > > + visit_type_int(v, name, &value, errp);
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > > + const char *name, void *opaque,
> > > > + Error **errp)
> > > > +{
> > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > + Error *local_err = NULL;
> > > > + int64_t value;
> > > > +
> > > > + visit_type_int(v, name, &value, &local_err);
> > > > + if (local_err) {
> > > > + error_propagate(errp, local_err);
> > > > + return;
> > > > + }
> > > > +
> > > > + /* Allow only homogeneous configuration */
> > > > + if (value != smp_threads) {
> > > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > > + return;
> > > > + }
> > > > +
> > > > + core->nr_threads = value;
> > > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > > +
> > > > + if (core->nr_threads && core->cpu_model) {
> > > > + spapr_cpu_core_create_threads(core, errp);
> > > > + }
> > > > +}
> > > > +
> > > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > > +{
> > > > + object_property_add(obj, "nr_threads", "int",
> > > > + spapr_cpu_core_prop_get_nr_threads,
> > > > + spapr_cpu_core_prop_set_nr_threads,
> > > > + NULL, NULL, NULL);
> > > > + object_property_add_str(obj, "cpu_model",
> > > > + spapr_cpu_core_prop_get_cpu_model,
> > > > + spapr_cpu_core_prop_set_cpu_model,
> > > > + NULL);
> > > > + object_property_add_str(obj, "slot",
> > > > + spapr_cpu_core_prop_get_slot,
> > > > + spapr_cpu_core_prop_set_slot,
> > > > + NULL);
> > >
> > > "slot" seems intended to be a machine-agnostic of mapping device
> > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> >
> > Correct.
> >
> > >
> > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > right approach, but I don't know how we could expose that as
> > > a property. I guess it's somewhat implied that this "interface"
> > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > if something a bit more formal should be modeled to make the
> > > implementation requirements a bit clearer.
> > >
> > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > class method, expose them via "slot" property, then have the
> > > defaults generate "not implemented" errors?
> >
> > Yes makes sense. In fact David has often times said that generic
> > properties/routines should be pushed to base class wherever possible.
> >
> > I didn't do that in this first iteration to keep the generic changes
> > as minimum as possible, but yes slot should be a property of the
> > base class of core or socket.
> Then what will happen to slot if there isn't any core/socket device
> to query it, i.e. cpu hasn't been plugged in yet?
> To me slot looks like a machine belonged feature.
Yes slot belongs to the machine and it is represented by a link that
is created b/n the machine object and the core object that sits in
the slot.
In the context of this thread, slot is actually the slot name that
identifies the machine slot which the core occupies or will occupy after
hotplug. Thus slot name which is named slot here, it is a property of the
core device.
(qemu) device_add spapr-cpu-core,slot=core[2]
^
Regards,
Bharata.
>
> >
> > Regards,
> > Bharata.
> >
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-29 12:55 ` Bharata B Rao
@ 2016-02-29 15:15 ` Igor Mammedov
2016-03-01 1:21 ` David Gibson
2016-03-01 8:17 ` Bharata B Rao
0 siblings, 2 replies; 45+ messages in thread
From: Igor Mammedov @ 2016-02-29 15:15 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Mon, 29 Feb 2016 18:25:25 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > On Mon, 29 Feb 2016 11:20:19 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > > > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > > > Creating this core device will result in creation of all the CPU thread
> > > > > devices that are part of this core.
> > > > >
> > > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > ---
> > > > > hw/ppc/Makefile.objs | 1 +
> > > > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > > > 3 files changed, 243 insertions(+)
> > > > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > > > >
> > > > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > > > index c1ffc77..5cc6608 100644
> > > > > --- a/hw/ppc/Makefile.objs
> > > > > +++ b/hw/ppc/Makefile.objs
> > > > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > > > obj-y += spapr_pci_vfio.o
> > > > > endif
> > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > > new file mode 100644
> > > > > index 0000000..c44eb61
> > > > > --- /dev/null
> > > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > > @@ -0,0 +1,210 @@
> > > > > +/*
> > > > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > > > + *
> > > > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > + *
> > > > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > > + * See the COPYING file in the top-level directory.
> > > > > + */
> > > > > +#include "hw/cpu/core.h"
> > > > > +#include "hw/ppc/spapr_cpu_core.h"
> > > > > +#include "hw/ppc/spapr.h"
> > > > > +#include "hw/boards.h"
> > > > > +#include "qemu/error-report.h"
> > > > > +#include "qapi/visitor.h"
> > > > > +#include <sysemu/cpus.h>
> > > > > +
> > > > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > > > +{
> > > > > + Error **errp = opaque;
> > > > > +
> > > > > + object_property_set_bool(child, true, "realized", errp);
> > > > > + if (*errp) {
> > > > > + return 1;
> > > > > + }
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > > + Error *local_err = NULL;
> > > > > +
> > > > > + if (!core->nr_threads) {
> > > > > + error_setg(errp, "nr_threads property can't be 0");
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + if (!core->cpu_model) {
> > > > > + error_setg(errp, "cpu_model property isn't set");
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + /*
> > > > > + * TODO: If slot isn't specified, plug this core into
> > > > > + * an existing empty slot.
> > > > > + */
> > > > > + if (!core->slot) {
> > > > > + error_setg(errp, "slot property isn't set");
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > > > + &local_err);
> > > > > + if (local_err) {
> > > > > + error_propagate(errp, local_err);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > > > +}
> > > > > +
> > > > > +/*
> > > > > + * This creates the CPU threads for a given @core.
> > > > > + *
> > > > > + * In order to create the threads, we need two inputs - number of
> > > > > + * threads and the cpu_model. These are set as core object's properties.
> > > > > + * When both of them become available/set, this routine will be called from
> > > > > + * either property's set handler to create the threads.
> > > > > + *
> > > > > + * TODO: Dependence of threads creation on two properties is resulting
> > > > > + * in this not-so-clean way of creating threads from either of the
> > > > > + * property setters based on the order in which they get set. Check if
> > > > > + * this can be handled in a better manner.
> > > > > + */
> > > > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > > > +{
> > > > > + int i;
> > > > > +
> > > > > + for (i = 0; i < core->nr_threads; i++) {
> > > > > + char id[32];
> > > > > + char type[32];
> > > > > +
> > > > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > > > + TYPE_POWERPC_CPU);
> > > > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > > > +
> > > > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > > > + errp);
> > > > > + }
> > > > > +}
> > > > > +
> > > > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > +
> > > > > + return core->slot;
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > > > + Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > +
> > > > > + core->slot = g_strdup(val);
> > > > > +}
> > > > > +
> > > > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > +
> > > > > + return core->cpu_model;
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > > > + Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > > > +
> > > > > + /*
> > > > > + * cpu_model can't be different from what is specified with -cpu
> > > > > + */
> > > > > + if (strcmp(val, machine->cpu_model)) {
> > > > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + core->cpu_model = g_strdup(val);
> > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > + }
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > > > + const char *name, void *opaque,
> > > > > + Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > + int64_t value = core->nr_threads;
> > > > > +
> > > > > + visit_type_int(v, name, &value, errp);
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > > > + const char *name, void *opaque,
> > > > > + Error **errp)
> > > > > +{
> > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > + Error *local_err = NULL;
> > > > > + int64_t value;
> > > > > +
> > > > > + visit_type_int(v, name, &value, &local_err);
> > > > > + if (local_err) {
> > > > > + error_propagate(errp, local_err);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + /* Allow only homogeneous configuration */
> > > > > + if (value != smp_threads) {
> > > > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + core->nr_threads = value;
> > > > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > > > +
> > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > + }
> > > > > +}
> > > > > +
> > > > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > > > +{
> > > > > + object_property_add(obj, "nr_threads", "int",
> > > > > + spapr_cpu_core_prop_get_nr_threads,
> > > > > + spapr_cpu_core_prop_set_nr_threads,
> > > > > + NULL, NULL, NULL);
> > > > > + object_property_add_str(obj, "cpu_model",
> > > > > + spapr_cpu_core_prop_get_cpu_model,
> > > > > + spapr_cpu_core_prop_set_cpu_model,
> > > > > + NULL);
> > > > > + object_property_add_str(obj, "slot",
> > > > > + spapr_cpu_core_prop_get_slot,
> > > > > + spapr_cpu_core_prop_set_slot,
> > > > > + NULL);
> > > >
> > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > >
> > > Correct.
> > >
> > > >
> > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > right approach, but I don't know how we could expose that as
> > > > a property. I guess it's somewhat implied that this "interface"
> > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > if something a bit more formal should be modeled to make the
> > > > implementation requirements a bit clearer.
> > > >
> > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > class method, expose them via "slot" property, then have the
> > > > defaults generate "not implemented" errors?
> > >
> > > Yes makes sense. In fact David has often times said that generic
> > > properties/routines should be pushed to base class wherever possible.
> > >
> > > I didn't do that in this first iteration to keep the generic changes
> > > as minimum as possible, but yes slot should be a property of the
> > > base class of core or socket.
> > Then what will happen to slot if there isn't any core/socket device
> > to query it, i.e. cpu hasn't been plugged in yet?
> > To me slot looks like a machine belonged feature.
>
> Yes slot belongs to the machine and it is represented by a link that
> is created b/n the machine object and the core object that sits in
> the slot.
>
> In the context of this thread, slot is actually the slot name that
> identifies the machine slot which the core occupies or will occupy after
> hotplug. Thus slot name which is named slot here, it is a property of the
> core device.
>
> (qemu) device_add spapr-cpu-core,slot=core[2]
> ^
Is 'slot' a term used by SPAPR on real hardware?
I'd thought that it's 'core', that's why I suggested to use
'core' for POWER as that matched real world concept, see
my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
of this series.
> Regards,
> Bharata.
> >
> > >
> > > Regards,
> > > Bharata.
> > >
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-29 15:15 ` Igor Mammedov
@ 2016-03-01 1:21 ` David Gibson
2016-03-01 9:27 ` Igor Mammedov
2016-03-01 8:17 ` Bharata B Rao
1 sibling, 1 reply; 45+ messages in thread
From: David Gibson @ 2016-03-01 1:21 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, Michael Roth, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, Bharata B Rao, pbonzini, afaerber,
ehabkost
[-- Attachment #1: Type: text/plain, Size: 3227 bytes --]
On Mon, Feb 29, 2016 at 04:15:25PM +0100, Igor Mammedov wrote:
> On Mon, 29 Feb 2016 18:25:25 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > > On Mon, 29 Feb 2016 11:20:19 +0530
> > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
[snip]
> > > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > > >
> > > > Correct.
> > > >
> > > > >
> > > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > > right approach, but I don't know how we could expose that as
> > > > > a property. I guess it's somewhat implied that this "interface"
> > > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > > if something a bit more formal should be modeled to make the
> > > > > implementation requirements a bit clearer.
> > > > >
> > > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > > class method, expose them via "slot" property, then have the
> > > > > defaults generate "not implemented" errors?
> > > >
> > > > Yes makes sense. In fact David has often times said that generic
> > > > properties/routines should be pushed to base class wherever possible.
> > > >
> > > > I didn't do that in this first iteration to keep the generic changes
> > > > as minimum as possible, but yes slot should be a property of the
> > > > base class of core or socket.
> > > Then what will happen to slot if there isn't any core/socket device
> > > to query it, i.e. cpu hasn't been plugged in yet?
> > > To me slot looks like a machine belonged feature.
> >
> > Yes slot belongs to the machine and it is represented by a link that
> > is created b/n the machine object and the core object that sits in
> > the slot.
> >
> > In the context of this thread, slot is actually the slot name that
> > identifies the machine slot which the core occupies or will occupy after
> > hotplug. Thus slot name which is named slot here, it is a property of the
> > core device.
> >
> > (qemu) device_add spapr-cpu-core,slot=core[2]
> > ^
> Is 'slot' a term used by SPAPR on real hardware?
So.. PAPR is a para-virtualized interface, so it never appears on real
hardware.
But, no, "slot" is not a term used by PAPR.
> I'd thought that it's 'core', that's why I suggested to use
> 'core' for POWER as that matched real world concept, see
> my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
> of this series.
I don't think it uses "core" either, I believe it uses just "cpu" but
meaning a multi-thread core, rather than a single logical cpu thread.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-02-29 4:42 ` Bharata B Rao
@ 2016-03-01 7:58 ` Bharata B Rao
2016-03-02 0:53 ` David Gibson
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-03-01 7:58 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
On Mon, Feb 29, 2016 at 10:12:10AM +0530, Bharata B Rao wrote:
> > > diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> > > index b7c5ebd..cc0369e 100644
> > > --- a/hw/ppc/spapr_rtas.c
> > > +++ b/hw/ppc/spapr_rtas.c
> > > @@ -34,6 +34,7 @@
> > >
> > > #include "hw/ppc/spapr.h"
> > > #include "hw/ppc/spapr_vio.h"
> > > +#include "hw/ppc/ppc.h"
> > > #include "qapi-event.h"
> > > #include "hw/boards.h"
> > >
> > > @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> > > rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> > > }
> > >
> > > +/*
> > > + * Set the timebase offset of the CPU to that of first CPU.
> > > + * This helps hotplugged CPU to have the correct timebase offset.
> > > + */
> > > +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> > > +{
> > > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > > +
> > > + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> > > +}
> > > +
> > > +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> > > +{
> > > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > > + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> > > +
> > > + if (!pcc->interrupts_big_endian(fcpu)) {
> > > + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> > > + }
> > > +}
> > > +
> >
> > Any particular reason for doing these things at rtas_start_cpu() time,
> > but other initialization at plug time? Could you consolidate it to
> > one place or the other?
>
> Those board specific things that are needed to be done have been consolidated
> into spapr_cpu_init() which will be called from the plug handler. We have
> discussed this earlier at:
>
> https://lists.nongnu.org/archive/html/qemu-devel/2015-02/msg04399.html
>
> It has been a while but there was a good reason why setting endianness
> here rather than in plug handler is necessary. W/o this LE hotplug on guests
> wouldn't work, I will dig up and come back on what exactly necessiated
> this change.
If we set LPCR_ILE in cpu->env.spr[SPR_LPCR] at plug time
(from spapr_cpu_init()), there are at least two places later where it gets
over-written. One is spapr_cpu_reset() and the other one when
kvm_cpu_synchronize_state() is called from rtas_start_cpu(). We could
probably issue a kvm_arch_put_registers(), but I found rtas_start_cpu()
as a place where this change is guaranteed to get reflected.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-02-29 15:15 ` Igor Mammedov
2016-03-01 1:21 ` David Gibson
@ 2016-03-01 8:17 ` Bharata B Rao
2016-03-01 9:16 ` Igor Mammedov
1 sibling, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-03-01 8:17 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Mon, Feb 29, 2016 at 04:15:25PM +0100, Igor Mammedov wrote:
> On Mon, 29 Feb 2016 18:25:25 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > > On Mon, 29 Feb 2016 11:20:19 +0530
> > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > >
> > > > On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > > > > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > > > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > > > > Creating this core device will result in creation of all the CPU thread
> > > > > > devices that are part of this core.
> > > > > >
> > > > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > ---
> > > > > > hw/ppc/Makefile.objs | 1 +
> > > > > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > > > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > > > > 3 files changed, 243 insertions(+)
> > > > > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > > > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > > > > >
> > > > > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > > > > index c1ffc77..5cc6608 100644
> > > > > > --- a/hw/ppc/Makefile.objs
> > > > > > +++ b/hw/ppc/Makefile.objs
> > > > > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > > > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > > > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > > > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > > > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > > > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > > > > obj-y += spapr_pci_vfio.o
> > > > > > endif
> > > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > > > new file mode 100644
> > > > > > index 0000000..c44eb61
> > > > > > --- /dev/null
> > > > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > > > @@ -0,0 +1,210 @@
> > > > > > +/*
> > > > > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > > > > + *
> > > > > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > + *
> > > > > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > > > + * See the COPYING file in the top-level directory.
> > > > > > + */
> > > > > > +#include "hw/cpu/core.h"
> > > > > > +#include "hw/ppc/spapr_cpu_core.h"
> > > > > > +#include "hw/ppc/spapr.h"
> > > > > > +#include "hw/boards.h"
> > > > > > +#include "qemu/error-report.h"
> > > > > > +#include "qapi/visitor.h"
> > > > > > +#include <sysemu/cpus.h>
> > > > > > +
> > > > > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > > > > +{
> > > > > > + Error **errp = opaque;
> > > > > > +
> > > > > > + object_property_set_bool(child, true, "realized", errp);
> > > > > > + if (*errp) {
> > > > > > + return 1;
> > > > > > + }
> > > > > > + return 0;
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > > > + Error *local_err = NULL;
> > > > > > +
> > > > > > + if (!core->nr_threads) {
> > > > > > + error_setg(errp, "nr_threads property can't be 0");
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + if (!core->cpu_model) {
> > > > > > + error_setg(errp, "cpu_model property isn't set");
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + /*
> > > > > > + * TODO: If slot isn't specified, plug this core into
> > > > > > + * an existing empty slot.
> > > > > > + */
> > > > > > + if (!core->slot) {
> > > > > > + error_setg(errp, "slot property isn't set");
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > > > > + &local_err);
> > > > > > + if (local_err) {
> > > > > > + error_propagate(errp, local_err);
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > > > > +}
> > > > > > +
> > > > > > +/*
> > > > > > + * This creates the CPU threads for a given @core.
> > > > > > + *
> > > > > > + * In order to create the threads, we need two inputs - number of
> > > > > > + * threads and the cpu_model. These are set as core object's properties.
> > > > > > + * When both of them become available/set, this routine will be called from
> > > > > > + * either property's set handler to create the threads.
> > > > > > + *
> > > > > > + * TODO: Dependence of threads creation on two properties is resulting
> > > > > > + * in this not-so-clean way of creating threads from either of the
> > > > > > + * property setters based on the order in which they get set. Check if
> > > > > > + * this can be handled in a better manner.
> > > > > > + */
> > > > > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > > > > +{
> > > > > > + int i;
> > > > > > +
> > > > > > + for (i = 0; i < core->nr_threads; i++) {
> > > > > > + char id[32];
> > > > > > + char type[32];
> > > > > > +
> > > > > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > > > > + TYPE_POWERPC_CPU);
> > > > > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > > > > +
> > > > > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > > > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > > > > + errp);
> > > > > > + }
> > > > > > +}
> > > > > > +
> > > > > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > +
> > > > > > + return core->slot;
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > > > > + Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > +
> > > > > > + core->slot = g_strdup(val);
> > > > > > +}
> > > > > > +
> > > > > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > +
> > > > > > + return core->cpu_model;
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > > > > + Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > > > > +
> > > > > > + /*
> > > > > > + * cpu_model can't be different from what is specified with -cpu
> > > > > > + */
> > > > > > + if (strcmp(val, machine->cpu_model)) {
> > > > > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + core->cpu_model = g_strdup(val);
> > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > + }
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > > > > + const char *name, void *opaque,
> > > > > > + Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > + int64_t value = core->nr_threads;
> > > > > > +
> > > > > > + visit_type_int(v, name, &value, errp);
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > > > > + const char *name, void *opaque,
> > > > > > + Error **errp)
> > > > > > +{
> > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > + Error *local_err = NULL;
> > > > > > + int64_t value;
> > > > > > +
> > > > > > + visit_type_int(v, name, &value, &local_err);
> > > > > > + if (local_err) {
> > > > > > + error_propagate(errp, local_err);
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + /* Allow only homogeneous configuration */
> > > > > > + if (value != smp_threads) {
> > > > > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + core->nr_threads = value;
> > > > > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > > > > +
> > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > + }
> > > > > > +}
> > > > > > +
> > > > > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > > > > +{
> > > > > > + object_property_add(obj, "nr_threads", "int",
> > > > > > + spapr_cpu_core_prop_get_nr_threads,
> > > > > > + spapr_cpu_core_prop_set_nr_threads,
> > > > > > + NULL, NULL, NULL);
> > > > > > + object_property_add_str(obj, "cpu_model",
> > > > > > + spapr_cpu_core_prop_get_cpu_model,
> > > > > > + spapr_cpu_core_prop_set_cpu_model,
> > > > > > + NULL);
> > > > > > + object_property_add_str(obj, "slot",
> > > > > > + spapr_cpu_core_prop_get_slot,
> > > > > > + spapr_cpu_core_prop_set_slot,
> > > > > > + NULL);
> > > > >
> > > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > > >
> > > > Correct.
> > > >
> > > > >
> > > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > > right approach, but I don't know how we could expose that as
> > > > > a property. I guess it's somewhat implied that this "interface"
> > > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > > if something a bit more formal should be modeled to make the
> > > > > implementation requirements a bit clearer.
> > > > >
> > > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > > class method, expose them via "slot" property, then have the
> > > > > defaults generate "not implemented" errors?
> > > >
> > > > Yes makes sense. In fact David has often times said that generic
> > > > properties/routines should be pushed to base class wherever possible.
> > > >
> > > > I didn't do that in this first iteration to keep the generic changes
> > > > as minimum as possible, but yes slot should be a property of the
> > > > base class of core or socket.
> > > Then what will happen to slot if there isn't any core/socket device
> > > to query it, i.e. cpu hasn't been plugged in yet?
> > > To me slot looks like a machine belonged feature.
> >
> > Yes slot belongs to the machine and it is represented by a link that
> > is created b/n the machine object and the core object that sits in
> > the slot.
> >
> > In the context of this thread, slot is actually the slot name that
> > identifies the machine slot which the core occupies or will occupy after
> > hotplug. Thus slot name which is named slot here, it is a property of the
> > core device.
> >
> > (qemu) device_add spapr-cpu-core,slot=core[2]
> > ^
> Is 'slot' a term used by SPAPR on real hardware?
> I'd thought that it's 'core', that's why I suggested to use
> 'core' for POWER as that matched real world concept, see
> my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
> of this series.
I don't see any reply from you to that thread. I guess you mean
"[RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices"
thread where you suggest to use CORE type rather than TYPE_CPU and
enumerate threads under CORE type.
In case, it didn't come out clearly, I am indeed using core object and
hotplugging at core granularity for sPAPR. The core sits in the machine slot.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-02-29 10:46 ` Igor Mammedov
@ 2016-03-01 9:09 ` Bharata B Rao
2016-03-01 13:55 ` Igor Mammedov
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-03-01 9:09 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Mon, Feb 29, 2016 at 11:46:42AM +0100, Igor Mammedov wrote:
> On Thu, 25 Feb 2016 21:52:41 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > Implement query cpu-slots that provides information about hot-plugged
> > as well as hot-pluggable CPU slots that the machine supports.
> >
> > TODO: As Eric suggested use enum for type instead of str.
> > TODO: @hotplug-granularity probably isn't required.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/core/machine.c | 19 +++++++++
> > hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > include/hw/boards.h | 4 ++
> > qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> > qmp-commands.hx | 47 ++++++++++++++++++++++
> > 5 files changed, 267 insertions(+)
> >
> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index 6d1a0d8..3055ef8 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -17,6 +17,25 @@
> > #include "hw/sysbus.h"
> > #include "sysemu/sysemu.h"
> > #include "qemu/error-report.h"
> > +#include "qmp-commands.h"
> > +
> > +/*
> > + * QMP: query-cpu-slots
> > + *
> > + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> > + */
> > +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> > +{
> > + MachineState *ms = MACHINE(qdev_get_machine());
> > + MachineClass *mc = MACHINE_GET_CLASS(ms);
> > +
> > + if (!mc->cpu_slots) {
> > + error_setg(errp, QERR_UNSUPPORTED);
> > + return NULL;
> > + }
> > +
> > + return mc->cpu_slots(ms);
> > +}
> >
> > static char *machine_get_accel(Object *obj, Error **errp)
> > {
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 780cd00..b76ed85 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> > return cpu_index / smp_threads / smp_cores;
> > }
> >
> > +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> > +{
> > + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> > + CPUInfoList ***prev = opaque;
> > +
> > + if (object_dynamic_cast(obj, TYPE_CPU)) {
> > + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> > + CPUInfo *s = g_new0(CPUInfo, 1);
> > + CPUState *cpu = CPU(obj);
> > + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> > +
> > + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> > + s->type = g_strdup(object_get_typename(obj));
> > + s->thread = cpu->cpu_index;
> > + s->has_thread = true;
> > + s->core = cpu->cpu_index / smp_threads;
> > + s->has_core = true;
> > + if (mc->cpu_index_to_socket_id) {
> > + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> > + } else {
> > + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> > + }
> > + s->has_socket = true;
> > + s->node = cpu->numa_node;
> > + s->has_node = true;
> > + s->qom_path = object_get_canonical_path(obj);
> > + s->has_qom_path = true;
> > +
> > + elem->value = s;
> > + elem->next = NULL;
> > + **prev = elem;
> > + *prev = &elem->next;
> > + }
> > + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> > + return 0;
> > +}
> > +
> > +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> > +{
> > + CPUSlotInfoList *head = NULL;
> > + CPUSlotInfoList **prev = &head;
> > + Object *root_container;
> > + ObjectProperty *prop;
> > + ObjectPropertyIterator iter;
> > +
> > + /*
> > + * TODO: There surely must be a better/easier way to walk all
> > + * the link properties of an object ?
> > + */
> > + root_container = container_get(object_get_root(), "/machine");
> > + object_property_iter_init(&iter, root_container);
> > +
> > + while ((prop = object_property_iter_next(&iter))) {
> > + Object *obj;
> > + DeviceState *dev;
> > + CPUSlotInfoList *elem;
> > + CPUSlotInfo *s;
> > + CPUInfoList *cpu_head = NULL;
> > + CPUInfoList **cpu_prev = &cpu_head;
> > + sPAPRCPUCore *core;
> > +
> > + if (!strstart(prop->type, "link<", NULL)) {
> > + continue;
> > + }
> > +
> > + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> > + continue;
> > + }
> > +
> > + elem = g_new0(CPUSlotInfoList, 1);
> > + s = g_new0(CPUSlotInfo, 1);
> > +
> > + obj = object_property_get_link(root_container, prop->name, NULL);
> > + if (obj) {
> > + /* Slot populated */
> > + dev = DEVICE(obj);
> > + core = SPAPR_CPU_CORE(obj);
> > +
> > + if (dev->id) {
> > + s->has_id = true;
> > + s->id = g_strdup(dev->id);
> > + }
> > + s->realized = object_property_get_bool(obj, "realized", NULL);
> > + s->nr_cpus = core->nr_threads;
> > + s->has_nr_cpus = true;
> > + s->qom_path = object_get_canonical_path(obj);
> > + s->has_qom_path = true;
> > + if (s->realized) {
> > + spapr_cpuinfo_list(obj, &cpu_prev);
> > + }
> > + s->has_cpus = true;
> > + } else {
> > + /* Slot empty */
> > + s->has_id = false;
> > + s->has_nr_cpus = false;
> > + s->has_qom_path = false;
> > + s->has_cpus = false;
> > + s->realized = false;
> > + }
> > + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> > + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> > + s->slot_id = g_strdup(prop->name);
> > + s->cpus = cpu_head;
> > + elem->value = s;
> > + elem->next = NULL;
> > + *prev = elem;
> > + prev = &elem->next;
> > + }
> > + return head;
> > +}
> > +
> > static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > {
> > MachineClass *mc = MACHINE_CLASS(oc);
> > @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > hc->plug = spapr_machine_device_plug;
> > hc->unplug = spapr_machine_device_unplug;
> > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> > + mc->cpu_slots = spapr_cpu_slots;
> >
> > smc->dr_lmb_enabled = true;
> > smc->dr_cpu_enabled = true;
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 0f30959..d888a02 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> > * Set only by old machines because they need to keep
> > * compatibility on code that exposed QEMU_VERSION to guests in
> > * the past (and now use qemu_hw_version()).
> > + * @cpu_slots:
> > + * Provides information about populated and yet-to-be populated
> > + * CPU slots in the machine. Used by QMP query-cpu-slots.
> > */
> > struct MachineClass {
> > /*< private >*/
> > @@ -99,6 +102,7 @@ struct MachineClass {
> > HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> > DeviceState *dev);
> > unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> > + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> > };
> >
> > /**
> > diff --git a/qapi-schema.json b/qapi-schema.json
> > index 8d04897..e9a52a2 100644
> > --- a/qapi-schema.json
> > +++ b/qapi-schema.json
> > @@ -4083,3 +4083,88 @@
> > ##
> > { 'enum': 'ReplayMode',
> > 'data': [ 'none', 'record', 'play' ] }
> > +
> > +##
> > +# @CPUInfo:
> > +#
> > +# Information about CPUs
> > +#
> > +# @arch-id: Arch-specific ID for the CPU.
> > +#
> > +# @type: QOM type of the CPU.
> > +#
> > +# @thread: Thread ID of the CPU.
> > +#
> > +# @core: Core ID of the CPU.
> > +#
> > +# @socket: Socket ID of the CPU.
> > +#
> > +# @node: NUMA node to which the CPU belongs.
> > +#
> > +# @qom-path: QOM path of the CPU object
> > +#
> > +# Since: 2.6
> > +##
> > +
> > +{ 'struct': 'CPUInfo',
> > + 'data': { 'arch-id': 'int',
> > + 'type': 'str',
> > + '*thread': 'int',
> > + '*core': 'int',
> > + '*socket' : 'int',
> > + '*node' : 'int',
> > + '*qom-path': 'str'
> > + }
> > +}
> > +
> > +##
> > +# @CPUSlotInfo:
> > +#
> > +# Information about CPU Slots
> > +#
> > +# @id: Device ID of the CPU composite object that occupies the slot.
> > +#
> > +# @type: QOM type of the CPU composite object that occupies the slot.
> > +#
> > +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> > +# can be thread, core or socket.
> > +#
> > +# @slot-id: Slot's identifier.
> > +#
> > +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> > +#
> > +# @realized: A boolean that indicates whether the slot is filled or empty.
> > +#
> > +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> > +# this slot.
> > +#
> > +# @cpus: An array of @CPUInfo elements where each element describes one
> > +# CPU that is part of this slot's CPU composite object.
> > +#
> > +# @type: QOM type
> > +#
> > +# Since: 2.6
> > +##
> > +
> > +{ 'struct': 'CPUSlotInfo',
> > + 'data': { '*id': 'str',
> > + 'type': 'str',
> > + 'hotplug-granularity' : 'str',
> Does it convey any useful info, if yes how it will be used by mgmt?
As I noted in the patch desc, this field is useless, will remove.
>
> > + 'slot-id' : 'str',
> > + '*qom-path': 'str',
> > + 'realized': 'bool',
> field's redundant, presence of qom-path answers this question
Ok, makes sense.
>
> > + '*nr-cpus': 'int',
> > + '*cpus' : ['CPUInfo']
> I'd suggest to drop above 2 fields as it's impl dependent,
> qom-path already point's to socket/core/thread object and
> its internal composition can be explored by other means if needed.
>
> Moreover 'CPUInfo' is confusing and sort of conflicts with existing
> 'CpuInfo'.
Ah I see, should change the naming if we eventually stick with this
implementation.
> I'd drop CPUInfo altogether and introduce only 'CPUSlotInfo' here,
> existing thread enumeration's already implemented query-cpus.
Ok.
>
> > + }
> > +}
> What I miss here is that CPUSlotInfo doesn't provide any
> information to about where CPU might be hotplugged to.
>
> Maybe use following tuple instead of slot-id?
>
> { 'struct': 'CPUSlotProperties',
> 'data': { '*node': 'int',
> '*socket': 'int',
> '*core': 'int',
> '*thread': 'int'
> }
> }
Hmm not sure. If I undestand Andreas' proposal correctly, slot is the
place where the CPU sits. Machine determines the type of the slot and it
can be socket slot, core slot or thread slot based on the granularity
of the hotplug supported by the machine. With this I don't see why
anything else apart from slot-id/slot-name is required to figure out where
to hoplug CPU.
In the current implementation, sPAPR hotplug is at core granularity.
CPUSlotInfo.type advertises the type as spapr-cpu-core. The number of
threads in the core and the CPU model of the threads are either machine
default or specified by user during hotplug time.
I believe NUMA node should be the property of the slot. The information
about which slots are part of which NUMA node should be known beforehand
at machine init time. Hotplugging CPU thread, core or socket to a slot will
make that thread, core or socket part of that NUMA node.
> it's generic for the most targets, which should work for spapr, s390, x86, ARM
> and could be extended for other cases adding other board specific
> properties if it's needed.
I don't about ARM, but s390 doesn't care about topology iiuc.
For sPAPR, as I described above, we don't care about sockets and hence
I believe we can live with hotplugging a core into a machine slot by
referring to the slot-id.
I see that CPUSlotProperties that you are suggesting here would make sense
for x86 depending on the granularity at which hotplug is done. I know there
has been discussion on thread vs socket hotplug granularity for x86, has
there been a consensus on what it is going to be ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-03-01 8:17 ` Bharata B Rao
@ 2016-03-01 9:16 ` Igor Mammedov
2016-03-01 9:45 ` Bharata B Rao
0 siblings, 1 reply; 45+ messages in thread
From: Igor Mammedov @ 2016-03-01 9:16 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Tue, 1 Mar 2016 13:47:27 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Mon, Feb 29, 2016 at 04:15:25PM +0100, Igor Mammedov wrote:
> > On Mon, 29 Feb 2016 18:25:25 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > > > On Mon, 29 Feb 2016 11:20:19 +0530
> > > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > > >
> > > > > On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > > > > > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > > > > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > > > > > Creating this core device will result in creation of all the CPU thread
> > > > > > > devices that are part of this core.
> > > > > > >
> > > > > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > > ---
> > > > > > > hw/ppc/Makefile.objs | 1 +
> > > > > > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > > > > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > > > > > 3 files changed, 243 insertions(+)
> > > > > > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > > > > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > > > > > >
> > > > > > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > > > > > index c1ffc77..5cc6608 100644
> > > > > > > --- a/hw/ppc/Makefile.objs
> > > > > > > +++ b/hw/ppc/Makefile.objs
> > > > > > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > > > > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > > > > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > > > > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > > > > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > > > > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > > > > > obj-y += spapr_pci_vfio.o
> > > > > > > endif
> > > > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > > > > new file mode 100644
> > > > > > > index 0000000..c44eb61
> > > > > > > --- /dev/null
> > > > > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > > > > @@ -0,0 +1,210 @@
> > > > > > > +/*
> > > > > > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > > > > > + *
> > > > > > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > > + *
> > > > > > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > > > > + * See the COPYING file in the top-level directory.
> > > > > > > + */
> > > > > > > +#include "hw/cpu/core.h"
> > > > > > > +#include "hw/ppc/spapr_cpu_core.h"
> > > > > > > +#include "hw/ppc/spapr.h"
> > > > > > > +#include "hw/boards.h"
> > > > > > > +#include "qemu/error-report.h"
> > > > > > > +#include "qapi/visitor.h"
> > > > > > > +#include <sysemu/cpus.h>
> > > > > > > +
> > > > > > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > > > > > +{
> > > > > > > + Error **errp = opaque;
> > > > > > > +
> > > > > > > + object_property_set_bool(child, true, "realized", errp);
> > > > > > > + if (*errp) {
> > > > > > > + return 1;
> > > > > > > + }
> > > > > > > + return 0;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > > > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > > > > + Error *local_err = NULL;
> > > > > > > +
> > > > > > > + if (!core->nr_threads) {
> > > > > > > + error_setg(errp, "nr_threads property can't be 0");
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + if (!core->cpu_model) {
> > > > > > > + error_setg(errp, "cpu_model property isn't set");
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + /*
> > > > > > > + * TODO: If slot isn't specified, plug this core into
> > > > > > > + * an existing empty slot.
> > > > > > > + */
> > > > > > > + if (!core->slot) {
> > > > > > > + error_setg(errp, "slot property isn't set");
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > > > > > + &local_err);
> > > > > > > + if (local_err) {
> > > > > > > + error_propagate(errp, local_err);
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > > > > > +}
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * This creates the CPU threads for a given @core.
> > > > > > > + *
> > > > > > > + * In order to create the threads, we need two inputs - number of
> > > > > > > + * threads and the cpu_model. These are set as core object's properties.
> > > > > > > + * When both of them become available/set, this routine will be called from
> > > > > > > + * either property's set handler to create the threads.
> > > > > > > + *
> > > > > > > + * TODO: Dependence of threads creation on two properties is resulting
> > > > > > > + * in this not-so-clean way of creating threads from either of the
> > > > > > > + * property setters based on the order in which they get set. Check if
> > > > > > > + * this can be handled in a better manner.
> > > > > > > + */
> > > > > > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > > > > > +{
> > > > > > > + int i;
> > > > > > > +
> > > > > > > + for (i = 0; i < core->nr_threads; i++) {
> > > > > > > + char id[32];
> > > > > > > + char type[32];
> > > > > > > +
> > > > > > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > > > > > + TYPE_POWERPC_CPU);
> > > > > > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > > > > > +
> > > > > > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > > > > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > > > > > + errp);
> > > > > > > + }
> > > > > > > +}
> > > > > > > +
> > > > > > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > +
> > > > > > > + return core->slot;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > > > > > + Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > +
> > > > > > > + core->slot = g_strdup(val);
> > > > > > > +}
> > > > > > > +
> > > > > > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > +
> > > > > > > + return core->cpu_model;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > > > > > + Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > > > > > +
> > > > > > > + /*
> > > > > > > + * cpu_model can't be different from what is specified with -cpu
> > > > > > > + */
> > > > > > > + if (strcmp(val, machine->cpu_model)) {
> > > > > > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + core->cpu_model = g_strdup(val);
> > > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > > + }
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > > > > > + const char *name, void *opaque,
> > > > > > > + Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > + int64_t value = core->nr_threads;
> > > > > > > +
> > > > > > > + visit_type_int(v, name, &value, errp);
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > > > > > + const char *name, void *opaque,
> > > > > > > + Error **errp)
> > > > > > > +{
> > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > + Error *local_err = NULL;
> > > > > > > + int64_t value;
> > > > > > > +
> > > > > > > + visit_type_int(v, name, &value, &local_err);
> > > > > > > + if (local_err) {
> > > > > > > + error_propagate(errp, local_err);
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + /* Allow only homogeneous configuration */
> > > > > > > + if (value != smp_threads) {
> > > > > > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > > > > > + return;
> > > > > > > + }
> > > > > > > +
> > > > > > > + core->nr_threads = value;
> > > > > > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > > > > > +
> > > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > > + }
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > > > > > +{
> > > > > > > + object_property_add(obj, "nr_threads", "int",
> > > > > > > + spapr_cpu_core_prop_get_nr_threads,
> > > > > > > + spapr_cpu_core_prop_set_nr_threads,
> > > > > > > + NULL, NULL, NULL);
> > > > > > > + object_property_add_str(obj, "cpu_model",
> > > > > > > + spapr_cpu_core_prop_get_cpu_model,
> > > > > > > + spapr_cpu_core_prop_set_cpu_model,
> > > > > > > + NULL);
> > > > > > > + object_property_add_str(obj, "slot",
> > > > > > > + spapr_cpu_core_prop_get_slot,
> > > > > > > + spapr_cpu_core_prop_set_slot,
> > > > > > > + NULL);
> > > > > >
> > > > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > > > >
> > > > > Correct.
> > > > >
> > > > > >
> > > > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > > > right approach, but I don't know how we could expose that as
> > > > > > a property. I guess it's somewhat implied that this "interface"
> > > > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > > > if something a bit more formal should be modeled to make the
> > > > > > implementation requirements a bit clearer.
> > > > > >
> > > > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > > > class method, expose them via "slot" property, then have the
> > > > > > defaults generate "not implemented" errors?
> > > > >
> > > > > Yes makes sense. In fact David has often times said that generic
> > > > > properties/routines should be pushed to base class wherever possible.
> > > > >
> > > > > I didn't do that in this first iteration to keep the generic changes
> > > > > as minimum as possible, but yes slot should be a property of the
> > > > > base class of core or socket.
> > > > Then what will happen to slot if there isn't any core/socket device
> > > > to query it, i.e. cpu hasn't been plugged in yet?
> > > > To me slot looks like a machine belonged feature.
> > >
> > > Yes slot belongs to the machine and it is represented by a link that
> > > is created b/n the machine object and the core object that sits in
> > > the slot.
> > >
> > > In the context of this thread, slot is actually the slot name that
> > > identifies the machine slot which the core occupies or will occupy after
> > > hotplug. Thus slot name which is named slot here, it is a property of the
> > > core device.
> > >
> > > (qemu) device_add spapr-cpu-core,slot=core[2]
> > > ^
> > Is 'slot' a term used by SPAPR on real hardware?
> > I'd thought that it's 'core', that's why I suggested to use
> > 'core' for POWER as that matched real world concept, see
> > my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
> > of this series.
>
> I don't see any reply from you to that thread. I guess you mean
> "[RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices"
no it's another thread, sorry for mixing things up.
it was thread about QMP interface:
http://lists.gnu.org/archive/html/qemu-devel/2016-02/msg06540.html
> thread where you suggest to use CORE type rather than TYPE_CPU and
> enumerate threads under CORE type.
>
> In case, it didn't come out clearly, I am indeed using core object and
> hotplugging at core granularity for sPAPR. The core sits in the machine slot.
There I was suggesting to handle only CORE at spapr_machine_device_plug,
making core.realize() to complete underlying threads initialization if possible,
and if it you have to do some wiring at machine level for threads then
do it while handling CORE branch.
That way hotplug call-chain would look like:
device_add(core)
-> device_realize(core)
-> core_realize()-> {realize threads}
-> spapr_machine_device_plug(core)
connect to board internals for core (maybe threads)
vs current way:
device_add(core)
-> device_realize(core)
-> core_realize()-> {realize threads}
-> device_realize(thread)
-> spapr_machine_device_plug(thread)
some board specific wiring
...
-> spapr_machine_device_plug(core)
connect to board internals
the former is easier to read/follow
>
> Regards,
> Bharata.
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-03-01 1:21 ` David Gibson
@ 2016-03-01 9:27 ` Igor Mammedov
0 siblings, 0 replies; 45+ messages in thread
From: Igor Mammedov @ 2016-03-01 9:27 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, qemu-devel, agraf,
Michael Roth, borntraeger, qemu-ppc, Bharata B Rao, pbonzini,
afaerber, armbru
On Tue, 1 Mar 2016 12:21:27 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:
> On Mon, Feb 29, 2016 at 04:15:25PM +0100, Igor Mammedov wrote:
> > On Mon, 29 Feb 2016 18:25:25 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > > On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > > > On Mon, 29 Feb 2016 11:20:19 +0530
> > > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> [snip]
> > > > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > > > >
> > > > > Correct.
> > > > >
> > > > > >
> > > > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > > > right approach, but I don't know how we could expose that as
> > > > > > a property. I guess it's somewhat implied that this "interface"
> > > > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > > > if something a bit more formal should be modeled to make the
> > > > > > implementation requirements a bit clearer.
> > > > > >
> > > > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > > > class method, expose them via "slot" property, then have the
> > > > > > defaults generate "not implemented" errors?
> > > > >
> > > > > Yes makes sense. In fact David has often times said that generic
> > > > > properties/routines should be pushed to base class wherever possible.
> > > > >
> > > > > I didn't do that in this first iteration to keep the generic changes
> > > > > as minimum as possible, but yes slot should be a property of the
> > > > > base class of core or socket.
> > > > Then what will happen to slot if there isn't any core/socket device
> > > > to query it, i.e. cpu hasn't been plugged in yet?
> > > > To me slot looks like a machine belonged feature.
> > >
> > > Yes slot belongs to the machine and it is represented by a link that
> > > is created b/n the machine object and the core object that sits in
> > > the slot.
> > >
> > > In the context of this thread, slot is actually the slot name that
> > > identifies the machine slot which the core occupies or will occupy after
> > > hotplug. Thus slot name which is named slot here, it is a property of the
> > > core device.
> > >
> > > (qemu) device_add spapr-cpu-core,slot=core[2]
> > > ^
> > Is 'slot' a term used by SPAPR on real hardware?
>
> So.. PAPR is a para-virtualized interface, so it never appears on real
> hardware.
>
> But, no, "slot" is not a term used by PAPR.
>
> > I'd thought that it's 'core', that's why I suggested to use
> > 'core' for POWER as that matched real world concept, see
> > my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
> > of this series.
>
> I don't think it uses "core" either, I believe it uses just "cpu" but
> meaning a multi-thread core, rather than a single logical cpu thread.
then calling property 'cpu' is fine or one could go by meaning and
use 'core' property (reusing 'core' property)
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device
2016-03-01 9:16 ` Igor Mammedov
@ 2016-03-01 9:45 ` Bharata B Rao
0 siblings, 0 replies; 45+ messages in thread
From: Bharata B Rao @ 2016-03-01 9:45 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, borntraeger,
qemu-devel, armbru, Michael Roth, aik, qemu-ppc, pbonzini,
afaerber, david
On Tue, Mar 01, 2016 at 10:16:18AM +0100, Igor Mammedov wrote:
> On Tue, 1 Mar 2016 13:47:27 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > On Mon, Feb 29, 2016 at 04:15:25PM +0100, Igor Mammedov wrote:
> > > On Mon, 29 Feb 2016 18:25:25 +0530
> > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > >
> > > > On Mon, Feb 29, 2016 at 11:03:16AM +0100, Igor Mammedov wrote:
> > > > > On Mon, 29 Feb 2016 11:20:19 +0530
> > > > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > > > >
> > > > > > On Fri, Feb 26, 2016 at 12:13:39PM -0600, Michael Roth wrote:
> > > > > > > Quoting Bharata B Rao (2016-02-25 10:22:38)
> > > > > > > > Add sPAPR specific CPU core device that is based on generic CPU core device.
> > > > > > > > Creating this core device will result in creation of all the CPU thread
> > > > > > > > devices that are part of this core.
> > > > > > > >
> > > > > > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > > > ---
> > > > > > > > hw/ppc/Makefile.objs | 1 +
> > > > > > > > hw/ppc/spapr_cpu_core.c | 210 ++++++++++++++++++++++++++++++++++++++++
> > > > > > > > include/hw/ppc/spapr_cpu_core.h | 32 ++++++
> > > > > > > > 3 files changed, 243 insertions(+)
> > > > > > > > create mode 100644 hw/ppc/spapr_cpu_core.c
> > > > > > > > create mode 100644 include/hw/ppc/spapr_cpu_core.h
> > > > > > > >
> > > > > > > > diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
> > > > > > > > index c1ffc77..5cc6608 100644
> > > > > > > > --- a/hw/ppc/Makefile.objs
> > > > > > > > +++ b/hw/ppc/Makefile.objs
> > > > > > > > @@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
> > > > > > > > obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
> > > > > > > > obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
> > > > > > > > obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
> > > > > > > > +obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
> > > > > > > > ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
> > > > > > > > obj-y += spapr_pci_vfio.o
> > > > > > > > endif
> > > > > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > > > > > new file mode 100644
> > > > > > > > index 0000000..c44eb61
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > > > > > @@ -0,0 +1,210 @@
> > > > > > > > +/*
> > > > > > > > + * sPAPR CPU core device, acts as container of CPU thread devices.
> > > > > > > > + *
> > > > > > > > + * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > > > > + *
> > > > > > > > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > > > > > > > + * See the COPYING file in the top-level directory.
> > > > > > > > + */
> > > > > > > > +#include "hw/cpu/core.h"
> > > > > > > > +#include "hw/ppc/spapr_cpu_core.h"
> > > > > > > > +#include "hw/ppc/spapr.h"
> > > > > > > > +#include "hw/boards.h"
> > > > > > > > +#include "qemu/error-report.h"
> > > > > > > > +#include "qapi/visitor.h"
> > > > > > > > +#include <sysemu/cpus.h>
> > > > > > > > +
> > > > > > > > +static int spapr_cpu_core_realize_child(Object *child, void *opaque)
> > > > > > > > +{
> > > > > > > > + Error **errp = opaque;
> > > > > > > > +
> > > > > > > > + object_property_set_bool(child, true, "realized", errp);
> > > > > > > > + if (*errp) {
> > > > > > > > + return 1;
> > > > > > > > + }
> > > > > > > > + return 0;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > > > > > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > > > > > > > + Error *local_err = NULL;
> > > > > > > > +
> > > > > > > > + if (!core->nr_threads) {
> > > > > > > > + error_setg(errp, "nr_threads property can't be 0");
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + if (!core->cpu_model) {
> > > > > > > > + error_setg(errp, "cpu_model property isn't set");
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + /*
> > > > > > > > + * TODO: If slot isn't specified, plug this core into
> > > > > > > > + * an existing empty slot.
> > > > > > > > + */
> > > > > > > > + if (!core->slot) {
> > > > > > > > + error_setg(errp, "slot property isn't set");
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + object_property_set_link(OBJECT(spapr), OBJECT(core), core->slot,
> > > > > > > > + &local_err);
> > > > > > > > + if (local_err) {
> > > > > > > > + error_propagate(errp, local_err);
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + object_child_foreach(OBJECT(dev), spapr_cpu_core_realize_child, errp);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +/*
> > > > > > > > + * This creates the CPU threads for a given @core.
> > > > > > > > + *
> > > > > > > > + * In order to create the threads, we need two inputs - number of
> > > > > > > > + * threads and the cpu_model. These are set as core object's properties.
> > > > > > > > + * When both of them become available/set, this routine will be called from
> > > > > > > > + * either property's set handler to create the threads.
> > > > > > > > + *
> > > > > > > > + * TODO: Dependence of threads creation on two properties is resulting
> > > > > > > > + * in this not-so-clean way of creating threads from either of the
> > > > > > > > + * property setters based on the order in which they get set. Check if
> > > > > > > > + * this can be handled in a better manner.
> > > > > > > > + */
> > > > > > > > +static void spapr_cpu_core_create_threads(sPAPRCPUCore *core, Error **errp)
> > > > > > > > +{
> > > > > > > > + int i;
> > > > > > > > +
> > > > > > > > + for (i = 0; i < core->nr_threads; i++) {
> > > > > > > > + char id[32];
> > > > > > > > + char type[32];
> > > > > > > > +
> > > > > > > > + snprintf(type, sizeof(type), "%s-%s", core->cpu_model,
> > > > > > > > + TYPE_POWERPC_CPU);
> > > > > > > > + object_initialize(&core->threads[i], sizeof(core->threads[i]), type);
> > > > > > > > +
> > > > > > > > + snprintf(id, sizeof(id), "thread[%d]", i);
> > > > > > > > + object_property_add_child(OBJECT(core), id, OBJECT(&core->threads[i]),
> > > > > > > > + errp);
> > > > > > > > + }
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static char *spapr_cpu_core_prop_get_slot(Object *obj, Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > +
> > > > > > > > + return core->slot;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_prop_set_slot(Object *obj, const char *val,
> > > > > > > > + Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > +
> > > > > > > > + core->slot = g_strdup(val);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static char *spapr_cpu_core_prop_get_cpu_model(Object *obj, Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > +
> > > > > > > > + return core->cpu_model;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_prop_set_cpu_model(Object *obj, const char *val,
> > > > > > > > + Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > + MachineState *machine = MACHINE(qdev_get_machine());
> > > > > > > > +
> > > > > > > > + /*
> > > > > > > > + * cpu_model can't be different from what is specified with -cpu
> > > > > > > > + */
> > > > > > > > + if (strcmp(val, machine->cpu_model)) {
> > > > > > > > + error_setg(errp, "cpu_model should be %s", machine->cpu_model);
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + core->cpu_model = g_strdup(val);
> > > > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > > > + }
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_prop_get_nr_threads(Object *obj, Visitor *v,
> > > > > > > > + const char *name, void *opaque,
> > > > > > > > + Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > + int64_t value = core->nr_threads;
> > > > > > > > +
> > > > > > > > + visit_type_int(v, name, &value, errp);
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_prop_set_nr_threads(Object *obj, Visitor *v,
> > > > > > > > + const char *name, void *opaque,
> > > > > > > > + Error **errp)
> > > > > > > > +{
> > > > > > > > + sPAPRCPUCore *core = SPAPR_CPU_CORE(obj);
> > > > > > > > + Error *local_err = NULL;
> > > > > > > > + int64_t value;
> > > > > > > > +
> > > > > > > > + visit_type_int(v, name, &value, &local_err);
> > > > > > > > + if (local_err) {
> > > > > > > > + error_propagate(errp, local_err);
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + /* Allow only homogeneous configuration */
> > > > > > > > + if (value != smp_threads) {
> > > > > > > > + error_setg(errp, "nr_threads should be %d", smp_threads);
> > > > > > > > + return;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + core->nr_threads = value;
> > > > > > > > + core->threads = g_malloc0(core->nr_threads * sizeof(PowerPCCPU));
> > > > > > > > +
> > > > > > > > + if (core->nr_threads && core->cpu_model) {
> > > > > > > > + spapr_cpu_core_create_threads(core, errp);
> > > > > > > > + }
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void spapr_cpu_core_instance_init(Object *obj)
> > > > > > > > +{
> > > > > > > > + object_property_add(obj, "nr_threads", "int",
> > > > > > > > + spapr_cpu_core_prop_get_nr_threads,
> > > > > > > > + spapr_cpu_core_prop_set_nr_threads,
> > > > > > > > + NULL, NULL, NULL);
> > > > > > > > + object_property_add_str(obj, "cpu_model",
> > > > > > > > + spapr_cpu_core_prop_get_cpu_model,
> > > > > > > > + spapr_cpu_core_prop_set_cpu_model,
> > > > > > > > + NULL);
> > > > > > > > + object_property_add_str(obj, "slot",
> > > > > > > > + spapr_cpu_core_prop_get_slot,
> > > > > > > > + spapr_cpu_core_prop_set_slot,
> > > > > > > > + NULL);
> > > > > > >
> > > > > > > "slot" seems intended to be a machine-agnostic of mapping device
> > > > > > > types discovered from qmp_query_cpu_slots() to an appropriate
> > > > > > > "bus" location, but here's it a field specific to TYPE_SPAPR_CPU_CORE.
> > > > > > > It seems like maybe TYPE_CPU_CORE is a better place, but then on
> > > > > > > x86 I suppose it might be TYPE_CPU_SOCKET or something instead...
> > > > > >
> > > > > > Correct.
> > > > > >
> > > > > > >
> > > > > > > It almost seems like a TYPE_INTERFACE_SLOTABLE would be the
> > > > > > > right approach, but I don't know how we could expose that as
> > > > > > > a property. I guess it's somewhat implied that this "interface"
> > > > > > > exists if qmp_query_cpu_slots() returns the type, but I wonder
> > > > > > > if something a bit more formal should be modeled to make the
> > > > > > > implementation requirements a bit clearer.
> > > > > > >
> > > > > > > Maybe have TYPE_CPU_{CORE,SOCKET} classes have a get_slot/set_slot
> > > > > > > class method, expose them via "slot" property, then have the
> > > > > > > defaults generate "not implemented" errors?
> > > > > >
> > > > > > Yes makes sense. In fact David has often times said that generic
> > > > > > properties/routines should be pushed to base class wherever possible.
> > > > > >
> > > > > > I didn't do that in this first iteration to keep the generic changes
> > > > > > as minimum as possible, but yes slot should be a property of the
> > > > > > base class of core or socket.
> > > > > Then what will happen to slot if there isn't any core/socket device
> > > > > to query it, i.e. cpu hasn't been plugged in yet?
> > > > > To me slot looks like a machine belonged feature.
> > > >
> > > > Yes slot belongs to the machine and it is represented by a link that
> > > > is created b/n the machine object and the core object that sits in
> > > > the slot.
> > > >
> > > > In the context of this thread, slot is actually the slot name that
> > > > identifies the machine slot which the core occupies or will occupy after
> > > > hotplug. Thus slot name which is named slot here, it is a property of the
> > > > core device.
> > > >
> > > > (qemu) device_add spapr-cpu-core,slot=core[2]
> > > > ^
> > > Is 'slot' a term used by SPAPR on real hardware?
> > > I'd thought that it's 'core', that's why I suggested to use
> > > 'core' for POWER as that matched real world concept, see
> > > my other reply in "[RFC PATCH v0 4/6] spapr: CPU hotplug support" thread
> > > of this series.
> >
> > I don't see any reply from you to that thread. I guess you mean
> > "[RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices"
> no it's another thread, sorry for mixing things up.
> it was thread about QMP interface:
> http://lists.gnu.org/archive/html/qemu-devel/2016-02/msg06540.html
>
> > thread where you suggest to use CORE type rather than TYPE_CPU and
> > enumerate threads under CORE type.
> >
> > In case, it didn't come out clearly, I am indeed using core object and
> > hotplugging at core granularity for sPAPR. The core sits in the machine slot.
> There I was suggesting to handle only CORE at spapr_machine_device_plug,
> making core.realize() to complete underlying threads initialization if possible,
> and if it you have to do some wiring at machine level for threads then
> do it while handling CORE branch.
>
> That way hotplug call-chain would look like:
> device_add(core)
> -> device_realize(core)
> -> core_realize()-> {realize threads}
> -> spapr_machine_device_plug(core)
> connect to board internals for core (maybe threads)
>
> vs current way:
> device_add(core)
> -> device_realize(core)
> -> core_realize()-> {realize threads}
> -> device_realize(thread)
> -> spapr_machine_device_plug(thread)
> some board specific wiring
> ...
> -> spapr_machine_device_plug(core)
> connect to board internals
Sure, I am changing this in next version.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (5 preceding siblings ...)
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 6/6] hmp: Implement 'info cpu-slots' Bharata B Rao
@ 2016-03-01 10:00 ` Bharata B Rao
2016-03-01 13:59 ` Andreas Färber
6 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-03-01 10:00 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, mdroth, afaerber,
david
On Thu, Feb 25, 2016 at 09:52:36PM +0530, Bharata B Rao wrote:
> Hi,
>
> This is an attempt to implement CPU hotplug for PowerPC sPAPR based on
> the approach suggested by Andreas. While I say that, I should also explicitly
> add that I have tried to follow Andreas' suggestions to the best of my
> understanding and hence there could be bits which are still not
> as per expectations.
>
> I have tried to model this similarly to what Andreas did for x86 an year back at
> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04858.html
Andreas - Do you have any comments on this implementation ? Is it worth
pursuing further in your view ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-03-01 9:09 ` Bharata B Rao
@ 2016-03-01 13:55 ` Igor Mammedov
2016-03-03 9:30 ` Bharata B Rao
0 siblings, 1 reply; 45+ messages in thread
From: Igor Mammedov @ 2016-03-01 13:55 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Tue, 1 Mar 2016 14:39:51 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Mon, Feb 29, 2016 at 11:46:42AM +0100, Igor Mammedov wrote:
> > On Thu, 25 Feb 2016 21:52:41 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > Implement query cpu-slots that provides information about hot-plugged
> > > as well as hot-pluggable CPU slots that the machine supports.
> > >
> > > TODO: As Eric suggested use enum for type instead of str.
> > > TODO: @hotplug-granularity probably isn't required.
> > >
> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > ---
> > > hw/core/machine.c | 19 +++++++++
> > > hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > include/hw/boards.h | 4 ++
> > > qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> > > qmp-commands.hx | 47 ++++++++++++++++++++++
> > > 5 files changed, 267 insertions(+)
> > >
> > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > index 6d1a0d8..3055ef8 100644
> > > --- a/hw/core/machine.c
> > > +++ b/hw/core/machine.c
> > > @@ -17,6 +17,25 @@
> > > #include "hw/sysbus.h"
> > > #include "sysemu/sysemu.h"
> > > #include "qemu/error-report.h"
> > > +#include "qmp-commands.h"
> > > +
> > > +/*
> > > + * QMP: query-cpu-slots
> > > + *
> > > + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> > > + */
> > > +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> > > +{
> > > + MachineState *ms = MACHINE(qdev_get_machine());
> > > + MachineClass *mc = MACHINE_GET_CLASS(ms);
> > > +
> > > + if (!mc->cpu_slots) {
> > > + error_setg(errp, QERR_UNSUPPORTED);
> > > + return NULL;
> > > + }
> > > +
> > > + return mc->cpu_slots(ms);
> > > +}
> > >
> > > static char *machine_get_accel(Object *obj, Error **errp)
> > > {
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 780cd00..b76ed85 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> > > return cpu_index / smp_threads / smp_cores;
> > > }
> > >
> > > +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> > > +{
> > > + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> > > + CPUInfoList ***prev = opaque;
> > > +
> > > + if (object_dynamic_cast(obj, TYPE_CPU)) {
> > > + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> > > + CPUInfo *s = g_new0(CPUInfo, 1);
> > > + CPUState *cpu = CPU(obj);
> > > + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> > > +
> > > + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> > > + s->type = g_strdup(object_get_typename(obj));
> > > + s->thread = cpu->cpu_index;
> > > + s->has_thread = true;
> > > + s->core = cpu->cpu_index / smp_threads;
> > > + s->has_core = true;
> > > + if (mc->cpu_index_to_socket_id) {
> > > + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> > > + } else {
> > > + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> > > + }
> > > + s->has_socket = true;
> > > + s->node = cpu->numa_node;
> > > + s->has_node = true;
> > > + s->qom_path = object_get_canonical_path(obj);
> > > + s->has_qom_path = true;
> > > +
> > > + elem->value = s;
> > > + elem->next = NULL;
> > > + **prev = elem;
> > > + *prev = &elem->next;
> > > + }
> > > + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> > > + return 0;
> > > +}
> > > +
> > > +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> > > +{
> > > + CPUSlotInfoList *head = NULL;
> > > + CPUSlotInfoList **prev = &head;
> > > + Object *root_container;
> > > + ObjectProperty *prop;
> > > + ObjectPropertyIterator iter;
> > > +
> > > + /*
> > > + * TODO: There surely must be a better/easier way to walk all
> > > + * the link properties of an object ?
> > > + */
> > > + root_container = container_get(object_get_root(), "/machine");
> > > + object_property_iter_init(&iter, root_container);
> > > +
> > > + while ((prop = object_property_iter_next(&iter))) {
> > > + Object *obj;
> > > + DeviceState *dev;
> > > + CPUSlotInfoList *elem;
> > > + CPUSlotInfo *s;
> > > + CPUInfoList *cpu_head = NULL;
> > > + CPUInfoList **cpu_prev = &cpu_head;
> > > + sPAPRCPUCore *core;
> > > +
> > > + if (!strstart(prop->type, "link<", NULL)) {
> > > + continue;
> > > + }
> > > +
> > > + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> > > + continue;
> > > + }
> > > +
> > > + elem = g_new0(CPUSlotInfoList, 1);
> > > + s = g_new0(CPUSlotInfo, 1);
> > > +
> > > + obj = object_property_get_link(root_container, prop->name, NULL);
> > > + if (obj) {
> > > + /* Slot populated */
> > > + dev = DEVICE(obj);
> > > + core = SPAPR_CPU_CORE(obj);
> > > +
> > > + if (dev->id) {
> > > + s->has_id = true;
> > > + s->id = g_strdup(dev->id);
> > > + }
> > > + s->realized = object_property_get_bool(obj, "realized", NULL);
> > > + s->nr_cpus = core->nr_threads;
> > > + s->has_nr_cpus = true;
> > > + s->qom_path = object_get_canonical_path(obj);
> > > + s->has_qom_path = true;
> > > + if (s->realized) {
> > > + spapr_cpuinfo_list(obj, &cpu_prev);
> > > + }
> > > + s->has_cpus = true;
> > > + } else {
> > > + /* Slot empty */
> > > + s->has_id = false;
> > > + s->has_nr_cpus = false;
> > > + s->has_qom_path = false;
> > > + s->has_cpus = false;
> > > + s->realized = false;
> > > + }
> > > + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> > > + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> > > + s->slot_id = g_strdup(prop->name);
> > > + s->cpus = cpu_head;
> > > + elem->value = s;
> > > + elem->next = NULL;
> > > + *prev = elem;
> > > + prev = &elem->next;
> > > + }
> > > + return head;
> > > +}
> > > +
> > > static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > {
> > > MachineClass *mc = MACHINE_CLASS(oc);
> > > @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > hc->plug = spapr_machine_device_plug;
> > > hc->unplug = spapr_machine_device_unplug;
> > > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> > > + mc->cpu_slots = spapr_cpu_slots;
> > >
> > > smc->dr_lmb_enabled = true;
> > > smc->dr_cpu_enabled = true;
> > > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > > index 0f30959..d888a02 100644
> > > --- a/include/hw/boards.h
> > > +++ b/include/hw/boards.h
> > > @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> > > * Set only by old machines because they need to keep
> > > * compatibility on code that exposed QEMU_VERSION to guests in
> > > * the past (and now use qemu_hw_version()).
> > > + * @cpu_slots:
> > > + * Provides information about populated and yet-to-be populated
> > > + * CPU slots in the machine. Used by QMP query-cpu-slots.
> > > */
> > > struct MachineClass {
> > > /*< private >*/
> > > @@ -99,6 +102,7 @@ struct MachineClass {
> > > HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> > > DeviceState *dev);
> > > unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> > > + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> > > };
> > >
> > > /**
> > > diff --git a/qapi-schema.json b/qapi-schema.json
> > > index 8d04897..e9a52a2 100644
> > > --- a/qapi-schema.json
> > > +++ b/qapi-schema.json
> > > @@ -4083,3 +4083,88 @@
> > > ##
> > > { 'enum': 'ReplayMode',
> > > 'data': [ 'none', 'record', 'play' ] }
> > > +
> > > +##
> > > +# @CPUInfo:
> > > +#
> > > +# Information about CPUs
> > > +#
> > > +# @arch-id: Arch-specific ID for the CPU.
> > > +#
> > > +# @type: QOM type of the CPU.
> > > +#
> > > +# @thread: Thread ID of the CPU.
> > > +#
> > > +# @core: Core ID of the CPU.
> > > +#
> > > +# @socket: Socket ID of the CPU.
> > > +#
> > > +# @node: NUMA node to which the CPU belongs.
> > > +#
> > > +# @qom-path: QOM path of the CPU object
> > > +#
> > > +# Since: 2.6
> > > +##
> > > +
> > > +{ 'struct': 'CPUInfo',
> > > + 'data': { 'arch-id': 'int',
> > > + 'type': 'str',
> > > + '*thread': 'int',
> > > + '*core': 'int',
> > > + '*socket' : 'int',
> > > + '*node' : 'int',
> > > + '*qom-path': 'str'
> > > + }
> > > +}
> > > +
> > > +##
> > > +# @CPUSlotInfo:
> > > +#
> > > +# Information about CPU Slots
> > > +#
> > > +# @id: Device ID of the CPU composite object that occupies the slot.
> > > +#
> > > +# @type: QOM type of the CPU composite object that occupies the slot.
> > > +#
> > > +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> > > +# can be thread, core or socket.
> > > +#
> > > +# @slot-id: Slot's identifier.
> > > +#
> > > +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> > > +#
> > > +# @realized: A boolean that indicates whether the slot is filled or empty.
> > > +#
> > > +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> > > +# this slot.
> > > +#
> > > +# @cpus: An array of @CPUInfo elements where each element describes one
> > > +# CPU that is part of this slot's CPU composite object.
> > > +#
> > > +# @type: QOM type
> > > +#
> > > +# Since: 2.6
> > > +##
> > > +
> > > +{ 'struct': 'CPUSlotInfo',
> > > + 'data': { '*id': 'str',
> > > + 'type': 'str',
> > > + 'hotplug-granularity' : 'str',
> > Does it convey any useful info, if yes how it will be used by mgmt?
>
> As I noted in the patch desc, this field is useless, will remove.
>
> >
> > > + 'slot-id' : 'str',
> > > + '*qom-path': 'str',
> > > + 'realized': 'bool',
> > field's redundant, presence of qom-path answers this question
>
> Ok, makes sense.
>
> >
> > > + '*nr-cpus': 'int',
> > > + '*cpus' : ['CPUInfo']
> > I'd suggest to drop above 2 fields as it's impl dependent,
> > qom-path already point's to socket/core/thread object and
> > its internal composition can be explored by other means if needed.
> >
> > Moreover 'CPUInfo' is confusing and sort of conflicts with existing
> > 'CpuInfo'.
>
> Ah I see, should change the naming if we eventually stick with this
> implementation.
>
> > I'd drop CPUInfo altogether and introduce only 'CPUSlotInfo' here,
> > existing thread enumeration's already implemented query-cpus.
>
> Ok.
>
> >
> > > + }
> > > +}
> > What I miss here is that CPUSlotInfo doesn't provide any
> > information to about where CPU might be hotplugged to.
> >
> > Maybe use following tuple instead of slot-id?
> >
> > { 'struct': 'CPUSlotProperties',
> > 'data': { '*node': 'int',
> > '*socket': 'int',
> > '*core': 'int',
> > '*thread': 'int'
> > }
> > }
>
> Hmm not sure. If I undestand Andreas' proposal correctly, slot is the
> place where the CPU sits. Machine determines the type of the slot and it
> can be socket slot, core slot or thread slot based on the granularity
> of the hotplug supported by the machine. With this I don't see why
> anything else apart from slot-id/slot-name is required to figure out where
> to hoplug CPU.
Of cause it's possible to create and keep at board level map of
slot-names to whatever other info needed to create a CPU object
at machine init time, and then make board somehow to apply it
to the new CPU object before realizing it.
But it doesn't give mgmt any information whatsoever about where CPU
is being hotplugged. So for x86/arm we would end up with adding
yet another interface that would tell it. If CPUSlotProperties is used
instead of slot-name, it could convey that information while
keeping interface compatible with a range targets we care about
and extendable to other targets in future.
> In the current implementation, sPAPR hotplug is at core granularity.
> CPUSlotInfo.type advertises the type as spapr-cpu-core. The number of
> threads in the core and the CPU model of the threads are either machine
> default or specified by user during hotplug time.
now imagine extending in future sPAPR to support NUMA, which way would
you extend interface, how would mgmt know which CPU belongs to what node?
As you may note in CPUSlotProperties fields are optional so target would
use only ones it needs. For current sPAPR, query result could be
something like this:
{
{
type: spapr-cpu-core
properties: { core: 0 }
qom-path: /machine/unattached/device[xxx]
}
{
type: spapr-cpu-core
properties: { core: 1 }
}
...
{
type: spapr-cpu-core
properties: { core: 0 }
}
}
Later if numa was needed, the board could set 'numa' property as well.
Would that work for sPAPR?
> I believe NUMA node should be the property of the slot. The information
> about which slots are part of which NUMA node should be known beforehand
> at machine init time. Hotplugging CPU thread, core or socket to a slot will
> make that thread, core or socket part of that NUMA node.
As analogy PCI devices also have 'bus' property which is used to tell
device_add machinery where to attach the device. The same goes for DIMM
device which also has 'numa' option so mgmt could say to which node device
is being plugged.
CPUSlotProperties is basically location where CPU is being plugged
and in some cases location includes 'numa' property or some other.
The difference vs slot-name is that CPUSlotProperties is more
verbose and flexible and tries to describe location in natural
for target therms.
> > it's generic for the most targets, which should work for spapr, s390, x86, ARM
> > and could be extended for other cases adding other board specific
> > properties if it's needed.
>
> I don't about ARM, but s390 doesn't care about topology iiuc.
>
> For sPAPR, as I described above, we don't care about sockets and hence
> I believe we can live with hotplugging a core into a machine slot by
> referring to the slot-id.
>
> I see that CPUSlotProperties that you are suggesting here would make sense
> for x86 depending on the granularity at which hotplug is done. I know there
> has been discussion on thread vs socket hotplug granularity for x86, has
> there been a consensus on what it is going to be ?
I don't agree with switching to socket model on x86 for a number of reasons:
- it can't be done without breaking compatibility.
- from user pov, he/she would like to hotplug CPU at thread level to get
necessary CPU horsepower without over-paying for over-provisioned threads.
Limiting it to socket level would be crippling qemu based hypervisors
in comparison to other hypervisors, which do it currently at thread level.
Practice show that Linux and Windows handle thread level hotplug well
enough perhaps because they try match ACPI spec where hotplug happens
at thread level.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR
2016-03-01 10:00 ` [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
@ 2016-03-01 13:59 ` Andreas Färber
0 siblings, 0 replies; 45+ messages in thread
From: Andreas Färber @ 2016-03-01 13:59 UTC (permalink / raw)
To: bharata, qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, mdroth, david
Am 01.03.2016 um 11:00 schrieb Bharata B Rao:
> On Thu, Feb 25, 2016 at 09:52:36PM +0530, Bharata B Rao wrote:
>> Hi,
>>
>> This is an attempt to implement CPU hotplug for PowerPC sPAPR based on
>> the approach suggested by Andreas. While I say that, I should also explicitly
>> add that I have tried to follow Andreas' suggestions to the best of my
>> understanding and hence there could be bits which are still not
>> as per expectations.
>>
>> I have tried to model this similarly to what Andreas did for x86 an year back at
>> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04858.html
>
> Andreas - Do you have any comments on this implementation ? Is it worth
> pursuing further in your view ?
As indicated to some of you, I was absent last week - that means I did
not yet read the flood of emails I got from David, Igor and others.
Maybe you can give a summary of open questions and problems on the call?
Regards,
Andreas
--
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support
2016-03-01 7:58 ` Bharata B Rao
@ 2016-03-02 0:53 ` David Gibson
0 siblings, 0 replies; 45+ messages in thread
From: David Gibson @ 2016-03-02 0:53 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 3022 bytes --]
On Tue, Mar 01, 2016 at 01:28:56PM +0530, Bharata B Rao wrote:
> On Mon, Feb 29, 2016 at 10:12:10AM +0530, Bharata B Rao wrote:
> > > > diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> > > > index b7c5ebd..cc0369e 100644
> > > > --- a/hw/ppc/spapr_rtas.c
> > > > +++ b/hw/ppc/spapr_rtas.c
> > > > @@ -34,6 +34,7 @@
> > > >
> > > > #include "hw/ppc/spapr.h"
> > > > #include "hw/ppc/spapr_vio.h"
> > > > +#include "hw/ppc/ppc.h"
> > > > #include "qapi-event.h"
> > > > #include "hw/boards.h"
> > > >
> > > > @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> > > > rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> > > > }
> > > >
> > > > +/*
> > > > + * Set the timebase offset of the CPU to that of first CPU.
> > > > + * This helps hotplugged CPU to have the correct timebase offset.
> > > > + */
> > > > +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> > > > +{
> > > > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > > > +
> > > > + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> > > > +}
> > > > +
> > > > +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> > > > +{
> > > > + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> > > > + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> > > > +
> > > > + if (!pcc->interrupts_big_endian(fcpu)) {
> > > > + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> > > > + }
> > > > +}
> > > > +
> > >
> > > Any particular reason for doing these things at rtas_start_cpu() time,
> > > but other initialization at plug time? Could you consolidate it to
> > > one place or the other?
> >
> > Those board specific things that are needed to be done have been consolidated
> > into spapr_cpu_init() which will be called from the plug handler. We have
> > discussed this earlier at:
> >
> > https://lists.nongnu.org/archive/html/qemu-devel/2015-02/msg04399.html
> >
> > It has been a while but there was a good reason why setting endianness
> > here rather than in plug handler is necessary. W/o this LE hotplug on guests
> > wouldn't work, I will dig up and come back on what exactly necessiated
> > this change.
>
> If we set LPCR_ILE in cpu->env.spr[SPR_LPCR] at plug time
> (from spapr_cpu_init()), there are at least two places later where it gets
> over-written. One is spapr_cpu_reset() and the other one when
> kvm_cpu_synchronize_state() is called from rtas_start_cpu(). We could
> probably issue a kvm_arch_put_registers(), but I found rtas_start_cpu()
> as a place where this change is guaranteed to get reflected.
Ok, makes sense.
In that case can we move all, or nearly all, of the PAPR specific
thread initialization to rtas_start_cpu()? Obviously we'd need a
separate call to the same stuff for the boot cpu.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-03-01 13:55 ` Igor Mammedov
@ 2016-03-03 9:30 ` Bharata B Rao
2016-03-03 15:54 ` Igor Mammedov
0 siblings, 1 reply; 45+ messages in thread
From: Bharata B Rao @ 2016-03-03 9:30 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Tue, Mar 01, 2016 at 02:55:02PM +0100, Igor Mammedov wrote:
> On Tue, 1 Mar 2016 14:39:51 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > On Mon, Feb 29, 2016 at 11:46:42AM +0100, Igor Mammedov wrote:
> > > On Thu, 25 Feb 2016 21:52:41 +0530
> > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > >
> > > > Implement query cpu-slots that provides information about hot-plugged
> > > > as well as hot-pluggable CPU slots that the machine supports.
> > > >
> > > > TODO: As Eric suggested use enum for type instead of str.
> > > > TODO: @hotplug-granularity probably isn't required.
> > > >
> > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > ---
> > > > hw/core/machine.c | 19 +++++++++
> > > > hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > include/hw/boards.h | 4 ++
> > > > qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> > > > qmp-commands.hx | 47 ++++++++++++++++++++++
> > > > 5 files changed, 267 insertions(+)
> > > >
> > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > index 6d1a0d8..3055ef8 100644
> > > > --- a/hw/core/machine.c
> > > > +++ b/hw/core/machine.c
> > > > @@ -17,6 +17,25 @@
> > > > #include "hw/sysbus.h"
> > > > #include "sysemu/sysemu.h"
> > > > #include "qemu/error-report.h"
> > > > +#include "qmp-commands.h"
> > > > +
> > > > +/*
> > > > + * QMP: query-cpu-slots
> > > > + *
> > > > + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> > > > + */
> > > > +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> > > > +{
> > > > + MachineState *ms = MACHINE(qdev_get_machine());
> > > > + MachineClass *mc = MACHINE_GET_CLASS(ms);
> > > > +
> > > > + if (!mc->cpu_slots) {
> > > > + error_setg(errp, QERR_UNSUPPORTED);
> > > > + return NULL;
> > > > + }
> > > > +
> > > > + return mc->cpu_slots(ms);
> > > > +}
> > > >
> > > > static char *machine_get_accel(Object *obj, Error **errp)
> > > > {
> > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > index 780cd00..b76ed85 100644
> > > > --- a/hw/ppc/spapr.c
> > > > +++ b/hw/ppc/spapr.c
> > > > @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> > > > return cpu_index / smp_threads / smp_cores;
> > > > }
> > > >
> > > > +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> > > > +{
> > > > + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> > > > + CPUInfoList ***prev = opaque;
> > > > +
> > > > + if (object_dynamic_cast(obj, TYPE_CPU)) {
> > > > + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> > > > + CPUInfo *s = g_new0(CPUInfo, 1);
> > > > + CPUState *cpu = CPU(obj);
> > > > + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> > > > +
> > > > + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> > > > + s->type = g_strdup(object_get_typename(obj));
> > > > + s->thread = cpu->cpu_index;
> > > > + s->has_thread = true;
> > > > + s->core = cpu->cpu_index / smp_threads;
> > > > + s->has_core = true;
> > > > + if (mc->cpu_index_to_socket_id) {
> > > > + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> > > > + } else {
> > > > + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> > > > + }
> > > > + s->has_socket = true;
> > > > + s->node = cpu->numa_node;
> > > > + s->has_node = true;
> > > > + s->qom_path = object_get_canonical_path(obj);
> > > > + s->has_qom_path = true;
> > > > +
> > > > + elem->value = s;
> > > > + elem->next = NULL;
> > > > + **prev = elem;
> > > > + *prev = &elem->next;
> > > > + }
> > > > + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> > > > +{
> > > > + CPUSlotInfoList *head = NULL;
> > > > + CPUSlotInfoList **prev = &head;
> > > > + Object *root_container;
> > > > + ObjectProperty *prop;
> > > > + ObjectPropertyIterator iter;
> > > > +
> > > > + /*
> > > > + * TODO: There surely must be a better/easier way to walk all
> > > > + * the link properties of an object ?
> > > > + */
> > > > + root_container = container_get(object_get_root(), "/machine");
> > > > + object_property_iter_init(&iter, root_container);
> > > > +
> > > > + while ((prop = object_property_iter_next(&iter))) {
> > > > + Object *obj;
> > > > + DeviceState *dev;
> > > > + CPUSlotInfoList *elem;
> > > > + CPUSlotInfo *s;
> > > > + CPUInfoList *cpu_head = NULL;
> > > > + CPUInfoList **cpu_prev = &cpu_head;
> > > > + sPAPRCPUCore *core;
> > > > +
> > > > + if (!strstart(prop->type, "link<", NULL)) {
> > > > + continue;
> > > > + }
> > > > +
> > > > + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> > > > + continue;
> > > > + }
> > > > +
> > > > + elem = g_new0(CPUSlotInfoList, 1);
> > > > + s = g_new0(CPUSlotInfo, 1);
> > > > +
> > > > + obj = object_property_get_link(root_container, prop->name, NULL);
> > > > + if (obj) {
> > > > + /* Slot populated */
> > > > + dev = DEVICE(obj);
> > > > + core = SPAPR_CPU_CORE(obj);
> > > > +
> > > > + if (dev->id) {
> > > > + s->has_id = true;
> > > > + s->id = g_strdup(dev->id);
> > > > + }
> > > > + s->realized = object_property_get_bool(obj, "realized", NULL);
> > > > + s->nr_cpus = core->nr_threads;
> > > > + s->has_nr_cpus = true;
> > > > + s->qom_path = object_get_canonical_path(obj);
> > > > + s->has_qom_path = true;
> > > > + if (s->realized) {
> > > > + spapr_cpuinfo_list(obj, &cpu_prev);
> > > > + }
> > > > + s->has_cpus = true;
> > > > + } else {
> > > > + /* Slot empty */
> > > > + s->has_id = false;
> > > > + s->has_nr_cpus = false;
> > > > + s->has_qom_path = false;
> > > > + s->has_cpus = false;
> > > > + s->realized = false;
> > > > + }
> > > > + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> > > > + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> > > > + s->slot_id = g_strdup(prop->name);
> > > > + s->cpus = cpu_head;
> > > > + elem->value = s;
> > > > + elem->next = NULL;
> > > > + *prev = elem;
> > > > + prev = &elem->next;
> > > > + }
> > > > + return head;
> > > > +}
> > > > +
> > > > static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > > {
> > > > MachineClass *mc = MACHINE_CLASS(oc);
> > > > @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > > hc->plug = spapr_machine_device_plug;
> > > > hc->unplug = spapr_machine_device_unplug;
> > > > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> > > > + mc->cpu_slots = spapr_cpu_slots;
> > > >
> > > > smc->dr_lmb_enabled = true;
> > > > smc->dr_cpu_enabled = true;
> > > > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > > > index 0f30959..d888a02 100644
> > > > --- a/include/hw/boards.h
> > > > +++ b/include/hw/boards.h
> > > > @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> > > > * Set only by old machines because they need to keep
> > > > * compatibility on code that exposed QEMU_VERSION to guests in
> > > > * the past (and now use qemu_hw_version()).
> > > > + * @cpu_slots:
> > > > + * Provides information about populated and yet-to-be populated
> > > > + * CPU slots in the machine. Used by QMP query-cpu-slots.
> > > > */
> > > > struct MachineClass {
> > > > /*< private >*/
> > > > @@ -99,6 +102,7 @@ struct MachineClass {
> > > > HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> > > > DeviceState *dev);
> > > > unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> > > > + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> > > > };
> > > >
> > > > /**
> > > > diff --git a/qapi-schema.json b/qapi-schema.json
> > > > index 8d04897..e9a52a2 100644
> > > > --- a/qapi-schema.json
> > > > +++ b/qapi-schema.json
> > > > @@ -4083,3 +4083,88 @@
> > > > ##
> > > > { 'enum': 'ReplayMode',
> > > > 'data': [ 'none', 'record', 'play' ] }
> > > > +
> > > > +##
> > > > +# @CPUInfo:
> > > > +#
> > > > +# Information about CPUs
> > > > +#
> > > > +# @arch-id: Arch-specific ID for the CPU.
> > > > +#
> > > > +# @type: QOM type of the CPU.
> > > > +#
> > > > +# @thread: Thread ID of the CPU.
> > > > +#
> > > > +# @core: Core ID of the CPU.
> > > > +#
> > > > +# @socket: Socket ID of the CPU.
> > > > +#
> > > > +# @node: NUMA node to which the CPU belongs.
> > > > +#
> > > > +# @qom-path: QOM path of the CPU object
> > > > +#
> > > > +# Since: 2.6
> > > > +##
> > > > +
> > > > +{ 'struct': 'CPUInfo',
> > > > + 'data': { 'arch-id': 'int',
> > > > + 'type': 'str',
> > > > + '*thread': 'int',
> > > > + '*core': 'int',
> > > > + '*socket' : 'int',
> > > > + '*node' : 'int',
> > > > + '*qom-path': 'str'
> > > > + }
> > > > +}
> > > > +
> > > > +##
> > > > +# @CPUSlotInfo:
> > > > +#
> > > > +# Information about CPU Slots
> > > > +#
> > > > +# @id: Device ID of the CPU composite object that occupies the slot.
> > > > +#
> > > > +# @type: QOM type of the CPU composite object that occupies the slot.
> > > > +#
> > > > +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> > > > +# can be thread, core or socket.
> > > > +#
> > > > +# @slot-id: Slot's identifier.
> > > > +#
> > > > +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> > > > +#
> > > > +# @realized: A boolean that indicates whether the slot is filled or empty.
> > > > +#
> > > > +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> > > > +# this slot.
> > > > +#
> > > > +# @cpus: An array of @CPUInfo elements where each element describes one
> > > > +# CPU that is part of this slot's CPU composite object.
> > > > +#
> > > > +# @type: QOM type
> > > > +#
> > > > +# Since: 2.6
> > > > +##
> > > > +
> > > > +{ 'struct': 'CPUSlotInfo',
> > > > + 'data': { '*id': 'str',
> > > > + 'type': 'str',
> > > > + 'hotplug-granularity' : 'str',
> > > Does it convey any useful info, if yes how it will be used by mgmt?
> >
> > As I noted in the patch desc, this field is useless, will remove.
> >
> > >
> > > > + 'slot-id' : 'str',
> > > > + '*qom-path': 'str',
> > > > + 'realized': 'bool',
> > > field's redundant, presence of qom-path answers this question
> >
> > Ok, makes sense.
> >
> > >
> > > > + '*nr-cpus': 'int',
> > > > + '*cpus' : ['CPUInfo']
> > > I'd suggest to drop above 2 fields as it's impl dependent,
> > > qom-path already point's to socket/core/thread object and
> > > its internal composition can be explored by other means if needed.
> > >
> > > Moreover 'CPUInfo' is confusing and sort of conflicts with existing
> > > 'CpuInfo'.
> >
> > Ah I see, should change the naming if we eventually stick with this
> > implementation.
> >
> > > I'd drop CPUInfo altogether and introduce only 'CPUSlotInfo' here,
> > > existing thread enumeration's already implemented query-cpus.
> >
> > Ok.
> >
> > >
> > > > + }
> > > > +}
> > > What I miss here is that CPUSlotInfo doesn't provide any
> > > information to about where CPU might be hotplugged to.
> > >
> > > Maybe use following tuple instead of slot-id?
> > >
> > > { 'struct': 'CPUSlotProperties',
> > > 'data': { '*node': 'int',
> > > '*socket': 'int',
> > > '*core': 'int',
> > > '*thread': 'int'
> > > }
> > > }
> >
> > Hmm not sure. If I undestand Andreas' proposal correctly, slot is the
> > place where the CPU sits. Machine determines the type of the slot and it
> > can be socket slot, core slot or thread slot based on the granularity
> > of the hotplug supported by the machine. With this I don't see why
> > anything else apart from slot-id/slot-name is required to figure out where
> > to hoplug CPU.
> Of cause it's possible to create and keep at board level map of
> slot-names to whatever other info needed to create a CPU object
> at machine init time, and then make board somehow to apply it
> to the new CPU object before realizing it.
>
> But it doesn't give mgmt any information whatsoever about where CPU
> is being hotplugged. So for x86/arm we would end up with adding
> yet another interface that would tell it. If CPUSlotProperties is used
> instead of slot-name, it could convey that information while
> keeping interface compatible with a range targets we care about
> and extendable to other targets in future.
>
> > In the current implementation, sPAPR hotplug is at core granularity.
> > CPUSlotInfo.type advertises the type as spapr-cpu-core. The number of
> > threads in the core and the CPU model of the threads are either machine
> > default or specified by user during hotplug time.
> now imagine extending in future sPAPR to support NUMA, which way would
> you extend interface, how would mgmt know which CPU belongs to what node?
>
> As you may note in CPUSlotProperties fields are optional so target would
> use only ones it needs. For current sPAPR, query result could be
> something like this:
>
> {
> {
> type: spapr-cpu-core
> properties: { core: 0 }
> qom-path: /machine/unattached/device[xxx]
> }
> {
> type: spapr-cpu-core
> properties: { core: 1 }
> }
> ...
> {
> type: spapr-cpu-core
> properties: { core: 0 }
> }
> }
>
> Later if numa was needed, the board could set 'numa' property as well.
> Would that work for sPAPR?
It could work, however I am finding it difficult to reconcile this with
the machine object to core links that I have in this implementation which
I believe is what Andreas originally suggested. Such a link essentially
represents a slot and it has a name associated with it which identifies the
location where the core is plugged.
Now in order to incorporate CPUSlotProperties, these properties must
be tied to a slot/location object which in the present implementation
doesn't exist. So are you suggesting that we create slot/location
object (abstract ?) that has CPUSlotProperties ? If so, then I guess we
don't need machine object to core object links ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine
2016-03-03 9:30 ` Bharata B Rao
@ 2016-03-03 15:54 ` Igor Mammedov
0 siblings, 0 replies; 45+ messages in thread
From: Igor Mammedov @ 2016-03-03 15:54 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Thu, 3 Mar 2016 15:00:32 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Tue, Mar 01, 2016 at 02:55:02PM +0100, Igor Mammedov wrote:
> > On Tue, 1 Mar 2016 14:39:51 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > On Mon, Feb 29, 2016 at 11:46:42AM +0100, Igor Mammedov wrote:
> > > > On Thu, 25 Feb 2016 21:52:41 +0530
> > > > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> > > >
> > > > > Implement query cpu-slots that provides information about hot-plugged
> > > > > as well as hot-pluggable CPU slots that the machine supports.
> > > > >
> > > > > TODO: As Eric suggested use enum for type instead of str.
> > > > > TODO: @hotplug-granularity probably isn't required.
> > > > >
> > > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > > ---
> > > > > hw/core/machine.c | 19 +++++++++
> > > > > hw/ppc/spapr.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > > include/hw/boards.h | 4 ++
> > > > > qapi-schema.json | 85 +++++++++++++++++++++++++++++++++++++++
> > > > > qmp-commands.hx | 47 ++++++++++++++++++++++
> > > > > 5 files changed, 267 insertions(+)
> > > > >
> > > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > > index 6d1a0d8..3055ef8 100644
> > > > > --- a/hw/core/machine.c
> > > > > +++ b/hw/core/machine.c
> > > > > @@ -17,6 +17,25 @@
> > > > > #include "hw/sysbus.h"
> > > > > #include "sysemu/sysemu.h"
> > > > > #include "qemu/error-report.h"
> > > > > +#include "qmp-commands.h"
> > > > > +
> > > > > +/*
> > > > > + * QMP: query-cpu-slots
> > > > > + *
> > > > > + * TODO: Ascertain if this is the right place to for this arch-neutral routine.
> > > > > + */
> > > > > +CPUSlotInfoList *qmp_query_cpu_slots(Error **errp)
> > > > > +{
> > > > > + MachineState *ms = MACHINE(qdev_get_machine());
> > > > > + MachineClass *mc = MACHINE_GET_CLASS(ms);
> > > > > +
> > > > > + if (!mc->cpu_slots) {
> > > > > + error_setg(errp, QERR_UNSUPPORTED);
> > > > > + return NULL;
> > > > > + }
> > > > > +
> > > > > + return mc->cpu_slots(ms);
> > > > > +}
> > > > >
> > > > > static char *machine_get_accel(Object *obj, Error **errp)
> > > > > {
> > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > index 780cd00..b76ed85 100644
> > > > > --- a/hw/ppc/spapr.c
> > > > > +++ b/hw/ppc/spapr.c
> > > > > @@ -2453,6 +2453,117 @@ static unsigned spapr_cpu_index_to_socket_id(unsigned cpu_index)
> > > > > return cpu_index / smp_threads / smp_cores;
> > > > > }
> > > > >
> > > > > +static int spapr_cpuinfo_list(Object *obj, void *opaque)
> > > > > +{
> > > > > + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
> > > > > + CPUInfoList ***prev = opaque;
> > > > > +
> > > > > + if (object_dynamic_cast(obj, TYPE_CPU)) {
> > > > > + CPUInfoList *elem = g_new0(CPUInfoList, 1);
> > > > > + CPUInfo *s = g_new0(CPUInfo, 1);
> > > > > + CPUState *cpu = CPU(obj);
> > > > > + PowerPCCPU *pcpu = POWERPC_CPU(cpu);
> > > > > +
> > > > > + s->arch_id = ppc_get_vcpu_dt_id(pcpu);
> > > > > + s->type = g_strdup(object_get_typename(obj));
> > > > > + s->thread = cpu->cpu_index;
> > > > > + s->has_thread = true;
> > > > > + s->core = cpu->cpu_index / smp_threads;
> > > > > + s->has_core = true;
> > > > > + if (mc->cpu_index_to_socket_id) {
> > > > > + s->socket = mc->cpu_index_to_socket_id(cpu->cpu_index);
> > > > > + } else {
> > > > > + s->socket = cpu->cpu_index / smp_threads / smp_cores;
> > > > > + }
> > > > > + s->has_socket = true;
> > > > > + s->node = cpu->numa_node;
> > > > > + s->has_node = true;
> > > > > + s->qom_path = object_get_canonical_path(obj);
> > > > > + s->has_qom_path = true;
> > > > > +
> > > > > + elem->value = s;
> > > > > + elem->next = NULL;
> > > > > + **prev = elem;
> > > > > + *prev = &elem->next;
> > > > > + }
> > > > > + object_child_foreach(obj, spapr_cpuinfo_list, opaque);
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > +static CPUSlotInfoList *spapr_cpu_slots(MachineState *machine)
> > > > > +{
> > > > > + CPUSlotInfoList *head = NULL;
> > > > > + CPUSlotInfoList **prev = &head;
> > > > > + Object *root_container;
> > > > > + ObjectProperty *prop;
> > > > > + ObjectPropertyIterator iter;
> > > > > +
> > > > > + /*
> > > > > + * TODO: There surely must be a better/easier way to walk all
> > > > > + * the link properties of an object ?
> > > > > + */
> > > > > + root_container = container_get(object_get_root(), "/machine");
> > > > > + object_property_iter_init(&iter, root_container);
> > > > > +
> > > > > + while ((prop = object_property_iter_next(&iter))) {
> > > > > + Object *obj;
> > > > > + DeviceState *dev;
> > > > > + CPUSlotInfoList *elem;
> > > > > + CPUSlotInfo *s;
> > > > > + CPUInfoList *cpu_head = NULL;
> > > > > + CPUInfoList **cpu_prev = &cpu_head;
> > > > > + sPAPRCPUCore *core;
> > > > > +
> > > > > + if (!strstart(prop->type, "link<", NULL)) {
> > > > > + continue;
> > > > > + }
> > > > > +
> > > > > + if (!strstart(prop->name, SPAPR_MACHINE_CPU_CORE_PROP, NULL)) {
> > > > > + continue;
> > > > > + }
> > > > > +
> > > > > + elem = g_new0(CPUSlotInfoList, 1);
> > > > > + s = g_new0(CPUSlotInfo, 1);
> > > > > +
> > > > > + obj = object_property_get_link(root_container, prop->name, NULL);
> > > > > + if (obj) {
> > > > > + /* Slot populated */
> > > > > + dev = DEVICE(obj);
> > > > > + core = SPAPR_CPU_CORE(obj);
> > > > > +
> > > > > + if (dev->id) {
> > > > > + s->has_id = true;
> > > > > + s->id = g_strdup(dev->id);
> > > > > + }
> > > > > + s->realized = object_property_get_bool(obj, "realized", NULL);
> > > > > + s->nr_cpus = core->nr_threads;
> > > > > + s->has_nr_cpus = true;
> > > > > + s->qom_path = object_get_canonical_path(obj);
> > > > > + s->has_qom_path = true;
> > > > > + if (s->realized) {
> > > > > + spapr_cpuinfo_list(obj, &cpu_prev);
> > > > > + }
> > > > > + s->has_cpus = true;
> > > > > + } else {
> > > > > + /* Slot empty */
> > > > > + s->has_id = false;
> > > > > + s->has_nr_cpus = false;
> > > > > + s->has_qom_path = false;
> > > > > + s->has_cpus = false;
> > > > > + s->realized = false;
> > > > > + }
> > > > > + s->type = g_strdup(TYPE_SPAPR_CPU_CORE);
> > > > > + s->hotplug_granularity = g_strdup(SPAPR_MACHINE_CPU_CORE_PROP);
> > > > > + s->slot_id = g_strdup(prop->name);
> > > > > + s->cpus = cpu_head;
> > > > > + elem->value = s;
> > > > > + elem->next = NULL;
> > > > > + *prev = elem;
> > > > > + prev = &elem->next;
> > > > > + }
> > > > > + return head;
> > > > > +}
> > > > > +
> > > > > static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > > > {
> > > > > MachineClass *mc = MACHINE_CLASS(oc);
> > > > > @@ -2482,6 +2593,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > > > > hc->plug = spapr_machine_device_plug;
> > > > > hc->unplug = spapr_machine_device_unplug;
> > > > > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
> > > > > + mc->cpu_slots = spapr_cpu_slots;
> > > > >
> > > > > smc->dr_lmb_enabled = true;
> > > > > smc->dr_cpu_enabled = true;
> > > > > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > > > > index 0f30959..d888a02 100644
> > > > > --- a/include/hw/boards.h
> > > > > +++ b/include/hw/boards.h
> > > > > @@ -57,6 +57,9 @@ bool machine_mem_merge(MachineState *machine);
> > > > > * Set only by old machines because they need to keep
> > > > > * compatibility on code that exposed QEMU_VERSION to guests in
> > > > > * the past (and now use qemu_hw_version()).
> > > > > + * @cpu_slots:
> > > > > + * Provides information about populated and yet-to-be populated
> > > > > + * CPU slots in the machine. Used by QMP query-cpu-slots.
> > > > > */
> > > > > struct MachineClass {
> > > > > /*< private >*/
> > > > > @@ -99,6 +102,7 @@ struct MachineClass {
> > > > > HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> > > > > DeviceState *dev);
> > > > > unsigned (*cpu_index_to_socket_id)(unsigned cpu_index);
> > > > > + CPUSlotInfoList *(*cpu_slots)(MachineState *machine);
> > > > > };
> > > > >
> > > > > /**
> > > > > diff --git a/qapi-schema.json b/qapi-schema.json
> > > > > index 8d04897..e9a52a2 100644
> > > > > --- a/qapi-schema.json
> > > > > +++ b/qapi-schema.json
> > > > > @@ -4083,3 +4083,88 @@
> > > > > ##
> > > > > { 'enum': 'ReplayMode',
> > > > > 'data': [ 'none', 'record', 'play' ] }
> > > > > +
> > > > > +##
> > > > > +# @CPUInfo:
> > > > > +#
> > > > > +# Information about CPUs
> > > > > +#
> > > > > +# @arch-id: Arch-specific ID for the CPU.
> > > > > +#
> > > > > +# @type: QOM type of the CPU.
> > > > > +#
> > > > > +# @thread: Thread ID of the CPU.
> > > > > +#
> > > > > +# @core: Core ID of the CPU.
> > > > > +#
> > > > > +# @socket: Socket ID of the CPU.
> > > > > +#
> > > > > +# @node: NUMA node to which the CPU belongs.
> > > > > +#
> > > > > +# @qom-path: QOM path of the CPU object
> > > > > +#
> > > > > +# Since: 2.6
> > > > > +##
> > > > > +
> > > > > +{ 'struct': 'CPUInfo',
> > > > > + 'data': { 'arch-id': 'int',
> > > > > + 'type': 'str',
> > > > > + '*thread': 'int',
> > > > > + '*core': 'int',
> > > > > + '*socket' : 'int',
> > > > > + '*node' : 'int',
> > > > > + '*qom-path': 'str'
> > > > > + }
> > > > > +}
> > > > > +
> > > > > +##
> > > > > +# @CPUSlotInfo:
> > > > > +#
> > > > > +# Information about CPU Slots
> > > > > +#
> > > > > +# @id: Device ID of the CPU composite object that occupies the slot.
> > > > > +#
> > > > > +# @type: QOM type of the CPU composite object that occupies the slot.
> > > > > +#
> > > > > +# @hotplug-granularity: Granularity of CPU composite hotplug for this slot,
> > > > > +# can be thread, core or socket.
> > > > > +#
> > > > > +# @slot-id: Slot's identifier.
> > > > > +#
> > > > > +# @qom-path: QOM path of the CPU composite object that occupies the slot.
> > > > > +#
> > > > > +# @realized: A boolean that indicates whether the slot is filled or empty.
> > > > > +#
> > > > > +# @nr-cpus: Number of CPUs that are part of CPU composite object that occupies
> > > > > +# this slot.
> > > > > +#
> > > > > +# @cpus: An array of @CPUInfo elements where each element describes one
> > > > > +# CPU that is part of this slot's CPU composite object.
> > > > > +#
> > > > > +# @type: QOM type
> > > > > +#
> > > > > +# Since: 2.6
> > > > > +##
> > > > > +
> > > > > +{ 'struct': 'CPUSlotInfo',
> > > > > + 'data': { '*id': 'str',
> > > > > + 'type': 'str',
> > > > > + 'hotplug-granularity' : 'str',
> > > > Does it convey any useful info, if yes how it will be used by mgmt?
> > >
> > > As I noted in the patch desc, this field is useless, will remove.
> > >
> > > >
> > > > > + 'slot-id' : 'str',
> > > > > + '*qom-path': 'str',
> > > > > + 'realized': 'bool',
> > > > field's redundant, presence of qom-path answers this question
> > >
> > > Ok, makes sense.
> > >
> > > >
> > > > > + '*nr-cpus': 'int',
> > > > > + '*cpus' : ['CPUInfo']
> > > > I'd suggest to drop above 2 fields as it's impl dependent,
> > > > qom-path already point's to socket/core/thread object and
> > > > its internal composition can be explored by other means if needed.
> > > >
> > > > Moreover 'CPUInfo' is confusing and sort of conflicts with existing
> > > > 'CpuInfo'.
> > >
> > > Ah I see, should change the naming if we eventually stick with this
> > > implementation.
> > >
> > > > I'd drop CPUInfo altogether and introduce only 'CPUSlotInfo' here,
> > > > existing thread enumeration's already implemented query-cpus.
> > >
> > > Ok.
> > >
> > > >
> > > > > + }
> > > > > +}
> > > > What I miss here is that CPUSlotInfo doesn't provide any
> > > > information to about where CPU might be hotplugged to.
> > > >
> > > > Maybe use following tuple instead of slot-id?
> > > >
> > > > { 'struct': 'CPUSlotProperties',
> > > > 'data': { '*node': 'int',
> > > > '*socket': 'int',
> > > > '*core': 'int',
> > > > '*thread': 'int'
> > > > }
> > > > }
> > >
> > > Hmm not sure. If I undestand Andreas' proposal correctly, slot is the
> > > place where the CPU sits. Machine determines the type of the slot and it
> > > can be socket slot, core slot or thread slot based on the granularity
> > > of the hotplug supported by the machine. With this I don't see why
> > > anything else apart from slot-id/slot-name is required to figure out where
> > > to hoplug CPU.
> > Of cause it's possible to create and keep at board level map of
> > slot-names to whatever other info needed to create a CPU object
> > at machine init time, and then make board somehow to apply it
> > to the new CPU object before realizing it.
> >
> > But it doesn't give mgmt any information whatsoever about where CPU
> > is being hotplugged. So for x86/arm we would end up with adding
> > yet another interface that would tell it. If CPUSlotProperties is used
> > instead of slot-name, it could convey that information while
> > keeping interface compatible with a range targets we care about
> > and extendable to other targets in future.
> >
> > > In the current implementation, sPAPR hotplug is at core granularity.
> > > CPUSlotInfo.type advertises the type as spapr-cpu-core. The number of
> > > threads in the core and the CPU model of the threads are either machine
> > > default or specified by user during hotplug time.
> > now imagine extending in future sPAPR to support NUMA, which way would
> > you extend interface, how would mgmt know which CPU belongs to what node?
> >
> > As you may note in CPUSlotProperties fields are optional so target would
> > use only ones it needs. For current sPAPR, query result could be
> > something like this:
> >
> > {
> > {
> > type: spapr-cpu-core
> > properties: { core: 0 }
> > qom-path: /machine/unattached/device[xxx]
> > }
> > {
> > type: spapr-cpu-core
> > properties: { core: 1 }
> > }
> > ...
> > {
> > type: spapr-cpu-core
> > properties: { core: 0 }
> > }
> > }
> >
> > Later if numa was needed, the board could set 'numa' property as well.
> > Would that work for sPAPR?
>
> It could work, however I am finding it difficult to reconcile this with
> the machine object to core links that I have in this implementation which
> I believe is what Andreas originally suggested. Such a link essentially
> represents a slot and it has a name associated with it which identifies the
> location where the core is plugged.
>
> Now in order to incorporate CPUSlotProperties, these properties must
> be tied to a slot/location object which in the present implementation
> doesn't exist. So are you suggesting that we create slot/location
> object (abstract ?) that has CPUSlotProperties ? If so, then I guess we
> don't need machine object to core object links ?
I consider links as QOM interface to access Existing CPUs and
they links are more or less useless when it comes to possible CPUs.
If you really have to have links, you can associate them with
cores instead of slots, so it would look like:
/machine/
-> cpu-core[0]
-> cpu-core[1]
...
I'm not suggesting anything related to QOM interface or QOM
in general when we discussing QMP interface in this patch.
On contrary this QMP query should be QOM agnostic so each target
could implement it regardless of what QOM model it uses.
So CPUSlotProperties is a pure QMP construct not tied to QOM
and it's up to machine callback to return CPUs layout description
like it has been proposed in https://patchwork.ozlabs.org/patch/583036/
I think in that thread, I've succeed in convincing Eduardo
and David to set aside QOM interface and implement only QMP command.
(Perhaps looking through that thread will convince you too if you
re-read it). I'll repost that patch taking in account comments
on that and this mail-thread using CPUSlotProperties.
That way we separate interface issue from QOM interface, making
QOM modeling a target implementation detail. Then each target
can implement it's own QOM device model (including links if it wishes)
to play with but still use common QMP interface which will not
be affected by it.
So mgmt would get stable well described/documented interface,
and we could eventually move with device_add based CPU hotplug
from stalemate point. QOM interface would be left for future
exercise or as experimental thing until it became at parity
with QMP one and had time settle down more or less.
>
> Regards,
> Bharata.
>
^ permalink raw reply [flat|nested] 45+ messages in thread
end of thread, other threads:[~2016-03-03 15:55 UTC | newest]
Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-25 16:22 [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 1/6] cpu: Abstract CPU core type Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 2/6] spapr: CPU core device Bharata B Rao
2016-02-26 2:57 ` David Gibson
2016-02-26 5:39 ` Bharata B Rao
2016-02-26 10:46 ` Thomas Huth
2016-02-29 5:39 ` Bharata B Rao
2016-02-26 18:13 ` Michael Roth
2016-02-29 3:44 ` David Gibson
2016-02-29 5:50 ` Bharata B Rao
2016-02-29 10:03 ` Igor Mammedov
2016-02-29 12:55 ` Bharata B Rao
2016-02-29 15:15 ` Igor Mammedov
2016-03-01 1:21 ` David Gibson
2016-03-01 9:27 ` Igor Mammedov
2016-03-01 8:17 ` Bharata B Rao
2016-03-01 9:16 ` Igor Mammedov
2016-03-01 9:45 ` Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 3/6] spapr: Represent boot CPUs as spapr-cpu-core devices Bharata B Rao
2016-02-26 3:45 ` David Gibson
2016-02-26 4:02 ` Bharata B Rao
2016-02-26 15:18 ` Igor Mammedov
2016-02-29 5:35 ` Bharata B Rao
2016-02-29 7:11 ` David Gibson
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 4/6] spapr: CPU hotplug support Bharata B Rao
2016-02-26 3:51 ` David Gibson
2016-02-29 4:42 ` Bharata B Rao
2016-03-01 7:58 ` Bharata B Rao
2016-03-02 0:53 ` David Gibson
2016-02-26 13:03 ` Thomas Huth
2016-02-26 14:54 ` Bharata B Rao
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 5/6] qmp, spapr: Show hot-plugged/pluggable CPU slots in the Machine Bharata B Rao
2016-02-26 4:03 ` David Gibson
2016-02-26 9:40 ` Bharata B Rao
2016-02-26 15:58 ` Eric Blake
2016-02-29 5:43 ` Bharata B Rao
2016-02-26 16:33 ` Thomas Huth
2016-02-29 10:46 ` Igor Mammedov
2016-03-01 9:09 ` Bharata B Rao
2016-03-01 13:55 ` Igor Mammedov
2016-03-03 9:30 ` Bharata B Rao
2016-03-03 15:54 ` Igor Mammedov
2016-02-25 16:22 ` [Qemu-devel] [RFC PATCH v0 6/6] hmp: Implement 'info cpu-slots' Bharata B Rao
2016-03-01 10:00 ` [Qemu-devel] [RFC PATCH v0 0/6] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-03-01 13:59 ` Andreas Färber
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.