* [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR
@ 2016-03-31 8:39 Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 01/12] exec: Remove cpu from cpus list during cpu_exec_exit() Bharata B Rao
` (12 more replies)
0 siblings, 13 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Hi,
This is v2.1 of "Core based CPU hotplug for PowerPC sPAPR". v2 was posted
here: https://lists.gnu.org/archive/html/qemu-ppc/2016-03/msg00201.html
I am making this v2.1 instead of v3 as this introduces a change in
device_add semantics and if we don't want to continue this, I will go back
to the previous v2 semantics from v3 onwards. v2 had spapr-cpu-core device
that served as CPU core device for all types of PowerPCCPU threads.
In this version however, spapr-cpu-core becomes an abstract device and we
will have different core devices for each of the PowerPCCPU type. So
in this version, the hotplug semantics looks like this:
(qemu) device_add POWER8-spapr-cpu-core,id=core2,core=16[,threads=4]
cpu_model specification is gone as it becomes redundant with different
core types for each CPU type. CPU core types are defined only for
host, POWER7 and POWER8 models only (and not for their variants/aliases)
yet.
I have mostly taken care of all the review comments that I got for v2.
Major changes in this version (v2.1)
------------------------------------
- Semantics change: spapr-cpu-core to POWER8-spapr-cpu-core and likewise
- Included Igor's pre_plug patch.
Bharata B Rao (10):
exec: Remove cpu from cpus list during cpu_exec_exit()
exec: Do vmstate unregistration from cpu_exec_exit()
cpu: Add a sync version of cpu_remove()
cpu: Abstract CPU core type
spapr: Abstract CPU core device
spapr: Add CPU type specific core devices
spapr: convert boot CPUs into CPU core devices
spapr: CPU hotplug support
xics,xics_kvm: Handle CPU unplug correctly
spapr: CPU hot unplug support
Gu Zheng (1):
cpu: Reclaim vCPU objects
Igor Mammedov (1):
qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
cpus.c | 51 +++++-
exec.c | 41 ++++-
hw/core/hotplug.c | 11 ++
hw/core/qdev.c | 9 +-
hw/cpu/Makefile.objs | 1 +
hw/cpu/core.c | 87 ++++++++++
hw/intc/xics.c | 14 ++
hw/intc/xics_kvm.c | 8 +-
hw/ppc/Makefile.objs | 1 +
hw/ppc/spapr.c | 150 +++++++++++++++--
hw/ppc/spapr_cpu_core.c | 349 ++++++++++++++++++++++++++++++++++++++++
hw/ppc/spapr_events.c | 3 +
hw/ppc/spapr_rtas.c | 24 +++
include/hw/cpu/core.h | 31 ++++
include/hw/hotplug.h | 14 +-
include/hw/ppc/spapr.h | 6 +
include/hw/ppc/spapr_cpu_core.h | 79 +++++++++
include/hw/ppc/xics.h | 1 +
include/qom/cpu.h | 18 +++
include/sysemu/kvm.h | 1 +
kvm-all.c | 57 ++++++-
kvm-stub.c | 5 +
22 files changed, 932 insertions(+), 29 deletions(-)
create mode 100644 hw/cpu/core.c
create mode 100644 hw/ppc/spapr_cpu_core.c
create mode 100644 include/hw/cpu/core.h
create mode 100644 include/hw/ppc/spapr_cpu_core.h
--
2.1.0
^ permalink raw reply [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 01/12] exec: Remove cpu from cpus list during cpu_exec_exit()
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 02/12] exec: Do vmstate unregistration from cpu_exec_exit() Bharata B Rao
` (11 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
CPUState *cpu gets added to the cpus list during cpu_exec_init(). It
should be removed from cpu_exec_exit().
cpu_exec_exit() is called from generic CPU::instance_finalize and some
archs like PowerPC call it from CPU unrealizefn. So ensure that we
dequeue the cpu only once.
Now -1 value for cpu->cpu_index indicates that we have already dequeued
the cpu for CONFIG_USER_ONLY case also.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
exec.c | 32 ++++++++++++++++++++++++--------
1 file changed, 24 insertions(+), 8 deletions(-)
diff --git a/exec.c b/exec.c
index f09dd4e..49ae593 100644
--- a/exec.c
+++ b/exec.c
@@ -609,15 +609,9 @@ static int cpu_get_free_index(Error **errp)
return cpu;
}
-void cpu_exec_exit(CPUState *cpu)
+static void cpu_release_index(CPUState *cpu)
{
- if (cpu->cpu_index == -1) {
- /* cpu_index was never allocated by this @cpu or was already freed. */
- return;
- }
-
bitmap_clear(cpu_index_map, cpu->cpu_index, 1);
- cpu->cpu_index = -1;
}
#else
@@ -632,11 +626,33 @@ static int cpu_get_free_index(Error **errp)
return cpu_index;
}
-void cpu_exec_exit(CPUState *cpu)
+static void cpu_release_index(CPUState *cpu)
{
+ return;
}
#endif
+void cpu_exec_exit(CPUState *cpu)
+{
+#if defined(CONFIG_USER_ONLY)
+ cpu_list_lock();
+#endif
+ if (cpu->cpu_index == -1) {
+ /* cpu_index was never allocated by this @cpu or was already freed. */
+#if defined(CONFIG_USER_ONLY)
+ cpu_list_unlock();
+#endif
+ return;
+ }
+
+ QTAILQ_REMOVE(&cpus, cpu, node);
+ cpu_release_index(cpu);
+ cpu->cpu_index = -1;
+#if defined(CONFIG_USER_ONLY)
+ cpu_list_unlock();
+#endif
+}
+
void cpu_exec_init(CPUState *cpu, Error **errp)
{
CPUClass *cc = CPU_GET_CLASS(cpu);
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 02/12] exec: Do vmstate unregistration from cpu_exec_exit()
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 01/12] exec: Remove cpu from cpus list during cpu_exec_exit() Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 03/12] cpu: Reclaim vCPU objects Bharata B Rao
` (10 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
cpu_exec_init() does vmstate_register for the CPU device. This needs to be
undone from cpu_exec_exit(). This change is needed to support CPU hot
removal.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
exec.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/exec.c b/exec.c
index 49ae593..8ddca6b 100644
--- a/exec.c
+++ b/exec.c
@@ -634,6 +634,8 @@ static void cpu_release_index(CPUState *cpu)
void cpu_exec_exit(CPUState *cpu)
{
+ CPUClass *cc = CPU_GET_CLASS(cpu);
+
#if defined(CONFIG_USER_ONLY)
cpu_list_lock();
#endif
@@ -651,6 +653,13 @@ void cpu_exec_exit(CPUState *cpu)
#if defined(CONFIG_USER_ONLY)
cpu_list_unlock();
#endif
+
+ if (cc->vmsd != NULL) {
+ vmstate_unregister(NULL, cc->vmsd, cpu);
+ }
+ if (qdev_get_vmsd(DEVICE(cpu)) == NULL) {
+ vmstate_unregister(NULL, &vmstate_cpu_common, cpu);
+ }
}
void cpu_exec_init(CPUState *cpu, Error **errp)
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 03/12] cpu: Reclaim vCPU objects
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 01/12] exec: Remove cpu from cpus list during cpu_exec_exit() Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 02/12] exec: Do vmstate unregistration from cpu_exec_exit() Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 04/12] cpu: Add a sync version of cpu_remove() Bharata B Rao
` (9 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, Zhu Guihua, ehabkost, aik,
Bharata B Rao, armbru, agraf, borntraeger, qemu-ppc, Chen Fan,
pbonzini, Gu Zheng, imammedo, mdroth, afaerber, david
From: Gu Zheng <guz.fnst@cn.fujitsu.com>
In order to deal well with the kvm vcpus (which can not be removed without any
protection), we do not close KVM vcpu fd, just record and mark it as stopped
into a list, so that we can reuse it for the appending cpu hot-add request if
possible. It is also the approach that kvm guys suggested:
https://www.mail-archive.com/kvm@vger.kernel.org/msg102839.html
Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
[- Explicit CPU_REMOVE() from qemu_kvm/tcg_destroy_vcpu()
isn't needed as it is done from cpu_exec_exit()
- Use iothread mutex instead of global mutex during
destroy
- Don't cleanup vCPU object from vCPU thread context
but leave it to the callers (device_add/device_del)]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
cpus.c | 39 +++++++++++++++++++++++++++++++++--
include/qom/cpu.h | 10 +++++++++
include/sysemu/kvm.h | 1 +
kvm-all.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++-
kvm-stub.c | 5 +++++
5 files changed, 109 insertions(+), 3 deletions(-)
diff --git a/cpus.c b/cpus.c
index bc774e2..be0ac6a 100644
--- a/cpus.c
+++ b/cpus.c
@@ -953,6 +953,18 @@ void async_run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data)
qemu_cpu_kick(cpu);
}
+static void qemu_kvm_destroy_vcpu(CPUState *cpu)
+{
+ if (kvm_destroy_vcpu(cpu) < 0) {
+ error_report("kvm_destroy_vcpu failed");
+ exit(EXIT_FAILURE);
+ }
+}
+
+static void qemu_tcg_destroy_vcpu(CPUState *cpu)
+{
+}
+
static void flush_queued_work(CPUState *cpu)
{
struct qemu_work_item *wi;
@@ -1045,7 +1057,7 @@ static void *qemu_kvm_cpu_thread_fn(void *arg)
cpu->created = true;
qemu_cond_signal(&qemu_cpu_cond);
- while (1) {
+ do {
if (cpu_can_run(cpu)) {
r = kvm_cpu_exec(cpu);
if (r == EXCP_DEBUG) {
@@ -1053,8 +1065,10 @@ static void *qemu_kvm_cpu_thread_fn(void *arg)
}
}
qemu_kvm_wait_io_event(cpu);
- }
+ } while (!cpu->unplug || cpu_can_run(cpu));
+ qemu_kvm_destroy_vcpu(cpu);
+ qemu_mutex_unlock_iothread();
return NULL;
}
@@ -1108,6 +1122,7 @@ static void tcg_exec_all(void);
static void *qemu_tcg_cpu_thread_fn(void *arg)
{
CPUState *cpu = arg;
+ CPUState *remove_cpu = NULL;
rcu_register_thread();
@@ -1145,6 +1160,16 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
}
}
qemu_tcg_wait_io_event(QTAILQ_FIRST(&cpus));
+ CPU_FOREACH(cpu) {
+ if (cpu->unplug && !cpu_can_run(cpu)) {
+ remove_cpu = cpu;
+ break;
+ }
+ }
+ if (remove_cpu) {
+ qemu_tcg_destroy_vcpu(remove_cpu);
+ remove_cpu = NULL;
+ }
}
return NULL;
@@ -1301,6 +1326,13 @@ void resume_all_vcpus(void)
}
}
+void cpu_remove(CPUState *cpu)
+{
+ cpu->stop = true;
+ cpu->unplug = true;
+ qemu_cpu_kick(cpu);
+}
+
/* For temporary buffers for forming a name */
#define VCPU_THREAD_NAME_SIZE 16
@@ -1517,6 +1549,9 @@ static void tcg_exec_all(void)
break;
}
} else if (cpu->stop || cpu->stopped) {
+ if (cpu->unplug) {
+ next_cpu = CPU_NEXT(cpu);
+ }
break;
}
}
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 7052eee..0720dd7 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -237,6 +237,7 @@ struct kvm_run;
* @halted: Nonzero if the CPU is in suspended state.
* @stop: Indicates a pending stop request.
* @stopped: Indicates the CPU has been artificially stopped.
+ * @unplug: Indicates a pending CPU unplug request.
* @crash_occurred: Indicates the OS reported a crash (panic) for this CPU
* @tcg_exit_req: Set to force TCG to stop executing linked TBs for this
* CPU and return to its top level loop.
@@ -289,6 +290,7 @@ struct CPUState {
bool created;
bool stop;
bool stopped;
+ bool unplug;
bool crash_occurred;
bool exit_request;
uint32_t interrupt_request;
@@ -756,6 +758,14 @@ void cpu_exit(CPUState *cpu);
void cpu_resume(CPUState *cpu);
/**
+ * cpu_remove:
+ * @cpu: The CPU to remove.
+ *
+ * Requests the CPU to be removed.
+ */
+void cpu_remove(CPUState *cpu);
+
+/**
* qemu_init_vcpu:
* @cpu: The vCPU to initialize.
*
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 6695fa7..5d5b602 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -216,6 +216,7 @@ int kvm_has_intx_set_mask(void);
int kvm_init_vcpu(CPUState *cpu);
int kvm_cpu_exec(CPUState *cpu);
+int kvm_destroy_vcpu(CPUState *cpu);
#ifdef NEED_CPU_H
diff --git a/kvm-all.c b/kvm-all.c
index 44c0464..35c0621 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -61,6 +61,12 @@
#define KVM_MSI_HASHTAB_SIZE 256
+struct KVMParkedVcpu {
+ unsigned long vcpu_id;
+ int kvm_fd;
+ QLIST_ENTRY(KVMParkedVcpu) node;
+};
+
struct KVMState
{
AccelState parent_obj;
@@ -94,6 +100,7 @@ struct KVMState
QTAILQ_HEAD(msi_hashtab, KVMMSIRoute) msi_hashtab[KVM_MSI_HASHTAB_SIZE];
#endif
KVMMemoryListener memory_listener;
+ QLIST_HEAD(, KVMParkedVcpu) kvm_parked_vcpus;
};
KVMState *kvm_state;
@@ -237,6 +244,53 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot)
return kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
}
+int kvm_destroy_vcpu(CPUState *cpu)
+{
+ KVMState *s = kvm_state;
+ long mmap_size;
+ struct KVMParkedVcpu *vcpu = NULL;
+ int ret = 0;
+
+ DPRINTF("kvm_destroy_vcpu\n");
+
+ mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
+ if (mmap_size < 0) {
+ ret = mmap_size;
+ DPRINTF("KVM_GET_VCPU_MMAP_SIZE failed\n");
+ goto err;
+ }
+
+ ret = munmap(cpu->kvm_run, mmap_size);
+ if (ret < 0) {
+ goto err;
+ }
+
+ vcpu = g_malloc0(sizeof(*vcpu));
+ vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
+ vcpu->kvm_fd = cpu->kvm_fd;
+ QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
+err:
+ return ret;
+}
+
+static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
+{
+ struct KVMParkedVcpu *cpu;
+
+ QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
+ if (cpu->vcpu_id == vcpu_id) {
+ int kvm_fd;
+
+ QLIST_REMOVE(cpu, node);
+ kvm_fd = cpu->kvm_fd;
+ g_free(cpu);
+ return kvm_fd;
+ }
+ }
+
+ return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
+}
+
int kvm_init_vcpu(CPUState *cpu)
{
KVMState *s = kvm_state;
@@ -245,7 +299,7 @@ int kvm_init_vcpu(CPUState *cpu)
DPRINTF("kvm_init_vcpu\n");
- ret = kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)kvm_arch_vcpu_id(cpu));
+ ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
if (ret < 0) {
DPRINTF("kvm_create_vcpu failed\n");
goto err;
@@ -1495,6 +1549,7 @@ static int kvm_init(MachineState *ms)
#ifdef KVM_CAP_SET_GUEST_DEBUG
QTAILQ_INIT(&s->kvm_sw_breakpoints);
#endif
+ QLIST_INIT(&s->kvm_parked_vcpus);
s->vmfd = -1;
s->fd = qemu_open("/dev/kvm", O_RDWR);
if (s->fd == -1) {
diff --git a/kvm-stub.c b/kvm-stub.c
index b962b24..61f9d5c 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -33,6 +33,11 @@ bool kvm_allowed;
bool kvm_readonly_mem_allowed;
bool kvm_ioeventfd_any_length_allowed;
+int kvm_destroy_vcpu(CPUState *cpu)
+{
+ return -ENOSYS;
+}
+
int kvm_init_vcpu(CPUState *cpu)
{
return -ENOSYS;
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 04/12] cpu: Add a sync version of cpu_remove()
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (2 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 03/12] cpu: Reclaim vCPU objects Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback Bharata B Rao
` (8 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
This sync API will be used by the CPU hotplug code to wait for the CPU to
completely get removed before flagging the failure to the device_add
command.
Sync version of this call is needed to correctly recover from CPU
realization failures when ->plug() handler fails.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
cpus.c | 12 ++++++++++++
include/qom/cpu.h | 8 ++++++++
2 files changed, 20 insertions(+)
diff --git a/cpus.c b/cpus.c
index be0ac6a..05cb096 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1068,6 +1068,8 @@ static void *qemu_kvm_cpu_thread_fn(void *arg)
} while (!cpu->unplug || cpu_can_run(cpu));
qemu_kvm_destroy_vcpu(cpu);
+ cpu->created = false;
+ qemu_cond_signal(&qemu_cpu_cond);
qemu_mutex_unlock_iothread();
return NULL;
}
@@ -1168,6 +1170,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
}
if (remove_cpu) {
qemu_tcg_destroy_vcpu(remove_cpu);
+ cpu->created = false;
+ qemu_cond_signal(&qemu_cpu_cond);
remove_cpu = NULL;
}
}
@@ -1333,6 +1337,14 @@ void cpu_remove(CPUState *cpu)
qemu_cpu_kick(cpu);
}
+void cpu_remove_sync(CPUState *cpu)
+{
+ cpu_remove(cpu);
+ while (cpu->created) {
+ qemu_cond_wait(&qemu_cpu_cond, &qemu_global_mutex);
+ }
+}
+
/* For temporary buffers for forming a name */
#define VCPU_THREAD_NAME_SIZE 16
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 0720dd7..6e20119 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -765,6 +765,14 @@ void cpu_resume(CPUState *cpu);
*/
void cpu_remove(CPUState *cpu);
+ /**
+ * cpu_remove_sync:
+ * @cpu: The CPU to remove.
+ *
+ * Requests the CPU to be removed and waits till it is removed.
+ */
+void cpu_remove_sync(CPUState *cpu);
+
/**
* qemu_init_vcpu:
* @cpu: The vCPU to initialize.
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (3 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 04/12] cpu: Add a sync version of cpu_remove() Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-01 3:30 ` David Gibson
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 06/12] cpu: Abstract CPU core type Bharata B Rao
` (7 subsequent siblings)
12 siblings, 1 reply; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
From: Igor Mammedov <imammedo@redhat.com>
pre_plug callback is to be called before device.realize() is executed.
This would allow to check/set device's properties from HotplugHandler.
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/core/hotplug.c | 11 +++++++++++
hw/core/qdev.c | 9 ++++++++-
include/hw/hotplug.h | 14 +++++++++++++-
3 files changed, 32 insertions(+), 2 deletions(-)
diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c
index 645cfca..17ac986 100644
--- a/hw/core/hotplug.c
+++ b/hw/core/hotplug.c
@@ -13,6 +13,17 @@
#include "hw/hotplug.h"
#include "qemu/module.h"
+void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
+ DeviceState *plugged_dev,
+ Error **errp)
+{
+ HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler);
+
+ if (hdc->pre_plug) {
+ hdc->pre_plug(plug_handler, plugged_dev, errp);
+ }
+}
+
void hotplug_handler_plug(HotplugHandler *plug_handler,
DeviceState *plugged_dev,
Error **errp)
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index db41aa1..a0b3aad 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -1062,6 +1062,14 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
g_free(name);
}
+ hotplug_ctrl = qdev_get_hotplug_handler(dev);
+ if (hotplug_ctrl) {
+ hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err);
+ if (local_err != NULL) {
+ goto fail;
+ }
+ }
+
if (dc->realize) {
dc->realize(dev, &local_err);
}
@@ -1072,7 +1080,6 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
DEVICE_LISTENER_CALL(realize, Forward, dev);
- hotplug_ctrl = qdev_get_hotplug_handler(dev);
if (hotplug_ctrl) {
hotplug_handler_plug(hotplug_ctrl, dev, &local_err);
}
diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h
index 2db025d..50d84e9 100644
--- a/include/hw/hotplug.h
+++ b/include/hw/hotplug.h
@@ -46,7 +46,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler,
* hardware (un)plug functions.
*
* @parent: Opaque parent interface.
- * @plug: plug callback.
+ * @pre_plug: pre plug callback called at start of device.realize(true)
+ * @plug: plug callback called at end of device.realize(true).
* @unplug_request: unplug request callback.
* Used as a means to initiate device unplug for devices that
* require asynchronous unplug handling.
@@ -59,6 +60,7 @@ typedef struct HotplugHandlerClass {
InterfaceClass parent;
/* <public> */
+ hotplug_fn pre_plug;
hotplug_fn plug;
hotplug_fn unplug_request;
hotplug_fn unplug;
@@ -74,6 +76,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler,
Error **errp);
/**
+ * hotplug_handler_pre_plug:
+ *
+ * Call #HotplugHandlerClass.pre_plug callback of @plug_handler.
+ */
+void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
+ DeviceState *plugged_dev,
+ Error **errp);
+
+
+/**
* hotplug_handler_unplug_request:
*
* Calls #HotplugHandlerClass.unplug_request callback of @plug_handler.
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 06/12] cpu: Abstract CPU core type
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (4 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 07/12] spapr: Abstract CPU core device Bharata B Rao
` (6 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Add an abstract CPU core type that could be used by machines that want
to define and hotplug CPUs in core granularity.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
[Integer core property]
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
hw/cpu/Makefile.objs | 1 +
hw/cpu/core.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++
include/hw/cpu/core.h | 31 ++++++++++++++++++
3 files changed, 119 insertions(+)
create mode 100644 hw/cpu/core.c
create mode 100644 include/hw/cpu/core.h
diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
index 0954a18..942a4bb 100644
--- a/hw/cpu/Makefile.objs
+++ b/hw/cpu/Makefile.objs
@@ -2,4 +2,5 @@ obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
obj-$(CONFIG_REALVIEW) += realview_mpcore.o
obj-$(CONFIG_A9MPCORE) += a9mpcore.o
obj-$(CONFIG_A15MPCORE) += a15mpcore.o
+obj-y += core.o
diff --git a/hw/cpu/core.c b/hw/cpu/core.c
new file mode 100644
index 0000000..3faf53d
--- /dev/null
+++ b/hw/cpu/core.c
@@ -0,0 +1,87 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+#include "qapi/visitor.h"
+#include "sysemu/cpus.h"
+
+static void core_prop_get_core(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CPUCore *core = CPU_CORE(obj);
+ int64_t value = core->core;
+
+ visit_type_int(v, name, &value, errp);
+}
+
+static void core_prop_set_core(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CPUCore *core = CPU_CORE(obj);
+ Error *local_err = NULL;
+ int64_t value;
+
+ visit_type_int(v, name, &value, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ core->core = value;
+}
+
+static void core_prop_get_threads(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CPUCore *core = CPU_CORE(obj);
+ int64_t value = core->threads;
+
+ visit_type_int(v, name, &value, errp);
+}
+
+static void core_prop_set_threads(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CPUCore *core = CPU_CORE(obj);
+ Error *local_err = NULL;
+ int64_t value;
+
+ visit_type_int(v, name, &value, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ core->threads = value;
+}
+
+static void cpu_core_instance_init(Object *obj)
+{
+ CPUCore *core = CPU_CORE(obj);
+
+ object_property_add(obj, "core", "int", core_prop_get_core,
+ core_prop_set_core, NULL, NULL, NULL);
+ object_property_add(obj, "threads", "int", core_prop_get_threads,
+ core_prop_set_threads, NULL, NULL, NULL);
+ core->threads = smp_threads;
+}
+
+static const TypeInfo cpu_core_type_info = {
+ .name = TYPE_CPU_CORE,
+ .parent = TYPE_DEVICE,
+ .abstract = true,
+ .instance_size = sizeof(CPUCore),
+ .instance_init = cpu_core_instance_init,
+};
+
+static void cpu_core_register_types(void)
+{
+ type_register_static(&cpu_core_type_info);
+}
+
+type_init(cpu_core_register_types)
diff --git a/include/hw/cpu/core.h b/include/hw/cpu/core.h
new file mode 100644
index 0000000..a2a5a04
--- /dev/null
+++ b/include/hw/cpu/core.h
@@ -0,0 +1,31 @@
+/*
+ * CPU core abstract device
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_CPU_CORE_H
+#define HW_CPU_CORE_H
+
+#include "qemu/osdep.h"
+#include "hw/qdev.h"
+
+#define TYPE_CPU_CORE "cpu-core"
+
+#define CPU_CORE(obj) \
+ OBJECT_CHECK(CPUCore, (obj), TYPE_CPU_CORE)
+
+typedef struct CPUCore {
+ /*< private >*/
+ DeviceState parent_obj;
+
+ /*< public >*/
+ int core;
+ int threads;
+} CPUCore;
+
+#define CPU_CORE_PROP_CORE "core"
+
+#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 07/12] spapr: Abstract CPU core device
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (5 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 06/12] cpu: Abstract CPU core type Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
` (5 subsequent siblings)
12 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Add sPAPR specific abastract CPU core device that is based on generic
CPU core device. This will serve as base type for all PowerPC CPU
specific core devices.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/Makefile.objs | 1 +
hw/ppc/spapr_cpu_core.c | 30 ++++++++++++++++++++++++++++++
include/hw/ppc/spapr_cpu_core.h | 27 +++++++++++++++++++++++++++
3 files changed, 58 insertions(+)
create mode 100644 hw/ppc/spapr_cpu_core.c
create mode 100644 include/hw/ppc/spapr_cpu_core.h
diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs
index c1ffc77..5cc6608 100644
--- a/hw/ppc/Makefile.objs
+++ b/hw/ppc/Makefile.objs
@@ -4,6 +4,7 @@ obj-y += ppc.o ppc_booke.o
obj-$(CONFIG_PSERIES) += spapr.o spapr_vio.o spapr_events.o
obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o
obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o
+obj-$(CONFIG_PSERIES) += spapr_cpu_core.o
ifeq ($(CONFIG_PCI)$(CONFIG_PSERIES)$(CONFIG_LINUX), yyy)
obj-y += spapr_pci_vfio.o
endif
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
new file mode 100644
index 0000000..8cbe2a5
--- /dev/null
+++ b/hw/ppc/spapr_cpu_core.c
@@ -0,0 +1,30 @@
+/*
+ * sPAPR CPU core device, acts as container of CPU thread devices.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/cpu/core.h"
+#include "hw/ppc/spapr_cpu_core.h"
+#include "hw/ppc/spapr.h"
+#include "hw/boards.h"
+#include "qemu/error-report.h"
+#include "qapi/visitor.h"
+#include <sysemu/cpus.h>
+#include "target-ppc/kvm_ppc.h"
+
+static const TypeInfo spapr_cpu_core_type_info = {
+ .name = TYPE_SPAPR_CPU_CORE,
+ .parent = TYPE_CPU_CORE,
+ .abstract = true,
+ .instance_size = sizeof(sPAPRCPUCore),
+};
+
+static void spapr_cpu_core_register_types(void)
+{
+ type_register_static(&spapr_cpu_core_type_info);
+}
+
+type_init(spapr_cpu_core_register_types)
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
new file mode 100644
index 0000000..e3340ea
--- /dev/null
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -0,0 +1,27 @@
+/*
+ * sPAPR CPU core device.
+ *
+ * Copyright (C) 2016 Bharata B Rao <bharata@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef HW_SPAPR_CPU_CORE_H
+#define HW_SPAPR_CPU_CORE_H
+
+#include "hw/qdev.h"
+#include "hw/cpu/core.h"
+
+#define TYPE_SPAPR_CPU_CORE "spapr-cpu-core"
+#define SPAPR_CPU_CORE(obj) \
+ OBJECT_CHECK(sPAPRCPUCore, (obj), TYPE_SPAPR_CPU_CORE)
+
+typedef struct sPAPRCPUCore {
+ /*< private >*/
+ CPUCore parent_obj;
+
+ /*< public >*/
+ PowerPCCPU *threads;
+} sPAPRCPUCore;
+
+#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (6 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 07/12] spapr: Abstract CPU core device Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-01 5:08 ` David Gibson
` (2 more replies)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU " Bharata B Rao
` (4 subsequent siblings)
12 siblings, 3 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Introduce core devices for each CPU type supported by sPAPR. These
core devices are derived from the base spapr-cpu-core device type.
TODO:
- Add core types for other remaining CPU types
- Handle CPU model alias correctly
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 3 +-
hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
include/hw/ppc/spapr.h | 1 +
include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
4 files changed, 156 insertions(+), 2 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 64c4acc..45ac5dc 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
machine->boot_order = g_strdup(boot_device);
}
-static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
- Error **errp)
+void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
{
CPUPPCState *env = &cpu->env;
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 8cbe2a5..3751a54 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
.instance_size = sizeof(sPAPRCPUCore),
};
+#define DEFINE_SPAPR_CPU_CORE(_name) \
+static void \
+glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
+ Error **errp) \
+{ \
+ int i; \
+ Error *local_err = NULL; \
+ sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
+ glue(_name, sPAPRCPUCore) * core = \
+ glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
+ \
+ for (i = 0; i < threads; i++) { \
+ char id[32]; \
+ \
+ object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
+ object_class_get_name(core->cpu)); \
+ snprintf(id, sizeof(id), "thread[%d]", i); \
+ object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
+ &local_err); \
+ if (local_err) { \
+ goto err; \
+ } \
+ } \
+ return; \
+ \
+err: \
+ while (--i) { \
+ object_unparent(OBJECT(&sc->threads[i])); \
+ } \
+ error_propagate(errp, local_err); \
+} \
+ \
+static int \
+glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
+{ \
+ Error **errp = opaque; \
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
+ CPUState *cs = CPU(child); \
+ PowerPCCPU *cpu = POWERPC_CPU(cs); \
+ \
+ object_property_set_bool(child, true, "realized", errp); \
+ if (*errp) { \
+ return 1; \
+ } \
+ \
+ spapr_cpu_init(spapr, cpu, errp); \
+ if (*errp) { \
+ return 1; \
+ } \
+ return 0; \
+} \
+ \
+static void \
+glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
+{ \
+ sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
+ CPUCore *cc = CPU_CORE(OBJECT(dev)); \
+ Error *local_err = NULL; \
+ \
+ /* \
+ * TODO: This is CPU model specific CPU core's realize routine. \
+ * However I am initializing "threads" field of the parent type \
+ * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
+ * part of CPU model specific CPU core type and have different plug() \
+ * handlers for each type instead of having a common plug() handler \
+ * for all core types. \
+ */ \
+ sc->threads = g_new0(PowerPCCPU, cc->threads); \
+ glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
+ if (local_err) { \
+ goto out; \
+ } \
+ \
+ object_child_foreach(OBJECT(dev), \
+ glue(_name, _spapr_cpu_core_realize_child), \
+ &local_err); \
+ \
+out: \
+ if (local_err) { \
+ g_free(sc->threads); \
+ error_propagate(errp, local_err); \
+ } \
+} \
+ \
+static void \
+glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
+{ \
+ glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
+ const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
+ ObjectClass *oc = object_class_by_name(type); \
+ \
+ core->cpu = oc; \
+} \
+ \
+static void \
+glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
+{ \
+ \
+ DeviceClass *dc = DEVICE_CLASS(oc); \
+ dc->realize = glue(_name, _spapr_cpu_core_realize); \
+} \
+ \
+static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
+{ \
+ .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
+ .parent = TYPE_SPAPR_CPU_CORE, \
+ .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
+ .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
+ .class_init = glue(_name, _spapr_cpu_core_class_init), \
+};
+
+DEFINE_SPAPR_CPU_CORE(host);
+DEFINE_SPAPR_CPU_CORE(POWER7);
+DEFINE_SPAPR_CPU_CORE(POWER8);
+
static void spapr_cpu_core_register_types(void)
{
type_register_static(&spapr_cpu_core_type_info);
+ type_register_static(&host_spapr_cpu_core_type_info);
+ type_register_static(&POWER7_spapr_cpu_core_type_info);
+ type_register_static(&POWER8_spapr_cpu_core_type_info);
}
type_init(spapr_cpu_core_register_types)
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 098d85d..0fdf448 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
uint32_t count);
void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
uint32_t count);
+void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
/* rtas-configure-connector state */
struct sPAPRConfigureConnectorState {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index e3340ea..71e69c0 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
PowerPCCPU *threads;
} sPAPRCPUCore;
+#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
+#define host_SPAPR_CPU_CORE(obj) \
+ OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
+
+typedef struct hostsPAPRCPUCore {
+ /*< private >*/
+ sPAPRCPUCore parent_obj;
+
+ /*< public >*/
+ ObjectClass *cpu;
+} hostsPAPRCPUCore;
+
+#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
+#define POWER7_SPAPR_CPU_CORE(obj) \
+ OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
+
+typedef struct POWER7sPAPRCPUCore {
+ /*< private >*/
+ sPAPRCPUCore parent_obj;
+
+ /*< public >*/
+ ObjectClass *cpu;
+} POWER7sPAPRCPUCore;
+
+#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
+#define POWER8_SPAPR_CPU_CORE(obj) \
+ OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
+
+typedef struct POWER8sPAPRCPUCore {
+ /*< private >*/
+ sPAPRCPUCore parent_obj;
+
+ /*< public >*/
+ ObjectClass *cpu;
+} POWER8sPAPRCPUCore;
+
#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU core devices
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (7 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-01 5:12 ` David Gibson
2016-04-08 23:35 ` Michael Roth
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
` (3 subsequent siblings)
12 siblings, 2 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Introduce sPAPRMachineClass.dr_cpu_enabled to indicate support for
CPU core hotplug. Initialize boot time CPUs as core deivces and prevent
topologies that result in partially filled cores. Both of these are done
only if CPU core hotplug is supported.
Note: An unrelated change in the call to xics_system_init() is done
in this patch as it makes sense to use the local variable smt introduced
in this patch instead of kvmppc_smt_threads() call here.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 73 +++++++++++++++++++++++++++++++++++------
hw/ppc/spapr_cpu_core.c | 45 +++++++++++++++++++++++++
include/hw/ppc/spapr.h | 2 ++
include/hw/ppc/spapr_cpu_core.h | 3 ++
4 files changed, 113 insertions(+), 10 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 45ac5dc..1ead043 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -64,6 +64,7 @@
#include "hw/compat.h"
#include "qemu-common.h"
+#include "hw/ppc/spapr_cpu_core.h"
#include <libfdt.h>
@@ -1614,6 +1615,10 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
machine->boot_order = g_strdup(boot_device);
}
+/*
+ * TODO: Check if some of these can be moved to rtas_start_cpu() where
+ * a few other things required for hotplugged CPUs are being done.
+ */
void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
{
CPUPPCState *env = &cpu->env;
@@ -1644,6 +1649,7 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
xics_cpu_setup(spapr->icp, cpu);
qemu_register_reset(spapr_cpu_reset, cpu);
+ spapr_cpu_reset(cpu);
}
/*
@@ -1727,7 +1733,6 @@ static void ppc_spapr_init(MachineState *machine)
const char *kernel_filename = machine->kernel_filename;
const char *kernel_cmdline = machine->kernel_cmdline;
const char *initrd_filename = machine->initrd_filename;
- PowerPCCPU *cpu;
PCIHostState *phb;
int i;
MemoryRegion *sysmem = get_system_memory();
@@ -1741,6 +1746,22 @@ static void ppc_spapr_init(MachineState *machine)
long load_limit, fw_size;
bool kernel_le = false;
char *filename;
+ int smt = kvmppc_smt_threads();
+ int spapr_cores = smp_cpus / smp_threads;
+ int spapr_max_cores = max_cpus / smp_threads;
+
+ if (smc->dr_cpu_enabled) {
+ if (smp_cpus % smp_threads) {
+ error_report("smp_cpus (%u) must be multiple of threads (%u)",
+ smp_cpus, smp_threads);
+ exit(1);
+ }
+ if (max_cpus % smp_threads) {
+ error_report("max_cpus (%u) must be multiple of threads (%u)",
+ max_cpus, smp_threads);
+ exit(1);
+ }
+ }
msi_supported = true;
@@ -1787,8 +1808,7 @@ static void ppc_spapr_init(MachineState *machine)
/* Set up Interrupt Controller before we create the VCPUs */
spapr->icp = xics_system_init(machine,
- DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(),
- smp_threads),
+ DIV_ROUND_UP(max_cpus * smt, smp_threads),
XICS_IRQS, &error_fatal);
if (smc->dr_lmb_enabled) {
@@ -1799,13 +1819,34 @@ static void ppc_spapr_init(MachineState *machine)
if (machine->cpu_model == NULL) {
machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
}
- for (i = 0; i < smp_cpus; i++) {
- cpu = cpu_ppc_init(machine->cpu_model);
- if (cpu == NULL) {
- error_report("Unable to find PowerPC CPU definition");
- exit(1);
+
+ if (smc->dr_cpu_enabled) {
+ spapr->cores = g_new0(Object *, spapr_max_cores);
+
+ for (i = 0; i < spapr_max_cores; i++) {
+ int core_dt_id = i * smt;
+
+ if (i < spapr_cores) {
+ char *type = spapr_get_cpu_core_type(machine->cpu_model);
+ Object *core = object_new(type);
+
+ g_free(type);
+ object_property_set_int(core, smp_threads, "threads",
+ &error_fatal);
+ object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE,
+ &error_fatal);
+ object_property_set_bool(core, true, "realized", &error_fatal);
+ }
}
- spapr_cpu_init(spapr, cpu, &error_fatal);
+ } else {
+ for (i = 0; i < smp_cpus; i++) {
+ PowerPCCPU *cpu = cpu_ppc_init(machine->cpu_model);
+ if (cpu == NULL) {
+ error_report("Unable to find PowerPC CPU definition");
+ exit(1);
+ }
+ spapr_cpu_init(spapr, cpu, &error_fatal);
+ }
}
if (kvm_enabled()) {
@@ -2257,10 +2298,19 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
}
}
+static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
+ DeviceState *dev, Error **errp)
+{
+ if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+ spapr_core_pre_plug(hotplug_dev, dev, errp);
+ }
+}
+
static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
DeviceState *dev)
{
- if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
+ if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
+ object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
return HOTPLUG_HANDLER(machine);
}
return NULL;
@@ -2299,11 +2349,13 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
mc->has_dynamic_sysbus = true;
mc->pci_allow_0_address = true;
mc->get_hotplug_handler = spapr_get_hotpug_handler;
+ hc->pre_plug = spapr_machine_device_pre_plug;
hc->plug = spapr_machine_device_plug;
hc->unplug = spapr_machine_device_unplug;
mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
smc->dr_lmb_enabled = true;
+ smc->dr_cpu_enabled = true;
fwc->get_dev_path = spapr_get_fw_dev_path;
nc->nmi_monitor_handler = spapr_nmi;
}
@@ -2383,6 +2435,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
spapr_machine_2_6_class_options(mc);
smc->use_ohci_by_default = true;
+ smc->dr_cpu_enabled = false;
SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
}
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 3751a54..640d143 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -15,6 +15,40 @@
#include <sysemu/cpus.h>
#include "target-ppc/kvm_ppc.h"
+void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp)
+{
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+ int spapr_max_cores = max_cpus / smp_threads;
+ int index;
+ int smt = kvmppc_smt_threads();
+ Error *local_err = NULL;
+ CPUCore *cc = CPU_CORE(dev);
+
+ if (cc->threads != smp_threads) {
+ error_setg(&local_err, "threads must be %d", smp_threads);
+ goto out;
+ }
+
+ if (cc->core % smt) {
+ error_setg(&local_err, "invalid core id %d\n", cc->core);
+ goto out;
+ }
+
+ index = cc->core / smt;
+ if (index < 0 || index >= spapr_max_cores) {
+ error_setg(&local_err, "core id %d out of range", cc->core);
+ goto out;
+ }
+
+ if (spapr->cores[index]) {
+ error_setg(&local_err, "core %d already populated", cc->core);
+ }
+
+out:
+ error_propagate(errp, local_err);
+}
+
static const TypeInfo spapr_cpu_core_type_info = {
.name = TYPE_SPAPR_CPU_CORE,
.parent = TYPE_CPU_CORE,
@@ -146,3 +180,14 @@ static void spapr_cpu_core_register_types(void)
}
type_init(spapr_cpu_core_register_types)
+
+/*
+ * TODO: Looks fragile :(
+ */
+char *spapr_get_cpu_core_type(const char *model)
+{
+ char core_type[32];
+
+ snprintf(core_type, 32, "%s-%s", model, TYPE_SPAPR_CPU_CORE);
+ return g_strdup(core_type);
+}
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 0fdf448..a6956c0 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -36,6 +36,7 @@ struct sPAPRMachineClass {
/*< public >*/
bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */
+ bool dr_cpu_enabled; /* enable dynamic-reconfig/hotplug of CPUs */
bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */
};
@@ -79,6 +80,7 @@ struct sPAPRMachineState {
/*< public >*/
char *kvm_type;
MemoryHotplugState hotplug_memory;
+ Object **cores;
};
#define H_SUCCESS 0
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 71e69c0..f08f291 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -60,4 +60,7 @@ typedef struct POWER8sPAPRCPUCore {
ObjectClass *cpu;
} POWER8sPAPRCPUCore;
+void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp);
+char *spapr_get_cpu_core_type(const char *model);
#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (8 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU " Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-04 4:23 ` David Gibson
` (2 more replies)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly Bharata B Rao
` (2 subsequent siblings)
12 siblings, 3 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Set up device tree entries for the hotplugged CPU core and use the
exising RTAS event logging infrastructure to send CPU hotplug notification
to the guest.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 58 ++++++++++++++++++++++++++++++++++
hw/ppc/spapr_cpu_core.c | 70 +++++++++++++++++++++++++++++++++++++++++
hw/ppc/spapr_events.c | 3 ++
hw/ppc/spapr_rtas.c | 24 ++++++++++++++
include/hw/ppc/spapr.h | 2 ++
include/hw/ppc/spapr_cpu_core.h | 2 ++
6 files changed, 159 insertions(+)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1ead043..1a5dbd9 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -603,6 +603,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
size_t page_sizes_prop_size;
uint32_t vcpus_per_socket = smp_threads * smp_cores;
uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
+ sPAPRDRConnector *drc;
+ sPAPRDRConnectorClass *drck;
+ int drc_index;
+
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
+ if (drc) {
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drc_index = drck->get_index(drc);
+ _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
+ }
/* Note: we keep CI large pages off for now because a 64K capable guest
* provisioned with large pages might otherwise try to map a qemu
@@ -987,6 +997,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
_FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
}
+ if (smc->dr_cpu_enabled) {
+ int offset = fdt_path_offset(fdt, "/cpus");
+ ret = spapr_drc_populate_dt(fdt, offset, NULL,
+ SPAPR_DR_CONNECTOR_TYPE_CPU);
+ if (ret < 0) {
+ error_report("Couldn't set up CPU DR device tree properties");
+ exit(1);
+ }
+ }
+
_FDT((fdt_pack(fdt)));
if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
@@ -1622,6 +1642,8 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
{
CPUPPCState *env = &cpu->env;
+ CPUState *cs = CPU(cpu);
+ int i;
/* Set time-base frequency to 512 MHz */
cpu_ppc_tb_init(env, TIMEBASE_FREQ);
@@ -1646,6 +1668,14 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
}
}
+ /* Set NUMA node for the added CPUs */
+ for (i = 0; i < nb_numa_nodes; i++) {
+ if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
+ cs->numa_node = i;
+ break;
+ }
+ }
+
xics_cpu_setup(spapr->icp, cpu);
qemu_register_reset(spapr_cpu_reset, cpu);
@@ -1825,6 +1855,11 @@ static void ppc_spapr_init(MachineState *machine)
for (i = 0; i < spapr_max_cores; i++) {
int core_dt_id = i * smt;
+ sPAPRDRConnector *drc =
+ spapr_dr_connector_new(OBJECT(spapr),
+ SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
+
+ qemu_register_reset(spapr_drc_reset, drc);
if (i < spapr_cores) {
char *type = spapr_get_cpu_core_type(machine->cpu_model);
@@ -2247,6 +2282,27 @@ out:
error_propagate(errp, local_err);
}
+void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
+ sPAPRMachineState *spapr)
+{
+ PowerPCCPU *cpu = POWERPC_CPU(cs);
+ DeviceClass *dc = DEVICE_GET_CLASS(cs);
+ int id = ppc_get_vcpu_dt_id(cpu);
+ void *fdt;
+ int offset, fdt_size;
+ char *nodename;
+
+ fdt = create_device_tree(&fdt_size);
+ nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
+ offset = fdt_add_subnode(fdt, 0, nodename);
+
+ spapr_populate_cpu_dt(cs, fdt, offset, spapr);
+ g_free(nodename);
+
+ *fdt_offset = offset;
+ return fdt;
+}
+
static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
DeviceState *dev, Error **errp)
{
@@ -2287,6 +2343,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
}
spapr_memory_plug(hotplug_dev, dev, node, errp);
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+ spapr_core_plug(hotplug_dev, dev, errp);
}
}
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 640d143..a9ba843 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -18,6 +18,7 @@
void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
Error **errp)
{
+ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
int spapr_max_cores = max_cpus / smp_threads;
int index;
@@ -25,6 +26,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
Error *local_err = NULL;
CPUCore *cc = CPU_CORE(dev);
+ if (!smc->dr_cpu_enabled && dev->hotplugged) {
+ error_setg(&local_err, "CPU hotplug not supported for this machine");
+ goto out;
+ }
+
if (cc->threads != smp_threads) {
error_setg(&local_err, "threads must be %d", smp_threads);
goto out;
@@ -49,6 +55,70 @@ out:
error_propagate(errp, local_err);
}
+void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp)
+{
+ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+ CPUCore *cc = CPU_CORE(dev);
+ CPUState *cs = CPU(&core->threads[0]);
+ sPAPRDRConnector *drc;
+ sPAPRDRConnectorClass *drck;
+ Error *local_err = NULL;
+ void *fdt = NULL;
+ int fdt_offset = 0;
+ int index;
+ int smt = kvmppc_smt_threads();
+
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core);
+ index = cc->core / smt;
+ spapr->cores[index] = OBJECT(dev);
+
+ if (!smc->dr_cpu_enabled) {
+ /*
+ * This is a cold plugged CPU core but the machine doesn't support
+ * DR. So skip the hotplug path ensuring that the core is brought
+ * up online with out an associated DR connector.
+ */
+ return;
+ }
+
+ g_assert(drc);
+
+ /*
+ * Setup CPU DT entries only for hotplugged CPUs. For boot time or
+ * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
+ */
+ if (dev->hotplugged) {
+ fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
+ dev->hotplugged = true;
+ }
+
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
+ if (local_err) {
+ g_free(fdt);
+ spapr->cores[index] = NULL;
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ if (dev->hotplugged) {
+ /*
+ * Send hotplug notification interrupt to the guest only in case
+ * of hotplugged CPUs.
+ */
+ spapr_hotplug_req_add_by_index(drc);
+ } else {
+ /*
+ * Set the right DRC states for cold plugged CPU.
+ */
+ drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
+ drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
+ }
+}
+
static const TypeInfo spapr_cpu_core_type_info = {
.name = TYPE_SPAPR_CPU_CORE,
.parent = TYPE_CPU_CORE,
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 39f4682..10340e1 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
case SPAPR_DR_CONNECTOR_TYPE_LMB:
hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
break;
+ case SPAPR_DR_CONNECTOR_TYPE_CPU:
+ hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
+ break;
default:
/* we shouldn't be signaling hotplug events for resources
* that don't support them
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index b7c5ebd..cc0369e 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -34,6 +34,7 @@
#include "hw/ppc/spapr.h"
#include "hw/ppc/spapr_vio.h"
+#include "hw/ppc/ppc.h"
#include "qapi-event.h"
#include "hw/boards.h"
@@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
}
+/*
+ * Set the timebase offset of the CPU to that of first CPU.
+ * This helps hotplugged CPU to have the correct timebase offset.
+ */
+static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
+{
+ PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+
+ cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
+}
+
+static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
+{
+ PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
+ PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
+
+ if (!pcc->interrupts_big_endian(fcpu)) {
+ cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
+ }
+}
+
static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
uint32_t token, uint32_t nargs,
target_ulong args,
@@ -197,6 +219,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
env->nip = start;
env->gpr[3] = r3;
cs->halted = 0;
+ spapr_cpu_set_endianness(cpu);
+ spapr_cpu_update_tb_offset(cpu);
qemu_cpu_kick(cs);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index a6956c0..619db98 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -588,6 +588,8 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
uint32_t count);
void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
+void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
+ sPAPRMachineState *spapr);
/* rtas-configure-connector state */
struct sPAPRConfigureConnectorState {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index f08f291..165af7c 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -63,4 +63,6 @@ typedef struct POWER8sPAPRCPUCore {
void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
Error **errp);
char *spapr_get_cpu_core_type(const char *model);
+void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp);
#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (9 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-06 0:24 ` Michael Roth
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support Bharata B Rao
2016-04-04 14:44 ` [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Igor Mammedov
12 siblings, 1 reply; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
XICS is setup for each CPU during initialization. Provide a routine
to undo the same when CPU is unplugged. While here, move ss->cs management
into xics from xics_kvm since there is nothing KVM specific in it.
Also ensure xics reset doesn't set irq for CPUs that are already unplugged.
This allows reboot of a VM that has undergone CPU hotplug and unplug
to work correctly.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
hw/intc/xics.c | 14 ++++++++++++++
hw/intc/xics_kvm.c | 8 ++++----
include/hw/ppc/xics.h | 1 +
3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/hw/intc/xics.c b/hw/intc/xics.c
index 213a370..9fdb551 100644
--- a/hw/intc/xics.c
+++ b/hw/intc/xics.c
@@ -45,6 +45,18 @@ static int get_cpu_index_by_dt_id(int cpu_dt_id)
return -1;
}
+void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu)
+{
+ CPUState *cs = CPU(cpu);
+ ICPState *ss = &icp->ss[cs->cpu_index];
+
+ assert(cs->cpu_index < icp->nr_servers);
+ assert(cs == ss->cs);
+
+ ss->output = NULL;
+ ss->cs = NULL;
+}
+
void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
{
CPUState *cs = CPU(cpu);
@@ -54,6 +66,8 @@ void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
assert(cs->cpu_index < icp->nr_servers);
+ ss->cs = cs;
+
if (info->cpu_setup) {
info->cpu_setup(icp, cpu);
}
diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c
index 9fe0667..7aab4a1 100644
--- a/hw/intc/xics_kvm.c
+++ b/hw/intc/xics_kvm.c
@@ -110,8 +110,10 @@ static void icp_kvm_reset(DeviceState *dev)
icp->pending_priority = 0xff;
icp->mfrr = 0xff;
- /* Make all outputs are deasserted */
- qemu_set_irq(icp->output, 0);
+ /* Make all outputs as deasserted only if the CPU thread is in use */
+ if (icp->output) {
+ qemu_set_irq(icp->output, 0);
+ }
icp_set_kvm_state(icp, 1);
}
@@ -344,8 +346,6 @@ static void xics_kvm_cpu_setup(XICSState *icp, PowerPCCPU *cpu)
if (icpkvm->kernel_xics_fd != -1) {
int ret;
- ss->cs = cs;
-
ret = kvm_vcpu_enable_cap(cs, KVM_CAP_IRQ_XICS, 0,
icpkvm->kernel_xics_fd, kvm_arch_vcpu_id(cs));
if (ret < 0) {
diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
index f60b06a..9091054 100644
--- a/include/hw/ppc/xics.h
+++ b/include/hw/ppc/xics.h
@@ -167,5 +167,6 @@ int xics_alloc_block(XICSState *icp, int src, int num, bool lsi, bool align,
void xics_free(XICSState *icp, int irq, int num);
void xics_cpu_setup(XICSState *icp, PowerPCCPU *cpu);
+void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu);
#endif /* __XICS_H__ */
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (10 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly Bharata B Rao
@ 2016-03-31 8:39 ` Bharata B Rao
2016-04-04 4:27 ` David Gibson
2016-04-04 14:44 ` [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Igor Mammedov
12 siblings, 1 reply; 40+ messages in thread
From: Bharata B Rao @ 2016-03-31 8:39 UTC (permalink / raw)
To: qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, Bharata B Rao, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, imammedo, mdroth,
afaerber, david
Remove the CPU core device by removing the underlying CPU thread devices.
Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
notification to the guest. Release the vCPU object after CPU hot unplug so
that vCPU fd can be parked and reused.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 16 ++++++++
hw/ppc/spapr_cpu_core.c | 86 +++++++++++++++++++++++++++++++++++++++++
include/hw/ppc/spapr.h | 1 +
include/hw/ppc/spapr_cpu_core.h | 11 ++++++
4 files changed, 114 insertions(+)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1a5dbd9..74cdcf2 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2348,11 +2348,27 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
}
}
+void spapr_cpu_destroy(PowerPCCPU *cpu)
+{
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+
+ xics_cpu_destroy(spapr->icp, cpu);
+ qemu_unregister_reset(spapr_cpu_reset, cpu);
+}
+
static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
DeviceState *dev, Error **errp)
{
+ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
+
if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
error_setg(errp, "Memory hot unplug not supported by sPAPR");
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+ if (!smc->dr_cpu_enabled) {
+ error_setg(errp, "CPU hot unplug not supported on this machine");
+ return;
+ }
+ spapr_core_unplug(hotplug_dev, dev, errp);
}
}
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index a9ba843..09a592e 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -119,6 +119,92 @@ void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
}
}
+static void spapr_cpu_core_cleanup(struct sPAPRCPUUnplugList *unplug_list)
+{
+ sPAPRCPUUnplug *unplug, *next;
+ Object *cpu;
+
+ QLIST_FOREACH_SAFE(unplug, unplug_list, node, next) {
+ cpu = unplug->cpu;
+ object_unparent(cpu);
+ QLIST_REMOVE(unplug, node);
+ g_free(unplug);
+ }
+}
+
+static void spapr_add_cpu_to_unplug_list(Object *cpu,
+ struct sPAPRCPUUnplugList *unplug_list)
+{
+ sPAPRCPUUnplug *unplug = g_malloc(sizeof(*unplug));
+
+ unplug->cpu = cpu;
+ QLIST_INSERT_HEAD(unplug_list, unplug, node);
+}
+
+static int spapr_cpu_release(Object *obj, void *opaque)
+{
+ DeviceState *dev = DEVICE(obj);
+ CPUState *cs = CPU(dev);
+ PowerPCCPU *cpu = POWERPC_CPU(cs);
+ struct sPAPRCPUUnplugList *unplug_list = opaque;
+
+ spapr_cpu_destroy(cpu);
+ cpu_remove_sync(cs);
+
+ /*
+ * We are still walking the core object's children list, and
+ * hence can't cleanup this CPU thread object just yet. Put
+ * it on a list for later removal.
+ */
+ spapr_add_cpu_to_unplug_list(obj, unplug_list);
+ return 0;
+}
+
+static void spapr_core_release(DeviceState *dev)
+{
+ struct sPAPRCPUUnplugList unplug_list;
+ sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+ CPUCore *cc = CPU_CORE(dev);
+ int smt = kvmppc_smt_threads();
+
+ QLIST_INIT(&unplug_list);
+ object_child_foreach(OBJECT(dev), spapr_cpu_release, &unplug_list);
+ spapr_cpu_core_cleanup(&unplug_list);
+ spapr->cores[cc->core / smt] = NULL;
+
+ g_free(core->threads);
+}
+
+static void spapr_core_release_unparent(DeviceState *dev, void *opaque)
+{
+ spapr_core_release(dev);
+ object_unparent(OBJECT(dev));
+}
+
+void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp)
+{
+ sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
+ PowerPCCPU *cpu = &core->threads[0];
+ int id = ppc_get_vcpu_dt_id(cpu);
+ sPAPRDRConnector *drc =
+ spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
+ sPAPRDRConnectorClass *drck;
+ Error *local_err = NULL;
+
+ g_assert(drc);
+
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drck->detach(drc, dev, spapr_core_release_unparent, NULL, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ spapr_hotplug_req_remove_by_index(drc);
+}
+
static const TypeInfo spapr_cpu_core_type_info = {
.name = TYPE_SPAPR_CPU_CORE,
.parent = TYPE_CPU_CORE,
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 619db98..c5a4a15 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -590,6 +590,7 @@ void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
sPAPRMachineState *spapr);
+void spapr_cpu_destroy(PowerPCCPU *cpu);
/* rtas-configure-connector state */
struct sPAPRConfigureConnectorState {
diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
index 165af7c..9bc2e01 100644
--- a/include/hw/ppc/spapr_cpu_core.h
+++ b/include/hw/ppc/spapr_cpu_core.h
@@ -65,4 +65,15 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
char *spapr_get_cpu_core_type(const char *model);
void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
Error **errp);
+void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp);
+
+/* List to store unplugged CPU objects for cleanup during unplug */
+typedef struct sPAPRCPUUnplug {
+ Object *cpu;
+ QLIST_ENTRY(sPAPRCPUUnplug) node;
+} sPAPRCPUUnplug;
+
+QLIST_HEAD(sPAPRCPUUnplugList, sPAPRCPUUnplug);
+
#endif
--
2.1.0
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback Bharata B Rao
@ 2016-04-01 3:30 ` David Gibson
2016-04-01 10:38 ` Paolo Bonzini
0 siblings, 1 reply; 40+ messages in thread
From: David Gibson @ 2016-04-01 3:30 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 4116 bytes --]
On Thu, Mar 31, 2016 at 02:09:14PM +0530, Bharata B Rao wrote:
> From: Igor Mammedov <imammedo@redhat.com>
>
> pre_plug callback is to be called before device.realize() is executed.
> This would allow to check/set device's properties from HotplugHandler.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
It would be really nice to get some opinion on this from Andreas or
Paolo.
> ---
> hw/core/hotplug.c | 11 +++++++++++
> hw/core/qdev.c | 9 ++++++++-
> include/hw/hotplug.h | 14 +++++++++++++-
> 3 files changed, 32 insertions(+), 2 deletions(-)
>
> diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c
> index 645cfca..17ac986 100644
> --- a/hw/core/hotplug.c
> +++ b/hw/core/hotplug.c
> @@ -13,6 +13,17 @@
> #include "hw/hotplug.h"
> #include "qemu/module.h"
>
> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
> + DeviceState *plugged_dev,
> + Error **errp)
> +{
> + HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler);
> +
> + if (hdc->pre_plug) {
> + hdc->pre_plug(plug_handler, plugged_dev, errp);
> + }
> +}
> +
> void hotplug_handler_plug(HotplugHandler *plug_handler,
> DeviceState *plugged_dev,
> Error **errp)
> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> index db41aa1..a0b3aad 100644
> --- a/hw/core/qdev.c
> +++ b/hw/core/qdev.c
> @@ -1062,6 +1062,14 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> g_free(name);
> }
>
> + hotplug_ctrl = qdev_get_hotplug_handler(dev);
> + if (hotplug_ctrl) {
> + hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err);
> + if (local_err != NULL) {
> + goto fail;
> + }
> + }
> +
> if (dc->realize) {
> dc->realize(dev, &local_err);
> }
> @@ -1072,7 +1080,6 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
>
> DEVICE_LISTENER_CALL(realize, Forward, dev);
>
> - hotplug_ctrl = qdev_get_hotplug_handler(dev);
> if (hotplug_ctrl) {
> hotplug_handler_plug(hotplug_ctrl, dev, &local_err);
> }
> diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h
> index 2db025d..50d84e9 100644
> --- a/include/hw/hotplug.h
> +++ b/include/hw/hotplug.h
> @@ -46,7 +46,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler,
> * hardware (un)plug functions.
> *
> * @parent: Opaque parent interface.
> - * @plug: plug callback.
> + * @pre_plug: pre plug callback called at start of device.realize(true)
> + * @plug: plug callback called at end of device.realize(true).
> * @unplug_request: unplug request callback.
> * Used as a means to initiate device unplug for devices that
> * require asynchronous unplug handling.
> @@ -59,6 +60,7 @@ typedef struct HotplugHandlerClass {
> InterfaceClass parent;
>
> /* <public> */
> + hotplug_fn pre_plug;
> hotplug_fn plug;
> hotplug_fn unplug_request;
> hotplug_fn unplug;
> @@ -74,6 +76,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler,
> Error **errp);
>
> /**
> + * hotplug_handler_pre_plug:
> + *
> + * Call #HotplugHandlerClass.pre_plug callback of @plug_handler.
> + */
> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
> + DeviceState *plugged_dev,
> + Error **errp);
> +
> +
> +/**
> * hotplug_handler_unplug_request:
> *
> * Calls #HotplugHandlerClass.unplug_request callback of @plug_handler.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
@ 2016-04-01 5:08 ` David Gibson
2016-04-01 6:12 ` Bharata B Rao
2016-04-04 0:16 ` David Gibson
2016-04-08 23:35 ` Michael Roth
2 siblings, 1 reply; 40+ messages in thread
From: David Gibson @ 2016-04-01 5:08 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 14141 bytes --]
On Thu, Mar 31, 2016 at 02:09:17PM +0530, Bharata B Rao wrote:
> Introduce core devices for each CPU type supported by sPAPR. These
> core devices are derived from the base spapr-cpu-core device type.
>
> TODO:
> - Add core types for other remaining CPU types
> - Handle CPU model alias correctly
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 3 +-
> hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 1 +
> include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> 4 files changed, 156 insertions(+), 2 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 64c4acc..45ac5dc 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> machine->boot_order = g_strdup(boot_device);
> }
>
> -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> - Error **errp)
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 8cbe2a5..3751a54 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> .instance_size = sizeof(sPAPRCPUCore),
> };
>
> +#define DEFINE_SPAPR_CPU_CORE(_name) \
> +static void \
> +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> + Error **errp) \
> +{ \
> + int i; \
> + Error *local_err = NULL; \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + glue(_name, sPAPRCPUCore) * core = \
> + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> + \
> + for (i = 0; i < threads; i++) { \
> + char id[32]; \
> + \
> + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> + object_class_get_name(core->cpu)); \
> + snprintf(id, sizeof(id), "thread[%d]", i); \
> + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> + &local_err); \
> + if (local_err) { \
> + goto err; \
> + } \
> + } \
> + return; \
> + \
> +err: \
> + while (--i) { \
> + object_unparent(OBJECT(&sc->threads[i])); \
> + } \
> + error_propagate(errp, local_err); \
> +} \
> + \
> +static int \
> +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> +{ \
> + Error **errp = opaque; \
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> + CPUState *cs = CPU(child); \
> + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> + \
> + object_property_set_bool(child, true, "realized", errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + \
> + spapr_cpu_init(spapr, cpu, errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + return 0; \
> +} \
If you put the ObjectClass * for the threads in the base abstract
class's class structure, then you can move most of this logic to the
abstract class as well and make the macro-ized stuff much smaller.
The realize_child stuff doesn't even need the ObjectClass* in the base
class to factor out.
> +static void \
> +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> +{ \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> + Error *local_err = NULL; \
> + \
> + /* \
> + * TODO: This is CPU model specific CPU core's realize routine. \
> + * However I am initializing "threads" field of the parent type \
> + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> + * part of CPU model specific CPU core type and have different plug() \
> + * handlers for each type instead of having a common plug() handler \
> + * for all core types. \
> + */ \
> + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> + if (local_err) { \
> + goto out; \
> + } \
> + \
> + object_child_foreach(OBJECT(dev), \
> + glue(_name, _spapr_cpu_core_realize_child), \
> + &local_err); \
> + \
> +out: \
> + if (local_err) { \
> + g_free(sc->threads); \
> + error_propagate(errp, local_err); \
> + } \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> +{ \
> + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> + ObjectClass *oc = object_class_by_name(type); \
> + \
> + core->cpu = oc; \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> +{ \
> + \
> + DeviceClass *dc = DEVICE_CLASS(oc); \
> + dc->realize = glue(_name, _spapr_cpu_core_realize); \
I think the only callback you should need to construct in the macro is
class_init to initialize the ObjectClass* field.
> +} \
> + \
> +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> +{ \
> + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> + .parent = TYPE_SPAPR_CPU_CORE, \
> + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> +};
> +
> +DEFINE_SPAPR_CPU_CORE(host);
> +DEFINE_SPAPR_CPU_CORE(POWER7);
> +DEFINE_SPAPR_CPU_CORE(POWER8);
> +
> static void spapr_cpu_core_register_types(void)
> {
> type_register_static(&spapr_cpu_core_type_info);
> + type_register_static(&host_spapr_cpu_core_type_info);
> + type_register_static(&POWER7_spapr_cpu_core_type_info);
> + type_register_static(&POWER8_spapr_cpu_core_type_info);
> }
>
> type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 098d85d..0fdf448 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index e3340ea..71e69c0 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> PowerPCCPU *threads;
> } sPAPRCPUCore;
>
> +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> +#define host_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> +
> +typedef struct hostsPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} hostsPAPRCPUCore;
> +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> +#define POWER7_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> +
> +typedef struct POWER7sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER7sPAPRCPUCore;
> +
> +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> +#define POWER8_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> +
> +typedef struct POWER8sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER8sPAPRCPUCore;
These are all identical so should also be macro constructed as well.
I don't think there's actually any need for the structures to be
exposed in a header file either, so you should be able to do it in the
same macro that constructs the implementation.
Uh.. except if you move the ObjectClass* to the base class you won't
even need these.
> #endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU core devices
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU " Bharata B Rao
@ 2016-04-01 5:12 ` David Gibson
2016-04-08 23:35 ` Michael Roth
1 sibling, 0 replies; 40+ messages in thread
From: David Gibson @ 2016-04-01 5:12 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 10300 bytes --]
On Thu, Mar 31, 2016 at 02:09:18PM +0530, Bharata B Rao wrote:
> Introduce sPAPRMachineClass.dr_cpu_enabled to indicate support for
> CPU core hotplug. Initialize boot time CPUs as core deivces and prevent
> topologies that result in partially filled cores. Both of these are done
> only if CPU core hotplug is supported.
>
> Note: An unrelated change in the call to xics_system_init() is done
> in this patch as it makes sense to use the local variable smt introduced
> in this patch instead of kvmppc_smt_threads() call here.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> hw/ppc/spapr.c | 73 +++++++++++++++++++++++++++++++++++------
> hw/ppc/spapr_cpu_core.c | 45 +++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 2 ++
> include/hw/ppc/spapr_cpu_core.h | 3 ++
> 4 files changed, 113 insertions(+), 10 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 45ac5dc..1ead043 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -64,6 +64,7 @@
>
> #include "hw/compat.h"
> #include "qemu-common.h"
> +#include "hw/ppc/spapr_cpu_core.h"
>
> #include <libfdt.h>
>
> @@ -1614,6 +1615,10 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> machine->boot_order = g_strdup(boot_device);
> }
>
> +/*
> + * TODO: Check if some of these can be moved to rtas_start_cpu() where
> + * a few other things required for hotplugged CPUs are being done.
> + */
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
> @@ -1644,6 +1649,7 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> xics_cpu_setup(spapr->icp, cpu);
>
> qemu_register_reset(spapr_cpu_reset, cpu);
> + spapr_cpu_reset(cpu);
> }
>
> /*
> @@ -1727,7 +1733,6 @@ static void ppc_spapr_init(MachineState *machine)
> const char *kernel_filename = machine->kernel_filename;
> const char *kernel_cmdline = machine->kernel_cmdline;
> const char *initrd_filename = machine->initrd_filename;
> - PowerPCCPU *cpu;
> PCIHostState *phb;
> int i;
> MemoryRegion *sysmem = get_system_memory();
> @@ -1741,6 +1746,22 @@ static void ppc_spapr_init(MachineState *machine)
> long load_limit, fw_size;
> bool kernel_le = false;
> char *filename;
> + int smt = kvmppc_smt_threads();
> + int spapr_cores = smp_cpus / smp_threads;
> + int spapr_max_cores = max_cpus / smp_threads;
> +
> + if (smc->dr_cpu_enabled) {
> + if (smp_cpus % smp_threads) {
> + error_report("smp_cpus (%u) must be multiple of threads (%u)",
> + smp_cpus, smp_threads);
> + exit(1);
> + }
> + if (max_cpus % smp_threads) {
> + error_report("max_cpus (%u) must be multiple of threads (%u)",
> + max_cpus, smp_threads);
> + exit(1);
> + }
> + }
>
> msi_supported = true;
>
> @@ -1787,8 +1808,7 @@ static void ppc_spapr_init(MachineState *machine)
>
> /* Set up Interrupt Controller before we create the VCPUs */
> spapr->icp = xics_system_init(machine,
> - DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(),
> - smp_threads),
> + DIV_ROUND_UP(max_cpus * smt, smp_threads),
> XICS_IRQS, &error_fatal);
>
> if (smc->dr_lmb_enabled) {
> @@ -1799,13 +1819,34 @@ static void ppc_spapr_init(MachineState *machine)
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> }
> - for (i = 0; i < smp_cpus; i++) {
> - cpu = cpu_ppc_init(machine->cpu_model);
> - if (cpu == NULL) {
> - error_report("Unable to find PowerPC CPU definition");
> - exit(1);
> +
> + if (smc->dr_cpu_enabled) {
> + spapr->cores = g_new0(Object *, spapr_max_cores);
> +
> + for (i = 0; i < spapr_max_cores; i++) {
> + int core_dt_id = i * smt;
> +
> + if (i < spapr_cores) {
> + char *type = spapr_get_cpu_core_type(machine->cpu_model);
> + Object *core = object_new(type);
> +
> + g_free(type);
> + object_property_set_int(core, smp_threads, "threads",
> + &error_fatal);
> + object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE,
> + &error_fatal);
> + object_property_set_bool(core, true, "realized", &error_fatal);
> + }
> }
> - spapr_cpu_init(spapr, cpu, &error_fatal);
> + } else {
> + for (i = 0; i < smp_cpus; i++) {
> + PowerPCCPU *cpu = cpu_ppc_init(machine->cpu_model);
> + if (cpu == NULL) {
> + error_report("Unable to find PowerPC CPU definition");
> + exit(1);
> + }
> + spapr_cpu_init(spapr, cpu, &error_fatal);
> + }
> }
>
> if (kvm_enabled()) {
> @@ -2257,10 +2298,19 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> }
> }
>
> +static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
> + DeviceState *dev, Error **errp)
> +{
> + if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_pre_plug(hotplug_dev, dev, errp);
> + }
> +}
> +
> static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> - if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> @@ -2299,11 +2349,13 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> mc->has_dynamic_sysbus = true;
> mc->pci_allow_0_address = true;
> mc->get_hotplug_handler = spapr_get_hotpug_handler;
> + hc->pre_plug = spapr_machine_device_pre_plug;
> hc->plug = spapr_machine_device_plug;
> hc->unplug = spapr_machine_device_unplug;
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
>
> smc->dr_lmb_enabled = true;
> + smc->dr_cpu_enabled = true;
> fwc->get_dev_path = spapr_get_fw_dev_path;
> nc->nmi_monitor_handler = spapr_nmi;
> }
> @@ -2383,6 +2435,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
>
> spapr_machine_2_6_class_options(mc);
> smc->use_ohci_by_default = true;
> + smc->dr_cpu_enabled = false;
> SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 3751a54..640d143 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -15,6 +15,40 @@
> #include <sysemu/cpus.h>
> #include "target-ppc/kvm_ppc.h"
>
> +void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + int spapr_max_cores = max_cpus / smp_threads;
> + int index;
> + int smt = kvmppc_smt_threads();
> + Error *local_err = NULL;
> + CPUCore *cc = CPU_CORE(dev);
> +
> + if (cc->threads != smp_threads) {
> + error_setg(&local_err, "threads must be %d", smp_threads);
> + goto out;
> + }
> +
> + if (cc->core % smt) {
> + error_setg(&local_err, "invalid core id %d\n", cc->core);
> + goto out;
> + }
> +
> + index = cc->core / smt;
> + if (index < 0 || index >= spapr_max_cores) {
> + error_setg(&local_err, "core id %d out of range", cc->core);
> + goto out;
> + }
> +
> + if (spapr->cores[index]) {
> + error_setg(&local_err, "core %d already populated", cc->core);
> + }
> +
> +out:
> + error_propagate(errp, local_err);
> +}
> +
> static const TypeInfo spapr_cpu_core_type_info = {
> .name = TYPE_SPAPR_CPU_CORE,
> .parent = TYPE_CPU_CORE,
> @@ -146,3 +180,14 @@ static void spapr_cpu_core_register_types(void)
> }
>
> type_init(spapr_cpu_core_register_types)
> +
> +/*
> + * TODO: Looks fragile :(
> + */
> +char *spapr_get_cpu_core_type(const char *model)
> +{
> + char core_type[32];
> +
> + snprintf(core_type, 32, "%s-%s", model, TYPE_SPAPR_CPU_CORE);
> + return g_strdup(core_type);
> +}
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 0fdf448..a6956c0 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -36,6 +36,7 @@ struct sPAPRMachineClass {
>
> /*< public >*/
> bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */
> + bool dr_cpu_enabled; /* enable dynamic-reconfig/hotplug of CPUs */
> bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */
> };
>
> @@ -79,6 +80,7 @@ struct sPAPRMachineState {
> /*< public >*/
> char *kvm_type;
> MemoryHotplugState hotplug_memory;
> + Object **cores;
> };
>
> #define H_SUCCESS 0
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index 71e69c0..f08f291 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -60,4 +60,7 @@ typedef struct POWER8sPAPRCPUCore {
> ObjectClass *cpu;
> } POWER8sPAPRCPUCore;
>
> +void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp);
> +char *spapr_get_cpu_core_type(const char *model);
> #endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-04-01 5:08 ` David Gibson
@ 2016-04-01 6:12 ` Bharata B Rao
2016-04-04 0:13 ` David Gibson
0 siblings, 1 reply; 40+ messages in thread
From: Bharata B Rao @ 2016-04-01 6:12 UTC (permalink / raw)
To: David Gibson
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
On Fri, Apr 01, 2016 at 04:08:44PM +1100, David Gibson wrote:
> On Thu, Mar 31, 2016 at 02:09:17PM +0530, Bharata B Rao wrote:
> > Introduce core devices for each CPU type supported by sPAPR. These
> > core devices are derived from the base spapr-cpu-core device type.
> >
> > TODO:
> > - Add core types for other remaining CPU types
> > - Handle CPU model alias correctly
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 3 +-
> > hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> > include/hw/ppc/spapr.h | 1 +
> > include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> > 4 files changed, 156 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 64c4acc..45ac5dc 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> > machine->boot_order = g_strdup(boot_device);
> > }
> >
> > -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> > - Error **errp)
> > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > {
> > CPUPPCState *env = &cpu->env;
> >
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > index 8cbe2a5..3751a54 100644
> > --- a/hw/ppc/spapr_cpu_core.c
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> > .instance_size = sizeof(sPAPRCPUCore),
> > };
> >
> > +#define DEFINE_SPAPR_CPU_CORE(_name) \
> > +static void \
> > +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> > + Error **errp) \
> > +{ \
> > + int i; \
> > + Error *local_err = NULL; \
> > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > + glue(_name, sPAPRCPUCore) * core = \
> > + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> > + \
> > + for (i = 0; i < threads; i++) { \
> > + char id[32]; \
> > + \
> > + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> > + object_class_get_name(core->cpu)); \
> > + snprintf(id, sizeof(id), "thread[%d]", i); \
> > + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> > + &local_err); \
> > + if (local_err) { \
> > + goto err; \
> > + } \
> > + } \
> > + return; \
> > + \
> > +err: \
> > + while (--i) { \
> > + object_unparent(OBJECT(&sc->threads[i])); \
> > + } \
> > + error_propagate(errp, local_err); \
> > +} \
> > + \
> > +static int \
> > +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> > +{ \
> > + Error **errp = opaque; \
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> > + CPUState *cs = CPU(child); \
> > + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> > + \
> > + object_property_set_bool(child, true, "realized", errp); \
> > + if (*errp) { \
> > + return 1; \
> > + } \
> > + \
> > + spapr_cpu_init(spapr, cpu, errp); \
> > + if (*errp) { \
> > + return 1; \
> > + } \
> > + return 0; \
> > +} \
>
> If you put the ObjectClass * for the threads in the base abstract
> class's class structure, then you can move most of this logic to the
> abstract class as well and make the macro-ized stuff much smaller.
>
> The realize_child stuff doesn't even need the ObjectClass* in the base
> class to factor out.
>
> > +static void \
> > +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> > +{ \
> > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> > + Error *local_err = NULL; \
> > + \
> > + /* \
> > + * TODO: This is CPU model specific CPU core's realize routine. \
> > + * However I am initializing "threads" field of the parent type \
> > + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> > + * part of CPU model specific CPU core type and have different plug() \
> > + * handlers for each type instead of having a common plug() handler \
> > + * for all core types. \
> > + */ \
> > + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> > + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> > + if (local_err) { \
> > + goto out; \
> > + } \
> > + \
> > + object_child_foreach(OBJECT(dev), \
> > + glue(_name, _spapr_cpu_core_realize_child), \
> > + &local_err); \
> > + \
> > +out: \
> > + if (local_err) { \
> > + g_free(sc->threads); \
> > + error_propagate(errp, local_err); \
> > + } \
> > +} \
> > + \
> > +static void \
> > +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> > +{ \
> > + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> > + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> > + ObjectClass *oc = object_class_by_name(type); \
> > + \
> > + core->cpu = oc; \
> > +} \
> > + \
> > +static void \
> > +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> > +{ \
> > + \
> > + DeviceClass *dc = DEVICE_CLASS(oc); \
> > + dc->realize = glue(_name, _spapr_cpu_core_realize); \
>
> I think the only callback you should need to construct in the macro is
> class_init to initialize the ObjectClass* field.
>
> > +} \
> > + \
> > +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> > +{ \
> > + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> > + .parent = TYPE_SPAPR_CPU_CORE, \
> > + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> > + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> > + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> > +};
> > +
> > +DEFINE_SPAPR_CPU_CORE(host);
> > +DEFINE_SPAPR_CPU_CORE(POWER7);
> > +DEFINE_SPAPR_CPU_CORE(POWER8);
> > +
> > static void spapr_cpu_core_register_types(void)
> > {
> > type_register_static(&spapr_cpu_core_type_info);
> > + type_register_static(&host_spapr_cpu_core_type_info);
> > + type_register_static(&POWER7_spapr_cpu_core_type_info);
> > + type_register_static(&POWER8_spapr_cpu_core_type_info);
> > }
> >
> > type_init(spapr_cpu_core_register_types)
> > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > index 098d85d..0fdf448 100644
> > --- a/include/hw/ppc/spapr.h
> > +++ b/include/hw/ppc/spapr.h
> > @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> > uint32_t count);
> > void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> > uint32_t count);
> > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> >
> > /* rtas-configure-connector state */
> > struct sPAPRConfigureConnectorState {
> > diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> > index e3340ea..71e69c0 100644
> > --- a/include/hw/ppc/spapr_cpu_core.h
> > +++ b/include/hw/ppc/spapr_cpu_core.h
> > @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> > PowerPCCPU *threads;
> > } sPAPRCPUCore;
> >
> > +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> > +#define host_SPAPR_CPU_CORE(obj) \
> > + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> > +
> > +typedef struct hostsPAPRCPUCore {
> > + /*< private >*/
> > + sPAPRCPUCore parent_obj;
> > +
> > + /*< public >*/
> > + ObjectClass *cpu;
> > +} hostsPAPRCPUCore;
> > +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> > +#define POWER7_SPAPR_CPU_CORE(obj) \
> > + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> > +
> > +typedef struct POWER7sPAPRCPUCore {
> > + /*< private >*/
> > + sPAPRCPUCore parent_obj;
> > +
> > + /*< public >*/
> > + ObjectClass *cpu;
> > +} POWER7sPAPRCPUCore;
> > +
> > +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> > +#define POWER8_SPAPR_CPU_CORE(obj) \
> > + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> > +
> > +typedef struct POWER8sPAPRCPUCore {
> > + /*< private >*/
> > + sPAPRCPUCore parent_obj;
> > +
> > + /*< public >*/
> > + ObjectClass *cpu;
> > +} POWER8sPAPRCPUCore;
>
> These are all identical so should also be macro constructed as well.
> I don't think there's actually any need for the structures to be
> exposed in a header file either, so you should be able to do it in the
> same macro that constructs the implementation.
>
> Uh.. except if you move the ObjectClass* to the base class you won't
> even need these.
The only reason (currently) POWER8sPAPRCPUCore exists separately from
the base class sPAPRCPUCore is that it represents POWER8 core which
is stored as ObjectClass*.
Now if we don't track cpu type (ObjectClass *) as part of
POWER8sPAPRCPUCore but push that up to sPAPRCPUCore, I am not
sure if that would be at the right abstraction level.
Apart from the above, can you also look at the TODO I have put on the top
of _spapr_cpu_core_create_threads() routine in this patch ? I am not sure
if what I have done there is the right thing to do.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
2016-04-01 3:30 ` David Gibson
@ 2016-04-01 10:38 ` Paolo Bonzini
2016-04-04 0:09 ` David Gibson
0 siblings, 1 reply; 40+ messages in thread
From: Paolo Bonzini @ 2016-04-01 10:38 UTC (permalink / raw)
To: David Gibson, Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, imammedo, afaerber, mdroth
[-- Attachment #1.1: Type: text/plain, Size: 4169 bytes --]
On 01/04/2016 05:30, David Gibson wrote:
> On Thu, Mar 31, 2016 at 02:09:14PM +0530, Bharata B Rao wrote:
>> From: Igor Mammedov <imammedo@redhat.com>
>>
>> pre_plug callback is to be called before device.realize() is executed.
>> This would allow to check/set device's properties from HotplugHandler.
>>
>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
>
> It would be really nice to get some opinion on this from Andreas or
> Paolo.
Certainly okay for me, Igor did all of the HotplugHandler design and work.
Paolo
>> ---
>> hw/core/hotplug.c | 11 +++++++++++
>> hw/core/qdev.c | 9 ++++++++-
>> include/hw/hotplug.h | 14 +++++++++++++-
>> 3 files changed, 32 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c
>> index 645cfca..17ac986 100644
>> --- a/hw/core/hotplug.c
>> +++ b/hw/core/hotplug.c
>> @@ -13,6 +13,17 @@
>> #include "hw/hotplug.h"
>> #include "qemu/module.h"
>>
>> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
>> + DeviceState *plugged_dev,
>> + Error **errp)
>> +{
>> + HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler);
>> +
>> + if (hdc->pre_plug) {
>> + hdc->pre_plug(plug_handler, plugged_dev, errp);
>> + }
>> +}
>> +
>> void hotplug_handler_plug(HotplugHandler *plug_handler,
>> DeviceState *plugged_dev,
>> Error **errp)
>> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
>> index db41aa1..a0b3aad 100644
>> --- a/hw/core/qdev.c
>> +++ b/hw/core/qdev.c
>> @@ -1062,6 +1062,14 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
>> g_free(name);
>> }
>>
>> + hotplug_ctrl = qdev_get_hotplug_handler(dev);
>> + if (hotplug_ctrl) {
>> + hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err);
>> + if (local_err != NULL) {
>> + goto fail;
>> + }
>> + }
>> +
>> if (dc->realize) {
>> dc->realize(dev, &local_err);
>> }
>> @@ -1072,7 +1080,6 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
>>
>> DEVICE_LISTENER_CALL(realize, Forward, dev);
>>
>> - hotplug_ctrl = qdev_get_hotplug_handler(dev);
>> if (hotplug_ctrl) {
>> hotplug_handler_plug(hotplug_ctrl, dev, &local_err);
>> }
>> diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h
>> index 2db025d..50d84e9 100644
>> --- a/include/hw/hotplug.h
>> +++ b/include/hw/hotplug.h
>> @@ -46,7 +46,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler,
>> * hardware (un)plug functions.
>> *
>> * @parent: Opaque parent interface.
>> - * @plug: plug callback.
>> + * @pre_plug: pre plug callback called at start of device.realize(true)
>> + * @plug: plug callback called at end of device.realize(true).
>> * @unplug_request: unplug request callback.
>> * Used as a means to initiate device unplug for devices that
>> * require asynchronous unplug handling.
>> @@ -59,6 +60,7 @@ typedef struct HotplugHandlerClass {
>> InterfaceClass parent;
>>
>> /* <public> */
>> + hotplug_fn pre_plug;
>> hotplug_fn plug;
>> hotplug_fn unplug_request;
>> hotplug_fn unplug;
>> @@ -74,6 +76,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler,
>> Error **errp);
>>
>> /**
>> + * hotplug_handler_pre_plug:
>> + *
>> + * Call #HotplugHandlerClass.pre_plug callback of @plug_handler.
>> + */
>> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
>> + DeviceState *plugged_dev,
>> + Error **errp);
>> +
>> +
>> +/**
>> * hotplug_handler_unplug_request:
>> *
>> * Calls #HotplugHandlerClass.unplug_request callback of @plug_handler.
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
2016-04-01 10:38 ` Paolo Bonzini
@ 2016-04-04 0:09 ` David Gibson
0 siblings, 0 replies; 40+ messages in thread
From: David Gibson @ 2016-04-04 0:09 UTC (permalink / raw)
To: Paolo Bonzini
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, Bharata B Rao, imammedo,
afaerber, mdroth
[-- Attachment #1: Type: text/plain, Size: 4704 bytes --]
On Fri, Apr 01, 2016 at 12:38:28PM +0200, Paolo Bonzini wrote:
>
>
> On 01/04/2016 05:30, David Gibson wrote:
> > On Thu, Mar 31, 2016 at 02:09:14PM +0530, Bharata B Rao wrote:
> >> From: Igor Mammedov <imammedo@redhat.com>
> >>
> >> pre_plug callback is to be called before device.realize() is executed.
> >> This would allow to check/set device's properties from HotplugHandler.
> >>
> >> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> >
> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> >
> > It would be really nice to get some opinion on this from Andreas or
> > Paolo.
>
> Certainly okay for me, Igor did all of the HotplugHandler design and
> work.
Ok, good to hear. Thanks.
>
> Paolo
>
> >> ---
> >> hw/core/hotplug.c | 11 +++++++++++
> >> hw/core/qdev.c | 9 ++++++++-
> >> include/hw/hotplug.h | 14 +++++++++++++-
> >> 3 files changed, 32 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c
> >> index 645cfca..17ac986 100644
> >> --- a/hw/core/hotplug.c
> >> +++ b/hw/core/hotplug.c
> >> @@ -13,6 +13,17 @@
> >> #include "hw/hotplug.h"
> >> #include "qemu/module.h"
> >>
> >> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
> >> + DeviceState *plugged_dev,
> >> + Error **errp)
> >> +{
> >> + HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler);
> >> +
> >> + if (hdc->pre_plug) {
> >> + hdc->pre_plug(plug_handler, plugged_dev, errp);
> >> + }
> >> +}
> >> +
> >> void hotplug_handler_plug(HotplugHandler *plug_handler,
> >> DeviceState *plugged_dev,
> >> Error **errp)
> >> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> >> index db41aa1..a0b3aad 100644
> >> --- a/hw/core/qdev.c
> >> +++ b/hw/core/qdev.c
> >> @@ -1062,6 +1062,14 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> >> g_free(name);
> >> }
> >>
> >> + hotplug_ctrl = qdev_get_hotplug_handler(dev);
> >> + if (hotplug_ctrl) {
> >> + hotplug_handler_pre_plug(hotplug_ctrl, dev, &local_err);
> >> + if (local_err != NULL) {
> >> + goto fail;
> >> + }
> >> + }
> >> +
> >> if (dc->realize) {
> >> dc->realize(dev, &local_err);
> >> }
> >> @@ -1072,7 +1080,6 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> >>
> >> DEVICE_LISTENER_CALL(realize, Forward, dev);
> >>
> >> - hotplug_ctrl = qdev_get_hotplug_handler(dev);
> >> if (hotplug_ctrl) {
> >> hotplug_handler_plug(hotplug_ctrl, dev, &local_err);
> >> }
> >> diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h
> >> index 2db025d..50d84e9 100644
> >> --- a/include/hw/hotplug.h
> >> +++ b/include/hw/hotplug.h
> >> @@ -46,7 +46,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler,
> >> * hardware (un)plug functions.
> >> *
> >> * @parent: Opaque parent interface.
> >> - * @plug: plug callback.
> >> + * @pre_plug: pre plug callback called at start of device.realize(true)
> >> + * @plug: plug callback called at end of device.realize(true).
> >> * @unplug_request: unplug request callback.
> >> * Used as a means to initiate device unplug for devices that
> >> * require asynchronous unplug handling.
> >> @@ -59,6 +60,7 @@ typedef struct HotplugHandlerClass {
> >> InterfaceClass parent;
> >>
> >> /* <public> */
> >> + hotplug_fn pre_plug;
> >> hotplug_fn plug;
> >> hotplug_fn unplug_request;
> >> hotplug_fn unplug;
> >> @@ -74,6 +76,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler,
> >> Error **errp);
> >>
> >> /**
> >> + * hotplug_handler_pre_plug:
> >> + *
> >> + * Call #HotplugHandlerClass.pre_plug callback of @plug_handler.
> >> + */
> >> +void hotplug_handler_pre_plug(HotplugHandler *plug_handler,
> >> + DeviceState *plugged_dev,
> >> + Error **errp);
> >> +
> >> +
> >> +/**
> >> * hotplug_handler_unplug_request:
> >> *
> >> * Calls #HotplugHandlerClass.unplug_request callback of @plug_handler.
> >
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-04-01 6:12 ` Bharata B Rao
@ 2016-04-04 0:13 ` David Gibson
2016-04-09 2:21 ` Michael Roth
0 siblings, 1 reply; 40+ messages in thread
From: David Gibson @ 2016-04-04 0:13 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 15931 bytes --]
On Fri, Apr 01, 2016 at 11:42:23AM +0530, Bharata B Rao wrote:
> On Fri, Apr 01, 2016 at 04:08:44PM +1100, David Gibson wrote:
> > On Thu, Mar 31, 2016 at 02:09:17PM +0530, Bharata B Rao wrote:
> > > Introduce core devices for each CPU type supported by sPAPR. These
> > > core devices are derived from the base spapr-cpu-core device type.
> > >
> > > TODO:
> > > - Add core types for other remaining CPU types
> > > - Handle CPU model alias correctly
> > >
> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > ---
> > > hw/ppc/spapr.c | 3 +-
> > > hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> > > include/hw/ppc/spapr.h | 1 +
> > > include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> > > 4 files changed, 156 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 64c4acc..45ac5dc 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> > > machine->boot_order = g_strdup(boot_device);
> > > }
> > >
> > > -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> > > - Error **errp)
> > > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > > {
> > > CPUPPCState *env = &cpu->env;
> > >
> > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > index 8cbe2a5..3751a54 100644
> > > --- a/hw/ppc/spapr_cpu_core.c
> > > +++ b/hw/ppc/spapr_cpu_core.c
> > > @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> > > .instance_size = sizeof(sPAPRCPUCore),
> > > };
> > >
> > > +#define DEFINE_SPAPR_CPU_CORE(_name) \
> > > +static void \
> > > +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> > > + Error **errp) \
> > > +{ \
> > > + int i; \
> > > + Error *local_err = NULL; \
> > > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > > + glue(_name, sPAPRCPUCore) * core = \
> > > + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> > > + \
> > > + for (i = 0; i < threads; i++) { \
> > > + char id[32]; \
> > > + \
> > > + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> > > + object_class_get_name(core->cpu)); \
> > > + snprintf(id, sizeof(id), "thread[%d]", i); \
> > > + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> > > + &local_err); \
> > > + if (local_err) { \
> > > + goto err; \
> > > + } \
> > > + } \
> > > + return; \
> > > + \
> > > +err: \
> > > + while (--i) { \
> > > + object_unparent(OBJECT(&sc->threads[i])); \
> > > + } \
> > > + error_propagate(errp, local_err); \
> > > +} \
> > > + \
> > > +static int \
> > > +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> > > +{ \
> > > + Error **errp = opaque; \
> > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> > > + CPUState *cs = CPU(child); \
> > > + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> > > + \
> > > + object_property_set_bool(child, true, "realized", errp); \
> > > + if (*errp) { \
> > > + return 1; \
> > > + } \
> > > + \
> > > + spapr_cpu_init(spapr, cpu, errp); \
> > > + if (*errp) { \
> > > + return 1; \
> > > + } \
> > > + return 0; \
> > > +} \
> >
> > If you put the ObjectClass * for the threads in the base abstract
> > class's class structure, then you can move most of this logic to the
> > abstract class as well and make the macro-ized stuff much smaller.
> >
> > The realize_child stuff doesn't even need the ObjectClass* in the base
> > class to factor out.
> >
> > > +static void \
> > > +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> > > +{ \
> > > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > > + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> > > + Error *local_err = NULL; \
> > > + \
> > > + /* \
> > > + * TODO: This is CPU model specific CPU core's realize routine. \
> > > + * However I am initializing "threads" field of the parent type \
> > > + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> > > + * part of CPU model specific CPU core type and have different plug() \
> > > + * handlers for each type instead of having a common plug() handler \
> > > + * for all core types. \
> > > + */ \
> > > + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> > > + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> > > + if (local_err) { \
> > > + goto out; \
> > > + } \
> > > + \
> > > + object_child_foreach(OBJECT(dev), \
> > > + glue(_name, _spapr_cpu_core_realize_child), \
> > > + &local_err); \
> > > + \
> > > +out: \
> > > + if (local_err) { \
> > > + g_free(sc->threads); \
> > > + error_propagate(errp, local_err); \
> > > + } \
> > > +} \
> > > + \
> > > +static void \
> > > +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> > > +{ \
> > > + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> > > + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> > > + ObjectClass *oc = object_class_by_name(type); \
> > > + \
> > > + core->cpu = oc; \
> > > +} \
> > > + \
> > > +static void \
> > > +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> > > +{ \
> > > + \
> > > + DeviceClass *dc = DEVICE_CLASS(oc); \
> > > + dc->realize = glue(_name, _spapr_cpu_core_realize); \
> >
> > I think the only callback you should need to construct in the macro is
> > class_init to initialize the ObjectClass* field.
> >
> > > +} \
> > > + \
> > > +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> > > +{ \
> > > + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> > > + .parent = TYPE_SPAPR_CPU_CORE, \
> > > + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> > > + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> > > + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> > > +};
> > > +
> > > +DEFINE_SPAPR_CPU_CORE(host);
> > > +DEFINE_SPAPR_CPU_CORE(POWER7);
> > > +DEFINE_SPAPR_CPU_CORE(POWER8);
> > > +
> > > static void spapr_cpu_core_register_types(void)
> > > {
> > > type_register_static(&spapr_cpu_core_type_info);
> > > + type_register_static(&host_spapr_cpu_core_type_info);
> > > + type_register_static(&POWER7_spapr_cpu_core_type_info);
> > > + type_register_static(&POWER8_spapr_cpu_core_type_info);
> > > }
> > >
> > > type_init(spapr_cpu_core_register_types)
> > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > > index 098d85d..0fdf448 100644
> > > --- a/include/hw/ppc/spapr.h
> > > +++ b/include/hw/ppc/spapr.h
> > > @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> > > uint32_t count);
> > > void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> > > uint32_t count);
> > > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> > >
> > > /* rtas-configure-connector state */
> > > struct sPAPRConfigureConnectorState {
> > > diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> > > index e3340ea..71e69c0 100644
> > > --- a/include/hw/ppc/spapr_cpu_core.h
> > > +++ b/include/hw/ppc/spapr_cpu_core.h
> > > @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> > > PowerPCCPU *threads;
> > > } sPAPRCPUCore;
> > >
> > > +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> > > +#define host_SPAPR_CPU_CORE(obj) \
> > > + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> > > +
> > > +typedef struct hostsPAPRCPUCore {
> > > + /*< private >*/
> > > + sPAPRCPUCore parent_obj;
> > > +
> > > + /*< public >*/
> > > + ObjectClass *cpu;
> > > +} hostsPAPRCPUCore;
> > > +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> > > +#define POWER7_SPAPR_CPU_CORE(obj) \
> > > + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> > > +
> > > +typedef struct POWER7sPAPRCPUCore {
> > > + /*< private >*/
> > > + sPAPRCPUCore parent_obj;
> > > +
> > > + /*< public >*/
> > > + ObjectClass *cpu;
> > > +} POWER7sPAPRCPUCore;
> > > +
> > > +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> > > +#define POWER8_SPAPR_CPU_CORE(obj) \
> > > + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> > > +
> > > +typedef struct POWER8sPAPRCPUCore {
> > > + /*< private >*/
> > > + sPAPRCPUCore parent_obj;
> > > +
> > > + /*< public >*/
> > > + ObjectClass *cpu;
> > > +} POWER8sPAPRCPUCore;
> >
> > These are all identical so should also be macro constructed as well.
> > I don't think there's actually any need for the structures to be
> > exposed in a header file either, so you should be able to do it in the
> > same macro that constructs the implementation.
> >
> > Uh.. except if you move the ObjectClass* to the base class you won't
> > even need these.
>
> The only reason (currently) POWER8sPAPRCPUCore exists separately from
> the base class sPAPRCPUCore is that it represents POWER8 core which
> is stored as ObjectClass*.
>
> Now if we don't track cpu type (ObjectClass *) as part of
> POWER8sPAPRCPUCore but push that up to sPAPRCPUCore, I am not
> sure if that would be at the right abstraction level.
Not as part of sPAPRCPUCore, but as part of sPAPRCPUCoreClass. We
don't have a structure for the class at present, but we can add one.
You can think of it as a method that sPAPRCPUCore and subclasses have
which returns the correct ObjectClass *, except that we don't actually
need a method (function pointer) - a simple data pointer in the class
will suffice.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
2016-04-01 5:08 ` David Gibson
@ 2016-04-04 0:16 ` David Gibson
2016-04-08 23:35 ` Michael Roth
2 siblings, 0 replies; 40+ messages in thread
From: David Gibson @ 2016-04-04 0:16 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 13575 bytes --]
On Thu, Mar 31, 2016 at 02:09:17PM +0530, Bharata B Rao wrote:
> Introduce core devices for each CPU type supported by sPAPR. These
> core devices are derived from the base spapr-cpu-core device type.
>
> TODO:
> - Add core types for other remaining CPU types
> - Handle CPU model alias correctly
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 3 +-
> hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 1 +
> include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> 4 files changed, 156 insertions(+), 2 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 64c4acc..45ac5dc 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> machine->boot_order = g_strdup(boot_device);
> }
>
> -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> - Error **errp)
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 8cbe2a5..3751a54 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> .instance_size = sizeof(sPAPRCPUCore),
> };
>
> +#define DEFINE_SPAPR_CPU_CORE(_name) \
> +static void \
> +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> + Error **errp) \
> +{ \
> + int i; \
> + Error *local_err = NULL; \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + glue(_name, sPAPRCPUCore) * core = \
> + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> + \
> + for (i = 0; i < threads; i++) { \
> + char id[32]; \
> + \
> + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> + object_class_get_name(core->cpu)); \
> + snprintf(id, sizeof(id), "thread[%d]", i); \
> + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> + &local_err); \
> + if (local_err) { \
> + goto err; \
> + } \
> + } \
> + return; \
> + \
> +err: \
> + while (--i) { \
> + object_unparent(OBJECT(&sc->threads[i])); \
> + } \
> + error_propagate(errp, local_err); \
> +} \
> + \
> +static int \
> +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> +{ \
> + Error **errp = opaque; \
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> + CPUState *cs = CPU(child); \
> + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> + \
> + object_property_set_bool(child, true, "realized", errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + \
> + spapr_cpu_init(spapr, cpu, errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + return 0; \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> +{ \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> + Error *local_err = NULL; \
> + \
> + /* \
> + * TODO: This is CPU model specific CPU core's realize routine. \
> + * However I am initializing "threads" field of the parent type \
> + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> + * part of CPU model specific CPU core type and have different plug() \
> + * handlers for each type instead of having a common plug() handler \
> + * for all core types. \
> + */
> \
I think that should be fine - it's generally ok for methods to
manipulate superclass data fields (technically this is what C++ calls
a "protected" as opposed to "private" field).
> + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> + if (local_err) { \
> + goto out; \
> + } \
> + \
> + object_child_foreach(OBJECT(dev), \
> + glue(_name, _spapr_cpu_core_realize_child), \
> + &local_err); \
> + \
> +out: \
> + if (local_err) { \
> + g_free(sc->threads); \
> + error_propagate(errp, local_err); \
> + } \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> +{ \
> + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> + ObjectClass *oc = object_class_by_name(type); \
> + \
> + core->cpu = oc; \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> +{ \
> + \
> + DeviceClass *dc = DEVICE_CLASS(oc); \
> + dc->realize = glue(_name, _spapr_cpu_core_realize); \
> +} \
> + \
> +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> +{ \
> + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> + .parent = TYPE_SPAPR_CPU_CORE, \
> + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> +};
> +
> +DEFINE_SPAPR_CPU_CORE(host);
> +DEFINE_SPAPR_CPU_CORE(POWER7);
> +DEFINE_SPAPR_CPU_CORE(POWER8);
> +
> static void spapr_cpu_core_register_types(void)
> {
> type_register_static(&spapr_cpu_core_type_info);
> + type_register_static(&host_spapr_cpu_core_type_info);
> + type_register_static(&POWER7_spapr_cpu_core_type_info);
> + type_register_static(&POWER8_spapr_cpu_core_type_info);
> }
>
> type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 098d85d..0fdf448 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index e3340ea..71e69c0 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> PowerPCCPU *threads;
> } sPAPRCPUCore;
>
> +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> +#define host_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> +
> +typedef struct hostsPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} hostsPAPRCPUCore;
> +
> +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> +#define POWER7_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> +
> +typedef struct POWER7sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER7sPAPRCPUCore;
> +
> +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> +#define POWER8_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> +
> +typedef struct POWER8sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER8sPAPRCPUCore;
> +
> #endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
@ 2016-04-04 4:23 ` David Gibson
2016-04-05 23:47 ` Michael Roth
2016-05-06 8:57 ` Igor Mammedov
2 siblings, 0 replies; 40+ messages in thread
From: David Gibson @ 2016-04-04 4:23 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 12624 bytes --]
On Thu, Mar 31, 2016 at 02:09:19PM +0530, Bharata B Rao wrote:
> Set up device tree entries for the hotplugged CPU core and use the
> exising RTAS event logging infrastructure to send CPU hotplug notification
> to the guest.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> hw/ppc/spapr.c | 58 ++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_cpu_core.c | 70 +++++++++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_events.c | 3 ++
> hw/ppc/spapr_rtas.c | 24 ++++++++++++++
> include/hw/ppc/spapr.h | 2 ++
> include/hw/ppc/spapr_cpu_core.h | 2 ++
> 6 files changed, 159 insertions(+)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1ead043..1a5dbd9 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -603,6 +603,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> size_t page_sizes_prop_size;
> uint32_t vcpus_per_socket = smp_threads * smp_cores;
> uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + int drc_index;
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> + if (drc) {
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drc_index = drck->get_index(drc);
> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> + }
It seems like now we have a concrete representation for cpu cores, we
ought to be passing a cpu core instead of a thread state to the dt
populate function. I guess that complicates backwards compat stuff
for older machine types though.
> /* Note: we keep CI large pages off for now because a 64K capable guest
> * provisioned with large pages might otherwise try to map a qemu
> @@ -987,6 +997,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> }
>
> + if (smc->dr_cpu_enabled) {
> + int offset = fdt_path_offset(fdt, "/cpus");
> + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> + SPAPR_DR_CONNECTOR_TYPE_CPU);
> + if (ret < 0) {
> + error_report("Couldn't set up CPU DR device tree properties");
> + exit(1);
> + }
> + }
> +
> _FDT((fdt_pack(fdt)));
>
> if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> @@ -1622,6 +1642,8 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
> + CPUState *cs = CPU(cpu);
> + int i;
>
> /* Set time-base frequency to 512 MHz */
> cpu_ppc_tb_init(env, TIMEBASE_FREQ);
> @@ -1646,6 +1668,14 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> }
> }
>
> + /* Set NUMA node for the added CPUs */
> + for (i = 0; i < nb_numa_nodes; i++) {
> + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> + cs->numa_node = i;
> + break;
> + }
> + }
> +
> xics_cpu_setup(spapr->icp, cpu);
>
> qemu_register_reset(spapr_cpu_reset, cpu);
> @@ -1825,6 +1855,11 @@ static void ppc_spapr_init(MachineState *machine)
>
> for (i = 0; i < spapr_max_cores; i++) {
> int core_dt_id = i * smt;
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_new(OBJECT(spapr),
> + SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
> +
> + qemu_register_reset(spapr_drc_reset, drc);
>
> if (i < spapr_cores) {
> char *type = spapr_get_cpu_core_type(machine->cpu_model);
> @@ -2247,6 +2282,27 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> + sPAPRMachineState *spapr)
> +{
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + void *fdt;
> + int offset, fdt_size;
> + char *nodename;
> +
> + fdt = create_device_tree(&fdt_size);
> + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> + offset = fdt_add_subnode(fdt, 0, nodename);
> +
> + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> + g_free(nodename);
> +
> + *fdt_offset = offset;
> + return fdt;
> +}
> +
Ugh. I've really got to find time to clean up a bunch of the DT
construction stuff in spapr. Not something in the scope of this
patch, though.
> static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> @@ -2287,6 +2343,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
>
> spapr_memory_plug(hotplug_dev, dev, node, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_plug(hotplug_dev, dev, errp);
> }
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 640d143..a9ba843 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -18,6 +18,7 @@
> void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp)
> {
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> int spapr_max_cores = max_cpus / smp_threads;
> int index;
> @@ -25,6 +26,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error *local_err = NULL;
> CPUCore *cc = CPU_CORE(dev);
>
> + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> + error_setg(&local_err, "CPU hotplug not supported for this machine");
> + goto out;
> + }
> +
> if (cc->threads != smp_threads) {
> error_setg(&local_err, "threads must be %d", smp_threads);
> goto out;
> @@ -49,6 +55,70 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + CPUCore *cc = CPU_CORE(dev);
> + CPUState *cs = CPU(&core->threads[0]);
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> + void *fdt = NULL;
> + int fdt_offset = 0;
> + int index;
> + int smt = kvmppc_smt_threads();
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core);
> + index = cc->core / smt;
> + spapr->cores[index] = OBJECT(dev);
> +
> + if (!smc->dr_cpu_enabled) {
> + /*
> + * This is a cold plugged CPU core but the machine doesn't support
> + * DR. So skip the hotplug path ensuring that the core is brought
> + * up online with out an associated DR connector.
> + */
> + return;
> + }
> +
> + g_assert(drc);
> +
> + /*
> + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> + */
> + if (dev->hotplugged) {
> + fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
> + dev->hotplugged = true;
> + }
> +
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
> + if (local_err) {
> + g_free(fdt);
> + spapr->cores[index] = NULL;
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + if (dev->hotplugged) {
> + /*
> + * Send hotplug notification interrupt to the guest only in case
> + * of hotplugged CPUs.
> + */
> + spapr_hotplug_req_add_by_index(drc);
> + } else {
> + /*
> + * Set the right DRC states for cold plugged CPU.
> + */
> + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
> + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
> + }
> +}
> +
> static const TypeInfo spapr_cpu_core_type_info = {
> .name = TYPE_SPAPR_CPU_CORE,
> .parent = TYPE_CPU_CORE,
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index 39f4682..10340e1 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
> case SPAPR_DR_CONNECTOR_TYPE_LMB:
> hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
> break;
> + case SPAPR_DR_CONNECTOR_TYPE_CPU:
> + hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
> + break;
> default:
> /* we shouldn't be signaling hotplug events for resources
> * that don't support them
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index b7c5ebd..cc0369e 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -34,6 +34,7 @@
>
> #include "hw/ppc/spapr.h"
> #include "hw/ppc/spapr_vio.h"
> +#include "hw/ppc/ppc.h"
> #include "qapi-event.h"
> #include "hw/boards.h"
>
> @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> }
>
> +/*
> + * Set the timebase offset of the CPU to that of first CPU.
> + * This helps hotplugged CPU to have the correct timebase offset.
> + */
> +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> +
> + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> +}
> +
> +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> +
> + if (!pcc->interrupts_big_endian(fcpu)) {
> + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> + }
> +}
> +
> static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> uint32_t token, uint32_t nargs,
> target_ulong args,
> @@ -197,6 +219,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> env->nip = start;
> env->gpr[3] = r3;
> cs->halted = 0;
> + spapr_cpu_set_endianness(cpu);
> + spapr_cpu_update_tb_offset(cpu);
I'd kind of like to see these rtas_start_cpu() changes split out to a
separate patch, but it's not worth respinning just for that.
> qemu_cpu_kick(cs);
>
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index a6956c0..619db98 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -588,6 +588,8 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> + sPAPRMachineState *spapr);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index f08f291..165af7c 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -63,4 +63,6 @@ typedef struct POWER8sPAPRCPUCore {
> void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp);
> char *spapr_get_cpu_core_type(const char *model);
> +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp);
> #endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support Bharata B Rao
@ 2016-04-04 4:27 ` David Gibson
2016-05-09 4:24 ` Bharata B Rao
0 siblings, 1 reply; 40+ messages in thread
From: David Gibson @ 2016-04-04 4:27 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, pbonzini, imammedo, afaerber,
mdroth
[-- Attachment #1: Type: text/plain, Size: 6939 bytes --]
On Thu, Mar 31, 2016 at 02:09:21PM +0530, Bharata B Rao wrote:
> Remove the CPU core device by removing the underlying CPU thread devices.
> Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
> notification to the guest. Release the vCPU object after CPU hot unplug so
> that vCPU fd can be parked and reused.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 16 ++++++++
> hw/ppc/spapr_cpu_core.c | 86 +++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 1 +
> include/hw/ppc/spapr_cpu_core.h | 11 ++++++
> 4 files changed, 114 insertions(+)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1a5dbd9..74cdcf2 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2348,11 +2348,27 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
> }
>
> +void spapr_cpu_destroy(PowerPCCPU *cpu)
> +{
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> +
> + xics_cpu_destroy(spapr->icp, cpu);
> + qemu_unregister_reset(spapr_cpu_reset, cpu);
> +}
> +
> static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> +
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> error_setg(errp, "Memory hot unplug not supported by sPAPR");
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + if (!smc->dr_cpu_enabled) {
> + error_setg(errp, "CPU hot unplug not supported on this machine");
> + return;
> + }
> + spapr_core_unplug(hotplug_dev, dev, errp);
> }
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index a9ba843..09a592e 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -119,6 +119,92 @@ void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> }
> }
>
> +static void spapr_cpu_core_cleanup(struct sPAPRCPUUnplugList *unplug_list)
> +{
> + sPAPRCPUUnplug *unplug, *next;
> + Object *cpu;
> +
> + QLIST_FOREACH_SAFE(unplug, unplug_list, node, next) {
> + cpu = unplug->cpu;
> + object_unparent(cpu);
Is there any danger in the fact that the cpu object is still in the
QOM tree until unparented here? My usual expectation would be that
you'd remove the object from the tree immediately, but defer the
actual free. But I'm a bit unclear on how QOM removals are supposed
to work.
> + QLIST_REMOVE(unplug, node);
> + g_free(unplug);
> + }
> +}
> +
> +static void spapr_add_cpu_to_unplug_list(Object *cpu,
> + struct sPAPRCPUUnplugList *unplug_list)
> +{
> + sPAPRCPUUnplug *unplug = g_malloc(sizeof(*unplug));
> +
> + unplug->cpu = cpu;
> + QLIST_INSERT_HEAD(unplug_list, unplug, node);
> +}
> +
> +static int spapr_cpu_release(Object *obj, void *opaque)
> +{
> + DeviceState *dev = DEVICE(obj);
> + CPUState *cs = CPU(dev);
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + struct sPAPRCPUUnplugList *unplug_list = opaque;
> +
> + spapr_cpu_destroy(cpu);
> + cpu_remove_sync(cs);
> +
> + /*
> + * We are still walking the core object's children list, and
> + * hence can't cleanup this CPU thread object just yet. Put
> + * it on a list for later removal.
> + */
> + spapr_add_cpu_to_unplug_list(obj, unplug_list);
> + return 0;
> +}
> +
> +static void spapr_core_release(DeviceState *dev)
> +{
> + struct sPAPRCPUUnplugList unplug_list;
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + CPUCore *cc = CPU_CORE(dev);
> + int smt = kvmppc_smt_threads();
> +
> + QLIST_INIT(&unplug_list);
> + object_child_foreach(OBJECT(dev), spapr_cpu_release, &unplug_list);
> + spapr_cpu_core_cleanup(&unplug_list);
> + spapr->cores[cc->core / smt] = NULL;
> +
> + g_free(core->threads);
> +}
> +
> +static void spapr_core_release_unparent(DeviceState *dev, void *opaque)
> +{
> + spapr_core_release(dev);
> + object_unparent(OBJECT(dev));
> +}
> +
> +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + PowerPCCPU *cpu = &core->threads[0];
> + int id = ppc_get_vcpu_dt_id(cpu);
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id);
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> +
> + g_assert(drc);
> +
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drck->detach(drc, dev, spapr_core_release_unparent, NULL, &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + spapr_hotplug_req_remove_by_index(drc);
> +}
> +
> static const TypeInfo spapr_cpu_core_type_info = {
> .name = TYPE_SPAPR_CPU_CORE,
> .parent = TYPE_CPU_CORE,
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 619db98..c5a4a15 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -590,6 +590,7 @@ void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> sPAPRMachineState *spapr);
> +void spapr_cpu_destroy(PowerPCCPU *cpu);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index 165af7c..9bc2e01 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -65,4 +65,15 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> char *spapr_get_cpu_core_type(const char *model);
> void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp);
> +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp);
> +
> +/* List to store unplugged CPU objects for cleanup during unplug */
> +typedef struct sPAPRCPUUnplug {
> + Object *cpu;
> + QLIST_ENTRY(sPAPRCPUUnplug) node;
> +} sPAPRCPUUnplug;
> +
> +QLIST_HEAD(sPAPRCPUUnplugList, sPAPRCPUUnplug);
> +
> #endif
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
` (11 preceding siblings ...)
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support Bharata B Rao
@ 2016-04-04 14:44 ` Igor Mammedov
2016-04-05 14:55 ` Bharata B Rao
12 siblings, 1 reply; 40+ messages in thread
From: Igor Mammedov @ 2016-04-04 14:44 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber,
david
On Thu, 31 Mar 2016 14:09:09 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> Hi,
>
> This is v2.1 of "Core based CPU hotplug for PowerPC sPAPR". v2 was
> posted here:
> https://lists.gnu.org/archive/html/qemu-ppc/2016-03/msg00201.html
>
> I am making this v2.1 instead of v3 as this introduces a change in
> device_add semantics and if we don't want to continue this, I will go
> back to the previous v2 semantics from v3 onwards. v2 had
> spapr-cpu-core device that served as CPU core device for all types of
> PowerPCCPU threads. In this version however, spapr-cpu-core becomes
> an abstract device and we will have different core devices for each
> of the PowerPCCPU type. So in this version, the hotplug semantics
> looks like this:
>
> (qemu) device_add POWER8-spapr-cpu-core,id=core2,core=16[,threads=4]
>
> cpu_model specification is gone as it becomes redundant with different
> core types for each CPU type. CPU core types are defined only for
> host, POWER7 and POWER8 models only (and not for their
> variants/aliases) yet.
>
> I have mostly taken care of all the review comments that I got for v2.
Could you rebase on top of current master, pls?
and fix compile error:
hw/cpu/core.c: In function ‘core_prop_set_core’:
hw/cpu/core.c:31:9: error: implicit declaration of function ‘error_propagate’ [-Werror=implicit-function-declaration]
error_propagate(errp, local_err);
>
> Major changes in this version (v2.1)
> ------------------------------------
> - Semantics change: spapr-cpu-core to POWER8-spapr-cpu-core and
> likewise
> - Included Igor's pre_plug patch.
>
> Bharata B Rao (10):
> exec: Remove cpu from cpus list during cpu_exec_exit()
> exec: Do vmstate unregistration from cpu_exec_exit()
> cpu: Add a sync version of cpu_remove()
> cpu: Abstract CPU core type
> spapr: Abstract CPU core device
> spapr: Add CPU type specific core devices
> spapr: convert boot CPUs into CPU core devices
> spapr: CPU hotplug support
> xics,xics_kvm: Handle CPU unplug correctly
> spapr: CPU hot unplug support
>
> Gu Zheng (1):
> cpu: Reclaim vCPU objects
>
> Igor Mammedov (1):
> qdev: hotplug: Introduce HotplugHandler.pre_plug() callback
>
> cpus.c | 51 +++++-
> exec.c | 41 ++++-
> hw/core/hotplug.c | 11 ++
> hw/core/qdev.c | 9 +-
> hw/cpu/Makefile.objs | 1 +
> hw/cpu/core.c | 87 ++++++++++
> hw/intc/xics.c | 14 ++
> hw/intc/xics_kvm.c | 8 +-
> hw/ppc/Makefile.objs | 1 +
> hw/ppc/spapr.c | 150 +++++++++++++++--
> hw/ppc/spapr_cpu_core.c | 349
> ++++++++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_events.c | 3 +
> hw/ppc/spapr_rtas.c | 24 +++
> include/hw/cpu/core.h | 31 ++++
> include/hw/hotplug.h | 14 +-
> include/hw/ppc/spapr.h | 6 +
> include/hw/ppc/spapr_cpu_core.h | 79 +++++++++
> include/hw/ppc/xics.h | 1 +
> include/qom/cpu.h | 18 +++
> include/sysemu/kvm.h | 1 +
> kvm-all.c | 57 ++++++-
> kvm-stub.c | 5 + 22 files changed, 932
> insertions(+), 29 deletions(-) create mode 100644 hw/cpu/core.c
> create mode 100644 hw/ppc/spapr_cpu_core.c
> create mode 100644 include/hw/cpu/core.h
> create mode 100644 include/hw/ppc/spapr_cpu_core.h
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR
2016-04-04 14:44 ` [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Igor Mammedov
@ 2016-04-05 14:55 ` Bharata B Rao
2016-04-05 18:40 ` Igor Mammedov
2016-04-05 21:58 ` Igor Mammedov
0 siblings, 2 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-04-05 14:55 UTC (permalink / raw)
To: Igor Mammedov
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber,
david
On Mon, Apr 04, 2016 at 04:44:27PM +0200, Igor Mammedov wrote:
> On Thu, 31 Mar 2016 14:09:09 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > Hi,
> >
> > This is v2.1 of "Core based CPU hotplug for PowerPC sPAPR". v2 was
> > posted here:
> > https://lists.gnu.org/archive/html/qemu-ppc/2016-03/msg00201.html
> >
> > I am making this v2.1 instead of v3 as this introduces a change in
> > device_add semantics and if we don't want to continue this, I will go
> > back to the previous v2 semantics from v3 onwards. v2 had
> > spapr-cpu-core device that served as CPU core device for all types of
> > PowerPCCPU threads. In this version however, spapr-cpu-core becomes
> > an abstract device and we will have different core devices for each
> > of the PowerPCCPU type. So in this version, the hotplug semantics
> > looks like this:
> >
> > (qemu) device_add POWER8-spapr-cpu-core,id=core2,core=16[,threads=4]
> >
> > cpu_model specification is gone as it becomes redundant with different
> > core types for each CPU type. CPU core types are defined only for
> > host, POWER7 and POWER8 models only (and not for their
> > variants/aliases) yet.
> >
> > I have mostly taken care of all the review comments that I got for v2.
> Could you rebase on top of current master, pls?
Sure, I usually update, but missed it this time.
>
> and fix compile error:
> hw/cpu/core.c: In function ‘core_prop_set_core’:
> hw/cpu/core.c:31:9: error: implicit declaration of function ‘error_propagate’ [-Werror=implicit-function-declaration]
> error_propagate(errp, local_err);
I don't see this when compiling all the targets either on x86 or PPC systems.
May be this is seen with latest master, in any case will check this out
when spinning next version.
However do you have any specific comment to be addressed from this version
before I post the next one ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR
2016-04-05 14:55 ` Bharata B Rao
@ 2016-04-05 18:40 ` Igor Mammedov
2016-04-05 21:58 ` Igor Mammedov
1 sibling, 0 replies; 40+ messages in thread
From: Igor Mammedov @ 2016-04-05 18:40 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber,
david
On Tue, 5 Apr 2016 20:25:57 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Mon, Apr 04, 2016 at 04:44:27PM +0200, Igor Mammedov wrote:
> > On Thu, 31 Mar 2016 14:09:09 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > Hi,
> > >
> > > This is v2.1 of "Core based CPU hotplug for PowerPC sPAPR". v2 was
> > > posted here:
> > > https://lists.gnu.org/archive/html/qemu-ppc/2016-03/msg00201.html
> > >
> > > I am making this v2.1 instead of v3 as this introduces a change in
> > > device_add semantics and if we don't want to continue this, I
> > > will go back to the previous v2 semantics from v3 onwards. v2 had
> > > spapr-cpu-core device that served as CPU core device for all
> > > types of PowerPCCPU threads. In this version however,
> > > spapr-cpu-core becomes an abstract device and we will have
> > > different core devices for each of the PowerPCCPU type. So in
> > > this version, the hotplug semantics looks like this:
> > >
> > > (qemu) device_add
> > > POWER8-spapr-cpu-core,id=core2,core=16[,threads=4]
> > >
> > > cpu_model specification is gone as it becomes redundant with
> > > different core types for each CPU type. CPU core types are
> > > defined only for host, POWER7 and POWER8 models only (and not for
> > > their variants/aliases) yet.
> > >
> > > I have mostly taken care of all the review comments that I got
> > > for v2.
> > Could you rebase on top of current master, pls?
>
> Sure, I usually update, but missed it this time.
>
> >
> > and fix compile error:
> > hw/cpu/core.c: In function ‘core_prop_set_core’:
> > hw/cpu/core.c:31:9: error: implicit declaration of function
> > ‘error_propagate’ [-Werror=implicit-function-declaration]
> > error_propagate(errp, local_err);
>
> I don't see this when compiling all the targets either on x86 or PPC
> systems. May be this is seen with latest master, in any case will
> check this out when spinning next version.
>
> However do you have any specific comment to be addressed from this
> version before I post the next one ?
Sorry, I haven't had a spare time to review it yet.
But device_add example looks good and similar what x86 will do,
which is good from libvirt point of view as it would be able to
implement hotplug in a uniform way for both targets.
>
> Regards,
> Bharata.
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR
2016-04-05 14:55 ` Bharata B Rao
2016-04-05 18:40 ` Igor Mammedov
@ 2016-04-05 21:58 ` Igor Mammedov
1 sibling, 0 replies; 40+ messages in thread
From: Igor Mammedov @ 2016-04-05 21:58 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, qemu-devel, agraf,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Tue, 5 Apr 2016 20:25:57 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Mon, Apr 04, 2016 at 04:44:27PM +0200, Igor Mammedov wrote:
> > On Thu, 31 Mar 2016 14:09:09 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > Hi,
> > >
> > > This is v2.1 of "Core based CPU hotplug for PowerPC sPAPR". v2 was
> > > posted here:
> > > https://lists.gnu.org/archive/html/qemu-ppc/2016-03/msg00201.html
> > >
> > > I am making this v2.1 instead of v3 as this introduces a change in
> > > device_add semantics and if we don't want to continue this, I
> > > will go back to the previous v2 semantics from v3 onwards. v2 had
> > > spapr-cpu-core device that served as CPU core device for all
> > > types of PowerPCCPU threads. In this version however,
> > > spapr-cpu-core becomes an abstract device and we will have
> > > different core devices for each of the PowerPCCPU type. So in
> > > this version, the hotplug semantics looks like this:
> > >
> > > (qemu) device_add
> > > POWER8-spapr-cpu-core,id=core2,core=16[,threads=4]
> > >
> > > cpu_model specification is gone as it becomes redundant with
> > > different core types for each CPU type. CPU core types are
> > > defined only for host, POWER7 and POWER8 models only (and not for
> > > their variants/aliases) yet.
> > >
> > > I have mostly taken care of all the review comments that I got
> > > for v2.
> > Could you rebase on top of current master, pls?
>
> Sure, I usually update, but missed it this time.
>
> >
> > and fix compile error:
> > hw/cpu/core.c: In function ‘core_prop_set_core’:
> > hw/cpu/core.c:31:9: error: implicit declaration of function
> > ‘error_propagate’ [-Werror=implicit-function-declaration]
> > error_propagate(errp, local_err);
>
> I don't see this when compiling all the targets either on x86 or PPC
> systems. May be this is seen with latest master, in any case will
> check this out when spinning next version.
Series also breaks 'make check'
gdb ./ppc64-softmmu/qemu-system-ppc64
Program received signal SIGSEGV, Segmentation fault.
object_class_get_name (klass=0x0) at qom/object.c:748
748 return klass->type->name;
(gdb) bt
#0 object_class_get_name (klass=0x0) at qom/object.c:748
#1 0x0000555555795aff in POWER7_spapr_cpu_core_create_threads (errp=0x7fffffffdc00, threads=0x1, dev=0x5555565bf930) at /home/imammedo/builds/qemu/hw/ppc/spapr_cpu_core.c:328
#2 POWER7_spapr_cpu_core_realize (dev=0x5555565bf930, errp=0x7fffffffdc80) at /home/imammedo/builds/qemu/hw/ppc/spapr_cpu_core.c:328
#3 0x00005555558af84c in device_set_realized (obj=0x5555565bf930, value=<optimized out>, errp=0x55555649fbd0 <error_fatal>) at hw/core/qdev.c:1074
#4 0x00005555559b542e in property_set_bool (obj=0x5555565bf930, v=<optimized out>, name=<optimized out>, opaque=0x5555565bfb20, errp=0x55555649fbd0 <error_fatal>)
at qom/object.c:1853
#5 0x00005555559b92b7 in object_property_set_qobject (obj=obj@entry=0x5555565bf930, value=value@entry=0x5555565bedd0, name=name@entry=0x555555ad403e "realized",
errp=errp@entry=0x55555649fbd0 <error_fatal>) at qom/qom-qobject.c:26
#6 0x00005555559b7130 in object_property_set_bool (obj=0x5555565bf930, value=<optimized out>, name=0x555555ad403e "realized", errp=0x55555649fbd0 <error_fatal>)
at qom/object.c:1150
#7 0x0000555555786fc1 in ppc_spapr_init (machine=0x55555659fe50) at /home/imammedo/builds/qemu/hw/ppc/spapr.c:1859
#8 0x00005555556e54af in main (argc=<optimized out>, argv=<optimized
out>, envp=<optimized out>) at vl.c:4513
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
2016-04-04 4:23 ` David Gibson
@ 2016-04-05 23:47 ` Michael Roth
2016-05-05 9:22 ` Bharata B Rao
2016-05-06 8:57 ` Igor Mammedov
2 siblings, 1 reply; 40+ messages in thread
From: Michael Roth @ 2016-04-05 23:47 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, afaerber, david
Quoting Bharata B Rao (2016-03-31 03:39:19)
> Set up device tree entries for the hotplugged CPU core and use the
> exising RTAS event logging infrastructure to send CPU hotplug notification
> to the guest.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 58 ++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_cpu_core.c | 70 +++++++++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_events.c | 3 ++
> hw/ppc/spapr_rtas.c | 24 ++++++++++++++
> include/hw/ppc/spapr.h | 2 ++
> include/hw/ppc/spapr_cpu_core.h | 2 ++
> 6 files changed, 159 insertions(+)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1ead043..1a5dbd9 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -603,6 +603,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> size_t page_sizes_prop_size;
> uint32_t vcpus_per_socket = smp_threads * smp_cores;
> uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + int drc_index;
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> + if (drc) {
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drc_index = drck->get_index(drc);
> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> + }
>
> /* Note: we keep CI large pages off for now because a 64K capable guest
> * provisioned with large pages might otherwise try to map a qemu
> @@ -987,6 +997,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> }
>
> + if (smc->dr_cpu_enabled) {
> + int offset = fdt_path_offset(fdt, "/cpus");
> + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> + SPAPR_DR_CONNECTOR_TYPE_CPU);
> + if (ret < 0) {
> + error_report("Couldn't set up CPU DR device tree properties");
> + exit(1);
> + }
> + }
> +
> _FDT((fdt_pack(fdt)));
>
> if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> @@ -1622,6 +1642,8 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
> + CPUState *cs = CPU(cpu);
> + int i;
>
> /* Set time-base frequency to 512 MHz */
> cpu_ppc_tb_init(env, TIMEBASE_FREQ);
> @@ -1646,6 +1668,14 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> }
> }
>
> + /* Set NUMA node for the added CPUs */
> + for (i = 0; i < nb_numa_nodes; i++) {
> + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> + cs->numa_node = i;
> + break;
> + }
> + }
> +
> xics_cpu_setup(spapr->icp, cpu);
>
> qemu_register_reset(spapr_cpu_reset, cpu);
> @@ -1825,6 +1855,11 @@ static void ppc_spapr_init(MachineState *machine)
>
> for (i = 0; i < spapr_max_cores; i++) {
> int core_dt_id = i * smt;
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_new(OBJECT(spapr),
> + SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
> +
> + qemu_register_reset(spapr_drc_reset, drc);
>
> if (i < spapr_cores) {
> char *type = spapr_get_cpu_core_type(machine->cpu_model);
> @@ -2247,6 +2282,27 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> + sPAPRMachineState *spapr)
> +{
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + void *fdt;
> + int offset, fdt_size;
> + char *nodename;
> +
> + fdt = create_device_tree(&fdt_size);
> + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> + offset = fdt_add_subnode(fdt, 0, nodename);
> +
> + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> + g_free(nodename);
> +
> + *fdt_offset = offset;
> + return fdt;
> +}
> +
> static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> @@ -2287,6 +2343,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
>
> spapr_memory_plug(hotplug_dev, dev, node, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_plug(hotplug_dev, dev, errp);
> }
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 640d143..a9ba843 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -18,6 +18,7 @@
> void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp)
> {
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> int spapr_max_cores = max_cpus / smp_threads;
> int index;
> @@ -25,6 +26,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error *local_err = NULL;
> CPUCore *cc = CPU_CORE(dev);
>
> + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> + error_setg(&local_err, "CPU hotplug not supported for this machine");
> + goto out;
> + }
> +
> if (cc->threads != smp_threads) {
> error_setg(&local_err, "threads must be %d", smp_threads);
> goto out;
> @@ -49,6 +55,70 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + CPUCore *cc = CPU_CORE(dev);
> + CPUState *cs = CPU(&core->threads[0]);
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> + void *fdt = NULL;
> + int fdt_offset = 0;
> + int index;
> + int smt = kvmppc_smt_threads();
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core);
> + index = cc->core / smt;
> + spapr->cores[index] = OBJECT(dev);
> +
> + if (!smc->dr_cpu_enabled) {
> + /*
> + * This is a cold plugged CPU core but the machine doesn't support
> + * DR. So skip the hotplug path ensuring that the core is brought
> + * up online with out an associated DR connector.
> + */
> + return;
> + }
> +
> + g_assert(drc);
> +
> + /*
> + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> + */
> + if (dev->hotplugged) {
> + fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
> + dev->hotplugged = true;
This doesn't seem necessary ^
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly Bharata B Rao
@ 2016-04-06 0:24 ` Michael Roth
2016-04-06 0:43 ` David Gibson
0 siblings, 1 reply; 40+ messages in thread
From: Michael Roth @ 2016-04-06 0:24 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: mjrosato, thuth, pkrempa, ehabkost, aik, armbru, agraf,
borntraeger, qemu-ppc, pbonzini, imammedo, afaerber, david
Quoting Bharata B Rao (2016-03-31 03:39:20)
> XICS is setup for each CPU during initialization. Provide a routine
> to undo the same when CPU is unplugged. While here, move ss->cs management
> into xics from xics_kvm since there is nothing KVM specific in it.
> Also ensure xics reset doesn't set irq for CPUs that are already unplugged.
>
> This allows reboot of a VM that has undergone CPU hotplug and unplug
> to work correctly.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> hw/intc/xics.c | 14 ++++++++++++++
> hw/intc/xics_kvm.c | 8 ++++----
> include/hw/ppc/xics.h | 1 +
> 3 files changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> index 213a370..9fdb551 100644
> --- a/hw/intc/xics.c
> +++ b/hw/intc/xics.c
> @@ -45,6 +45,18 @@ static int get_cpu_index_by_dt_id(int cpu_dt_id)
> return -1;
> }
>
> +void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu)
> +{
> + CPUState *cs = CPU(cpu);
> + ICPState *ss = &icp->ss[cs->cpu_index];
> +
If the following assertion is false, I think you'd get an OOB before you
reached it due to the assignment above.
> + assert(cs->cpu_index < icp->nr_servers);
> + assert(cs == ss->cs);
> +
> + ss->output = NULL;
> + ss->cs = NULL;
> +}
> +
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly
2016-04-06 0:24 ` Michael Roth
@ 2016-04-06 0:43 ` David Gibson
2016-04-08 23:40 ` Michael Roth
0 siblings, 1 reply; 40+ messages in thread
From: David Gibson @ 2016-04-06 0:43 UTC (permalink / raw)
To: Michael Roth
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, armbru,
qemu-devel, borntraeger, pbonzini, qemu-ppc, Bharata B Rao,
imammedo, afaerber
[-- Attachment #1: Type: text/plain, Size: 1849 bytes --]
On Tue, Apr 05, 2016 at 07:24:39PM -0500, Michael Roth wrote:
> Quoting Bharata B Rao (2016-03-31 03:39:20)
> > XICS is setup for each CPU during initialization. Provide a routine
> > to undo the same when CPU is unplugged. While here, move ss->cs management
> > into xics from xics_kvm since there is nothing KVM specific in it.
> > Also ensure xics reset doesn't set irq for CPUs that are already unplugged.
> >
> > This allows reboot of a VM that has undergone CPU hotplug and unplug
> > to work correctly.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > ---
> > hw/intc/xics.c | 14 ++++++++++++++
> > hw/intc/xics_kvm.c | 8 ++++----
> > include/hw/ppc/xics.h | 1 +
> > 3 files changed, 19 insertions(+), 4 deletions(-)
> >
> > diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> > index 213a370..9fdb551 100644
> > --- a/hw/intc/xics.c
> > +++ b/hw/intc/xics.c
> > @@ -45,6 +45,18 @@ static int get_cpu_index_by_dt_id(int cpu_dt_id)
> > return -1;
> > }
> >
> > +void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu)
> > +{
> > + CPUState *cs = CPU(cpu);
> > + ICPState *ss = &icp->ss[cs->cpu_index];
> > +
>
> If the following assertion is false, I think you'd get an OOB before you
> reached it due to the assignment above.
It should be ok. The assignment above will compute an out of bounds
address, but it doesn't dereference it.
> > + assert(cs->cpu_index < icp->nr_servers);
> > + assert(cs == ss->cs);
> > +
> > + ss->output = NULL;
> > + ss->cs = NULL;
> > +}
> > +
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
2016-04-01 5:08 ` David Gibson
2016-04-04 0:16 ` David Gibson
@ 2016-04-08 23:35 ` Michael Roth
2 siblings, 0 replies; 40+ messages in thread
From: Michael Roth @ 2016-04-08 23:35 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: qemu-ppc, afaerber, david, imammedo, armbru, thuth, aik, agraf,
pbonzini, ehabkost, pkrempa, eblake, mjrosato, borntraeger
Quoting Bharata B Rao (2016-03-31 03:39:17)
> Introduce core devices for each CPU type supported by sPAPR. These
> core devices are derived from the base spapr-cpu-core device type.
>
> TODO:
> - Add core types for other remaining CPU types
> - Handle CPU model alias correctly
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 3 +-
> hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 1 +
> include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> 4 files changed, 156 insertions(+), 2 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 64c4acc..45ac5dc 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> machine->boot_order = g_strdup(boot_device);
> }
>
> -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> - Error **errp)
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 8cbe2a5..3751a54 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> .instance_size = sizeof(sPAPRCPUCore),
> };
>
> +#define DEFINE_SPAPR_CPU_CORE(_name) \
> +static void \
> +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> + Error **errp) \
> +{ \
> + int i; \
> + Error *local_err = NULL; \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + glue(_name, sPAPRCPUCore) * core = \
> + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> + \
> + for (i = 0; i < threads; i++) { \
> + char id[32]; \
> + \
> + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> + object_class_get_name(core->cpu)); \
> + snprintf(id, sizeof(id), "thread[%d]", i); \
> + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> + &local_err); \
> + if (local_err) { \
> + goto err; \
> + } \
> + } \
> + return; \
> + \
> +err: \
> + while (--i) { \
> + object_unparent(OBJECT(&sc->threads[i])); \
> + } \
> + error_propagate(errp, local_err); \
> +} \
> + \
> +static int \
> +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> +{ \
> + Error **errp = opaque; \
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> + CPUState *cs = CPU(child); \
> + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> + \
> + object_property_set_bool(child, true, "realized", errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + \
> + spapr_cpu_init(spapr, cpu, errp); \
> + if (*errp) { \
> + return 1; \
> + } \
> + return 0; \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> +{ \
> + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> + Error *local_err = NULL; \
> + \
> + /* \
> + * TODO: This is CPU model specific CPU core's realize routine. \
> + * However I am initializing "threads" field of the parent type \
> + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> + * part of CPU model specific CPU core type and have different plug() \
> + * handlers for each type instead of having a common plug() handler \
> + * for all core types. \
> + */ \
Not sure, but there seems to be precedence for descendants of CPUClass
at least. e.g. arm_cpu_realizefn() modifies it's parent CPUState.
But do we even need a model-specific realize? It seems like once
core->cpu is set via the model's instance_init, the same realize /
create_threads functions can be shared by everyone thanks to
object_class_get_name(). So maybe we could move ObjectClass *cpu up into
sPAPRCPUCore, and use a common realize set via spapr_cpu_core_class_init?
If that's doable, we could drop all the macros, or at least limit them
to basically just a model-specific instance_init that sets
sPAPRCPUCore->cpu. You might not even need typedefs for the different
models at that point, since instance_size == sizeof(sPAPRCPUCore) for
everyone at that point. Currently, at least.
In fact, arm_cpu_register_types() seems to do something very similar
for registering TypeInfos for tuples of (model name, init fn).
> + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> + if (local_err) { \
> + goto out; \
> + } \
> + \
> + object_child_foreach(OBJECT(dev), \
> + glue(_name, _spapr_cpu_core_realize_child), \
> + &local_err); \
> + \
> +out: \
> + if (local_err) { \
> + g_free(sc->threads); \
> + error_propagate(errp, local_err); \
> + } \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> +{ \
> + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> + ObjectClass *oc = object_class_by_name(type); \
> + \
> + core->cpu = oc; \
> +} \
> + \
> +static void \
> +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> +{ \
> + \
> + DeviceClass *dc = DEVICE_CLASS(oc); \
> + dc->realize = glue(_name, _spapr_cpu_core_realize); \
> +} \
> + \
> +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> +{ \
> + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> + .parent = TYPE_SPAPR_CPU_CORE, \
> + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> +};
> +
> +DEFINE_SPAPR_CPU_CORE(host);
> +DEFINE_SPAPR_CPU_CORE(POWER7);
> +DEFINE_SPAPR_CPU_CORE(POWER8);
> +
> static void spapr_cpu_core_register_types(void)
> {
> type_register_static(&spapr_cpu_core_type_info);
> + type_register_static(&host_spapr_cpu_core_type_info);
> + type_register_static(&POWER7_spapr_cpu_core_type_info);
> + type_register_static(&POWER8_spapr_cpu_core_type_info);
> }
>
> type_init(spapr_cpu_core_register_types)
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 098d85d..0fdf448 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index e3340ea..71e69c0 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> PowerPCCPU *threads;
> } sPAPRCPUCore;
>
> +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> +#define host_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> +
> +typedef struct hostsPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} hostsPAPRCPUCore;
> +
> +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> +#define POWER7_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> +
> +typedef struct POWER7sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER7sPAPRCPUCore;
> +
> +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> +#define POWER8_SPAPR_CPU_CORE(obj) \
> + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> +
> +typedef struct POWER8sPAPRCPUCore {
> + /*< private >*/
> + sPAPRCPUCore parent_obj;
> +
> + /*< public >*/
> + ObjectClass *cpu;
> +} POWER8sPAPRCPUCore;
> +
> #endif
> --
> 2.1.0
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU core devices
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU " Bharata B Rao
2016-04-01 5:12 ` David Gibson
@ 2016-04-08 23:35 ` Michael Roth
2016-05-05 9:19 ` Bharata B Rao
1 sibling, 1 reply; 40+ messages in thread
From: Michael Roth @ 2016-04-08 23:35 UTC (permalink / raw)
To: Bharata B Rao, qemu-devel
Cc: qemu-ppc, afaerber, david, imammedo, armbru, thuth, aik, agraf,
pbonzini, ehabkost, pkrempa, eblake, mjrosato, borntraeger
Quoting Bharata B Rao (2016-03-31 03:39:18)
> Introduce sPAPRMachineClass.dr_cpu_enabled to indicate support for
> CPU core hotplug. Initialize boot time CPUs as core deivces and prevent
> topologies that result in partially filled cores. Both of these are done
> only if CPU core hotplug is supported.
>
> Note: An unrelated change in the call to xics_system_init() is done
> in this patch as it makes sense to use the local variable smt introduced
> in this patch instead of kvmppc_smt_threads() call here.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 73 +++++++++++++++++++++++++++++++++++------
> hw/ppc/spapr_cpu_core.c | 45 +++++++++++++++++++++++++
> include/hw/ppc/spapr.h | 2 ++
> include/hw/ppc/spapr_cpu_core.h | 3 ++
> 4 files changed, 113 insertions(+), 10 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 45ac5dc..1ead043 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -64,6 +64,7 @@
>
> #include "hw/compat.h"
> #include "qemu-common.h"
> +#include "hw/ppc/spapr_cpu_core.h"
>
> #include <libfdt.h>
>
> @@ -1614,6 +1615,10 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> machine->boot_order = g_strdup(boot_device);
> }
>
> +/*
> + * TODO: Check if some of these can be moved to rtas_start_cpu() where
> + * a few other things required for hotplugged CPUs are being done.
> + */
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
> @@ -1644,6 +1649,7 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> xics_cpu_setup(spapr->icp, cpu);
>
> qemu_register_reset(spapr_cpu_reset, cpu);
> + spapr_cpu_reset(cpu);
> }
>
> /*
> @@ -1727,7 +1733,6 @@ static void ppc_spapr_init(MachineState *machine)
> const char *kernel_filename = machine->kernel_filename;
> const char *kernel_cmdline = machine->kernel_cmdline;
> const char *initrd_filename = machine->initrd_filename;
> - PowerPCCPU *cpu;
> PCIHostState *phb;
> int i;
> MemoryRegion *sysmem = get_system_memory();
> @@ -1741,6 +1746,22 @@ static void ppc_spapr_init(MachineState *machine)
> long load_limit, fw_size;
> bool kernel_le = false;
> char *filename;
> + int smt = kvmppc_smt_threads();
> + int spapr_cores = smp_cpus / smp_threads;
> + int spapr_max_cores = max_cpus / smp_threads;
> +
> + if (smc->dr_cpu_enabled) {
> + if (smp_cpus % smp_threads) {
> + error_report("smp_cpus (%u) must be multiple of threads (%u)",
> + smp_cpus, smp_threads);
> + exit(1);
> + }
> + if (max_cpus % smp_threads) {
> + error_report("max_cpus (%u) must be multiple of threads (%u)",
> + max_cpus, smp_threads);
> + exit(1);
> + }
> + }
>
> msi_supported = true;
>
> @@ -1787,8 +1808,7 @@ static void ppc_spapr_init(MachineState *machine)
>
> /* Set up Interrupt Controller before we create the VCPUs */
> spapr->icp = xics_system_init(machine,
> - DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(),
> - smp_threads),
> + DIV_ROUND_UP(max_cpus * smt, smp_threads),
> XICS_IRQS, &error_fatal);
>
> if (smc->dr_lmb_enabled) {
> @@ -1799,13 +1819,34 @@ static void ppc_spapr_init(MachineState *machine)
> if (machine->cpu_model == NULL) {
> machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> }
> - for (i = 0; i < smp_cpus; i++) {
> - cpu = cpu_ppc_init(machine->cpu_model);
> - if (cpu == NULL) {
> - error_report("Unable to find PowerPC CPU definition");
> - exit(1);
> +
> + if (smc->dr_cpu_enabled) {
> + spapr->cores = g_new0(Object *, spapr_max_cores);
> +
> + for (i = 0; i < spapr_max_cores; i++) {
> + int core_dt_id = i * smt;
> +
> + if (i < spapr_cores) {
Is there any reason to not just have the for() loop stop at spapr_cores?
Maybe I missed something in the subsequent patches, but it seems like we
never end up doing anything beyond i < spapr_cores.
> + char *type = spapr_get_cpu_core_type(machine->cpu_model);
> + Object *core = object_new(type);
> +
> + g_free(type);
> + object_property_set_int(core, smp_threads, "threads",
> + &error_fatal);
> + object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE,
> + &error_fatal);
> + object_property_set_bool(core, true, "realized", &error_fatal);
> + }
> }
> - spapr_cpu_init(spapr, cpu, &error_fatal);
> + } else {
> + for (i = 0; i < smp_cpus; i++) {
> + PowerPCCPU *cpu = cpu_ppc_init(machine->cpu_model);
> + if (cpu == NULL) {
> + error_report("Unable to find PowerPC CPU definition");
> + exit(1);
> + }
> + spapr_cpu_init(spapr, cpu, &error_fatal);
Is there anything that prevents us using core modeling in the non-DR
case as well? Having management of invididual threads/spapr_cpu_init
contained in within one place seems beneficial beyond hotplug. Is it
because cores are currently limited to cpus == "threads" via pre_plug
hook? Maybe it makes sense to loosen that restriction in the special
case where DR isn't enabled?
> + }
> }
>
> if (kvm_enabled()) {
> @@ -2257,10 +2298,19 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> }
> }
>
> +static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
> + DeviceState *dev, Error **errp)
> +{
> + if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_pre_plug(hotplug_dev, dev, errp);
> + }
> +}
> +
> static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine,
> DeviceState *dev)
> {
> - if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> return HOTPLUG_HANDLER(machine);
> }
> return NULL;
> @@ -2299,11 +2349,13 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> mc->has_dynamic_sysbus = true;
> mc->pci_allow_0_address = true;
> mc->get_hotplug_handler = spapr_get_hotpug_handler;
> + hc->pre_plug = spapr_machine_device_pre_plug;
> hc->plug = spapr_machine_device_plug;
> hc->unplug = spapr_machine_device_unplug;
> mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
>
> smc->dr_lmb_enabled = true;
> + smc->dr_cpu_enabled = true;
> fwc->get_dev_path = spapr_get_fw_dev_path;
> nc->nmi_monitor_handler = spapr_nmi;
> }
> @@ -2383,6 +2435,7 @@ static void spapr_machine_2_5_class_options(MachineClass *mc)
>
> spapr_machine_2_6_class_options(mc);
> smc->use_ohci_by_default = true;
> + smc->dr_cpu_enabled = false;
> SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_5);
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 3751a54..640d143 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -15,6 +15,40 @@
> #include <sysemu/cpus.h>
> #include "target-ppc/kvm_ppc.h"
>
> +void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + int spapr_max_cores = max_cpus / smp_threads;
> + int index;
> + int smt = kvmppc_smt_threads();
> + Error *local_err = NULL;
> + CPUCore *cc = CPU_CORE(dev);
> +
> + if (cc->threads != smp_threads) {
> + error_setg(&local_err, "threads must be %d", smp_threads);
> + goto out;
> + }
> +
> + if (cc->core % smt) {
> + error_setg(&local_err, "invalid core id %d\n", cc->core);
> + goto out;
> + }
> +
> + index = cc->core / smt;
> + if (index < 0 || index >= spapr_max_cores) {
> + error_setg(&local_err, "core id %d out of range", cc->core);
> + goto out;
> + }
> +
> + if (spapr->cores[index]) {
> + error_setg(&local_err, "core %d already populated", cc->core);
> + }
> +
> +out:
> + error_propagate(errp, local_err);
> +}
> +
> static const TypeInfo spapr_cpu_core_type_info = {
> .name = TYPE_SPAPR_CPU_CORE,
> .parent = TYPE_CPU_CORE,
> @@ -146,3 +180,14 @@ static void spapr_cpu_core_register_types(void)
> }
>
> type_init(spapr_cpu_core_register_types)
> +
> +/*
> + * TODO: Looks fragile :(
> + */
> +char *spapr_get_cpu_core_type(const char *model)
> +{
> + char core_type[32];
> +
> + snprintf(core_type, 32, "%s-%s", model, TYPE_SPAPR_CPU_CORE);
> + return g_strdup(core_type);
> +}
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 0fdf448..a6956c0 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -36,6 +36,7 @@ struct sPAPRMachineClass {
>
> /*< public >*/
> bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */
> + bool dr_cpu_enabled; /* enable dynamic-reconfig/hotplug of CPUs */
> bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */
> };
>
> @@ -79,6 +80,7 @@ struct sPAPRMachineState {
> /*< public >*/
> char *kvm_type;
> MemoryHotplugState hotplug_memory;
> + Object **cores;
> };
>
> #define H_SUCCESS 0
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index 71e69c0..f08f291 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -60,4 +60,7 @@ typedef struct POWER8sPAPRCPUCore {
> ObjectClass *cpu;
> } POWER8sPAPRCPUCore;
>
> +void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp);
> +char *spapr_get_cpu_core_type(const char *model);
> #endif
> --
> 2.1.0
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly
2016-04-06 0:43 ` David Gibson
@ 2016-04-08 23:40 ` Michael Roth
0 siblings, 0 replies; 40+ messages in thread
From: Michael Roth @ 2016-04-08 23:40 UTC (permalink / raw)
To: David Gibson
Cc: Bharata B Rao, qemu-devel, qemu-ppc, afaerber, imammedo, armbru,
thuth, aik, agraf, pbonzini, ehabkost, pkrempa, eblake, mjrosato,
borntraeger
Quoting David Gibson (2016-04-05 19:43:46)
> On Tue, Apr 05, 2016 at 07:24:39PM -0500, Michael Roth wrote:
> > Quoting Bharata B Rao (2016-03-31 03:39:20)
> > > XICS is setup for each CPU during initialization. Provide a routine
> > > to undo the same when CPU is unplugged. While here, move ss->cs management
> > > into xics from xics_kvm since there is nothing KVM specific in it.
> > > Also ensure xics reset doesn't set irq for CPUs that are already unplugged.
> > >
> > > This allows reboot of a VM that has undergone CPU hotplug and unplug
> > > to work correctly.
> > >
> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > > ---
> > > hw/intc/xics.c | 14 ++++++++++++++
> > > hw/intc/xics_kvm.c | 8 ++++----
> > > include/hw/ppc/xics.h | 1 +
> > > 3 files changed, 19 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> > > index 213a370..9fdb551 100644
> > > --- a/hw/intc/xics.c
> > > +++ b/hw/intc/xics.c
> > > @@ -45,6 +45,18 @@ static int get_cpu_index_by_dt_id(int cpu_dt_id)
> > > return -1;
> > > }
> > >
> > > +void xics_cpu_destroy(XICSState *icp, PowerPCCPU *cpu)
> > > +{
> > > + CPUState *cs = CPU(cpu);
> > > + ICPState *ss = &icp->ss[cs->cpu_index];
> > > +
> >
> > If the following assertion is false, I think you'd get an OOB before you
> > reached it due to the assignment above.
>
> It should be ok. The assignment above will compute an out of bounds
> address, but it doesn't dereference it.
Ahh, right. I was thinking there was an actual access for some reason.
I might still prefer moving the assignment after the assert from a
stylistic standpoint, in case other variables get introduced in the
future that rely on *ss. Just a nit though.
>
> > > + assert(cs->cpu_index < icp->nr_servers);
> > > + assert(cs == ss->cs);
> > > +
> > > + ss->output = NULL;
> > > + ss->cs = NULL;
> > > +}
> > > +
> >
>
> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices
2016-04-04 0:13 ` David Gibson
@ 2016-04-09 2:21 ` Michael Roth
0 siblings, 0 replies; 40+ messages in thread
From: Michael Roth @ 2016-04-09 2:21 UTC (permalink / raw)
To: David Gibson, Bharata B Rao
Cc: qemu-devel, qemu-ppc, afaerber, imammedo, armbru, thuth, aik,
agraf, pbonzini, ehabkost, pkrempa, eblake, mjrosato,
borntraeger
Quoting David Gibson (2016-04-03 19:13:54)
> On Fri, Apr 01, 2016 at 11:42:23AM +0530, Bharata B Rao wrote:
> > On Fri, Apr 01, 2016 at 04:08:44PM +1100, David Gibson wrote:
> > > On Thu, Mar 31, 2016 at 02:09:17PM +0530, Bharata B Rao wrote:
> > > > Introduce core devices for each CPU type supported by sPAPR. These
> > > > core devices are derived from the base spapr-cpu-core device type.
> > > >
> > > > TODO:
> > > > - Add core types for other remaining CPU types
> > > > - Handle CPU model alias correctly
> > > >
> > > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > > > ---
> > > > hw/ppc/spapr.c | 3 +-
> > > > hw/ppc/spapr_cpu_core.c | 118 ++++++++++++++++++++++++++++++++++++++++
> > > > include/hw/ppc/spapr.h | 1 +
> > > > include/hw/ppc/spapr_cpu_core.h | 36 ++++++++++++
> > > > 4 files changed, 156 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > index 64c4acc..45ac5dc 100644
> > > > --- a/hw/ppc/spapr.c
> > > > +++ b/hw/ppc/spapr.c
> > > > @@ -1614,8 +1614,7 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> > > > machine->boot_order = g_strdup(boot_device);
> > > > }
> > > >
> > > > -static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> > > > - Error **errp)
> > > > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > > > {
> > > > CPUPPCState *env = &cpu->env;
> > > >
> > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > > > index 8cbe2a5..3751a54 100644
> > > > --- a/hw/ppc/spapr_cpu_core.c
> > > > +++ b/hw/ppc/spapr_cpu_core.c
> > > > @@ -22,9 +22,127 @@ static const TypeInfo spapr_cpu_core_type_info = {
> > > > .instance_size = sizeof(sPAPRCPUCore),
> > > > };
> > > >
> > > > +#define DEFINE_SPAPR_CPU_CORE(_name) \
> > > > +static void \
> > > > +glue(_name, _spapr_cpu_core_create_threads)(DeviceState *dev, int threads, \
> > > > + Error **errp) \
> > > > +{ \
> > > > + int i; \
> > > > + Error *local_err = NULL; \
> > > > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > > > + glue(_name, sPAPRCPUCore) * core = \
> > > > + glue(_name, _SPAPR_CPU_CORE)(OBJECT(dev)); \
> > > > + \
> > > > + for (i = 0; i < threads; i++) { \
> > > > + char id[32]; \
> > > > + \
> > > > + object_initialize(&sc->threads[i], sizeof(sc->threads[i]), \
> > > > + object_class_get_name(core->cpu)); \
> > > > + snprintf(id, sizeof(id), "thread[%d]", i); \
> > > > + object_property_add_child(OBJECT(core), id, OBJECT(&sc->threads[i]), \
> > > > + &local_err); \
> > > > + if (local_err) { \
> > > > + goto err; \
> > > > + } \
> > > > + } \
> > > > + return; \
> > > > + \
> > > > +err: \
> > > > + while (--i) { \
> > > > + object_unparent(OBJECT(&sc->threads[i])); \
> > > > + } \
> > > > + error_propagate(errp, local_err); \
> > > > +} \
> > > > + \
> > > > +static int \
> > > > +glue(_name, _spapr_cpu_core_realize_child)(Object *child, void *opaque) \
> > > > +{ \
> > > > + Error **errp = opaque; \
> > > > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); \
> > > > + CPUState *cs = CPU(child); \
> > > > + PowerPCCPU *cpu = POWERPC_CPU(cs); \
> > > > + \
> > > > + object_property_set_bool(child, true, "realized", errp); \
> > > > + if (*errp) { \
> > > > + return 1; \
> > > > + } \
> > > > + \
> > > > + spapr_cpu_init(spapr, cpu, errp); \
> > > > + if (*errp) { \
> > > > + return 1; \
> > > > + } \
> > > > + return 0; \
> > > > +} \
> > >
> > > If you put the ObjectClass * for the threads in the base abstract
> > > class's class structure, then you can move most of this logic to the
> > > abstract class as well and make the macro-ized stuff much smaller.
> > >
> > > The realize_child stuff doesn't even need the ObjectClass* in the base
> > > class to factor out.
> > >
> > > > +static void \
> > > > +glue(_name, _spapr_cpu_core_realize)(DeviceState *dev, Error **errp) \
> > > > +{ \
> > > > + sPAPRCPUCore *sc = SPAPR_CPU_CORE(OBJECT(dev)); \
> > > > + CPUCore *cc = CPU_CORE(OBJECT(dev)); \
> > > > + Error *local_err = NULL; \
> > > > + \
> > > > + /* \
> > > > + * TODO: This is CPU model specific CPU core's realize routine. \
> > > > + * However I am initializing "threads" field of the parent type \
> > > > + * sPAPRCPUCore here. Is this ok ? If not I will have make "threads" \
> > > > + * part of CPU model specific CPU core type and have different plug() \
> > > > + * handlers for each type instead of having a common plug() handler \
> > > > + * for all core types. \
> > > > + */ \
> > > > + sc->threads = g_new0(PowerPCCPU, cc->threads); \
> > > > + glue(_name, _spapr_cpu_core_create_threads)(dev, cc->threads, &local_err); \
> > > > + if (local_err) { \
> > > > + goto out; \
> > > > + } \
> > > > + \
> > > > + object_child_foreach(OBJECT(dev), \
> > > > + glue(_name, _spapr_cpu_core_realize_child), \
> > > > + &local_err); \
> > > > + \
> > > > +out: \
> > > > + if (local_err) { \
> > > > + g_free(sc->threads); \
> > > > + error_propagate(errp, local_err); \
> > > > + } \
> > > > +} \
> > > > + \
> > > > +static void \
> > > > +glue(_name, _spapr_cpu_core_instance_init)(Object *obj) \
> > > > +{ \
> > > > + glue(_name, sPAPRCPUCore) * core = glue(_name, _SPAPR_CPU_CORE)(obj); \
> > > > + const char *type = stringify(_name) "-" TYPE_POWERPC_CPU; \
> > > > + ObjectClass *oc = object_class_by_name(type); \
> > > > + \
> > > > + core->cpu = oc; \
> > > > +} \
> > > > + \
> > > > +static void \
> > > > +glue(_name, _spapr_cpu_core_class_init)(ObjectClass *oc, void *data) \
> > > > +{ \
> > > > + \
> > > > + DeviceClass *dc = DEVICE_CLASS(oc); \
> > > > + dc->realize = glue(_name, _spapr_cpu_core_realize); \
> > >
> > > I think the only callback you should need to construct in the macro is
> > > class_init to initialize the ObjectClass* field.
> > >
> > > > +} \
> > > > + \
> > > > +static const TypeInfo glue(_name, _spapr_cpu_core_type_info) = \
> > > > +{ \
> > > > + .name = stringify(_name) "-" TYPE_SPAPR_CPU_CORE, \
> > > > + .parent = TYPE_SPAPR_CPU_CORE, \
> > > > + .instance_size = sizeof(glue(_name, sPAPRCPUCore)), \
> > > > + .instance_init = glue(_name, _spapr_cpu_core_instance_init), \
> > > > + .class_init = glue(_name, _spapr_cpu_core_class_init), \
> > > > +};
> > > > +
> > > > +DEFINE_SPAPR_CPU_CORE(host);
> > > > +DEFINE_SPAPR_CPU_CORE(POWER7);
> > > > +DEFINE_SPAPR_CPU_CORE(POWER8);
> > > > +
> > > > static void spapr_cpu_core_register_types(void)
> > > > {
> > > > type_register_static(&spapr_cpu_core_type_info);
> > > > + type_register_static(&host_spapr_cpu_core_type_info);
> > > > + type_register_static(&POWER7_spapr_cpu_core_type_info);
> > > > + type_register_static(&POWER8_spapr_cpu_core_type_info);
> > > > }
> > > >
> > > > type_init(spapr_cpu_core_register_types)
> > > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > > > index 098d85d..0fdf448 100644
> > > > --- a/include/hw/ppc/spapr.h
> > > > +++ b/include/hw/ppc/spapr.h
> > > > @@ -585,6 +585,7 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> > > > uint32_t count);
> > > > void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> > > > uint32_t count);
> > > > +void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> > > >
> > > > /* rtas-configure-connector state */
> > > > struct sPAPRConfigureConnectorState {
> > > > diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> > > > index e3340ea..71e69c0 100644
> > > > --- a/include/hw/ppc/spapr_cpu_core.h
> > > > +++ b/include/hw/ppc/spapr_cpu_core.h
> > > > @@ -24,4 +24,40 @@ typedef struct sPAPRCPUCore {
> > > > PowerPCCPU *threads;
> > > > } sPAPRCPUCore;
> > > >
> > > > +#define TYPE_host_SPAPR_CPU_CORE "host-spapr-cpu-core"
> > > > +#define host_SPAPR_CPU_CORE(obj) \
> > > > + OBJECT_CHECK(hostsPAPRCPUCore, (obj), TYPE_host_SPAPR_CPU_CORE)
> > > > +
> > > > +typedef struct hostsPAPRCPUCore {
> > > > + /*< private >*/
> > > > + sPAPRCPUCore parent_obj;
> > > > +
> > > > + /*< public >*/
> > > > + ObjectClass *cpu;
> > > > +} hostsPAPRCPUCore;
> > > > +#define TYPE_POWER7_SPAPR_CPU_CORE "POWER7-spapr-cpu-core"
> > > > +#define POWER7_SPAPR_CPU_CORE(obj) \
> > > > + OBJECT_CHECK(POWER7sPAPRCPUCore, (obj), TYPE_POWER7_SPAPR_CPU_CORE)
> > > > +
> > > > +typedef struct POWER7sPAPRCPUCore {
> > > > + /*< private >*/
> > > > + sPAPRCPUCore parent_obj;
> > > > +
> > > > + /*< public >*/
> > > > + ObjectClass *cpu;
> > > > +} POWER7sPAPRCPUCore;
> > > > +
> > > > +#define TYPE_POWER8_SPAPR_CPU_CORE "POWER8-spapr-cpu-core"
> > > > +#define POWER8_SPAPR_CPU_CORE(obj) \
> > > > + OBJECT_CHECK(POWER8sPAPRCPUCore, (obj), TYPE_POWER8_SPAPR_CPU_CORE)
> > > > +
> > > > +typedef struct POWER8sPAPRCPUCore {
> > > > + /*< private >*/
> > > > + sPAPRCPUCore parent_obj;
> > > > +
> > > > + /*< public >*/
> > > > + ObjectClass *cpu;
> > > > +} POWER8sPAPRCPUCore;
> > >
> > > These are all identical so should also be macro constructed as well.
> > > I don't think there's actually any need for the structures to be
> > > exposed in a header file either, so you should be able to do it in the
> > > same macro that constructs the implementation.
> > >
> > > Uh.. except if you move the ObjectClass* to the base class you won't
> > > even need these.
> >
> > The only reason (currently) POWER8sPAPRCPUCore exists separately from
> > the base class sPAPRCPUCore is that it represents POWER8 core which
> > is stored as ObjectClass*.
> >
> > Now if we don't track cpu type (ObjectClass *) as part of
> > POWER8sPAPRCPUCore but push that up to sPAPRCPUCore, I am not
> > sure if that would be at the right abstraction level.
>
> Not as part of sPAPRCPUCore, but as part of sPAPRCPUCoreClass. We
> don't have a structure for the class at present, but we can add one.
> You can think of it as a method that sPAPRCPUCore and subclasses have
> which returns the correct ObjectClass *, except that we don't actually
> need a method (function pointer) - a simple data pointer in the class
> will suffice.
Sorry, I'd missed this thread when I posted my other reply.
I agree it should be pushed up a level so we can drop most of the
macro stuff. arm_cpu_register_types() seems to provide a fairly
straightforward example of defining derivative types by modifying
parent Object's state via their instance_init functions.
But it does seem unecessary in this case to do it for every
instance, since that state (ObjectClass *cpu) will be
common/immutable for all instances.
So I think both approaches are workable and acceptable, but doing
it your way vs. what I suggested in my other reply does seem like
the nicer way to handle it.
>
> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU core devices
2016-04-08 23:35 ` Michael Roth
@ 2016-05-05 9:19 ` Bharata B Rao
0 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-05-05 9:19 UTC (permalink / raw)
To: Michael Roth
Cc: qemu-devel, qemu-ppc, afaerber, david, imammedo, armbru, thuth,
aik, agraf, pbonzini, ehabkost, pkrempa, eblake, mjrosato,
borntraeger
On Fri, Apr 08, 2016 at 06:35:19PM -0500, Michael Roth wrote:
> Quoting Bharata B Rao (2016-03-31 03:39:18)
> > Introduce sPAPRMachineClass.dr_cpu_enabled to indicate support for
> > CPU core hotplug. Initialize boot time CPUs as core deivces and prevent
> > topologies that result in partially filled cores. Both of these are done
> > only if CPU core hotplug is supported.
> >
> > Note: An unrelated change in the call to xics_system_init() is done
> > in this patch as it makes sense to use the local variable smt introduced
> > in this patch instead of kvmppc_smt_threads() call here.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 73 +++++++++++++++++++++++++++++++++++------
> > hw/ppc/spapr_cpu_core.c | 45 +++++++++++++++++++++++++
> > include/hw/ppc/spapr.h | 2 ++
> > include/hw/ppc/spapr_cpu_core.h | 3 ++
> > 4 files changed, 113 insertions(+), 10 deletions(-)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 45ac5dc..1ead043 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -64,6 +64,7 @@
> >
> > #include "hw/compat.h"
> > #include "qemu-common.h"
> > +#include "hw/ppc/spapr_cpu_core.h"
> >
> > #include <libfdt.h>
> >
> > @@ -1614,6 +1615,10 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> > machine->boot_order = g_strdup(boot_device);
> > }
> >
> > +/*
> > + * TODO: Check if some of these can be moved to rtas_start_cpu() where
> > + * a few other things required for hotplugged CPUs are being done.
> > + */
> > void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > {
> > CPUPPCState *env = &cpu->env;
> > @@ -1644,6 +1649,7 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > xics_cpu_setup(spapr->icp, cpu);
> >
> > qemu_register_reset(spapr_cpu_reset, cpu);
> > + spapr_cpu_reset(cpu);
> > }
> >
> > /*
> > @@ -1727,7 +1733,6 @@ static void ppc_spapr_init(MachineState *machine)
> > const char *kernel_filename = machine->kernel_filename;
> > const char *kernel_cmdline = machine->kernel_cmdline;
> > const char *initrd_filename = machine->initrd_filename;
> > - PowerPCCPU *cpu;
> > PCIHostState *phb;
> > int i;
> > MemoryRegion *sysmem = get_system_memory();
> > @@ -1741,6 +1746,22 @@ static void ppc_spapr_init(MachineState *machine)
> > long load_limit, fw_size;
> > bool kernel_le = false;
> > char *filename;
> > + int smt = kvmppc_smt_threads();
> > + int spapr_cores = smp_cpus / smp_threads;
> > + int spapr_max_cores = max_cpus / smp_threads;
> > +
> > + if (smc->dr_cpu_enabled) {
> > + if (smp_cpus % smp_threads) {
> > + error_report("smp_cpus (%u) must be multiple of threads (%u)",
> > + smp_cpus, smp_threads);
> > + exit(1);
> > + }
> > + if (max_cpus % smp_threads) {
> > + error_report("max_cpus (%u) must be multiple of threads (%u)",
> > + max_cpus, smp_threads);
> > + exit(1);
> > + }
> > + }
> >
> > msi_supported = true;
> >
> > @@ -1787,8 +1808,7 @@ static void ppc_spapr_init(MachineState *machine)
> >
> > /* Set up Interrupt Controller before we create the VCPUs */
> > spapr->icp = xics_system_init(machine,
> > - DIV_ROUND_UP(max_cpus * kvmppc_smt_threads(),
> > - smp_threads),
> > + DIV_ROUND_UP(max_cpus * smt, smp_threads),
> > XICS_IRQS, &error_fatal);
> >
> > if (smc->dr_lmb_enabled) {
> > @@ -1799,13 +1819,34 @@ static void ppc_spapr_init(MachineState *machine)
> > if (machine->cpu_model == NULL) {
> > machine->cpu_model = kvm_enabled() ? "host" : "POWER7";
> > }
> > - for (i = 0; i < smp_cpus; i++) {
> > - cpu = cpu_ppc_init(machine->cpu_model);
> > - if (cpu == NULL) {
> > - error_report("Unable to find PowerPC CPU definition");
> > - exit(1);
> > +
> > + if (smc->dr_cpu_enabled) {
> > + spapr->cores = g_new0(Object *, spapr_max_cores);
> > +
> > + for (i = 0; i < spapr_max_cores; i++) {
> > + int core_dt_id = i * smt;
> > +
> > + if (i < spapr_cores) {
>
> Is there any reason to not just have the for() loop stop at spapr_cores?
> Maybe I missed something in the subsequent patches, but it seems like we
> never end up doing anything beyond i < spapr_cores.
In the next patch, I create DR connectors for all spapr_max_cores but
create only spapr_cores number of cores (boot time cores). Having said that
I will have the for loop changed to spapr_cores in this patch and
move to spapr_max_cores in the next patch.
>
> > + char *type = spapr_get_cpu_core_type(machine->cpu_model);
> > + Object *core = object_new(type);
> > +
> > + g_free(type);
> > + object_property_set_int(core, smp_threads, "threads",
> > + &error_fatal);
> > + object_property_set_int(core, core_dt_id, CPU_CORE_PROP_CORE,
> > + &error_fatal);
> > + object_property_set_bool(core, true, "realized", &error_fatal);
> > + }
> > }
> > - spapr_cpu_init(spapr, cpu, &error_fatal);
> > + } else {
> > + for (i = 0; i < smp_cpus; i++) {
> > + PowerPCCPU *cpu = cpu_ppc_init(machine->cpu_model);
> > + if (cpu == NULL) {
> > + error_report("Unable to find PowerPC CPU definition");
> > + exit(1);
> > + }
> > + spapr_cpu_init(spapr, cpu, &error_fatal);
>
> Is there anything that prevents us using core modeling in the non-DR
> case as well? Having management of invididual threads/spapr_cpu_init
> contained in within one place seems beneficial beyond hotplug. Is it
> because cores are currently limited to cpus == "threads" via pre_plug
> hook? Maybe it makes sense to loosen that restriction in the special
> case where DR isn't enabled?
CPU core devices are created only when CPU DR capability is present. To support
migration of older guests that don't know about CPU core devices, I retain the
CPU thread initialization code as above.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-04-05 23:47 ` Michael Roth
@ 2016-05-05 9:22 ` Bharata B Rao
0 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-05-05 9:22 UTC (permalink / raw)
To: Michael Roth
Cc: qemu-devel, qemu-ppc, afaerber, david, imammedo, armbru, thuth,
aik, agraf, pbonzini, ehabkost, pkrempa, eblake, mjrosato,
borntraeger
On Tue, Apr 05, 2016 at 06:47:16PM -0500, Michael Roth wrote:
> Quoting Bharata B Rao (2016-03-31 03:39:19)
> > Set up device tree entries for the hotplugged CPU core and use the
> > exising RTAS event logging infrastructure to send CPU hotplug notification
> > to the guest.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 58 ++++++++++++++++++++++++++++++++++
> > hw/ppc/spapr_cpu_core.c | 70 +++++++++++++++++++++++++++++++++++++++++
> > hw/ppc/spapr_events.c | 3 ++
> > hw/ppc/spapr_rtas.c | 24 ++++++++++++++
> > include/hw/ppc/spapr.h | 2 ++
> > include/hw/ppc/spapr_cpu_core.h | 2 ++
> > 6 files changed, 159 insertions(+)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 1ead043..1a5dbd9 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -603,6 +603,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> > size_t page_sizes_prop_size;
> > uint32_t vcpus_per_socket = smp_threads * smp_cores;
> > uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> > + sPAPRDRConnector *drc;
> > + sPAPRDRConnectorClass *drck;
> > + int drc_index;
> > +
> > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> > + if (drc) {
> > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> > + drc_index = drck->get_index(drc);
> > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> > + }
> >
> > /* Note: we keep CI large pages off for now because a 64K capable guest
> > * provisioned with large pages might otherwise try to map a qemu
> > @@ -987,6 +997,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> > _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> > }
> >
> > + if (smc->dr_cpu_enabled) {
> > + int offset = fdt_path_offset(fdt, "/cpus");
> > + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > + SPAPR_DR_CONNECTOR_TYPE_CPU);
> > + if (ret < 0) {
> > + error_report("Couldn't set up CPU DR device tree properties");
> > + exit(1);
> > + }
> > + }
> > +
> > _FDT((fdt_pack(fdt)));
> >
> > if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> > @@ -1622,6 +1642,8 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> > void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > {
> > CPUPPCState *env = &cpu->env;
> > + CPUState *cs = CPU(cpu);
> > + int i;
> >
> > /* Set time-base frequency to 512 MHz */
> > cpu_ppc_tb_init(env, TIMEBASE_FREQ);
> > @@ -1646,6 +1668,14 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> > }
> > }
> >
> > + /* Set NUMA node for the added CPUs */
> > + for (i = 0; i < nb_numa_nodes; i++) {
> > + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> > + cs->numa_node = i;
> > + break;
> > + }
> > + }
> > +
> > xics_cpu_setup(spapr->icp, cpu);
> >
> > qemu_register_reset(spapr_cpu_reset, cpu);
> > @@ -1825,6 +1855,11 @@ static void ppc_spapr_init(MachineState *machine)
> >
> > for (i = 0; i < spapr_max_cores; i++) {
> > int core_dt_id = i * smt;
> > + sPAPRDRConnector *drc =
> > + spapr_dr_connector_new(OBJECT(spapr),
> > + SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
> > +
> > + qemu_register_reset(spapr_drc_reset, drc);
> >
> > if (i < spapr_cores) {
> > char *type = spapr_get_cpu_core_type(machine->cpu_model);
> > @@ -2247,6 +2282,27 @@ out:
> > error_propagate(errp, local_err);
> > }
> >
> > +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> > + sPAPRMachineState *spapr)
> > +{
> > + PowerPCCPU *cpu = POWERPC_CPU(cs);
> > + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> > + int id = ppc_get_vcpu_dt_id(cpu);
> > + void *fdt;
> > + int offset, fdt_size;
> > + char *nodename;
> > +
> > + fdt = create_device_tree(&fdt_size);
> > + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> > + offset = fdt_add_subnode(fdt, 0, nodename);
> > +
> > + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> > + g_free(nodename);
> > +
> > + *fdt_offset = offset;
> > + return fdt;
> > +}
> > +
> > static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > DeviceState *dev, Error **errp)
> > {
> > @@ -2287,6 +2343,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > }
> >
> > spapr_memory_plug(hotplug_dev, dev, node, errp);
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > + spapr_core_plug(hotplug_dev, dev, errp);
> > }
> > }
> >
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > index 640d143..a9ba843 100644
> > --- a/hw/ppc/spapr_cpu_core.c
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -18,6 +18,7 @@
> > void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > Error **errp)
> > {
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > int spapr_max_cores = max_cpus / smp_threads;
> > int index;
> > @@ -25,6 +26,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > Error *local_err = NULL;
> > CPUCore *cc = CPU_CORE(dev);
> >
> > + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> > + error_setg(&local_err, "CPU hotplug not supported for this machine");
> > + goto out;
> > + }
> > +
> > if (cc->threads != smp_threads) {
> > error_setg(&local_err, "threads must be %d", smp_threads);
> > goto out;
> > @@ -49,6 +55,70 @@ out:
> > error_propagate(errp, local_err);
> > }
> >
> > +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > + Error **errp)
> > +{
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> > + CPUCore *cc = CPU_CORE(dev);
> > + CPUState *cs = CPU(&core->threads[0]);
> > + sPAPRDRConnector *drc;
> > + sPAPRDRConnectorClass *drck;
> > + Error *local_err = NULL;
> > + void *fdt = NULL;
> > + int fdt_offset = 0;
> > + int index;
> > + int smt = kvmppc_smt_threads();
> > +
> > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core);
> > + index = cc->core / smt;
> > + spapr->cores[index] = OBJECT(dev);
> > +
> > + if (!smc->dr_cpu_enabled) {
> > + /*
> > + * This is a cold plugged CPU core but the machine doesn't support
> > + * DR. So skip the hotplug path ensuring that the core is brought
> > + * up online with out an associated DR connector.
> > + */
> > + return;
> > + }
> > +
> > + g_assert(drc);
> > +
> > + /*
> > + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> > + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> > + */
> > + if (dev->hotplugged) {
> > + fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
> > + dev->hotplugged = true;
>
> This doesn't seem necessary ^
Yes, this hunk refuses to die, I just ensured that it is removed finally.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
2016-04-04 4:23 ` David Gibson
2016-04-05 23:47 ` Michael Roth
@ 2016-05-06 8:57 ` Igor Mammedov
2016-05-06 10:14 ` Bharata B Rao
2 siblings, 1 reply; 40+ messages in thread
From: Igor Mammedov @ 2016-05-06 8:57 UTC (permalink / raw)
To: Bharata B Rao
Cc: qemu-devel, mjrosato, thuth, pkrempa, ehabkost, aik, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber, david
On Thu, 31 Mar 2016 14:09:19 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> Set up device tree entries for the hotplugged CPU core and use the
> exising RTAS event logging infrastructure to send CPU hotplug notification
> to the guest.
perhaps you could reuse:
[RFC,20/42] machine: add cpu-hotplug machine option
http://patchwork.ozlabs.org/patch/617516/
to enable cpu hotplug explicitly,
and by default machine would be hotplug less saving resources.
>
> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr.c | 58 ++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_cpu_core.c | 70 +++++++++++++++++++++++++++++++++++++++++
> hw/ppc/spapr_events.c | 3 ++
> hw/ppc/spapr_rtas.c | 24 ++++++++++++++
> include/hw/ppc/spapr.h | 2 ++
> include/hw/ppc/spapr_cpu_core.h | 2 ++
> 6 files changed, 159 insertions(+)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1ead043..1a5dbd9 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -603,6 +603,16 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset,
> size_t page_sizes_prop_size;
> uint32_t vcpus_per_socket = smp_threads * smp_cores;
> uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + int drc_index;
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index);
> + if (drc) {
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drc_index = drck->get_index(drc);
> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index)));
> + }
>
> /* Note: we keep CI large pages off for now because a 64K capable guest
> * provisioned with large pages might otherwise try to map a qemu
> @@ -987,6 +997,16 @@ static void spapr_finalize_fdt(sPAPRMachineState *spapr,
> _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_LMB));
> }
>
> + if (smc->dr_cpu_enabled) {
> + int offset = fdt_path_offset(fdt, "/cpus");
> + ret = spapr_drc_populate_dt(fdt, offset, NULL,
> + SPAPR_DR_CONNECTOR_TYPE_CPU);
> + if (ret < 0) {
> + error_report("Couldn't set up CPU DR device tree properties");
> + exit(1);
> + }
> + }
> +
> _FDT((fdt_pack(fdt)));
>
> if (fdt_totalsize(fdt) > FDT_MAX_SIZE) {
> @@ -1622,6 +1642,8 @@ static void spapr_boot_set(void *opaque, const char *boot_device,
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> {
> CPUPPCState *env = &cpu->env;
> + CPUState *cs = CPU(cpu);
> + int i;
>
> /* Set time-base frequency to 512 MHz */
> cpu_ppc_tb_init(env, TIMEBASE_FREQ);
> @@ -1646,6 +1668,14 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
> }
> }
>
> + /* Set NUMA node for the added CPUs */
> + for (i = 0; i < nb_numa_nodes; i++) {
> + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
> + cs->numa_node = i;
> + break;
> + }
> + }
> +
> xics_cpu_setup(spapr->icp, cpu);
>
> qemu_register_reset(spapr_cpu_reset, cpu);
> @@ -1825,6 +1855,11 @@ static void ppc_spapr_init(MachineState *machine)
>
> for (i = 0; i < spapr_max_cores; i++) {
> int core_dt_id = i * smt;
> + sPAPRDRConnector *drc =
> + spapr_dr_connector_new(OBJECT(spapr),
> + SPAPR_DR_CONNECTOR_TYPE_CPU, core_dt_id);
> +
> + qemu_register_reset(spapr_drc_reset, drc);
>
> if (i < spapr_cores) {
> char *type = spapr_get_cpu_core_type(machine->cpu_model);
> @@ -2247,6 +2282,27 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> + sPAPRMachineState *spapr)
> +{
> + PowerPCCPU *cpu = POWERPC_CPU(cs);
> + DeviceClass *dc = DEVICE_GET_CLASS(cs);
> + int id = ppc_get_vcpu_dt_id(cpu);
> + void *fdt;
> + int offset, fdt_size;
> + char *nodename;
> +
> + fdt = create_device_tree(&fdt_size);
> + nodename = g_strdup_printf("%s@%x", dc->fw_name, id);
> + offset = fdt_add_subnode(fdt, 0, nodename);
> +
> + spapr_populate_cpu_dt(cs, fdt, offset, spapr);
> + g_free(nodename);
> +
> + *fdt_offset = offset;
> + return fdt;
> +}
> +
> static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
> {
> @@ -2287,6 +2343,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> }
>
> spapr_memory_plug(hotplug_dev, dev, node, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> + spapr_core_plug(hotplug_dev, dev, errp);
> }
> }
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 640d143..a9ba843 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -18,6 +18,7 @@
> void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp)
> {
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
looks like hotplug_dev is machine,
so why don't just cast it instead of qdev_get_machine().
the same applies to other plug handlers.
> int spapr_max_cores = max_cpus / smp_threads;
> int index;
> @@ -25,6 +26,11 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error *local_err = NULL;
> CPUCore *cc = CPU_CORE(dev);
>
> + if (!smc->dr_cpu_enabled && dev->hotplugged) {
> + error_setg(&local_err, "CPU hotplug not supported for this machine");
> + goto out;
> + }
> +
> if (cc->threads != smp_threads) {
> error_setg(&local_err, "threads must be %d", smp_threads);
> goto out;
> @@ -49,6 +55,70 @@ out:
> error_propagate(errp, local_err);
> }
>
> +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp)
> +{
> + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev));
> + CPUCore *cc = CPU_CORE(dev);
> + CPUState *cs = CPU(&core->threads[0]);
> + sPAPRDRConnector *drc;
> + sPAPRDRConnectorClass *drck;
> + Error *local_err = NULL;
> + void *fdt = NULL;
> + int fdt_offset = 0;
> + int index;
> + int smt = kvmppc_smt_threads();
> +
> + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core);
> + index = cc->core / smt;
> + spapr->cores[index] = OBJECT(dev);
> +
> + if (!smc->dr_cpu_enabled) {
> + /*
> + * This is a cold plugged CPU core but the machine doesn't support
> + * DR. So skip the hotplug path ensuring that the core is brought
> + * up online with out an associated DR connector.
> + */
> + return;
> + }
> +
> + g_assert(drc);
> +
> + /*
> + * Setup CPU DT entries only for hotplugged CPUs. For boot time or
> + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt().
> + */
> + if (dev->hotplugged) {
> + fdt = spapr_populate_hotplug_cpu_dt(cs, &fdt_offset, spapr);
> + dev->hotplugged = true;
> + }
> +
> + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err);
> + if (local_err) {
> + g_free(fdt);
> + spapr->cores[index] = NULL;
> + error_propagate(errp, local_err);
> + return;
> + }
> +
> + if (dev->hotplugged) {
> + /*
> + * Send hotplug notification interrupt to the guest only in case
> + * of hotplugged CPUs.
> + */
> + spapr_hotplug_req_add_by_index(drc);
> + } else {
> + /*
> + * Set the right DRC states for cold plugged CPU.
> + */
> + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE);
> + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED);
> + }
> +}
> +
> static const TypeInfo spapr_cpu_core_type_info = {
> .name = TYPE_SPAPR_CPU_CORE,
> .parent = TYPE_CPU_CORE,
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index 39f4682..10340e1 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -437,6 +437,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
> case SPAPR_DR_CONNECTOR_TYPE_LMB:
> hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY;
> break;
> + case SPAPR_DR_CONNECTOR_TYPE_CPU:
> + hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
> + break;
> default:
> /* we shouldn't be signaling hotplug events for resources
> * that don't support them
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index b7c5ebd..cc0369e 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -34,6 +34,7 @@
>
> #include "hw/ppc/spapr.h"
> #include "hw/ppc/spapr_vio.h"
> +#include "hw/ppc/ppc.h"
> #include "qapi-event.h"
> #include "hw/boards.h"
>
> @@ -161,6 +162,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_,
> rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> }
>
> +/*
> + * Set the timebase offset of the CPU to that of first CPU.
> + * This helps hotplugged CPU to have the correct timebase offset.
> + */
> +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> +
> + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset;
> +}
> +
> +static void spapr_cpu_set_endianness(PowerPCCPU *cpu)
> +{
> + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu);
> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu);
> +
> + if (!pcc->interrupts_big_endian(fcpu)) {
> + cpu->env.spr[SPR_LPCR] |= LPCR_ILE;
> + }
> +}
> +
> static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> uint32_t token, uint32_t nargs,
> target_ulong args,
> @@ -197,6 +219,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr,
> env->nip = start;
> env->gpr[3] = r3;
> cs->halted = 0;
> + spapr_cpu_set_endianness(cpu);
> + spapr_cpu_update_tb_offset(cpu);
>
> qemu_cpu_kick(cs);
>
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index a6956c0..619db98 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -588,6 +588,8 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
> void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
> uint32_t count);
> void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
> +void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
> + sPAPRMachineState *spapr);
>
> /* rtas-configure-connector state */
> struct sPAPRConfigureConnectorState {
> diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h
> index f08f291..165af7c 100644
> --- a/include/hw/ppc/spapr_cpu_core.h
> +++ b/include/hw/ppc/spapr_cpu_core.h
> @@ -63,4 +63,6 @@ typedef struct POWER8sPAPRCPUCore {
> void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp);
> char *spapr_get_cpu_core_type(const char *model);
> +void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> + Error **errp);
> #endif
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-05-06 8:57 ` Igor Mammedov
@ 2016-05-06 10:14 ` Bharata B Rao
2016-05-06 11:01 ` Igor Mammedov
0 siblings, 1 reply; 40+ messages in thread
From: Bharata B Rao @ 2016-05-06 10:14 UTC (permalink / raw)
To: Igor Mammedov
Cc: qemu-devel, mjrosato, thuth, pkrempa, ehabkost, aik, armbru,
agraf, borntraeger, qemu-ppc, pbonzini, mdroth, afaerber, david
On Fri, May 06, 2016 at 10:57:21AM +0200, Igor Mammedov wrote:
> On Thu, 31 Mar 2016 14:09:19 +0530
> Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
>
> > Set up device tree entries for the hotplugged CPU core and use the
> > exising RTAS event logging infrastructure to send CPU hotplug notification
> > to the guest.
>
> perhaps you could reuse:
>
> [RFC,20/42] machine: add cpu-hotplug machine option
> http://patchwork.ozlabs.org/patch/617516/
>
> to enable cpu hotplug explicitly,
> and by default machine would be hotplug less saving resources.
We explicitly enable hotplug for specific machine type versions by
setting sPAPRMachineClass.dr_cpu_enabled. So we check it to ensure
that there are no DR connector objects (the resources needed for hotplug)
created for a machine that doesn't support CPU hotplug. Similar is
the case for memory hotplug.
So you are suggesting that we switch from sAPPR specific check
(sPAPRMachineClass.dr_cpu_enabled) to the above MachineState.cpu_hotplug ?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support
2016-05-06 10:14 ` Bharata B Rao
@ 2016-05-06 11:01 ` Igor Mammedov
0 siblings, 0 replies; 40+ messages in thread
From: Igor Mammedov @ 2016-05-06 11:01 UTC (permalink / raw)
To: Bharata B Rao
Cc: mjrosato, agraf, thuth, pkrempa, ehabkost, aik, qemu-devel,
armbru, borntraeger, qemu-ppc, afaerber, pbonzini, mdroth, david
On Fri, 6 May 2016 15:44:10 +0530
Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> On Fri, May 06, 2016 at 10:57:21AM +0200, Igor Mammedov wrote:
> > On Thu, 31 Mar 2016 14:09:19 +0530
> > Bharata B Rao <bharata@linux.vnet.ibm.com> wrote:
> >
> > > Set up device tree entries for the hotplugged CPU core and use the
> > > exising RTAS event logging infrastructure to send CPU hotplug notification
> > > to the guest.
> >
> > perhaps you could reuse:
> >
> > [RFC,20/42] machine: add cpu-hotplug machine option
> > http://patchwork.ozlabs.org/patch/617516/
> >
> > to enable cpu hotplug explicitly,
> > and by default machine would be hotplug less saving resources.
>
> We explicitly enable hotplug for specific machine type versions by
> setting sPAPRMachineClass.dr_cpu_enabled. So we check it to ensure
> that there are no DR connector objects (the resources needed for hotplug)
> created for a machine that doesn't support CPU hotplug. Similar is
> the case for memory hotplug.
>
> So you are suggesting that we switch from sAPPR specific check
> (sPAPRMachineClass.dr_cpu_enabled) to the above MachineState.cpu_hotplug ?
yep, i.e. make user explicitly say that he want cpu hotplug enabled.
instead of allocating resources for it when user doesn't care about it.
It saves some memory/io ports for x86, not sure what it would save for sPAPR,
but at least one doesn't have to guess if hotplug is needed by looking at
smp_cpus & max_cpus.
>
> Regards,
> Bharata.
>
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support
2016-04-04 4:27 ` David Gibson
@ 2016-05-09 4:24 ` Bharata B Rao
0 siblings, 0 replies; 40+ messages in thread
From: Bharata B Rao @ 2016-05-09 4:24 UTC (permalink / raw)
To: David Gibson
Cc: qemu-devel, qemu-ppc, afaerber, imammedo, armbru, thuth, aik,
agraf, pbonzini, ehabkost, pkrempa, mdroth, eblake, mjrosato,
borntraeger
On Mon, Apr 04, 2016 at 02:27:24PM +1000, David Gibson wrote:
> On Thu, Mar 31, 2016 at 02:09:21PM +0530, Bharata B Rao wrote:
> > Remove the CPU core device by removing the underlying CPU thread devices.
> > Hot removal of CPU for sPAPR guests is achieved by sending the hot unplug
> > notification to the guest. Release the vCPU object after CPU hot unplug so
> > that vCPU fd can be parked and reused.
> >
> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
> > ---
> > hw/ppc/spapr.c | 16 ++++++++
> > hw/ppc/spapr_cpu_core.c | 86 +++++++++++++++++++++++++++++++++++++++++
> > include/hw/ppc/spapr.h | 1 +
> > include/hw/ppc/spapr_cpu_core.h | 11 ++++++
> > 4 files changed, 114 insertions(+)
> >
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 1a5dbd9..74cdcf2 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2348,11 +2348,27 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > }
> > }
> >
> > +void spapr_cpu_destroy(PowerPCCPU *cpu)
> > +{
> > + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
> > +
> > + xics_cpu_destroy(spapr->icp, cpu);
> > + qemu_unregister_reset(spapr_cpu_reset, cpu);
> > +}
> > +
> > static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
> > DeviceState *dev, Error **errp)
> > {
> > + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
> > +
> > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> > error_setg(errp, "Memory hot unplug not supported by sPAPR");
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > + if (!smc->dr_cpu_enabled) {
> > + error_setg(errp, "CPU hot unplug not supported on this machine");
> > + return;
> > + }
> > + spapr_core_unplug(hotplug_dev, dev, errp);
> > }
> > }
> >
> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > index a9ba843..09a592e 100644
> > --- a/hw/ppc/spapr_cpu_core.c
> > +++ b/hw/ppc/spapr_cpu_core.c
> > @@ -119,6 +119,92 @@ void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > }
> > }
> >
> > +static void spapr_cpu_core_cleanup(struct sPAPRCPUUnplugList *unplug_list)
> > +{
> > + sPAPRCPUUnplug *unplug, *next;
> > + Object *cpu;
> > +
> > + QLIST_FOREACH_SAFE(unplug, unplug_list, node, next) {
> > + cpu = unplug->cpu;
> > + object_unparent(cpu);
>
> Is there any danger in the fact that the cpu object is still in the
> QOM tree until unparented here? My usual expectation would be that
> you'd remove the object from the tree immediately, but defer the
> actual free. But I'm a bit unclear on how QOM removals are supposed
> to work.
As per my understanding, object_unparent() removes the object from
its parent and finalizes the child too.
The reason I defer unparenting of each CPU thread object like this is because
from the parent core object's detach_cb routine (spapr_core_release), we are
still walking the parent core's child list and can't immediately unparent the
child thread objects.
Regards,
Bharata.
^ permalink raw reply [flat|nested] 40+ messages in thread
end of thread, other threads:[~2016-05-09 4:25 UTC | newest]
Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-31 8:39 [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 01/12] exec: Remove cpu from cpus list during cpu_exec_exit() Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 02/12] exec: Do vmstate unregistration from cpu_exec_exit() Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 03/12] cpu: Reclaim vCPU objects Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 04/12] cpu: Add a sync version of cpu_remove() Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 05/12] qdev: hotplug: Introduce HotplugHandler.pre_plug() callback Bharata B Rao
2016-04-01 3:30 ` David Gibson
2016-04-01 10:38 ` Paolo Bonzini
2016-04-04 0:09 ` David Gibson
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 06/12] cpu: Abstract CPU core type Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 07/12] spapr: Abstract CPU core device Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 08/12] spapr: Add CPU type specific core devices Bharata B Rao
2016-04-01 5:08 ` David Gibson
2016-04-01 6:12 ` Bharata B Rao
2016-04-04 0:13 ` David Gibson
2016-04-09 2:21 ` Michael Roth
2016-04-04 0:16 ` David Gibson
2016-04-08 23:35 ` Michael Roth
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 09/12] spapr: convert boot CPUs into CPU " Bharata B Rao
2016-04-01 5:12 ` David Gibson
2016-04-08 23:35 ` Michael Roth
2016-05-05 9:19 ` Bharata B Rao
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 10/12] spapr: CPU hotplug support Bharata B Rao
2016-04-04 4:23 ` David Gibson
2016-04-05 23:47 ` Michael Roth
2016-05-05 9:22 ` Bharata B Rao
2016-05-06 8:57 ` Igor Mammedov
2016-05-06 10:14 ` Bharata B Rao
2016-05-06 11:01 ` Igor Mammedov
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 11/12] xics, xics_kvm: Handle CPU unplug correctly Bharata B Rao
2016-04-06 0:24 ` Michael Roth
2016-04-06 0:43 ` David Gibson
2016-04-08 23:40 ` Michael Roth
2016-03-31 8:39 ` [Qemu-devel] [RFC PATCH v2.1 12/12] spapr: CPU hot unplug support Bharata B Rao
2016-04-04 4:27 ` David Gibson
2016-05-09 4:24 ` Bharata B Rao
2016-04-04 14:44 ` [Qemu-devel] [RFC PATCH v2.1 00/12] Core based CPU hotplug for PowerPC sPAPR Igor Mammedov
2016-04-05 14:55 ` Bharata B Rao
2016-04-05 18:40 ` Igor Mammedov
2016-04-05 21:58 ` Igor Mammedov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.