[Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

This series is based on chen fan's previous i386 cpu hot remove patchset:
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
OS remove one vCPU online, the fireware will store removed bitmap to
QEMU, then QEMU could know to notify the assigned vCPU of exiting.
Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
QEMU itself.

The whole work is based on the new hot plug/unplug framework, ,the unplug request
callback does the pre-check and send the request, unplug callback does the
removal handling.

This series depends on tangchen's common hot plug/unplug enhance patchset.
[RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html

The is the second half of the previous series:                                      
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html             

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
patchset first:
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html

---
Changelog since v1:
 -rebase on the latest version.
 -delete patch i386/cpu: add instance finalize callback, and put it into patchset
  [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.

Changelog since RFC:
 -splited the i386 cpu hot remove into single thread.
 -replaced apic_no with apic_id, so does the related stuff to make it
  work with arbitrary CPU hotadd.
 -add the icc_device_unrealize callback to handle apic unrealize.
 -rework on the new hot plug/unplug platform.
---

Chen Fan (2):
  x86: add x86_cpu_unrealizefn() for cpu apic remove
  cpu hotplug: implement function cpu_status_write() for vcpu ejection

Gu Zheng (5):
  acpi/cpu: add cpu hot unplug request callback function
  acpi/piix4: add cpu hot unplug callback support
  acpi/ich9: add cpu hot unplug support
  pc: add cpu hot unplug callback support
  cpus: reclaim allocated vCPU objects

Zhu Guihua (4):
  acpi/piix4: add cpu hot unplug request callback support
  acpi/ich9: add cpu hot unplug request callback support
  pc: add cpu hot unplug request callback support
  acpi/cpu: add cpu hot unplug callback function

 cpus.c                            | 44 ++++++++++++++++++++
 hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
 hw/acpi/ich9.c                    | 17 ++++++--
 hw/acpi/piix4.c                   | 12 +++++-
 hw/core/qdev.c                    |  2 +-
 hw/cpu/icc_bus.c                  | 11 +++++
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
 hw/i386/kvm/apic.c                |  8 ++++
 hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
 hw/intc/apic.c                    | 10 +++++
 hw/intc/apic_common.c             | 21 ++++++----
 include/hw/acpi/cpu_hotplug.h     |  8 ++++
 include/hw/cpu/icc_bus.h          |  1 +
 include/hw/i386/apic_internal.h   |  1 +
 include/hw/qdev-core.h            |  1 +
 include/qom/cpu.h                 |  9 ++++
 include/sysemu/kvm.h              |  1 +
 kvm-all.c                         | 57 ++++++++++++++++++++++++-
 target-i386/cpu.c                 | 46 ++++++++++++++++++++
 19 files changed, 378 insertions(+), 27 deletions(-)

-- 
1.9.3

[Qemu-devel] [PATCH v2 01/11] x86: add x86_cpu_unrealizefn() for cpu apic remove

[Zhu Guihua]

From: Chen Fan <chen.fan.fnst@cn.fujitsu.com>

Implement x86_cpu_unrealizefn() for corresponding x86_cpu_realizefn(),
which is mostly used to clean the apic related allocation and vmstates
at here.

Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/cpu/icc_bus.c                | 11 ++++++++++
 hw/i386/kvm/apic.c              |  8 +++++++
 hw/intc/apic.c                  | 10 +++++++++
 hw/intc/apic_common.c           | 21 ++++++++++++-------
 include/hw/cpu/icc_bus.h        |  1 +
 include/hw/i386/apic_internal.h |  1 +
 target-i386/cpu.c               | 46 +++++++++++++++++++++++++++++++++++++++++
 7 files changed, 90 insertions(+), 8 deletions(-)

diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
index f455a20..3e7f62f 100644
--- a/hw/cpu/icc_bus.c
+++ b/hw/cpu/icc_bus.c
@@ -44,11 +44,22 @@ static void icc_device_realize(DeviceState *dev, Error **errp)
 
 }
 
+static void icc_device_unrealize(DeviceState *dev, Error **errp)
+{
+    ICCDeviceClass *idc = ICC_DEVICE_GET_CLASS(dev);
+
+    /* convert to QOM */
+    if (idc->unrealize) {
+        idc->unrealize(dev, errp);
+    }
+}
+
 static void icc_device_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
 
     dc->realize = icc_device_realize;
+    dc->unrealize = icc_device_unrealize;
     dc->bus_type = TYPE_ICC_BUS;
 }
 
diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index 5b47056..781c296 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -189,11 +189,19 @@ static void kvm_apic_realize(DeviceState *dev, Error **errp)
     }
 }
 
+static void kvm_apic_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+
+    object_unparent(OBJECT(&s->io_memory));
+}
+
 static void kvm_apic_class_init(ObjectClass *klass, void *data)
 {
     APICCommonClass *k = APIC_COMMON_CLASS(klass);
 
     k->realize = kvm_apic_realize;
+    k->unrealize = kvm_apic_unrealize;
     k->reset = kvm_apic_reset;
     k->set_base = kvm_apic_set_base;
     k->set_tpr = kvm_apic_set_tpr;
diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 0f97b47..2f3a7a3 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -889,11 +889,21 @@ static void apic_realize(DeviceState *dev, Error **errp)
     msi_supported = true;
 }
 
+static void apic_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+
+    object_unparent(OBJECT(&s->io_memory));
+    timer_free(s->timer);
+    local_apics[s->idx] = NULL;
+}
+
 static void apic_class_init(ObjectClass *klass, void *data)
 {
     APICCommonClass *k = APIC_COMMON_CLASS(klass);
 
     k->realize = apic_realize;
+    k->unrealize = apic_unrealize;
     k->set_base = apic_set_base;
     k->set_tpr = apic_set_tpr;
     k->get_tpr = apic_get_tpr;
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index 719f74d..912ba64 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -299,16 +299,8 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
     APICCommonState *s = APIC_COMMON(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
-    static int apic_no;
     static bool mmio_registered;
 
-    if (apic_no >= MAX_APICS) {
-        error_setg(errp, "%s initialization failed.",
-                   object_get_typename(OBJECT(dev)));
-        return;
-    }
-    s->idx = apic_no++;
-
     info = APIC_COMMON_GET_CLASS(s);
     info->realize(dev, errp);
     if (!mmio_registered) {
@@ -329,6 +321,18 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
 
 }
 
+static void apic_common_unrealize(DeviceState *dev, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(dev);
+    APICCommonClass *info = APIC_COMMON_GET_CLASS(s);
+
+    info->unrealize(dev, errp);
+
+    if (apic_report_tpr_access && info->enable_tpr_reporting) {
+        info->enable_tpr_reporting(s, false);
+    }
+}
+
 static int apic_pre_load(void *opaque)
 {
     APICCommonState *s = APIC_COMMON(opaque);
@@ -437,6 +441,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
     dc->reset = apic_reset_common;
     dc->props = apic_properties_common;
     idc->realize = apic_common_realize;
+    idc->unrealize = apic_common_unrealize;
     /*
      * Reason: APIC and CPU need to be wired up by
      * x86_cpu_apic_create()
diff --git a/include/hw/cpu/icc_bus.h b/include/hw/cpu/icc_bus.h
index 7e53afb..0b2ddd5 100644
--- a/include/hw/cpu/icc_bus.h
+++ b/include/hw/cpu/icc_bus.h
@@ -69,6 +69,7 @@ typedef struct ICCDeviceClass {
     /*< public >*/
 
     DeviceRealize realize;
+    DeviceUnrealize unrealize;
 } ICCDeviceClass;
 
 #define TYPE_ICC_DEVICE "icc-device"
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 00d29e2..b46a3af 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -82,6 +82,7 @@ typedef struct APICCommonClass
     ICCDeviceClass parent_class;
 
     DeviceRealize realize;
+    DeviceUnrealize unrealize;
     void (*set_base)(APICCommonState *s, uint64_t val);
     void (*set_tpr)(APICCommonState *s, uint8_t val);
     uint8_t (*get_tpr)(APICCommonState *s);
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 4746814..ca0c1f6 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2786,6 +2786,8 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
     qdev_prop_set_uint8(cpu->apic_state, "id", apic_id);
     /* TODO: convert to link<> */
     apic = APIC_COMMON(cpu->apic_state);
+    g_assert(apic_id < MAX_APICS);
+    apic->idx = apic_id;
     apic->cpu = cpu;
 }
 
@@ -2807,10 +2809,32 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
         return;
     }
 }
+
+static void x86_cpu_apic_unrealize(X86CPU *cpu, Error **errp)
+{
+    Error *local_err = NULL;
+
+    if (cpu->apic_state == NULL) {
+        return;
+    }
+
+    object_property_set_bool(OBJECT(cpu->apic_state),
+                             false, "realized", &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    vmstate_unregister(NULL, &vmstate_apic_common, cpu->apic_state);
+    object_unparent(OBJECT(cpu->apic_state));
+}
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
 }
+static void x86_cpu_apic_unrealize(X86CPU *cpu, Error **errp)
+{
+}
 #endif
 
 
@@ -2893,6 +2917,27 @@ out:
     }
 }
 
+static void x86_cpu_unrealizefn(DeviceState *dev, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(dev);
+    CPUClass *cc = CPU_GET_CLASS(dev);
+    Error *local_err = NULL;
+
+    if (qdev_get_vmsd(DEVICE(cpu)) == NULL) {
+        vmstate_unregister(NULL, &vmstate_cpu_common, cpu);
+    }
+
+    if (cc->vmsd != NULL) {
+        vmstate_unregister(NULL, cc->vmsd, cpu);
+    }
+
+    x86_cpu_apic_unrealize(cpu, &local_err);
+    if (local_err != NULL) {
+        error_propagate(errp, local_err);
+        return;
+    }
+}
+
 /* Enables contiguous-apic-ID mode, for compatibility */
 static bool compat_apic_id_mode;
 
@@ -3059,6 +3104,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->unrealize = x86_cpu_unrealizefn;
     dc->bus_type = TYPE_ICC_BUS;
     dc->props = x86_cpu_properties;
 
-- 
1.9.3

[Qemu-devel] [PATCH v2 02/11] acpi/cpu: add cpu hot unplug request callback function

[Zhu Guihua]

From: Gu Zheng <guz.fnst@cn.fujitsu.com>

Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/acpi/cpu_hotplug.c         | 38 +++++++++++++++++++++++++++++++++-----
 include/hw/acpi/cpu_hotplug.h |  4 ++++
 2 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index 4047294..f8a10d2 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -12,6 +12,11 @@
 #include "hw/hw.h"
 #include "hw/acpi/cpu_hotplug.h"
 
+typedef enum STS_OPT {
+    SET,
+    CLEAR,
+} STS_OPT;
+
 static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 {
     AcpiCpuHotplug *cpus = opaque;
@@ -36,8 +41,8 @@ static const MemoryRegionOps AcpiCpuHotplug_ops = {
     },
 };
 
-static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
-                                     Error **errp)
+static void acpi_update_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
+                                        STS_OPT opt, Error **errp)
 {
     CPUClass *k = CPU_GET_CLASS(cpu);
     int64_t cpu_id;
@@ -48,13 +53,23 @@ static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
         return;
     }
 
-    g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    switch (opt) {
+    case SET:
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+        break;
+    case CLEAR:
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+        break;
+    default:
+        g_assert(0);
+        break;
+    }
 }
 
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
                       AcpiCpuHotplug *g, DeviceState *dev, Error **errp)
 {
-    acpi_set_cpu_present_bit(g, CPU(dev), errp);
+    acpi_update_cpu_present_bit(g, CPU(dev), SET, errp);
     if (*errp != NULL) {
         return;
     }
@@ -66,13 +81,26 @@ void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
     }
 }
 
+void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
+                                AcpiCpuHotplug *g, DeviceState *dev,
+                                Error **errp)
+{
+    acpi_update_cpu_present_bit(g, CPU(dev), CLEAR, errp);
+    if (*errp != NULL) {
+        return;
+    }
+
+    ar->gpe.sts[0] |= ACPI_CPU_HOTPLUG_STATUS;
+    acpi_update_sci(ar, irq);
+}
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base)
 {
     CPUState *cpu;
 
     CPU_FOREACH(cpu) {
-        acpi_set_cpu_present_bit(gpe_cpu, cpu, &error_abort);
+        acpi_update_cpu_present_bit(gpe_cpu, cpu, SET, &error_abort);
     }
     memory_region_init_io(&gpe_cpu->io, owner, &AcpiCpuHotplug_ops,
                           gpe_cpu, "acpi-cpu-hotplug", ACPI_GPE_PROC_LEN);
diff --git a/include/hw/acpi/cpu_hotplug.h b/include/hw/acpi/cpu_hotplug.h
index f6d358d..8b15a3d 100644
--- a/include/hw/acpi/cpu_hotplug.h
+++ b/include/hw/acpi/cpu_hotplug.h
@@ -23,6 +23,10 @@ typedef struct AcpiCpuHotplug {
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
                       AcpiCpuHotplug *g, DeviceState *dev, Error **errp);
 
+void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
+                                AcpiCpuHotplug *g, DeviceState *dev,
+                                Error **errp);
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base);
 #endif
-- 
1.9.3

[Qemu-devel] [PATCH v2 03/11] acpi/piix4: add cpu hot unplug request callback support

[Zhu Guihua]

Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 hw/acpi/piix4.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 4407388..9b75780 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -364,6 +364,8 @@ static void piix4_device_unplug_request_cb(HotplugHandler *hotplug_dev,
     if (object_dynamic_cast(OBJECT(dev), TYPE_PCI_DEVICE)) {
         acpi_pcihp_device_unplug_cb(&s->ar, s->irq, &s->acpi_pci_hotplug, dev,
                                     errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_request_cb(&s->ar, s->irq, &s->gpe_cpu, dev, errp);
     } else {
         error_setg(errp, "acpi: device unplug request for not supported device"
                    " type: %s", object_get_typename(OBJECT(dev)));
-- 
1.9.3

[Qemu-devel] [PATCH v2 04/11] acpi/ich9: add cpu hot unplug request callback support

[Zhu Guihua]

Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 hw/acpi/ich9.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c
index c48d176..b795f8f 100644
--- a/hw/acpi/ich9.c
+++ b/hw/acpi/ich9.c
@@ -304,8 +304,13 @@ void ich9_pm_device_plug_cb(ICH9LPCPMRegs *pm, DeviceState *dev, Error **errp)
 void ich9_pm_device_unplug_request_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
                                       Error **errp)
 {
-    error_setg(errp, "acpi: device unplug request for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_request_cb(&pm->acpi_regs, pm->irq,
+                                   &pm->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug request for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 void ich9_pm_device_unplug_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
-- 
1.9.3

[Qemu-devel] [PATCH v2 05/11] pc: add cpu hot unplug request callback support

[Zhu Guihua]

Add cpu hot unplug request callback support, and a pre-check about the
active cpus is added to avoid removing the last cpu.

Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 hw/i386/pc.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 9bdec16..dd6e8da 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1694,6 +1694,34 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void pc_cpu_unplug_request(HotplugHandler *hotplug_dev,
+                                  DeviceState *dev, Error **errp)
+{
+    HotplugHandlerClass *hhc;
+    Error *local_err = NULL;
+    PCMachineState *pcms;
+
+    if (smp_cpus == 1) {
+        error_setg(&local_err,
+                   "This is the last cpu, should not be removed!");
+        goto out;
+    }
+
+    pcms = PC_MACHINE(hotplug_dev);
+    if (!pcms->acpi_dev) {
+        if (dev->hotplugged) {
+            error_setg(&local_err,
+                       "cpu hot unplug is not enabled: missing acpi device");
+        }
+        goto out;
+    }
+
+    hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
+    hhc->unplug_request(HOTPLUG_HANDLER(pcms->acpi_dev), dev, &local_err);
+out:
+    error_propagate(errp, local_err);
+}
+
 static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -1707,8 +1735,12 @@ static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
 static void pc_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev,
                                                 DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug request for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        pc_cpu_unplug_request(hotplug_dev, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug request for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static void pc_machine_device_unplug_cb(HotplugHandler *hotplug_dev,
-- 
1.9.3

[Qemu-devel] [PATCH v2 06/11] acpi/cpu: add cpu hot unplug callback function

[Zhu Guihua]

Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 cpus.c                        | 7 +++++++
 hw/acpi/cpu_hotplug.c         | 8 ++++++++
 include/hw/acpi/cpu_hotplug.h | 3 +++
 include/qom/cpu.h             | 9 +++++++++
 4 files changed, 27 insertions(+)

diff --git a/cpus.c b/cpus.c
index d5e35c0..1c25054 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1205,6 +1205,13 @@ void resume_all_vcpus(void)
     }
 }
 
+void cpu_remove(CPUState *cpu)
+{
+    cpu->stop = true;
+    cpu->exit = true;
+    qemu_cpu_kick(cpu);
+}
+
 /* For temporary buffers for forming a name */
 #define VCPU_THREAD_NAME_SIZE 16
 
diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index f8a10d2..36feb6a 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -94,6 +94,14 @@ void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
     acpi_update_sci(ar, irq);
 }
 
+void acpi_cpu_unplug_cb(ACPIREGS *ar, qemu_irq irq, AcpiCpuHotplug *g,
+                        DeviceState *dev, Error **errp)
+{
+    CPUState *cpu = CPU(dev);
+
+    cpu_remove(cpu);
+}
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base)
 {
diff --git a/include/hw/acpi/cpu_hotplug.h b/include/hw/acpi/cpu_hotplug.h
index 8b15a3d..0f84adb 100644
--- a/include/hw/acpi/cpu_hotplug.h
+++ b/include/hw/acpi/cpu_hotplug.h
@@ -27,6 +27,9 @@ void acpi_cpu_unplug_request_cb(ACPIREGS *ar, qemu_irq irq,
                                 AcpiCpuHotplug *g, DeviceState *dev,
                                 Error **errp);
 
+void acpi_cpu_unplug_cb(ACPIREGS *ar, qemu_irq irq,
+                        AcpiCpuHotplug *g, DeviceState *dev, Error **errp);
+
 void acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
                            AcpiCpuHotplug *gpe_cpu, uint16_t base);
 #endif
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index f663199..0592b4d 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -249,6 +249,7 @@ struct CPUState {
     bool created;
     bool stop;
     bool stopped;
+    bool exit;
     volatile sig_atomic_t exit_request;
     uint32_t interrupt_request;
     int singlestep_enabled;
@@ -613,6 +614,14 @@ void cpu_exit(CPUState *cpu);
 void cpu_resume(CPUState *cpu);
 
 /**
+ * cpu_remove:
+ * @cpu: The vCPU to remove.
+ *
+ * Requests the CPU @cpu to be removed.
+ */
+void cpu_remove(CPUState *cpu);
+
+/**
  * qemu_init_vcpu:
  * @cpu: The vCPU to initialize.
  *
-- 
1.9.3

[Qemu-devel] [PATCH v2 07/11] acpi/piix4: add cpu hot unplug callback support

[Zhu Guihua]

From: Gu Zheng <guz.fnst@cn.fujitsu.com>

Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/acpi/piix4.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 9b75780..2a923fc 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -375,8 +375,14 @@ static void piix4_device_unplug_request_cb(HotplugHandler *hotplug_dev,
 static void piix4_device_unplug_cb(HotplugHandler *hotplug_dev,
                                    DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    PIIX4PMState *s = PIIX4_PM(hotplug_dev);
+
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_cb(&s->ar, s->irq, &s->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static void piix4_update_bus_hotplug(PCIBus *pci_bus, void *opaque)
-- 
1.9.3

[Qemu-devel] [PATCH v2 08/11] acpi/ich9: add cpu hot unplug support

[Zhu Guihua]

From: Gu Zheng <guz.fnst@cn.fujitsu.com>

Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/acpi/ich9.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c
index b795f8f..a892e5d 100644
--- a/hw/acpi/ich9.c
+++ b/hw/acpi/ich9.c
@@ -316,8 +316,12 @@ void ich9_pm_device_unplug_request_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
 void ich9_pm_device_unplug_cb(ICH9LPCPMRegs *pm, DeviceState *dev,
                               Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        acpi_cpu_unplug_cb(&pm->acpi_regs, pm->irq, &pm->gpe_cpu, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 void ich9_pm_ospm_status(AcpiDeviceIf *adev, ACPIOSTInfoList ***list)
-- 
1.9.3

[Qemu-devel] [PATCH v2 09/11] pc: add cpu hot unplug callback support

[Zhu Guihua]

From: Gu Zheng <guz.fnst@cn.fujitsu.com>

Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/i386/pc.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index dd6e8da..b7d5712 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1722,6 +1722,24 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void pc_cpu_unplug(HotplugHandler *hotplug_dev,
+                          DeviceState *dev, Error **errp)
+{
+    HotplugHandlerClass *hhc;
+    Error *local_err = NULL;
+    PCMachineState *pcms = PC_MACHINE(hotplug_dev);
+
+    hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
+    hhc->unplug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    /* decrement the number of CPUs */
+    rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) - 1);
+}
+
 static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -1746,8 +1764,12 @@ static void pc_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev,
 static void pc_machine_device_unplug_cb(HotplugHandler *hotplug_dev,
                                         DeviceState *dev, Error **errp)
 {
-    error_setg(errp, "acpi: device unplug for not supported device"
-               " type: %s", object_get_typename(OBJECT(dev)));
+    if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+        pc_cpu_unplug(hotplug_dev, dev, errp);
+    } else {
+        error_setg(errp, "acpi: device unplug for not supported device"
+                   " type: %s", object_get_typename(OBJECT(dev)));
+    }
 }
 
 static HotplugHandler *pc_get_hotpug_handler(MachineState *machine,
-- 
1.9.3

[Qemu-devel] [PATCH v2 10/11] cpu hotplug: implement function cpu_status_write() for vcpu ejection

[Zhu Guihua]

From: Chen Fan <chen.fan.fnst@cn.fujitsu.com>

When OS ejected a vcpu (like: echo 1 > /sys/bus/acpi/devices/LNXCPUXX/eject),
it would call acpi EJ0 method, the firmware need to write the new cpumap, QEMU
would know which vcpu need to be ejected.

Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 hw/acpi/cpu_hotplug.c             | 42 ++++++++++++++++++++++++++++++++++++++-
 hw/core/qdev.c                    |  2 +-
 hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 +++++-
 include/hw/acpi/cpu_hotplug.h     |  1 +
 include/hw/qdev-core.h            |  1 +
 5 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index 36feb6a..2a56df0 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -17,6 +17,26 @@ typedef enum STS_OPT {
     CLEAR,
 } STS_OPT;
 
+static void acpi_eject_vcpu(AcpiCpuHotplug *cpus_status, int64_t cpu_id)
+{
+    CPUState *cpu;
+
+    CPU_FOREACH(cpu) {
+        CPUClass *cc = CPU_GET_CLASS(cpu);
+        int64_t id = cc->get_arch_id(cpu);
+        HotplugHandler *hotplug_ctrl;
+
+        if (cpu_id == id) {
+            cpus_status->old_sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+            cpus_status->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+
+            hotplug_ctrl = qdev_get_hotplug_handler(DEVICE(cpu));
+            hotplug_handler_unplug(hotplug_ctrl, DEVICE(cpu), NULL);
+            break;
+        }
+    }
+}
+
 static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 {
     AcpiCpuHotplug *cpus = opaque;
@@ -28,7 +48,26 @@ static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned int size)
 static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
                              unsigned int size)
 {
-    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+    AcpiCpuHotplug *cpus = opaque;
+    uint8_t val;
+    int i;
+    int64_t cpu_id = -1;
+
+    val = cpus->old_sts[addr] ^ data;
+
+    if (val == 0) {
+        return;
+    }
+
+    for (i = 0; i < 8; i++) {
+        if (val & 1 << i) {
+            cpu_id = 8 * addr + i;
+        }
+    }
+
+    if (cpu_id != -1) {
+        acpi_eject_vcpu(cpus, cpu_id);
+    }
 }
 
 static const MemoryRegionOps AcpiCpuHotplug_ops = {
@@ -56,6 +95,7 @@ static void acpi_update_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
     switch (opt) {
     case SET:
         g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+        g->old_sts[cpu_id / 8] |= (1 << (cpu_id % 8));
         break;
     case CLEAR:
         g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index 901f289..9f08fe6 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -223,7 +223,7 @@ void qdev_set_legacy_instance_id(DeviceState *dev, int alias_id,
     dev->alias_required_for_version = required_for_version;
 }
 
-static HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev)
+HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev)
 {
     HotplugHandler *hotplug_ctrl = NULL;
 
diff --git a/hw/i386/acpi-dsdt-cpu-hotplug.dsl b/hw/i386/acpi-dsdt-cpu-hotplug.dsl
index 34aab5a..9485f12 100644
--- a/hw/i386/acpi-dsdt-cpu-hotplug.dsl
+++ b/hw/i386/acpi-dsdt-cpu-hotplug.dsl
@@ -50,7 +50,11 @@ Scope(\_SB) {
     }
     Method(CPEJ, 2, NotSerialized) {
         // _EJ0 method - eject callback
-        Sleep(200)
+        Store(Zero, Index(CPON, ToInteger(Arg0)))
+        Store(One, Local0)
+        ShiftLeft(Local0, Arg0, Local0)
+        Not(Local0, Local0)
+        And(PRS, Local0, PRS)
     }
 
 #define CPU_STATUS_LEN ACPI_GPE_PROC_LEN
diff --git a/include/hw/acpi/cpu_hotplug.h b/include/hw/acpi/cpu_hotplug.h
index 0f84adb..abbb29e 100644
--- a/include/hw/acpi/cpu_hotplug.h
+++ b/include/hw/acpi/cpu_hotplug.h
@@ -18,6 +18,7 @@
 typedef struct AcpiCpuHotplug {
     MemoryRegion io;
     uint8_t sts[ACPI_GPE_PROC_LEN];
+    uint8_t old_sts[ACPI_GPE_PROC_LEN];
 } AcpiCpuHotplug;
 
 void acpi_cpu_plug_cb(ACPIREGS *ar, qemu_irq irq,
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 589bbe7..5b28a4f 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -263,6 +263,7 @@ void qdev_set_legacy_instance_id(DeviceState *dev, int alias_id,
 void qdev_unplug(DeviceState *dev, Error **errp);
 void qdev_simple_device_unplug_cb(HotplugHandler *hotplug_dev,
                                   DeviceState *dev, Error **errp);
+HotplugHandler *qdev_get_hotplug_handler(DeviceState *dev);
 void qdev_machine_creation_done(void);
 bool qdev_machine_modified(void);
 
-- 
1.9.3

[Qemu-devel] [PATCH v2 11/11] cpus: reclaim allocated vCPU objects

[Zhu Guihua]

From: Gu Zheng <guz.fnst@cn.fujitsu.com>

After ACPI get a signal to eject a vCPU, the vCPU must be
removed from CPU list，before the vCPU really removed,  then
release the all related vCPU objects.

In order to deal well with the kvm vcpus (which can not be removed without any
protection), we do not close KVM vcpu fd, just record and mark it as stopped
into a list, so that we can reuse it for the appending cpu hot-add request if
possible. It is also the approach that kvm guys suggested:
https://www.mail-archive.com/kvm@vger.kernel.org/msg102839.html

Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
---
 cpus.c               | 37 ++++++++++++++++++++++++++++++++++
 include/sysemu/kvm.h |  1 +
 kvm-all.c            | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 94 insertions(+), 1 deletion(-)

diff --git a/cpus.c b/cpus.c
index 1c25054..b5b0c91 100644
--- a/cpus.c
+++ b/cpus.c
@@ -871,6 +871,24 @@ void async_run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data)
     qemu_cpu_kick(cpu);
 }
 
+static void qemu_kvm_destroy_vcpu(CPUState *cpu)
+{
+    CPU_REMOVE(cpu);
+
+    if (kvm_destroy_vcpu(cpu) < 0) {
+        error_report("kvm_destroy_vcpu failed.\n");
+        exit(EXIT_FAILURE);
+    }
+
+    object_unparent(OBJECT(cpu));
+}
+
+static void qemu_tcg_destroy_vcpu(CPUState *cpu)
+{
+    CPU_REMOVE(cpu);
+    object_unparent(OBJECT(cpu));
+}
+
 static void flush_queued_work(CPUState *cpu)
 {
     struct qemu_work_item *wi;
@@ -963,6 +981,11 @@ static void *qemu_kvm_cpu_thread_fn(void *arg)
             }
         }
         qemu_kvm_wait_io_event(cpu);
+        if (cpu->exit && !cpu_can_run(cpu)) {
+            qemu_kvm_destroy_vcpu(cpu);
+            qemu_mutex_unlock(&qemu_global_mutex);
+            return NULL;
+        }
     }
 
     return NULL;
@@ -1016,6 +1039,7 @@ static void tcg_exec_all(void);
 static void *qemu_tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
+    CPUState *remove_cpu = NULL;
 
     qemu_tcg_init_cpu_signals();
     qemu_thread_get_self(cpu->thread);
@@ -1049,6 +1073,16 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
             }
         }
         qemu_tcg_wait_io_event();
+        CPU_FOREACH(cpu) {
+            if (cpu->exit && !cpu_can_run(cpu)) {
+                remove_cpu = cpu;
+                break;
+            }
+        }
+        if (remove_cpu) {
+            qemu_tcg_destroy_vcpu(remove_cpu);
+            remove_cpu = NULL;
+        }
     }
 
     return NULL;
@@ -1406,6 +1440,9 @@ static void tcg_exec_all(void)
                 break;
             }
         } else if (cpu->stop || cpu->stopped) {
+            if (cpu->exit) {
+                next_cpu = CPU_NEXT(cpu);
+            }
             break;
         }
     }
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 30cb84d..560caef 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -188,6 +188,7 @@ int kvm_has_intx_set_mask(void);
 
 int kvm_init_vcpu(CPUState *cpu);
 int kvm_cpu_exec(CPUState *cpu);
+int kvm_destroy_vcpu(CPUState *cpu);
 
 #ifdef NEED_CPU_H
 
diff --git a/kvm-all.c b/kvm-all.c
index 2f21a4e..6ea3d70 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -71,6 +71,12 @@ typedef struct KVMSlot
 
 typedef struct kvm_dirty_log KVMDirtyLog;
 
+struct KVMParkedVcpu {
+    unsigned long vcpu_id;
+    int kvm_fd;
+    QLIST_ENTRY(KVMParkedVcpu) node;
+};
+
 struct KVMState
 {
     AccelState parent_obj;
@@ -107,6 +113,7 @@ struct KVMState
     QTAILQ_HEAD(msi_hashtab, KVMMSIRoute) msi_hashtab[KVM_MSI_HASHTAB_SIZE];
     bool direct_msi;
 #endif
+    QLIST_HEAD(, KVMParkedVcpu) kvm_parked_vcpus;
 };
 
 #define TYPE_KVM_ACCEL ACCEL_CLASS_NAME("kvm")
@@ -247,6 +254,53 @@ static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot)
     return kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
 }
 
+int kvm_destroy_vcpu(CPUState *cpu)
+{
+    KVMState *s = kvm_state;
+    long mmap_size;
+    struct KVMParkedVcpu *vcpu = NULL;
+    int ret = 0;
+
+    DPRINTF("kvm_destroy_vcpu\n");
+
+    mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
+    if (mmap_size < 0) {
+        ret = mmap_size;
+        DPRINTF("KVM_GET_VCPU_MMAP_SIZE failed\n");
+        goto err;
+    }
+
+    ret = munmap(cpu->kvm_run, mmap_size);
+    if (ret < 0) {
+        goto err;
+    }
+
+    vcpu = g_malloc0(sizeof(*vcpu));
+    vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
+    vcpu->kvm_fd = cpu->kvm_fd;
+    QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
+err:
+    return ret;
+}
+
+static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
+{
+    struct KVMParkedVcpu *cpu;
+
+    QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
+        if (cpu->vcpu_id == vcpu_id) {
+            int kvm_fd;
+
+            QLIST_REMOVE(cpu, node);
+            kvm_fd = cpu->kvm_fd;
+            g_free(cpu);
+            return kvm_fd;
+        }
+    }
+
+    return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
+}
+
 int kvm_init_vcpu(CPUState *cpu)
 {
     KVMState *s = kvm_state;
@@ -255,7 +309,7 @@ int kvm_init_vcpu(CPUState *cpu)
 
     DPRINTF("kvm_init_vcpu\n");
 
-    ret = kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)kvm_arch_vcpu_id(cpu));
+    ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
     if (ret < 0) {
         DPRINTF("kvm_create_vcpu failed\n");
         goto err;
@@ -1448,6 +1502,7 @@ static int kvm_init(MachineState *ms)
 #ifdef KVM_CAP_SET_GUEST_DEBUG
     QTAILQ_INIT(&s->kvm_sw_breakpoints);
 #endif
+    QLIST_INIT(&s->kvm_parked_vcpus);
     s->vmfd = -1;
     s->fd = qemu_open("/dev/kvm", O_RDWR);
     if (s->fd == -1) {
-- 
1.9.3

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Alexandre DERUMIER]

Hello,

I'm currently testing the new cpu unplug features,
Works fine here with debian guests and kernel 3.14.

But I have notice some small potential bugs, but I'm not sure I'm doing it right.

1)first, to unplug cpu, we need an id for cpu

The problem is that the current qemu command line
-smp 1,sockets=2,cores=1,maxcpus=2

for example, will create 1 cpu on apic-id 0  without any id, so we can't unplug it.


So, I have tried with

-smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0

But this give me an error:
"-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"

(also try to set -smp 0, but it's not working).



2) second problem, if I start with
-smp 1,sockets=2,cores=1,maxcpus=2

then hmp: 
device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

then hmp : device_del cpu1

Got an error:"
This is the last cpu, should not be removed!"



This is coming from
[PATCH 06/12] pc: add cpu hot unplug request callback	support
+    if (smp_cpus == 1) {
+        error_setg(&local_err,
+                   "This is the last cpu, should not be removed!");
+        goto out;
+    }



So, the only way unplug is working for me, is to start with -smp 2 minimum
-smp 2,sockets=2,cores=1,maxcpus=4

Then I can hotplug|unplug cpuid >= 2



Regards,

Alexandre Derumier

----- Mail original -----
De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
À: "qemu-devel" <qemu-devel@nongnu.org>
Cc: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
Envoyé: Mercredi 14 Janvier 2015 08:44:53
Objet: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

This series is based on chen fan's previous i386 cpu hot remove patchset: 
https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html 

Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest 
OS remove one vCPU online, the fireware will store removed bitmap to 
QEMU, then QEMU could know to notify the assigned vCPU of exiting. 
Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from 
QEMU itself. 

The whole work is based on the new hot plug/unplug framework, ,the unplug request 
callback does the pre-check and send the request, unplug callback does the 
removal handling. 

This series depends on tangchen's common hot plug/unplug enhance patchset. 
[RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug 
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html 

The is the second half of the previous series: 
[RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support 
https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html 

If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu' 
patchset first: 
[PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support 
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html 

--- 
Changelog since v1: 
-rebase on the latest version. 
-delete patch i386/cpu: add instance finalize callback, and put it into patchset 
[PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support. 

Changelog since RFC: 
-splited the i386 cpu hot remove into single thread. 
-replaced apic_no with apic_id, so does the related stuff to make it 
work with arbitrary CPU hotadd. 
-add the icc_device_unrealize callback to handle apic unrealize. 
-rework on the new hot plug/unplug platform. 
--- 

Chen Fan (2): 
x86: add x86_cpu_unrealizefn() for cpu apic remove 
cpu hotplug: implement function cpu_status_write() for vcpu ejection 

Gu Zheng (5): 
acpi/cpu: add cpu hot unplug request callback function 
acpi/piix4: add cpu hot unplug callback support 
acpi/ich9: add cpu hot unplug support 
pc: add cpu hot unplug callback support 
cpus: reclaim allocated vCPU objects 

Zhu Guihua (4): 
acpi/piix4: add cpu hot unplug request callback support 
acpi/ich9: add cpu hot unplug request callback support 
pc: add cpu hot unplug request callback support 
acpi/cpu: add cpu hot unplug callback function 

cpus.c | 44 ++++++++++++++++++++ 
hw/acpi/cpu_hotplug.c | 88 ++++++++++++++++++++++++++++++++++++--- 
hw/acpi/ich9.c | 17 ++++++-- 
hw/acpi/piix4.c | 12 +++++- 
hw/core/qdev.c | 2 +- 
hw/cpu/icc_bus.c | 11 +++++ 
hw/i386/acpi-dsdt-cpu-hotplug.dsl | 6 ++- 
hw/i386/kvm/apic.c | 8 ++++ 
hw/i386/pc.c | 62 +++++++++++++++++++++++++-- 
hw/intc/apic.c | 10 +++++ 
hw/intc/apic_common.c | 21 ++++++---- 
include/hw/acpi/cpu_hotplug.h | 8 ++++ 
include/hw/cpu/icc_bus.h | 1 + 
include/hw/i386/apic_internal.h | 1 + 
include/hw/qdev-core.h | 1 + 
include/qom/cpu.h | 9 ++++ 
include/sysemu/kvm.h | 1 + 
kvm-all.c | 57 ++++++++++++++++++++++++- 
target-i386/cpu.c | 46 ++++++++++++++++++++ 
19 files changed, 378 insertions(+), 27 deletions(-) 

-- 
1.9.3 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote:
> Hello,
> 
> I'm currently testing the new cpu unplug features,
> Works fine here with debian guests and kernel 3.14.
> 

Thanks for your test.

> But I have notice some small potential bugs, but I'm not sure I'm doing it right.
> 
> 1)first, to unplug cpu, we need an id for cpu
> 

Yes, if you want to unplug cpu, you must have an id for cpu.

> The problem is that the current qemu command line
> -smp 1,sockets=2,cores=1,maxcpus=2
> 
> for example, will create 1 cpu on apic-id 0  without any id, so we can't unplug it.
> 
> 
> So, I have tried with
> 
> -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0
> 
> But this give me an error:
> "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists"
> 

APIC ID 0 was used by the cpu of '-smp 1'.
So you should use apic-id=1

> (also try to set -smp 0, but it's not working).
> 
> 
> 
> 2) second problem, if I start with
> -smp 1,sockets=2,cores=1,maxcpus=2
> 
> then hmp: 
> device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1
> 
> then hmp : device_del cpu1
> 
> Got an error:"
> This is the last cpu, should not be removed!"
> 
> 

Oh, it's our problem, thanks for your pointing out.
I will fix it in next version.

Regards,
Zhu

> 
> This is coming from
> [PATCH 06/12] pc: add cpu hot unplug request callback	support
> +    if (smp_cpus == 1) {
> +        error_setg(&local_err,
> +                   "This is the last cpu, should not be removed!");
> +        goto out;
> +    }
> 
> 
> 
> So, the only way unplug is working for me, is to start with -smp 2 minimum
> -smp 2,sockets=2,cores=1,maxcpus=4
> 
> Then I can hotplug|unplug cpuid >= 2
> 
> 
> 
> Regards,
> 
> Alexandre Derumier
[...]

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Alexandre DERUMIER]

Thanks for your reply.

2 others things:

1)
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu

starting with a guest with 3cpus:

guest: #ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu2

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972
  CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973
  CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102


then unplug cpu2
hmp : device_del cpu2

guest:

dmesg:
[  176.219754] Unregister pv shared memory for cpu 2
[  176.278881] smpboot: CPU 2 is now offline

#ls -lah /sys/devices/system/ |grep cpu
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1

hmp: # info cpus
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972
  CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973
  CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102




2)when numa is used, the hotplugged cpu is always on numa node 0
  (cpu_add or device_add cpu)


starting a guest, with 2 sockets,1 cores

-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp:
# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 1
node 1 size: 256 MB

ok

now

starting with same topology, but with 1cpu at start
-smp 2,sockets=2,cores=1,maxcpus=2
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

# info numa
2 nodes
node 0 cpus: 0
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

hotpluging a cpu
# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1

# info numa
2 nodes
node 0 cpus: 0 1
node 0 size: 256 MB
node 1 cpus: 
node 1 size: 256 MB

cpu1 should be on node1, not node0.


Regards,

Alexandre

----- Mail original -----
De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
À: "aderumier" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
Envoyé: Lundi 26 Janvier 2015 03:01:48
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> Hello, 
> 
> I'm currently testing the new cpu unplug features, 
> Works fine here with debian guests and kernel 3.14. 
> 

Thanks for your test. 

> But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> 
> 1)first, to unplug cpu, we need an id for cpu 
> 

Yes, if you want to unplug cpu, you must have an id for cpu. 

> The problem is that the current qemu command line 
> -smp 1,sockets=2,cores=1,maxcpus=2 
> 
> for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> 
> 
> So, I have tried with 
> 
> -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> 
> But this give me an error: 
> "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> 

APIC ID 0 was used by the cpu of '-smp 1'. 
So you should use apic-id=1 

> (also try to set -smp 0, but it's not working). 
> 
> 
> 
> 2) second problem, if I start with 
> -smp 1,sockets=2,cores=1,maxcpus=2 
> 
> then hmp: 
> device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> 
> then hmp : device_del cpu1 
> 
> Got an error:" 
> This is the last cpu, should not be removed!" 
> 
> 

Oh, it's our problem, thanks for your pointing out. 
I will fix it in next version. 

Regards, 
Zhu 

> 
> This is coming from 
> [PATCH 06/12] pc: add cpu hot unplug request callback support 
> + if (smp_cpus == 1) { 
> + error_setg(&local_err, 
> + "This is the last cpu, should not be removed!"); 
> + goto out; 
> + } 
> 
> 
> 
> So, the only way unplug is working for me, is to start with -smp 2 minimum 
> -smp 2,sockets=2,cores=1,maxcpus=4 
> 
> Then I can hotplug|unplug cpuid >= 2 
> 
> 
> 
> Regards, 
> 
> Alexandre Derumier 
[...] 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Alexandre DERUMIER]

>>2)when numa is used, the hotplugged cpu is always on numa node 0 
>>(cpu_add or device_add cpu) 

About this, it seem to be a display bug in "info numa",
cpu is correctly assigned to numa node1 in the guest.


----- Mail original -----
De: "aderumier" <aderumier@odiso.com>
À: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
Envoyé: Lundi 26 Janvier 2015 04:19:52
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

Thanks for your reply. 

2 others things: 

1) 
on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu 

starting with a guest with 3cpus: 

guest: #ls -lah /sys/devices/system/ |grep cpu 
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 

hmp: # info cpus 
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 


then unplug cpu2 
hmp : device_del cpu2 

guest: 

dmesg: 
[ 176.219754] Unregister pv shared memory for cpu 2 
[ 176.278881] smpboot: CPU 2 is now offline 

#ls -lah /sys/devices/system/ |grep cpu 
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 

hmp: # info cpus 
* CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 




2)when numa is used, the hotplugged cpu is always on numa node 0 
(cpu_add or device_add cpu) 


starting a guest, with 2 sockets,1 cores 

-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

hmp: 
# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 1 
node 1 size: 256 MB 

ok 

now 

starting with same topology, but with 1cpu at start 
-smp 2,sockets=2,cores=1,maxcpus=2 
-object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
-object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 

# info numa 
2 nodes 
node 0 cpus: 0 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

hotpluging a cpu 
# device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 

# info numa 
2 nodes 
node 0 cpus: 0 1 
node 0 size: 256 MB 
node 1 cpus: 
node 1 size: 256 MB 

cpu1 should be on node1, not node0. 


Regards, 

Alexandre 

----- Mail original ----- 
De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com> 
À: "aderumier" <aderumier@odiso.com> 
Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de> 
Envoyé: Lundi 26 Janvier 2015 03:01:48 
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support 

On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> Hello, 
> 
> I'm currently testing the new cpu unplug features, 
> Works fine here with debian guests and kernel 3.14. 
> 

Thanks for your test. 

> But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> 
> 1)first, to unplug cpu, we need an id for cpu 
> 

Yes, if you want to unplug cpu, you must have an id for cpu. 

> The problem is that the current qemu command line 
> -smp 1,sockets=2,cores=1,maxcpus=2 
> 
> for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> 
> 
> So, I have tried with 
> 
> -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> 
> But this give me an error: 
> "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> 

APIC ID 0 was used by the cpu of '-smp 1'. 
So you should use apic-id=1 

> (also try to set -smp 0, but it's not working). 
> 
> 
> 
> 2) second problem, if I start with 
> -smp 1,sockets=2,cores=1,maxcpus=2 
> 
> then hmp: 
> device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> 
> then hmp : device_del cpu1 
> 
> Got an error:" 
> This is the last cpu, should not be removed!" 
> 
> 

Oh, it's our problem, thanks for your pointing out. 
I will fix it in next version. 

Regards, 
Zhu 

> 
> This is coming from 
> [PATCH 06/12] pc: add cpu hot unplug request callback support 
> + if (smp_cpus == 1) { 
> + error_setg(&local_err, 
> + "This is the last cpu, should not be removed!"); 
> + goto out; 
> + } 
> 
> 
> 
> So, the only way unplug is working for me, is to start with -smp 2 minimum 
> -smp 2,sockets=2,cores=1,maxcpus=4 
> 
> Then I can hotplug|unplug cpuid >= 2 
> 
> 
> 
> Regards, 
> 
> Alexandre Derumier 
[...] 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Mon, 2015-01-26 at 04:25 +0100, Alexandre DERUMIER wrote:
> >>2)when numa is used, the hotplugged cpu is always on numa node 0 
> >>(cpu_add or device_add cpu) 
> 
> About this, it seem to be a display bug in "info numa",
> cpu is correctly assigned to numa node1 in the guest.
> 
> 

Now in the guest, VCPUs are assigned round-robin.
But in qemu, vcpus' property 'numa_node' was assigned the default
value 0.

We have made a patchset to solve this successfully.
But we plan to send the patchset to the community later.

Regards,
Zhu

> ----- Mail original -----
> De: "aderumier" <aderumier@odiso.com>
> À: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
> Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
> Envoyé: Lundi 26 Janvier 2015 04:19:52
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> Thanks for your reply. 
> 
> 2 others things: 
> 
> 1) 
> on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu 
> 
> starting with a guest with 3cpus: 
> 
> guest: #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> then unplug cpu2 
> hmp : device_del cpu2 
> 
> guest: 
> 
> dmesg: 
> [ 176.219754] Unregister pv shared memory for cpu 2 
> [ 176.278881] smpboot: CPU 2 is now offline 
> 
> #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> 
> 
> 2)when numa is used, the hotplugged cpu is always on numa node 0 
> (cpu_add or device_add cpu) 
> 
> 
> starting a guest, with 2 sockets,1 cores 
> 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> hmp: 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 1 
> node 1 size: 256 MB 
> 
> ok 
> 
> now 
> 
> starting with same topology, but with 1cpu at start 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> hotpluging a cpu 
> # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 1 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> cpu1 should be on node1, not node0. 
> 
> 
> Regards, 
> 
> Alexandre 
> 
> ----- Mail original ----- 
> De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com> 
> À: "aderumier" <aderumier@odiso.com> 
> Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de> 
> Envoyé: Lundi 26 Janvier 2015 03:01:48 
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support 
> 
> On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > Hello, 
> > 
> > I'm currently testing the new cpu unplug features, 
> > Works fine here with debian guests and kernel 3.14. 
> > 
> 
> Thanks for your test. 
> 
> > But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> > 
> > 1)first, to unplug cpu, we need an id for cpu 
> > 
> 
> Yes, if you want to unplug cpu, you must have an id for cpu. 
> 
> > The problem is that the current qemu command line 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> > 
> > 
> > So, I have tried with 
> > 
> > -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > 
> > But this give me an error: 
> > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > 
> 
> APIC ID 0 was used by the cpu of '-smp 1'. 
> So you should use apic-id=1 
> 
> > (also try to set -smp 0, but it's not working). 
> > 
> > 
> > 
> > 2) second problem, if I start with 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > then hmp: 
> > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > then hmp : device_del cpu1 
> > 
> > Got an error:" 
> > This is the last cpu, should not be removed!" 
> > 
> > 
> 
> Oh, it's our problem, thanks for your pointing out. 
> I will fix it in next version. 
> 
> Regards, 
> Zhu 
> 
> > 
> > This is coming from 
> > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > + if (smp_cpus == 1) { 
> > + error_setg(&local_err, 
> > + "This is the last cpu, should not be removed!"); 
> > + goto out; 
> > + } 
> > 
> > 
> > 
> > So, the only way unplug is working for me, is to start with -smp 2 minimum 
> > -smp 2,sockets=2,cores=1,maxcpus=4 
> > 
> > Then I can hotplug|unplug cpuid >= 2 
> > 
> > 
> > 
> > Regards, 
> > 
> > Alexandre Derumier 
> [...] 
> 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Mon, 2015-01-26 at 04:19 +0100, Alexandre DERUMIER wrote:
> Thanks for your reply.
> 
> 2 others things:
> 
> 1)
> on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu
> 

About this, I can do it successfully on my qemu.
So can you tell us more information about your operation?

And I found the patchset you applied is the last version in your last
email. I think you'd better apply the latest version.

Regards,
Zhu

> starting with a guest with 3cpus:
> 
> guest: #ls -lah /sys/devices/system/ |grep cpu
> drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
> drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1
> drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu2
> 
> hmp: # info cpus
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972
>   CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973
>   CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102
> 
> 
> then unplug cpu2
> hmp : device_del cpu2
> 
> guest:
> 
> dmesg:
> [  176.219754] Unregister pv shared memory for cpu 2
> [  176.278881] smpboot: CPU 2 is now offline
> 
> #ls -lah /sys/devices/system/ |grep cpu
> drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu0
> drwxr-xr-x 6 root root    0 Jan 25 22:16 cpu1
> 
> hmp: # info cpus
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972
>   CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973
>   CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102
> 
> 
> 
> 
> 2)when numa is used, the hotplugged cpu is always on numa node 0
>   (cpu_add or device_add cpu)
> 
> 
> starting a guest, with 2 sockets,1 cores
> 
> -smp 2,sockets=2,cores=1,maxcpus=2
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> hmp:
> # info numa
> 2 nodes
> node 0 cpus: 0
> node 0 size: 256 MB
> node 1 cpus: 1
> node 1 size: 256 MB
> 
> ok
> 
> now
> 
> starting with same topology, but with 1cpu at start
> -smp 2,sockets=2,cores=1,maxcpus=2
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> # info numa
> 2 nodes
> node 0 cpus: 0
> node 0 size: 256 MB
> node 1 cpus: 
> node 1 size: 256 MB
> 
> hotpluging a cpu
> # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1
> 
> # info numa
> 2 nodes
> node 0 cpus: 0 1
> node 0 size: 256 MB
> node 1 cpus: 
> node 1 size: 256 MB
> 
> cpu1 should be on node1, not node0.
> 
> 
> Regards,
> 
> Alexandre
> 
> ----- Mail original -----
> De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
> À: "aderumier" <aderumier@odiso.com>
> Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
> Envoyé: Lundi 26 Janvier 2015 03:01:48
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > Hello, 
> > 
> > I'm currently testing the new cpu unplug features, 
> > Works fine here with debian guests and kernel 3.14. 
> > 
> 
> Thanks for your test. 
> 
> > But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> > 
> > 1)first, to unplug cpu, we need an id for cpu 
> > 
> 
> Yes, if you want to unplug cpu, you must have an id for cpu. 
> 
> > The problem is that the current qemu command line 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> > 
> > 
> > So, I have tried with 
> > 
> > -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > 
> > But this give me an error: 
> > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > 
> 
> APIC ID 0 was used by the cpu of '-smp 1'. 
> So you should use apic-id=1 
> 
> > (also try to set -smp 0, but it's not working). 
> > 
> > 
> > 
> > 2) second problem, if I start with 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > then hmp: 
> > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > then hmp : device_del cpu1 
> > 
> > Got an error:" 
> > This is the last cpu, should not be removed!" 
> > 
> > 
> 
> Oh, it's our problem, thanks for your pointing out. 
> I will fix it in next version. 
> 
> Regards, 
> Zhu 
> 
> > 
> > This is coming from 
> > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > + if (smp_cpus == 1) { 
> > + error_setg(&local_err, 
> > + "This is the last cpu, should not be removed!"); 
> > + goto out; 
> > + } 
> > 
> > 
> > 
> > So, the only way unplug is working for me, is to start with -smp 2 minimum 
> > -smp 2,sockets=2,cores=1,maxcpus=4 
> > 
> > Then I can hotplug|unplug cpuid >= 2 
> > 
> > 
> > 
> > Regards, 
> > 
> > Alexandre Derumier 
> [...] 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Alexandre DERUMIER]

>>About this, I can do it successfully on my qemu. 
>>So can you tell us more information about your operation? 
simply start with

-smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2

then

#device_del cpu2

Guest return
[  324.195024] Unregister pv shared memory for cpu 2
[  324.250579] smpboot: CPU 2 is now offline

but cpu is not remove in qemu.

I had also try to eject manualy from guest
echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject

But I have this error message
ACPI: \_SB_.CP02: Eject incomplete - status 0xf

(maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)


>>And I found the patchset you applied is the last version in your last 
>>email. I think you'd better apply the latest version. 

I have applied on top of git theses patches series

https://www.mail-archive.com/qemu-devel%40nongnu.org/msg272745.html
[Qemu-devel] [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug.
https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
[Qemu-devel] [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu	support
https://www.mail-archive.com/qemu-devel@nongnu.org/msg273870.html
[Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

----- Mail original -----
De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
À: "aderumier" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
Envoyé: Lundi 26 Janvier 2015 04:47:13
Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

On Mon, 2015-01-26 at 04:19 +0100, Alexandre DERUMIER wrote: 
> Thanks for your reply. 
> 
> 2 others things: 
> 
> 1) 
> on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu 
> 

About this, I can do it successfully on my qemu. 
So can you tell us more information about your operation? 

And I found the patchset you applied is the last version in your last 
email. I think you'd better apply the latest version. 

Regards, 
Zhu 

> starting with a guest with 3cpus: 
> 
> guest: #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> then unplug cpu2 
> hmp : device_del cpu2 
> 
> guest: 
> 
> dmesg: 
> [ 176.219754] Unregister pv shared memory for cpu 2 
> [ 176.278881] smpboot: CPU 2 is now offline 
> 
> #ls -lah /sys/devices/system/ |grep cpu 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> 
> hmp: # info cpus 
> * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> 
> 
> 
> 
> 2)when numa is used, the hotplugged cpu is always on numa node 0 
> (cpu_add or device_add cpu) 
> 
> 
> starting a guest, with 2 sockets,1 cores 
> 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> hmp: 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 1 
> node 1 size: 256 MB 
> 
> ok 
> 
> now 
> 
> starting with same topology, but with 1cpu at start 
> -smp 2,sockets=2,cores=1,maxcpus=2 
> -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> hotpluging a cpu 
> # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> 
> # info numa 
> 2 nodes 
> node 0 cpus: 0 1 
> node 0 size: 256 MB 
> node 1 cpus: 
> node 1 size: 256 MB 
> 
> cpu1 should be on node1, not node0. 
> 
> 
> Regards, 
> 
> Alexandre 
> 
> ----- Mail original ----- 
> De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com> 
> À: "aderumier" <aderumier@odiso.com> 
> Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de> 
> Envoyé: Lundi 26 Janvier 2015 03:01:48 
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support 
> 
> On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > Hello, 
> > 
> > I'm currently testing the new cpu unplug features, 
> > Works fine here with debian guests and kernel 3.14. 
> > 
> 
> Thanks for your test. 
> 
> > But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> > 
> > 1)first, to unplug cpu, we need an id for cpu 
> > 
> 
> Yes, if you want to unplug cpu, you must have an id for cpu. 
> 
> > The problem is that the current qemu command line 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> > 
> > 
> > So, I have tried with 
> > 
> > -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > 
> > But this give me an error: 
> > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > 
> 
> APIC ID 0 was used by the cpu of '-smp 1'. 
> So you should use apic-id=1 
> 
> > (also try to set -smp 0, but it's not working). 
> > 
> > 
> > 
> > 2) second problem, if I start with 
> > -smp 1,sockets=2,cores=1,maxcpus=2 
> > 
> > then hmp: 
> > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > then hmp : device_del cpu1 
> > 
> > Got an error:" 
> > This is the last cpu, should not be removed!" 
> > 
> > 
> 
> Oh, it's our problem, thanks for your pointing out. 
> I will fix it in next version. 
> 
> Regards, 
> Zhu 
> 
> > 
> > This is coming from 
> > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > + if (smp_cpus == 1) { 
> > + error_setg(&local_err, 
> > + "This is the last cpu, should not be removed!"); 
> > + goto out; 
> > + } 
> > 
> > 
> > 
> > So, the only way unplug is working for me, is to start with -smp 2 minimum 
> > -smp 2,sockets=2,cores=1,maxcpus=4 
> > 
> > Then I can hotplug|unplug cpuid >= 2 
> > 
> > 
> > 
> > Regards, 
> > 
> > Alexandre Derumier 
> [...] 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Mon, 2015-01-26 at 12:27 +0100, Alexandre DERUMIER wrote:
> >>About this, I can do it successfully on my qemu. 
> >>So can you tell us more information about your operation? 
> simply start with
> 
> -smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2
> 
> then
> 
> #device_del cpu2
> 
> Guest return
> [  324.195024] Unregister pv shared memory for cpu 2
> [  324.250579] smpboot: CPU 2 is now offline
> 
> but cpu is not remove in qemu.
> 
> I had also try to eject manualy from guest
> echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject
> 
> But I have this error message
> ACPI: \_SB_.CP02: Eject incomplete - status 0xf
> 
> (maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)
> 

According to your description, the guest has invoked _EJ0 method, but
the method was executed unsuccessfully.
Thus, only hot-unplug request was invoked in qemu, but hot-unplug was
not so that cpu was still in qemu.

Did you test any other kernel version? I will try kernel 3.16 to
investigate this problem later.

Regards,
Zhu

> 
> >>And I found the patchset you applied is the last version in your last 
> >>email. I think you'd better apply the latest version. 
> 
> I have applied on top of git theses patches series
> 
> https://www.mail-archive.com/qemu-devel%40nongnu.org/msg272745.html
> [Qemu-devel] [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug.
> https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
> [Qemu-devel] [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu	support
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg273870.html
> [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> ----- Mail original -----
> De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com>
> À: "aderumier" <aderumier@odiso.com>
> Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de>
> Envoyé: Lundi 26 Janvier 2015 04:47:13
> Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support
> 
> On Mon, 2015-01-26 at 04:19 +0100, Alexandre DERUMIER wrote: 
> > Thanks for your reply. 
> > 
> > 2 others things: 
> > 
> > 1) 
> > on cpu unplug, I see that the cpu is correctly removed from my linux guest but not from qemu 
> > 
> 
> About this, I can do it successfully on my qemu. 
> So can you tell us more information about your operation? 
> 
> And I found the patchset you applied is the last version in your last 
> email. I think you'd better apply the latest version. 
> 
> Regards, 
> Zhu 
> 
> > starting with a guest with 3cpus: 
> > 
> > guest: #ls -lah /sys/devices/system/ |grep cpu 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu2 
> > 
> > hmp: # info cpus 
> > * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> > CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> > CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> > 
> > 
> > then unplug cpu2 
> > hmp : device_del cpu2 
> > 
> > guest: 
> > 
> > dmesg: 
> > [ 176.219754] Unregister pv shared memory for cpu 2 
> > [ 176.278881] smpboot: CPU 2 is now offline 
> > 
> > #ls -lah /sys/devices/system/ |grep cpu 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu0 
> > drwxr-xr-x 6 root root 0 Jan 25 22:16 cpu1 
> > 
> > hmp: # info cpus 
> > * CPU #0: pc=0xffffffff81057022 (halted) thread_id=24972 
> > CPU #1: pc=0xffffffff81057022 (halted) thread_id=24973 
> > CPU #2: pc=0xffffffff81048bc1 (halted) thread_id=25102 
> > 
> > 
> > 
> > 
> > 2)when numa is used, the hotplugged cpu is always on numa node 0 
> > (cpu_add or device_add cpu) 
> > 
> > 
> > starting a guest, with 2 sockets,1 cores 
> > 
> > -smp 2,sockets=2,cores=1,maxcpus=2 
> > -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> > -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> > 
> > hmp: 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 
> > node 0 size: 256 MB 
> > node 1 cpus: 1 
> > node 1 size: 256 MB 
> > 
> > ok 
> > 
> > now 
> > 
> > starting with same topology, but with 1cpu at start 
> > -smp 2,sockets=2,cores=1,maxcpus=2 
> > -object memory-backend-ram,size=256M,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0 
> > -object memory-backend-ram,size=256M,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1 
> > 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 
> > node 0 size: 256 MB 
> > node 1 cpus: 
> > node 1 size: 256 MB 
> > 
> > hotpluging a cpu 
> > # device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > 
> > # info numa 
> > 2 nodes 
> > node 0 cpus: 0 1 
> > node 0 size: 256 MB 
> > node 1 cpus: 
> > node 1 size: 256 MB 
> > 
> > cpu1 should be on node1, not node0. 
> > 
> > 
> > Regards, 
> > 
> > Alexandre 
> > 
> > ----- Mail original ----- 
> > De: "Zhu Guihua" <zhugh.fnst@cn.fujitsu.com> 
> > À: "aderumier" <aderumier@odiso.com> 
> > Cc: "qemu-devel" <qemu-devel@nongnu.org>, tangchen@cn.fujitsu.com, "guz fnst" <guz.fnst@cn.fujitsu.com>, "isimatu yasuaki" <isimatu.yasuaki@jp.fujitsu.com>, "Anshul Makkar" <anshul.makkar@profitbricks.com>, "chen fan fnst" <chen.fan.fnst@cn.fujitsu.com>, "Igor Mammedov" <imammedo@redhat.com>, "afaerber" <afaerber@suse.de> 
> > Envoyé: Lundi 26 Janvier 2015 03:01:48 
> > Objet: Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support 
> > 
> > On Fri, 2015-01-23 at 11:24 +0100, Alexandre DERUMIER wrote: 
> > > Hello, 
> > > 
> > > I'm currently testing the new cpu unplug features, 
> > > Works fine here with debian guests and kernel 3.14. 
> > > 
> > 
> > Thanks for your test. 
> > 
> > > But I have notice some small potential bugs, but I'm not sure I'm doing it right. 
> > > 
> > > 1)first, to unplug cpu, we need an id for cpu 
> > > 
> > 
> > Yes, if you want to unplug cpu, you must have an id for cpu. 
> > 
> > > The problem is that the current qemu command line 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 
> > > 
> > > for example, will create 1 cpu on apic-id 0 without any id, so we can't unplug it. 
> > > 
> > > 
> > > So, I have tried with 
> > > 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 -device kvm64-x86_64-cpu,apic-id=0,id=cpu0 
> > > 
> > > But this give me an error: 
> > > "-device kvm64-x86_64-cpu,apic-id=0,id=cpu0: CPU with APIC ID 0 exists" 
> > > 
> > 
> > APIC ID 0 was used by the cpu of '-smp 1'. 
> > So you should use apic-id=1 
> > 
> > > (also try to set -smp 0, but it's not working). 
> > > 
> > > 
> > > 
> > > 2) second problem, if I start with 
> > > -smp 1,sockets=2,cores=1,maxcpus=2 
> > > 
> > > then hmp: 
> > > device_add kvm64-x86_64-cpu,apic-id=1,id=cpu1 
> > > 
> > > then hmp : device_del cpu1 
> > > 
> > > Got an error:" 
> > > This is the last cpu, should not be removed!" 
> > > 
> > > 
> > 
> > Oh, it's our problem, thanks for your pointing out. 
> > I will fix it in next version. 
> > 
> > Regards, 
> > Zhu 
> > 
> > > 
> > > This is coming from 
> > > [PATCH 06/12] pc: add cpu hot unplug request callback support 
> > > + if (smp_cpus == 1) { 
> > > + error_setg(&local_err, 
> > > + "This is the last cpu, should not be removed!"); 
> > > + goto out; 
> > > + } 
> > > 
> > > 
> > > 
> > > So, the only way unplug is working for me, is to start with -smp 2 minimum 
> > > -smp 2,sockets=2,cores=1,maxcpus=4 
> > > 
> > > Then I can hotplug|unplug cpuid >= 2 
> > > 
> > > 
> > > 
> > > Regards, 
> > > 
> > > Alexandre Derumier 
> > [...] 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Bharata B Rao]

On Tue, Jan 27, 2015 at 7:30 AM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> On Mon, 2015-01-26 at 12:27 +0100, Alexandre DERUMIER wrote:
>> >>About this, I can do it successfully on my qemu.
>> >>So can you tell us more information about your operation?
>> simply start with
>>
>> -smp 2,sockets=2,cores=2,maxcpus=4 -device kvm64-x86_64-cpu,apic-id=2,id=cpu2
>>
>> then
>>
>> #device_del cpu2
>>
>> Guest return
>> [  324.195024] Unregister pv shared memory for cpu 2
>> [  324.250579] smpboot: CPU 2 is now offline
>>
>> but cpu is not remove in qemu.
>>
>> I had also try to eject manualy from guest
>> echo 1 > /sys/bus/acpi/devices/LNXCPU\:02/eject
>>
>> But I have this error message
>> ACPI: \_SB_.CP02: Eject incomplete - status 0xf
>>
>> (maybe is it normal ? or maybe is it a guest bug (kernel 3.16 from debian wheezy backports) ?)
>>
>
> According to your description, the guest has invoked _EJ0 method, but
> the method was executed unsuccessfully.
> Thus, only hot-unplug request was invoked in qemu, but hot-unplug was
> not so that cpu was still in qemu.
>
> Did you test any other kernel version? I will try kernel 3.16 to
> investigate this problem later.

Not sure if it is exactly same, but I had similar problem as seen by
Alexandre when I had tried this patchset earlier and it was resolved
by using iasl and regenerating the ACPI tables.

Ref: http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg02993.html

Regards,
Bharata.

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhi Yong Wu]

HI,

Can you push the patchset to a branch on github? It will be convenient
for other guys to do some tests.

On Wed, Jan 14, 2015 at 3:44 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> This series is based on chen fan's previous i386 cpu hot remove patchset:
> https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html
>
> Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
> OS remove one vCPU online, the fireware will store removed bitmap to
> QEMU, then QEMU could know to notify the assigned vCPU of exiting.
> Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
> QEMU itself.
>
> The whole work is based on the new hot plug/unplug framework, ,the unplug request
> callback does the pre-check and send the request, unplug callback does the
> removal handling.
>
> This series depends on tangchen's common hot plug/unplug enhance patchset.
> [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
> https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html
>
> The is the second half of the previous series:
> [RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
> https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html
>
> If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
> patchset first:
> [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
> https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
>
> ---
> Changelog since v1:
>  -rebase on the latest version.
>  -delete patch i386/cpu: add instance finalize callback, and put it into patchset
>   [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.
>
> Changelog since RFC:
>  -splited the i386 cpu hot remove into single thread.
>  -replaced apic_no with apic_id, so does the related stuff to make it
>   work with arbitrary CPU hotadd.
>  -add the icc_device_unrealize callback to handle apic unrealize.
>  -rework on the new hot plug/unplug platform.
> ---
>
> Chen Fan (2):
>   x86: add x86_cpu_unrealizefn() for cpu apic remove
>   cpu hotplug: implement function cpu_status_write() for vcpu ejection
>
> Gu Zheng (5):
>   acpi/cpu: add cpu hot unplug request callback function
>   acpi/piix4: add cpu hot unplug callback support
>   acpi/ich9: add cpu hot unplug support
>   pc: add cpu hot unplug callback support
>   cpus: reclaim allocated vCPU objects
>
> Zhu Guihua (4):
>   acpi/piix4: add cpu hot unplug request callback support
>   acpi/ich9: add cpu hot unplug request callback support
>   pc: add cpu hot unplug request callback support
>   acpi/cpu: add cpu hot unplug callback function
>
>  cpus.c                            | 44 ++++++++++++++++++++
>  hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
>  hw/acpi/ich9.c                    | 17 ++++++--
>  hw/acpi/piix4.c                   | 12 +++++-
>  hw/core/qdev.c                    |  2 +-
>  hw/cpu/icc_bus.c                  | 11 +++++
>  hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
>  hw/i386/kvm/apic.c                |  8 ++++
>  hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
>  hw/intc/apic.c                    | 10 +++++
>  hw/intc/apic_common.c             | 21 ++++++----
>  include/hw/acpi/cpu_hotplug.h     |  8 ++++
>  include/hw/cpu/icc_bus.h          |  1 +
>  include/hw/i386/apic_internal.h   |  1 +
>  include/hw/qdev-core.h            |  1 +
>  include/qom/cpu.h                 |  9 ++++
>  include/sysemu/kvm.h              |  1 +
>  kvm-all.c                         | 57 ++++++++++++++++++++++++-
>  target-i386/cpu.c                 | 46 ++++++++++++++++++++
>  19 files changed, 378 insertions(+), 27 deletions(-)
>
> --
> 1.9.3
>
>



-- 
Regards,

Zhi Yong Wu

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Tue, 2015-02-03 at 16:41 +0800, Zhi Yong Wu wrote:
> HI,
> 
> Can you push the patchset to a branch on github? It will be convenient
> for other guys to do some tests.

sorry for late reply.
I had pushed into https://github.com/zhugh/qemu.git
The patchset is on branch cpu-hotplug.

Welcome to test, Thanks.

Regards,
Zhu

> 
> On Wed, Jan 14, 2015 at 3:44 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> > This series is based on chen fan's previous i386 cpu hot remove patchset:
> > https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html
> >
> > Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
> > OS remove one vCPU online, the fireware will store removed bitmap to
> > QEMU, then QEMU could know to notify the assigned vCPU of exiting.
> > Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
> > QEMU itself.
> >
> > The whole work is based on the new hot plug/unplug framework, ,the unplug request
> > callback does the pre-check and send the request, unplug callback does the
> > removal handling.
> >
> > This series depends on tangchen's common hot plug/unplug enhance patchset.
> > [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html
> >
> > The is the second half of the previous series:
> > [RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
> > https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html
> >
> > If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
> > patchset first:
> > [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
> >
> > ---
> > Changelog since v1:
> >  -rebase on the latest version.
> >  -delete patch i386/cpu: add instance finalize callback, and put it into patchset
> >   [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.
> >
> > Changelog since RFC:
> >  -splited the i386 cpu hot remove into single thread.
> >  -replaced apic_no with apic_id, so does the related stuff to make it
> >   work with arbitrary CPU hotadd.
> >  -add the icc_device_unrealize callback to handle apic unrealize.
> >  -rework on the new hot plug/unplug platform.
> > ---
> >
> > Chen Fan (2):
> >   x86: add x86_cpu_unrealizefn() for cpu apic remove
> >   cpu hotplug: implement function cpu_status_write() for vcpu ejection
> >
> > Gu Zheng (5):
> >   acpi/cpu: add cpu hot unplug request callback function
> >   acpi/piix4: add cpu hot unplug callback support
> >   acpi/ich9: add cpu hot unplug support
> >   pc: add cpu hot unplug callback support
> >   cpus: reclaim allocated vCPU objects
> >
> > Zhu Guihua (4):
> >   acpi/piix4: add cpu hot unplug request callback support
> >   acpi/ich9: add cpu hot unplug request callback support
> >   pc: add cpu hot unplug request callback support
> >   acpi/cpu: add cpu hot unplug callback function
> >
> >  cpus.c                            | 44 ++++++++++++++++++++
> >  hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
> >  hw/acpi/ich9.c                    | 17 ++++++--
> >  hw/acpi/piix4.c                   | 12 +++++-
> >  hw/core/qdev.c                    |  2 +-
> >  hw/cpu/icc_bus.c                  | 11 +++++
> >  hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
> >  hw/i386/kvm/apic.c                |  8 ++++
> >  hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
> >  hw/intc/apic.c                    | 10 +++++
> >  hw/intc/apic_common.c             | 21 ++++++----
> >  include/hw/acpi/cpu_hotplug.h     |  8 ++++
> >  include/hw/cpu/icc_bus.h          |  1 +
> >  include/hw/i386/apic_internal.h   |  1 +
> >  include/hw/qdev-core.h            |  1 +
> >  include/qom/cpu.h                 |  9 ++++
> >  include/sysemu/kvm.h              |  1 +
> >  kvm-all.c                         | 57 ++++++++++++++++++++++++-
> >  target-i386/cpu.c                 | 46 ++++++++++++++++++++
> >  19 files changed, 378 insertions(+), 27 deletions(-)
> >
> > --
> > 1.9.3
> >
> >
> 
> 
> 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhi Yong Wu]

On Fri, Feb 6, 2015 at 3:54 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> On Tue, 2015-02-03 at 16:41 +0800, Zhi Yong Wu wrote:
>> HI,
>>
>> Can you push the patchset to a branch on github? It will be convenient
>> for other guys to do some tests.
>
> sorry for late reply.
> I had pushed into https://github.com/zhugh/qemu.git
> The patchset is on branch cpu-hotplug.
>
> Welcome to test, Thanks.
HI,

Can you let me know why device_add can't support the vCPU with 'host' model?

>
> Regards,
> Zhu
>
>>
>> On Wed, Jan 14, 2015 at 3:44 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
>> > This series is based on chen fan's previous i386 cpu hot remove patchset:
>> > https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html
>> >
>> > Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
>> > OS remove one vCPU online, the fireware will store removed bitmap to
>> > QEMU, then QEMU could know to notify the assigned vCPU of exiting.
>> > Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
>> > QEMU itself.
>> >
>> > The whole work is based on the new hot plug/unplug framework, ,the unplug request
>> > callback does the pre-check and send the request, unplug callback does the
>> > removal handling.
>> >
>> > This series depends on tangchen's common hot plug/unplug enhance patchset.
>> > [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
>> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html
>> >
>> > The is the second half of the previous series:
>> > [RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
>> > https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html
>> >
>> > If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
>> > patchset first:
>> > [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
>> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
>> >
>> > ---
>> > Changelog since v1:
>> >  -rebase on the latest version.
>> >  -delete patch i386/cpu: add instance finalize callback, and put it into patchset
>> >   [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.
>> >
>> > Changelog since RFC:
>> >  -splited the i386 cpu hot remove into single thread.
>> >  -replaced apic_no with apic_id, so does the related stuff to make it
>> >   work with arbitrary CPU hotadd.
>> >  -add the icc_device_unrealize callback to handle apic unrealize.
>> >  -rework on the new hot plug/unplug platform.
>> > ---
>> >
>> > Chen Fan (2):
>> >   x86: add x86_cpu_unrealizefn() for cpu apic remove
>> >   cpu hotplug: implement function cpu_status_write() for vcpu ejection
>> >
>> > Gu Zheng (5):
>> >   acpi/cpu: add cpu hot unplug request callback function
>> >   acpi/piix4: add cpu hot unplug callback support
>> >   acpi/ich9: add cpu hot unplug support
>> >   pc: add cpu hot unplug callback support
>> >   cpus: reclaim allocated vCPU objects
>> >
>> > Zhu Guihua (4):
>> >   acpi/piix4: add cpu hot unplug request callback support
>> >   acpi/ich9: add cpu hot unplug request callback support
>> >   pc: add cpu hot unplug request callback support
>> >   acpi/cpu: add cpu hot unplug callback function
>> >
>> >  cpus.c                            | 44 ++++++++++++++++++++
>> >  hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
>> >  hw/acpi/ich9.c                    | 17 ++++++--
>> >  hw/acpi/piix4.c                   | 12 +++++-
>> >  hw/core/qdev.c                    |  2 +-
>> >  hw/cpu/icc_bus.c                  | 11 +++++
>> >  hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
>> >  hw/i386/kvm/apic.c                |  8 ++++
>> >  hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
>> >  hw/intc/apic.c                    | 10 +++++
>> >  hw/intc/apic_common.c             | 21 ++++++----
>> >  include/hw/acpi/cpu_hotplug.h     |  8 ++++
>> >  include/hw/cpu/icc_bus.h          |  1 +
>> >  include/hw/i386/apic_internal.h   |  1 +
>> >  include/hw/qdev-core.h            |  1 +
>> >  include/qom/cpu.h                 |  9 ++++
>> >  include/sysemu/kvm.h              |  1 +
>> >  kvm-all.c                         | 57 ++++++++++++++++++++++++-
>> >  target-i386/cpu.c                 | 46 ++++++++++++++++++++
>> >  19 files changed, 378 insertions(+), 27 deletions(-)
>> >
>> > --
>> > 1.9.3
>> >
>> >
>>
>>
>>
>
>



-- 
Regards,

Zhi Yong Wu

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Tue, 2015-02-10 at 20:38 +0800, Zhi Yong Wu wrote:
> On Fri, Feb 6, 2015 at 3:54 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> > On Tue, 2015-02-03 at 16:41 +0800, Zhi Yong Wu wrote:
> >> HI,
> >>
> >> Can you push the patchset to a branch on github? It will be convenient
> >> for other guys to do some tests.
> >
> > sorry for late reply.
> > I had pushed into https://github.com/zhugh/qemu.git
> > The patchset is on branch cpu-hotplug.
> >
> > Welcome to test, Thanks.
> HI,
> 
> Can you let me know why device_add can't support the vCPU with 'host' model?
> 
> >

I have not noticed this before, thanks for your mentioning.

host_x86_cpu does not support hotplug by default, we will fix this in
next version.

Regards,
Zhu

> > Regards,
> > Zhu
> >
> >>
> >> On Wed, Jan 14, 2015 at 3:44 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> >> > This series is based on chen fan's previous i386 cpu hot remove patchset:
> >> > https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html
> >> >
> >> > Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
> >> > OS remove one vCPU online, the fireware will store removed bitmap to
> >> > QEMU, then QEMU could know to notify the assigned vCPU of exiting.
> >> > Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
> >> > QEMU itself.
> >> >
> >> > The whole work is based on the new hot plug/unplug framework, ,the unplug request
> >> > callback does the pre-check and send the request, unplug callback does the
> >> > removal handling.
> >> >
> >> > This series depends on tangchen's common hot plug/unplug enhance patchset.
> >> > [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
> >> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html
> >> >
> >> > The is the second half of the previous series:
> >> > [RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
> >> > https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html
> >> >
> >> > If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
> >> > patchset first:
> >> > [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
> >> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
> >> >
> >> > ---
> >> > Changelog since v1:
> >> >  -rebase on the latest version.
> >> >  -delete patch i386/cpu: add instance finalize callback, and put it into patchset
> >> >   [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.
> >> >
> >> > Changelog since RFC:
> >> >  -splited the i386 cpu hot remove into single thread.
> >> >  -replaced apic_no with apic_id, so does the related stuff to make it
> >> >   work with arbitrary CPU hotadd.
> >> >  -add the icc_device_unrealize callback to handle apic unrealize.
> >> >  -rework on the new hot plug/unplug platform.
> >> > ---
> >> >
> >> > Chen Fan (2):
> >> >   x86: add x86_cpu_unrealizefn() for cpu apic remove
> >> >   cpu hotplug: implement function cpu_status_write() for vcpu ejection
> >> >
> >> > Gu Zheng (5):
> >> >   acpi/cpu: add cpu hot unplug request callback function
> >> >   acpi/piix4: add cpu hot unplug callback support
> >> >   acpi/ich9: add cpu hot unplug support
> >> >   pc: add cpu hot unplug callback support
> >> >   cpus: reclaim allocated vCPU objects
> >> >
> >> > Zhu Guihua (4):
> >> >   acpi/piix4: add cpu hot unplug request callback support
> >> >   acpi/ich9: add cpu hot unplug request callback support
> >> >   pc: add cpu hot unplug request callback support
> >> >   acpi/cpu: add cpu hot unplug callback function
> >> >
> >> >  cpus.c                            | 44 ++++++++++++++++++++
> >> >  hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
> >> >  hw/acpi/ich9.c                    | 17 ++++++--
> >> >  hw/acpi/piix4.c                   | 12 +++++-
> >> >  hw/core/qdev.c                    |  2 +-
> >> >  hw/cpu/icc_bus.c                  | 11 +++++
> >> >  hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
> >> >  hw/i386/kvm/apic.c                |  8 ++++
> >> >  hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
> >> >  hw/intc/apic.c                    | 10 +++++
> >> >  hw/intc/apic_common.c             | 21 ++++++----
> >> >  include/hw/acpi/cpu_hotplug.h     |  8 ++++
> >> >  include/hw/cpu/icc_bus.h          |  1 +
> >> >  include/hw/i386/apic_internal.h   |  1 +
> >> >  include/hw/qdev-core.h            |  1 +
> >> >  include/qom/cpu.h                 |  9 ++++
> >> >  include/sysemu/kvm.h              |  1 +
> >> >  kvm-all.c                         | 57 ++++++++++++++++++++++++-
> >> >  target-i386/cpu.c                 | 46 ++++++++++++++++++++
> >> >  19 files changed, 378 insertions(+), 27 deletions(-)
> >> >
> >> > --
> >> > 1.9.3
> >> >
> >> >
> >>
> >>
> >>
> >
> >
> 
> 
> 

Re: [Qemu-devel] [PATCH v2 00/11] cpu: add i386 cpu hot remove support

[Zhu Guihua]

On Thu, 2015-02-12 at 19:49 +0800, Zhu Guihua wrote:
> On Tue, 2015-02-10 at 20:38 +0800, Zhi Yong Wu wrote:
> > On Fri, Feb 6, 2015 at 3:54 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> > > On Tue, 2015-02-03 at 16:41 +0800, Zhi Yong Wu wrote:
> > >> HI,
> > >>
> > >> Can you push the patchset to a branch on github? It will be convenient
> > >> for other guys to do some tests.
> > >
> > > sorry for late reply.
> > > I had pushed into https://github.com/zhugh/qemu.git
> > > The patchset is on branch cpu-hotplug.
> > >
> > > Welcome to test, Thanks.
> > HI,
> > 
> > Can you let me know why device_add can't support the vCPU with 'host' model?
> > 
> > >
> 
> I have not noticed this before, thanks for your mentioning.
> 
> host_x86_cpu does not support hotplug by default, we will fix this in
> next version.

I underestimated the complexity of this issue, and expect to handle this
in v5 for x86_64-cpu device_add. Thanks.

Regards,
Zhu

> Regards,
> Zhu
> 
> > > Regards,
> > > Zhu
> > >
> > >>
> > >> On Wed, Jan 14, 2015 at 3:44 PM, Zhu Guihua <zhugh.fnst@cn.fujitsu.com> wrote:
> > >> > This series is based on chen fan's previous i386 cpu hot remove patchset:
> > >> > https://lists.nongnu.org/archive/html/qemu-devel/2013-12/msg04266.html
> > >> >
> > >> > Via implementing ACPI standard methods _EJ0 in ACPI table, after Guest
> > >> > OS remove one vCPU online, the fireware will store removed bitmap to
> > >> > QEMU, then QEMU could know to notify the assigned vCPU of exiting.
> > >> > Meanwhile, intruduce the QOM command 'device_del' to remove vCPU from
> > >> > QEMU itself.
> > >> >
> > >> > The whole work is based on the new hot plug/unplug framework, ,the unplug request
> > >> > callback does the pre-check and send the request, unplug callback does the
> > >> > removal handling.
> > >> >
> > >> > This series depends on tangchen's common hot plug/unplug enhance patchset.
> > >> > [RESEND PATCH v1 0/5] Common unplug and unplug request cb for memory and CPU hot-unplug
> > >> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg00429.html
> > >> >
> > >> > The is the second half of the previous series:
> > >> > [RFC V2 00/10] cpu: add device_add foo-x86_64-cpu and i386 cpu hot remove support
> > >> > https://lists.nongnu.org/archive/html/qemu-devel/2014-08/msg04779.html
> > >> >
> > >> > If you want to test the series, you need to apply the 'device_add foo-x86_64-cpu'
> > >> > patchset first:
> > >> > [PATCH v3 0/7] cpu: add device_add foo-x86_64-cpu support
> > >> > https://lists.nongnu.org/archive/html/qemu-devel/2015-01/msg01552.html
> > >> >
> > >> > ---
> > >> > Changelog since v1:
> > >> >  -rebase on the latest version.
> > >> >  -delete patch i386/cpu: add instance finalize callback, and put it into patchset
> > >> >   [PATCH v3 0/6] cpu: add device_add foo-x86_64-cpu support.
> > >> >
> > >> > Changelog since RFC:
> > >> >  -splited the i386 cpu hot remove into single thread.
> > >> >  -replaced apic_no with apic_id, so does the related stuff to make it
> > >> >   work with arbitrary CPU hotadd.
> > >> >  -add the icc_device_unrealize callback to handle apic unrealize.
> > >> >  -rework on the new hot plug/unplug platform.
> > >> > ---
> > >> >
> > >> > Chen Fan (2):
> > >> >   x86: add x86_cpu_unrealizefn() for cpu apic remove
> > >> >   cpu hotplug: implement function cpu_status_write() for vcpu ejection
> > >> >
> > >> > Gu Zheng (5):
> > >> >   acpi/cpu: add cpu hot unplug request callback function
> > >> >   acpi/piix4: add cpu hot unplug callback support
> > >> >   acpi/ich9: add cpu hot unplug support
> > >> >   pc: add cpu hot unplug callback support
> > >> >   cpus: reclaim allocated vCPU objects
> > >> >
> > >> > Zhu Guihua (4):
> > >> >   acpi/piix4: add cpu hot unplug request callback support
> > >> >   acpi/ich9: add cpu hot unplug request callback support
> > >> >   pc: add cpu hot unplug request callback support
> > >> >   acpi/cpu: add cpu hot unplug callback function
> > >> >
> > >> >  cpus.c                            | 44 ++++++++++++++++++++
> > >> >  hw/acpi/cpu_hotplug.c             | 88 ++++++++++++++++++++++++++++++++++++---
> > >> >  hw/acpi/ich9.c                    | 17 ++++++--
> > >> >  hw/acpi/piix4.c                   | 12 +++++-
> > >> >  hw/core/qdev.c                    |  2 +-
> > >> >  hw/cpu/icc_bus.c                  | 11 +++++
> > >> >  hw/i386/acpi-dsdt-cpu-hotplug.dsl |  6 ++-
> > >> >  hw/i386/kvm/apic.c                |  8 ++++
> > >> >  hw/i386/pc.c                      | 62 +++++++++++++++++++++++++--
> > >> >  hw/intc/apic.c                    | 10 +++++
> > >> >  hw/intc/apic_common.c             | 21 ++++++----
> > >> >  include/hw/acpi/cpu_hotplug.h     |  8 ++++
> > >> >  include/hw/cpu/icc_bus.h          |  1 +
> > >> >  include/hw/i386/apic_internal.h   |  1 +
> > >> >  include/hw/qdev-core.h            |  1 +
> > >> >  include/qom/cpu.h                 |  9 ++++
> > >> >  include/sysemu/kvm.h              |  1 +
> > >> >  kvm-all.c                         | 57 ++++++++++++++++++++++++-
> > >> >  target-i386/cpu.c                 | 46 ++++++++++++++++++++
> > >> >  19 files changed, 378 insertions(+), 27 deletions(-)
> > >> >
> > >> > --
> > >> > 1.9.3
> > >> >
> > >> >
> > >>
> > >>
> > >>
> > >
> > >
> > 
> > 
> > 
> 
> 
>