[Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Li Zhang]

From: Li Zhang <li.zhang@hxt-semitech.com>

virt machine doesn't support hot_add_cpu interface. This patchset is to implement
hot_add_cpu interface. A CPU can be added by QMP command with QEMU monitor.

Here is command to add a CPU with QMP command.

* qemu-system-aarch64 -machine virt -cpu cortex-a15 -smp 1,maxcpus=4 \
    -monitor telnet:127.0.0.1:4444,server,nowait -nographic
* connect monitor:      telnet 127.0.0.1 4444
* execute qmp command:  cpu-add 1

Currently, when KVM is enabled and add a CPU with QMP command, QEMU 
reports error "kvm_init_vcpu failed: Device or resourc busy". KVM can't
create a new CPU when vgic has been initialized and irqchip_in_kernel in
function kvm_arch_vcpu_create. It needs to change KVM code in the future.

Li Zhang (3):
  arm/virt: Refine fdt_add_cpu_nodes code
  arm/virt: Refine code of machvirt_init
  arm/virt: Implement hot_add_cpu interface

 hw/arm/virt.c | 344 ++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 202 insertions(+), 142 deletions(-)

-- 
2.7.4

[Qemu-devel] [PATCH 1/3] arm/virt: Refine fdt_add_cpu_nodes code

[Li Zhang]

From: Li Zhang <li.zhang@hxt-semitech.com>

Refind fdt_add_cpu_nodes code and add a new function fdt_add_cpu_node,
which can be called by hot_add_cpu function.

Signed-off-by: Li Zhang <li.zhang@hxt-semitech.com>
---
 hw/arm/virt.c | 106 +++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 61 insertions(+), 45 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index c7c8159..73c3cf7 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -334,68 +334,84 @@ static void fdt_add_timer_nodes(const VirtMachineState *vms)
                        GIC_FDT_IRQ_TYPE_PPI, ARCH_TIMER_NS_EL2_IRQ, irqflags);
 }
 
-static void fdt_add_cpu_nodes(const VirtMachineState *vms)
+/*
+ * From Documentation/devicetree/bindings/arm/cpus.txt
+ *  On ARM v8 64-bit systems value should be set to 2,
+ *  that corresponds to the MPIDR_EL1 register size.
+ *  If MPIDR_EL1[63:32] value is equal to 0 on all CPUs
+ *  in the system, #address-cells can be set to 1, since
+ *  MPIDR_EL1[63:32] bits are not used for CPUs
+ *  identification.
+ *
+ *  Here we actually don't know whether our system is 32- or 64-bit one.
+ *  The simplest way to go is to examine affinity IDs of all our CPUs. If
+ *  at least one of them has Aff3 populated, we set #address-cells to 2.
+ */
+
+static int fdt_get_addr_cells(const VirtMachineState *vms)
 {
     int cpu;
     int addr_cells = 1;
-    const MachineState *ms = MACHINE(vms);
 
-    /*
-     * From Documentation/devicetree/bindings/arm/cpus.txt
-     *  On ARM v8 64-bit systems value should be set to 2,
-     *  that corresponds to the MPIDR_EL1 register size.
-     *  If MPIDR_EL1[63:32] value is equal to 0 on all CPUs
-     *  in the system, #address-cells can be set to 1, since
-     *  MPIDR_EL1[63:32] bits are not used for CPUs
-     *  identification.
-     *
-     *  Here we actually don't know whether our system is 32- or 64-bit one.
-     *  The simplest way to go is to examine affinity IDs of all our CPUs. If
-     *  at least one of them has Aff3 populated, we set #address-cells to 2.
-     */
     for (cpu = 0; cpu < vms->smp_cpus; cpu++) {
         ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(cpu));
-
         if (armcpu->mp_affinity & ARM_AFF3_MASK) {
             addr_cells = 2;
             break;
         }
     }
+    return addr_cells;
+}
 
-    qemu_fdt_add_subnode(vms->fdt, "/cpus");
-    qemu_fdt_setprop_cell(vms->fdt, "/cpus", "#address-cells", addr_cells);
-    qemu_fdt_setprop_cell(vms->fdt, "/cpus", "#size-cells", 0x0);
 
-    for (cpu = vms->smp_cpus - 1; cpu >= 0; cpu--) {
-        char *nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
-        ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(cpu));
-        CPUState *cs = CPU(armcpu);
+static void fdt_add_cpu_node(const VirtMachineState *vms, int cpu)
+{
+    char *nodename;
+    int addr_cells = fdt_get_addr_cells(vms);
+    const MachineState *ms = MACHINE(vms);
+    ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(cpu));
+    CPUState *cs = CPU(armcpu);
 
-        qemu_fdt_add_subnode(vms->fdt, nodename);
-        qemu_fdt_setprop_string(vms->fdt, nodename, "device_type", "cpu");
-        qemu_fdt_setprop_string(vms->fdt, nodename, "compatible",
-                                    armcpu->dtb_compatible);
-
-        if (vms->psci_conduit != QEMU_PSCI_CONDUIT_DISABLED
-            && vms->smp_cpus > 1) {
-            qemu_fdt_setprop_string(vms->fdt, nodename,
-                                        "enable-method", "psci");
-        }
+    if (cpu < 0 || cpu >= max_cpus) {
+        error_report("Invalid cpu index.");
+        return;
+    }
 
-        if (addr_cells == 2) {
-            qemu_fdt_setprop_u64(vms->fdt, nodename, "reg",
-                                 armcpu->mp_affinity);
-        } else {
-            qemu_fdt_setprop_cell(vms->fdt, nodename, "reg",
-                                  armcpu->mp_affinity);
-        }
+    nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
 
-        if (ms->possible_cpus->cpus[cs->cpu_index].props.has_node_id) {
-            qemu_fdt_setprop_cell(vms->fdt, nodename, "numa-node-id",
-                ms->possible_cpus->cpus[cs->cpu_index].props.node_id);
-        }
+    qemu_fdt_add_subnode(vms->fdt, nodename);
+    qemu_fdt_setprop_string(vms->fdt, nodename, "device_type", "cpu");
+    qemu_fdt_setprop_string(vms->fdt, nodename, "compatible",
+                            armcpu->dtb_compatible);
+    if (vms->psci_conduit != QEMU_PSCI_CONDUIT_DISABLED) {
+        qemu_fdt_setprop_string(vms->fdt, nodename, "enable-method", "psci");
+    }
 
-        g_free(nodename);
+    if (addr_cells == 2) {
+        qemu_fdt_setprop_u64(vms->fdt, nodename, "reg", armcpu->mp_affinity);
+    } else {
+        qemu_fdt_setprop_cell(vms->fdt, nodename, "reg", armcpu->mp_affinity);
+    }
+
+    if (ms->possible_cpus->cpus[cs->cpu_index].props.has_node_id) {
+        qemu_fdt_setprop_cell(vms->fdt, nodename, "numa-node-id",
+            ms->possible_cpus->cpus[cs->cpu_index].props.node_id);
+    }
+
+    g_free(nodename);
+}
+
+static void fdt_add_cpu_nodes(const VirtMachineState *vms)
+{
+    int cpu;
+    int addr_cells = fdt_get_addr_cells(vms);
+
+    qemu_fdt_add_subnode(vms->fdt, "/cpus");
+    qemu_fdt_setprop_cell(vms->fdt, "/cpus", "#address-cells", addr_cells);
+    qemu_fdt_setprop_cell(vms->fdt, "/cpus", "#size-cells", 0x0);
+
+    for (cpu = vms->smp_cpus - 1; cpu >= 0; cpu--) {
+        fdt_add_cpu_node(vms, cpu);
     }
 }
 
-- 
2.7.4

[Qemu-devel] [PATCH 2/3] arm/virt: Refine code of machvirt_init

[Li Zhang]

From: Li Zhang <li.zhang@hxt-semitech.com>

This patch is to refine cpu related in machvirt_init and
add virt_new_cpu function which can be called by hot_add_cpu.

Signed-off-by: Li Zhang <li.zhang@hxt-semitech.com>
---
 hw/arm/virt.c | 231 ++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 128 insertions(+), 103 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 73c3cf7..31314c1 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -110,6 +110,8 @@ static ARMPlatformBusSystemParams platform_bus_params;
 #define RAMLIMIT_GB 255
 #define RAMLIMIT_BYTES (RAMLIMIT_GB * 1024ULL * 1024 * 1024)
 
+static MemoryRegion *secure_sysmem;
+
 /* Addresses and sizes of our components.
  * 0..128MB is space for a flash device so we can run bootrom code such as UEFI.
  * 128MB..256MB is used for miscellaneous device I/O.
@@ -415,6 +417,116 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
     }
 }
 
+static void virt_new_cpu(MachineState *ms, int id, Error **errp)
+{
+    const char *typename;
+    char **cpustr;
+    int node_id;
+
+    ObjectClass *oc;
+    CPUClass *cc;
+    CPUState *cs;
+    Object *cpuobj;
+    const CPUArchIdList *possible_cpus;
+
+    Error *err = NULL;
+    const char *cpu_model = ms->cpu_model;
+    MemoryRegion *sysmem = get_system_memory();
+
+    VirtMachineState *vms = VIRT_MACHINE(ms);
+    VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(ms);
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    CPU_FOREACH(cs) {
+        if (cs->cpu_index == id) {
+            error_report("CPU %d has been created.", id);
+            return;
+        }
+    }
+    possible_cpus = mc->possible_cpu_arch_ids(ms);
+    /* Separate the actual CPU model name from any appended features */
+    cpustr = g_strsplit(cpu_model, ",", 2);
+    if (!cpuname_valid(cpustr[0])) {
+        error_report("mach-virt: CPU %s not supported", cpustr[0]);
+        goto out;
+    }
+    oc = cpu_class_by_name(TYPE_ARM_CPU, cpustr[0]);
+    if (!oc) {
+        error_report("Unable to find CPU definition");
+        goto out;
+    }
+
+    typename = object_class_get_name(oc);
+
+    cc = CPU_CLASS(oc);
+    cc->parse_features(typename, cpustr[1], &err);
+    if (err) {
+        error_report_err(err);
+        goto out;
+    }
+
+    cpuobj = object_new(typename);
+    object_property_set_int(cpuobj, possible_cpus->cpus[id].arch_id,
+                            "mp-affinity", NULL);
+    cs = CPU(cpuobj);
+    cs->cpu_index = id;
+
+    node_id = possible_cpus->cpus[cs->cpu_index].props.node_id;
+    if (!possible_cpus->cpus[cs->cpu_index].props.has_node_id) {
+        /* by default CPUState::numa_node was 0 if it's not set via CLI
+         * keep it this way for now but in future we probably should
+         * refuse to start up with incomplete numa mapping */
+        node_id = 0;
+    }
+    if (cs->numa_node == CPU_UNSET_NUMA_NODE_ID) {
+        cs->numa_node = node_id;
+    } else {
+        /* CPU isn't device_add compatible yet, this shouldn't happen */
+        error_setg(&error_abort, "user set node-id not implemented");
+    }
+
+    if (!vms->secure) {
+        object_property_set_bool(cpuobj, false, "has_el3", NULL);
+    }
+
+    if (!vms->virt && object_property_find(cpuobj, "has_el2", NULL)) {
+        object_property_set_bool(cpuobj, false, "has_el2", NULL);
+    }
+
+    if (vms->psci_conduit != QEMU_PSCI_CONDUIT_DISABLED) {
+        object_property_set_int(cpuobj, vms->psci_conduit,
+                                "psci-conduit", NULL);
+        /* Secondary CPUs start in PSCI powered-down state */
+        if (id > 0) {
+            object_property_set_bool(cpuobj, true, "start-powered-off", NULL);
+        }
+    }
+
+    if (vmc->no_pmu && object_property_find(cpuobj, "pmu", NULL)) {
+        object_property_set_bool(cpuobj, false, "pmu", NULL);
+    }
+
+    if (object_property_find(cpuobj, "reset-cbar", NULL)) {
+        object_property_set_int(cpuobj, vms->memmap[VIRT_CPUPERIPHS].base,
+                                "reset-cbar", &error_abort);
+    }
+
+    object_property_set_link(cpuobj, OBJECT(sysmem), "memory", &error_abort);
+
+    if (vms->secure) {
+        object_property_set_link(cpuobj, OBJECT(secure_sysmem),
+                                  "secure-memory", &error_abort);
+    }
+
+    object_property_set_int(cpuobj, id, "id", NULL);
+    object_property_set_bool(cpuobj, true, "realized", NULL);
+    object_unref(cpuobj);
+
+out:
+    g_strfreev(cpustr);
+    error_propagate(errp, err);
+}
+
 static void fdt_add_its_gic_node(VirtMachineState *vms)
 {
     vms->msi_phandle = qemu_fdt_alloc_phandle(vms->fdt);
@@ -1237,35 +1349,29 @@ static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx)
 static void machvirt_init(MachineState *machine)
 {
     VirtMachineState *vms = VIRT_MACHINE(machine);
-    VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(machine);
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     const CPUArchIdList *possible_cpus;
     qemu_irq pic[NUM_IRQS];
     MemoryRegion *sysmem = get_system_memory();
-    MemoryRegion *secure_sysmem = NULL;
     int n, virt_max_cpus;
     MemoryRegion *ram = g_new(MemoryRegion, 1);
     const char *cpu_model = machine->cpu_model;
-    char **cpustr;
-    ObjectClass *oc;
-    const char *typename;
-    CPUClass *cc;
     Error *err = NULL;
     bool firmware_loaded = bios_name || drive_get(IF_PFLASH, 0, 0);
 
     if (!cpu_model) {
-        cpu_model = "cortex-a15";
+        cpu_model = kvm_enabled() ? "host" : "cortex-a15";
+        machine->cpu_model = cpu_model;
     }
 
     /* We can probe only here because during property set
      * KVM is not available yet
      */
+    if (!vms->gic_version && !kvm_enabled()) {
+        error_report("gic-version=host requires KVM");
+        exit(1);
+    }
     if (!vms->gic_version) {
-        if (!kvm_enabled()) {
-            error_report("gic-version=host requires KVM");
-            exit(1);
-        }
-
         vms->gic_version = kvm_arm_vgic_probe();
         if (!vms->gic_version) {
             error_report("Unable to determine GIC version supported by host");
@@ -1273,14 +1379,6 @@ static void machvirt_init(MachineState *machine)
         }
     }
 
-    /* Separate the actual CPU model name from any appended features */
-    cpustr = g_strsplit(cpu_model, ",", 2);
-
-    if (!cpuname_valid(cpustr[0])) {
-        error_report("mach-virt: CPU %s not supported", cpustr[0]);
-        exit(1);
-    }
-
     /* If we have an EL3 boot ROM then the assumption is that it will
      * implement PSCI itself, so disable QEMU's internal implementation
      * so it doesn't get in the way. Instead of starting secondary
@@ -1328,12 +1426,12 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->secure) {
-        if (kvm_enabled()) {
-            error_report("mach-virt: KVM does not support Security extensions");
-            exit(1);
-        }
+    if (vms->secure && kvm_enabled()) {
+        error_report("mach-virt: KVM does not support Security extensions");
+        exit(1);
+    }
 
+    if (vms->secure) {
         /* The Secure view of the world is the same as the NonSecure,
          * but with a few extra devices. Create it as a container region
          * containing the system memory at low priority; any secure-only
@@ -1347,91 +1445,18 @@ static void machvirt_init(MachineState *machine)
 
     create_fdt(vms);
 
-    oc = cpu_class_by_name(TYPE_ARM_CPU, cpustr[0]);
-    if (!oc) {
-        error_report("Unable to find CPU definition");
-        exit(1);
-    }
-    typename = object_class_get_name(oc);
-
-    /* convert -smp CPU options specified by the user into global props */
-    cc = CPU_CLASS(oc);
-    cc->parse_features(typename, cpustr[1], &err);
-    g_strfreev(cpustr);
-    if (err) {
-        error_report_err(err);
-        exit(1);
-    }
-
     possible_cpus = mc->possible_cpu_arch_ids(machine);
     for (n = 0; n < possible_cpus->len; n++) {
-        Object *cpuobj;
-        CPUState *cs;
-        int node_id;
-
         if (n >= smp_cpus) {
             break;
         }
-
-        cpuobj = object_new(typename);
-        object_property_set_int(cpuobj, possible_cpus->cpus[n].arch_id,
-                                "mp-affinity", NULL);
-
-        cs = CPU(cpuobj);
-        cs->cpu_index = n;
-
-        node_id = possible_cpus->cpus[cs->cpu_index].props.node_id;
-        if (!possible_cpus->cpus[cs->cpu_index].props.has_node_id) {
-            /* by default CPUState::numa_node was 0 if it's not set via CLI
-             * keep it this way for now but in future we probably should
-             * refuse to start up with incomplete numa mapping */
-             node_id = 0;
-        }
-        if (cs->numa_node == CPU_UNSET_NUMA_NODE_ID) {
-            cs->numa_node = node_id;
-        } else {
-            /* CPU isn't device_add compatible yet, this shouldn't happen */
-            error_setg(&error_abort, "user set node-id not implemented");
-        }
-
-        if (!vms->secure) {
-            object_property_set_bool(cpuobj, false, "has_el3", NULL);
-        }
-
-        if (!vms->virt && object_property_find(cpuobj, "has_el2", NULL)) {
-            object_property_set_bool(cpuobj, false, "has_el2", NULL);
-        }
-
-        if (vms->psci_conduit != QEMU_PSCI_CONDUIT_DISABLED) {
-            object_property_set_int(cpuobj, vms->psci_conduit,
-                                    "psci-conduit", NULL);
-
-            /* Secondary CPUs start in PSCI powered-down state */
-            if (n > 0) {
-                object_property_set_bool(cpuobj, true,
-                                         "start-powered-off", NULL);
-            }
-        }
-
-        if (vmc->no_pmu && object_property_find(cpuobj, "pmu", NULL)) {
-            object_property_set_bool(cpuobj, false, "pmu", NULL);
-        }
-
-        if (object_property_find(cpuobj, "reset-cbar", NULL)) {
-            object_property_set_int(cpuobj, vms->memmap[VIRT_CPUPERIPHS].base,
-                                    "reset-cbar", &error_abort);
-        }
-
-        object_property_set_link(cpuobj, OBJECT(sysmem), "memory",
-                                 &error_abort);
-        if (vms->secure) {
-            object_property_set_link(cpuobj, OBJECT(secure_sysmem),
-                                     "secure-memory", &error_abort);
+        virt_new_cpu(machine, n, &err);
+        if (err) {
+            error_report("mach-virt: creating a new cpu failed.");
+            exit(1);
         }
-
-        object_property_set_bool(cpuobj, true, "realized", NULL);
-        object_unref(cpuobj);
     }
+
     fdt_add_timer_nodes(vms);
     fdt_add_cpu_nodes(vms);
     fdt_add_psci_node(vms);
-- 
2.7.4

[Qemu-devel] [PATCH 3/3] arm/virt: Implement hot_add_cpu interface

[Li Zhang]

From: Li Zhang <li.zhang@hxt-semitech.com>

This patch is to add add_hot_cpu interface.

Signed-off-by: Li Zhang <li.zhang@hxt-semitech.com>
---
 hw/arm/virt.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 31314c1..c7e8322 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -527,6 +527,24 @@ out:
     error_propagate(errp, err);
 }
 
+static void virt_hot_add_cpu(const int64_t id, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());
+    VirtMachineState *vms = VIRT_MACHINE(ms);
+
+    if (id < 0 || id >= max_cpus) {
+        error_setg(errp, "Invalid CPU id: %" PRIi64, id);
+        return;
+    }
+
+    virt_new_cpu(ms, id, errp);
+    if (errp) {
+        return;
+    }
+
+    fdt_add_cpu_node(vms, id);
+}
+
 static void fdt_add_its_gic_node(VirtMachineState *vms)
 {
     vms->msi_phandle = qemu_fdt_alloc_phandle(vms->fdt);
@@ -1654,6 +1672,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     mc->minimum_page_bits = 12;
     mc->possible_cpu_arch_ids = virt_possible_cpu_arch_ids;
     mc->cpu_index_to_instance_props = virt_cpu_index_to_props;
+    mc->hot_add_cpu = virt_hot_add_cpu;
 }
 
 static const TypeInfo virt_machine_info = {
-- 
2.7.4

Re: [Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Igor Mammedov]

On Fri, 26 May 2017 17:21:05 +0800
Li Zhang <zhlcindy@gmail.com> wrote:

> From: Li Zhang <li.zhang@hxt-semitech.com>
> 
> virt machine doesn't support hot_add_cpu interface. This patchset is to implement
> hot_add_cpu interface. A CPU can be added by QMP command with QEMU monitor.
> 
> Here is command to add a CPU with QMP command.
> 
> * qemu-system-aarch64 -machine virt -cpu cortex-a15 -smp 1,maxcpus=4 \
>     -monitor telnet:127.0.0.1:4444,server,nowait -nographic
> * connect monitor:      telnet 127.0.0.1 4444
> * execute qmp command:  cpu-add 1
cpu-add command shouldn't be used for new cpu hotplug,
pls use generic device_add for that.

The last time I looked at it, virt machine needed quite a bit of
re-factoring of the way it creates/wires up CPUs.

Pls see usage of following callbacks for example on how to implement
device_add based cpu hotplug:

  get_hotplug_handler
  pc_cpu_pre_plug
  pc_cpu_plug
  pc_possible_cpu_arch_ids

and pc_cpus_init() for initial cpu creation with above callbacks in use

> 
> Currently, when KVM is enabled and add a CPU with QMP command, QEMU 
> reports error "kvm_init_vcpu failed: Device or resourc busy". KVM can't
> create a new CPU when vgic has been initialized and irqchip_in_kernel in
> function kvm_arch_vcpu_create. It needs to change KVM code in the future.
if KVM isn't capable do it yet, then qemu should refuse cpu hotplug if running
with KVM accelarator and allow it only in TCG mode.

> 
> Li Zhang (3):
>   arm/virt: Refine fdt_add_cpu_nodes code
>   arm/virt: Refine code of machvirt_init
>   arm/virt: Implement hot_add_cpu interface
> 
>  hw/arm/virt.c | 344 ++++++++++++++++++++++++++++++++++------------------------
>  1 file changed, 202 insertions(+), 142 deletions(-)
> 

Re: [Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Li Zhang]

On Fri, May 26, 2017 at 7:21 PM, Igor Mammedov <imammedo@redhat.com> wrote:
> On Fri, 26 May 2017 17:21:05 +0800
> Li Zhang <zhlcindy@gmail.com> wrote:
>
>> From: Li Zhang <li.zhang@hxt-semitech.com>
>>
>> virt machine doesn't support hot_add_cpu interface. This patchset is to implement
>> hot_add_cpu interface. A CPU can be added by QMP command with QEMU monitor.
>>
>> Here is command to add a CPU with QMP command.
>>
>> * qemu-system-aarch64 -machine virt -cpu cortex-a15 -smp 1,maxcpus=4 \
>>     -monitor telnet:127.0.0.1:4444,server,nowait -nographic
>> * connect monitor:      telnet 127.0.0.1 4444
>> * execute qmp command:  cpu-add 1
> cpu-add command shouldn't be used for new cpu hotplug,
> pls use generic device_add for that.

Ok,  I will drop this command and look at device_add.

>
> The last time I looked at it, virt machine needed quite a bit of
> re-factoring of the way it creates/wires up CPUs.

 Yes, I am looking into it and see if I can do something:)

>
> Pls see usage of following callbacks for example on how to implement
> device_add based cpu hotplug:
>
>   get_hotplug_handler
>   pc_cpu_pre_plug
>   pc_cpu_plug
>   pc_possible_cpu_arch_ids
>
> and pc_cpus_init() for initial cpu creation with above callbacks in use
>
Thanks, I will see it. :)

>>
>> Currently, when KVM is enabled and add a CPU with QMP command, QEMU
>> reports error "kvm_init_vcpu failed: Device or resourc busy". KVM can't
>> create a new CPU when vgic has been initialized and irqchip_in_kernel in
>> function kvm_arch_vcpu_create. It needs to change KVM code in the future.
> if KVM isn't capable do it yet, then qemu should refuse cpu hotplug if running
> with KVM accelarator and allow it only in TCG mode.

OK, I will add an assertion to allow hotplug only in TCG mode.

>
>>
>> Li Zhang (3):
>>   arm/virt: Refine fdt_add_cpu_nodes code
>>   arm/virt: Refine code of machvirt_init
>>   arm/virt: Implement hot_add_cpu interface
>>
>>  hw/arm/virt.c | 344 ++++++++++++++++++++++++++++++++++------------------------
>>  1 file changed, 202 insertions(+), 142 deletions(-)
>>
>



-- 

Best Regards
-Li

Re: [Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Igor Mammedov]

On Sat, 27 May 2017 10:32:33 +0800
Li Zhang <zhlcindy@gmail.com> wrote:

> On Fri, May 26, 2017 at 7:21 PM, Igor Mammedov <imammedo@redhat.com> wrote:
> > On Fri, 26 May 2017 17:21:05 +0800
> > Li Zhang <zhlcindy@gmail.com> wrote:
> >  
> >> From: Li Zhang <li.zhang@hxt-semitech.com>
> >>
> >> virt machine doesn't support hot_add_cpu interface. This patchset is to implement
> >> hot_add_cpu interface. A CPU can be added by QMP command with QEMU monitor.
> >>
> >> Here is command to add a CPU with QMP command.
> >>
> >> * qemu-system-aarch64 -machine virt -cpu cortex-a15 -smp 1,maxcpus=4 \
> >>     -monitor telnet:127.0.0.1:4444,server,nowait -nographic
> >> * connect monitor:      telnet 127.0.0.1 4444
> >> * execute qmp command:  cpu-add 1  
> > cpu-add command shouldn't be used for new cpu hotplug,
> > pls use generic device_add for that.  
> 
> Ok,  I will drop this command and look at device_add.
> 
> >
> > The last time I looked at it, virt machine needed quite a bit of
> > re-factoring of the way it creates/wires up CPUs.  
> 
>  Yes, I am looking into it and see if I can do something:)
> 
> >
> > Pls see usage of following callbacks for example on how to implement
> > device_add based cpu hotplug:
> >
> >   get_hotplug_handler
> >   pc_cpu_pre_plug
> >   pc_cpu_plug
> >   pc_possible_cpu_arch_ids
> >
> > and pc_cpus_init() for initial cpu creation with above callbacks in use
> >  
> Thanks, I will see it. :)
> 
> >>
> >> Currently, when KVM is enabled and add a CPU with QMP command, QEMU
> >> reports error "kvm_init_vcpu failed: Device or resourc busy". KVM can't
> >> create a new CPU when vgic has been initialized and irqchip_in_kernel in
> >> function kvm_arch_vcpu_create. It needs to change KVM code in the future.  
> > if KVM isn't capable do it yet, then qemu should refuse cpu hotplug if running
> > with KVM accelarator and allow it only in TCG mode.  
> 
> OK, I will add an assertion to allow hotplug only in TCG mode.
instead of assert which will crash guest on hotplug,
just disable hotplug gracefully (for example set error in pre_plug handler)

> 
> >  
> >>
> >> Li Zhang (3):
> >>   arm/virt: Refine fdt_add_cpu_nodes code
> >>   arm/virt: Refine code of machvirt_init
> >>   arm/virt: Implement hot_add_cpu interface
> >>
> >>  hw/arm/virt.c | 344 ++++++++++++++++++++++++++++++++++------------------------
> >>  1 file changed, 202 insertions(+), 142 deletions(-)
> >>  
> >  
> 
> 
> 

Re: [Qemu-devel] [PATCH 0/3] arm/virt: refine virt.c code and implement hot_add_cpu interface

[Li Zhang]

> 在 2017年5月29日，下午6:43，Igor Mammedov <imammedo@redhat.com> 写道：
> 
> On Sat, 27 May 2017 10:32:33 +0800
> Li Zhang <zhlcindy@gmail.com> wrote:
> 
>>> On Fri, May 26, 2017 at 7:21 PM, Igor Mammedov <imammedo@redhat.com> wrote:
>>> On Fri, 26 May 2017 17:21:05 +0800
>>> Li Zhang <zhlcindy@gmail.com> wrote:
>>> 
>>>> From: Li Zhang <li.zhang@hxt-semitech.com>
>>>> 
>>>> virt machine doesn't support hot_add_cpu interface. This patchset is to implement
>>>> hot_add_cpu interface. A CPU can be added by QMP command with QEMU monitor.
>>>> 
>>>> Here is command to add a CPU with QMP command.
>>>> 
>>>> * qemu-system-aarch64 -machine virt -cpu cortex-a15 -smp 1,maxcpus=4 \
>>>>    -monitor telnet:127.0.0.1:4444,server,nowait -nographic
>>>> * connect monitor:      telnet 127.0.0.1 4444
>>>> * execute qmp command:  cpu-add 1  
>>> cpu-add command shouldn't be used for new cpu hotplug,
>>> pls use generic device_add for that.  
>> 
>> Ok,  I will drop this command and look at device_add.
>> 
>>> 
>>> The last time I looked at it, virt machine needed quite a bit of
>>> re-factoring of the way it creates/wires up CPUs.  
>> 
>> Yes, I am looking into it and see if I can do something:)
>> 
>>> 
>>> Pls see usage of following callbacks for example on how to implement
>>> device_add based cpu hotplug:
>>> 
>>>  get_hotplug_handler
>>>  pc_cpu_pre_plug
>>>  pc_cpu_plug
>>>  pc_possible_cpu_arch_ids
>>> 
>>> and pc_cpus_init() for initial cpu creation with above callbacks in use
>>> 
>> Thanks, I will see it. :)
>> 
>>>> 
>>>> Currently, when KVM is enabled and add a CPU with QMP command, QEMU
>>>> reports error "kvm_init_vcpu failed: Device or resourc busy". KVM can't
>>>> create a new CPU when vgic has been initialized and irqchip_in_kernel in
>>>> function kvm_arch_vcpu_create. It needs to change KVM code in the future.  
>>> if KVM isn't capable do it yet, then qemu should refuse cpu hotplug if running
>>> with KVM accelarator and allow it only in TCG mode.  
>> 
>> OK, I will add an assertion to allow hotplug only in TCG mode.
> instead of assert which will crash guest on hotplug,
> just disable hotplug gracefully (for example set error in pre_plug handler)

OK, got it.

> 
>> 
>>> 
>>>> 
>>>> Li Zhang (3):
>>>>  arm/virt: Refine fdt_add_cpu_nodes code
>>>>  arm/virt: Refine code of machvirt_init
>>>>  arm/virt: Implement hot_add_cpu interface
>>>> 
>>>> hw/arm/virt.c | 344 ++++++++++++++++++++++++++++++++++------------------------
>>>> 1 file changed, 202 insertions(+), 142 deletions(-)
>>>> 
>>> 
>> 
>> 
>> 
>