[Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Igor Mammedov]

Changes since v3:
  - simplify series by dropping idea of showing property values in "qom-list-properties"
    and use MachineInfo in QAPI schema instead

Changes since v2:
  - taking in account previous review, implement a way for mgmt to intospect if
    '-numa node,mem' is supported by machine type as suggested by Daniel at
             https://www.mail-archive.com/qemu-devel@nongnu.org/msg601220.html
      * ammend "qom-list-properties" to show property values
      * add "numa-mem-supported" machine property to reflect if '-numa node,mem=SZ'
        is supported. It culd be used with '-machine none' or at runtime with
        --preconfig before numa memory mapping are configured
  * minor fixes to deprecation documentation mentioning "numa-mem-supported" property

 1) "I'm considering to deprecating -mem-path/prealloc CLI options and replacing
them with a single memdev Machine property to allow interested users to pick
used backend for initial RAM (fixes mixed -mem-path+hostmem backends issues)
and as a transition step to modeling initial RAM as a Device instead of
(ab)using MemoryRegion APIs."
(for more details see: https://www.mail-archive.com/qemu-devel@nongnu.org/msg596314.html)

However there is a couple of roadblocks on the way (s390x and numa memory handling).
I think I finally thought out a way to hack s390x in migration compatible manner,
but I don't see any way to do it for -numa node,mem and default RAM assignement
to nodes. Considering both numa usecases aren't meaningfully using NUMA (aside
guest side testing) and could be replaced with explicitly used memdev parameter,
I'd like to propose removing these fake NUMA friends on new machine types,
hence this deprecation. And once the last machie type that supported the option
is removed we would be able to remove option altogether.

As result of removing deprecated options and replacing initial RAM allocation
with 'memdev's (1), QEMU will allocate guest RAM in consistent way, fixing mixed
use-case and allowing boards to move towards modelling initial RAM as Device(s).
Which in its own turn should allow to cleanup NUMA/HMP/memory accounting code
more by dropping ad-hoc node_mem tracking and reusing memory device enumeration
instead.

Reference to previous versions:
 * https://www.mail-archive.com/qemu-devel@nongnu.org/msg617694.html

CC: libvir-list@redhat.com
CC: ehabkost@redhat.com
CC: pbonzini@redhat.com
CC: berrange@redhat.com
CC: armbru@redhat.com

Igor Mammedov (3):
  machine: show if CLI option '-numa node,mem' is supported in QAPI
    schema
  numa: deprecate 'mem' parameter of '-numa node' option
  numa: deprecate implict memory distribution between nodes

 include/hw/boards.h  |  3 +++
 hw/arm/virt.c        |  1 +
 hw/i386/pc.c         |  1 +
 hw/ppc/spapr.c       |  1 +
 numa.c               |  5 +++++
 qapi/misc.json       |  5 ++++-
 qemu-deprecated.texi | 24 ++++++++++++++++++++++++
 vl.c                 |  1 +
 8 files changed, 40 insertions(+), 1 deletion(-)

-- 
2.7.4

[Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Igor Mammedov]

Legacy '-numa node,mem' option has a number of issues and mgmt often
defaults to it. Unfortunately it's no possible to replace it with
an alternative '-numa memdev' without breaking migration compatibility.
What's possible though is to deprecate it, keeping option working with
old machine types only.

In order to help users to find out if being deprecated CLI option
'-numa node,mem' is still supported by particular machine type, add new
"numa-mem-supported" property to MachineInfo description in QAPI schema.

"numa-mem-supported" is set to 'true' for machines that currently support
NUMA, but it will be flipped to 'false' later on, once deprecation period
expires and kept 'true' only for old machine types that used to support
the legacy option so it won't break existing configuration that are using
it.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---

Notes:
    v4:
      * drop idea to use "qom-list-properties" and use MachineInfo instead
        which could be inspected with 'query-machines'

 include/hw/boards.h | 3 +++
 hw/arm/virt.c       | 1 +
 hw/i386/pc.c        | 1 +
 hw/ppc/spapr.c      | 1 +
 qapi/misc.json      | 5 ++++-
 vl.c                | 1 +
 6 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6f7916f..86894b6 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -158,6 +158,8 @@ typedef struct {
  * @kvm_type:
  *    Return the type of KVM corresponding to the kvm-type string option or
  *    computed based on other criteria such as the host kernel capabilities.
+ * @numa_mem_supported:
+ *    true if '--numa node.mem' option is supported and false otherwise
  */
 struct MachineClass {
     /*< private >*/
@@ -210,6 +212,7 @@ struct MachineClass {
     bool ignore_boot_device_suffixes;
     bool smbus_no_migration_support;
     bool nvdimm_supported;
+    bool numa_mem_supported;
 
     HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
                                            DeviceState *dev);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index bf54f10..481a603 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     assert(!mc->get_hotplug_handler);
     mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
     hc->plug = virt_machine_device_plug_cb;
+    mc->numa_mem_supported = true;
 }
 
 static void virt_instance_init(Object *obj)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2632b73..05b8368 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2747,6 +2747,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     nc->nmi_monitor_handler = x86_nmi;
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = true;
+    mc->numa_mem_supported = true;
 
     object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
         pc_machine_get_device_memory_region_size, NULL,
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2ef3ce4..265ecfb 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4336,6 +4336,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
      * in which LMBs are represented and hot-added
      */
     mc->numa_mem_align_shift = 28;
+    mc->numa_mem_supported = true;
 
     smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
     smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;
diff --git a/qapi/misc.json b/qapi/misc.json
index 8b3ca4f..d0bdccb 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -2018,12 +2018,15 @@
 #
 # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
 #
+# @numa-mem-supported: true if '-numa node,mem' option is supported by machine
+#                      type and false otherwise (since 4.1)
+#
 # Since: 1.2.0
 ##
 { 'struct': 'MachineInfo',
   'data': { 'name': 'str', '*alias': 'str',
             '*is-default': 'bool', 'cpu-max': 'int',
-            'hotpluggable-cpus': 'bool'} }
+            'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool'} }
 
 ##
 # @query-machines:
diff --git a/vl.c b/vl.c
index 5550bd7..5bf17f5 100644
--- a/vl.c
+++ b/vl.c
@@ -1520,6 +1520,7 @@ MachineInfoList *qmp_query_machines(Error **errp)
         info->name = g_strdup(mc->name);
         info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus;
         info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
+        info->numa_mem_supported = mc->numa_mem_supported;
 
         entry = g_malloc0(sizeof(*entry));
         entry->value = info;
-- 
2.7.4

Re: [Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Markus Armbruster]

Igor Mammedov <imammedo@redhat.com> writes:

> Legacy '-numa node,mem' option has a number of issues and mgmt often
> defaults to it. Unfortunately it's no possible to replace it with
> an alternative '-numa memdev' without breaking migration compatibility.
> What's possible though is to deprecate it, keeping option working with
> old machine types only.
>
> In order to help users to find out if being deprecated CLI option
> '-numa node,mem' is still supported by particular machine type, add new
> "numa-mem-supported" property to MachineInfo description in QAPI schema.

Suggest s/to MachineInfo description in QAPI schema/to output of
query-machines/, because query-machines is the external interface people
know.

> "numa-mem-supported" is set to 'true' for machines that currently support
> NUMA, but it will be flipped to 'false' later on, once deprecation period
> expires and kept 'true' only for old machine types that used to support
> the legacy option so it won't break existing configuration that are using
> it.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>
> Notes:
>     v4:
>       * drop idea to use "qom-list-properties" and use MachineInfo instead
>         which could be inspected with 'query-machines'
>
>  include/hw/boards.h | 3 +++
>  hw/arm/virt.c       | 1 +
>  hw/i386/pc.c        | 1 +
>  hw/ppc/spapr.c      | 1 +
>  qapi/misc.json      | 5 ++++-
>  vl.c                | 1 +
>  6 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 6f7916f..86894b6 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -158,6 +158,8 @@ typedef struct {
>   * @kvm_type:
>   *    Return the type of KVM corresponding to the kvm-type string option or
>   *    computed based on other criteria such as the host kernel capabilities.
> + * @numa_mem_supported:
> + *    true if '--numa node.mem' option is supported and false otherwise
>   */
>  struct MachineClass {
>      /*< private >*/
> @@ -210,6 +212,7 @@ struct MachineClass {
>      bool ignore_boot_device_suffixes;
>      bool smbus_no_migration_support;
>      bool nvdimm_supported;
> +    bool numa_mem_supported;
>  
>      HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
>                                             DeviceState *dev);
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index bf54f10..481a603 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
>      assert(!mc->get_hotplug_handler);
>      mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
>      hc->plug = virt_machine_device_plug_cb;
> +    mc->numa_mem_supported = true;
>  }
>  
>  static void virt_instance_init(Object *obj)
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 2632b73..05b8368 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -2747,6 +2747,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
>      nc->nmi_monitor_handler = x86_nmi;
>      mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
>      mc->nvdimm_supported = true;
> +    mc->numa_mem_supported = true;
>  
>      object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
>          pc_machine_get_device_memory_region_size, NULL,
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 2ef3ce4..265ecfb 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -4336,6 +4336,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
>       * in which LMBs are represented and hot-added
>       */
>      mc->numa_mem_align_shift = 28;
> +    mc->numa_mem_supported = true;
>  
>      smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
>      smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;

This is correct when the TYPE_VIRT_MACHINE, TYPE_PC_MACHINE and
TYPE_SPAPR_MACHINE are exactly the machines supporting NUMA.  How could
I check that?

> diff --git a/qapi/misc.json b/qapi/misc.json
> index 8b3ca4f..d0bdccb 100644
> --- a/qapi/misc.json
> +++ b/qapi/misc.json
> @@ -2018,12 +2018,15 @@
>  #
>  # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
>  #
> +# @numa-mem-supported: true if '-numa node,mem' option is supported by machine
> +#                      type and false otherwise (since 4.1)


"by the machine type", for consistency with @cpu-max.  Also, rather long
line.

> +#
>  # Since: 1.2.0
>  ##
>  { 'struct': 'MachineInfo',
>    'data': { 'name': 'str', '*alias': 'str',
>              '*is-default': 'bool', 'cpu-max': 'int',
> -            'hotpluggable-cpus': 'bool'} }
> +            'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool'} }
>  
>  ##
>  # @query-machines:
> diff --git a/vl.c b/vl.c
> index 5550bd7..5bf17f5 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -1520,6 +1520,7 @@ MachineInfoList *qmp_query_machines(Error **errp)
>          info->name = g_strdup(mc->name);
>          info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus;
>          info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
> +        info->numa_mem_supported = mc->numa_mem_supported;
>  
>          entry = g_malloc0(sizeof(*entry));
>          entry->value = info;

Re: [Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Eduardo Habkost]

On Fri, Jun 07, 2019 at 07:39:17PM +0200, Markus Armbruster wrote:
> This is correct when the TYPE_VIRT_MACHINE, TYPE_PC_MACHINE and
> TYPE_SPAPR_MACHINE are exactly the machines supporting NUMA.  How could
> I check that?

parse_numa_node() rejects the -numa option if the machine doesn't
implement MachineClass::get_default_cpu_node_id().

Grepping for it:

$ git grep -pw get_default_cpu_node_id
hw/arm/virt.c=static void virt_machine_class_init(ObjectClass *oc, void *data)
hw/arm/virt.c:    mc->get_default_cpu_node_id = virt_get_default_cpu_node_id;
hw/core/machine.c=static void machine_numa_finish_cpu_init(MachineState *machine)
hw/core/machine.c:            props.node_id = mc->get_default_cpu_node_id(machine, i);
hw/i386/pc.c=static void pc_machine_class_init(ObjectClass *oc, void *data)
hw/i386/pc.c:    mc->get_default_cpu_node_id = pc_get_default_cpu_node_id;
hw/ppc/spapr.c=static void spapr_machine_class_init(ObjectClass *oc, void *data)
hw/ppc/spapr.c:    mc->get_default_cpu_node_id = spapr_get_default_cpu_node_id;
include/hw/boards.h=typedef struct {
include/hw/boards.h: * @get_default_cpu_node_id:
include/hw/boards.h=struct MachineClass {
include/hw/boards.h:    int64_t (*get_default_cpu_node_id)(const MachineState *ms, int idx);
numa.c=static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
numa.c:    if (!mc->cpu_index_to_instance_props || !mc->get_default_cpu_node_id) {


Related:
  [PATCH v4 01/11] numa: move numa global variable nb_numa_nodes into MachineState
which adds a MachineClass::numa_supported flag to those machines.

-- 
Eduardo

Re: [Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Markus Armbruster]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Jun 07, 2019 at 07:39:17PM +0200, Markus Armbruster wrote:
>> This is correct when the TYPE_VIRT_MACHINE, TYPE_PC_MACHINE and
>> TYPE_SPAPR_MACHINE are exactly the machines supporting NUMA.  How could
>> I check that?
>
> parse_numa_node() rejects the -numa option if the machine doesn't
> implement MachineClass::get_default_cpu_node_id().
>
> Grepping for it:
>
> $ git grep -pw get_default_cpu_node_id
> hw/arm/virt.c=static void virt_machine_class_init(ObjectClass *oc, void *data)
> hw/arm/virt.c:    mc->get_default_cpu_node_id = virt_get_default_cpu_node_id;
> hw/core/machine.c=static void machine_numa_finish_cpu_init(MachineState *machine)
> hw/core/machine.c:            props.node_id = mc->get_default_cpu_node_id(machine, i);
> hw/i386/pc.c=static void pc_machine_class_init(ObjectClass *oc, void *data)
> hw/i386/pc.c:    mc->get_default_cpu_node_id = pc_get_default_cpu_node_id;
> hw/ppc/spapr.c=static void spapr_machine_class_init(ObjectClass *oc, void *data)
> hw/ppc/spapr.c:    mc->get_default_cpu_node_id = spapr_get_default_cpu_node_id;
> include/hw/boards.h=typedef struct {
> include/hw/boards.h: * @get_default_cpu_node_id:
> include/hw/boards.h=struct MachineClass {
> include/hw/boards.h:    int64_t (*get_default_cpu_node_id)(const MachineState *ms, int idx);
> numa.c=static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> numa.c:    if (!mc->cpu_index_to_instance_props || !mc->get_default_cpu_node_id) {
>
>
> Related:
>   [PATCH v4 01/11] numa: move numa global variable nb_numa_nodes into MachineState
> which adds a MachineClass::numa_supported flag to those machines.

Thanks, Eduardo!

Preferably with commit message and doc comment tweaked:
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Igor Mammedov]

On Fri, 07 Jun 2019 19:39:17 +0200
Markus Armbruster <armbru@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> writes:
> 
> > Legacy '-numa node,mem' option has a number of issues and mgmt often
> > defaults to it. Unfortunately it's no possible to replace it with
> > an alternative '-numa memdev' without breaking migration compatibility.
> > What's possible though is to deprecate it, keeping option working with
> > old machine types only.
> >
> > In order to help users to find out if being deprecated CLI option
> > '-numa node,mem' is still supported by particular machine type, add new
> > "numa-mem-supported" property to MachineInfo description in QAPI schema.  
> 
> Suggest s/to MachineInfo description in QAPI schema/to output of
> query-machines/, because query-machines is the external interface people
> know.

fixed

> 
> > "numa-mem-supported" is set to 'true' for machines that currently support
> > NUMA, but it will be flipped to 'false' later on, once deprecation period
> > expires and kept 'true' only for old machine types that used to support
> > the legacy option so it won't break existing configuration that are using
> > it.
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >
> > Notes:
> >     v4:
> >       * drop idea to use "qom-list-properties" and use MachineInfo instead
> >         which could be inspected with 'query-machines'
> >
> >  include/hw/boards.h | 3 +++
> >  hw/arm/virt.c       | 1 +
> >  hw/i386/pc.c        | 1 +
> >  hw/ppc/spapr.c      | 1 +
> >  qapi/misc.json      | 5 ++++-
> >  vl.c                | 1 +
> >  6 files changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 6f7916f..86894b6 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -158,6 +158,8 @@ typedef struct {
> >   * @kvm_type:
> >   *    Return the type of KVM corresponding to the kvm-type string option or
> >   *    computed based on other criteria such as the host kernel capabilities.
> > + * @numa_mem_supported:
> > + *    true if '--numa node.mem' option is supported and false otherwise
> >   */
> >  struct MachineClass {
> >      /*< private >*/
> > @@ -210,6 +212,7 @@ struct MachineClass {
> >      bool ignore_boot_device_suffixes;
> >      bool smbus_no_migration_support;
> >      bool nvdimm_supported;
> > +    bool numa_mem_supported;
> >  
> >      HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> >                                             DeviceState *dev);
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index bf54f10..481a603 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
> >      assert(!mc->get_hotplug_handler);
> >      mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
> >      hc->plug = virt_machine_device_plug_cb;
> > +    mc->numa_mem_supported = true;
> >  }
> >  
> >  static void virt_instance_init(Object *obj)
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2632b73..05b8368 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -2747,6 +2747,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
> >      nc->nmi_monitor_handler = x86_nmi;
> >      mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
> >      mc->nvdimm_supported = true;
> > +    mc->numa_mem_supported = true;
> >  
> >      object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
> >          pc_machine_get_device_memory_region_size, NULL,
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 2ef3ce4..265ecfb 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -4336,6 +4336,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> >       * in which LMBs are represented and hot-added
> >       */
> >      mc->numa_mem_align_shift = 28;
> > +    mc->numa_mem_supported = true;
> >  
> >      smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
> >      smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;  
> 
> This is correct when the TYPE_VIRT_MACHINE, TYPE_PC_MACHINE and
> TYPE_SPAPR_MACHINE are exactly the machines supporting NUMA.  How could
> I check that?

We don't have an interface to communicate that to users as far as I know.

Eduardo pointed out to the patch that might serve as starting point for it though.

> 
> > diff --git a/qapi/misc.json b/qapi/misc.json
> > index 8b3ca4f..d0bdccb 100644
> > --- a/qapi/misc.json
> > +++ b/qapi/misc.json
> > @@ -2018,12 +2018,15 @@
> >  #
> >  # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
> >  #
> > +# @numa-mem-supported: true if '-numa node,mem' option is supported by machine
> > +#                      type and false otherwise (since 4.1)  
> 
> 
> "by the machine type", for consistency with @cpu-max.  Also, rather long
> line.

fixed

> 
> > +#
> >  # Since: 1.2.0
> >  ##
> >  { 'struct': 'MachineInfo',
> >    'data': { 'name': 'str', '*alias': 'str',
> >              '*is-default': 'bool', 'cpu-max': 'int',
> > -            'hotpluggable-cpus': 'bool'} }
> > +            'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool'} }
> >  
> >  ##
> >  # @query-machines:
> > diff --git a/vl.c b/vl.c
> > index 5550bd7..5bf17f5 100644
> > --- a/vl.c
> > +++ b/vl.c
> > @@ -1520,6 +1520,7 @@ MachineInfoList *qmp_query_machines(Error **errp)
> >          info->name = g_strdup(mc->name);
> >          info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus;
> >          info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
> > +        info->numa_mem_supported = mc->numa_mem_supported;
> >  
> >          entry = g_malloc0(sizeof(*entry));
> >          entry->value = info;  

[Qemu-devel] [PATCH v5 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Igor Mammedov]

Legacy '-numa node,mem' option has a number of issues and mgmt often
defaults to it. Unfortunately it's no possible to replace it with
an alternative '-numa memdev' without breaking migration compatibility.
What's possible though is to deprecate it, keeping option working with
old machine types only.

In order to help users to find out if being deprecated CLI option
'-numa node,mem' is still supported by particular machine type, add new
"numa-mem-supported" property to output of query-machines.

"numa-mem-supported" is set to 'true' for machines that currently support
NUMA, but it will be flipped to 'false' later on, once deprecation period
expires and kept 'true' only for old machine types that used to support
the legacy option so it won't break existing configuration that are using
it.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
  v5:
     (Markus Armbruster <armbru@redhat.com>)
        * s/by machine type/by the machine type/
        * ammend commit message
           s/to MachineInfo description in QAPI schema/to output of query-machines/
  v4:
      * drop idea to use "qom-list-properties" and use MachineInfo instead
        which could be inspected with 'query-machines'

 include/hw/boards.h | 3 +++
 hw/arm/virt.c       | 1 +
 hw/i386/pc.c        | 1 +
 hw/ppc/spapr.c      | 1 +
 qapi/misc.json      | 5 ++++-
 vl.c                | 1 +
 6 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6ff02bf..ab6badc 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -158,6 +158,8 @@ typedef struct {
  * @kvm_type:
  *    Return the type of KVM corresponding to the kvm-type string option or
  *    computed based on other criteria such as the host kernel capabilities.
+ * @numa_mem_supported:
+ *    true if '--numa node.mem' option is supported and false otherwise
  */
 struct MachineClass {
     /*< private >*/
@@ -210,6 +212,7 @@ struct MachineClass {
     bool ignore_boot_device_suffixes;
     bool smbus_no_migration_support;
     bool nvdimm_supported;
+    bool numa_mem_supported;
 
     HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
                                            DeviceState *dev);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index bf54f10..481a603 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     assert(!mc->get_hotplug_handler);
     mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
     hc->plug = virt_machine_device_plug_cb;
+    mc->numa_mem_supported = true;
 }
 
 static void virt_instance_init(Object *obj)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index edc240b..25146d7 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2750,6 +2750,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     nc->nmi_monitor_handler = x86_nmi;
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = true;
+    mc->numa_mem_supported = true;
 
     object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
         pc_machine_get_device_memory_region_size, NULL,
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e2b33e5..89d5814 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4340,6 +4340,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
      * in which LMBs are represented and hot-added
      */
     mc->numa_mem_align_shift = 28;
+    mc->numa_mem_supported = true;
 
     smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
     smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;
diff --git a/qapi/misc.json b/qapi/misc.json
index 8b3ca4f..2dbfdf0 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -2018,12 +2018,15 @@
 #
 # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
 #
+# @numa-mem-supported: true if '-numa node,mem' option is supported by
+#                      the machine type and false otherwise (since 4.1)
+#
 # Since: 1.2.0
 ##
 { 'struct': 'MachineInfo',
   'data': { 'name': 'str', '*alias': 'str',
             '*is-default': 'bool', 'cpu-max': 'int',
-            'hotpluggable-cpus': 'bool'} }
+            'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool'} }
 
 ##
 # @query-machines:
diff --git a/vl.c b/vl.c
index cd1fbc4..f5b083f 100644
--- a/vl.c
+++ b/vl.c
@@ -1428,6 +1428,7 @@ MachineInfoList *qmp_query_machines(Error **errp)
         info->name = g_strdup(mc->name);
         info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus;
         info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
+        info->numa_mem_supported = mc->numa_mem_supported;
 
         entry = g_malloc0(sizeof(*entry));
         entry->value = info;
-- 
2.7.4

Re: [Qemu-devel] [PATCH v5 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Markus Armbruster]

Igor Mammedov <imammedo@redhat.com> writes:

> Legacy '-numa node,mem' option has a number of issues and mgmt often
> defaults to it. Unfortunately it's no possible to replace it with
> an alternative '-numa memdev' without breaking migration compatibility.
> What's possible though is to deprecate it, keeping option working with
> old machine types only.
>
> In order to help users to find out if being deprecated CLI option
> '-numa node,mem' is still supported by particular machine type, add new
> "numa-mem-supported" property to output of query-machines.
>
> "numa-mem-supported" is set to 'true' for machines that currently support
> NUMA, but it will be flipped to 'false' later on, once deprecation period
> expires and kept 'true' only for old machine types that used to support
> the legacy option so it won't break existing configuration that are using
> it.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

[Qemu-devel] [PATCH v4 2/3] numa: deprecate 'mem' parameter of '-numa node' option

[Igor Mammedov]

The parameter allows to configure fake NUMA topology where guest
VM simulates NUMA topology but not actually getting performance
benefits from it. The same or better results could be achieved
using 'memdev' parameter.
Beside of unpredictable performance, '-numa node.mem' option has
other issues when it's used with combination of -mem-path +
+ -mem-prealloc + memdev backends (pc-dimm), breaking binding of
memdev backends since mem-path/mem-prealloc are global and affect
the most of RAM allocations.

It's possible to make memdevs and global -mem-path/mem-prealloc
to play nicely together but that will just complicate already
complicated code and add unobious ways it could break on 2
different memmory allocation pathes and their combinations.

Instead of it, consolidate all guest RAM allocation over memdev
which still allows to create fake NUMA configurations if desired
and leaves one simplifyed code path to consider when it comes
to guest RAM allocation.

To achieve desired simplification deprecate 'mem' parameter as its
ad-hoc partitioning of initial RAM MemoryRegion can't be translated
to memdev based backend transparently to users and in compatible
manner (migration wise).

Later down the road that will allow to consolidate means of how
guest RAM is allocated and would permit us to clean up quite
a bit memory allocations and numa code, leaving only 'memdev'
implementation in place.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---

Notes:
    v4:
      * fix up documentation to mention where users should look
        to check if -numa node.mem is supported

 numa.c               |  2 ++
 qemu-deprecated.texi | 16 ++++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/numa.c b/numa.c
index 3875e1e..2205773 100644
--- a/numa.c
+++ b/numa.c
@@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
 
     if (node->has_mem) {
         numa_info[nodenr].node_mem = node->mem;
+        warn_report("Parameter -numa node,mem is deprecated,"
+                    " use -numa node,memdev instead");
     }
     if (node->has_memdev) {
         Object *o;
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 50292d8..eb347f5 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -82,6 +82,22 @@ The @code{-realtime mlock=on|off} argument has been replaced by the
 The ``-virtfs_synth'' argument is now deprecated. Please use ``-fsdev synth''
 and ``-device virtio-9p-...'' instead.
 
+@subsection -numa node,mem=@var{size} (since 4.1)
+
+The parameter @option{mem} of @option{-numa node} is used to assign a part of
+guest RAM to a NUMA node. But when using it, it's impossible to manage specified
+RAM chunk on the host side (like bind it to a host node, setting bind policy, ...),
+so guest end-ups with the fake NUMA configuration with suboptiomal performance.
+However since 2014 there is an alternative way to assign RAM to a NUMA node
+using parameter @option{memdev}, which does the same as @option{mem} and adds
+means to actualy manage node RAM on the host side. Use parameter @option{memdev}
+with @var{memory-backend-ram} backend as an replacement for parameter @option{mem}
+to achieve the same fake NUMA effect or a properly configured
+@var{memory-backend-file} backend to actually benefit from NUMA configuration.
+In future new machine versions will not accept the option but it will still
+work with old machine types. User can check QAPI schema to see if the legacy
+option is supported by looking at MachineInfo::numa-mem-supported property.
+
 @section QEMU Machine Protocol (QMP) commands
 
 @subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-- 
2.7.4

[Qemu-devel] [PATCH v4 3/3] numa: deprecate implict memory distribution between nodes

[Igor Mammedov]

Implicit RAM distribution between nodes has exactly the same issues as:
  "numa: deprecate 'mem' parameter of '-numa node' option"
only with QEMU being the user that's 'adding' 'mem' parameter.

Deprecate it, to get it out of the way so that we could consolidate
guest RAM allocation using memory backends making it consistent and
possibly later on transition to using memory devices instead of
adhoc memory mapping for the initial RAM.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 numa.c               | 3 +++
 qemu-deprecated.texi | 8 ++++++++
 2 files changed, 11 insertions(+)

diff --git a/numa.c b/numa.c
index 2205773..6d45a1f 100644
--- a/numa.c
+++ b/numa.c
@@ -409,6 +409,9 @@ void numa_complete_configuration(MachineState *ms)
         if (i == nb_numa_nodes) {
             assert(mc->numa_auto_assign_ram);
             mc->numa_auto_assign_ram(mc, numa_info, nb_numa_nodes, ram_size);
+            warn_report("Default splitting of RAM between nodes is deprecated,"
+                        " Use '-numa node,memdev' to explictly define RAM"
+                        " allocation per node");
         }
 
         numa_total = 0;
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index eb347f5..c744ba9 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -98,6 +98,14 @@ In future new machine versions will not accept the option but it will still
 work with old machine types. User can check QAPI schema to see if the legacy
 option is supported by looking at MachineInfo::numa-mem-supported property.
 
+@subsection -numa node (without memory specified) (since 4.1)
+
+Splitting RAM by default between NUMA nodes has the same issues as @option{mem}
+parameter described above with the difference that the role of the user plays
+QEMU using implicit generic or board specific splitting rule.
+Use @option{memdev} with @var{memory-backend-ram} backend or @option{mem} (if
+it's supported by used machine type) to define mapping explictly instead.
+
 @section QEMU Machine Protocol (QMP) commands
 
 @subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-- 
2.7.4

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Daniel P. Berrangé]

To summarize implications on libvirt...

This patch currently doesn't change the deprecation status of any machine
type, so it is effectively no-op metadata addition right now.

Libvirt version N can implement support for this today, and nothing would
change from a functional POV with QEMU 4.1.

At some later time, a QEMU release X.Y will mark a new machine type has
having this deprecation enabled, at which point the dormant libvirt logic
will activate, causing libvirt to use the new syntax for NUMA.


IOW, libvirt version N onwards can live migrate QEMU with machine type
pc-X.Y to a host with arbitrary future libvirt or QEMU version.

Libvirt version N-1  can *not* live migrate QEMU with machine type
pc-X.Y to a host with libvirt version N or newer.

This restriction won't be a problem in practice if there's a moderate
time window between libvirt implementing support for this concept,
and the future QEMU X.Y which actually enables the deprecation.

The main downside of this approach is the one Eduardo raised previously
which is that the dormant libvirt logic will not be testable/tested until
that QEMU X.Y actually arrives.

The testing gap is not nice, but I don't think it is sufficient to reject
this approach entirely.

I wonder if there's a way to close the testing gap somehow ? Random idea
would be a non-versioned "pc-no-deprecated" machine type, which blocks
all use of deprecated features and does not promise any migration compat.
Essentially it would exist just for testing purposem as a way todo
functional tests for libvirt & mgmt apps to prove they don't use any
deprecated features (any deprecated features, not merely this  NUMA one).

On Thu, May 30, 2019 at 10:33:16AM +0200, Igor Mammedov wrote:
> Changes since v3:
>   - simplify series by dropping idea of showing property values in "qom-list-properties"
>     and use MachineInfo in QAPI schema instead
> 
> Changes since v2:
>   - taking in account previous review, implement a way for mgmt to intospect if
>     '-numa node,mem' is supported by machine type as suggested by Daniel at
>              https://www.mail-archive.com/qemu-devel@nongnu.org/msg601220.html
>       * ammend "qom-list-properties" to show property values
>       * add "numa-mem-supported" machine property to reflect if '-numa node,mem=SZ'
>         is supported. It culd be used with '-machine none' or at runtime with
>         --preconfig before numa memory mapping are configured
>   * minor fixes to deprecation documentation mentioning "numa-mem-supported" property
> 
>  1) "I'm considering to deprecating -mem-path/prealloc CLI options and replacing
> them with a single memdev Machine property to allow interested users to pick
> used backend for initial RAM (fixes mixed -mem-path+hostmem backends issues)
> and as a transition step to modeling initial RAM as a Device instead of
> (ab)using MemoryRegion APIs."
> (for more details see: https://www.mail-archive.com/qemu-devel@nongnu.org/msg596314.html)
> 
> However there is a couple of roadblocks on the way (s390x and numa memory handling).
> I think I finally thought out a way to hack s390x in migration compatible manner,
> but I don't see any way to do it for -numa node,mem and default RAM assignement
> to nodes. Considering both numa usecases aren't meaningfully using NUMA (aside
> guest side testing) and could be replaced with explicitly used memdev parameter,
> I'd like to propose removing these fake NUMA friends on new machine types,
> hence this deprecation. And once the last machie type that supported the option
> is removed we would be able to remove option altogether.
> 
> As result of removing deprecated options and replacing initial RAM allocation
> with 'memdev's (1), QEMU will allocate guest RAM in consistent way, fixing mixed
> use-case and allowing boards to move towards modelling initial RAM as Device(s).
> Which in its own turn should allow to cleanup NUMA/HMP/memory accounting code
> more by dropping ad-hoc node_mem tracking and reusing memory device enumeration
> instead.
> 
> Reference to previous versions:
>  * https://www.mail-archive.com/qemu-devel@nongnu.org/msg617694.html
> 
> CC: libvir-list@redhat.com
> CC: ehabkost@redhat.com
> CC: pbonzini@redhat.com
> CC: berrange@redhat.com
> CC: armbru@redhat.com
> 
> Igor Mammedov (3):
>   machine: show if CLI option '-numa node,mem' is supported in QAPI
>     schema
>   numa: deprecate 'mem' parameter of '-numa node' option
>   numa: deprecate implict memory distribution between nodes
> 
>  include/hw/boards.h  |  3 +++
>  hw/arm/virt.c        |  1 +
>  hw/i386/pc.c         |  1 +
>  hw/ppc/spapr.c       |  1 +
>  numa.c               |  5 +++++
>  qapi/misc.json       |  5 ++++-
>  qemu-deprecated.texi | 24 ++++++++++++++++++++++++
>  vl.c                 |  1 +
>  8 files changed, 40 insertions(+), 1 deletion(-)
> 
> -- 
> 2.7.4
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Eduardo Habkost]

On Wed, Jun 05, 2019 at 06:33:11PM +0100, Daniel P. Berrangé wrote:
[...]
> I wonder if there's a way to close the testing gap somehow ? Random idea
> would be a non-versioned "pc-no-deprecated" machine type, which blocks
> all use of deprecated features and does not promise any migration compat.
> Essentially it would exist just for testing purposem as a way todo
> functional tests for libvirt & mgmt apps to prove they don't use any
> deprecated features (any deprecated features, not merely this  NUMA one).

This isn't the first time I wish we had a machine type with
experimental features enabled.  What about calling it "pc-next"?

-- 
Eduardo

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Daniel P. Berrangé]

On Wed, Jun 05, 2019 at 03:06:08PM -0300, Eduardo Habkost wrote:
> On Wed, Jun 05, 2019 at 06:33:11PM +0100, Daniel P. Berrangé wrote:
> [...]
> > I wonder if there's a way to close the testing gap somehow ? Random idea
> > would be a non-versioned "pc-no-deprecated" machine type, which blocks
> > all use of deprecated features and does not promise any migration compat.
> > Essentially it would exist just for testing purposem as a way todo
> > functional tests for libvirt & mgmt apps to prove they don't use any
> > deprecated features (any deprecated features, not merely this  NUMA one).
> 
> This isn't the first time I wish we had a machine type with
> experimental features enabled.  What about calling it "pc-next"?

No objection from me.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Markus Armbruster]

Igor Mammedov <imammedo@redhat.com> writes:

> Changes since v3:
>   - simplify series by dropping idea of showing property values in "qom-list-properties"
>     and use MachineInfo in QAPI schema instead

Where did "[PATCH v3 1/6] pc: fix possible NULL pointer dereference in
pc_machine_get_device_memory_region_size()" go?  It fixes a crash bug...

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Igor Mammedov]

On Fri, 07 Jun 2019 19:28:58 +0200
Markus Armbruster <armbru@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> writes:
> 
> > Changes since v3:
> >   - simplify series by dropping idea of showing property values in "qom-list-properties"
> >     and use MachineInfo in QAPI schema instead  
> 
> Where did "[PATCH v3 1/6] pc: fix possible NULL pointer dereference in
> pc_machine_get_device_memory_region_size()" go?  It fixes a crash bug...

I'll post it as separate patch as it's not more related to this series

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Igor Mammedov]

On Thu, 30 May 2019 10:33:16 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

> Changes since v3:
>   - simplify series by dropping idea of showing property values in "qom-list-properties"
>     and use MachineInfo in QAPI schema instead
> 
> Changes since v2:
>   - taking in account previous review, implement a way for mgmt to intospect if
>     '-numa node,mem' is supported by machine type as suggested by Daniel at
>              https://www.mail-archive.com/qemu-devel@nongnu.org/msg601220.html
>       * ammend "qom-list-properties" to show property values
>       * add "numa-mem-supported" machine property to reflect if '-numa node,mem=SZ'
>         is supported. It culd be used with '-machine none' or at runtime with
>         --preconfig before numa memory mapping are configured
>   * minor fixes to deprecation documentation mentioning "numa-mem-supported" property
> 
>  1) "I'm considering to deprecating -mem-path/prealloc CLI options and replacing
> them with a single memdev Machine property to allow interested users to pick
> used backend for initial RAM (fixes mixed -mem-path+hostmem backends issues)
> and as a transition step to modeling initial RAM as a Device instead of
> (ab)using MemoryRegion APIs."
> (for more details see: https://www.mail-archive.com/qemu-devel@nongnu.org/msg596314.html)
> 
> However there is a couple of roadblocks on the way (s390x and numa memory handling).
> I think I finally thought out a way to hack s390x in migration compatible manner,
> but I don't see any way to do it for -numa node,mem and default RAM assignement
> to nodes. Considering both numa usecases aren't meaningfully using NUMA (aside
> guest side testing) and could be replaced with explicitly used memdev parameter,
> I'd like to propose removing these fake NUMA friends on new machine types,
> hence this deprecation. And once the last machie type that supported the option
> is removed we would be able to remove option altogether.
> 
> As result of removing deprecated options and replacing initial RAM allocation
> with 'memdev's (1), QEMU will allocate guest RAM in consistent way, fixing mixed
> use-case and allowing boards to move towards modelling initial RAM as Device(s).
> Which in its own turn should allow to cleanup NUMA/HMP/memory accounting code
> more by dropping ad-hoc node_mem tracking and reusing memory device enumeration
> instead.

Eduardo,

could you take and merge it via numa/machine tree?

> 
> Reference to previous versions:
>  * https://www.mail-archive.com/qemu-devel@nongnu.org/msg617694.html
> 
> CC: libvir-list@redhat.com
> CC: ehabkost@redhat.com
> CC: pbonzini@redhat.com
> CC: berrange@redhat.com
> CC: armbru@redhat.com
> 
> Igor Mammedov (3):
>   machine: show if CLI option '-numa node,mem' is supported in QAPI
>     schema
>   numa: deprecate 'mem' parameter of '-numa node' option
>   numa: deprecate implict memory distribution between nodes
> 
>  include/hw/boards.h  |  3 +++
>  hw/arm/virt.c        |  1 +
>  hw/i386/pc.c         |  1 +
>  hw/ppc/spapr.c       |  1 +
>  numa.c               |  5 +++++
>  qapi/misc.json       |  5 ++++-
>  qemu-deprecated.texi | 24 ++++++++++++++++++++++++
>  vl.c                 |  1 +
>  8 files changed, 40 insertions(+), 1 deletion(-)
>