[Qemu-devel] [PATCH v8 0/3] machvirt dynamic sysbus device instantiation

[Qemu-devel] [PATCH v8 0/3] machvirt dynamic sysbus device instantiation

[Eric Auger]

This patch series enables machvirt to dynamically instantiate sysbus
devices from command line (using -device option).

All those sysbus devices are plugged onto a platform bus. This latter
device is instantiated in machvirt and takes care of the binding of
children sysbus devices on a machine init done notifier. The device
tree node generation for children dynamic sysbus device also happens
on a subsequent notifier that must be executed after the above one.
machvirt registers that notifier before the platform bus creation to
make sure notifiers are executed in the right order: dt generation after
actual QOM binding.

Very few sysbus devices are supposed to be instantiated that
way. VFIO devices belong to them.

Node creation really is architecture specific. On ARM the dynamic
sysbus device node creation is implemented in a new C module,
hw/arm/sysbus-fdt.c and not in the machine file.

Machvirt transformations and sysbus-fdt are largely inspired from Alex work.

The patch series can be found at:
http://git.linaro.org/people/eric.auger/qemu.git,
branch official_dynsysbus_v8
Previous version on official_dynsysbus_v7

Best Regards

Eric

v7 -> v8:
- rebase on 2.2.0 (boot.c and virt.c)
- in virt.c machvirt_init, create_platform_bus simply is added
  after the arm_load_kernel call instead of moving this latter.
  should ease the review.
- Add Alex & Shannon Reviewed-by

v6 -> v7:
Take into account Shannon's comments:
- hw/arm/sysbus-fdt.c: revert indentation in add_fdt_node_functions
- hw/arm/virt.c:
  - add an additional comment in a15irqmap related to PLATFORM_BUS_NUM_IRQS
  - correct platform bus size to 0x400000
  - remove PLATFORM_BUS_FIRST_IRQ macro

v5 -> v6:
Take into account Peter's comments:
- dtb overload mechanism rewritten: arm_load_kernel code is moved into a
  machine init done notifier notify instead. arm_load_kernel only registers
  that notifier. As a consequence the dtb is loaded once.
- v5 1-4 patch files are removed and replaced by a single patch file moving
  arm_load_kernel in the notifier (2).
- as a consequence arm_load_kernel must be called before sysbus-fdt
  arm_register_platform_bus_fdt_creator.
- In virt, platform_bus_params not a const anymore since its fields are
  initialized from vbi->memmap and vbi->irqmap. Hence create_platform_bus
  proto can be simplified.
- In sysbus-fdt add_all_platform_bus_fdt_nodes now takes a handle to
  an ARMPlatformBusFdtParams. This is not a modify_dtb function anymore
  fdt pointer is checked in case the callback is called after the load_dtb
  (this latter deallocated fdt pointer). Check of fdt_filename moved in here.
  upgrade_dtb is removed. copyright aligned between .h and .c.

v4 -> v5:
- in virt.c: platform_bus_params becomes static const
- sysbus-fdt: change indentation in add_fdt_node_functions array init
- s/load_dtb/arm_load_dtb in one boot.c comment

v3 -> v4:
- dyn_sysbus_binding removed since binding stuff now are implemented by
  the platform bus device
- due to a change in ARM load_dtb implementation using rom_add_blob_fixed,
  the dt no more is generated in a reset notifier but is generated on a
  machine init done notifier
- the augmented device tree is not generated from scratch anymore but is
  added using a modify_dtb function. This required some small change in
  boot.c
- the case where the user provides a dtb file now is handled
- some cleanup in virt additions
- implement a list of dyanmically instantiable devices in sysbus-fdt

v2 -> v3:
- patch now applies on top of Alex full patchset
- dyn_sysbus_devtree: add arm_prefix to emphasize the fact those
  functions are arm specific; arm_sysbus_device_create_devtree
  becomes static
- load_dtb renamed into arm_load_dtb
- add copyright in hw/arm/dyn_sysbus_devtree.c

v1 -> v2:
- device node generation no more in sysbus device but in
  dyn_sysbus_devtree
- VFIO region shrinked to 4MB and relocated in machvirt to avoid PCI
  shrink (dynamic vfio-mmio support might come latter)
- platform_bus_base removed from PlatformDevtreeData


Eric Auger (3):
  hw/arm/sysbus-fdt: helpers for platform bus nodes addition
  hw/arm/boot: arm_load_kernel implemented as a machine init done
    notifier
  hw/arm/virt: add dynamic sysbus device support

 hw/arm/Makefile.objs        |   1 +
 hw/arm/boot.c               |  14 +++-
 hw/arm/sysbus-fdt.c         | 173 ++++++++++++++++++++++++++++++++++++++++++++
 hw/arm/virt.c               |  77 +++++++++++++++++---
 include/hw/arm/arm.h        |  28 +++++++
 include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
 6 files changed, 343 insertions(+), 10 deletions(-)
 create mode 100644 hw/arm/sysbus-fdt.c
 create mode 100644 include/hw/arm/sysbus-fdt.h

-- 
1.8.3.2

[Qemu-devel] [PATCH v8 1/3] hw/arm/sysbus-fdt: helpers for platform bus nodes addition

[Eric Auger]

This new C module will be used by ARM machine files to generate
platform bus node and their dynamic sysbus device tree nodes.

Dynamic sysbus device node addition is done in a machine init
done notifier. arm_register_platform_bus_fdt_creator does the
registration of this latter and is supposed to be called by
ARM machine files that support platform bus and their dynamic
sysbus. Addition of dynamic sysbus nodes is done only if the
user did not provide any dtb.

Signed-off-by: Alexander Graf <agraf@suse.de>
Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Alexander Graf <agraf@suse.de>

---
v7 -> v8:
add Reviewed-by from Alex and Shannon

v6 -> v7:
- revert indentation in add_fdt_node_functions

v5 -> v6:
- add_all_platform_bus_fdt_nodes is not a modify_dtb function anymore
- it now takes a handle to an ARMPlatformBusFdtParams.
- fdt pointer is checked in case this notifier is executed after the
  one that executes the load_dtb (this latter deallocates the fdt pointer)
- check of fdt_filename moved in here.
- upgrade_dtb is removed
- copyright aligned between .h and .c

v4 -> v5:
- change indentation in add_fdt_node_functions. Also becomes a
  static const.
- ARMPlatformBusFdtParams.system_params becomes a pointer to
  a const ARMPlatformBusSystemParams
- removes platform-bus.h second inclusion

v3 -> v4:
- dyn_sysbus_devtree.c renamed into sysbus-fdt.c
- use new PlatformBusDevice object
- the dtb upgrade is done through modify_dtb. Before the fdt
  was recreated from scratch. When the user provided a dtb this
  latter was overwritten which was not correct.
- an array contains the association between device type names
  and their node creation function
- I must aknowledge I did not find any cleaner way to implement
  a FDT_BUILDER interface, as suggested by Paolo. The class method
  would need to be initialized somewhere and since it cannot
  happen in the device itself - according to Alex & Peter comments -,
  I don't see when I shall associate the device type and its
  interface implementation.

v2 -> v3:
- add arm_ prefix
- arm_sysbus_device_create_devtree becomes static

v1 -> v2:
- Code moved in an arch specific file to accomodate architecture
  dependent specificities.
- remove platform_bus_base from PlatformDevtreeData

v1: code originally written by Alex Graf in e500.c and reused for
ARM [Eric Auger]
---
 hw/arm/Makefile.objs        |   1 +
 hw/arm/sysbus-fdt.c         | 173 ++++++++++++++++++++++++++++++++++++++++++++
 include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
 3 files changed, 234 insertions(+)
 create mode 100644 hw/arm/sysbus-fdt.c
 create mode 100644 include/hw/arm/sysbus-fdt.h

diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
index 6088e53..0cc63e1 100644
--- a/hw/arm/Makefile.objs
+++ b/hw/arm/Makefile.objs
@@ -3,6 +3,7 @@ obj-$(CONFIG_DIGIC) += digic_boards.o
 obj-y += integratorcp.o kzm.o mainstone.o musicpal.o nseries.o
 obj-y += omap_sx1.o palm.o realview.o spitz.o stellaris.o
 obj-y += tosa.o versatilepb.o vexpress.o virt.o xilinx_zynq.o z2.o
+obj-y += sysbus-fdt.o
 
 obj-y += armv7m.o exynos4210.o pxa2xx.o pxa2xx_gpio.o pxa2xx_pic.o
 obj-$(CONFIG_DIGIC) += digic.o
diff --git a/hw/arm/sysbus-fdt.c b/hw/arm/sysbus-fdt.c
new file mode 100644
index 0000000..a1dc0aa
--- /dev/null
+++ b/hw/arm/sysbus-fdt.c
@@ -0,0 +1,173 @@
+/*
+ * ARM Platform Bus device tree generation helpers
+ *
+ * Copyright (c) 2014 Linaro Limited
+ *
+ * Authors:
+ *  Alex Graf <agraf@suse.de>
+ *  Eric Auger <eric.auger@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "hw/arm/sysbus-fdt.h"
+#include "qemu/error-report.h"
+#include "sysemu/device_tree.h"
+#include "hw/platform-bus.h"
+#include "sysemu/sysemu.h"
+
+/*
+ * internal struct that contains the information to create dynamic
+ * sysbus device node
+ */
+typedef struct PlatformBusFdtData {
+    void *fdt; /* device tree handle */
+    int irq_start; /* index of the first IRQ usable by platform bus devices */
+    const char *pbus_node_name; /* name of the platform bus node */
+    PlatformBusDevice *pbus;
+} PlatformBusFdtData;
+
+/*
+ * struct used when calling the machine init done notifier
+ * that constructs the fdt nodes of platform bus devices
+ */
+typedef struct PlatformBusFdtNotifierParams {
+    ARMPlatformBusFdtParams *fdt_params;
+    Notifier notifier;
+} PlatformBusFdtNotifierParams;
+
+/* struct that associates a device type name and a node creation function */
+typedef struct NodeCreationPair {
+    const char *typename;
+    int (*add_fdt_node_fn)(SysBusDevice *sbdev, void *opaque);
+} NodeCreationPair;
+
+/* list of supported dynamic sysbus devices */
+static const NodeCreationPair add_fdt_node_functions[] = {
+    {"", NULL}, /*last element*/
+};
+
+/**
+ * add_fdt_node - add the device tree node of a dynamic sysbus device
+ *
+ * @sbdev: handle to the sysbus device
+ * @opaque: handle to the PlatformBusFdtData
+ *
+ * Checks the sysbus type belongs to the list of device types that
+ * are dynamically instantiable and in the positive call the node
+ * creation function.
+ */
+static int add_fdt_node(SysBusDevice *sbdev, void *opaque)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(add_fdt_node_functions); i++) {
+        if (!strcmp(object_get_typename(OBJECT(sbdev)),
+                    add_fdt_node_functions[i].typename)) {
+            add_fdt_node_functions[i].add_fdt_node_fn(sbdev, opaque);
+            return 0;
+        }
+    }
+    error_report("Device %s can not be dynamically instantiated",
+                     qdev_fw_name(DEVICE(sbdev)));
+    return 1;
+}
+
+/**
+ * add_all_platform_bus_fdt_nodes - create all the platform bus nodes
+ *
+ * builds the parent platform bus node and all the nodes of dynamic
+ * sysbus devices attached to it.
+ */
+static void add_all_platform_bus_fdt_nodes(ARMPlatformBusFdtParams *fdt_params)
+{
+    const char platcomp[] = "qemu,platform\0simple-bus";
+    PlatformBusDevice *pbus;
+    DeviceState *dev;
+    gchar *node;
+    uint64_t addr, size;
+    int irq_start, dtb_size;
+    struct arm_boot_info *info = fdt_params->binfo;
+    const ARMPlatformBusSystemParams *params = fdt_params->system_params;
+    const char *intc = fdt_params->intc;
+    void *fdt = info->get_dtb(info, &dtb_size);
+
+    /*
+     * In case the user provided a dtb, we assume he already integrated the
+     * dynamic sysbus nodes. This corresponds to a use case where the dynamic
+     * sysbus nodes are complex and their generation is not yet supported. In
+     * case the use can take in charge the guest dt while qemu takes in charge
+     * the qom stuff.
+     */
+    if (info->dtb_filename) {
+        return;
+    }
+
+    if (!fdt) {
+        fprintf(stderr,
+                "fdt NULL pointer: platform bus nodes cannot be added\n") ;
+        return;
+    }
+
+    node = g_strdup_printf("/platform@%"PRIx64, params->platform_bus_base);
+    addr = params->platform_bus_base;
+    size = params->platform_bus_size;
+    irq_start = params->platform_bus_first_irq;
+
+    /* Create a /platform node that we can put all devices into */
+    qemu_fdt_add_subnode(fdt, node);
+    qemu_fdt_setprop(fdt, node, "compatible", platcomp, sizeof(platcomp));
+
+    /* Our platform bus region is less than 32bit big, so 1 cell is enough for
+       address and size */
+    qemu_fdt_setprop_cells(fdt, node, "#size-cells", 1);
+    qemu_fdt_setprop_cells(fdt, node, "#address-cells", 1);
+    qemu_fdt_setprop_cells(fdt, node, "ranges", 0, addr >> 32, addr, size);
+
+    qemu_fdt_setprop_phandle(fdt, node, "interrupt-parent", intc);
+
+    dev = qdev_find_recursive(sysbus_get_default(), TYPE_PLATFORM_BUS_DEVICE);
+    pbus = PLATFORM_BUS_DEVICE(dev);
+
+    /* We can only create dt nodes for dynamic devices when they're ready */
+    if (pbus->done_gathering) {
+        PlatformBusFdtData data = {
+            .fdt = fdt,
+            .irq_start = irq_start,
+            .pbus_node_name = node,
+            .pbus = pbus,
+        };
+
+        /* Loop through all dynamic sysbus devices and create their node */
+        foreach_dynamic_sysbus_device(add_fdt_node, &data);
+    }
+
+    g_free(node);
+}
+
+static void platform_bus_fdt_notify(Notifier *notifier, void *data)
+{
+    PlatformBusFdtNotifierParams *p =
+        container_of(notifier, PlatformBusFdtNotifierParams, notifier);
+    add_all_platform_bus_fdt_nodes(p->fdt_params);
+}
+
+void arm_register_platform_bus_fdt_creator(ARMPlatformBusFdtParams *fdt_params)
+{
+    PlatformBusFdtNotifierParams *p = g_new(PlatformBusFdtNotifierParams, 1);
+
+    p->fdt_params = fdt_params;
+    p->notifier.notify = platform_bus_fdt_notify;
+    qemu_add_machine_init_done_notifier(&p->notifier);
+}
diff --git a/include/hw/arm/sysbus-fdt.h b/include/hw/arm/sysbus-fdt.h
new file mode 100644
index 0000000..7ad9442
--- /dev/null
+++ b/include/hw/arm/sysbus-fdt.h
@@ -0,0 +1,60 @@
+/*
+ * Dynamic sysbus device tree node generation API
+ *
+ * Copyright Linaro Limited, 2014
+ *
+ * Authors:
+ *  Alex Graf <agraf@suse.de>
+ *  Eric Auger <eric.auger@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef HW_ARM_SYSBUS_FDT_H
+#define HW_ARM_SYSBUS_FDT_H
+
+#include "hw/arm/arm.h"
+#include "qemu-common.h"
+#include "hw/sysbus.h"
+
+/*
+ * struct that contains dimensioning parameters of the platform bus
+ */
+typedef struct {
+    hwaddr platform_bus_base; /* start address of the bus */
+    hwaddr platform_bus_size; /* size of the bus */
+    int platform_bus_first_irq; /* first hwirq assigned to the bus */
+    int platform_bus_num_irqs; /* number of hwirq assigned to the bus */
+} ARMPlatformBusSystemParams;
+
+/*
+ * struct that contains all relevant info to build the fdt nodes of
+ * platform bus and attached dynamic sysbus devices
+ * in the future might be augmented with additional info
+ * such as PHY, CLK handles ...
+ */
+typedef struct {
+    const ARMPlatformBusSystemParams *system_params;
+    struct arm_boot_info *binfo;
+    const char *intc; /* parent interrupt controller name */
+} ARMPlatformBusFdtParams;
+
+/**
+ * arm_register_platform_bus_fdt_creator - register a machine init done
+ * notifier that creates the device tree nodes of the platform bus and
+ * associated dynamic sysbus devices
+ */
+void arm_register_platform_bus_fdt_creator(ARMPlatformBusFdtParams *fdt_params);
+
+#endif
-- 
1.8.3.2

[Qemu-devel] [PATCH v8 2/3] hw/arm/boot: arm_load_kernel implemented as a machine init done notifier

[Eric Auger]

Device tree nodes for the platform bus and its children dynamic sysbus
devices are added in a machine init done notifier. To load the dtb once,
after those latter nodes are built and before ROM freeze, the actual
arm_load_kernel existing code is moved into a notifier notify function,
arm_load_kernel_notify. arm_load_kernel now only registers the
corresponding notifier.

Machine files that do not support platform bus stay unchanged. Machine
files willing to support dynamic sysbus devices must call arm_load_kernel
before sysbus-fdt arm_register_platform_bus_fdt_creator to make sure
dynamic sysbus device nodes are integrated in the dtb.

Signed-off-by: Eric Auger <eric.auger@linaro.org>
Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Alexander Graf <agraf@suse.de>

---

v7 -> v8:
- Add Reviewed-by from Alex & Shannon
- rebase on 2.2.0

v6: creation of this patch file
---
 hw/arm/boot.c        | 14 +++++++++++++-
 include/hw/arm/arm.h | 28 ++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 52ebd8b..51f01aa 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -537,7 +537,7 @@ static void load_image_to_fw_cfg(FWCfgState *fw_cfg, uint16_t size_key,
     fw_cfg_add_bytes(fw_cfg, data_key, data, size);
 }
 
-void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
+static void arm_load_kernel_notify(Notifier *notifier, void *data)
 {
     CPUState *cs;
     int kernel_size;
@@ -548,6 +548,11 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
     hwaddr entry, kernel_load_offset;
     int big_endian;
     static const ARMInsnFixup *primary_loader;
+    ArmLoadKernelNotifier *n =
+        container_of(notifier, ArmLoadKernelNotifier, notifier);
+    ARMCPU *cpu = n->cpu;
+    struct arm_boot_info *info =
+        container_of(n, struct arm_boot_info, load_kernel_notifier);
 
     /* CPU objects (unlike devices) are not automatically reset on system
      * reset, so we must always register a handler to do so. If we're
@@ -755,3 +760,10 @@ void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
         ARM_CPU(cs)->env.boot_info = info;
     }
 }
+
+void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info)
+{
+    info->load_kernel_notifier.cpu = cpu;
+    info->load_kernel_notifier.notifier.notify = arm_load_kernel_notify;
+    qemu_add_machine_init_done_notifier(&info->load_kernel_notifier.notifier);
+}
diff --git a/include/hw/arm/arm.h b/include/hw/arm/arm.h
index c4bf56d..aa6787f 100644
--- a/include/hw/arm/arm.h
+++ b/include/hw/arm/arm.h
@@ -19,6 +19,16 @@ qemu_irq *armv7m_init(MemoryRegion *system_memory,
                       int flash_size, int sram_size,
                       const char *kernel_filename, const char *cpu_model);
 
+/*
+ * struct used as a parameter of the arm_load_kernel machine init
+ * done notifier
+ */
+typedef struct {
+    ARMCPU *cpu; /* handle to the first cpu object */
+    Notifier notifier; /* actual notifier */
+} ArmLoadKernelNotifier;
+
+
 /* arm_boot.c */
 struct arm_boot_info {
     uint64_t ram_size;
@@ -65,6 +75,8 @@ struct arm_boot_info {
      * the user it should implement this hook.
      */
     void (*modify_dtb)(const struct arm_boot_info *info, void *fdt);
+    /* machine init done notifier executing arm_load_dtb */
+    ArmLoadKernelNotifier load_kernel_notifier;
     /* Used internally by arm_boot.c */
     int is_linux;
     hwaddr initrd_start;
@@ -76,6 +88,22 @@ struct arm_boot_info {
      */
     bool firmware_loaded;
 };
+
+/**
+ * arm_load_kernel - Loads memory with everything needed to boot
+ *
+ * @cpu: handle to the first CPU object
+ * @info: handle to the boot info struct
+ * Registers a machine init done notifier that copies to memory
+ * everything needed to boot, depending on machine and user options:
+ * kernel image, boot loaders, initrd, dtb. Also registers the CPU
+ * reset handler.
+ *
+ * In case the machine file supports the platform bus device and its
+ * dynamically instantiable sysbus devices, this function must be called
+ * before sysbus-fdt arm_register_platform_bus_fdt_creator. Indeed the
+ * machine init done notifiers are called in registration reverse order.
+ */
 void arm_load_kernel(ARMCPU *cpu, struct arm_boot_info *info);
 
 /* Multiplication factor to convert from system clock ticks to qemu timer
-- 
1.8.3.2

[Qemu-devel] [PATCH v8 3/3] hw/arm/virt: add dynamic sysbus device support

[Eric Auger]

Allows sysbus devices to be instantiated from command line by
using -device option. Machvirt creates a platform bus at init.
The dynamic sysbus devices are attached to this platform bus device.

The platform bus device registers a machine init done notifier
whose role will be to bind the dynamic sysbus devices. Indeed
dynamic sysbus devices are created after machine init.

machvirt also registers a notifier that will build the device
tree nodes for the platform bus and its children dynamic sysbus
devices.

Signed-off-by: Alexander Graf <agraf@suse.de>
Signed-off-by: Eric Auger <eric.auger@linaro.org>

---
v7 -> v8:
- rebase on 2.2.0
- in machvirt_init, create_platform_bus simply is added
  after the arm_load_kernel call instead of moving this latter.
  Related comment slighly reworded.
- Due to those changes I dropped Alex and Shannon's Reviewed-by

v6 -> v7:
Take into account Shannon comments:
- remove PLATFORM_BUS_FIRST_IRQ macro
- correct platform bus size to 0x400000
- add an additional comment in a15irqmap related to
  PLATFORM_BUS_NUM_IRQS

v5 -> v6:
- Take into account Peter's comments:
  - platform_bus_params initialized from vbi->memmap and vbi->irqmap.
    As a consequence, const is removed. Also alignment in a15memmap
    is slightly changed.
  - ARMPlatformBusSystemParams handle removed from create_platform_bus
    prototype
- arm_load_kernel has become a machine init done notifier registration.
  It must be called before platform_bus_create to guarantee the correct
  notifier execution sequence

v4 -> v5:
- platform_bus_params becomes static const
- reword comment in create_platform_bus
- reword the commit message

v3 -> v4:
- use platform bus object, instantiated in create_platform_bus
- device tree generation for platform bus and children dynamic
  sysbus devices is no more handled at reset but in a
  machine_init_done_notifier (due to the change in implementaion
  of ARM load dtb using rom_add_blob_fixed).
- device tree enhancement now takes into account the case of
  user provided dtb. Before the user dtb was overwritten which
  was wrong. However in case the dtb is provided by the user,
  dynamic sysbus nodes are not added there.
- renaming of MACHVIRT_PLATFORM defines
- MACHVIRT_PLATFORM_PAGE_SHIFT and SIZE_PAGES not needed anymore,
  hence removed.
- DynSysbusParams struct renamed into ARMPlatformBusSystemParams
  and above params removed.
- separation of dt creation and QEMU binding is not mandated anymore
  since the device tree is not created from scratch anymore. Instead
  the modify_dtb function is used.
- create_platform_bus registers another machine init done notifier
  to start VFIO IRQ handling. This latter executes after the
  dynamic sysbus device binding.

v2 -> v3:
- renaming of arm_platform_bus_create_devtree and arm_load_dtb
- add copyright in hw/arm/dyn_sysbus_devtree.c

v1 -> v2:
- remove useless vfio-platform.h include file
- s/MACHVIRT_PLATFORM_HOLE/MACHVIRT_PLATFORM_SIZE
- use dyn_sysbus_binding and dyn_sysbus_devtree
- dynamic sysbus platform buse size shrinked to 4MB and
  moved between RTC and MMIO

v1:

Inspired from what Alex Graf did in ppc e500
https://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00012.html

Conflicts:
	hw/arm/sysbus-fdt.c

Conflicts:
	hw/arm/virt.c
---
 hw/arm/virt.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 68 insertions(+), 9 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 2353440..40f4f0f 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -42,6 +42,8 @@
 #include "exec/address-spaces.h"
 #include "qemu/bitops.h"
 #include "qemu/error-report.h"
+#include "hw/arm/sysbus-fdt.h"
+#include "hw/platform-bus.h"
 
 #define NUM_VIRTIO_TRANSPORTS 32
 
@@ -59,6 +61,8 @@
 #define GIC_FDT_IRQ_PPI_CPU_START 8
 #define GIC_FDT_IRQ_PPI_CPU_WIDTH 8
 
+#define PLATFORM_BUS_NUM_IRQS     20
+
 enum {
     VIRT_FLASH,
     VIRT_MEM,
@@ -69,8 +73,11 @@ enum {
     VIRT_MMIO,
     VIRT_RTC,
     VIRT_FW_CFG,
+    VIRT_PLATFORM_BUS,
 };
 
+static ARMPlatformBusSystemParams platform_bus_params;
+
 typedef struct MemMapEntry {
     hwaddr base;
     hwaddr size;
@@ -119,24 +126,26 @@ typedef struct {
  */
 static const MemMapEntry a15memmap[] = {
     /* Space up to 0x8000000 is reserved for a boot ROM */
-    [VIRT_FLASH] =      {          0, 0x08000000 },
-    [VIRT_CPUPERIPHS] = { 0x08000000, 0x00020000 },
+    [VIRT_FLASH] =          {          0, 0x08000000 },
+    [VIRT_CPUPERIPHS] =     { 0x08000000, 0x00020000 },
     /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
-    [VIRT_GIC_DIST] =   { 0x08000000, 0x00010000 },
-    [VIRT_GIC_CPU] =    { 0x08010000, 0x00010000 },
-    [VIRT_UART] =       { 0x09000000, 0x00001000 },
-    [VIRT_RTC] =        { 0x09010000, 0x00001000 },
-    [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
-    [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
+    [VIRT_GIC_DIST] =       { 0x08000000, 0x00010000 },
+    [VIRT_GIC_CPU] =        { 0x08010000, 0x00010000 },
+    [VIRT_UART] =           { 0x09000000, 0x00001000 },
+    [VIRT_RTC] =            { 0x09010000, 0x00001000 },
+    [VIRT_FW_CFG] =         { 0x09020000, 0x0000000a },
+    [VIRT_PLATFORM_BUS] =   { 0x09400000, 0x00400000 },
+    [VIRT_MMIO] =           { 0x0a000000, 0x00000200 },
     /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
     /* 0x10000000 .. 0x40000000 reserved for PCI */
-    [VIRT_MEM] =        { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
+    [VIRT_MEM] =            { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
 };
 
 static const int a15irqmap[] = {
     [VIRT_UART] = 1,
     [VIRT_RTC] = 2,
     [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */
+    [VIRT_PLATFORM_BUS] = 48, /* .. to 48 + PLATFORM_BUS_NUM_IRQS -1*/
 };
 
 static VirtBoardInfo machines[] = {
@@ -556,6 +565,47 @@ static void create_fw_cfg(const VirtBoardInfo *vbi)
     g_free(nodename);
 }
 
+static void create_platform_bus(VirtBoardInfo *vbi, qemu_irq *pic)
+{
+    DeviceState *dev;
+    SysBusDevice *s;
+    int i;
+    ARMPlatformBusFdtParams *fdt_params = g_new(ARMPlatformBusFdtParams, 1);
+    MemoryRegion *sysmem = get_system_memory();
+
+    platform_bus_params.platform_bus_base = vbi->memmap[VIRT_PLATFORM_BUS].base;
+    platform_bus_params.platform_bus_size = vbi->memmap[VIRT_PLATFORM_BUS].size;
+    platform_bus_params.platform_bus_first_irq = vbi->irqmap[VIRT_PLATFORM_BUS];
+    platform_bus_params.platform_bus_num_irqs = PLATFORM_BUS_NUM_IRQS;
+
+    fdt_params->system_params = &platform_bus_params;
+    fdt_params->binfo = &vbi->bootinfo;
+    fdt_params->intc = "/intc";
+    /*
+     * register a machine init done notifier that creates the device tree
+     * nodes of the platform bus and its children dynamic sysbus devices
+     */
+    arm_register_platform_bus_fdt_creator(fdt_params);
+
+    dev = qdev_create(NULL, TYPE_PLATFORM_BUS_DEVICE);
+    dev->id = TYPE_PLATFORM_BUS_DEVICE;
+    qdev_prop_set_uint32(dev, "num_irqs",
+        platform_bus_params.platform_bus_num_irqs);
+    qdev_prop_set_uint32(dev, "mmio_size",
+        platform_bus_params.platform_bus_size);
+    qdev_init_nofail(dev);
+    s = SYS_BUS_DEVICE(dev);
+
+    for (i = 0; i < platform_bus_params.platform_bus_num_irqs; i++) {
+        int irqn = platform_bus_params.platform_bus_first_irq + i;
+        sysbus_connect_irq(s, i, pic[irqn]);
+    }
+
+    memory_region_add_subregion(sysmem,
+                                platform_bus_params.platform_bus_base,
+                                sysbus_mmio_get_region(s, 0));
+}
+
 static void *machvirt_dtb(const struct arm_boot_info *binfo, int *fdt_size)
 {
     const VirtBoardInfo *board = (const VirtBoardInfo *)binfo;
@@ -658,6 +708,14 @@ static void machvirt_init(MachineState *machine)
     vbi->bootinfo.get_dtb = machvirt_dtb;
     vbi->bootinfo.firmware_loaded = bios_name || drive_get(IF_PFLASH, 0, 0);
     arm_load_kernel(ARM_CPU(first_cpu), &vbi->bootinfo);
+
+    /*
+     * arm_load_kernel machine init done notifier registration must
+     * happen before the platform_bus_create call. In this latter,
+     * another notifier is registered which adds platform bus nodes.
+     * Notifiers are executed in registration reverse order.
+     */
+    create_platform_bus(vbi, pic);
 }
 
 static bool virt_get_secure(Object *obj, Error **errp)
@@ -696,6 +754,7 @@ static void virt_class_init(ObjectClass *oc, void *data)
     mc->desc = "ARM Virtual Machine",
     mc->init = machvirt_init;
     mc->max_cpus = 8;
+    mc->has_dynamic_sysbus = true;
 }
 
 static const TypeInfo machvirt_info = {
-- 
1.8.3.2

Re: [Qemu-devel] [PATCH v8 2/3] hw/arm/boot: arm_load_kernel implemented as a machine init done notifier

[Peter Maydell]

On 5 January 2015 at 16:14, Eric Auger <eric.auger@linaro.org> wrote:
> Device tree nodes for the platform bus and its children dynamic sysbus
> devices are added in a machine init done notifier. To load the dtb once,
> after those latter nodes are built and before ROM freeze, the actual
> arm_load_kernel existing code is moved into a notifier notify function,
> arm_load_kernel_notify. arm_load_kernel now only registers the
> corresponding notifier.
>
> Machine files that do not support platform bus stay unchanged. Machine
> files willing to support dynamic sysbus devices must call arm_load_kernel
> before sysbus-fdt arm_register_platform_bus_fdt_creator to make sure
> dynamic sysbus device nodes are integrated in the dtb.

> --- a/include/hw/arm/arm.h
> +++ b/include/hw/arm/arm.h
> @@ -19,6 +19,16 @@ qemu_irq *armv7m_init(MemoryRegion *system_memory,
>                        int flash_size, int sram_size,
>                        const char *kernel_filename, const char *cpu_model);
>
> +/*
> + * struct used as a parameter of the arm_load_kernel machine init
> + * done notifier
> + */
> +typedef struct {
> +    ARMCPU *cpu; /* handle to the first cpu object */
> +    Notifier notifier; /* actual notifier */

This is an odd order -- why not put 'notifier' first?

> +} ArmLoadKernelNotifier;
> +
> +

This breaks building of the linux-user targets:

In file included from
/home/petmay01/linaro/qemu-from-laptop/qemu/target-arm/cpu.c:29:0:
/home/petmay01/linaro/qemu-from-laptop/qemu/include/hw/arm/arm.h:28:5:
error: unknown type name ‘Notifier’
     Notifier notifier; /* actual notifier */
     ^
make[1]: *** [target-arm/cpu.o] Error 1

Including "qemu/notify.h" in this file fixes it.

Also, stray blank line.

Otherwise looks OK. (It seems a slight shame to have to drag
notify.h into everything that includes arm.h when really the
notifier is just a detail of the implementation of boot.c,
but I can't see a nice way round that, so never mind.)

-- PMM

Re: [Qemu-devel] [PATCH v8 1/3] hw/arm/sysbus-fdt: helpers for platform bus nodes addition

[Peter Maydell]

On 5 January 2015 at 16:14, Eric Auger <eric.auger@linaro.org> wrote:
> This new C module will be used by ARM machine files to generate
> platform bus node and their dynamic sysbus device tree nodes.
>
> Dynamic sysbus device node addition is done in a machine init
> done notifier. arm_register_platform_bus_fdt_creator does the
> registration of this latter and is supposed to be called by
> ARM machine files that support platform bus and their dynamic
> sysbus. Addition of dynamic sysbus nodes is done only if the
> user did not provide any dtb.
>
> Signed-off-by: Alexander Graf <agraf@suse.de>
> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
> Reviewed-by: Alexander Graf <agraf@suse.de>
>
> ---
> v7 -> v8:
> add Reviewed-by from Alex and Shannon
>
> v6 -> v7:
> - revert indentation in add_fdt_node_functions
>
> v5 -> v6:
> - add_all_platform_bus_fdt_nodes is not a modify_dtb function anymore
> - it now takes a handle to an ARMPlatformBusFdtParams.
> - fdt pointer is checked in case this notifier is executed after the
>   one that executes the load_dtb (this latter deallocates the fdt pointer)
> - check of fdt_filename moved in here.
> - upgrade_dtb is removed
> - copyright aligned between .h and .c
>
> v4 -> v5:
> - change indentation in add_fdt_node_functions. Also becomes a
>   static const.
> - ARMPlatformBusFdtParams.system_params becomes a pointer to
>   a const ARMPlatformBusSystemParams
> - removes platform-bus.h second inclusion
>
> v3 -> v4:
> - dyn_sysbus_devtree.c renamed into sysbus-fdt.c
> - use new PlatformBusDevice object
> - the dtb upgrade is done through modify_dtb. Before the fdt
>   was recreated from scratch. When the user provided a dtb this
>   latter was overwritten which was not correct.
> - an array contains the association between device type names
>   and their node creation function
> - I must aknowledge I did not find any cleaner way to implement
>   a FDT_BUILDER interface, as suggested by Paolo. The class method
>   would need to be initialized somewhere and since it cannot
>   happen in the device itself - according to Alex & Peter comments -,
>   I don't see when I shall associate the device type and its
>   interface implementation.
>
> v2 -> v3:
> - add arm_ prefix
> - arm_sysbus_device_create_devtree becomes static
>
> v1 -> v2:
> - Code moved in an arch specific file to accomodate architecture
>   dependent specificities.
> - remove platform_bus_base from PlatformDevtreeData
>
> v1: code originally written by Alex Graf in e500.c and reused for
> ARM [Eric Auger]
> ---
>  hw/arm/Makefile.objs        |   1 +
>  hw/arm/sysbus-fdt.c         | 173 ++++++++++++++++++++++++++++++++++++++++++++
>  include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
>  3 files changed, 234 insertions(+)
>  create mode 100644 hw/arm/sysbus-fdt.c
>  create mode 100644 include/hw/arm/sysbus-fdt.h
>
> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
> index 6088e53..0cc63e1 100644
> --- a/hw/arm/Makefile.objs
> +++ b/hw/arm/Makefile.objs
> @@ -3,6 +3,7 @@ obj-$(CONFIG_DIGIC) += digic_boards.o
>  obj-y += integratorcp.o kzm.o mainstone.o musicpal.o nseries.o
>  obj-y += omap_sx1.o palm.o realview.o spitz.o stellaris.o
>  obj-y += tosa.o versatilepb.o vexpress.o virt.o xilinx_zynq.o z2.o
> +obj-y += sysbus-fdt.o
>
>  obj-y += armv7m.o exynos4210.o pxa2xx.o pxa2xx_gpio.o pxa2xx_pic.o
>  obj-$(CONFIG_DIGIC) += digic.o
> diff --git a/hw/arm/sysbus-fdt.c b/hw/arm/sysbus-fdt.c
> new file mode 100644
> index 0000000..a1dc0aa
> --- /dev/null
> +++ b/hw/arm/sysbus-fdt.c
> @@ -0,0 +1,173 @@
> +/*
> + * ARM Platform Bus device tree generation helpers
> + *
> + * Copyright (c) 2014 Linaro Limited
> + *
> + * Authors:
> + *  Alex Graf <agraf@suse.de>
> + *  Eric Auger <eric.auger@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2 or later, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +
> +#include "hw/arm/sysbus-fdt.h"
> +#include "qemu/error-report.h"
> +#include "sysemu/device_tree.h"
> +#include "hw/platform-bus.h"
> +#include "sysemu/sysemu.h"
> +
> +/*
> + * internal struct that contains the information to create dynamic
> + * sysbus device node
> + */
> +typedef struct PlatformBusFdtData {

FDT is an acronym, so "PlatformBusFDTData" would be nicer.
(Similar remarks apply for some other struct type names.)

> +    void *fdt; /* device tree handle */
> +    int irq_start; /* index of the first IRQ usable by platform bus devices */
> +    const char *pbus_node_name; /* name of the platform bus node */
> +    PlatformBusDevice *pbus;
> +} PlatformBusFdtData;
> +
> +/*
> + * struct used when calling the machine init done notifier
> + * that constructs the fdt nodes of platform bus devices
> + */
> +typedef struct PlatformBusFdtNotifierParams {
> +    ARMPlatformBusFdtParams *fdt_params;
> +    Notifier notifier;

Weird ordering again. If you put notifier first then you can do

   PlatformBusFdtNotifier *p = DO_UPCAST(PlatformBusFdtNotifier,
                                         notifier, notifier);

rather than raw container_of calls. (This is of course assuming
you see this as a type-of-notifier, which seems reasonable to
me...hence the tweak to the struct name.)

> +} PlatformBusFdtNotifierParams;
> +
> +/* struct that associates a device type name and a node creation function */
> +typedef struct NodeCreationPair {
> +    const char *typename;
> +    int (*add_fdt_node_fn)(SysBusDevice *sbdev, void *opaque);
> +} NodeCreationPair;
> +
> +/* list of supported dynamic sysbus devices */
> +static const NodeCreationPair add_fdt_node_functions[] = {
> +    {"", NULL}, /*last element*/

Missing spaces: /* last element */

> +};
> +
> +/**
> + * add_fdt_node - add the device tree node of a dynamic sysbus device
> + *
> + * @sbdev: handle to the sysbus device
> + * @opaque: handle to the PlatformBusFdtData
> + *
> + * Checks the sysbus type belongs to the list of device types that
> + * are dynamically instantiable and in the positive call the node

s/and in the positive/and if so/

> + * creation function.
> + */
> +static int add_fdt_node(SysBusDevice *sbdev, void *opaque)
> +{
> +    int i;
> +
> +    for (i = 0; i < ARRAY_SIZE(add_fdt_node_functions); i++) {
> +        if (!strcmp(object_get_typename(OBJECT(sbdev)),
> +                    add_fdt_node_functions[i].typename)) {
> +            add_fdt_node_functions[i].add_fdt_node_fn(sbdev, opaque);
> +            return 0;
> +        }
> +    }
> +    error_report("Device %s can not be dynamically instantiated",
> +                     qdev_fw_name(DEVICE(sbdev)));
> +    return 1;
> +}
> +
> +/**
> + * add_all_platform_bus_fdt_nodes - create all the platform bus nodes
> + *
> + * builds the parent platform bus node and all the nodes of dynamic
> + * sysbus devices attached to it.
> + */
> +static void add_all_platform_bus_fdt_nodes(ARMPlatformBusFdtParams *fdt_params)
> +{
> +    const char platcomp[] = "qemu,platform\0simple-bus";
> +    PlatformBusDevice *pbus;
> +    DeviceState *dev;
> +    gchar *node;
> +    uint64_t addr, size;
> +    int irq_start, dtb_size;
> +    struct arm_boot_info *info = fdt_params->binfo;
> +    const ARMPlatformBusSystemParams *params = fdt_params->system_params;
> +    const char *intc = fdt_params->intc;
> +    void *fdt = info->get_dtb(info, &dtb_size);
> +
> +    /*
> +     * In case the user provided a dtb, we assume he already integrated the

s/In case/If/. s/he/they/.

> +     * dynamic sysbus nodes. This corresponds to a use case where the dynamic
> +     * sysbus nodes are complex and their generation is not yet supported. In
> +     * case the use can take in charge the guest dt while qemu takes in charge
> +     * the qom stuff.

"can take charge of".

> +     */
> +    if (info->dtb_filename) {
> +        return;
> +    }
> +
> +    if (!fdt) {
> +        fprintf(stderr,
> +                "fdt NULL pointer: platform bus nodes cannot be added\n") ;
> +        return;
> +    }

When can this happen? If it can happen due to user command line error
or similar, the error message should be more useful to the user.
If it can only happen due to a bug in the board model (which I think
is the case), then just assert(fdt);

> +
> +    node = g_strdup_printf("/platform@%"PRIx64, params->platform_bus_base);
> +    addr = params->platform_bus_base;
> +    size = params->platform_bus_size;
> +    irq_start = params->platform_bus_first_irq;
> +
> +    /* Create a /platform node that we can put all devices into */
> +    qemu_fdt_add_subnode(fdt, node);
> +    qemu_fdt_setprop(fdt, node, "compatible", platcomp, sizeof(platcomp));
> +
> +    /* Our platform bus region is less than 32bit big, so 1 cell is enough for
> +       address and size */

"32 bits".
 /* multiline comments
  * should look like this
  */

> +    qemu_fdt_setprop_cells(fdt, node, "#size-cells", 1);
> +    qemu_fdt_setprop_cells(fdt, node, "#address-cells", 1);
> +    qemu_fdt_setprop_cells(fdt, node, "ranges", 0, addr >> 32, addr, size);
> +
> +    qemu_fdt_setprop_phandle(fdt, node, "interrupt-parent", intc);
> +
> +    dev = qdev_find_recursive(sysbus_get_default(), TYPE_PLATFORM_BUS_DEVICE);
> +    pbus = PLATFORM_BUS_DEVICE(dev);
> +
> +    /* We can only create dt nodes for dynamic devices when they're ready */
> +    if (pbus->done_gathering) {

Isn't it a bug if we get here with done_gathering false? If not,
then what arranges for us to get called again later?

> +        PlatformBusFdtData data = {
> +            .fdt = fdt,
> +            .irq_start = irq_start,
> +            .pbus_node_name = node,
> +            .pbus = pbus,
> +        };
> +
> +        /* Loop through all dynamic sysbus devices and create their node */
> +        foreach_dynamic_sysbus_device(add_fdt_node, &data);
> +    }
> +
> +    g_free(node);
> +}

Rest looks OK I think.

-- PMM

Re: [Qemu-devel] [PATCH v8 3/3] hw/arm/virt: add dynamic sysbus device support

[Peter Maydell]

On 5 January 2015 at 16:14, Eric Auger <eric.auger@linaro.org> wrote:
> Allows sysbus devices to be instantiated from command line by
> using -device option. Machvirt creates a platform bus at init.
> The dynamic sysbus devices are attached to this platform bus device.

> @@ -59,6 +61,8 @@
>  #define GIC_FDT_IRQ_PPI_CPU_START 8
>  #define GIC_FDT_IRQ_PPI_CPU_WIDTH 8
>
> +#define PLATFORM_BUS_NUM_IRQS     20
> +
>  enum {
>      VIRT_FLASH,
>      VIRT_MEM,
> @@ -69,8 +73,11 @@ enum {
>      VIRT_MMIO,
>      VIRT_RTC,
>      VIRT_FW_CFG,
> +    VIRT_PLATFORM_BUS,
>  };
>
> +static ARMPlatformBusSystemParams platform_bus_params;
> +
>  typedef struct MemMapEntry {
>      hwaddr base;
>      hwaddr size;
> @@ -119,24 +126,26 @@ typedef struct {
>   */
>  static const MemMapEntry a15memmap[] = {
>      /* Space up to 0x8000000 is reserved for a boot ROM */
> -    [VIRT_FLASH] =      {          0, 0x08000000 },
> -    [VIRT_CPUPERIPHS] = { 0x08000000, 0x00020000 },
> +    [VIRT_FLASH] =          {          0, 0x08000000 },
> +    [VIRT_CPUPERIPHS] =     { 0x08000000, 0x00020000 },
>      /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
> -    [VIRT_GIC_DIST] =   { 0x08000000, 0x00010000 },
> -    [VIRT_GIC_CPU] =    { 0x08010000, 0x00010000 },
> -    [VIRT_UART] =       { 0x09000000, 0x00001000 },
> -    [VIRT_RTC] =        { 0x09010000, 0x00001000 },
> -    [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
> -    [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
> +    [VIRT_GIC_DIST] =       { 0x08000000, 0x00010000 },
> +    [VIRT_GIC_CPU] =        { 0x08010000, 0x00010000 },
> +    [VIRT_UART] =           { 0x09000000, 0x00001000 },
> +    [VIRT_RTC] =            { 0x09010000, 0x00001000 },
> +    [VIRT_FW_CFG] =         { 0x09020000, 0x0000000a },

Please don't re-indent unrelated lines in the same patch: it makes
it very hard to tell whether any of them have actually changed.

> +    [VIRT_PLATFORM_BUS] =   { 0x09400000, 0x00400000 },
> +    [VIRT_MMIO] =           { 0x0a000000, 0x00000200 },
>      /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
>      /* 0x10000000 .. 0x40000000 reserved for PCI */
> -    [VIRT_MEM] =        { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
> +    [VIRT_MEM] =            { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
>  };
>
>  static const int a15irqmap[] = {
>      [VIRT_UART] = 1,
>      [VIRT_RTC] = 2,
>      [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */
> +    [VIRT_PLATFORM_BUS] = 48, /* .. to 48 + PLATFORM_BUS_NUM_IRQS -1*/

Missing spaces again (also use '...to' for consistency with line above).

>  };

This patch generally looks OK, so I think the major question is:
are we happy with this memory and IRQ usage?

Why 20 for PLATFORM_BUS_NUM_IRQS? It seems a funny number.
Starting the platform bus IRQs at 48 means there is no scope
at all for raising NUM_VIRTIO_TRANSPORTS later, which is
not really what I had in mind, though in fact it seems
unlikely that we'll ever really want to do that given the
imminent advent of PCI. Still, I don't think it would
hurt to start at 64.

>  static VirtBoardInfo machines[] = {
> @@ -556,6 +565,47 @@ static void create_fw_cfg(const VirtBoardInfo *vbi)
>      g_free(nodename);
>  }
>
> +static void create_platform_bus(VirtBoardInfo *vbi, qemu_irq *pic)
> +{
> +    DeviceState *dev;
> +    SysBusDevice *s;
> +    int i;
> +    ARMPlatformBusFdtParams *fdt_params = g_new(ARMPlatformBusFdtParams, 1);

...does the arm_register_platform_bus_fdt_creator() function
implicitly agree to g_free() the pointer it's passed when it's
done with it, or are we just leaking this?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v8 1/3] hw/arm/sysbus-fdt: helpers for platform bus nodes addition

[Eric Auger]

Hi Peter,
On 01/06/2015 07:40 PM, Peter Maydell wrote:
> On 5 January 2015 at 16:14, Eric Auger <eric.auger@linaro.org> wrote:
>> This new C module will be used by ARM machine files to generate
>> platform bus node and their dynamic sysbus device tree nodes.
>>
>> Dynamic sysbus device node addition is done in a machine init
>> done notifier. arm_register_platform_bus_fdt_creator does the
>> registration of this latter and is supposed to be called by
>> ARM machine files that support platform bus and their dynamic
>> sysbus. Addition of dynamic sysbus nodes is done only if the
>> user did not provide any dtb.
>>
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>> Signed-off-by: Eric Auger <eric.auger@linaro.org>
>> Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
>> Reviewed-by: Alexander Graf <agraf@suse.de>
>>
>> ---
>> v7 -> v8:
>> add Reviewed-by from Alex and Shannon
>>
>> v6 -> v7:
>> - revert indentation in add_fdt_node_functions
>>
>> v5 -> v6:
>> - add_all_platform_bus_fdt_nodes is not a modify_dtb function anymore
>> - it now takes a handle to an ARMPlatformBusFdtParams.
>> - fdt pointer is checked in case this notifier is executed after the
>>   one that executes the load_dtb (this latter deallocates the fdt pointer)
>> - check of fdt_filename moved in here.
>> - upgrade_dtb is removed
>> - copyright aligned between .h and .c
>>
>> v4 -> v5:
>> - change indentation in add_fdt_node_functions. Also becomes a
>>   static const.
>> - ARMPlatformBusFdtParams.system_params becomes a pointer to
>>   a const ARMPlatformBusSystemParams
>> - removes platform-bus.h second inclusion
>>
>> v3 -> v4:
>> - dyn_sysbus_devtree.c renamed into sysbus-fdt.c
>> - use new PlatformBusDevice object
>> - the dtb upgrade is done through modify_dtb. Before the fdt
>>   was recreated from scratch. When the user provided a dtb this
>>   latter was overwritten which was not correct.
>> - an array contains the association between device type names
>>   and their node creation function
>> - I must aknowledge I did not find any cleaner way to implement
>>   a FDT_BUILDER interface, as suggested by Paolo. The class method
>>   would need to be initialized somewhere and since it cannot
>>   happen in the device itself - according to Alex & Peter comments -,
>>   I don't see when I shall associate the device type and its
>>   interface implementation.
>>
>> v2 -> v3:
>> - add arm_ prefix
>> - arm_sysbus_device_create_devtree becomes static
>>
>> v1 -> v2:
>> - Code moved in an arch specific file to accomodate architecture
>>   dependent specificities.
>> - remove platform_bus_base from PlatformDevtreeData
>>
>> v1: code originally written by Alex Graf in e500.c and reused for
>> ARM [Eric Auger]
>> ---
>>  hw/arm/Makefile.objs        |   1 +
>>  hw/arm/sysbus-fdt.c         | 173 ++++++++++++++++++++++++++++++++++++++++++++
>>  include/hw/arm/sysbus-fdt.h |  60 +++++++++++++++
>>  3 files changed, 234 insertions(+)
>>  create mode 100644 hw/arm/sysbus-fdt.c
>>  create mode 100644 include/hw/arm/sysbus-fdt.h
>>
>> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
>> index 6088e53..0cc63e1 100644
>> --- a/hw/arm/Makefile.objs
>> +++ b/hw/arm/Makefile.objs
>> @@ -3,6 +3,7 @@ obj-$(CONFIG_DIGIC) += digic_boards.o
>>  obj-y += integratorcp.o kzm.o mainstone.o musicpal.o nseries.o
>>  obj-y += omap_sx1.o palm.o realview.o spitz.o stellaris.o
>>  obj-y += tosa.o versatilepb.o vexpress.o virt.o xilinx_zynq.o z2.o
>> +obj-y += sysbus-fdt.o
>>
>>  obj-y += armv7m.o exynos4210.o pxa2xx.o pxa2xx_gpio.o pxa2xx_pic.o
>>  obj-$(CONFIG_DIGIC) += digic.o
>> diff --git a/hw/arm/sysbus-fdt.c b/hw/arm/sysbus-fdt.c
>> new file mode 100644
>> index 0000000..a1dc0aa
>> --- /dev/null
>> +++ b/hw/arm/sysbus-fdt.c
>> @@ -0,0 +1,173 @@
>> +/*
>> + * ARM Platform Bus device tree generation helpers
>> + *
>> + * Copyright (c) 2014 Linaro Limited
>> + *
>> + * Authors:
>> + *  Alex Graf <agraf@suse.de>
>> + *  Eric Auger <eric.auger@linaro.org>
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms and conditions of the GNU General Public License,
>> + * version 2 or later, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
>> + * more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along with
>> + * this program.  If not, see <http://www.gnu.org/licenses/>.
>> + *
>> + */
>> +
>> +#include "hw/arm/sysbus-fdt.h"
>> +#include "qemu/error-report.h"
>> +#include "sysemu/device_tree.h"
>> +#include "hw/platform-bus.h"
>> +#include "sysemu/sysemu.h"
>> +
>> +/*
>> + * internal struct that contains the information to create dynamic
>> + * sysbus device node
>> + */
>> +typedef struct PlatformBusFdtData {
> 
> FDT is an acronym, so "PlatformBusFDTData" would be nicer.
> (Similar remarks apply for some other struct type names.)
Sure
> 
>> +    void *fdt; /* device tree handle */
>> +    int irq_start; /* index of the first IRQ usable by platform bus devices */
>> +    const char *pbus_node_name; /* name of the platform bus node */
>> +    PlatformBusDevice *pbus;
>> +} PlatformBusFdtData;
>> +
>> +/*
>> + * struct used when calling the machine init done notifier
>> + * that constructs the fdt nodes of platform bus devices
>> + */
>> +typedef struct PlatformBusFdtNotifierParams {
>> +    ARMPlatformBusFdtParams *fdt_params;
>> +    Notifier notifier;
> 
> Weird ordering again. If you put notifier first then you can do
> 
>    PlatformBusFdtNotifier *p = DO_UPCAST(PlatformBusFdtNotifier,
>                                          notifier, notifier);
> 
> rather than raw container_of calls. (This is of course assuming
> you see this as a type-of-notifier, which seems reasonable to
> me...hence the tweak to the struct name.)
Yes it is the case, I will correct the order.
> 
>> +} PlatformBusFdtNotifierParams;
>> +
>> +/* struct that associates a device type name and a node creation function */
>> +typedef struct NodeCreationPair {
>> +    const char *typename;
>> +    int (*add_fdt_node_fn)(SysBusDevice *sbdev, void *opaque);
>> +} NodeCreationPair;
>> +
>> +/* list of supported dynamic sysbus devices */
>> +static const NodeCreationPair add_fdt_node_functions[] = {
>> +    {"", NULL}, /*last element*/
> 
> Missing spaces: /* last element */
sure
> 
>> +};
>> +
>> +/**
>> + * add_fdt_node - add the device tree node of a dynamic sysbus device
>> + *
>> + * @sbdev: handle to the sysbus device
>> + * @opaque: handle to the PlatformBusFdtData
>> + *
>> + * Checks the sysbus type belongs to the list of device types that
>> + * are dynamically instantiable and in the positive call the node
> 
> s/and in the positive/and if so/
thanks
> 
>> + * creation function.
>> + */
>> +static int add_fdt_node(SysBusDevice *sbdev, void *opaque)
>> +{
>> +    int i;
>> +
>> +    for (i = 0; i < ARRAY_SIZE(add_fdt_node_functions); i++) {
>> +        if (!strcmp(object_get_typename(OBJECT(sbdev)),
>> +                    add_fdt_node_functions[i].typename)) {
>> +            add_fdt_node_functions[i].add_fdt_node_fn(sbdev, opaque);
>> +            return 0;
>> +        }
>> +    }
>> +    error_report("Device %s can not be dynamically instantiated",
>> +                     qdev_fw_name(DEVICE(sbdev)));
>> +    return 1;
>> +}
>> +
>> +/**
>> + * add_all_platform_bus_fdt_nodes - create all the platform bus nodes
>> + *
>> + * builds the parent platform bus node and all the nodes of dynamic
>> + * sysbus devices attached to it.
>> + */
>> +static void add_all_platform_bus_fdt_nodes(ARMPlatformBusFdtParams *fdt_params)
>> +{
>> +    const char platcomp[] = "qemu,platform\0simple-bus";
>> +    PlatformBusDevice *pbus;
>> +    DeviceState *dev;
>> +    gchar *node;
>> +    uint64_t addr, size;
>> +    int irq_start, dtb_size;
>> +    struct arm_boot_info *info = fdt_params->binfo;
>> +    const ARMPlatformBusSystemParams *params = fdt_params->system_params;
>> +    const char *intc = fdt_params->intc;
>> +    void *fdt = info->get_dtb(info, &dtb_size);
>> +
>> +    /*
>> +     * In case the user provided a dtb, we assume he already integrated the
> 
> s/In case/If/. s/he/they/.
OK
> 
>> +     * dynamic sysbus nodes. This corresponds to a use case where the dynamic
>> +     * sysbus nodes are complex and their generation is not yet supported. In
>> +     * case the use can take in charge the guest dt while qemu takes in charge
>> +     * the qom stuff.
> 
> "can take charge of".
thanks
> 
>> +     */
>> +    if (info->dtb_filename) {
>> +        return;
>> +    }
>> +
>> +    if (!fdt) {
>> +        fprintf(stderr,
>> +                "fdt NULL pointer: platform bus nodes cannot be added\n") ;
>> +        return;
>> +    }
> 
> When can this happen? If it can happen due to user command line error
> or similar, the error message should be more useful to the user.
> If it can only happen due to a bug in the board model (which I think
> is the case), then just assert(fdt);
Indeed this was meant to warn about a bug in board model. I will use
assert then.
> 
>> +
>> +    node = g_strdup_printf("/platform@%"PRIx64, params->platform_bus_base);
>> +    addr = params->platform_bus_base;
>> +    size = params->platform_bus_size;
>> +    irq_start = params->platform_bus_first_irq;
>> +
>> +    /* Create a /platform node that we can put all devices into */
>> +    qemu_fdt_add_subnode(fdt, node);
>> +    qemu_fdt_setprop(fdt, node, "compatible", platcomp, sizeof(platcomp));
>> +
>> +    /* Our platform bus region is less than 32bit big, so 1 cell is enough for
>> +       address and size */
> 
> "32 bits".
>  /* multiline comments
>   * should look like this
>   */
OK
> 
>> +    qemu_fdt_setprop_cells(fdt, node, "#size-cells", 1);
>> +    qemu_fdt_setprop_cells(fdt, node, "#address-cells", 1);
>> +    qemu_fdt_setprop_cells(fdt, node, "ranges", 0, addr >> 32, addr, size);
>> +
>> +    qemu_fdt_setprop_phandle(fdt, node, "interrupt-parent", intc);
>> +
>> +    dev = qdev_find_recursive(sysbus_get_default(), TYPE_PLATFORM_BUS_DEVICE);
>> +    pbus = PLATFORM_BUS_DEVICE(dev);
>> +
>> +    /* We can only create dt nodes for dynamic devices when they're ready */
>> +    if (pbus->done_gathering) {
> 
> Isn't it a bug if we get here with done_gathering false? If not,
> then what arranges for us to get called again later?
Yes it would be bug in the board model. The "gathering" is done in the
notifier registered in platform_bus_realize while this function is
executed in the notifier registered in create_platform_bus, when calling
arm_register_platform_bus_fdt_creator. create_platform_bus takes care of
registering the fdt creator notifier first and then instantiating the
platform bus.

I will put an assert then.

Thanks for the review!

Best Regards

Eric
> 
>> +        PlatformBusFdtData data = {
>> +            .fdt = fdt,
>> +            .irq_start = irq_start,
>> +            .pbus_node_name = node,
>> +            .pbus = pbus,
>> +        };
>> +
>> +        /* Loop through all dynamic sysbus devices and create their node */
>> +        foreach_dynamic_sysbus_device(add_fdt_node, &data);
>> +    }
>> +
>> +    g_free(node);
>> +}
> 
> Rest looks OK I think.
> 
> -- PMM
> 

Re: [Qemu-devel] [PATCH v8 3/3] hw/arm/virt: add dynamic sysbus device support

[Eric Auger]

Hi Peter,
On 01/06/2015 07:57 PM, Peter Maydell wrote:
> On 5 January 2015 at 16:14, Eric Auger <eric.auger@linaro.org> wrote:
>> Allows sysbus devices to be instantiated from command line by
>> using -device option. Machvirt creates a platform bus at init.
>> The dynamic sysbus devices are attached to this platform bus device.
> 
>> @@ -59,6 +61,8 @@
>>  #define GIC_FDT_IRQ_PPI_CPU_START 8
>>  #define GIC_FDT_IRQ_PPI_CPU_WIDTH 8
>>
>> +#define PLATFORM_BUS_NUM_IRQS     20
>> +
>>  enum {
>>      VIRT_FLASH,
>>      VIRT_MEM,
>> @@ -69,8 +73,11 @@ enum {
>>      VIRT_MMIO,
>>      VIRT_RTC,
>>      VIRT_FW_CFG,
>> +    VIRT_PLATFORM_BUS,
>>  };
>>
>> +static ARMPlatformBusSystemParams platform_bus_params;
>> +
>>  typedef struct MemMapEntry {
>>      hwaddr base;
>>      hwaddr size;
>> @@ -119,24 +126,26 @@ typedef struct {
>>   */
>>  static const MemMapEntry a15memmap[] = {
>>      /* Space up to 0x8000000 is reserved for a boot ROM */
>> -    [VIRT_FLASH] =      {          0, 0x08000000 },
>> -    [VIRT_CPUPERIPHS] = { 0x08000000, 0x00020000 },
>> +    [VIRT_FLASH] =          {          0, 0x08000000 },
>> +    [VIRT_CPUPERIPHS] =     { 0x08000000, 0x00020000 },
>>      /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
>> -    [VIRT_GIC_DIST] =   { 0x08000000, 0x00010000 },
>> -    [VIRT_GIC_CPU] =    { 0x08010000, 0x00010000 },
>> -    [VIRT_UART] =       { 0x09000000, 0x00001000 },
>> -    [VIRT_RTC] =        { 0x09010000, 0x00001000 },
>> -    [VIRT_FW_CFG] =     { 0x09020000, 0x0000000a },
>> -    [VIRT_MMIO] =       { 0x0a000000, 0x00000200 },
>> +    [VIRT_GIC_DIST] =       { 0x08000000, 0x00010000 },
>> +    [VIRT_GIC_CPU] =        { 0x08010000, 0x00010000 },
>> +    [VIRT_UART] =           { 0x09000000, 0x00001000 },
>> +    [VIRT_RTC] =            { 0x09010000, 0x00001000 },
>> +    [VIRT_FW_CFG] =         { 0x09020000, 0x0000000a },
> 
> Please don't re-indent unrelated lines in the same patch: it makes
> it very hard to tell whether any of them have actually changed.
sure I will create another patch file dedicated to that
> 
>> +    [VIRT_PLATFORM_BUS] =   { 0x09400000, 0x00400000 },
>> +    [VIRT_MMIO] =           { 0x0a000000, 0x00000200 },
>>      /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */
>>      /* 0x10000000 .. 0x40000000 reserved for PCI */
>> -    [VIRT_MEM] =        { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
>> +    [VIRT_MEM] =            { 0x40000000, 30ULL * 1024 * 1024 * 1024 },
>>  };
>>
>>  static const int a15irqmap[] = {
>>      [VIRT_UART] = 1,
>>      [VIRT_RTC] = 2,
>>      [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */
>> +    [VIRT_PLATFORM_BUS] = 48, /* .. to 48 + PLATFORM_BUS_NUM_IRQS -1*/
> 
> Missing spaces again (also use '...to' for consistency with line above).
> 
>>  };
> 
> This patch generally looks OK, so I think the major question is:
> are we happy with this memory and IRQ usage?
> 
> Why 20 for PLATFORM_BUS_NUM_IRQS? It seems a funny number.
> Starting the platform bus IRQs at 48 means there is no scope
> at all for raising NUM_VIRTIO_TRANSPORTS later, which is
> not really what I had in mind, though in fact it seems
> unlikely that we'll ever really want to do that given the
> imminent advent of PCI. Still, I don't think it would
> hurt to start at 64.
Then I will start at 64.

About PLATFORM_BUS_NUM_IRQS= 20, I must acknowledge it is an arbitrary
figure. Alex used 10 in e500 as default value.

If I am not wrong a DMA330 can have up to 32 IRQ. It is another
candidate for VFIO. Would it be OK to use 32 instead of 20?

About memory, here we have space for 64 64kB pages. any other suggestions?

>  we have
>>  static VirtBoardInfo machines[] = {
>> @@ -556,6 +565,47 @@ static void create_fw_cfg(const VirtBoardInfo *vbi)
>>      g_free(nodename);
>>  }
>>
>> +static void create_platform_bus(VirtBoardInfo *vbi, qemu_irq *pic)
>> +{
>> +    DeviceState *dev;
>> +    SysBusDevice *s;
>> +    int i;
>> +    ARMPlatformBusFdtParams *fdt_params = g_new(ARMPlatformBusFdtParams, 1);
> 
> ...does the arm_register_platform_bus_fdt_creator() function
> implicitly agree to g_free() the pointer it's passed when it's
> done with it, or are we just leaking this?
Well I could deallocate in platform_bus_fdt_notify routine if you
recommend to do so.

Thanks for the review.

Eric
> 
> thanks
> -- PMM
>