[Qemu-devel] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Qemu-devel] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Greg Kurz]

This series is based on patches from Michel Roth posted in 2015:

https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg04246.html

It addresses comments made during the RFC review, and also provides a bunch
of preliminary cleanup/fix patches. Since we have reached hard-freeze, this
feature is provided by a new pseries-2.11 machine type, introduced by this
series. It is based on David Gibson's ppc-for-2.10 branch, and I believe some
of the preliminary fixes are eligible for 2.10.

Note that it also requires an updated SLOF that supports a new private hcall:
KVMPPC_H_UPDATE_PHANDLE. This is needed because SLOF changes FDT phandles to
Open Firmware phandles. Since the guest only sees the latter, QEMU must use
the updated value when populating the FDT for the hotplugged PHB (otherwise
the guest can't setup IRQs for the PCI devices). SLOF part is already upstream:

http://git.qemu.org/?p=SLOF.git;h=604d28cc3f791657414f8b21103921fa0147fc63

With these patches we support the following:

(qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
(qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
(qemu) device_del hp2.0
(qemu) device_del phb2

I could run some successful tests with a fedora25 guest:
- hotplug PHB + migrate + unplug PHB
- hotplug PHB + hotplug PCI device + unplug PHB => PCI device gets unplugged
- migrate before OS starts + hotplug PHB => destination uses OF phandles
- no regression observed with older machine types

All the patches are also available here:

https://github.com/gkurz/qemu/commits/spapr-hotplug-phb

Cheers,

--
Greg

---

Greg Kurz (14):
      spapr: move spapr_create_phb() to core machine code
      spapr_pci: use memory_region_add_subregion() with DMA windows
      spapr_iommu: use g_strdup_printf() instead of snprintf()
      spapr_drc: use g_strdup_printf() instead of snprintf()
      spapr_iommu: convert TCE table object to realize()
      spapr_pci: parent the MSI memory region to the PHB
      spapr_drc: fix realize and unrealize
      spapr_drc: add unrealize method to physical DRC class
      spapr_iommu: unregister vmstate at unrealize time
      spapr: add pseries-2.11 machine type
      spapr_pci: introduce drc_id property
      spapr: allow guest to update the XICS phandle
      spapr_pci: drop abusive sanity check when migrating the LSI table
      spapr: add hotplug hooks for PHB hotplug

Michael Roth (11):
      spapr_drc: pass object ownership to parent/owner
      spapr_iommu: pass object ownership to parent/owner
      pci: allow cleanup/unregistration of PCI buses
      qdev: store DeviceState's canonical path to use when unparenting
      spapr_pci: add PHB unrealize
      spapr: enable PHB hotplug for pseries-2.11
      spapr: create DR connectors for PHBs
      spapr_events: add support for phb hotplug events
      qdev: pass an Object * to qbus_set_hotplug_handler()
      spapr_pci: provide node start offset via spapr_populate_pci_dt()
      spapr_pci: add ibm, my-drc-index property for PHB hotplug

Nathan Fontenot (1):
      spapr: populate PHB DRC entries for root DT node


 hw/acpi/piix4.c               |    2 
 hw/char/virtio-serial-bus.c   |    2 
 hw/core/bus.c                 |   11 --
 hw/core/qdev.c                |   15 ++-
 hw/pci/pci.c                  |   33 +++++++
 hw/pci/pcie.c                 |    2 
 hw/pci/shpc.c                 |    2 
 hw/ppc/spapr.c                |  205 ++++++++++++++++++++++++++++++++++++++++-
 hw/ppc/spapr_drc.c            |   65 ++++++++++---
 hw/ppc/spapr_events.c         |    3 +
 hw/ppc/spapr_hcall.c          |   20 ++++
 hw/ppc/spapr_iommu.c          |   22 +++-
 hw/ppc/spapr_pci.c            |   86 +++++++++++++----
 hw/s390x/css-bridge.c         |    2 
 hw/s390x/s390-pci-bus.c       |    6 +
 hw/scsi/virtio-scsi.c         |    2 
 hw/scsi/vmw_pvscsi.c          |    2 
 hw/usb/dev-smartcard-reader.c |    2 
 include/hw/compat.h           |    3 +
 include/hw/pci-host/spapr.h   |    9 +-
 include/hw/pci/pci.h          |    3 +
 include/hw/ppc/spapr.h        |   15 +++
 include/hw/ppc/spapr_drc.h    |    8 ++
 include/hw/qdev-core.h        |    4 -
 24 files changed, 446 insertions(+), 78 deletions(-)

[Qemu-devel] [for-2.11 PATCH 01/26] spapr: move spapr_create_phb() to core machine code

[Greg Kurz]

This function is only used when creating the default PHB. Let's rename
it and move it to the core machine code for clarity.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c              |   13 ++++++++++++-
 hw/ppc/spapr_pci.c          |   11 -----------
 include/hw/pci-host/spapr.h |    2 --
 3 files changed, 12 insertions(+), 14 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2a3e53d5d58c..53969315ac24 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2142,6 +2142,17 @@ static void spapr_init_cpus(sPAPRMachineState *spapr)
     g_free(type);
 }
 
+static PCIHostState *spapr_create_default_phb(sPAPRMachineState *spapr)
+{
+    DeviceState *dev;
+
+    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
+    qdev_prop_set_uint32(dev, "index", 0);
+    qdev_init_nofail(dev);
+
+    return PCI_HOST_BRIDGE(dev);
+}
+
 /* pSeries LPAR / sPAPR hardware init */
 static void ppc_spapr_init(MachineState *machine)
 {
@@ -2374,7 +2385,7 @@ static void ppc_spapr_init(MachineState *machine)
     /* Set up PCI */
     spapr_pci_rtas_init();
 
-    phb = spapr_create_phb(spapr, 0);
+    phb = spapr_create_default_phb(spapr);
 
     for (i = 0; i < nb_nics; i++) {
         NICInfo *nd = &nd_table[i];
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index d84abf1070a0..70a99a032267 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1982,17 +1982,6 @@ static const TypeInfo spapr_phb_info = {
     }
 };
 
-PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index)
-{
-    DeviceState *dev;
-
-    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
-    qdev_prop_set_uint32(dev, "index", index);
-    qdev_init_nofail(dev);
-
-    return PCI_HOST_BRIDGE(dev);
-}
-
 typedef struct sPAPRFDT {
     void *fdt;
     int node_off;
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 38470b2f0e5c..5a4e9686d562 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -111,8 +111,6 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
     return xics_get_qirq(XICS_FABRIC(spapr), phb->lsi_table[pin].irq);
 }
 
-PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index);
-
 int spapr_populate_pci_dt(sPAPRPHBState *phb,
                           uint32_t xics_phandle,
                           void *fdt);

[Qemu-devel] [for-2.11 PATCH 02/26] spapr_pci: use memory_region_add_subregion() with DMA windows

[Greg Kurz]

Passing a null priority to memory_region_add_subregion_overlap() is
strictly equivalent to calling memory_region_add_subregion().

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_pci.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 70a99a032267..3fe7f3145467 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1752,8 +1752,8 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
                        i, sphb->dtbusname);
             return;
         }
-        memory_region_add_subregion_overlap(&sphb->iommu_root, 0,
-                                            spapr_tce_get_iommu(tcet), 0);
+        memory_region_add_subregion(&sphb->iommu_root, 0,
+                                    spapr_tce_get_iommu(tcet));
     }
 
     sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);

[Qemu-devel] [for-2.11 PATCH 03/26] spapr_iommu: use g_strdup_printf() instead of snprintf()

[Greg Kurz]

Passing a stack allocated buffer of arbitrary length to snprintf()
without checking the return value can cause the resultant strings
to be silently truncated.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_iommu.c |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index e614621a8317..740d42608b61 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -252,17 +252,19 @@ static int spapr_tce_table_realize(DeviceState *dev)
 {
     sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
     Object *tcetobj = OBJECT(tcet);
-    char tmp[32];
+    gchar *tmp;
 
     tcet->fd = -1;
     tcet->need_vfio = false;
-    snprintf(tmp, sizeof(tmp), "tce-root-%x", tcet->liobn);
+    tmp = g_strdup_printf("tce-root-%x", tcet->liobn);
     memory_region_init(&tcet->root, tcetobj, tmp, UINT64_MAX);
+    g_free(tmp);
 
-    snprintf(tmp, sizeof(tmp), "tce-iommu-%x", tcet->liobn);
+    tmp = g_strdup_printf("tce-iommu-%x", tcet->liobn);
     memory_region_init_iommu(&tcet->iommu, sizeof(tcet->iommu),
                              TYPE_SPAPR_IOMMU_MEMORY_REGION,
                              tcetobj, tmp, 0);
+    g_free(tmp);
 
     QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list);
 
@@ -307,7 +309,7 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
 sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
 {
     sPAPRTCETable *tcet;
-    char tmp[32];
+    gchar *tmp;
 
     if (spapr_tce_find_by_liobn(liobn)) {
         error_report("Attempted to create TCE table with duplicate"
@@ -318,8 +320,9 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
     tcet = SPAPR_TCE_TABLE(object_new(TYPE_SPAPR_TCE_TABLE));
     tcet->liobn = liobn;
 
-    snprintf(tmp, sizeof(tmp), "tce-table-%x", liobn);
+    tmp = g_strdup_printf("tce-table-%x", liobn);
     object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
+    g_free(tmp);
 
     object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
 

[Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[Greg Kurz]

Passing a stack allocated buffer of arbitrary length to snprintf()
without checking the return value can cause the resultant strings
to be silently truncated.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_drc.c |   15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 15bae5c216a9..e4e8383ec7b5 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -488,7 +488,7 @@ static void realize(DeviceState *d, Error **errp)
 {
     sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
     Object *root_container;
-    char link_name[256];
+    gchar *link_name;
     gchar *child_name;
     Error *err = NULL;
 
@@ -501,11 +501,12 @@ static void realize(DeviceState *d, Error **errp)
      * existing in the composition tree
      */
     root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
-    snprintf(link_name, sizeof(link_name), "%x", spapr_drc_index(drc));
+    link_name = g_strdup_printf("%x", spapr_drc_index(drc));
     child_name = object_get_canonical_path_component(OBJECT(drc));
     trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
     object_property_add_alias(root_container, link_name,
                               drc->owner, child_name, &err);
+    g_free(link_name);
     if (err) {
         error_report_err(err);
         object_unref(OBJECT(drc));
@@ -521,13 +522,14 @@ static void unrealize(DeviceState *d, Error **errp)
 {
     sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
     Object *root_container;
-    char name[256];
+    gchar *name;
     Error *err = NULL;
 
     trace_spapr_drc_unrealize(spapr_drc_index(drc));
     root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
-    snprintf(name, sizeof(name), "%x", spapr_drc_index(drc));
+    name = g_strdup_printf("%x", spapr_drc_index(drc));
     object_property_del(root_container, name, &err);
+    g_free(name);
     if (err) {
         error_report_err(err);
         object_unref(OBJECT(drc));
@@ -729,10 +731,11 @@ static const TypeInfo spapr_drc_lmb_info = {
 sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
 {
     Object *obj;
-    char name[256];
+    gchar *name;
 
-    snprintf(name, sizeof(name), "%s/%x", DRC_CONTAINER_PATH, index);
+    name = g_strdup_printf("%s/%x", DRC_CONTAINER_PATH, index);
     obj = object_resolve_path(name, NULL);
+    g_free(name);
 
     return !obj ? NULL : SPAPR_DR_CONNECTOR(obj);
 }

[Qemu-devel] [for-2.11 PATCH 05/26] spapr_iommu: convert TCE table object to realize()

[Greg Kurz]

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_iommu.c |    6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index 740d42608b61..d4eca2a77225 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -248,7 +248,7 @@ static const VMStateDescription vmstate_spapr_tce_table = {
     }
 };
 
-static int spapr_tce_table_realize(DeviceState *dev)
+static void spapr_tce_table_realize(DeviceState *dev, Error **errp)
 {
     sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
     Object *tcetobj = OBJECT(tcet);
@@ -270,8 +270,6 @@ static int spapr_tce_table_realize(DeviceState *dev)
 
     vmstate_register(DEVICE(tcet), tcet->liobn, &vmstate_spapr_tce_table,
                      tcet);
-
-    return 0;
 }
 
 void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
@@ -618,7 +616,7 @@ int spapr_tcet_dma_dt(void *fdt, int node_off, const char *propname,
 static void spapr_tce_table_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
-    dc->init = spapr_tce_table_realize;
+    dc->realize = spapr_tce_table_realize;
     dc->reset = spapr_tce_reset;
     dc->unrealize = spapr_tce_table_unrealize;
 

[Qemu-devel] [for-2.11 PATCH 06/26] spapr_pci: parent the MSI memory region to the PHB

[Greg Kurz]

This memory region should be owned by the PHB. This ensures the PHB
cannot be finalized as long as the the region is guest visible, or
used by a CPU or a device.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_pci.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 3fe7f3145467..4e4165b44b9a 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1703,7 +1703,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
     }
 #endif
 
-    memory_region_init_io(&sphb->msiwindow, NULL, &spapr_msi_ops, spapr,
+    memory_region_init_io(&sphb->msiwindow, OBJECT(sphb), &spapr_msi_ops, spapr,
                           "msi", msi_window_size);
     memory_region_add_subregion(&sphb->iommu_root, SPAPR_PCI_MSI_WINDOW,
                                 &sphb->msiwindow);

[Qemu-devel] [for-2.11 PATCH 07/26] spapr_drc: fix realize and unrealize

[Greg Kurz]

If object_property_add_alias() returns an error in realize(), we should
propagate it to the caller and certainly not unref the DRC.

Same thing goes for unrealize(). Since object_property_del() is the last
call, we can even get rid of the intermediate Error *.

And finally, unrealize() should undo all registrations performed by
realize().

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_drc.c |   15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index e4e8383ec7b5..d72453bcb42f 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -506,12 +506,12 @@ static void realize(DeviceState *d, Error **errp)
     trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
     object_property_add_alias(root_container, link_name,
                               drc->owner, child_name, &err);
+    g_free(child_name);
     g_free(link_name);
     if (err) {
-        error_report_err(err);
-        object_unref(OBJECT(drc));
+        error_propagate(errp, err);
+        return;
     }
-    g_free(child_name);
     vmstate_register(DEVICE(drc), spapr_drc_index(drc), &vmstate_spapr_drc,
                      drc);
     qemu_register_reset(drc_reset, drc);
@@ -523,17 +523,14 @@ static void unrealize(DeviceState *d, Error **errp)
     sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
     Object *root_container;
     gchar *name;
-    Error *err = NULL;
 
     trace_spapr_drc_unrealize(spapr_drc_index(drc));
+    qemu_unregister_reset(drc_reset, drc);
+    vmstate_unregister(DEVICE(drc), &vmstate_spapr_drc, drc);
     root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
     name = g_strdup_printf("%x", spapr_drc_index(drc));
-    object_property_del(root_container, name, &err);
+    object_property_del(root_container, name, errp);
     g_free(name);
-    if (err) {
-        error_report_err(err);
-        object_unref(OBJECT(drc));
-    }
 }
 
 sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,

[Qemu-devel] [for-2.11 PATCH 08/26] spapr_drc: add unrealize method to physical DRC class

[Greg Kurz]

When hot-unplugging a PHB, all its PCI DRC connectors get unrealized. This
patch adds an unrealize method to the physical DRC class, in order to undo
registrations performed in realize_physical().

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_drc.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index d72453bcb42f..105ce85b6d04 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -626,12 +626,28 @@ static void realize_physical(DeviceState *d, Error **errp)
     qemu_register_reset(drc_physical_reset, drcp);
 }
 
+static void unrealize_physical(DeviceState *d, Error **errp)
+{
+    sPAPRDRCPhysical *drcp = SPAPR_DRC_PHYSICAL(d);
+    Error *local_err = NULL;
+
+    unrealize(d, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    vmstate_unregister(DEVICE(drcp), &vmstate_spapr_drc_physical, drcp);
+    qemu_unregister_reset(drc_physical_reset, drcp);
+}
+
 static void spapr_drc_physical_class_init(ObjectClass *k, void *data)
 {
     DeviceClass *dk = DEVICE_CLASS(k);
     sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
 
     dk->realize = realize_physical;
+    dk->unrealize = unrealize_physical;
     drck->dr_entity_sense = physical_entity_sense;
     drck->isolate = drc_isolate_physical;
     drck->unisolate = drc_unisolate_physical;

[Qemu-devel] [for-2.11 PATCH 09/26] spapr_drc: pass object ownership to parent/owner

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

DRC objects attach themselves to an owner as a child
property. unref afterward to allow them to be finalized
when their owner is finalized.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
---
 hw/ppc/spapr_drc.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 105ce85b6d04..eb8024d37c54 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -544,6 +544,7 @@ sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
     prop_name = g_strdup_printf("dr-connector[%"PRIu32"]",
                                 spapr_drc_index(drc));
     object_property_add_child(owner, prop_name, OBJECT(drc), NULL);
+    object_unref(OBJECT(drc));
     object_property_set_bool(OBJECT(drc), true, "realized", NULL);
     g_free(prop_name);
 

[Qemu-devel] [for-2.11 PATCH 10/26] spapr_iommu: pass object ownership to parent/owner

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

TCE table objects attach themselves to an owner as a child
property. unref afterward to allow them to be finalized
when their owner is finalized.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- s/DRC/TCE table/ in changelog
---
 hw/ppc/spapr_iommu.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index d4eca2a77225..8ec4e7a40834 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -321,6 +321,7 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
     tmp = g_strdup_printf("tce-table-%x", liobn);
     object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
     g_free(tmp);
+    object_unref(OBJECT(tcet));
 
     object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
 

[Qemu-devel] [for-2.11 PATCH 11/26] spapr_iommu: unregister vmstate at unrealize time

[Greg Kurz]

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_iommu.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index 8ec4e7a40834..40a44c563824 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -374,6 +374,8 @@ static void spapr_tce_table_unrealize(DeviceState *dev, Error **errp)
 {
     sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
 
+    vmstate_unregister(DEVICE(tcet), &vmstate_spapr_tce_table, tcet);
+
     QLIST_REMOVE(tcet, list);
 
     spapr_tce_table_disable(tcet);

[Qemu-devel] [for-2.11 PATCH 12/26] pci: allow cleanup/unregistration of PCI buses

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

This adds cleanup counterparts to pci_register_bus(),
pci_bus_new(), and pci_bus_irqs().

These cleanup routines are needed in the case of hotpluggable
PCIHostBridge implementations. Currently we can rely on the
object_unparent()'ing of the PCIHostState recursively unparenting
and cleaning up it's child buses, but we need explicit calls
to also:

  1) remove the PCIHostState from pci_host_bridges global list.
     otherwise, we risk accessing freed memory when we access
     the list later
  2) clean up memory allocated in pci_bus_irqs()

Both are handled outside the context of any particular bus or
host bridge's init/realize functions, making it difficult to
avoid the need for explicit cleanup functions without remodeling
how PCIHostBridges are created. So keep it simple and just add
them for now.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- renamed arg of pci_host_bus_register() to host for consistency
---
 hw/pci/pci.c         |   33 +++++++++++++++++++++++++++++++++
 include/hw/pci/pci.h |    3 +++
 2 files changed, 36 insertions(+)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 258fbe51e2ee..22765b8529d1 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -319,6 +319,13 @@ static void pci_host_bus_register(DeviceState *host)
     QLIST_INSERT_HEAD(&pci_host_bridges, host_bridge, next);
 }
 
+static void pci_host_bus_unregister(DeviceState *host)
+{
+    PCIHostState *host_bridge = PCI_HOST_BRIDGE(host);
+
+    QLIST_REMOVE(host_bridge, next);
+}
+
 PCIBus *pci_find_primary_bus(void)
 {
     PCIBus *primary_bus = NULL;
@@ -380,6 +387,11 @@ static void pci_bus_init(PCIBus *bus, DeviceState *parent,
     pci_host_bus_register(parent);
 }
 
+static void pci_bus_uninit(PCIBus *bus)
+{
+    pci_host_bus_unregister(BUS(bus)->parent);
+}
+
 bool pci_bus_is_express(PCIBus *bus)
 {
     return object_dynamic_cast(OBJECT(bus), TYPE_PCIE_BUS);
@@ -412,6 +424,12 @@ PCIBus *pci_bus_new(DeviceState *parent, const char *name,
     return bus;
 }
 
+void pci_bus_cleanup(PCIBus *bus)
+{
+    pci_bus_uninit(bus);
+    object_unparent(OBJECT(bus));
+}
+
 void pci_bus_irqs(PCIBus *bus, pci_set_irq_fn set_irq, pci_map_irq_fn map_irq,
                   void *irq_opaque, int nirq)
 {
@@ -422,6 +440,15 @@ void pci_bus_irqs(PCIBus *bus, pci_set_irq_fn set_irq, pci_map_irq_fn map_irq,
     bus->irq_count = g_malloc0(nirq * sizeof(bus->irq_count[0]));
 }
 
+void pci_bus_irqs_cleanup(PCIBus *bus)
+{
+    bus->set_irq = NULL;
+    bus->map_irq = NULL;
+    bus->irq_opaque = NULL;
+    bus->nirq = 0;
+    g_free(bus->irq_count);
+}
+
 PCIBus *pci_register_bus(DeviceState *parent, const char *name,
                          pci_set_irq_fn set_irq, pci_map_irq_fn map_irq,
                          void *irq_opaque,
@@ -437,6 +464,12 @@ PCIBus *pci_register_bus(DeviceState *parent, const char *name,
     return bus;
 }
 
+void pci_unregister_bus(PCIBus *bus)
+{
+    pci_bus_irqs_cleanup(bus);
+    pci_bus_cleanup(bus);
+}
+
 int pci_bus_num(PCIBus *s)
 {
     return PCI_BUS_GET_CLASS(s)->bus_num(s);
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index e598b095ebca..3667e4cda73e 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -400,8 +400,10 @@ PCIBus *pci_bus_new(DeviceState *parent, const char *name,
                     MemoryRegion *address_space_mem,
                     MemoryRegion *address_space_io,
                     uint8_t devfn_min, const char *typename);
+void pci_bus_cleanup(PCIBus *bus);
 void pci_bus_irqs(PCIBus *bus, pci_set_irq_fn set_irq, pci_map_irq_fn map_irq,
                   void *irq_opaque, int nirq);
+void pci_bus_irqs_cleanup(PCIBus *bus);
 int pci_bus_get_irq_level(PCIBus *bus, int irq_num);
 /* 0 <= pin <= 3 0 = INTA, 1 = INTB, 2 = INTC, 3 = INTD */
 int pci_swizzle_map_irq_fn(PCIDevice *pci_dev, int pin);
@@ -411,6 +413,7 @@ PCIBus *pci_register_bus(DeviceState *parent, const char *name,
                          MemoryRegion *address_space_mem,
                          MemoryRegion *address_space_io,
                          uint8_t devfn_min, int nirq, const char *typename);
+void pci_unregister_bus(PCIBus *bus);
 void pci_bus_set_route_irq_fn(PCIBus *, pci_route_irq_fn);
 PCIINTxRoute pci_device_route_intx_to_irq(PCIDevice *dev, int pin);
 bool pci_intx_route_changed(PCIINTxRoute *old, PCIINTxRoute *new);

[Qemu-devel] [for-2.11 PATCH 13/26] qdev: store DeviceState's canonical path to use when unparenting

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

device_unparent(dev, ...) is called when a device is unparented,
either directly, or as a result of a parent device being
finalized, and handles some final cleanup for the device. Part
of this includes emiting a DEVICE_DELETED QMP event to notify
management, which includes the device's path in the composition
tree as provided by object_get_canonical_path().

object_get_canonical_path() assumes the device is still connected
to the machine/root container, and will assert otherwise, but
in some situations this isn't the case:

If the parent is finalized as a result of object_unparent(), it
will still be attached to the composition tree at the time any
children are unparented as a result of that same call to
object_unparent(). However, in some cases, object_unparent()
will complete without finalizing the parent device, due to
lingering references that won't be released till some time later.
One such example is if the parent has MemoryRegion children (which
take a ref on their parent), who in turn have AddressSpace's (which
take a ref on their regions), since those AddressSpaces get cleaned
up asynchronously by the RCU thread.

In this case qdev:device_unparent() may be called for a child Device
that no longer has a path to the root/machine container, causing
object_get_canonical_path() to assert.

Fix this by storing the canonical path during realize() so the
information will still be available for device_unparent() in such
cases.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
---
 hw/core/qdev.c         |   15 ++++++++++++---
 include/hw/qdev-core.h |    1 +
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index 606ab53c42cd..a64b35c16251 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -928,6 +928,12 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
             goto post_realize_fail;
         }
 
+        /* always re-initialize since we clean up in device_unparent() instead
+         * of unrealize()
+         */
+        g_free(dev->canonical_path);
+        dev->canonical_path = object_get_canonical_path(OBJECT(dev));
+
         if (qdev_get_vmsd(dev)) {
             if (vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
                                                dev->instance_id_alias,
@@ -984,6 +990,7 @@ child_realize_fail:
     }
 
 post_realize_fail:
+    g_free(dev->canonical_path);
     if (dc->unrealize) {
         dc->unrealize(dev, NULL);
     }
@@ -1102,10 +1109,12 @@ static void device_unparent(Object *obj)
 
     /* Only send event if the device had been completely realized */
     if (dev->pending_deleted_event) {
-        gchar *path = object_get_canonical_path(OBJECT(dev));
+        g_assert(dev->canonical_path);
 
-        qapi_event_send_device_deleted(!!dev->id, dev->id, path, &error_abort);
-        g_free(path);
+        qapi_event_send_device_deleted(!!dev->id, dev->id, dev->canonical_path,
+                                       &error_abort);
+        g_free(dev->canonical_path);
+        dev->canonical_path = NULL;
     }
 
     qemu_opts_del(dev->opts);
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index ae317286a480..9237b6849ff3 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -153,6 +153,7 @@ struct DeviceState {
     /*< public >*/
 
     const char *id;
+    char *canonical_path;
     bool realized;
     bool pending_deleted_event;
     QemuOpts *opts;

[Qemu-devel] [for-2.11 PATCH 14/26] spapr_pci: add PHB unrealize

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

To support PHB hotplug we need to clean up lingering references,
memory, child properties, etc. prior to the PHB object being
finalized. Generally this will be called as a result of calling
object_unparent() on the PHB object, which in turn would normally
be called as the result of an unplug() operation.

When the PHB is finalized, child objects will be unparented in
turn, and finalized if the PHB was the only reference holder. so
we don't bother to explicitly unparent child objects of the PHB
(spapr_iommu, spapr_drc, etc).

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- s/object_unref/object_unparent/ in the changelog
- adapted to DDW and 64-bit MMIO window
- free dtbusname in finalize function
- don't bother to unparent memory regions: they seem to drop any reference
  on the PHB during unrealize (ie, I couldn't find a scenario where the
  PHB isn't finalized)
---
 hw/ppc/spapr_pci.c |   46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 4e4165b44b9a..994d2f36105f 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1530,6 +1530,50 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
     }
 }
 
+static void spapr_phb_finalizefn(Object *obj)
+{
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(obj);
+
+    g_free(sphb->dtbusname);
+    sphb->dtbusname = NULL;
+}
+
+static void spapr_phb_unrealize(DeviceState *dev, Error **errp)
+{
+    SysBusDevice *s = SYS_BUS_DEVICE(dev);
+    PCIHostState *phb = PCI_HOST_BRIDGE(s);
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(phb);
+    sPAPRTCETable *tcet;
+    int i;
+    const unsigned windows_supported =
+        sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1;
+
+    g_hash_table_unref(sphb->msi);
+
+    /* remove IO/MMIO subregions and aliases, rest should get cleaned
+     * via PHB's unrealize->object_finalize
+     */
+    for (i = windows_supported - 1; i >= 0; i--) {
+        tcet = spapr_tce_find_by_liobn(sphb->dma_liobn[i]);
+        g_assert(tcet);
+        memory_region_del_subregion(&sphb->iommu_root,
+                                    spapr_tce_get_iommu(tcet));
+    }
+
+    QLIST_REMOVE(sphb, list);
+
+    memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow);
+
+    address_space_destroy(&sphb->iommu_as);
+
+    qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort);
+    pci_unregister_bus(phb->bus);
+
+    memory_region_del_subregion(get_system_memory(), &sphb->iowindow);
+    memory_region_del_subregion(get_system_memory(), &sphb->mem64window);
+    memory_region_del_subregion(get_system_memory(), &sphb->mem32window);
+}
+
 static void spapr_phb_realize(DeviceState *dev, Error **errp)
 {
     sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
@@ -1961,6 +2005,7 @@ static void spapr_phb_class_init(ObjectClass *klass, void *data)
 
     hc->root_bus_path = spapr_phb_root_bus_path;
     dc->realize = spapr_phb_realize;
+    dc->unrealize = spapr_phb_unrealize;
     dc->props = spapr_phb_properties;
     dc->reset = spapr_phb_reset;
     dc->vmsd = &vmstate_spapr_pci;
@@ -1975,6 +2020,7 @@ static const TypeInfo spapr_phb_info = {
     .name          = TYPE_SPAPR_PCI_HOST_BRIDGE,
     .parent        = TYPE_PCI_HOST_BRIDGE,
     .instance_size = sizeof(sPAPRPHBState),
+    .instance_finalize = spapr_phb_finalizefn,
     .class_init    = spapr_phb_class_init,
     .interfaces    = (InterfaceInfo[]) {
         { TYPE_HOTPLUG_HANDLER },

[Qemu-devel] [for-2.11 PATCH 15/26] spapr: add pseries-2.11 machine type

[Greg Kurz]

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c      |   22 ++++++++++++++++++++--
 include/hw/compat.h |    3 +++
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 53969315ac24..277daa4f9645 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -3533,18 +3533,36 @@ static const TypeInfo spapr_machine_info = {
     type_init(spapr_machine_register_##suffix)
 
 /*
+ * pseries-2.11
+ */
+static void spapr_machine_2_11_instance_options(MachineState *machine)
+{
+}
+
+static void spapr_machine_2_11_class_options(MachineClass *mc)
+{
+    /* Defaults for the latest behaviour inherited from the base class */
+}
+
+DEFINE_SPAPR_MACHINE(2_11, "2.11", true);
+
+/*
  * pseries-2.10
  */
+#define SPAPR_COMPAT_2_10                                              \
+    HW_COMPAT_2_10                                                     \
+
 static void spapr_machine_2_10_instance_options(MachineState *machine)
 {
 }
 
 static void spapr_machine_2_10_class_options(MachineClass *mc)
 {
-    /* Defaults for the latest behaviour inherited from the base class */
+    spapr_machine_2_11_class_options(mc);
+    SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
 }
 
-DEFINE_SPAPR_MACHINE(2_10, "2.10", true);
+DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
 
 /*
  * pseries-2.9
diff --git a/include/hw/compat.h b/include/hw/compat.h
index 08f36004dad7..3e101f8f6783 100644
--- a/include/hw/compat.h
+++ b/include/hw/compat.h
@@ -1,6 +1,9 @@
 #ifndef HW_COMPAT_H
 #define HW_COMPAT_H
 
+#define HW_COMPAT_2_10 \
+    /* empty */
+
 #define HW_COMPAT_2_9 \
     {\
         .driver   = "pci-bridge",\

[Qemu-devel] [for-2.11 PATCH 16/26] spapr: enable PHB hotplug for pseries-2.11

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

The 'dr_phb_enabled' field of that class can be set as
part of machine-specific init code, and is then propagated
to sPAPREnvironment to conditionally enable creation of DRC
objects and device-tree description to facilitate hotplug
of PHBs.

Since we can't migrate this state to older machine types,
default the option to true and disable it for older machine
types.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- updated changelog and title
- set default to true and disable for 2.10 and older
---
 hw/ppc/spapr.c         |    6 ++++++
 include/hw/ppc/spapr.h |    3 +++
 2 files changed, 9 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 277daa4f9645..8dc505343c0f 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2382,6 +2382,8 @@ static void ppc_spapr_init(MachineState *machine)
     /* We always have at least the nvram device on VIO */
     spapr_create_nvram(spapr);
 
+    spapr->dr_phb_enabled = smc->dr_phb_enabled;
+
     /* Set up PCI */
     spapr_pci_rtas_init();
 
@@ -3482,6 +3484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
      * in which LMBs are represented and hot-added
      */
     mc->numa_mem_align_shift = 28;
+    smc->dr_phb_enabled = true;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -3558,8 +3561,11 @@ static void spapr_machine_2_10_instance_options(MachineState *machine)
 
 static void spapr_machine_2_10_class_options(MachineClass *mc)
 {
+    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
+
     spapr_machine_2_11_class_options(mc);
     SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
+    smc->dr_phb_enabled = false;
 }
 
 DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 2a303a705c17..8004d9c2ab2c 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -59,6 +59,7 @@ struct sPAPRMachineClass {
 
     /*< public >*/
     bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
+    bool dr_phb_enabled;       /* enable dynamic-reconfig/hotplug of PHBs */
     bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
     const char *tcg_default_cpu; /* which (TCG) CPU to simulate by default */
     bool pre_2_10_has_unused_icps;
@@ -122,6 +123,8 @@ struct sPAPRMachineState {
      * occurs during the unplug process. */
     QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
 
+    bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */
+
     /*< public >*/
     char *kvm_type;
     MemoryHotplugState hotplug_memory;

[Qemu-devel] [for-2.11 PATCH 17/26] spapr_pci: introduce drc_id property

[Greg Kurz]

With the addition of PHB hotplug, we have a static number of DRCs
that can be used to handle hotplug/unplug operations on our PHBs,
and need a consistent way to map PHBs to these connectors, and
assign a unique identifiers for the connectors.

This patch adds a drc_id property for that purpose.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_pci.c          |    1 +
 include/hw/pci-host/spapr.h |    2 ++
 2 files changed, 3 insertions(+)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 994d2f36105f..54533d8a3841 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1878,6 +1878,7 @@ static Property spapr_phb_properties[] = {
                      pre_2_8_migration, false),
     DEFINE_PROP_BOOL("pcie-extended-configuration-space", sPAPRPHBState,
                      pcie_ecs, true),
+    DEFINE_PROP_UINT32("drc_id", sPAPRPHBState, drc_id, -1),
     DEFINE_PROP_END_OF_LIST(),
 };
 
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 5a4e9686d562..31bae68167f2 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -82,6 +82,8 @@ struct sPAPRPHBState {
 
     bool pcie_ecs; /* Allow access to PCIe extended config space? */
 
+    uint32_t drc_id;
+
     /* Fields for migration compatibility hacks */
     bool pre_2_8_migration;
     uint32_t mig_liobn;

[Qemu-devel] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10 (reset hooks registering already merged)
- added new DRC type for PHB
---
 hw/ppc/spapr.c             |   15 +++++++++++++++
 hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
 include/hw/ppc/spapr_drc.h |    8 ++++++++
 3 files changed, 40 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 8dc505343c0f..5950c009ab7e 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -98,6 +98,9 @@
 
 #define PHANDLE_XICP            0x00001111
 
+/* maximum number of hotpluggable PHBs */
+#define SPAPR_DRC_MAX_PHB       256
+
 static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
                                   const char *type_ics,
                                   int nr_irqs, Error **errp)
@@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
 
     spapr->dr_phb_enabled = smc->dr_phb_enabled;
 
+    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
+     * connectors (described in root DT node's "ibm,drc-types" property)
+     * are pre-initialized here. additional child connectors (such as
+     * connectors for a PHBs PCI slots) are added as needed during their
+     * parent's realization.
+     */
+    if (spapr->dr_phb_enabled) {
+        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
+            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
+        }
+    }
+
     /* Set up PCI */
     spapr_pci_rtas_init();
 
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index eb8024d37c54..2e1049ce61c7 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
     drck->release = spapr_lmb_release;
 }
 
+static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
+{
+    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
+
+    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
+    drck->typename = "PHB";
+    drck->drc_name_prefix = "PHB ";
+}
+
 static const TypeInfo spapr_dr_connector_info = {
     .name          = TYPE_SPAPR_DR_CONNECTOR,
     .parent        = TYPE_DEVICE,
@@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
     .class_init    = spapr_drc_lmb_class_init,
 };
 
+static const TypeInfo spapr_drc_phb_info = {
+    .name          = TYPE_SPAPR_DRC_PHB,
+    .parent        = TYPE_SPAPR_DRC_LOGICAL,
+    .instance_size = sizeof(sPAPRDRConnector),
+    .class_init    = spapr_drc_phb_class_init,
+};
+
 /* helper functions for external users */
 
 sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
@@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
     type_register_static(&spapr_drc_cpu_info);
     type_register_static(&spapr_drc_pci_info);
     type_register_static(&spapr_drc_lmb_info);
+    type_register_static(&spapr_drc_phb_info);
 
     spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
                         rtas_set_indicator);
diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
index a7958d0a8d14..535fc61b98a8 100644
--- a/include/hw/ppc/spapr_drc.h
+++ b/include/hw/ppc/spapr_drc.h
@@ -69,6 +69,14 @@
 #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
                                         TYPE_SPAPR_DRC_LMB)
 
+#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
+#define SPAPR_DRC_PHB_GET_CLASS(obj) \
+        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
+#define SPAPR_DRC_PHB_CLASS(klass) \
+        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
+#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
+                                        TYPE_SPAPR_DRC_PHB)
+
 /*
  * Various hotplug types managed by sPAPRDRConnector
  *

[Qemu-devel] [for-2.11 PATCH 19/26] spapr: populate PHB DRC entries for root DT node

[Greg Kurz]

From: Nathan Fontenot <nfont@linux.vnet.ibm.com>

This add entries to the root OF node to advertise our PHBs as being
DR-capable in accordance with PAPR specification.

Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- use error_report
---
 hw/ppc/spapr.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 5950c009ab7e..632040f35ecc 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1151,6 +1151,16 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
         exit(1);
     }
 
+    if (smc->dr_phb_enabled) {
+        int offset = fdt_path_offset(fdt, "/");
+        ret = spapr_drc_populate_dt(fdt, offset, NULL,
+                                    SPAPR_DR_CONNECTOR_TYPE_PHB);
+        if (ret < 0) {
+            error_report("Couldn't set up PHB DR device tree properties");
+            exit(1);
+        }
+    }
+
     return fdt;
 }
 

[Qemu-devel] [for-2.11 PATCH 20/26] spapr_events: add support for phb hotplug events

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Extend the existing EPOW event format we use for PCI
devices to emit PHB plug/unplug events.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
---
 hw/ppc/spapr_events.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index f952b78237e1..24ea43d2b951 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -531,6 +531,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
     case SPAPR_DR_CONNECTOR_TYPE_CPU:
         hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
         break;
+    case SPAPR_DR_CONNECTOR_TYPE_PHB:
+        hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_PHB;
+        break;
     default:
         /* we shouldn't be signaling hotplug events for resources
          * that don't support them

[Qemu-devel] [for-2.11 PATCH 21/26] qdev: pass an Object * to qbus_set_hotplug_handler()

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Certain devices types, like memory/CPU, are now being handled using a
hotplug interface provided by a top-level MachineClass. Hotpluggable
host bridges are another such device where it makes sense to use a
machine-level hotplug handler. However, unlike those devices,
host-bridges have a parent bus (the main system bus), and devices with
a parent bus use a different mechanism for registering their hotplug
handlers: qbus_set_hotplug_handler(). This interface currently expects
a handler to be a subclass of DeviceClass, but this is not the case
for MachineClass, which derives directly from ObjectClass.

Internally, the interface only requires an ObjectClass, so expose that
in qbus_set_hotplug_handler().

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- changed qbus_set_hotplug_handler() declaration and dropped
  qbus_set_hotplug_handler_internal() (Paolo)
- updated title and changelog accordingly
---
 hw/acpi/piix4.c               |    2 +-
 hw/char/virtio-serial-bus.c   |    2 +-
 hw/core/bus.c                 |   11 ++---------
 hw/pci/pcie.c                 |    2 +-
 hw/pci/shpc.c                 |    2 +-
 hw/ppc/spapr_pci.c            |    2 +-
 hw/s390x/css-bridge.c         |    2 +-
 hw/s390x/s390-pci-bus.c       |    6 +++---
 hw/scsi/virtio-scsi.c         |    2 +-
 hw/scsi/vmw_pvscsi.c          |    2 +-
 hw/usb/dev-smartcard-reader.c |    2 +-
 include/hw/qdev-core.h        |    3 +--
 12 files changed, 15 insertions(+), 23 deletions(-)

diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index f276967365c4..f99c7438235e 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -447,7 +447,7 @@ static void piix4_update_bus_hotplug(PCIBus *pci_bus, void *opaque)
 
     /* pci_bus cannot outlive PIIX4PMState, because /machine keeps it alive
      * and it's not hot-unpluggable */
-    qbus_set_hotplug_handler(BUS(pci_bus), DEVICE(s), &error_abort);
+    qbus_set_hotplug_handler(BUS(pci_bus), OBJECT(s), &error_abort);
 }
 
 static void piix4_pm_machine_ready(Notifier *n, void *opaque)
diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index f5bc173844e4..4880236f4258 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -1042,7 +1042,7 @@ static void virtio_serial_device_realize(DeviceState *dev, Error **errp)
     /* Spawn a new virtio-serial bus on which the ports will ride as devices */
     qbus_create_inplace(&vser->bus, sizeof(vser->bus), TYPE_VIRTIO_SERIAL_BUS,
                         dev, vdev->bus_name);
-    qbus_set_hotplug_handler(BUS(&vser->bus), DEVICE(vser), errp);
+    qbus_set_hotplug_handler(BUS(&vser->bus), OBJECT(vser), errp);
     vser->bus.vser = vser;
     QTAILQ_INIT(&vser->ports);
 
diff --git a/hw/core/bus.c b/hw/core/bus.c
index 4651f244864c..e09843f6abea 100644
--- a/hw/core/bus.c
+++ b/hw/core/bus.c
@@ -22,22 +22,15 @@
 #include "hw/qdev.h"
 #include "qapi/error.h"
 
-static void qbus_set_hotplug_handler_internal(BusState *bus, Object *handler,
-                                              Error **errp)
+void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp)
 {
-
     object_property_set_link(OBJECT(bus), OBJECT(handler),
                              QDEV_HOTPLUG_HANDLER_PROPERTY, errp);
 }
 
-void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler, Error **errp)
-{
-    qbus_set_hotplug_handler_internal(bus, OBJECT(handler), errp);
-}
-
 void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp)
 {
-    qbus_set_hotplug_handler_internal(bus, OBJECT(bus), errp);
+    qbus_set_hotplug_handler(bus, OBJECT(bus), errp);
 }
 
 int qbus_walk_children(BusState *bus,
diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
index 32191f2a55f7..52d9ff947f7c 100644
--- a/hw/pci/pcie.c
+++ b/hw/pci/pcie.c
@@ -443,7 +443,7 @@ void pcie_cap_slot_init(PCIDevice *dev, uint16_t slot)
     dev->exp.hpev_notified = false;
 
     qbus_set_hotplug_handler(BUS(pci_bridge_get_sec_bus(PCI_BRIDGE(dev))),
-                             DEVICE(dev), NULL);
+                             OBJECT(dev), NULL);
 }
 
 void pcie_cap_slot_reset(PCIDevice *dev)
diff --git a/hw/pci/shpc.c b/hw/pci/shpc.c
index 69fc14b218d8..feed8a45e535 100644
--- a/hw/pci/shpc.c
+++ b/hw/pci/shpc.c
@@ -648,7 +648,7 @@ int shpc_init(PCIDevice *d, PCIBus *sec_bus, MemoryRegion *bar,
     shpc_cap_update_dword(d);
     memory_region_add_subregion(bar, offset, &shpc->mmio);
 
-    qbus_set_hotplug_handler(BUS(sec_bus), DEVICE(d), NULL);
+    qbus_set_hotplug_handler(BUS(sec_bus), OBJECT(d), NULL);
 
     d->cap_present |= QEMU_PCI_CAP_SHPC;
     return 0;
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 54533d8a3841..b73e099e0285 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1714,7 +1714,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
                            &sphb->memspace, &sphb->iospace,
                            PCI_DEVFN(0, 0), PCI_NUM_PINS, TYPE_PCI_BUS);
     phb->bus = bus;
-    qbus_set_hotplug_handler(BUS(phb->bus), DEVICE(sphb), NULL);
+    qbus_set_hotplug_handler(BUS(phb->bus), OBJECT(sphb), NULL);
 
     /*
      * Initialize PHB address space.
diff --git a/hw/s390x/css-bridge.c b/hw/s390x/css-bridge.c
index c4a9735d7108..19eda77979d1 100644
--- a/hw/s390x/css-bridge.c
+++ b/hw/s390x/css-bridge.c
@@ -107,7 +107,7 @@ VirtualCssBus *virtual_css_bus_init(void)
     cbus->squash_mcss = s390_get_squash_mcss();
 
     /* Enable hotplugging */
-    qbus_set_hotplug_handler(bus, dev, &error_abort);
+    qbus_set_hotplug_handler(bus, OBJECT(dev), &error_abort);
 
     css_register_io_adapters(CSS_IO_ADAPTER_VIRTIO, true, false,
                              0, &error_abort);
diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
index 61cfd2138f68..503b7ae05abb 100644
--- a/hw/s390x/s390-pci-bus.c
+++ b/hw/s390x/s390-pci-bus.c
@@ -566,11 +566,11 @@ static int s390_pcihost_init(SysBusDevice *dev)
     pci_setup_iommu(b, s390_pci_dma_iommu, s);
 
     bus = BUS(b);
-    qbus_set_hotplug_handler(bus, DEVICE(dev), NULL);
+    qbus_set_hotplug_handler(bus, OBJECT(dev), NULL);
     phb->bus = b;
 
     s->bus = S390_PCI_BUS(qbus_create(TYPE_S390_PCI_BUS, DEVICE(s), NULL));
-    qbus_set_hotplug_handler(BUS(s->bus), DEVICE(s), NULL);
+    qbus_set_hotplug_handler(BUS(s->bus), OBJECT(s), NULL);
 
     s->iommu_table = g_hash_table_new_full(g_int64_hash, g_int64_equal,
                                            NULL, g_free);
@@ -678,7 +678,7 @@ static void s390_pcihost_hot_plug(HotplugHandler *hotplug_dev,
         pci_setup_iommu(&pb->sec_bus, s390_pci_dma_iommu, s);
 
         bus = BUS(&pb->sec_bus);
-        qbus_set_hotplug_handler(bus, DEVICE(s), errp);
+        qbus_set_hotplug_handler(bus, OBJECT(s), errp);
 
         if (dev->hotplugged) {
             pci_default_write_config(pdev, PCI_PRIMARY_BUS, s->bus_no, 1);
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index eb639442d192..b4c26d4862b5 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -893,7 +893,7 @@ static void virtio_scsi_device_realize(DeviceState *dev, Error **errp)
     scsi_bus_new(&s->bus, sizeof(s->bus), dev,
                  &virtio_scsi_scsi_info, vdev->bus_name);
     /* override default SCSI bus hotplug-handler, with virtio-scsi's one */
-    qbus_set_hotplug_handler(BUS(&s->bus), dev, &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
 
     virtio_scsi_dataplane_setup(s, errp);
 }
diff --git a/hw/scsi/vmw_pvscsi.c b/hw/scsi/vmw_pvscsi.c
index 4a106da85646..036e44caca42 100644
--- a/hw/scsi/vmw_pvscsi.c
+++ b/hw/scsi/vmw_pvscsi.c
@@ -1146,7 +1146,7 @@ pvscsi_init(PCIDevice *pci_dev)
     scsi_bus_new(&s->bus, sizeof(s->bus), DEVICE(pci_dev),
                  &pvscsi_scsi_info, NULL);
     /* override default SCSI bus hotplug-handler, with pvscsi's one */
-    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(s), &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(s), &error_abort);
     pvscsi_reset_state(s);
 
     return 0;
diff --git a/hw/usb/dev-smartcard-reader.c b/hw/usb/dev-smartcard-reader.c
index bef1f03c426b..f2a86de6c556 100644
--- a/hw/usb/dev-smartcard-reader.c
+++ b/hw/usb/dev-smartcard-reader.c
@@ -1337,7 +1337,7 @@ static void ccid_realize(USBDevice *dev, Error **errp)
     usb_desc_init(dev);
     qbus_create_inplace(&s->bus, sizeof(s->bus), TYPE_CCID_BUS, DEVICE(dev),
                         NULL);
-    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(dev), &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
     s->intr = usb_ep_get(dev, USB_TOKEN_IN, CCID_INT_IN_EP);
     s->bulk = usb_ep_get(dev, USB_TOKEN_IN, CCID_BULK_IN_EP);
     s->card = NULL;
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 9237b6849ff3..ce01ff42850b 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -397,8 +397,7 @@ char *qdev_get_dev_path(DeviceState *dev);
 
 GSList *qdev_build_hotpluggable_device_list(Object *peripheral);
 
-void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler,
-                              Error **errp);
+void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp);
 
 void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp);
 

[Qemu-devel] [for-2.11 PATCH 22/26] spapr_pci: provide node start offset via spapr_populate_pci_dt()

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

PHB hotplug re-uses PHB device tree generation code and passes
it to a guest via RTAS. Doing this requires knowledge of where
exactly in the device tree the node describing the PHB begins.

Provide this via a new optional pointer that can be used to
store the PHB node's start offset.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
---
 hw/ppc/spapr.c              |    2 +-
 hw/ppc/spapr_pci.c          |    6 +++++-
 include/hw/pci-host/spapr.h |    3 ++-
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 632040f35ecc..1a6cd4efeb97 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1098,7 +1098,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
     }
 
     QLIST_FOREACH(phb, &spapr->phbs, list) {
-        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt);
+        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt, NULL);
         if (ret < 0) {
             error_report("couldn't setup PCI devices in fdt");
             exit(1);
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index b73e099e0285..79f10ff453d0 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -2109,7 +2109,8 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
 
 int spapr_populate_pci_dt(sPAPRPHBState *phb,
                           uint32_t xics_phandle,
-                          void *fdt)
+                          void *fdt,
+                          int *node_offset)
 {
     int bus_off, i, j, ret;
     char nodename[FDT_NAME_MAX];
@@ -2166,6 +2167,9 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
     if (bus_off < 0) {
         return bus_off;
     }
+    if (node_offset) {
+        *node_offset = bus_off;
+    }
 
     /* Write PHB properties */
     _FDT(fdt_setprop_string(fdt, bus_off, "device_type", "pci"));
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 31bae68167f2..7837fb0b1110 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -115,7 +115,8 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
 
 int spapr_populate_pci_dt(sPAPRPHBState *phb,
                           uint32_t xics_phandle,
-                          void *fdt);
+                          void *fdt,
+                          int *node_offset);
 
 void spapr_pci_rtas_init(void);
 

[Qemu-devel] [for-2.11 PATCH 23/26] spapr_pci: add ibm, my-drc-index property for PHB hotplug

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

This is needed to denote a boot-time PHB as being hot-pluggable.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
Changes since RFC:
- rebased against ppc-for-2.10
- use phb->drc_id
---
 hw/ppc/spapr_pci.c |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 79f10ff453d0..58406a1b7e93 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -2160,6 +2160,7 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
     sPAPRTCETable *tcet;
     PCIBus *bus = PCI_HOST_BRIDGE(phb)->bus;
     sPAPRFDT s_fdt;
+    sPAPRDRConnector *drc;
 
     /* Start populating the FDT */
     snprintf(nodename, FDT_NAME_MAX, "pci@%" PRIx64, phb->buid);
@@ -2229,6 +2230,14 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
                  tcet->liobn, tcet->bus_offset,
                  tcet->nb_table << tcet->page_shift);
 
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, phb->drc_id);
+    if (drc) {
+        uint32_t drc_index = cpu_to_be32(spapr_drc_index(drc));
+
+        _FDT(fdt_setprop(fdt, bus_off, "ibm,my-drc-index", &drc_index,
+                         sizeof(drc_index)));
+    }
+
     /* Walk the bridges and program the bus numbers*/
     spapr_phb_pci_enumerate(phb);
     _FDT(fdt_setprop_cell(fdt, bus_off, "qemu,phb-enumerated", 0x1));

[Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[Greg Kurz]

The "phandle" property of the XICS node is referenced by the "interrupt-map"
property of each PHB node. This is used by the guest OS to setup IRQs for
all PCI devices.

QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
this value to a SLOF specific one, which is then presented to the guest OS.

This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
by SLOF to communicate the patched phandle value back to QEMU. This value
is then cached and preserved accross migration until machine reset.

This is required to be able to support PHB hotplug.

Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
have to introduce its number even if QEMU currently doesn't implement it.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c         |   25 +++++++++++++++++++++++--
 hw/ppc/spapr_hcall.c   |   20 ++++++++++++++++++++
 include/hw/ppc/spapr.h |   11 ++++++++++-
 3 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1a6cd4efeb97..90485054c2e7 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -96,8 +96,6 @@
 
 #define MIN_RMA_SLOF            128UL
 
-#define PHANDLE_XICP            0x00001111
-
 /* maximum number of hotpluggable PHBs */
 #define SPAPR_DRC_MAX_PHB       256
 
@@ -1454,6 +1452,7 @@ static void ppc_spapr_reset(void)
     first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
 
     spapr->cas_reboot = false;
+    spapr->xics_phandle = UINT32_MAX;
 }
 
 static void spapr_create_nvram(sPAPRMachineState *spapr)
@@ -1652,6 +1651,26 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
     },
 };
 
+static bool spapr_xics_phandle_needed(void *opaque)
+{
+    sPAPRMachineState *spapr = opaque;
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(MACHINE(spapr));
+
+    /* Don't break older machine types that don't support PHB hotplug. */
+    return smc->dr_phb_enabled && spapr->xics_phandle != UINT32_MAX;
+}
+
+static const VMStateDescription vmstate_spapr_xics_phandle = {
+    .name = "spapr_xics_phandle",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = spapr_xics_phandle_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32(xics_phandle, sPAPRMachineState),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 static const VMStateDescription vmstate_spapr = {
     .name = "spapr",
     .version_id = 3,
@@ -1671,6 +1690,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_ov5_cas,
         &vmstate_spapr_patb_entry,
         &vmstate_spapr_pending_events,
+        &vmstate_spapr_xics_phandle,
         NULL
     }
 };
@@ -2702,6 +2722,7 @@ static void spapr_machine_initfn(Object *obj)
 
     spapr->htab_fd = -1;
     spapr->use_hotplug_event_source = true;
+    spapr->xics_phandle = UINT32_MAX;
     object_property_add_str(obj, "kvm-type",
                             spapr_get_kvm_type, spapr_set_kvm_type, NULL);
     object_property_set_description(obj, "kvm-type",
diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 72ea5a8247bf..ce8a9eb66b23 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -1623,6 +1623,25 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu,
     return H_SUCCESS;
 }
 
+static target_ulong h_update_phandle(PowerPCCPU *cpu, sPAPRMachineState *spapr,
+                                     target_ulong opcode, target_ulong *args)
+{
+    target_ulong old_phandle = args[0];
+    target_ulong new_phandle = args[1];
+
+    if (new_phandle >= UINT32_MAX) {
+        return H_PARAMETER;
+    }
+
+    /* We only have a "phandle" property in the XICS node at the moment. */
+    if (old_phandle != (uint32_t) PHANDLE_XICP) {
+        return H_PARAMETER;
+    }
+
+    spapr->xics_phandle = (uint32_t) new_phandle;
+    return H_SUCCESS;
+}
+
 static spapr_hcall_fn papr_hypercall_table[(MAX_HCALL_OPCODE / 4) + 1];
 static spapr_hcall_fn kvmppc_hypercall_table[KVMPPC_HCALL_MAX - KVMPPC_HCALL_BASE + 1];
 
@@ -1717,6 +1736,7 @@ static void hypercall_register_types(void)
 
     /* qemu/KVM-PPC specific hcalls */
     spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
+    spapr_register_hypercall(KVMPPC_H_UPDATE_PHANDLE, h_update_phandle);
 
     /* ibm,client-architecture-support support */
     spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 8004d9c2ab2c..f09c54d5bb94 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -125,6 +125,8 @@ struct sPAPRMachineState {
 
     bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */
 
+    uint32_t xics_phandle;
+
     /*< public >*/
     char *kvm_type;
     MemoryHotplugState hotplug_memory;
@@ -402,7 +404,9 @@ struct sPAPRMachineState {
 #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
 /* Client Architecture support */
 #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
-#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
+#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
+#define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4)
+#define KVMPPC_HCALL_MAX        KVMPPC_H_UPDATE_PHANDLE
 
 typedef struct sPAPRDeviceTreeUpdateHeader {
     uint32_t version_id;
@@ -707,4 +711,9 @@ void spapr_do_system_reset_on_cpu(CPUState *cs, run_on_cpu_data arg);
 
 #define HTAB_SIZE(spapr)        (1ULL << ((spapr)->htab_shift))
 
+/* Boot time value of the "phandle" property of the "interrupt-controller"
+ * node.
+ */
+#define PHANDLE_XICP            0x00001111
+
 #endif /* HW_SPAPR_H */

[Qemu-devel] [for-2.11 PATCH 25/26] spapr_pci: drop abusive sanity check when migrating the LSI table

[Greg Kurz]

The guest can allocate blocks of IRQs when calling the ibm,change-msi
RTAS call. This has an impact on the IRQ numbers returned by subsequent
calls to spapr_ics_alloc_block().

It doesn't cause any problem right now because PHB are cold plugged and
the LSI table have the same numbers at the destination and the source.

But this won't be the case anymore when we support PHB hotplug. In this
case the IRQ numbers in the LSI table are state that we should migrate.

This patch hence changes the destination to always accept the LSI table
from the migration stream, like it is already done for MSIs. No effort
is made to preserve the current behavior of failing migration when LSI
numbers are different with older machine types, for simplicity.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_pci.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 58406a1b7e93..157867af8178 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1887,8 +1887,7 @@ static const VMStateDescription vmstate_spapr_pci_lsi = {
     .version_id = 1,
     .minimum_version_id = 1,
     .fields = (VMStateField[]) {
-        VMSTATE_UINT32_EQUAL(irq, struct spapr_pci_lsi, NULL),
-
+        VMSTATE_UINT32(irq, struct spapr_pci_lsi),
         VMSTATE_END_OF_LIST()
     },
 };

Re: [Qemu-devel] [for-2.11 PATCH 19/26] spapr: populate PHB DRC entries for root DT node

[Michael Roth]

Quoting Greg Kurz (2017-07-25 13:02:03)
> From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> 
> This add entries to the root OF node to advertise our PHBs as being
> DR-capable in accordance with PAPR specification.
> 
> Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> - use error_report
> ---
>  hw/ppc/spapr.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 5950c009ab7e..632040f35ecc 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1151,6 +1151,16 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>          exit(1);
>      }
> 
> +    if (smc->dr_phb_enabled) {
> +        int offset = fdt_path_offset(fdt, "/");
> +        ret = spapr_drc_populate_dt(fdt, offset, NULL,
> +                                    SPAPR_DR_CONNECTOR_TYPE_PHB);

This might actually clobber LMB/MEM entries added earlier since they both
try to create the /ibm,drc* entries in / from scratch.
  
If that's the case, we can arrange the code something like:
  
  uint32_t root_drc_types = 0;
  
  if (smc->dr_lmb_enabled) {
      root_drc_types |= SPAPR_DR_CONNECTOR_TYPE_LMB;
  }
  
  if (smc->dr_phb_enabled) {
      root_drc_types |= SPAPR_DR_CONNECTOR_TYPE_PHB;
  }
  
  ...
  
  if (root_drc_types) {
    int offset = fdt_path_offset(fdt, "/");
    ret = spapr_drc_populate_dt(fdt, offset, NULL, root_drc_types);
  }


> +        if (ret < 0) {
> +            error_report("Couldn't set up PHB DR device tree properties");
> +            exit(1);
> +        }
> +    }
> +
>      return fdt;
>  }
> 
> 

Re: [Qemu-devel] [for-2.11 PATCH 01/26] spapr: move spapr_create_phb() to core machine code

[Alexey Kardashevskiy]

On 26/07/17 03:58, Greg Kurz wrote:
> This function is only used when creating the default PHB. Let's rename
> it and move it to the core machine code for clarity.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr.c              |   13 ++++++++++++-
>  hw/ppc/spapr_pci.c          |   11 -----------
>  include/hw/pci-host/spapr.h |    2 --
>  3 files changed, 12 insertions(+), 14 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 2a3e53d5d58c..53969315ac24 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2142,6 +2142,17 @@ static void spapr_init_cpus(sPAPRMachineState *spapr)
>      g_free(type);
>  }
>  
> +static PCIHostState *spapr_create_default_phb(sPAPRMachineState *spapr)


Nit: you could get rid of the *spapr here.

Anyway,

Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>



> +{
> +    DeviceState *dev;
> +
> +    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
> +    qdev_prop_set_uint32(dev, "index", 0);
> +    qdev_init_nofail(dev);
> +
> +    return PCI_HOST_BRIDGE(dev);
> +}
> +
>  /* pSeries LPAR / sPAPR hardware init */
>  static void ppc_spapr_init(MachineState *machine)
>  {
> @@ -2374,7 +2385,7 @@ static void ppc_spapr_init(MachineState *machine)
>      /* Set up PCI */
>      spapr_pci_rtas_init();
>  
> -    phb = spapr_create_phb(spapr, 0);
> +    phb = spapr_create_default_phb(spapr);
>  
>      for (i = 0; i < nb_nics; i++) {
>          NICInfo *nd = &nd_table[i];
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index d84abf1070a0..70a99a032267 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1982,17 +1982,6 @@ static const TypeInfo spapr_phb_info = {
>      }
>  };
>  
> -PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index)
> -{
> -    DeviceState *dev;
> -
> -    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
> -    qdev_prop_set_uint32(dev, "index", index);
> -    qdev_init_nofail(dev);
> -
> -    return PCI_HOST_BRIDGE(dev);
> -}
> -
>  typedef struct sPAPRFDT {
>      void *fdt;
>      int node_off;
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 38470b2f0e5c..5a4e9686d562 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -111,8 +111,6 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
>      return xics_get_qirq(XICS_FABRIC(spapr), phb->lsi_table[pin].irq);
>  }
>  
> -PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index);
> -
>  int spapr_populate_pci_dt(sPAPRPHBState *phb,
>                            uint32_t xics_phandle,
>                            void *fdt);
> 
> 


-- 
Alexey

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 02/26] spapr_pci: use memory_region_add_subregion() with DMA windows

[Alexey Kardashevskiy]

On 26/07/17 03:58, Greg Kurz wrote:
> Passing a null priority to memory_region_add_subregion_overlap() is
> strictly equivalent to calling memory_region_add_subregion().
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

> ---
>  hw/ppc/spapr_pci.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 70a99a032267..3fe7f3145467 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1752,8 +1752,8 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>                         i, sphb->dtbusname);
>              return;
>          }
> -        memory_region_add_subregion_overlap(&sphb->iommu_root, 0,
> -                                            spapr_tce_get_iommu(tcet), 0);
> +        memory_region_add_subregion(&sphb->iommu_root, 0,
> +                                    spapr_tce_get_iommu(tcet));
>      }
>  
>      sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 03/26] spapr_iommu: use g_strdup_printf() instead of snprintf()

[Alexey Kardashevskiy]

On 26/07/17 03:58, Greg Kurz wrote:
> Passing a stack allocated buffer of arbitrary length to snprintf()
> without checking the return value can cause the resultant strings
> to be silently truncated.

The strings it is touching cannot be silently truncated as
"tce-iommu-XXXXXXXX" are shorter than 32 chars.


> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr_iommu.c |   13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> index e614621a8317..740d42608b61 100644
> --- a/hw/ppc/spapr_iommu.c
> +++ b/hw/ppc/spapr_iommu.c
> @@ -252,17 +252,19 @@ static int spapr_tce_table_realize(DeviceState *dev)
>  {
>      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
>      Object *tcetobj = OBJECT(tcet);
> -    char tmp[32];
> +    gchar *tmp;
>  
>      tcet->fd = -1;
>      tcet->need_vfio = false;
> -    snprintf(tmp, sizeof(tmp), "tce-root-%x", tcet->liobn);
> +    tmp = g_strdup_printf("tce-root-%x", tcet->liobn);
>      memory_region_init(&tcet->root, tcetobj, tmp, UINT64_MAX);
> +    g_free(tmp);
>  
> -    snprintf(tmp, sizeof(tmp), "tce-iommu-%x", tcet->liobn);
> +    tmp = g_strdup_printf("tce-iommu-%x", tcet->liobn);
>      memory_region_init_iommu(&tcet->iommu, sizeof(tcet->iommu),
>                               TYPE_SPAPR_IOMMU_MEMORY_REGION,
>                               tcetobj, tmp, 0);
> +    g_free(tmp);
>  
>      QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list);
>  
> @@ -307,7 +309,7 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
>  sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
>  {
>      sPAPRTCETable *tcet;
> -    char tmp[32];
> +    gchar *tmp;
>  
>      if (spapr_tce_find_by_liobn(liobn)) {
>          error_report("Attempted to create TCE table with duplicate"
> @@ -318,8 +320,9 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
>      tcet = SPAPR_TCE_TABLE(object_new(TYPE_SPAPR_TCE_TABLE));
>      tcet->liobn = liobn;
>  
> -    snprintf(tmp, sizeof(tmp), "tce-table-%x", liobn);
> +    tmp = g_strdup_printf("tce-table-%x", liobn);
>      object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
> +    g_free(tmp);
>  
>      object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
>  
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Alexey Kardashevskiy]

On 26/07/17 03:57, Greg Kurz wrote:
> This series is based on patches from Michel Roth posted in 2015:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg04246.html
> 
> It addresses comments made during the RFC review, and also provides a bunch
> of preliminary cleanup/fix patches. Since we have reached hard-freeze, this
> feature is provided by a new pseries-2.11 machine type, introduced by this
> series. It is based on David Gibson's ppc-for-2.10 branch, and I believe some
> of the preliminary fixes are eligible for 2.10.
> 
> Note that it also requires an updated SLOF that supports a new private hcall:
> KVMPPC_H_UPDATE_PHANDLE. This is needed because SLOF changes FDT phandles to
> Open Firmware phandles. Since the guest only sees the latter, QEMU must use
> the updated value when populating the FDT for the hotplugged PHB (otherwise
> the guest can't setup IRQs for the PCI devices). SLOF part is already upstream:
> 
> http://git.qemu.org/?p=SLOF.git;h=604d28cc3f791657414f8b21103921fa0147fc63
> 
> With these patches we support the following:
> 
> (qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
> (qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
> (qemu) device_del hp2.0
> (qemu) device_del phb2
> 
> I could run some successful tests with a fedora25 guest:
> - hotplug PHB + migrate + unplug PHB
> - hotplug PHB + hotplug PCI device + unplug PHB => PCI device gets unplugged
> - migrate before OS starts + hotplug PHB => destination uses OF phandles
> - no regression observed with older machine types
> 
> All the patches are also available here:
> 
> https://github.com/gkurz/qemu/commits/spapr-hotplug-phb
> 
> Cheers,
> 
> --
> Greg
> 
> ---
> 
> Greg Kurz (14):
>       spapr: move spapr_create_phb() to core machine code
>       spapr_pci: use memory_region_add_subregion() with DMA windows
>       spapr_iommu: use g_strdup_printf() instead of snprintf()
>       spapr_drc: use g_strdup_printf() instead of snprintf()
>       spapr_iommu: convert TCE table object to realize()
>       spapr_pci: parent the MSI memory region to the PHB
>       spapr_drc: fix realize and unrealize
>       spapr_drc: add unrealize method to physical DRC class
>       spapr_iommu: unregister vmstate at unrealize time
>       spapr: add pseries-2.11 machine type

>       spapr_pci: introduce drc_id property
>       spapr: allow guest to update the XICS phandle
>       spapr_pci: drop abusive sanity check when migrating the LSI table
>       spapr: add hotplug hooks for PHB hotplug


This one did not make it to the lists.



> 
> Michael Roth (11):
>       spapr_drc: pass object ownership to parent/owner
>       spapr_iommu: pass object ownership to parent/owner
>       pci: allow cleanup/unregistration of PCI buses
>       qdev: store DeviceState's canonical path to use when unparenting
>       spapr_pci: add PHB unrealize
>       spapr: enable PHB hotplug for pseries-2.11>       spapr: create DR connectors for PHBs


>       spapr_events: add support for phb hotplug events
>       qdev: pass an Object * to qbus_set_hotplug_handler()
>       spapr_pci: provide node start offset via spapr_populate_pci_dt()
>       spapr_pci: add ibm, my-drc-index property for PHB hotplug
> 
> Nathan Fontenot (1):
>       spapr: populate PHB DRC entries for root DT node
> 
> 
>  hw/acpi/piix4.c               |    2 
>  hw/char/virtio-serial-bus.c   |    2 
>  hw/core/bus.c                 |   11 --
>  hw/core/qdev.c                |   15 ++-
>  hw/pci/pci.c                  |   33 +++++++
>  hw/pci/pcie.c                 |    2 
>  hw/pci/shpc.c                 |    2 
>  hw/ppc/spapr.c                |  205 ++++++++++++++++++++++++++++++++++++++++-
>  hw/ppc/spapr_drc.c            |   65 ++++++++++---
>  hw/ppc/spapr_events.c         |    3 +
>  hw/ppc/spapr_hcall.c          |   20 ++++
>  hw/ppc/spapr_iommu.c          |   22 +++-
>  hw/ppc/spapr_pci.c            |   86 +++++++++++++----
>  hw/s390x/css-bridge.c         |    2 
>  hw/s390x/s390-pci-bus.c       |    6 +
>  hw/scsi/virtio-scsi.c         |    2 
>  hw/scsi/vmw_pvscsi.c          |    2 
>  hw/usb/dev-smartcard-reader.c |    2 
>  include/hw/compat.h           |    3 +
>  include/hw/pci-host/spapr.h   |    9 +-
>  include/hw/pci/pci.h          |    3 +
>  include/hw/ppc/spapr.h        |   15 +++
>  include/hw/ppc/spapr_drc.h    |    8 ++
>  include/hw/qdev-core.h        |    4 -
>  24 files changed, 446 insertions(+), 78 deletions(-)
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 01/26] spapr: move spapr_create_phb() to core machine code

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]

On Wed, Jul 26, 2017 at 01:32:41PM +1000, Alexey Kardashevskiy wrote:
> On 26/07/17 03:58, Greg Kurz wrote:
> > This function is only used when creating the default PHB. Let's rename
> > it and move it to the core machine code for clarity.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr.c              |   13 ++++++++++++-
> >  hw/ppc/spapr_pci.c          |   11 -----------
> >  include/hw/pci-host/spapr.h |    2 --
> >  3 files changed, 12 insertions(+), 14 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 2a3e53d5d58c..53969315ac24 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2142,6 +2142,17 @@ static void spapr_init_cpus(sPAPRMachineState *spapr)
> >      g_free(type);
> >  }
> >  
> > +static PCIHostState *spapr_create_default_phb(sPAPRMachineState *spapr)
> 
> 
> Nit: you could get rid of the *spapr here.
> 
> Anyway,
> 
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 02/26] spapr_pci: use memory_region_add_subregion() with DMA windows

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Wed, Jul 26, 2017 at 01:33:04PM +1000, Alexey Kardashevskiy wrote:
> On 26/07/17 03:58, Greg Kurz wrote:
> > Passing a null priority to memory_region_add_subregion_overlap() is
> > strictly equivalent to calling memory_region_add_subregion().
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> 
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

> 
> > ---
> >  hw/ppc/spapr_pci.c |    4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index 70a99a032267..3fe7f3145467 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1752,8 +1752,8 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >                         i, sphb->dtbusname);
> >              return;
> >          }
> > -        memory_region_add_subregion_overlap(&sphb->iommu_root, 0,
> > -                                            spapr_tce_get_iommu(tcet), 0);
> > +        memory_region_add_subregion(&sphb->iommu_root, 0,
> > +                                    spapr_tce_get_iommu(tcet));
> >      }
> >  
> >      sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);
> > 
> > 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 02/26] spapr_pci: use memory_region_add_subregion() with DMA windows

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

On Wed, Jul 26, 2017 at 01:33:04PM +1000, Alexey Kardashevskiy wrote:
> On 26/07/17 03:58, Greg Kurz wrote:
> > Passing a null priority to memory_region_add_subregion_overlap() is
> > strictly equivalent to calling memory_region_add_subregion().
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> 
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

In fact, this cleanup makes sense on its own.  So, I've started a
ppc-for-2.11 branch and applied this to it.

> 
> > ---
> >  hw/ppc/spapr_pci.c |    4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index 70a99a032267..3fe7f3145467 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1752,8 +1752,8 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >                         i, sphb->dtbusname);
> >              return;
> >          }
> > -        memory_region_add_subregion_overlap(&sphb->iommu_root, 0,
> > -                                            spapr_tce_get_iommu(tcet), 0);
> > +        memory_region_add_subregion(&sphb->iommu_root, 0,
> > +                                    spapr_tce_get_iommu(tcet));
> >      }
> >  
> >      sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);
> > 
> > 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 03/26] spapr_iommu: use g_strdup_printf() instead of snprintf()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2932 bytes --]

On Wed, Jul 26, 2017 at 01:37:03PM +1000, Alexey Kardashevskiy wrote:
> On 26/07/17 03:58, Greg Kurz wrote:
> > Passing a stack allocated buffer of arbitrary length to snprintf()
> > without checking the return value can cause the resultant strings
> > to be silently truncated.
> 
> The strings it is touching cannot be silently truncated as
> "tce-iommu-XXXXXXXX" are shorter than 32 chars.

That's true.  But I think using strdup_printf() is more in keeping
with qemu common practice, so I've applied this to ppc-for-2.11.

> 
> 
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr_iommu.c |   13 ++++++++-----
> >  1 file changed, 8 insertions(+), 5 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> > index e614621a8317..740d42608b61 100644
> > --- a/hw/ppc/spapr_iommu.c
> > +++ b/hw/ppc/spapr_iommu.c
> > @@ -252,17 +252,19 @@ static int spapr_tce_table_realize(DeviceState *dev)
> >  {
> >      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
> >      Object *tcetobj = OBJECT(tcet);
> > -    char tmp[32];
> > +    gchar *tmp;
> >  
> >      tcet->fd = -1;
> >      tcet->need_vfio = false;
> > -    snprintf(tmp, sizeof(tmp), "tce-root-%x", tcet->liobn);
> > +    tmp = g_strdup_printf("tce-root-%x", tcet->liobn);
> >      memory_region_init(&tcet->root, tcetobj, tmp, UINT64_MAX);
> > +    g_free(tmp);
> >  
> > -    snprintf(tmp, sizeof(tmp), "tce-iommu-%x", tcet->liobn);
> > +    tmp = g_strdup_printf("tce-iommu-%x", tcet->liobn);
> >      memory_region_init_iommu(&tcet->iommu, sizeof(tcet->iommu),
> >                               TYPE_SPAPR_IOMMU_MEMORY_REGION,
> >                               tcetobj, tmp, 0);
> > +    g_free(tmp);
> >  
> >      QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list);
> >  
> > @@ -307,7 +309,7 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
> >  sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
> >  {
> >      sPAPRTCETable *tcet;
> > -    char tmp[32];
> > +    gchar *tmp;
> >  
> >      if (spapr_tce_find_by_liobn(liobn)) {
> >          error_report("Attempted to create TCE table with duplicate"
> > @@ -318,8 +320,9 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
> >      tcet = SPAPR_TCE_TABLE(object_new(TYPE_SPAPR_TCE_TABLE));
> >      tcet->liobn = liobn;
> >  
> > -    snprintf(tmp, sizeof(tmp), "tce-table-%x", liobn);
> > +    tmp = g_strdup_printf("tce-table-%x", liobn);
> >      object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
> > +    g_free(tmp);
> >  
> >      object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
> >  
> > 
> > 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2953 bytes --]

On Tue, Jul 25, 2017 at 07:58:53PM +0200, Greg Kurz wrote:
> Passing a stack allocated buffer of arbitrary length to snprintf()
> without checking the return value can cause the resultant strings
> to be silently truncated.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.

> ---
>  hw/ppc/spapr_drc.c |   15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index 15bae5c216a9..e4e8383ec7b5 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -488,7 +488,7 @@ static void realize(DeviceState *d, Error **errp)
>  {
>      sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
>      Object *root_container;
> -    char link_name[256];
> +    gchar *link_name;
>      gchar *child_name;
>      Error *err = NULL;
>  
> @@ -501,11 +501,12 @@ static void realize(DeviceState *d, Error **errp)
>       * existing in the composition tree
>       */
>      root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> -    snprintf(link_name, sizeof(link_name), "%x", spapr_drc_index(drc));
> +    link_name = g_strdup_printf("%x", spapr_drc_index(drc));
>      child_name = object_get_canonical_path_component(OBJECT(drc));
>      trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
>      object_property_add_alias(root_container, link_name,
>                                drc->owner, child_name, &err);
> +    g_free(link_name);
>      if (err) {
>          error_report_err(err);
>          object_unref(OBJECT(drc));
> @@ -521,13 +522,14 @@ static void unrealize(DeviceState *d, Error **errp)
>  {
>      sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
>      Object *root_container;
> -    char name[256];
> +    gchar *name;
>      Error *err = NULL;
>  
>      trace_spapr_drc_unrealize(spapr_drc_index(drc));
>      root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> -    snprintf(name, sizeof(name), "%x", spapr_drc_index(drc));
> +    name = g_strdup_printf("%x", spapr_drc_index(drc));
>      object_property_del(root_container, name, &err);
> +    g_free(name);
>      if (err) {
>          error_report_err(err);
>          object_unref(OBJECT(drc));
> @@ -729,10 +731,11 @@ static const TypeInfo spapr_drc_lmb_info = {
>  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
>  {
>      Object *obj;
> -    char name[256];
> +    gchar *name;
>  
> -    snprintf(name, sizeof(name), "%s/%x", DRC_CONTAINER_PATH, index);
> +    name = g_strdup_printf("%s/%x", DRC_CONTAINER_PATH, index);
>      obj = object_resolve_path(name, NULL);
> +    g_free(name);
>  
>      return !obj ? NULL : SPAPR_DR_CONNECTOR(obj);
>  }
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 05/26] spapr_iommu: convert TCE table object to realize()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

On Tue, Jul 25, 2017 at 07:59:06PM +0200, Greg Kurz wrote:
> Signed-off-by: Greg Kurz <groug@kaod.org>

Oh, wow, that was pretty messed up.

Applied to ppc-for-2.11.

> ---
>  hw/ppc/spapr_iommu.c |    6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> index 740d42608b61..d4eca2a77225 100644
> --- a/hw/ppc/spapr_iommu.c
> +++ b/hw/ppc/spapr_iommu.c
> @@ -248,7 +248,7 @@ static const VMStateDescription vmstate_spapr_tce_table = {
>      }
>  };
>  
> -static int spapr_tce_table_realize(DeviceState *dev)
> +static void spapr_tce_table_realize(DeviceState *dev, Error **errp)
>  {
>      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
>      Object *tcetobj = OBJECT(tcet);
> @@ -270,8 +270,6 @@ static int spapr_tce_table_realize(DeviceState *dev)
>  
>      vmstate_register(DEVICE(tcet), tcet->liobn, &vmstate_spapr_tce_table,
>                       tcet);
> -
> -    return 0;
>  }
>  
>  void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
> @@ -618,7 +616,7 @@ int spapr_tcet_dma_dt(void *fdt, int node_off, const char *propname,
>  static void spapr_tce_table_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> -    dc->init = spapr_tce_table_realize;
> +    dc->realize = spapr_tce_table_realize;
>      dc->reset = spapr_tce_reset;
>      dc->unrealize = spapr_tce_table_unrealize;
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 06/26] spapr_pci: parent the MSI memory region to the PHB

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1221 bytes --]

On Tue, Jul 25, 2017 at 07:59:18PM +0200, Greg Kurz wrote:
> This memory region should be owned by the PHB. This ensures the PHB
> cannot be finalized as long as the the region is guest visible, or
> used by a CPU or a device.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.

> ---
>  hw/ppc/spapr_pci.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 3fe7f3145467..4e4165b44b9a 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1703,7 +1703,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>      }
>  #endif
>  
> -    memory_region_init_io(&sphb->msiwindow, NULL, &spapr_msi_ops, spapr,
> +    memory_region_init_io(&sphb->msiwindow, OBJECT(sphb), &spapr_msi_ops, spapr,
>                            "msi", msi_window_size);
>      memory_region_add_subregion(&sphb->iommu_root, SPAPR_PCI_MSI_WINDOW,
>                                  &sphb->msiwindow);
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 07/26] spapr_drc: fix realize and unrealize

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2664 bytes --]

On Tue, Jul 25, 2017 at 07:59:31PM +0200, Greg Kurz wrote:
> If object_property_add_alias() returns an error in realize(), we should
> propagate it to the caller and certainly not unref the DRC.
> 
> Same thing goes for unrealize(). Since object_property_del() is the last
> call, we can even get rid of the intermediate Error *.
> 
> And finally, unrealize() should undo all registrations performed by
> realize().
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

I've applied this to ppc-for-2.11, but this looks like it could be a
real bug fix.  So I'm wondering if I should push it for 2.10 (we're in
hard freeze, but bugfixes can still be applied).

> ---
>  hw/ppc/spapr_drc.c |   15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index e4e8383ec7b5..d72453bcb42f 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -506,12 +506,12 @@ static void realize(DeviceState *d, Error **errp)
>      trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
>      object_property_add_alias(root_container, link_name,
>                                drc->owner, child_name, &err);
> +    g_free(child_name);
>      g_free(link_name);
>      if (err) {
> -        error_report_err(err);
> -        object_unref(OBJECT(drc));
> +        error_propagate(errp, err);
> +        return;
>      }
> -    g_free(child_name);
>      vmstate_register(DEVICE(drc), spapr_drc_index(drc), &vmstate_spapr_drc,
>                       drc);
>      qemu_register_reset(drc_reset, drc);
> @@ -523,17 +523,14 @@ static void unrealize(DeviceState *d, Error **errp)
>      sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
>      Object *root_container;
>      gchar *name;
> -    Error *err = NULL;
>  
>      trace_spapr_drc_unrealize(spapr_drc_index(drc));
> +    qemu_unregister_reset(drc_reset, drc);
> +    vmstate_unregister(DEVICE(drc), &vmstate_spapr_drc, drc);
>      root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
>      name = g_strdup_printf("%x", spapr_drc_index(drc));
> -    object_property_del(root_container, name, &err);
> +    object_property_del(root_container, name, errp);
>      g_free(name);
> -    if (err) {
> -        error_report_err(err);
> -        object_unref(OBJECT(drc));
> -    }
>  }
>  
>  sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 08/26] spapr_drc: add unrealize method to physical DRC class

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1986 bytes --]

On Tue, Jul 25, 2017 at 07:59:44PM +0200, Greg Kurz wrote:
> When hot-unplugging a PHB, all its PCI DRC connectors get unrealized. This
> patch adds an unrealize method to the physical DRC class, in order to undo
> registrations performed in realize_physical().
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.  In a sense this is fix that could be applied
to ppc-for-2.10, but unlike the last patch, AFAICT there's no way this
bug can be triggered in the current code.

> ---
>  hw/ppc/spapr_drc.c |   16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index d72453bcb42f..105ce85b6d04 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -626,12 +626,28 @@ static void realize_physical(DeviceState *d, Error **errp)
>      qemu_register_reset(drc_physical_reset, drcp);
>  }
>  
> +static void unrealize_physical(DeviceState *d, Error **errp)
> +{
> +    sPAPRDRCPhysical *drcp = SPAPR_DRC_PHYSICAL(d);
> +    Error *local_err = NULL;
> +
> +    unrealize(d, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +
> +    vmstate_unregister(DEVICE(drcp), &vmstate_spapr_drc_physical, drcp);
> +    qemu_unregister_reset(drc_physical_reset, drcp);
> +}
> +
>  static void spapr_drc_physical_class_init(ObjectClass *k, void *data)
>  {
>      DeviceClass *dk = DEVICE_CLASS(k);
>      sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
>  
>      dk->realize = realize_physical;
> +    dk->unrealize = unrealize_physical;
>      drck->dr_entity_sense = physical_entity_sense;
>      drck->isolate = drc_isolate_physical;
>      drck->unisolate = drc_unisolate_physical;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 09/26] spapr_drc: pass object ownership to parent/owner

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1437 bytes --]

On Tue, Jul 25, 2017 at 07:59:56PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> DRC objects attach themselves to an owner as a child
> property. unref afterward to allow them to be finalized
> when their owner is finalized.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.

> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> ---
>  hw/ppc/spapr_drc.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index 105ce85b6d04..eb8024d37c54 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -544,6 +544,7 @@ sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
>      prop_name = g_strdup_printf("dr-connector[%"PRIu32"]",
>                                  spapr_drc_index(drc));
>      object_property_add_child(owner, prop_name, OBJECT(drc), NULL);
> +    object_unref(OBJECT(drc));
>      object_property_set_bool(OBJECT(drc), true, "realized", NULL);
>      g_free(prop_name);
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 10/26] spapr_iommu: pass object ownership to parent/owner

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]

On Tue, Jul 25, 2017 at 08:00:09PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> TCE table objects attach themselves to an owner as a child
> property. unref afterward to allow them to be finalized
> when their owner is finalized.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.

> ---
> Changes since RFC:
> - s/DRC/TCE table/ in changelog
> ---
>  hw/ppc/spapr_iommu.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> index d4eca2a77225..8ec4e7a40834 100644
> --- a/hw/ppc/spapr_iommu.c
> +++ b/hw/ppc/spapr_iommu.c
> @@ -321,6 +321,7 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
>      tmp = g_strdup_printf("tce-table-%x", liobn);
>      object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
>      g_free(tmp);
> +    object_unref(OBJECT(tcet));
>  
>      object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 05/26] spapr_iommu: convert TCE table object to realize()

[Alexey Kardashevskiy]

On 26/07/17 03:59, Greg Kurz wrote:
> Signed-off-by: Greg Kurz <groug@kaod.org>

Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

> ---
>  hw/ppc/spapr_iommu.c |    6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> index 740d42608b61..d4eca2a77225 100644
> --- a/hw/ppc/spapr_iommu.c
> +++ b/hw/ppc/spapr_iommu.c
> @@ -248,7 +248,7 @@ static const VMStateDescription vmstate_spapr_tce_table = {
>      }
>  };
>  
> -static int spapr_tce_table_realize(DeviceState *dev)
> +static void spapr_tce_table_realize(DeviceState *dev, Error **errp)
>  {
>      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
>      Object *tcetobj = OBJECT(tcet);
> @@ -270,8 +270,6 @@ static int spapr_tce_table_realize(DeviceState *dev)
>  
>      vmstate_register(DEVICE(tcet), tcet->liobn, &vmstate_spapr_tce_table,
>                       tcet);
> -
> -    return 0;
>  }
>  
>  void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
> @@ -618,7 +616,7 @@ int spapr_tcet_dma_dt(void *fdt, int node_off, const char *propname,
>  static void spapr_tce_table_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> -    dc->init = spapr_tce_table_realize;
> +    dc->realize = spapr_tce_table_realize;
>      dc->reset = spapr_tce_reset;
>      dc->unrealize = spapr_tce_table_unrealize;
>  
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 11/26] spapr_iommu: unregister vmstate at unrealize time

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

On Tue, Jul 25, 2017 at 08:00:22PM +0200, Greg Kurz wrote:
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.

> ---
>  hw/ppc/spapr_iommu.c |    2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> index 8ec4e7a40834..40a44c563824 100644
> --- a/hw/ppc/spapr_iommu.c
> +++ b/hw/ppc/spapr_iommu.c
> @@ -374,6 +374,8 @@ static void spapr_tce_table_unrealize(DeviceState *dev, Error **errp)
>  {
>      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
>  
> +    vmstate_unregister(DEVICE(tcet), &vmstate_spapr_tce_table, tcet);
> +
>      QLIST_REMOVE(tcet, list);
>  
>      spapr_tce_table_disable(tcet);
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 06/26] spapr_pci: parent the MSI memory region to the PHB

[Alexey Kardashevskiy]

On 26/07/17 03:59, Greg Kurz wrote:
> This memory region should be owned by the PHB. This ensures the PHB
> cannot be finalized as long as the the region is guest visible, or
> used by a CPU or a device.

Out of curiosity - does it really ensure this? Passing a parent to
memory_region_init_io() adds a reference to a child (i.e. "msi" region),
not to the PHB object. It is probably a good thing to have an owner for
every MR anyway, I am just not sure about the commit log, what does not
work if you do not do this?


> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr_pci.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 3fe7f3145467..4e4165b44b9a 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1703,7 +1703,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>      }
>  #endif
>  
> -    memory_region_init_io(&sphb->msiwindow, NULL, &spapr_msi_ops, spapr,
> +    memory_region_init_io(&sphb->msiwindow, OBJECT(sphb), &spapr_msi_ops, spapr,
>                            "msi", msi_window_size);
>      memory_region_add_subregion(&sphb->iommu_root, SPAPR_PCI_MSI_WINDOW,
>                                  &sphb->msiwindow);
> 
> 


-- 
Alexey

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 16/26] spapr: enable PHB hotplug for pseries-2.11

[Alexey Kardashevskiy]

On 26/07/17 04:01, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> The 'dr_phb_enabled' field of that class can be set as
> part of machine-specific init code, and is then propagated
> to sPAPREnvironment to conditionally enable creation of DRC
> objects and device-tree description to facilitate hotplug
> of PHBs.
> 
> Since we can't migrate this state to older machine types,
> default the option to true and disable it for older machine
> types.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> - updated changelog and title
> - set default to true and disable for 2.10 and older
> ---
>  hw/ppc/spapr.c         |    6 ++++++
>  include/hw/ppc/spapr.h |    3 +++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 277daa4f9645..8dc505343c0f 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2382,6 +2382,8 @@ static void ppc_spapr_init(MachineState *machine)
>      /* We always have at least the nvram device on VIO */
>      spapr_create_nvram(spapr);
>  
> +    spapr->dr_phb_enabled = smc->dr_phb_enabled;
> +
>      /* Set up PCI */
>      spapr_pci_rtas_init();
>  
> @@ -3482,6 +3484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
>       * in which LMBs are represented and hot-added
>       */
>      mc->numa_mem_align_shift = 28;
> +    smc->dr_phb_enabled = true;
>  }
>  
>  static const TypeInfo spapr_machine_info = {
> @@ -3558,8 +3561,11 @@ static void spapr_machine_2_10_instance_options(MachineState *machine)
>  
>  static void spapr_machine_2_10_class_options(MachineClass *mc)
>  {
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
> +
>      spapr_machine_2_11_class_options(mc);
>      SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
> +    smc->dr_phb_enabled = false;
>  }
>  
>  DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 2a303a705c17..8004d9c2ab2c 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -59,6 +59,7 @@ struct sPAPRMachineClass {
>  
>      /*< public >*/
>      bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
> +    bool dr_phb_enabled;       /* enable dynamic-reconfig/hotplug of PHBs */
>      bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
>      const char *tcg_default_cpu; /* which (TCG) CPU to simulate by default */
>      bool pre_2_10_has_unused_icps;
> @@ -122,6 +123,8 @@ struct sPAPRMachineState {
>       * occurs during the unplug process. */
>      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
>  
> +    bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */

The one in the class seems sufficient unless we are going to allow
disabling this via the command line.



> +
>      /*< public >*/
>      char *kvm_type;
>      MemoryHotplugState hotplug_memory;
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 13/26] qdev: store DeviceState's canonical path to use when unparenting

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4571 bytes --]

On Tue, Jul 25, 2017 at 08:00:47PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> device_unparent(dev, ...) is called when a device is unparented,
> either directly, or as a result of a parent device being
> finalized, and handles some final cleanup for the device. Part
> of this includes emiting a DEVICE_DELETED QMP event to notify
> management, which includes the device's path in the composition
> tree as provided by object_get_canonical_path().
> 
> object_get_canonical_path() assumes the device is still connected
> to the machine/root container, and will assert otherwise, but
> in some situations this isn't the case:
> 
> If the parent is finalized as a result of object_unparent(), it
> will still be attached to the composition tree at the time any
> children are unparented as a result of that same call to
> object_unparent(). However, in some cases, object_unparent()
> will complete without finalizing the parent device, due to
> lingering references that won't be released till some time later.
> One such example is if the parent has MemoryRegion children (which
> take a ref on their parent), who in turn have AddressSpace's (which
> take a ref on their regions), since those AddressSpaces get cleaned
> up asynchronously by the RCU thread.
> 
> In this case qdev:device_unparent() may be called for a child Device
> that no longer has a path to the root/machine container, causing
> object_get_canonical_path() to assert.
> 
> Fix this by storing the canonical path during realize() so the
> information will still be available for device_unparent() in such
> cases.

Hm.  I'm no expert on the QOM model, but I'm not sure this is the
right approach.

I would have thought the right time to emit the DEVICE_DELETED message
would be when the device leaves the main composition tree, even if it
could be finalized later.

If we made that the case, does this problem go away?

> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> ---
>  hw/core/qdev.c         |   15 ++++++++++++---
>  include/hw/qdev-core.h |    1 +
>  2 files changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> index 606ab53c42cd..a64b35c16251 100644
> --- a/hw/core/qdev.c
> +++ b/hw/core/qdev.c
> @@ -928,6 +928,12 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
>              goto post_realize_fail;
>          }
>  
> +        /* always re-initialize since we clean up in device_unparent() instead
> +         * of unrealize()
> +         */
> +        g_free(dev->canonical_path);
> +        dev->canonical_path = object_get_canonical_path(OBJECT(dev));
> +
>          if (qdev_get_vmsd(dev)) {
>              if (vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
>                                                 dev->instance_id_alias,
> @@ -984,6 +990,7 @@ child_realize_fail:
>      }
>  
>  post_realize_fail:
> +    g_free(dev->canonical_path);
>      if (dc->unrealize) {
>          dc->unrealize(dev, NULL);
>      }
> @@ -1102,10 +1109,12 @@ static void device_unparent(Object *obj)
>  
>      /* Only send event if the device had been completely realized */
>      if (dev->pending_deleted_event) {
> -        gchar *path = object_get_canonical_path(OBJECT(dev));
> +        g_assert(dev->canonical_path);
>  
> -        qapi_event_send_device_deleted(!!dev->id, dev->id, path, &error_abort);
> -        g_free(path);
> +        qapi_event_send_device_deleted(!!dev->id, dev->id, dev->canonical_path,
> +                                       &error_abort);
> +        g_free(dev->canonical_path);
> +        dev->canonical_path = NULL;
>      }
>  
>      qemu_opts_del(dev->opts);
> diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> index ae317286a480..9237b6849ff3 100644
> --- a/include/hw/qdev-core.h
> +++ b/include/hw/qdev-core.h
> @@ -153,6 +153,7 @@ struct DeviceState {
>      /*< public >*/
>  
>      const char *id;
> +    char *canonical_path;
>      bool realized;
>      bool pending_deleted_event;
>      QemuOpts *opts;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 15/26] spapr: add pseries-2.11 machine type

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]

On Tue, Jul 25, 2017 at 08:01:12PM +0200, Greg Kurz wrote:
> Signed-off-by: Greg Kurz <groug@kaod.org>

Applied to ppc-for-2.11.  Might as well get the new machine type
template ready.

> ---
>  hw/ppc/spapr.c      |   22 ++++++++++++++++++++--
>  include/hw/compat.h |    3 +++
>  2 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 53969315ac24..277daa4f9645 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3533,18 +3533,36 @@ static const TypeInfo spapr_machine_info = {
>      type_init(spapr_machine_register_##suffix)
>  
>  /*
> + * pseries-2.11
> + */
> +static void spapr_machine_2_11_instance_options(MachineState *machine)
> +{
> +}
> +
> +static void spapr_machine_2_11_class_options(MachineClass *mc)
> +{
> +    /* Defaults for the latest behaviour inherited from the base class */
> +}
> +
> +DEFINE_SPAPR_MACHINE(2_11, "2.11", true);
> +
> +/*
>   * pseries-2.10
>   */
> +#define SPAPR_COMPAT_2_10                                              \
> +    HW_COMPAT_2_10                                                     \
> +
>  static void spapr_machine_2_10_instance_options(MachineState *machine)
>  {
>  }
>  
>  static void spapr_machine_2_10_class_options(MachineClass *mc)
>  {
> -    /* Defaults for the latest behaviour inherited from the base class */
> +    spapr_machine_2_11_class_options(mc);
> +    SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
>  }
>  
> -DEFINE_SPAPR_MACHINE(2_10, "2.10", true);
> +DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
>  
>  /*
>   * pseries-2.9
> diff --git a/include/hw/compat.h b/include/hw/compat.h
> index 08f36004dad7..3e101f8f6783 100644
> --- a/include/hw/compat.h
> +++ b/include/hw/compat.h
> @@ -1,6 +1,9 @@
>  #ifndef HW_COMPAT_H
>  #define HW_COMPAT_H
>  
> +#define HW_COMPAT_2_10 \
> +    /* empty */
> +
>  #define HW_COMPAT_2_9 \
>      {\
>          .driver   = "pci-bridge",\
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[Alexey Kardashevskiy]

On 26/07/17 04:03, Greg Kurz wrote:
> The "phandle" property of the XICS node is referenced by the "interrupt-map"
> property of each PHB node. This is used by the guest OS to setup IRQs for
> all PCI devices.
> 
> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
> this value to a SLOF specific one, which is then presented to the guest OS.
> 
> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
> by SLOF to communicate the patched phandle value back to QEMU. This value
> is then cached and preserved accross migration until machine reset.
> 
> This is required to be able to support PHB hotplug.
> 
> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> have to introduce its number even if QEMU currently doesn't implement it.
> 
> Suggested-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>

> ---
>  hw/ppc/spapr.c         |   25 +++++++++++++++++++++++--
>  hw/ppc/spapr_hcall.c   |   20 ++++++++++++++++++++
>  include/hw/ppc/spapr.h |   11 ++++++++++-
>  3 files changed, 53 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1a6cd4efeb97..90485054c2e7 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -96,8 +96,6 @@
>  
>  #define MIN_RMA_SLOF            128UL
>  
> -#define PHANDLE_XICP            0x00001111
> -
>  /* maximum number of hotpluggable PHBs */
>  #define SPAPR_DRC_MAX_PHB       256
>  
> @@ -1454,6 +1452,7 @@ static void ppc_spapr_reset(void)
>      first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
>  
>      spapr->cas_reboot = false;
> +    spapr->xics_phandle = UINT32_MAX;
>  }
>  
>  static void spapr_create_nvram(sPAPRMachineState *spapr)
> @@ -1652,6 +1651,26 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
>      },
>  };
>  
> +static bool spapr_xics_phandle_needed(void *opaque)
> +{
> +    sPAPRMachineState *spapr = opaque;
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(MACHINE(spapr));
> +
> +    /* Don't break older machine types that don't support PHB hotplug. */
> +    return smc->dr_phb_enabled && spapr->xics_phandle != UINT32_MAX;
> +}
> +
> +static const VMStateDescription vmstate_spapr_xics_phandle = {
> +    .name = "spapr_xics_phandle",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .needed = spapr_xics_phandle_needed,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT32(xics_phandle, sPAPRMachineState),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
>  static const VMStateDescription vmstate_spapr = {
>      .name = "spapr",
>      .version_id = 3,
> @@ -1671,6 +1690,7 @@ static const VMStateDescription vmstate_spapr = {
>          &vmstate_spapr_ov5_cas,
>          &vmstate_spapr_patb_entry,
>          &vmstate_spapr_pending_events,
> +        &vmstate_spapr_xics_phandle,
>          NULL
>      }
>  };
> @@ -2702,6 +2722,7 @@ static void spapr_machine_initfn(Object *obj)
>  
>      spapr->htab_fd = -1;
>      spapr->use_hotplug_event_source = true;
> +    spapr->xics_phandle = UINT32_MAX;
>      object_property_add_str(obj, "kvm-type",
>                              spapr_get_kvm_type, spapr_set_kvm_type, NULL);
>      object_property_set_description(obj, "kvm-type",
> diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
> index 72ea5a8247bf..ce8a9eb66b23 100644
> --- a/hw/ppc/spapr_hcall.c
> +++ b/hw/ppc/spapr_hcall.c
> @@ -1623,6 +1623,25 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu,
>      return H_SUCCESS;
>  }
>  
> +static target_ulong h_update_phandle(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> +                                     target_ulong opcode, target_ulong *args)
> +{
> +    target_ulong old_phandle = args[0];
> +    target_ulong new_phandle = args[1];
> +
> +    if (new_phandle >= UINT32_MAX) {
> +        return H_PARAMETER;
> +    }
> +
> +    /* We only have a "phandle" property in the XICS node at the moment. */
> +    if (old_phandle != (uint32_t) PHANDLE_XICP) {
> +        return H_PARAMETER;
> +    }
> +
> +    spapr->xics_phandle = (uint32_t) new_phandle;
> +    return H_SUCCESS;
> +}
> +
>  static spapr_hcall_fn papr_hypercall_table[(MAX_HCALL_OPCODE / 4) + 1];
>  static spapr_hcall_fn kvmppc_hypercall_table[KVMPPC_HCALL_MAX - KVMPPC_HCALL_BASE + 1];
>  
> @@ -1717,6 +1736,7 @@ static void hypercall_register_types(void)
>  
>      /* qemu/KVM-PPC specific hcalls */
>      spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
> +    spapr_register_hypercall(KVMPPC_H_UPDATE_PHANDLE, h_update_phandle);
>  
>      /* ibm,client-architecture-support support */
>      spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 8004d9c2ab2c..f09c54d5bb94 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -125,6 +125,8 @@ struct sPAPRMachineState {
>  
>      bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */
>  
> +    uint32_t xics_phandle;
> +
>      /*< public >*/
>      char *kvm_type;
>      MemoryHotplugState hotplug_memory;
> @@ -402,7 +404,9 @@ struct sPAPRMachineState {
>  #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
>  /* Client Architecture support */
>  #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
> -#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
> +#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
> +#define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4)
> +#define KVMPPC_HCALL_MAX        KVMPPC_H_UPDATE_PHANDLE
>  
>  typedef struct sPAPRDeviceTreeUpdateHeader {
>      uint32_t version_id;
> @@ -707,4 +711,9 @@ void spapr_do_system_reset_on_cpu(CPUState *cs, run_on_cpu_data arg);
>  
>  #define HTAB_SIZE(spapr)        (1ULL << ((spapr)->htab_shift))
>  
> +/* Boot time value of the "phandle" property of the "interrupt-controller"
> + * node.
> + */
> +#define PHANDLE_XICP            0x00001111
> +
>  #endif /* HW_SPAPR_H */
> 
> 


-- 
Alexey

Re: [Qemu-devel] [for-2.11 PATCH 19/26] spapr: populate PHB DRC entries for root DT node

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

On Tue, Jul 25, 2017 at 08:02:03PM +0200, Greg Kurz wrote:
> From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> 
> This add entries to the root OF node to advertise our PHBs as being
> DR-capable in accordance with PAPR specification.
> 
> Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> - use error_report
> ---
>  hw/ppc/spapr.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 5950c009ab7e..632040f35ecc 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1151,6 +1151,16 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>          exit(1);
>      }
>  
> +    if (smc->dr_phb_enabled) {
> +        int offset = fdt_path_offset(fdt, "/");

Offset of the root node is guaranteed to be 0, so you don't need fdt_path_offset().

> +        ret = spapr_drc_populate_dt(fdt, offset, NULL,
> +                                    SPAPR_DR_CONNECTOR_TYPE_PHB);
> +        if (ret < 0) {
> +            error_report("Couldn't set up PHB DR device tree properties");
> +            exit(1);
> +        }
> +    }
> +
>      return fdt;
>  }
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

Hotplugging PHBs is a machine-level operation, but PHBs reside on the
main system bus, so we register spapr machine as the handler for the
main system bus.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
- rebased against ppc-for-2.10
- converted to unplug_request
- handle drc_id at pre-plug
- reset hotplugged PHB at plug
- compatibility with older machine types
---
 hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_drc.c          |    1 
 hw/ppc/spapr_pci.c          |    2 -
 include/hw/pci-host/spapr.h |    2 +
 include/hw/ppc/spapr.h      |    1 
 5 files changed, 118 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 90485054c2e7..589f76ef9fb8 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
     register_savevm_live(NULL, "spapr/htab", -1, 1,
                          &savevm_htab_handlers, spapr);
 
+    if (spapr->dr_phb_enabled) {
+        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
+    }
+
     qemu_register_boot_set(spapr_boot_set, spapr);
 
     if (kvm_enabled()) {
@@ -3238,6 +3242,103 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                               Error **errp)
+{
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+
+    if (sphb->drc_id == (uint32_t)-1) {
+        sphb->drc_id = sphb->index;
+    }
+
+    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
+        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
+    }
+}
+
+static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                           Error **errp)
+{
+    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+    void *fdt = NULL;
+    int fdt_start_offset;
+    int fdt_size;
+    Error *local_err = NULL;
+    sPAPRDRConnector *drc;
+    uint32_t phandle;
+    int ret;
+    bool hotplugged = spapr_drc_hotplugged(dev);
+
+    if (!spapr->dr_phb_enabled) {
+        return;
+    }
+
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
+    /* hotplug hooks should check it's enabled before getting this far */
+    g_assert(drc);
+
+    if (hotplugged) {
+        if (spapr->xics_phandle == UINT32_MAX) {
+            error_setg(&local_err,
+                       "SLOF didn't update the XICS phandle. PHB hotplug cancelled");
+            goto out;
+        }
+        phandle = spapr->xics_phandle;
+
+        spapr_phb_reset(dev);
+    } else {
+        phandle = PHANDLE_XICP;
+    }
+
+    fdt = create_device_tree(&fdt_size);
+    ret = spapr_populate_pci_dt(sphb, phandle, fdt, &fdt_start_offset);
+    if (ret < 0) {
+        error_setg(&local_err, "unable to create FDT for %sPHB",
+                   dev->hotplugged ? "hotplugged " : "");
+        goto out;
+    }
+
+    if (hotplugged) {
+        /* generally SLOF creates these, for hotplug it's up to QEMU */
+        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
+    }
+
+    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
+out:
+    if (local_err) {
+        error_propagate(errp, local_err);
+        g_free(fdt);
+        return;
+    }
+
+    if (hotplugged) {
+        spapr_hotplug_req_add_by_index(drc);
+    } else if (drc) {
+        spapr_drc_reset(drc);
+    }
+}
+
+void spapr_phb_release(DeviceState *dev)
+{
+    object_unparent(OBJECT(dev));
+}
+
+static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
+                                     DeviceState *dev, Error **errp)
+{
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+    sPAPRDRConnector *drc;
+
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
+    g_assert(drc);
+
+    if (!spapr_drc_unplug_requested(drc)) {
+        spapr_drc_detach(drc);
+        spapr_hotplug_req_remove_by_index(drc);
+    }
+}
+
 static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -3284,6 +3385,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
         spapr_memory_plug(hotplug_dev, dev, node, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
         spapr_core_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        spapr_phb_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -3311,6 +3414,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
             return;
         }
         spapr_core_unplug_request(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        if (sms->dr_phb_enabled) {
+            spapr_phb_unplug_request(hotplug_dev, dev, errp);
+        } else {
+            error_setg(errp, "PHB hot unplug not supported on this machine");
+        }
     }
 }
 
@@ -3321,6 +3430,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
         spapr_memory_pre_plug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
         spapr_core_pre_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        spapr_phb_pre_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -3328,7 +3439,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
                                                  DeviceState *dev)
 {
     if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
-        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
+        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
         return HOTPLUG_HANDLER(machine);
     }
     return NULL;
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 2e1049ce61c7..845fcf70b932 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -704,6 +704,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
     drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
     drck->typename = "PHB";
     drck->drc_name_prefix = "PHB ";
+    drck->release = spapr_phb_release;
 }
 
 static const TypeInfo spapr_dr_connector_info = {
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 157867af8178..c12f71ae3e2d 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1833,7 +1833,7 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb)
                            sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT);
 }
 
-static void spapr_phb_reset(DeviceState *qdev)
+void spapr_phb_reset(DeviceState *qdev)
 {
     sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(qdev);
 
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 7837fb0b1110..15799cee4280 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -120,6 +120,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
 
 void spapr_pci_rtas_init(void);
 
+void spapr_phb_reset(DeviceState *qdev);
+
 sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *spapr, uint64_t buid);
 PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid,
                               uint32_t config_addr);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index f09c54d5bb94..a2f6782bdbbf 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -673,6 +673,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
 /* CPU and LMB DRC release callbacks. */
 void spapr_core_release(DeviceState *dev);
 void spapr_lmb_release(DeviceState *dev);
+void spapr_phb_release(DeviceState *dev);
 
 void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
 int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);

Re: [Qemu-devel] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 5115 bytes --]

On Wed, 26 Jul 2017 13:44:55 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 03:57, Greg Kurz wrote:
> > This series is based on patches from Michel Roth posted in 2015:
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg04246.html
> > 
> > It addresses comments made during the RFC review, and also provides a bunch
> > of preliminary cleanup/fix patches. Since we have reached hard-freeze, this
> > feature is provided by a new pseries-2.11 machine type, introduced by this
> > series. It is based on David Gibson's ppc-for-2.10 branch, and I believe some
> > of the preliminary fixes are eligible for 2.10.
> > 
> > Note that it also requires an updated SLOF that supports a new private hcall:
> > KVMPPC_H_UPDATE_PHANDLE. This is needed because SLOF changes FDT phandles to
> > Open Firmware phandles. Since the guest only sees the latter, QEMU must use
> > the updated value when populating the FDT for the hotplugged PHB (otherwise
> > the guest can't setup IRQs for the PCI devices). SLOF part is already upstream:
> > 
> > http://git.qemu.org/?p=SLOF.git;h=604d28cc3f791657414f8b21103921fa0147fc63
> > 
> > With these patches we support the following:
> > 
> > (qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
> > (qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
> > (qemu) device_del hp2.0
> > (qemu) device_del phb2
> > 
> > I could run some successful tests with a fedora25 guest:
> > - hotplug PHB + migrate + unplug PHB
> > - hotplug PHB + hotplug PCI device + unplug PHB => PCI device gets unplugged
> > - migrate before OS starts + hotplug PHB => destination uses OF phandles
> > - no regression observed with older machine types
> > 
> > All the patches are also available here:
> > 
> > https://github.com/gkurz/qemu/commits/spapr-hotplug-phb
> > 
> > Cheers,
> > 
> > --
> > Greg
> > 
> > ---
> > 
> > Greg Kurz (14):
> >       spapr: move spapr_create_phb() to core machine code
> >       spapr_pci: use memory_region_add_subregion() with DMA windows
> >       spapr_iommu: use g_strdup_printf() instead of snprintf()
> >       spapr_drc: use g_strdup_printf() instead of snprintf()
> >       spapr_iommu: convert TCE table object to realize()
> >       spapr_pci: parent the MSI memory region to the PHB
> >       spapr_drc: fix realize and unrealize
> >       spapr_drc: add unrealize method to physical DRC class
> >       spapr_iommu: unregister vmstate at unrealize time
> >       spapr: add pseries-2.11 machine type  
> 
> >       spapr_pci: introduce drc_id property
> >       spapr: allow guest to update the XICS phandle
> >       spapr_pci: drop abusive sanity check when migrating the LSI table
> >       spapr: add hotplug hooks for PHB hotplug  
> 
> 
> This one did not make it to the lists.
> 

Yeah, I've just realized that 'stg mail' failed to send this patch because
of a connection problem with OVH's mail relay... :-\

I've sent it again with --in-reply-to, so that it goes to the appropriate
thread.

> 
> 
> > 
> > Michael Roth (11):
> >       spapr_drc: pass object ownership to parent/owner
> >       spapr_iommu: pass object ownership to parent/owner
> >       pci: allow cleanup/unregistration of PCI buses
> >       qdev: store DeviceState's canonical path to use when unparenting
> >       spapr_pci: add PHB unrealize
> >       spapr: enable PHB hotplug for pseries-2.11>       spapr: create DR connectors for PHBs  
> 
> 
> >       spapr_events: add support for phb hotplug events
> >       qdev: pass an Object * to qbus_set_hotplug_handler()
> >       spapr_pci: provide node start offset via spapr_populate_pci_dt()
> >       spapr_pci: add ibm, my-drc-index property for PHB hotplug
> > 
> > Nathan Fontenot (1):
> >       spapr: populate PHB DRC entries for root DT node
> > 
> > 
> >  hw/acpi/piix4.c               |    2 
> >  hw/char/virtio-serial-bus.c   |    2 
> >  hw/core/bus.c                 |   11 --
> >  hw/core/qdev.c                |   15 ++-
> >  hw/pci/pci.c                  |   33 +++++++
> >  hw/pci/pcie.c                 |    2 
> >  hw/pci/shpc.c                 |    2 
> >  hw/ppc/spapr.c                |  205 ++++++++++++++++++++++++++++++++++++++++-
> >  hw/ppc/spapr_drc.c            |   65 ++++++++++---
> >  hw/ppc/spapr_events.c         |    3 +
> >  hw/ppc/spapr_hcall.c          |   20 ++++
> >  hw/ppc/spapr_iommu.c          |   22 +++-
> >  hw/ppc/spapr_pci.c            |   86 +++++++++++++----
> >  hw/s390x/css-bridge.c         |    2 
> >  hw/s390x/s390-pci-bus.c       |    6 +
> >  hw/scsi/virtio-scsi.c         |    2 
> >  hw/scsi/vmw_pvscsi.c          |    2 
> >  hw/usb/dev-smartcard-reader.c |    2 
> >  include/hw/compat.h           |    3 +
> >  include/hw/pci-host/spapr.h   |    9 +-
> >  include/hw/pci/pci.h          |    3 +
> >  include/hw/ppc/spapr.h        |   15 +++
> >  include/hw/ppc/spapr_drc.h    |    8 ++
> >  include/hw/qdev-core.h        |    4 -
> >  24 files changed, 446 insertions(+), 78 deletions(-)
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 01/26] spapr: move spapr_create_phb() to core machine code

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 3076 bytes --]

On Wed, 26 Jul 2017 13:32:41 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 03:58, Greg Kurz wrote:
> > This function is only used when creating the default PHB. Let's rename
> > it and move it to the core machine code for clarity.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr.c              |   13 ++++++++++++-
> >  hw/ppc/spapr_pci.c          |   11 -----------
> >  include/hw/pci-host/spapr.h |    2 --
> >  3 files changed, 12 insertions(+), 14 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 2a3e53d5d58c..53969315ac24 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2142,6 +2142,17 @@ static void spapr_init_cpus(sPAPRMachineState *spapr)
> >      g_free(type);
> >  }
> >  
> > +static PCIHostState *spapr_create_default_phb(sPAPRMachineState *spapr)  
> 
> 
> Nit: you could get rid of the *spapr here.
> 

Heh you're right, I'll do that in the next spin.

> Anyway,
> 
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> 
> 
> 
> > +{
> > +    DeviceState *dev;
> > +
> > +    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
> > +    qdev_prop_set_uint32(dev, "index", 0);
> > +    qdev_init_nofail(dev);
> > +
> > +    return PCI_HOST_BRIDGE(dev);
> > +}
> > +
> >  /* pSeries LPAR / sPAPR hardware init */
> >  static void ppc_spapr_init(MachineState *machine)
> >  {
> > @@ -2374,7 +2385,7 @@ static void ppc_spapr_init(MachineState *machine)
> >      /* Set up PCI */
> >      spapr_pci_rtas_init();
> >  
> > -    phb = spapr_create_phb(spapr, 0);
> > +    phb = spapr_create_default_phb(spapr);
> >  
> >      for (i = 0; i < nb_nics; i++) {
> >          NICInfo *nd = &nd_table[i];
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index d84abf1070a0..70a99a032267 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1982,17 +1982,6 @@ static const TypeInfo spapr_phb_info = {
> >      }
> >  };
> >  
> > -PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index)
> > -{
> > -    DeviceState *dev;
> > -
> > -    dev = qdev_create(NULL, TYPE_SPAPR_PCI_HOST_BRIDGE);
> > -    qdev_prop_set_uint32(dev, "index", index);
> > -    qdev_init_nofail(dev);
> > -
> > -    return PCI_HOST_BRIDGE(dev);
> > -}
> > -
> >  typedef struct sPAPRFDT {
> >      void *fdt;
> >      int node_off;
> > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > index 38470b2f0e5c..5a4e9686d562 100644
> > --- a/include/hw/pci-host/spapr.h
> > +++ b/include/hw/pci-host/spapr.h
> > @@ -111,8 +111,6 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
> >      return xics_get_qirq(XICS_FABRIC(spapr), phb->lsi_table[pin].irq);
> >  }
> >  
> > -PCIHostState *spapr_create_phb(sPAPRMachineState *spapr, int index);
> > -
> >  int spapr_populate_pci_dt(sPAPRPHBState *phb,
> >                            uint32_t xics_phandle,
> >                            void *fdt);
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 07/26] spapr_drc: fix realize and unrealize

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2910 bytes --]

On Wed, 26 Jul 2017 14:04:59 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Jul 25, 2017 at 07:59:31PM +0200, Greg Kurz wrote:
> > If object_property_add_alias() returns an error in realize(), we should
> > propagate it to the caller and certainly not unref the DRC.
> > 
> > Same thing goes for unrealize(). Since object_property_del() is the last
> > call, we can even get rid of the intermediate Error *.
> > 
> > And finally, unrealize() should undo all registrations performed by
> > realize().
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>  
> 
> I've applied this to ppc-for-2.11, but this looks like it could be a
> real bug fix.  So I'm wondering if I should push it for 2.10 (we're in
> hard freeze, but bugfixes can still be applied).
> 

Yeah I guess it should also be merged in 2.10 but it has a dependency
with patch 4 (spapr_drc: use g_strdup_printf() instead  of snprintf()).
If you prefer, maybe I can repost this as a standalone fix for 2.10 ?

> > ---
> >  hw/ppc/spapr_drc.c |   15 ++++++---------
> >  1 file changed, 6 insertions(+), 9 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > index e4e8383ec7b5..d72453bcb42f 100644
> > --- a/hw/ppc/spapr_drc.c
> > +++ b/hw/ppc/spapr_drc.c
> > @@ -506,12 +506,12 @@ static void realize(DeviceState *d, Error **errp)
> >      trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
> >      object_property_add_alias(root_container, link_name,
> >                                drc->owner, child_name, &err);
> > +    g_free(child_name);
> >      g_free(link_name);
> >      if (err) {
> > -        error_report_err(err);
> > -        object_unref(OBJECT(drc));
> > +        error_propagate(errp, err);
> > +        return;
> >      }
> > -    g_free(child_name);
> >      vmstate_register(DEVICE(drc), spapr_drc_index(drc), &vmstate_spapr_drc,
> >                       drc);
> >      qemu_register_reset(drc_reset, drc);
> > @@ -523,17 +523,14 @@ static void unrealize(DeviceState *d, Error **errp)
> >      sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
> >      Object *root_container;
> >      gchar *name;
> > -    Error *err = NULL;
> >  
> >      trace_spapr_drc_unrealize(spapr_drc_index(drc));
> > +    qemu_unregister_reset(drc_reset, drc);
> > +    vmstate_unregister(DEVICE(drc), &vmstate_spapr_drc, drc);
> >      root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> >      name = g_strdup_printf("%x", spapr_drc_index(drc));
> > -    object_property_del(root_container, name, &err);
> > +    object_property_del(root_container, name, errp);
> >      g_free(name);
> > -    if (err) {
> > -        error_report_err(err);
> > -        object_unref(OBJECT(drc));
> > -    }
> >  }
> >  
> >  sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 03/26] spapr_iommu: use g_strdup_printf() instead of snprintf()

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2877 bytes --]

On Wed, 26 Jul 2017 13:37:03 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 03:58, Greg Kurz wrote:
> > Passing a stack allocated buffer of arbitrary length to snprintf()
> > without checking the return value can cause the resultant strings
> > to be silently truncated.  
> 
> The strings it is touching cannot be silently truncated as
> "tce-iommu-XXXXXXXX" are shorter than 32 chars.
> 

True but if the string was to be changed (unlikely, I admit) then we should
ensure the array is large enough. And anyway, this means we waste stack space,
which is suboptimal. As noted by David, it is a common practice in QEMU to use
g_strdup_printf().

> 
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr_iommu.c |   13 ++++++++-----
> >  1 file changed, 8 insertions(+), 5 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
> > index e614621a8317..740d42608b61 100644
> > --- a/hw/ppc/spapr_iommu.c
> > +++ b/hw/ppc/spapr_iommu.c
> > @@ -252,17 +252,19 @@ static int spapr_tce_table_realize(DeviceState *dev)
> >  {
> >      sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev);
> >      Object *tcetobj = OBJECT(tcet);
> > -    char tmp[32];
> > +    gchar *tmp;
> >  
> >      tcet->fd = -1;
> >      tcet->need_vfio = false;
> > -    snprintf(tmp, sizeof(tmp), "tce-root-%x", tcet->liobn);
> > +    tmp = g_strdup_printf("tce-root-%x", tcet->liobn);
> >      memory_region_init(&tcet->root, tcetobj, tmp, UINT64_MAX);
> > +    g_free(tmp);
> >  
> > -    snprintf(tmp, sizeof(tmp), "tce-iommu-%x", tcet->liobn);
> > +    tmp = g_strdup_printf("tce-iommu-%x", tcet->liobn);
> >      memory_region_init_iommu(&tcet->iommu, sizeof(tcet->iommu),
> >                               TYPE_SPAPR_IOMMU_MEMORY_REGION,
> >                               tcetobj, tmp, 0);
> > +    g_free(tmp);
> >  
> >      QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list);
> >  
> > @@ -307,7 +309,7 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio)
> >  sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
> >  {
> >      sPAPRTCETable *tcet;
> > -    char tmp[32];
> > +    gchar *tmp;
> >  
> >      if (spapr_tce_find_by_liobn(liobn)) {
> >          error_report("Attempted to create TCE table with duplicate"
> > @@ -318,8 +320,9 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn)
> >      tcet = SPAPR_TCE_TABLE(object_new(TYPE_SPAPR_TCE_TABLE));
> >      tcet->liobn = liobn;
> >  
> > -    snprintf(tmp, sizeof(tmp), "tce-table-%x", liobn);
> > +    tmp = g_strdup_printf("tce-table-%x", liobn);
> >      object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL);
> > +    g_free(tmp);
> >  
> >      object_property_set_bool(OBJECT(tcet), true, "realized", NULL);
> >  
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 13/26] qdev: store DeviceState's canonical path to use when unparenting

[Michael Roth]

Quoting David Gibson (2017-07-26 00:24:43)
> On Tue, Jul 25, 2017 at 08:00:47PM +0200, Greg Kurz wrote:
> > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > 
> > device_unparent(dev, ...) is called when a device is unparented,
> > either directly, or as a result of a parent device being
> > finalized, and handles some final cleanup for the device. Part
> > of this includes emiting a DEVICE_DELETED QMP event to notify
> > management, which includes the device's path in the composition
> > tree as provided by object_get_canonical_path().
> > 
> > object_get_canonical_path() assumes the device is still connected
> > to the machine/root container, and will assert otherwise, but
> > in some situations this isn't the case:
> > 
> > If the parent is finalized as a result of object_unparent(), it
> > will still be attached to the composition tree at the time any
> > children are unparented as a result of that same call to
> > object_unparent(). However, in some cases, object_unparent()
> > will complete without finalizing the parent device, due to
> > lingering references that won't be released till some time later.
> > One such example is if the parent has MemoryRegion children (which
> > take a ref on their parent), who in turn have AddressSpace's (which
> > take a ref on their regions), since those AddressSpaces get cleaned
> > up asynchronously by the RCU thread.
> > 
> > In this case qdev:device_unparent() may be called for a child Device
> > that no longer has a path to the root/machine container, causing
> > object_get_canonical_path() to assert.
> > 
> > Fix this by storing the canonical path during realize() so the
> > information will still be available for device_unparent() in such
> > cases.
> 
> Hm.  I'm no expert on the QOM model, but I'm not sure this is the
> right approach.
> 
> I would have thought the right time to emit the DEVICE_DELETED message
> would be when the device leaves the main composition tree, even if it
> could be finalized later.

That probably is how it should've been approached originally, since
that's also the expectation for device_del on PCIDevices currently.

However, there was a recent discussion which suggests we should move
DEVICE_DELETED to finalize to fix a race with VFIO unplug in libvirt.

  https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg06887.html

Since the QOM path will be broken by that point, we likely would need a
way to cache the path so it can be reported in the DEVICE_DELETED message.
I was actually planning on using this patch as a prereq for that. I'll
send that out soon.

> 
> If we made that the case, does this problem go away?
> 
> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10
> > ---
> >  hw/core/qdev.c         |   15 ++++++++++++---
> >  include/hw/qdev-core.h |    1 +
> >  2 files changed, 13 insertions(+), 3 deletions(-)
> > 
> > diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> > index 606ab53c42cd..a64b35c16251 100644
> > --- a/hw/core/qdev.c
> > +++ b/hw/core/qdev.c
> > @@ -928,6 +928,12 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> >              goto post_realize_fail;
> >          }
> >  
> > +        /* always re-initialize since we clean up in device_unparent() instead
> > +         * of unrealize()
> > +         */
> > +        g_free(dev->canonical_path);
> > +        dev->canonical_path = object_get_canonical_path(OBJECT(dev));
> > +
> >          if (qdev_get_vmsd(dev)) {
> >              if (vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
> >                                                 dev->instance_id_alias,
> > @@ -984,6 +990,7 @@ child_realize_fail:
> >      }
> >  
> >  post_realize_fail:
> > +    g_free(dev->canonical_path);
> >      if (dc->unrealize) {
> >          dc->unrealize(dev, NULL);
> >      }
> > @@ -1102,10 +1109,12 @@ static void device_unparent(Object *obj)
> >  
> >      /* Only send event if the device had been completely realized */
> >      if (dev->pending_deleted_event) {
> > -        gchar *path = object_get_canonical_path(OBJECT(dev));
> > +        g_assert(dev->canonical_path);
> >  
> > -        qapi_event_send_device_deleted(!!dev->id, dev->id, path, &error_abort);
> > -        g_free(path);
> > +        qapi_event_send_device_deleted(!!dev->id, dev->id, dev->canonical_path,
> > +                                       &error_abort);
> > +        g_free(dev->canonical_path);
> > +        dev->canonical_path = NULL;
> >      }
> >  
> >      qemu_opts_del(dev->opts);
> > diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> > index ae317286a480..9237b6849ff3 100644
> > --- a/include/hw/qdev-core.h
> > +++ b/include/hw/qdev-core.h
> > @@ -153,6 +153,7 @@ struct DeviceState {
> >      /*< public >*/
> >  
> >      const char *id;
> > +    char *canonical_path;
> >      bool realized;
> >      bool pending_deleted_event;
> >      QemuOpts *opts;
> > 
> 
> -- 
> David Gibson                    | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
>                                 | _way_ _around_!
> http://www.ozlabs.org/~dgibson

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 06/26] spapr_pci: parent the MSI memory region to the PHB

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2100 bytes --]

On Wed, 26 Jul 2017 14:29:26 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 03:59, Greg Kurz wrote:
> > This memory region should be owned by the PHB. This ensures the PHB
> > cannot be finalized as long as the the region is guest visible, or
> > used by a CPU or a device.  
> 
> Out of curiosity - does it really ensure this? Passing a parent to
> memory_region_init_io() adds a reference to a child (i.e. "msi" region),
> not to the PHB object. 

You're right, being owner of the MR means the PHB is parent and takes
a reference on the MR. But it also means a reference on the PHB is
taken/dropped each time the MR gets referenced/unreferenced (ie, "guest
visible or used by a CPU or a device").

> It is probably a good thing to have an owner for
> every MR anyway, I am just not sure about the commit log, what does not
> work if you do not do this?
> 

The PHB could theorically get unrealized while the MSI region is still active.
But since the associated MMIO op spapr_msi_write() only uses the machine
object and not the PHB, I admit I don't have a scenario where this could
break something...

So even if doesn't fix anything obvious, as you say, it is probably a good
thing to have an owner for every MR anyway :)

> 
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr_pci.c |    2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index 3fe7f3145467..4e4165b44b9a 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1703,7 +1703,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >      }
> >  #endif
> >  
> > -    memory_region_init_io(&sphb->msiwindow, NULL, &spapr_msi_ops, spapr,
> > +    memory_region_init_io(&sphb->msiwindow, OBJECT(sphb), &spapr_msi_ops, spapr,
> >                            "msi", msi_window_size);
> >      memory_region_add_subregion(&sphb->iommu_root, SPAPR_PCI_MSI_WINDOW,
> >                                  &sphb->msiwindow);
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 08/26] spapr_drc: add unrealize method to physical DRC class

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2040 bytes --]

On Wed, 26 Jul 2017 14:06:29 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Jul 25, 2017 at 07:59:44PM +0200, Greg Kurz wrote:
> > When hot-unplugging a PHB, all its PCI DRC connectors get unrealized. This
> > patch adds an unrealize method to the physical DRC class, in order to undo
> > registrations performed in realize_physical().
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>  
> 
> Applied to ppc-for-2.11.  In a sense this is fix that could be applied
> to ppc-for-2.10, but unlike the last patch, AFAICT there's no way this
> bug can be triggered in the current code.
> 

Correct. The current code won't unrealize DRCs.

> > ---
> >  hw/ppc/spapr_drc.c |   16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > index d72453bcb42f..105ce85b6d04 100644
> > --- a/hw/ppc/spapr_drc.c
> > +++ b/hw/ppc/spapr_drc.c
> > @@ -626,12 +626,28 @@ static void realize_physical(DeviceState *d, Error **errp)
> >      qemu_register_reset(drc_physical_reset, drcp);
> >  }
> >  
> > +static void unrealize_physical(DeviceState *d, Error **errp)
> > +{
> > +    sPAPRDRCPhysical *drcp = SPAPR_DRC_PHYSICAL(d);
> > +    Error *local_err = NULL;
> > +
> > +    unrealize(d, &local_err);
> > +    if (local_err) {
> > +        error_propagate(errp, local_err);
> > +        return;
> > +    }
> > +
> > +    vmstate_unregister(DEVICE(drcp), &vmstate_spapr_drc_physical, drcp);
> > +    qemu_unregister_reset(drc_physical_reset, drcp);
> > +}
> > +
> >  static void spapr_drc_physical_class_init(ObjectClass *k, void *data)
> >  {
> >      DeviceClass *dk = DEVICE_CLASS(k);
> >      sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> >  
> >      dk->realize = realize_physical;
> > +    dk->unrealize = unrealize_physical;
> >      drck->dr_entity_sense = physical_entity_sense;
> >      drck->isolate = drc_isolate_physical;
> >      drck->unisolate = drc_unisolate_physical;
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 16/26] spapr: enable PHB hotplug for pseries-2.11

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 3562 bytes --]

On Wed, 26 Jul 2017 14:42:23 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 04:01, Greg Kurz wrote:
> > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > 
> > The 'dr_phb_enabled' field of that class can be set as
> > part of machine-specific init code, and is then propagated
> > to sPAPREnvironment to conditionally enable creation of DRC
> > objects and device-tree description to facilitate hotplug
> > of PHBs.
> > 
> > Since we can't migrate this state to older machine types,
> > default the option to true and disable it for older machine
> > types.
> > 
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10
> > - updated changelog and title
> > - set default to true and disable for 2.10 and older
> > ---
> >  hw/ppc/spapr.c         |    6 ++++++
> >  include/hw/ppc/spapr.h |    3 +++
> >  2 files changed, 9 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 277daa4f9645..8dc505343c0f 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2382,6 +2382,8 @@ static void ppc_spapr_init(MachineState *machine)
> >      /* We always have at least the nvram device on VIO */
> >      spapr_create_nvram(spapr);
> >  
> > +    spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > +
> >      /* Set up PCI */
> >      spapr_pci_rtas_init();
> >  
> > @@ -3482,6 +3484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> >       * in which LMBs are represented and hot-added
> >       */
> >      mc->numa_mem_align_shift = 28;
> > +    smc->dr_phb_enabled = true;
> >  }
> >  
> >  static const TypeInfo spapr_machine_info = {
> > @@ -3558,8 +3561,11 @@ static void spapr_machine_2_10_instance_options(MachineState *machine)
> >  
> >  static void spapr_machine_2_10_class_options(MachineClass *mc)
> >  {
> > +    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
> > +
> >      spapr_machine_2_11_class_options(mc);
> >      SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
> > +    smc->dr_phb_enabled = false;
> >  }
> >  
> >  DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
> > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > index 2a303a705c17..8004d9c2ab2c 100644
> > --- a/include/hw/ppc/spapr.h
> > +++ b/include/hw/ppc/spapr.h
> > @@ -59,6 +59,7 @@ struct sPAPRMachineClass {
> >  
> >      /*< public >*/
> >      bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
> > +    bool dr_phb_enabled;       /* enable dynamic-reconfig/hotplug of PHBs */
> >      bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
> >      const char *tcg_default_cpu; /* which (TCG) CPU to simulate by default */
> >      bool pre_2_10_has_unused_icps;
> > @@ -122,6 +123,8 @@ struct sPAPRMachineState {
> >       * occurs during the unplug process. */
> >      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
> >  
> > +    bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */  
> 
> The one in the class seems sufficient unless we are going to allow
> disabling this via the command line.
> 

I kept the original patch but you're right indeed. I don't see why
someone would want to disable this from the cmdline, so I guess I
should just drop the per-instance flag.

Mike, any thoughts ?

> 
> 
> > +
> >      /*< public >*/
> >      char *kvm_type;
> >      MemoryHotplugState hotplug_memory;
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 19/26] spapr: populate PHB DRC entries for root DT node

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]

On Wed, 26 Jul 2017 15:47:18 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Jul 25, 2017 at 08:02:03PM +0200, Greg Kurz wrote:
> > From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> > 
> > This add entries to the root OF node to advertise our PHBs as being
> > DR-capable in accordance with PAPR specification.
> > 
> > Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10
> > - use error_report
> > ---
> >  hw/ppc/spapr.c |   10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 5950c009ab7e..632040f35ecc 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -1151,6 +1151,16 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
> >          exit(1);
> >      }
> >  
> > +    if (smc->dr_phb_enabled) {
> > +        int offset = fdt_path_offset(fdt, "/");  
> 
> Offset of the root node is guaranteed to be 0, so you don't need fdt_path_offset().
> 

Oh, I didn't know that.

BTW, there are two other places that also need to be fixed then:

$ git grep -n 'fdt_path_offset(fdt, "/")'
hw/ppc/pnv.c:95:        cpus_offset = fdt_add_subnode(fdt, fdt_path_offset(fdt, "/"),
hw/ppc/spapr.c:355:            cpus_offset = fdt_add_subnode(fdt, fdt_path_offset(fdt, "/"),

I'll send a patch.

> > +        ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > +                                    SPAPR_DR_CONNECTOR_TYPE_PHB);
> > +        if (ret < 0) {
> > +            error_report("Couldn't set up PHB DR device tree properties");
> > +            exit(1);
> > +        }
> > +    }
> > +
> >      return fdt;
> >  }
> >  
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 19/26] spapr: populate PHB DRC entries for root DT node

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2238 bytes --]

On Tue, 25 Jul 2017 15:51:21 -0500
Michael Roth <mdroth@linux.vnet.ibm.com> wrote:

> Quoting Greg Kurz (2017-07-25 13:02:03)
> > From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> > 
> > This add entries to the root OF node to advertise our PHBs as being
> > DR-capable in accordance with PAPR specification.
> > 
> > Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10
> > - use error_report
> > ---
> >  hw/ppc/spapr.c |   10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 5950c009ab7e..632040f35ecc 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -1151,6 +1151,16 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
> >          exit(1);
> >      }
> > 
> > +    if (smc->dr_phb_enabled) {
> > +        int offset = fdt_path_offset(fdt, "/");
> > +        ret = spapr_drc_populate_dt(fdt, offset, NULL,
> > +                                    SPAPR_DR_CONNECTOR_TYPE_PHB);  
> 
> This might actually clobber LMB/MEM entries added earlier since they both
> try to create the /ibm,drc* entries in / from scratch.
>   

Oh, looking at the description of fdt_setprop() in <libfdt.h>, I guess you're
probably right.

> If that's the case, we can arrange the code something like:
>   
>   uint32_t root_drc_types = 0;
>   
>   if (smc->dr_lmb_enabled) {
>       root_drc_types |= SPAPR_DR_CONNECTOR_TYPE_LMB;
>   }
>   
>   if (smc->dr_phb_enabled) {
>       root_drc_types |= SPAPR_DR_CONNECTOR_TYPE_PHB;
>   }
>   
>   ...
>   
>   if (root_drc_types) {
>     int offset = fdt_path_offset(fdt, "/");
>     ret = spapr_drc_populate_dt(fdt, offset, NULL, root_drc_types);
>   }
> 

Good idea. I'll do just that.

Thanks!

> 
> > +        if (ret < 0) {
> > +            error_report("Couldn't set up PHB DR device tree properties");
> > +            exit(1);
> > +        }
> > +    }
> > +
> >      return fdt;
> >  }
> > 
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Daniel Henrique Barboza]

I've tested the patch set using Greg's Github branch. It worked fine in 
my tests
using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
though:

1 - This is not related to this patch set per se because it is 
reproducible on master, but
I think it is interfering with this new feature.  There is a 
warning/error message in
the kernel right after SLOF that goes:

(...)
  -> smp_release_cpus()
spinning_secondaries = 0
  <- smp_release_cpus()
Linux ppc64le
#1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
failed with rc=-22
[    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
[  OK  ] Started Security Auditing Service.
(...)

I get this same message after hotplugging a PCI device with this series, 
but the
hotplug shows up in the lspci as expected:

(qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
(qemu) [  412.233441] pci 0002:00:00.0: of_irq_parse_pci: failed with rc=-22


2 - when hotplugging the same PHB for the second time (device_add phb2,
device_del phb2, device_add phb2 again) the hotplug works but dmesg got 
spammed
by the messages:

(qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
(qemu) [  378.309490] rpaphp: rpaphp_register_slot: slot[C131106] is 
already registered
[  378.309674] rpaphp: rpaphp_register_slot: slot[C131074] is already 
registered
[  378.309841] rpaphp: rpaphp_register_slot: slot[C131087] is already 
registered
[  378.310847] rpaphp: rpaphp_register_slot: slot[C131078] is already 
registered
( .... about 250+ messages like that )

Looks like device_del isn't cleaning up everything after the first hotplug.



Thanks,


Daniel


On 07/25/2017 02:57 PM, Greg Kurz wrote:
> This series is based on patches from Michel Roth posted in 2015:
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg04246.html
>
> It addresses comments made during the RFC review, and also provides a bunch
> of preliminary cleanup/fix patches. Since we have reached hard-freeze, this
> feature is provided by a new pseries-2.11 machine type, introduced by this
> series. It is based on David Gibson's ppc-for-2.10 branch, and I believe some
> of the preliminary fixes are eligible for 2.10.
>
> Note that it also requires an updated SLOF that supports a new private hcall:
> KVMPPC_H_UPDATE_PHANDLE. This is needed because SLOF changes FDT phandles to
> Open Firmware phandles. Since the guest only sees the latter, QEMU must use
> the updated value when populating the FDT for the hotplugged PHB (otherwise
> the guest can't setup IRQs for the PCI devices). SLOF part is already upstream:
>
> http://git.qemu.org/?p=SLOF.git;h=604d28cc3f791657414f8b21103921fa0147fc63
>
> With these patches we support the following:
>
> (qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
> (qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
> (qemu) device_del hp2.0
> (qemu) device_del phb2
>
> I could run some successful tests with a fedora25 guest:
> - hotplug PHB + migrate + unplug PHB
> - hotplug PHB + hotplug PCI device + unplug PHB => PCI device gets unplugged
> - migrate before OS starts + hotplug PHB => destination uses OF phandles
> - no regression observed with older machine types
>
> All the patches are also available here:
>
> https://github.com/gkurz/qemu/commits/spapr-hotplug-phb
>
> Cheers,
>
> --
> Greg
>
> ---
>
> Greg Kurz (14):
>        spapr: move spapr_create_phb() to core machine code
>        spapr_pci: use memory_region_add_subregion() with DMA windows
>        spapr_iommu: use g_strdup_printf() instead of snprintf()
>        spapr_drc: use g_strdup_printf() instead of snprintf()
>        spapr_iommu: convert TCE table object to realize()
>        spapr_pci: parent the MSI memory region to the PHB
>        spapr_drc: fix realize and unrealize
>        spapr_drc: add unrealize method to physical DRC class
>        spapr_iommu: unregister vmstate at unrealize time
>        spapr: add pseries-2.11 machine type
>        spapr_pci: introduce drc_id property
>        spapr: allow guest to update the XICS phandle
>        spapr_pci: drop abusive sanity check when migrating the LSI table
>        spapr: add hotplug hooks for PHB hotplug
>
> Michael Roth (11):
>        spapr_drc: pass object ownership to parent/owner
>        spapr_iommu: pass object ownership to parent/owner
>        pci: allow cleanup/unregistration of PCI buses
>        qdev: store DeviceState's canonical path to use when unparenting
>        spapr_pci: add PHB unrealize
>        spapr: enable PHB hotplug for pseries-2.11
>        spapr: create DR connectors for PHBs
>        spapr_events: add support for phb hotplug events
>        qdev: pass an Object * to qbus_set_hotplug_handler()
>        spapr_pci: provide node start offset via spapr_populate_pci_dt()
>        spapr_pci: add ibm, my-drc-index property for PHB hotplug
>
> Nathan Fontenot (1):
>        spapr: populate PHB DRC entries for root DT node
>
>
>   hw/acpi/piix4.c               |    2
>   hw/char/virtio-serial-bus.c   |    2
>   hw/core/bus.c                 |   11 --
>   hw/core/qdev.c                |   15 ++-
>   hw/pci/pci.c                  |   33 +++++++
>   hw/pci/pcie.c                 |    2
>   hw/pci/shpc.c                 |    2
>   hw/ppc/spapr.c                |  205 ++++++++++++++++++++++++++++++++++++++++-
>   hw/ppc/spapr_drc.c            |   65 ++++++++++---
>   hw/ppc/spapr_events.c         |    3 +
>   hw/ppc/spapr_hcall.c          |   20 ++++
>   hw/ppc/spapr_iommu.c          |   22 +++-
>   hw/ppc/spapr_pci.c            |   86 +++++++++++++----
>   hw/s390x/css-bridge.c         |    2
>   hw/s390x/s390-pci-bus.c       |    6 +
>   hw/scsi/virtio-scsi.c         |    2
>   hw/scsi/vmw_pvscsi.c          |    2
>   hw/usb/dev-smartcard-reader.c |    2
>   include/hw/compat.h           |    3 +
>   include/hw/pci-host/spapr.h   |    9 +-
>   include/hw/pci/pci.h          |    3 +
>   include/hw/ppc/spapr.h        |   15 +++
>   include/hw/ppc/spapr_drc.h    |    8 ++
>   include/hw/qdev-core.h        |    4 -
>   24 files changed, 446 insertions(+), 78 deletions(-)
>
>

Re: [Qemu-devel] [for-2.11 PATCH 07/26] spapr_drc: fix realize and unrealize

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3361 bytes --]

On Wed, Jul 26, 2017 at 11:36:43AM +0200, Greg Kurz wrote:
> On Wed, 26 Jul 2017 14:04:59 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Tue, Jul 25, 2017 at 07:59:31PM +0200, Greg Kurz wrote:
> > > If object_property_add_alias() returns an error in realize(), we should
> > > propagate it to the caller and certainly not unref the DRC.
> > > 
> > > Same thing goes for unrealize(). Since object_property_del() is the last
> > > call, we can even get rid of the intermediate Error *.
> > > 
> > > And finally, unrealize() should undo all registrations performed by
> > > realize().
> > > 
> > > Signed-off-by: Greg Kurz <groug@kaod.org>  
> > 
> > I've applied this to ppc-for-2.11, but this looks like it could be a
> > real bug fix.  So I'm wondering if I should push it for 2.10 (we're in
> > hard freeze, but bugfixes can still be applied).
> > 
> 
> Yeah I guess it should also be merged in 2.10 but it has a dependency
> with patch 4 (spapr_drc: use g_strdup_printf() instead  of snprintf()).
> If you prefer, maybe I can repost this as a standalone fix for 2.10
> ?

Yes, that would be helpful, thank you.

> 
> > > ---
> > >  hw/ppc/spapr_drc.c |   15 ++++++---------
> > >  1 file changed, 6 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > index e4e8383ec7b5..d72453bcb42f 100644
> > > --- a/hw/ppc/spapr_drc.c
> > > +++ b/hw/ppc/spapr_drc.c
> > > @@ -506,12 +506,12 @@ static void realize(DeviceState *d, Error **errp)
> > >      trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
> > >      object_property_add_alias(root_container, link_name,
> > >                                drc->owner, child_name, &err);
> > > +    g_free(child_name);
> > >      g_free(link_name);
> > >      if (err) {
> > > -        error_report_err(err);
> > > -        object_unref(OBJECT(drc));
> > > +        error_propagate(errp, err);
> > > +        return;
> > >      }
> > > -    g_free(child_name);
> > >      vmstate_register(DEVICE(drc), spapr_drc_index(drc), &vmstate_spapr_drc,
> > >                       drc);
> > >      qemu_register_reset(drc_reset, drc);
> > > @@ -523,17 +523,14 @@ static void unrealize(DeviceState *d, Error **errp)
> > >      sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
> > >      Object *root_container;
> > >      gchar *name;
> > > -    Error *err = NULL;
> > >  
> > >      trace_spapr_drc_unrealize(spapr_drc_index(drc));
> > > +    qemu_unregister_reset(drc_reset, drc);
> > > +    vmstate_unregister(DEVICE(drc), &vmstate_spapr_drc, drc);
> > >      root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> > >      name = g_strdup_printf("%x", spapr_drc_index(drc));
> > > -    object_property_del(root_container, name, &err);
> > > +    object_property_del(root_container, name, errp);
> > >      g_free(name);
> > > -    if (err) {
> > > -        error_report_err(err);
> > > -        object_unref(OBJECT(drc));
> > > -    }
> > >  }
> > >  
> > >  sPAPRDRConnector *spapr_dr_connector_new(Object *owner, const char *type,
> > >   
> > 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Alexey Kardashevskiy]

On 26/07/17 18:40, Greg Kurz wrote:
> Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> main system bus, so we register spapr machine as the handler for the
> main system bus.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> - rebased against ppc-for-2.10
> - converted to unplug_request
> - handle drc_id at pre-plug
> - reset hotplugged PHB at plug
> - compatibility with older machine types
> ---
>  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
>  hw/ppc/spapr_drc.c          |    1 
>  hw/ppc/spapr_pci.c          |    2 -
>  include/hw/pci-host/spapr.h |    2 +
>  include/hw/ppc/spapr.h      |    1 
>  5 files changed, 118 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 90485054c2e7..589f76ef9fb8 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
>      register_savevm_live(NULL, "spapr/htab", -1, 1,
>                           &savevm_htab_handlers, spapr);
>  
> +    if (spapr->dr_phb_enabled) {
> +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> +    }
> +
>      qemu_register_boot_set(spapr_boot_set, spapr);
>  
>      if (kvm_enabled()) {
> @@ -3238,6 +3242,103 @@ out:
>      error_propagate(errp, local_err);
>  }
>  
> +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                               Error **errp)
> +{
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +
> +    if (sphb->drc_id == (uint32_t)-1) {
> +        sphb->drc_id = sphb->index;
> +    }
> +
> +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> +    }


sphb->index in considered 16bits in the existing code (even though it is
defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
same limit for both, either 256 or 65536 is fine for me.

It is actually a bit weird - it is possible to completely configure few
PHBs in the command line so they will have index==-1 but PCI hotplug code -
spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
this and just does (sphb->index << 16). May be just ditch drc_id, enforce
index not to be -1 and use it as drc_id?



> +}
> +
> +static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                           Error **errp)
> +{
> +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +    void *fdt = NULL;
> +    int fdt_start_offset;
> +    int fdt_size;
> +    Error *local_err = NULL;
> +    sPAPRDRConnector *drc;
> +    uint32_t phandle;
> +    int ret;
> +    bool hotplugged = spapr_drc_hotplugged(dev);
> +
> +    if (!spapr->dr_phb_enabled) {
> +        return;
> +    }
> +
> +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> +    /* hotplug hooks should check it's enabled before getting this far */
> +    g_assert(drc);
> +
> +    if (hotplugged) {
> +        if (spapr->xics_phandle == UINT32_MAX) {
> +            error_setg(&local_err,
> +                       "SLOF didn't update the XICS phandle. PHB hotplug cancelled");
> +            goto out;
> +        }
> +        phandle = spapr->xics_phandle;
> +
> +        spapr_phb_reset(dev);


It could be DEVICE_GET_CLASS(dev)->reset(dev) without exporting
spapr_phb_reset. Not sure how this fits into the current QEMU coding style
though.



> +    } else {
> +        phandle = PHANDLE_XICP;
> +    }
> +
> +    fdt = create_device_tree(&fdt_size);
> +    ret = spapr_populate_pci_dt(sphb, phandle, fdt, &fdt_start_offset);
> +    if (ret < 0) {
> +        error_setg(&local_err, "unable to create FDT for %sPHB",
> +                   dev->hotplugged ? "hotplugged " : "");
> +        goto out;
> +    }
> +
> +    if (hotplugged) {
> +        /* generally SLOF creates these, for hotplug it's up to QEMU */
> +        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
> +    }
> +
> +    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
> +out:
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        g_free(fdt);
> +        return;
> +    }
> +
> +    if (hotplugged) {
> +        spapr_hotplug_req_add_by_index(drc);
> +    } else if (drc) {
> +        spapr_drc_reset(drc);
> +    }
> +}
> +
> +void spapr_phb_release(DeviceState *dev)
> +{
> +    object_unparent(OBJECT(dev));
> +}
> +
> +static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
> +                                     DeviceState *dev, Error **errp)
> +{
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +    sPAPRDRConnector *drc;
> +
> +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> +    g_assert(drc);
> +
> +    if (!spapr_drc_unplug_requested(drc)) {
> +        spapr_drc_detach(drc);
> +        spapr_hotplug_req_remove_by_index(drc);
> +    }
> +}
> +
>  static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
>                                        DeviceState *dev, Error **errp)
>  {
> @@ -3284,6 +3385,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
>          spapr_memory_plug(hotplug_dev, dev, node, errp);
>      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
>          spapr_core_plug(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        spapr_phb_plug(hotplug_dev, dev, errp);
>      }
>  }
>  
> @@ -3311,6 +3414,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
>              return;
>          }
>          spapr_core_unplug_request(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        if (sms->dr_phb_enabled) {
> +            spapr_phb_unplug_request(hotplug_dev, dev, errp);
> +        } else {
> +            error_setg(errp, "PHB hot unplug not supported on this machine");
> +        }
>      }
>  }
>  
> @@ -3321,6 +3430,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
>          spapr_memory_pre_plug(hotplug_dev, dev, errp);
>      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
>          spapr_core_pre_plug(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        spapr_phb_pre_plug(hotplug_dev, dev, errp);
>      }
>  }
>  
> @@ -3328,7 +3439,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
>                                                   DeviceState *dev)
>  {
>      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> -        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
> +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
>          return HOTPLUG_HANDLER(machine);
>      }
>      return NULL;
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index 2e1049ce61c7..845fcf70b932 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -704,6 +704,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
>      drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
>      drck->typename = "PHB";
>      drck->drc_name_prefix = "PHB ";
> +    drck->release = spapr_phb_release;
>  }
>  
>  static const TypeInfo spapr_dr_connector_info = {
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 157867af8178..c12f71ae3e2d 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1833,7 +1833,7 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb)
>                             sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT);
>  }
>  
> -static void spapr_phb_reset(DeviceState *qdev)
> +void spapr_phb_reset(DeviceState *qdev)
>  {
>      sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(qdev);
>  
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 7837fb0b1110..15799cee4280 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -120,6 +120,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
>  
>  void spapr_pci_rtas_init(void);
>  
> +void spapr_phb_reset(DeviceState *qdev);
> +
>  sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *spapr, uint64_t buid);
>  PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid,
>                                uint32_t config_addr);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index f09c54d5bb94..a2f6782bdbbf 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -673,6 +673,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
>  /* CPU and LMB DRC release callbacks. */
>  void spapr_core_release(DeviceState *dev);
>  void spapr_lmb_release(DeviceState *dev);
> +void spapr_phb_release(DeviceState *dev);
>  
>  void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
>  int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);
> 
> 


-- 
Alexey

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 16/26] spapr: enable PHB hotplug for pseries-2.11

[Michael Roth]

Quoting Greg Kurz (2017-07-26 09:32:27)
> On Wed, 26 Jul 2017 14:42:23 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > On 26/07/17 04:01, Greg Kurz wrote:
> > > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > 
> > > The 'dr_phb_enabled' field of that class can be set as
> > > part of machine-specific init code, and is then propagated
> > > to sPAPREnvironment to conditionally enable creation of DRC
> > > objects and device-tree description to facilitate hotplug
> > > of PHBs.
> > > 
> > > Since we can't migrate this state to older machine types,
> > > default the option to true and disable it for older machine
> > > types.
> > > 
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > ---
> > > Changes since RFC:
> > > - rebased against ppc-for-2.10
> > > - updated changelog and title
> > > - set default to true and disable for 2.10 and older
> > > ---
> > >  hw/ppc/spapr.c         |    6 ++++++
> > >  include/hw/ppc/spapr.h |    3 +++
> > >  2 files changed, 9 insertions(+)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 277daa4f9645..8dc505343c0f 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -2382,6 +2382,8 @@ static void ppc_spapr_init(MachineState *machine)
> > >      /* We always have at least the nvram device on VIO */
> > >      spapr_create_nvram(spapr);
> > >  
> > > +    spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > > +
> > >      /* Set up PCI */
> > >      spapr_pci_rtas_init();
> > >  
> > > @@ -3482,6 +3484,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > >       * in which LMBs are represented and hot-added
> > >       */
> > >      mc->numa_mem_align_shift = 28;
> > > +    smc->dr_phb_enabled = true;
> > >  }
> > >  
> > >  static const TypeInfo spapr_machine_info = {
> > > @@ -3558,8 +3561,11 @@ static void spapr_machine_2_10_instance_options(MachineState *machine)
> > >  
> > >  static void spapr_machine_2_10_class_options(MachineClass *mc)
> > >  {
> > > +    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
> > > +
> > >      spapr_machine_2_11_class_options(mc);
> > >      SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_10);
> > > +    smc->dr_phb_enabled = false;
> > >  }
> > >  
> > >  DEFINE_SPAPR_MACHINE(2_10, "2.10", false);
> > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > > index 2a303a705c17..8004d9c2ab2c 100644
> > > --- a/include/hw/ppc/spapr.h
> > > +++ b/include/hw/ppc/spapr.h
> > > @@ -59,6 +59,7 @@ struct sPAPRMachineClass {
> > >  
> > >      /*< public >*/
> > >      bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
> > > +    bool dr_phb_enabled;       /* enable dynamic-reconfig/hotplug of PHBs */
> > >      bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
> > >      const char *tcg_default_cpu; /* which (TCG) CPU to simulate by default */
> > >      bool pre_2_10_has_unused_icps;
> > > @@ -122,6 +123,8 @@ struct sPAPRMachineState {
> > >       * occurs during the unplug process. */
> > >      QTAILQ_HEAD(, sPAPRDIMMState) pending_dimm_unplugs;
> > >  
> > > +    bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */  
> > 
> > The one in the class seems sufficient unless we are going to allow
> > disabling this via the command line.
> > 
> 
> I kept the original patch but you're right indeed. I don't see why
> someone would want to disable this from the cmdline, so I guess I
> should just drop the per-instance flag.
> 
> Mike, any thoughts ?

Not sure why I did it like that originally, and can't think of a good reason
we'd want to disable it specifically either, so I agree we should probably
just drop it.

> 
> > 
> > 
> > > +
> > >      /*< public >*/
> > >      char *kvm_type;
> > >      MemoryHotplugState hotplug_memory;
> > > 
> > >   
> > 
> > 
> 

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 8559 bytes --]

On Wed, 26 Jul 2017 17:31:17 -0300
Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote:

> I've tested the patch set using Greg's Github branch. It worked fine in 
> my tests
> using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
> though:
> 
> 1 - This is not related to this patch set per se because it is 
> reproducible on master, but
> I think it is interfering with this new feature.  There is a 
> warning/error message in
> the kernel right after SLOF that goes:
> 
> (...)
>   -> smp_release_cpus()  
> spinning_secondaries = 0
>   <- smp_release_cpus()
> Linux ppc64le
> #1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
> failed with rc=-22
> [    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
> [  OK  ] Started Security Auditing Service.
> (...)
> 

This is a regression in QEMU master introduced by this commit:

commit b87680427e8a3ff682f66514e99a8344e7437247
Author: Cédric Le Goater <clg@kaod.org>
Date:   Wed Jul 5 19:13:15 2017 +0200

    spapr: populate device tree depending on XIVE_EXPLOIT option
    
    When XIVE is supported, the device tree should be populated
    accordingly and the XIVE memory regions mapped to activate MMIOs.
    
    Depending on the design we choose, we could also allocate different
    ICS and ICP objects, or switch between objects. This needs to be
    discussed.
    
    Signed-off-by: Cédric Le Goater <clg@kaod.org>
    Signed-off-by: David Gibson <david@gibson.dropbear.id.au>

It is very similar to the issue that motivated the new KVMPPC_H_UPDATE_PHANDLE
hcall (see patch 24 and 26 in this series):

- QEMU creates an "interrupt-controller" node with a phandle property
  with the value 0x1111
- QEMU passes the FDT to SLOF
- SLOF converts all references to the phandle to an SLOF internal value

=> from now on (ie, until the next machine reset), the guest only knows
   the OF phandle.

- during CAS, if we go XICS, we send back an updated FDT with the
  phandle of the "interrupt-controller" node reverted to 0x1111

=> the guest complains because all cold-plugged devices nodes refer
   to the OF phandle, not 0x1111

A solution is to use the value set by KVMPPC_H_UPDATE_PHANDLE during CAS
instead of 0x1111. I could verify it makes the guest warning disappear.

I'll send a dedicated patchset to fix this in 2.10.

> I get this same message after hotplugging a PCI device with this series, 
> but the
> hotplug shows up in the lspci as expected:
> 
> (qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
> (qemu) [  412.233441] pci 0002:00:00.0: of_irq_parse_pci: failed with rc=-22
> 

This is the same error (the devices plugged in the new PHB all refer to
the OF phandle).

> 
> 2 - when hotplugging the same PHB for the second time (device_add phb2,
> device_del phb2, device_add phb2 again) the hotplug works but dmesg got 
> spammed
> by the messages:
> 
> (qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
> (qemu) [  378.309490] rpaphp: rpaphp_register_slot: slot[C131106] is 
> already registered
> [  378.309674] rpaphp: rpaphp_register_slot: slot[C131074] is already 
> registered
> [  378.309841] rpaphp: rpaphp_register_slot: slot[C131087] is already 
> registered
> [  378.310847] rpaphp: rpaphp_register_slot: slot[C131078] is already 
> registered

Yeah, I saw that but I haven't investigated that yet.

> ( .... about 250+ messages like that )
> 

The current default is to have 256 slots per PHB.

> Looks like device_del isn't cleaning up everything after the first hotplug.
> 

Or maybe the guest part (rpaphp or drmgr) ?

> 
> 
> Thanks,
> 


Thanks very much for the testing! :)

Cheers,

--
Greg

> 
> Daniel
> 
> 
> On 07/25/2017 02:57 PM, Greg Kurz wrote:
> > This series is based on patches from Michel Roth posted in 2015:
> >
> > https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg04246.html
> >
> > It addresses comments made during the RFC review, and also provides a bunch
> > of preliminary cleanup/fix patches. Since we have reached hard-freeze, this
> > feature is provided by a new pseries-2.11 machine type, introduced by this
> > series. It is based on David Gibson's ppc-for-2.10 branch, and I believe some
> > of the preliminary fixes are eligible for 2.10.
> >
> > Note that it also requires an updated SLOF that supports a new private hcall:
> > KVMPPC_H_UPDATE_PHANDLE. This is needed because SLOF changes FDT phandles to
> > Open Firmware phandles. Since the guest only sees the latter, QEMU must use
> > the updated value when populating the FDT for the hotplugged PHB (otherwise
> > the guest can't setup IRQs for the PCI devices). SLOF part is already upstream:
> >
> > http://git.qemu.org/?p=SLOF.git;h=604d28cc3f791657414f8b21103921fa0147fc63
> >
> > With these patches we support the following:
> >
> > (qemu) device_add spapr-pci-host-bridge,index=2,id=phb2
> > (qemu) device_add virtio-net-pci,id=hp2.0,bus=phb2.0
> > (qemu) device_del hp2.0
> > (qemu) device_del phb2
> >
> > I could run some successful tests with a fedora25 guest:
> > - hotplug PHB + migrate + unplug PHB
> > - hotplug PHB + hotplug PCI device + unplug PHB => PCI device gets unplugged
> > - migrate before OS starts + hotplug PHB => destination uses OF phandles
> > - no regression observed with older machine types
> >
> > All the patches are also available here:
> >
> > https://github.com/gkurz/qemu/commits/spapr-hotplug-phb
> >
> > Cheers,
> >
> > --
> > Greg
> >
> > ---
> >
> > Greg Kurz (14):
> >        spapr: move spapr_create_phb() to core machine code
> >        spapr_pci: use memory_region_add_subregion() with DMA windows
> >        spapr_iommu: use g_strdup_printf() instead of snprintf()
> >        spapr_drc: use g_strdup_printf() instead of snprintf()
> >        spapr_iommu: convert TCE table object to realize()
> >        spapr_pci: parent the MSI memory region to the PHB
> >        spapr_drc: fix realize and unrealize
> >        spapr_drc: add unrealize method to physical DRC class
> >        spapr_iommu: unregister vmstate at unrealize time
> >        spapr: add pseries-2.11 machine type
> >        spapr_pci: introduce drc_id property
> >        spapr: allow guest to update the XICS phandle
> >        spapr_pci: drop abusive sanity check when migrating the LSI table
> >        spapr: add hotplug hooks for PHB hotplug
> >
> > Michael Roth (11):
> >        spapr_drc: pass object ownership to parent/owner
> >        spapr_iommu: pass object ownership to parent/owner
> >        pci: allow cleanup/unregistration of PCI buses
> >        qdev: store DeviceState's canonical path to use when unparenting
> >        spapr_pci: add PHB unrealize
> >        spapr: enable PHB hotplug for pseries-2.11
> >        spapr: create DR connectors for PHBs
> >        spapr_events: add support for phb hotplug events
> >        qdev: pass an Object * to qbus_set_hotplug_handler()
> >        spapr_pci: provide node start offset via spapr_populate_pci_dt()
> >        spapr_pci: add ibm, my-drc-index property for PHB hotplug
> >
> > Nathan Fontenot (1):
> >        spapr: populate PHB DRC entries for root DT node
> >
> >
> >   hw/acpi/piix4.c               |    2
> >   hw/char/virtio-serial-bus.c   |    2
> >   hw/core/bus.c                 |   11 --
> >   hw/core/qdev.c                |   15 ++-
> >   hw/pci/pci.c                  |   33 +++++++
> >   hw/pci/pcie.c                 |    2
> >   hw/pci/shpc.c                 |    2
> >   hw/ppc/spapr.c                |  205 ++++++++++++++++++++++++++++++++++++++++-
> >   hw/ppc/spapr_drc.c            |   65 ++++++++++---
> >   hw/ppc/spapr_events.c         |    3 +
> >   hw/ppc/spapr_hcall.c          |   20 ++++
> >   hw/ppc/spapr_iommu.c          |   22 +++-
> >   hw/ppc/spapr_pci.c            |   86 +++++++++++++----
> >   hw/s390x/css-bridge.c         |    2
> >   hw/s390x/s390-pci-bus.c       |    6 +
> >   hw/scsi/virtio-scsi.c         |    2
> >   hw/scsi/vmw_pvscsi.c          |    2
> >   hw/usb/dev-smartcard-reader.c |    2
> >   include/hw/compat.h           |    3 +
> >   include/hw/pci-host/spapr.h   |    9 +-
> >   include/hw/pci/pci.h          |    3 +
> >   include/hw/ppc/spapr.h        |   15 +++
> >   include/hw/ppc/spapr_drc.h    |    8 ++
> >   include/hw/qdev-core.h        |    4 -
> >   24 files changed, 446 insertions(+), 78 deletions(-)
> >
> >  
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 13/26] qdev: store DeviceState's canonical path to use when unparenting

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 4915 bytes --]

On Wed, 26 Jul 2017 15:24:43 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Jul 25, 2017 at 08:00:47PM +0200, Greg Kurz wrote:
> > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > 
> > device_unparent(dev, ...) is called when a device is unparented,
> > either directly, or as a result of a parent device being
> > finalized, and handles some final cleanup for the device. Part
> > of this includes emiting a DEVICE_DELETED QMP event to notify
> > management, which includes the device's path in the composition
> > tree as provided by object_get_canonical_path().
> > 
> > object_get_canonical_path() assumes the device is still connected
> > to the machine/root container, and will assert otherwise, but
> > in some situations this isn't the case:
> > 
> > If the parent is finalized as a result of object_unparent(), it
> > will still be attached to the composition tree at the time any
> > children are unparented as a result of that same call to
> > object_unparent(). However, in some cases, object_unparent()
> > will complete without finalizing the parent device, due to
> > lingering references that won't be released till some time later.
> > One such example is if the parent has MemoryRegion children (which
> > take a ref on their parent), who in turn have AddressSpace's (which
> > take a ref on their regions), since those AddressSpaces get cleaned
> > up asynchronously by the RCU thread.
> > 
> > In this case qdev:device_unparent() may be called for a child Device
> > that no longer has a path to the root/machine container, causing
> > object_get_canonical_path() to assert.
> > 
> > Fix this by storing the canonical path during realize() so the
> > information will still be available for device_unparent() in such
> > cases.  
> 
> Hm.  I'm no expert on the QOM model, but I'm not sure this is the
> right approach.
> 
> I would have thought the right time to emit the DEVICE_DELETED message
> would be when the device leaves the main composition tree, even if it
> could be finalized later.
> 
> If we made that the case, does this problem go away?
> 

I'm no expert either and I confess I took this patch simply because it was
in Michael's original patchset. :)

But according to Michael's answer, it seems that the issue has a broader
scope than just PHB hotplug... 

> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10
> > ---
> >  hw/core/qdev.c         |   15 ++++++++++++---
> >  include/hw/qdev-core.h |    1 +
> >  2 files changed, 13 insertions(+), 3 deletions(-)
> > 
> > diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> > index 606ab53c42cd..a64b35c16251 100644
> > --- a/hw/core/qdev.c
> > +++ b/hw/core/qdev.c
> > @@ -928,6 +928,12 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> >              goto post_realize_fail;
> >          }
> >  
> > +        /* always re-initialize since we clean up in device_unparent() instead
> > +         * of unrealize()
> > +         */
> > +        g_free(dev->canonical_path);
> > +        dev->canonical_path = object_get_canonical_path(OBJECT(dev));
> > +
> >          if (qdev_get_vmsd(dev)) {
> >              if (vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
> >                                                 dev->instance_id_alias,
> > @@ -984,6 +990,7 @@ child_realize_fail:
> >      }
> >  
> >  post_realize_fail:
> > +    g_free(dev->canonical_path);
> >      if (dc->unrealize) {
> >          dc->unrealize(dev, NULL);
> >      }
> > @@ -1102,10 +1109,12 @@ static void device_unparent(Object *obj)
> >  
> >      /* Only send event if the device had been completely realized */
> >      if (dev->pending_deleted_event) {
> > -        gchar *path = object_get_canonical_path(OBJECT(dev));
> > +        g_assert(dev->canonical_path);
> >  
> > -        qapi_event_send_device_deleted(!!dev->id, dev->id, path, &error_abort);
> > -        g_free(path);
> > +        qapi_event_send_device_deleted(!!dev->id, dev->id, dev->canonical_path,
> > +                                       &error_abort);
> > +        g_free(dev->canonical_path);
> > +        dev->canonical_path = NULL;
> >      }
> >  
> >      qemu_opts_del(dev->opts);
> > diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> > index ae317286a480..9237b6849ff3 100644
> > --- a/include/hw/qdev-core.h
> > +++ b/include/hw/qdev-core.h
> > @@ -153,6 +153,7 @@ struct DeviceState {
> >      /*< public >*/
> >  
> >      const char *id;
> > +    char *canonical_path;
> >      bool realized;
> >      bool pending_deleted_event;
> >      QemuOpts *opts;
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 10320 bytes --]

On Thu, 27 Jul 2017 14:41:31 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 26/07/17 18:40, Greg Kurz wrote:
> > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > main system bus, so we register spapr machine as the handler for the
> > main system bus.
> > 
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> > - rebased against ppc-for-2.10
> > - converted to unplug_request
> > - handle drc_id at pre-plug
> > - reset hotplugged PHB at plug
> > - compatibility with older machine types
> > ---
> >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> >  hw/ppc/spapr_drc.c          |    1 
> >  hw/ppc/spapr_pci.c          |    2 -
> >  include/hw/pci-host/spapr.h |    2 +
> >  include/hw/ppc/spapr.h      |    1 
> >  5 files changed, 118 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 90485054c2e7..589f76ef9fb8 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> >                           &savevm_htab_handlers, spapr);
> >  
> > +    if (spapr->dr_phb_enabled) {
> > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > +    }
> > +
> >      qemu_register_boot_set(spapr_boot_set, spapr);
> >  
> >      if (kvm_enabled()) {
> > @@ -3238,6 +3242,103 @@ out:
> >      error_propagate(errp, local_err);
> >  }
> >  
> > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > +                               Error **errp)
> > +{
> > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > +
> > +    if (sphb->drc_id == (uint32_t)-1) {
> > +        sphb->drc_id = sphb->index;
> > +    }
> > +
> > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > +    }  
> 
> 
> sphb->index in considered 16bits in the existing code (even though it is
> defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> same limit for both, either 256 or 65536 is fine for me.
> 
> It is actually a bit weird - it is possible to completely configure few
> PHBs in the command line so they will have index==-1 but PCI hotplug code -
> spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> this and just does (sphb->index << 16).

You're right and this looks like a bug... I'll try to come up with a fix.

> May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> 

This was how Mike did it in the original patchset but David suggested
to introduce drc_id (to preserve existing setups I guess):

http://patchwork.ozlabs.org/patch/466262/

> 
> 
> > +}
> > +
> > +static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > +                           Error **errp)
> > +{
> > +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > +    void *fdt = NULL;
> > +    int fdt_start_offset;
> > +    int fdt_size;
> > +    Error *local_err = NULL;
> > +    sPAPRDRConnector *drc;
> > +    uint32_t phandle;
> > +    int ret;
> > +    bool hotplugged = spapr_drc_hotplugged(dev);
> > +
> > +    if (!spapr->dr_phb_enabled) {
> > +        return;
> > +    }
> > +
> > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > +    /* hotplug hooks should check it's enabled before getting this far */
> > +    g_assert(drc);
> > +
> > +    if (hotplugged) {
> > +        if (spapr->xics_phandle == UINT32_MAX) {
> > +            error_setg(&local_err,
> > +                       "SLOF didn't update the XICS phandle. PHB hotplug cancelled");
> > +            goto out;
> > +        }
> > +        phandle = spapr->xics_phandle;
> > +
> > +        spapr_phb_reset(dev);  
> 
> 
> It could be DEVICE_GET_CLASS(dev)->reset(dev) without exporting
> spapr_phb_reset. Not sure how this fits into the current QEMU coding style
> though.
> 
> 
> 
> > +    } else {
> > +        phandle = PHANDLE_XICP;
> > +    }
> > +
> > +    fdt = create_device_tree(&fdt_size);
> > +    ret = spapr_populate_pci_dt(sphb, phandle, fdt, &fdt_start_offset);
> > +    if (ret < 0) {
> > +        error_setg(&local_err, "unable to create FDT for %sPHB",
> > +                   dev->hotplugged ? "hotplugged " : "");
> > +        goto out;
> > +    }
> > +
> > +    if (hotplugged) {
> > +        /* generally SLOF creates these, for hotplug it's up to QEMU */
> > +        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
> > +    }
> > +
> > +    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
> > +out:
> > +    if (local_err) {
> > +        error_propagate(errp, local_err);
> > +        g_free(fdt);
> > +        return;
> > +    }
> > +
> > +    if (hotplugged) {
> > +        spapr_hotplug_req_add_by_index(drc);
> > +    } else if (drc) {
> > +        spapr_drc_reset(drc);
> > +    }
> > +}
> > +
> > +void spapr_phb_release(DeviceState *dev)
> > +{
> > +    object_unparent(OBJECT(dev));
> > +}
> > +
> > +static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
> > +                                     DeviceState *dev, Error **errp)
> > +{
> > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > +    sPAPRDRConnector *drc;
> > +
> > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > +    g_assert(drc);
> > +
> > +    if (!spapr_drc_unplug_requested(drc)) {
> > +        spapr_drc_detach(drc);
> > +        spapr_hotplug_req_remove_by_index(drc);
> > +    }
> > +}
> > +
> >  static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> >                                        DeviceState *dev, Error **errp)
> >  {
> > @@ -3284,6 +3385,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> >          spapr_memory_plug(hotplug_dev, dev, node, errp);
> >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> >          spapr_core_plug(hotplug_dev, dev, errp);
> > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > +        spapr_phb_plug(hotplug_dev, dev, errp);
> >      }
> >  }
> >  
> > @@ -3311,6 +3414,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
> >              return;
> >          }
> >          spapr_core_unplug_request(hotplug_dev, dev, errp);
> > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > +        if (sms->dr_phb_enabled) {
> > +            spapr_phb_unplug_request(hotplug_dev, dev, errp);
> > +        } else {
> > +            error_setg(errp, "PHB hot unplug not supported on this machine");
> > +        }
> >      }
> >  }
> >  
> > @@ -3321,6 +3430,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
> >          spapr_memory_pre_plug(hotplug_dev, dev, errp);
> >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> >          spapr_core_pre_plug(hotplug_dev, dev, errp);
> > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > +        spapr_phb_pre_plug(hotplug_dev, dev, errp);
> >      }
> >  }
> >  
> > @@ -3328,7 +3439,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
> >                                                   DeviceState *dev)
> >  {
> >      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> > -        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
> > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> >          return HOTPLUG_HANDLER(machine);
> >      }
> >      return NULL;
> > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > index 2e1049ce61c7..845fcf70b932 100644
> > --- a/hw/ppc/spapr_drc.c
> > +++ b/hw/ppc/spapr_drc.c
> > @@ -704,6 +704,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> >      drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> >      drck->typename = "PHB";
> >      drck->drc_name_prefix = "PHB ";
> > +    drck->release = spapr_phb_release;
> >  }
> >  
> >  static const TypeInfo spapr_dr_connector_info = {
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index 157867af8178..c12f71ae3e2d 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1833,7 +1833,7 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb)
> >                             sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT);
> >  }
> >  
> > -static void spapr_phb_reset(DeviceState *qdev)
> > +void spapr_phb_reset(DeviceState *qdev)
> >  {
> >      sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(qdev);
> >  
> > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > index 7837fb0b1110..15799cee4280 100644
> > --- a/include/hw/pci-host/spapr.h
> > +++ b/include/hw/pci-host/spapr.h
> > @@ -120,6 +120,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
> >  
> >  void spapr_pci_rtas_init(void);
> >  
> > +void spapr_phb_reset(DeviceState *qdev);
> > +
> >  sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *spapr, uint64_t buid);
> >  PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid,
> >                                uint32_t config_addr);
> > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > index f09c54d5bb94..a2f6782bdbbf 100644
> > --- a/include/hw/ppc/spapr.h
> > +++ b/include/hw/ppc/spapr.h
> > @@ -673,6 +673,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
> >  /* CPU and LMB DRC release callbacks. */
> >  void spapr_core_release(DeviceState *dev);
> >  void spapr_lmb_release(DeviceState *dev);
> > +void spapr_phb_release(DeviceState *dev);
> >  
> >  void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
> >  int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);
> > 
> >   
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Michael Roth]

Quoting Greg Kurz (2017-07-27 12:09:55)
> On Thu, 27 Jul 2017 14:41:31 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > On 26/07/17 18:40, Greg Kurz wrote:
> > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > main system bus, so we register spapr machine as the handler for the
> > > main system bus.
> > > 
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > ---
> > > - rebased against ppc-for-2.10
> > > - converted to unplug_request
> > > - handle drc_id at pre-plug
> > > - reset hotplugged PHB at plug
> > > - compatibility with older machine types
> > > ---
> > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > >  hw/ppc/spapr_drc.c          |    1 
> > >  hw/ppc/spapr_pci.c          |    2 -
> > >  include/hw/pci-host/spapr.h |    2 +
> > >  include/hw/ppc/spapr.h      |    1 
> > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 90485054c2e7..589f76ef9fb8 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > >                           &savevm_htab_handlers, spapr);
> > >  
> > > +    if (spapr->dr_phb_enabled) {
> > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > +    }
> > > +
> > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > >  
> > >      if (kvm_enabled()) {
> > > @@ -3238,6 +3242,103 @@ out:
> > >      error_propagate(errp, local_err);
> > >  }
> > >  
> > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > +                               Error **errp)
> > > +{
> > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > +
> > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > +        sphb->drc_id = sphb->index;
> > > +    }
> > > +
> > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > +    }  
> > 
> > 
> > sphb->index in considered 16bits in the existing code (even though it is
> > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > same limit for both, either 256 or 65536 is fine for me.
> > 
> > It is actually a bit weird - it is possible to completely configure few
> > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > this and just does (sphb->index << 16).
> 
> You're right and this looks like a bug... I'll try to come up with a fix.

Yup, that's a bug, and we can trigger it currently by adding 2
additional PHBs that don't have an index specified. QOM catches
and reports them as "attempt to add duplicate property", but it's
just reported by spapr_dr_connector_new() and doesn't seem to be
treated as fatal (and probably should be).

Might also see this more in practice now with the multi-phb support
in libvirt, though I'd imagine those would tend to rely on phb->index
being set.

Now that phb->drc_id is available though we can just use that instead.
I agree it should be limited to 16-bit or smaller to avoid any
possibility of overlap.

> 
> > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > 
> 
> This was how Mike did it in the original patchset but David suggested
> to introduce drc_id (to preserve existing setups I guess):
> 
> http://patchwork.ozlabs.org/patch/466262/

Althrough IIRC what David proposed was to handle it like the other
properties which are filled in automatically when 'index' is specified,
i.e. 'drc_id' would be set automatically by 'index' if index is
specified, or we can set it explicitly if 'index' not specified, but
if we try to set both 'index' and 'drc_id' we trigger the "Either "index"
or other parameters must be specified for PAPR PHB, not both" error.

And it looks like 'index' is limited to 30, so it fits within our
16-bit limit for drc_id.

> 
> > 
> > 
> > > +}
> > > +
> > > +static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > +                           Error **errp)
> > > +{
> > > +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > +    void *fdt = NULL;
> > > +    int fdt_start_offset;
> > > +    int fdt_size;
> > > +    Error *local_err = NULL;
> > > +    sPAPRDRConnector *drc;
> > > +    uint32_t phandle;
> > > +    int ret;
> > > +    bool hotplugged = spapr_drc_hotplugged(dev);
> > > +
> > > +    if (!spapr->dr_phb_enabled) {
> > > +        return;
> > > +    }
> > > +
> > > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > > +    /* hotplug hooks should check it's enabled before getting this far */
> > > +    g_assert(drc);
> > > +
> > > +    if (hotplugged) {
> > > +        if (spapr->xics_phandle == UINT32_MAX) {
> > > +            error_setg(&local_err,
> > > +                       "SLOF didn't update the XICS phandle. PHB hotplug cancelled");
> > > +            goto out;
> > > +        }
> > > +        phandle = spapr->xics_phandle;
> > > +
> > > +        spapr_phb_reset(dev);  
> > 
> > 
> > It could be DEVICE_GET_CLASS(dev)->reset(dev) without exporting
> > spapr_phb_reset. Not sure how this fits into the current QEMU coding style
> > though.
> > 
> > 
> > 
> > > +    } else {
> > > +        phandle = PHANDLE_XICP;
> > > +    }
> > > +
> > > +    fdt = create_device_tree(&fdt_size);
> > > +    ret = spapr_populate_pci_dt(sphb, phandle, fdt, &fdt_start_offset);
> > > +    if (ret < 0) {
> > > +        error_setg(&local_err, "unable to create FDT for %sPHB",
> > > +                   dev->hotplugged ? "hotplugged " : "");
> > > +        goto out;
> > > +    }
> > > +
> > > +    if (hotplugged) {
> > > +        /* generally SLOF creates these, for hotplug it's up to QEMU */
> > > +        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
> > > +    }
> > > +
> > > +    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
> > > +out:
> > > +    if (local_err) {
> > > +        error_propagate(errp, local_err);
> > > +        g_free(fdt);
> > > +        return;
> > > +    }
> > > +
> > > +    if (hotplugged) {
> > > +        spapr_hotplug_req_add_by_index(drc);
> > > +    } else if (drc) {
> > > +        spapr_drc_reset(drc);
> > > +    }
> > > +}
> > > +
> > > +void spapr_phb_release(DeviceState *dev)
> > > +{
> > > +    object_unparent(OBJECT(dev));
> > > +}
> > > +
> > > +static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
> > > +                                     DeviceState *dev, Error **errp)
> > > +{
> > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > +    sPAPRDRConnector *drc;
> > > +
> > > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > > +    g_assert(drc);
> > > +
> > > +    if (!spapr_drc_unplug_requested(drc)) {
> > > +        spapr_drc_detach(drc);
> > > +        spapr_hotplug_req_remove_by_index(drc);
> > > +    }
> > > +}
> > > +
> > >  static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > >                                        DeviceState *dev, Error **errp)
> > >  {
> > > @@ -3284,6 +3385,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > >          spapr_memory_plug(hotplug_dev, dev, node, errp);
> > >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > >          spapr_core_plug(hotplug_dev, dev, errp);
> > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > +        spapr_phb_plug(hotplug_dev, dev, errp);
> > >      }
> > >  }
> > >  
> > > @@ -3311,6 +3414,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
> > >              return;
> > >          }
> > >          spapr_core_unplug_request(hotplug_dev, dev, errp);
> > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > +        if (sms->dr_phb_enabled) {
> > > +            spapr_phb_unplug_request(hotplug_dev, dev, errp);
> > > +        } else {
> > > +            error_setg(errp, "PHB hot unplug not supported on this machine");
> > > +        }
> > >      }
> > >  }
> > >  
> > > @@ -3321,6 +3430,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
> > >          spapr_memory_pre_plug(hotplug_dev, dev, errp);
> > >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > >          spapr_core_pre_plug(hotplug_dev, dev, errp);
> > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > +        spapr_phb_pre_plug(hotplug_dev, dev, errp);
> > >      }
> > >  }
> > >  
> > > @@ -3328,7 +3439,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
> > >                                                   DeviceState *dev)
> > >  {
> > >      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> > > -        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
> > > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > >          return HOTPLUG_HANDLER(machine);
> > >      }
> > >      return NULL;
> > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > index 2e1049ce61c7..845fcf70b932 100644
> > > --- a/hw/ppc/spapr_drc.c
> > > +++ b/hw/ppc/spapr_drc.c
> > > @@ -704,6 +704,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > >      drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > >      drck->typename = "PHB";
> > >      drck->drc_name_prefix = "PHB ";
> > > +    drck->release = spapr_phb_release;
> > >  }
> > >  
> > >  static const TypeInfo spapr_dr_connector_info = {
> > > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > > index 157867af8178..c12f71ae3e2d 100644
> > > --- a/hw/ppc/spapr_pci.c
> > > +++ b/hw/ppc/spapr_pci.c
> > > @@ -1833,7 +1833,7 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb)
> > >                             sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT);
> > >  }
> > >  
> > > -static void spapr_phb_reset(DeviceState *qdev)
> > > +void spapr_phb_reset(DeviceState *qdev)
> > >  {
> > >      sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(qdev);
> > >  
> > > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > > index 7837fb0b1110..15799cee4280 100644
> > > --- a/include/hw/pci-host/spapr.h
> > > +++ b/include/hw/pci-host/spapr.h
> > > @@ -120,6 +120,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
> > >  
> > >  void spapr_pci_rtas_init(void);
> > >  
> > > +void spapr_phb_reset(DeviceState *qdev);
> > > +
> > >  sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *spapr, uint64_t buid);
> > >  PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid,
> > >                                uint32_t config_addr);
> > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > > index f09c54d5bb94..a2f6782bdbbf 100644
> > > --- a/include/hw/ppc/spapr.h
> > > +++ b/include/hw/ppc/spapr.h
> > > @@ -673,6 +673,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
> > >  /* CPU and LMB DRC release callbacks. */
> > >  void spapr_core_release(DeviceState *dev);
> > >  void spapr_lmb_release(DeviceState *dev);
> > > +void spapr_phb_release(DeviceState *dev);
> > >  
> > >  void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
> > >  int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);
> > > 
> > >   
> > 
> > 
> 

Re: [Qemu-devel] [for-2.11 PATCH 13/26] qdev: store DeviceState's canonical path to use when unparenting

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5586 bytes --]

On Thu, Jul 27, 2017 at 06:50:42PM +0200, Greg Kurz wrote:
> On Wed, 26 Jul 2017 15:24:43 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Tue, Jul 25, 2017 at 08:00:47PM +0200, Greg Kurz wrote:
> > > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > 
> > > device_unparent(dev, ...) is called when a device is unparented,
> > > either directly, or as a result of a parent device being
> > > finalized, and handles some final cleanup for the device. Part
> > > of this includes emiting a DEVICE_DELETED QMP event to notify
> > > management, which includes the device's path in the composition
> > > tree as provided by object_get_canonical_path().
> > > 
> > > object_get_canonical_path() assumes the device is still connected
> > > to the machine/root container, and will assert otherwise, but
> > > in some situations this isn't the case:
> > > 
> > > If the parent is finalized as a result of object_unparent(), it
> > > will still be attached to the composition tree at the time any
> > > children are unparented as a result of that same call to
> > > object_unparent(). However, in some cases, object_unparent()
> > > will complete without finalizing the parent device, due to
> > > lingering references that won't be released till some time later.
> > > One such example is if the parent has MemoryRegion children (which
> > > take a ref on their parent), who in turn have AddressSpace's (which
> > > take a ref on their regions), since those AddressSpaces get cleaned
> > > up asynchronously by the RCU thread.
> > > 
> > > In this case qdev:device_unparent() may be called for a child Device
> > > that no longer has a path to the root/machine container, causing
> > > object_get_canonical_path() to assert.
> > > 
> > > Fix this by storing the canonical path during realize() so the
> > > information will still be available for device_unparent() in such
> > > cases.  
> > 
> > Hm.  I'm no expert on the QOM model, but I'm not sure this is the
> > right approach.
> > 
> > I would have thought the right time to emit the DEVICE_DELETED message
> > would be when the device leaves the main composition tree, even if it
> > could be finalized later.
> > 
> > If we made that the case, does this problem go away?
> > 
> 
> I'm no expert either and I confess I took this patch simply because it was
> in Michael's original patchset. :)
> 
> But according to Michael's answer, it seems that the issue has a broader
> scope than just PHB hotplug...

Ok.  I see Michael has posted this and a couple of other things
separately.  Let's hope that can get resolved upstream, and rebase
this series on top of the result.

> 
> > > Cc: Michael S. Tsirkin <mst@redhat.com>
> > > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > ---
> > > Changes since RFC:
> > > - rebased against ppc-for-2.10
> > > ---
> > >  hw/core/qdev.c         |   15 ++++++++++++---
> > >  include/hw/qdev-core.h |    1 +
> > >  2 files changed, 13 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> > > index 606ab53c42cd..a64b35c16251 100644
> > > --- a/hw/core/qdev.c
> > > +++ b/hw/core/qdev.c
> > > @@ -928,6 +928,12 @@ static void device_set_realized(Object *obj, bool value, Error **errp)
> > >              goto post_realize_fail;
> > >          }
> > >  
> > > +        /* always re-initialize since we clean up in device_unparent() instead
> > > +         * of unrealize()
> > > +         */
> > > +        g_free(dev->canonical_path);
> > > +        dev->canonical_path = object_get_canonical_path(OBJECT(dev));
> > > +
> > >          if (qdev_get_vmsd(dev)) {
> > >              if (vmstate_register_with_alias_id(dev, -1, qdev_get_vmsd(dev), dev,
> > >                                                 dev->instance_id_alias,
> > > @@ -984,6 +990,7 @@ child_realize_fail:
> > >      }
> > >  
> > >  post_realize_fail:
> > > +    g_free(dev->canonical_path);
> > >      if (dc->unrealize) {
> > >          dc->unrealize(dev, NULL);
> > >      }
> > > @@ -1102,10 +1109,12 @@ static void device_unparent(Object *obj)
> > >  
> > >      /* Only send event if the device had been completely realized */
> > >      if (dev->pending_deleted_event) {
> > > -        gchar *path = object_get_canonical_path(OBJECT(dev));
> > > +        g_assert(dev->canonical_path);
> > >  
> > > -        qapi_event_send_device_deleted(!!dev->id, dev->id, path, &error_abort);
> > > -        g_free(path);
> > > +        qapi_event_send_device_deleted(!!dev->id, dev->id, dev->canonical_path,
> > > +                                       &error_abort);
> > > +        g_free(dev->canonical_path);
> > > +        dev->canonical_path = NULL;
> > >      }
> > >  
> > >      qemu_opts_del(dev->opts);
> > > diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> > > index ae317286a480..9237b6849ff3 100644
> > > --- a/include/hw/qdev-core.h
> > > +++ b/include/hw/qdev-core.h
> > > @@ -153,6 +153,7 @@ struct DeviceState {
> > >      /*< public >*/
> > >  
> > >      const char *id;
> > > +    char *canonical_path;
> > >      bool realized;
> > >      bool pending_deleted_event;
> > >      QemuOpts *opts;
> > >   
> > 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Alexey Kardashevskiy]

[-- Attachment #1: Type: text/plain, Size: 2792 bytes --]

On 28/07/17 02:39, Greg Kurz wrote:
> On Wed, 26 Jul 2017 17:31:17 -0300
> Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote:
> 
>> I've tested the patch set using Greg's Github branch. It worked fine in 
>> my tests
>> using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
>> though:
>>
>> 1 - This is not related to this patch set per se because it is 
>> reproducible on master, but
>> I think it is interfering with this new feature.  There is a 
>> warning/error message in
>> the kernel right after SLOF that goes:
>>
>> (...)
>>   -> smp_release_cpus()  
>> spinning_secondaries = 0
>>   <- smp_release_cpus()
>> Linux ppc64le
>> #1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
>> failed with rc=-22
>> [    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
>> [  OK  ] Started Security Auditing Service.
>> (...)
>>
> 
> This is a regression in QEMU master introduced by this commit:
> 
> commit b87680427e8a3ff682f66514e99a8344e7437247
> Author: Cédric Le Goater <clg@kaod.org>
> Date:   Wed Jul 5 19:13:15 2017 +0200
> 
>     spapr: populate device tree depending on XIVE_EXPLOIT option
>     
>     When XIVE is supported, the device tree should be populated
>     accordingly and the XIVE memory regions mapped to activate MMIOs.
>     
>     Depending on the design we choose, we could also allocate different
>     ICS and ICP objects, or switch between objects. This needs to be
>     discussed.
>     
>     Signed-off-by: Cédric Le Goater <clg@kaod.org>
>     Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> 
> It is very similar to the issue that motivated the new KVMPPC_H_UPDATE_PHANDLE
> hcall (see patch 24 and 26 in this series):
> 
> - QEMU creates an "interrupt-controller" node with a phandle property
>   with the value 0x1111
> - QEMU passes the FDT to SLOF
> - SLOF converts all references to the phandle to an SLOF internal value
> 
> => from now on (ie, until the next machine reset), the guest only knows
>    the OF phandle.
> 
> - during CAS, if we go XICS, we send back an updated FDT with the
>   phandle of the "interrupt-controller" node reverted to 0x1111
> 
> => the guest complains because all cold-plugged devices nodes refer
>    to the OF phandle, not 0x1111
> 
> A solution is to use the value set by KVMPPC_H_UPDATE_PHANDLE during CAS
> instead of 0x1111. I could verify it makes the guest warning disappear.
> 
> I'll send a dedicated patchset to fix this in 2.10.


The SLOF I pushed for 2.10 does not have it though. And the rest of XIVE is
not targeted for 2.10 anyway. So imho the solution is reverting "spapr:
populate device tree depending on XIVE_EXPLOIT option" for 2.10.


-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3237 bytes --]

On Fri, Jul 28, 2017 at 01:27:05PM +1000, Alexey Kardashevskiy wrote:
> On 28/07/17 02:39, Greg Kurz wrote:
> > On Wed, 26 Jul 2017 17:31:17 -0300
> > Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote:
> > 
> >> I've tested the patch set using Greg's Github branch. It worked fine in 
> >> my tests
> >> using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
> >> though:
> >>
> >> 1 - This is not related to this patch set per se because it is 
> >> reproducible on master, but
> >> I think it is interfering with this new feature.  There is a 
> >> warning/error message in
> >> the kernel right after SLOF that goes:
> >>
> >> (...)
> >>   -> smp_release_cpus()  
> >> spinning_secondaries = 0
> >>   <- smp_release_cpus()
> >> Linux ppc64le
> >> #1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
> >> failed with rc=-22
> >> [    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
> >> [  OK  ] Started Security Auditing Service.
> >> (...)
> >>
> > 
> > This is a regression in QEMU master introduced by this commit:
> > 
> > commit b87680427e8a3ff682f66514e99a8344e7437247
> > Author: Cédric Le Goater <clg@kaod.org>
> > Date:   Wed Jul 5 19:13:15 2017 +0200
> > 
> >     spapr: populate device tree depending on XIVE_EXPLOIT option
> >     
> >     When XIVE is supported, the device tree should be populated
> >     accordingly and the XIVE memory regions mapped to activate MMIOs.
> >     
> >     Depending on the design we choose, we could also allocate different
> >     ICS and ICP objects, or switch between objects. This needs to be
> >     discussed.
> >     
> >     Signed-off-by: Cédric Le Goater <clg@kaod.org>
> >     Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > 
> > It is very similar to the issue that motivated the new KVMPPC_H_UPDATE_PHANDLE
> > hcall (see patch 24 and 26 in this series):
> > 
> > - QEMU creates an "interrupt-controller" node with a phandle property
> >   with the value 0x1111
> > - QEMU passes the FDT to SLOF
> > - SLOF converts all references to the phandle to an SLOF internal value
> > 
> > => from now on (ie, until the next machine reset), the guest only knows
> >    the OF phandle.
> > 
> > - during CAS, if we go XICS, we send back an updated FDT with the
> >   phandle of the "interrupt-controller" node reverted to 0x1111
> > 
> > => the guest complains because all cold-plugged devices nodes refer
> >    to the OF phandle, not 0x1111
> > 
> > A solution is to use the value set by KVMPPC_H_UPDATE_PHANDLE during CAS
> > instead of 0x1111. I could verify it makes the guest warning disappear.
> > 
> > I'll send a dedicated patchset to fix this in 2.10.
> 
> 
> The SLOF I pushed for 2.10 does not have it though. And the rest of XIVE is
> not targeted for 2.10 anyway. So imho the solution is reverting "spapr:
> populate device tree depending on XIVE_EXPLOIT option" for 2.10.

I agree, I've applied the revert to ppc-for-2.10 now.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 17/26] spapr_pci: introduce drc_id property

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]

On Tue, Jul 25, 2017 at 08:01:38PM +0200, Greg Kurz wrote:
> With the addition of PHB hotplug, we have a static number of DRCs
> that can be used to handle hotplug/unplug operations on our PHBs,
> and need a consistent way to map PHBs to these connectors, and
> assign a unique identifiers for the connectors.
> 
> This patch adds a drc_id property for that purpose.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

I'd prefer to see this folded into the patch that actually uses this
property.

> ---
>  hw/ppc/spapr_pci.c          |    1 +
>  include/hw/pci-host/spapr.h |    2 ++
>  2 files changed, 3 insertions(+)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 994d2f36105f..54533d8a3841 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1878,6 +1878,7 @@ static Property spapr_phb_properties[] = {
>                       pre_2_8_migration, false),
>      DEFINE_PROP_BOOL("pcie-extended-configuration-space", sPAPRPHBState,
>                       pcie_ecs, true),
> +    DEFINE_PROP_UINT32("drc_id", sPAPRPHBState, drc_id, -1),
>      DEFINE_PROP_END_OF_LIST(),
>  };
>  
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 5a4e9686d562..31bae68167f2 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -82,6 +82,8 @@ struct sPAPRPHBState {
>  
>      bool pcie_ecs; /* Allow access to PCIe extended config space? */
>  
> +    uint32_t drc_id;
> +
>      /* Fields for migration compatibility hacks */
>      bool pre_2_8_migration;
>      uint32_t mig_liobn;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4637 bytes --]

On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>



> ---
> Changes since RFC:
> - rebased against ppc-for-2.10 (reset hooks registering already merged)
> - added new DRC type for PHB
> ---
>  hw/ppc/spapr.c             |   15 +++++++++++++++
>  hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
>  include/hw/ppc/spapr_drc.h |    8 ++++++++
>  3 files changed, 40 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 8dc505343c0f..5950c009ab7e 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -98,6 +98,9 @@
>  
>  #define PHANDLE_XICP            0x00001111
>  
> +/* maximum number of hotpluggable PHBs */
> +#define SPAPR_DRC_MAX_PHB       256

I wonder if we should actually make this a machine property.

>  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
>                                    const char *type_ics,
>                                    int nr_irqs, Error **errp)
> @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
>  
>      spapr->dr_phb_enabled = smc->dr_phb_enabled;
>  
> +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> +     * connectors (described in root DT node's "ibm,drc-types" property)
> +     * are pre-initialized here. additional child connectors (such as
> +     * connectors for a PHBs PCI slots) are added as needed during their
> +     * parent's realization.
> +     */
> +    if (spapr->dr_phb_enabled) {
> +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> +        }
> +    }
> +
>      /* Set up PCI */
>      spapr_pci_rtas_init();
>  
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index eb8024d37c54..2e1049ce61c7 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
>      drck->release = spapr_lmb_release;
>  }
>  
> +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> +{
> +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> +
> +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> +    drck->typename = "PHB";
> +    drck->drc_name_prefix = "PHB ";
> +}
> +
>  static const TypeInfo spapr_dr_connector_info = {
>      .name          = TYPE_SPAPR_DR_CONNECTOR,
>      .parent        = TYPE_DEVICE,
> @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
>      .class_init    = spapr_drc_lmb_class_init,
>  };
>  
> +static const TypeInfo spapr_drc_phb_info = {
> +    .name          = TYPE_SPAPR_DRC_PHB,
> +    .parent        = TYPE_SPAPR_DRC_LOGICAL,

I thought PHB DRCs were physical..

> +    .instance_size = sizeof(sPAPRDRConnector),
> +    .class_init    = spapr_drc_phb_class_init,
> +};
> +
>  /* helper functions for external users */
>  
>  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
>      type_register_static(&spapr_drc_cpu_info);
>      type_register_static(&spapr_drc_pci_info);
>      type_register_static(&spapr_drc_lmb_info);
> +    type_register_static(&spapr_drc_phb_info);
>  
>      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
>                          rtas_set_indicator);
> diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> index a7958d0a8d14..535fc61b98a8 100644
> --- a/include/hw/ppc/spapr_drc.h
> +++ b/include/hw/ppc/spapr_drc.h
> @@ -69,6 +69,14 @@
>  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
>                                          TYPE_SPAPR_DRC_LMB)
>  
> +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> +#define SPAPR_DRC_PHB_CLASS(klass) \
> +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> +                                        TYPE_SPAPR_DRC_PHB)
> +
>  /*
>   * Various hotplug types managed by sPAPRDRConnector
>   *
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 21/26] qdev: pass an Object * to qbus_set_hotplug_handler()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 10500 bytes --]

On Tue, Jul 25, 2017 at 08:02:28PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> Certain devices types, like memory/CPU, are now being handled using a
> hotplug interface provided by a top-level MachineClass. Hotpluggable
> host bridges are another such device where it makes sense to use a
> machine-level hotplug handler. However, unlike those devices,
> host-bridges have a parent bus (the main system bus), and devices with
> a parent bus use a different mechanism for registering their hotplug
> handlers: qbus_set_hotplug_handler(). This interface currently expects
> a handler to be a subclass of DeviceClass, but this is not the case
> for MachineClass, which derives directly from ObjectClass.
> 
> Internally, the interface only requires an ObjectClass, so expose that
> in qbus_set_hotplug_handler().
> 
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> - changed qbus_set_hotplug_handler() declaration and dropped
>   qbus_set_hotplug_handler_internal() (Paolo)
> - updated title and changelog accordingly
> ---
>  hw/acpi/piix4.c               |    2 +-
>  hw/char/virtio-serial-bus.c   |    2 +-
>  hw/core/bus.c                 |   11 ++---------
>  hw/pci/pcie.c                 |    2 +-
>  hw/pci/shpc.c                 |    2 +-
>  hw/ppc/spapr_pci.c            |    2 +-
>  hw/s390x/css-bridge.c         |    2 +-
>  hw/s390x/s390-pci-bus.c       |    6 +++---
>  hw/scsi/virtio-scsi.c         |    2 +-
>  hw/scsi/vmw_pvscsi.c          |    2 +-
>  hw/usb/dev-smartcard-reader.c |    2 +-
>  include/hw/qdev-core.h        |    3 +--
>  12 files changed, 15 insertions(+), 23 deletions(-)
> 
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index f276967365c4..f99c7438235e 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -447,7 +447,7 @@ static void piix4_update_bus_hotplug(PCIBus *pci_bus, void *opaque)
>  
>      /* pci_bus cannot outlive PIIX4PMState, because /machine keeps it alive
>       * and it's not hot-unpluggable */
> -    qbus_set_hotplug_handler(BUS(pci_bus), DEVICE(s), &error_abort);
> +    qbus_set_hotplug_handler(BUS(pci_bus), OBJECT(s), &error_abort);
>  }
>  
>  static void piix4_pm_machine_ready(Notifier *n, void *opaque)
> diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
> index f5bc173844e4..4880236f4258 100644
> --- a/hw/char/virtio-serial-bus.c
> +++ b/hw/char/virtio-serial-bus.c
> @@ -1042,7 +1042,7 @@ static void virtio_serial_device_realize(DeviceState *dev, Error **errp)
>      /* Spawn a new virtio-serial bus on which the ports will ride as devices */
>      qbus_create_inplace(&vser->bus, sizeof(vser->bus), TYPE_VIRTIO_SERIAL_BUS,
>                          dev, vdev->bus_name);
> -    qbus_set_hotplug_handler(BUS(&vser->bus), DEVICE(vser), errp);
> +    qbus_set_hotplug_handler(BUS(&vser->bus), OBJECT(vser), errp);
>      vser->bus.vser = vser;
>      QTAILQ_INIT(&vser->ports);
>  
> diff --git a/hw/core/bus.c b/hw/core/bus.c
> index 4651f244864c..e09843f6abea 100644
> --- a/hw/core/bus.c
> +++ b/hw/core/bus.c
> @@ -22,22 +22,15 @@
>  #include "hw/qdev.h"
>  #include "qapi/error.h"
>  
> -static void qbus_set_hotplug_handler_internal(BusState *bus, Object *handler,
> -                                              Error **errp)
> +void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp)
>  {
> -
>      object_property_set_link(OBJECT(bus), OBJECT(handler),
>                               QDEV_HOTPLUG_HANDLER_PROPERTY, errp);
>  }
>  
> -void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler, Error **errp)
> -{
> -    qbus_set_hotplug_handler_internal(bus, OBJECT(handler), errp);
> -}
> -
>  void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp)
>  {
> -    qbus_set_hotplug_handler_internal(bus, OBJECT(bus), errp);
> +    qbus_set_hotplug_handler(bus, OBJECT(bus), errp);
>  }
>  
>  int qbus_walk_children(BusState *bus,
> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
> index 32191f2a55f7..52d9ff947f7c 100644
> --- a/hw/pci/pcie.c
> +++ b/hw/pci/pcie.c
> @@ -443,7 +443,7 @@ void pcie_cap_slot_init(PCIDevice *dev, uint16_t slot)
>      dev->exp.hpev_notified = false;
>  
>      qbus_set_hotplug_handler(BUS(pci_bridge_get_sec_bus(PCI_BRIDGE(dev))),
> -                             DEVICE(dev), NULL);
> +                             OBJECT(dev), NULL);
>  }
>  
>  void pcie_cap_slot_reset(PCIDevice *dev)
> diff --git a/hw/pci/shpc.c b/hw/pci/shpc.c
> index 69fc14b218d8..feed8a45e535 100644
> --- a/hw/pci/shpc.c
> +++ b/hw/pci/shpc.c
> @@ -648,7 +648,7 @@ int shpc_init(PCIDevice *d, PCIBus *sec_bus, MemoryRegion *bar,
>      shpc_cap_update_dword(d);
>      memory_region_add_subregion(bar, offset, &shpc->mmio);
>  
> -    qbus_set_hotplug_handler(BUS(sec_bus), DEVICE(d), NULL);
> +    qbus_set_hotplug_handler(BUS(sec_bus), OBJECT(d), NULL);
>  
>      d->cap_present |= QEMU_PCI_CAP_SHPC;
>      return 0;
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 54533d8a3841..b73e099e0285 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1714,7 +1714,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>                             &sphb->memspace, &sphb->iospace,
>                             PCI_DEVFN(0, 0), PCI_NUM_PINS, TYPE_PCI_BUS);
>      phb->bus = bus;
> -    qbus_set_hotplug_handler(BUS(phb->bus), DEVICE(sphb), NULL);
> +    qbus_set_hotplug_handler(BUS(phb->bus), OBJECT(sphb), NULL);
>  
>      /*
>       * Initialize PHB address space.
> diff --git a/hw/s390x/css-bridge.c b/hw/s390x/css-bridge.c
> index c4a9735d7108..19eda77979d1 100644
> --- a/hw/s390x/css-bridge.c
> +++ b/hw/s390x/css-bridge.c
> @@ -107,7 +107,7 @@ VirtualCssBus *virtual_css_bus_init(void)
>      cbus->squash_mcss = s390_get_squash_mcss();
>  
>      /* Enable hotplugging */
> -    qbus_set_hotplug_handler(bus, dev, &error_abort);
> +    qbus_set_hotplug_handler(bus, OBJECT(dev), &error_abort);
>  
>      css_register_io_adapters(CSS_IO_ADAPTER_VIRTIO, true, false,
>                               0, &error_abort);
> diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
> index 61cfd2138f68..503b7ae05abb 100644
> --- a/hw/s390x/s390-pci-bus.c
> +++ b/hw/s390x/s390-pci-bus.c
> @@ -566,11 +566,11 @@ static int s390_pcihost_init(SysBusDevice *dev)
>      pci_setup_iommu(b, s390_pci_dma_iommu, s);
>  
>      bus = BUS(b);
> -    qbus_set_hotplug_handler(bus, DEVICE(dev), NULL);
> +    qbus_set_hotplug_handler(bus, OBJECT(dev), NULL);
>      phb->bus = b;
>  
>      s->bus = S390_PCI_BUS(qbus_create(TYPE_S390_PCI_BUS, DEVICE(s), NULL));
> -    qbus_set_hotplug_handler(BUS(s->bus), DEVICE(s), NULL);
> +    qbus_set_hotplug_handler(BUS(s->bus), OBJECT(s), NULL);
>  
>      s->iommu_table = g_hash_table_new_full(g_int64_hash, g_int64_equal,
>                                             NULL, g_free);
> @@ -678,7 +678,7 @@ static void s390_pcihost_hot_plug(HotplugHandler *hotplug_dev,
>          pci_setup_iommu(&pb->sec_bus, s390_pci_dma_iommu, s);
>  
>          bus = BUS(&pb->sec_bus);
> -        qbus_set_hotplug_handler(bus, DEVICE(s), errp);
> +        qbus_set_hotplug_handler(bus, OBJECT(s), errp);
>  
>          if (dev->hotplugged) {
>              pci_default_write_config(pdev, PCI_PRIMARY_BUS, s->bus_no, 1);
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index eb639442d192..b4c26d4862b5 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -893,7 +893,7 @@ static void virtio_scsi_device_realize(DeviceState *dev, Error **errp)
>      scsi_bus_new(&s->bus, sizeof(s->bus), dev,
>                   &virtio_scsi_scsi_info, vdev->bus_name);
>      /* override default SCSI bus hotplug-handler, with virtio-scsi's one */
> -    qbus_set_hotplug_handler(BUS(&s->bus), dev, &error_abort);
> +    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
>  
>      virtio_scsi_dataplane_setup(s, errp);
>  }
> diff --git a/hw/scsi/vmw_pvscsi.c b/hw/scsi/vmw_pvscsi.c
> index 4a106da85646..036e44caca42 100644
> --- a/hw/scsi/vmw_pvscsi.c
> +++ b/hw/scsi/vmw_pvscsi.c
> @@ -1146,7 +1146,7 @@ pvscsi_init(PCIDevice *pci_dev)
>      scsi_bus_new(&s->bus, sizeof(s->bus), DEVICE(pci_dev),
>                   &pvscsi_scsi_info, NULL);
>      /* override default SCSI bus hotplug-handler, with pvscsi's one */
> -    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(s), &error_abort);
> +    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(s), &error_abort);
>      pvscsi_reset_state(s);
>  
>      return 0;
> diff --git a/hw/usb/dev-smartcard-reader.c b/hw/usb/dev-smartcard-reader.c
> index bef1f03c426b..f2a86de6c556 100644
> --- a/hw/usb/dev-smartcard-reader.c
> +++ b/hw/usb/dev-smartcard-reader.c
> @@ -1337,7 +1337,7 @@ static void ccid_realize(USBDevice *dev, Error **errp)
>      usb_desc_init(dev);
>      qbus_create_inplace(&s->bus, sizeof(s->bus), TYPE_CCID_BUS, DEVICE(dev),
>                          NULL);
> -    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(dev), &error_abort);
> +    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
>      s->intr = usb_ep_get(dev, USB_TOKEN_IN, CCID_INT_IN_EP);
>      s->bulk = usb_ep_get(dev, USB_TOKEN_IN, CCID_BULK_IN_EP);
>      s->card = NULL;
> diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> index 9237b6849ff3..ce01ff42850b 100644
> --- a/include/hw/qdev-core.h
> +++ b/include/hw/qdev-core.h
> @@ -397,8 +397,7 @@ char *qdev_get_dev_path(DeviceState *dev);
>  
>  GSList *qdev_build_hotpluggable_device_list(Object *peripheral);
>  
> -void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler,
> -                              Error **errp);
> +void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp);
>  
>  void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp);
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 22/26] spapr_pci: provide node start offset via spapr_populate_pci_dt()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3261 bytes --]

On Tue, Jul 25, 2017 at 08:02:41PM +0200, Greg Kurz wrote:
> From: Michael Roth <mdroth@linux.vnet.ibm.com>
> 
> PHB hotplug re-uses PHB device tree generation code and passes
> it to a guest via RTAS. Doing this requires knowledge of where
> exactly in the device tree the node describing the PHB begins.
> 
> Provide this via a new optional pointer that can be used to
> store the PHB node's start offset.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Blech.  The patch is correct and you can't do much better at the
moment.  I really hope in the next cycle I get a chance to do a bunch
of the DT construction cleanups which should avoid this messy passing
of dt offsets around.

> ---
> Changes since RFC:
> - rebased against ppc-for-2.10
> ---
>  hw/ppc/spapr.c              |    2 +-
>  hw/ppc/spapr_pci.c          |    6 +++++-
>  include/hw/pci-host/spapr.h |    3 ++-
>  3 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 632040f35ecc..1a6cd4efeb97 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1098,7 +1098,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>      }
>  
>      QLIST_FOREACH(phb, &spapr->phbs, list) {
> -        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt);
> +        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt, NULL);
>          if (ret < 0) {
>              error_report("couldn't setup PCI devices in fdt");
>              exit(1);
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index b73e099e0285..79f10ff453d0 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -2109,7 +2109,8 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
>  
>  int spapr_populate_pci_dt(sPAPRPHBState *phb,
>                            uint32_t xics_phandle,
> -                          void *fdt)
> +                          void *fdt,
> +                          int *node_offset)
>  {
>      int bus_off, i, j, ret;
>      char nodename[FDT_NAME_MAX];
> @@ -2166,6 +2167,9 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
>      if (bus_off < 0) {
>          return bus_off;
>      }
> +    if (node_offset) {
> +        *node_offset = bus_off;
> +    }
>  
>      /* Write PHB properties */
>      _FDT(fdt_setprop_string(fdt, bus_off, "device_type", "pci"));
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 31bae68167f2..7837fb0b1110 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -115,7 +115,8 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
>  
>  int spapr_populate_pci_dt(sPAPRPHBState *phb,
>                            uint32_t xics_phandle,
> -                          void *fdt);
> +                          void *fdt,
> +                          int *node_offset);
>  
>  void spapr_pci_rtas_init(void);
>  
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 7085 bytes --]

On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:
> The "phandle" property of the XICS node is referenced by the "interrupt-map"
> property of each PHB node. This is used by the guest OS to setup IRQs for
> all PCI devices.
> 
> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
> this value to a SLOF specific one, which is then presented to the guest OS.
> 
> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
> by SLOF to communicate the patched phandle value back to QEMU. This value
> is then cached and preserved accross migration until machine reset.
> 
> This is required to be able to support PHB hotplug.
> 
> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> have to introduce its number even if QEMU currently doesn't implement it.
> 
> Suggested-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>

Ugh.  I really, really hope we can avoid this, though I don't
immediately see how.  Having to have two way communication between
qemu and SLOF about the device tree contents just seems like opening
the door to endless complexities.

This is basically a consequence of the fact that both qemu and partly
responsible for constructing the device tree for the guest, and that's
not easy to avoid.

Hrm.. Thomas, I know it's not really the OF way, but would it be
feasible to change SLOF to use the phandles as supplied by qemu rather
than creating its own?

> ---
>  hw/ppc/spapr.c         |   25 +++++++++++++++++++++++--
>  hw/ppc/spapr_hcall.c   |   20 ++++++++++++++++++++
>  include/hw/ppc/spapr.h |   11 ++++++++++-
>  3 files changed, 53 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1a6cd4efeb97..90485054c2e7 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -96,8 +96,6 @@
>  
>  #define MIN_RMA_SLOF            128UL
>  
> -#define PHANDLE_XICP            0x00001111
> -
>  /* maximum number of hotpluggable PHBs */
>  #define SPAPR_DRC_MAX_PHB       256
>  
> @@ -1454,6 +1452,7 @@ static void ppc_spapr_reset(void)
>      first_ppc_cpu->env.nip = SPAPR_ENTRY_POINT;
>  
>      spapr->cas_reboot = false;
> +    spapr->xics_phandle = UINT32_MAX;

Uh, is this defaulting to the phandle being (u32)(-1)?  That's one of
the two explicitly forbidden values for a phandle, so that's probably
not a good idea.

>  }
>  
>  static void spapr_create_nvram(sPAPRMachineState *spapr)
> @@ -1652,6 +1651,26 @@ static const VMStateDescription vmstate_spapr_patb_entry = {
>      },
>  };
>  
> +static bool spapr_xics_phandle_needed(void *opaque)
> +{
> +    sPAPRMachineState *spapr = opaque;
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(MACHINE(spapr));
> +
> +    /* Don't break older machine types that don't support PHB hotplug. */
> +    return smc->dr_phb_enabled && spapr->xics_phandle != UINT32_MAX;
> +}
> +
> +static const VMStateDescription vmstate_spapr_xics_phandle = {
> +    .name = "spapr_xics_phandle",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .needed = spapr_xics_phandle_needed,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT32(xics_phandle, sPAPRMachineState),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
>  static const VMStateDescription vmstate_spapr = {
>      .name = "spapr",
>      .version_id = 3,
> @@ -1671,6 +1690,7 @@ static const VMStateDescription vmstate_spapr = {
>          &vmstate_spapr_ov5_cas,
>          &vmstate_spapr_patb_entry,
>          &vmstate_spapr_pending_events,
> +        &vmstate_spapr_xics_phandle,
>          NULL
>      }
>  };
> @@ -2702,6 +2722,7 @@ static void spapr_machine_initfn(Object *obj)
>  
>      spapr->htab_fd = -1;
>      spapr->use_hotplug_event_source = true;
> +    spapr->xics_phandle = UINT32_MAX;
>      object_property_add_str(obj, "kvm-type",
>                              spapr_get_kvm_type, spapr_set_kvm_type, NULL);
>      object_property_set_description(obj, "kvm-type",
> diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
> index 72ea5a8247bf..ce8a9eb66b23 100644
> --- a/hw/ppc/spapr_hcall.c
> +++ b/hw/ppc/spapr_hcall.c
> @@ -1623,6 +1623,25 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu,
>      return H_SUCCESS;
>  }
>  
> +static target_ulong h_update_phandle(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> +                                     target_ulong opcode, target_ulong *args)
> +{
> +    target_ulong old_phandle = args[0];
> +    target_ulong new_phandle = args[1];
> +
> +    if (new_phandle >= UINT32_MAX) {
> +        return H_PARAMETER;
> +    }
> +
> +    /* We only have a "phandle" property in the XICS node at the moment. */
> +    if (old_phandle != (uint32_t) PHANDLE_XICP) {
> +        return H_PARAMETER;
> +    }
> +
> +    spapr->xics_phandle = (uint32_t) new_phandle;
> +    return H_SUCCESS;
> +}
> +
>  static spapr_hcall_fn papr_hypercall_table[(MAX_HCALL_OPCODE / 4) + 1];
>  static spapr_hcall_fn kvmppc_hypercall_table[KVMPPC_HCALL_MAX - KVMPPC_HCALL_BASE + 1];
>  
> @@ -1717,6 +1736,7 @@ static void hypercall_register_types(void)
>  
>      /* qemu/KVM-PPC specific hcalls */
>      spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
> +    spapr_register_hypercall(KVMPPC_H_UPDATE_PHANDLE, h_update_phandle);
>  
>      /* ibm,client-architecture-support support */
>      spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 8004d9c2ab2c..f09c54d5bb94 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -125,6 +125,8 @@ struct sPAPRMachineState {
>  
>      bool dr_phb_enabled; /* hotplug / dynamic-reconfiguration of PHBs */
>  
> +    uint32_t xics_phandle;
> +
>      /*< public >*/
>      char *kvm_type;
>      MemoryHotplugState hotplug_memory;
> @@ -402,7 +404,9 @@ struct sPAPRMachineState {
>  #define KVMPPC_H_LOGICAL_MEMOP  (KVMPPC_HCALL_BASE + 0x1)
>  /* Client Architecture support */
>  #define KVMPPC_H_CAS            (KVMPPC_HCALL_BASE + 0x2)
> -#define KVMPPC_HCALL_MAX        KVMPPC_H_CAS
> +#define KVMPPC_H_RTAS_UPDATE    (KVMPPC_HCALL_BASE + 0x3)
> +#define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4)
> +#define KVMPPC_HCALL_MAX        KVMPPC_H_UPDATE_PHANDLE
>  
>  typedef struct sPAPRDeviceTreeUpdateHeader {
>      uint32_t version_id;
> @@ -707,4 +711,9 @@ void spapr_do_system_reset_on_cpu(CPUState *cs, run_on_cpu_data arg);
>  
>  #define HTAB_SIZE(spapr)        (1ULL << ((spapr)->htab_shift))
>  
> +/* Boot time value of the "phandle" property of the "interrupt-controller"
> + * node.
> + */
> +#define PHANDLE_XICP            0x00001111
> +
>  #endif /* HW_SPAPR_H */
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 25/26] spapr_pci: drop abusive sanity check when migrating the LSI table

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

On Tue, Jul 25, 2017 at 08:03:21PM +0200, Greg Kurz wrote:
> The guest can allocate blocks of IRQs when calling the ibm,change-msi
> RTAS call. This has an impact on the IRQ numbers returned by subsequent
> calls to spapr_ics_alloc_block().
> 
> It doesn't cause any problem right now because PHB are cold plugged and
> the LSI table have the same numbers at the destination and the source.
> 
> But this won't be the case anymore when we support PHB hotplug. In this
> case the IRQ numbers in the LSI table are state that we should migrate.
> 
> This patch hence changes the destination to always accept the LSI table
> from the migration stream, like it is already done for MSIs. No effort
> is made to preserve the current behavior of failing migration when LSI
> numbers are different with older machine types, for simplicity.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

Urgh.  So, yes, there's a real problem here.  But fundamentally it's
because we're dynamically allocating the LSIs from a pool, rather than
having a fixed mapping (mea culpa, I did this in a bunch of places
before I knew better and it's come to bite us several times since).

Migrating the value like this will fix the problem, but I'm going to
have to think if it's the best way.  If we really do need an allocator
or something like it, then we'll have to migrate something - although
the fact that the LSIs will depend on the order you hotplug things is
kinda nasty still.

Making the LSIs fixed will address several other oddities in this
area, but will obviously been a wider ranging rework of the irq
assignment in the machine (and some machine version compatibility
cruft, of course).

> ---
>  hw/ppc/spapr_pci.c |    3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 58406a1b7e93..157867af8178 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1887,8 +1887,7 @@ static const VMStateDescription vmstate_spapr_pci_lsi = {
>      .version_id = 1,
>      .minimum_version_id = 1,
>      .fields = (VMStateField[]) {
> -        VMSTATE_UINT32_EQUAL(irq, struct spapr_pci_lsi, NULL),
> -
> +        VMSTATE_UINT32(irq, struct spapr_pci_lsi),
>          VMSTATE_END_OF_LIST()
>      },
>  };
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3694 bytes --]

On Thu, Jul 27, 2017 at 07:09:55PM +0200, Greg Kurz wrote:
> On Thu, 27 Jul 2017 14:41:31 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > On 26/07/17 18:40, Greg Kurz wrote:
> > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > main system bus, so we register spapr machine as the handler for the
> > > main system bus.
> > > 
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > ---
> > > - rebased against ppc-for-2.10
> > > - converted to unplug_request
> > > - handle drc_id at pre-plug
> > > - reset hotplugged PHB at plug
> > > - compatibility with older machine types
> > > ---
> > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > >  hw/ppc/spapr_drc.c          |    1 
> > >  hw/ppc/spapr_pci.c          |    2 -
> > >  include/hw/pci-host/spapr.h |    2 +
> > >  include/hw/ppc/spapr.h      |    1 
> > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 90485054c2e7..589f76ef9fb8 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > >                           &savevm_htab_handlers, spapr);
> > >  
> > > +    if (spapr->dr_phb_enabled) {
> > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > +    }
> > > +
> > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > >  
> > >      if (kvm_enabled()) {
> > > @@ -3238,6 +3242,103 @@ out:
> > >      error_propagate(errp, local_err);
> > >  }
> > >  
> > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > +                               Error **errp)
> > > +{
> > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > +
> > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > +        sphb->drc_id = sphb->index;
> > > +    }
> > > +
> > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > +    }  
> > 
> > 
> > sphb->index in considered 16bits in the existing code (even though it is
> > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > same limit for both, either 256 or 65536 is fine for me.
> > 
> > It is actually a bit weird - it is possible to completely configure few
> > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > this and just does (sphb->index << 16).
> 
> You're right and this looks like a bug... I'll try to come up with a fix.
> 
> > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > 
> 
> This was how Mike did it in the original patchset but David suggested
> to introduce drc_id (to preserve existing setups I guess):
> 
> http://patchwork.ozlabs.org/patch/466262/

Huh.  So I did.  But.. sorry, I've changed my mind.

The fact that needing a DRC forces us to have a reasonable small id
for each PHB seems like a good excuse to make index mandatory - I'm
not convinced anyone was actually creating PHBs without index, and
this does allow us to simplify a bunch of things.

I'd like to see that done as a preliminary cleanup patch, though.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Cédric Le Goater]

On 07/28/2017 05:40 AM, David Gibson wrote:
> On Fri, Jul 28, 2017 at 01:27:05PM +1000, Alexey Kardashevskiy wrote:
>> On 28/07/17 02:39, Greg Kurz wrote:
>>> On Wed, 26 Jul 2017 17:31:17 -0300
>>> Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote:
>>>
>>>> I've tested the patch set using Greg's Github branch. It worked fine in 
>>>> my tests
>>>> using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
>>>> though:
>>>>
>>>> 1 - This is not related to this patch set per se because it is 
>>>> reproducible on master, but
>>>> I think it is interfering with this new feature.  There is a 
>>>> warning/error message in
>>>> the kernel right after SLOF that goes:
>>>>
>>>> (...)
>>>>   -> smp_release_cpus()  
>>>> spinning_secondaries = 0
>>>>   <- smp_release_cpus()
>>>> Linux ppc64le
>>>> #1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
>>>> failed with rc=-22
>>>> [    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
>>>> [  OK  ] Started Security Auditing Service.
>>>> (...)
>>>>
>>>
>>> This is a regression in QEMU master introduced by this commit:
>>>
>>> commit b87680427e8a3ff682f66514e99a8344e7437247
>>> Author: Cédric Le Goater <clg@kaod.org>
>>> Date:   Wed Jul 5 19:13:15 2017 +0200
>>>
>>>     spapr: populate device tree depending on XIVE_EXPLOIT option
>>>     
>>>     When XIVE is supported, the device tree should be populated
>>>     accordingly and the XIVE memory regions mapped to activate MMIOs.
>>>     
>>>     Depending on the design we choose, we could also allocate different
>>>     ICS and ICP objects, or switch between objects. This needs to be
>>>     discussed.
>>>     
>>>     Signed-off-by: Cédric Le Goater <clg@kaod.org>
>>>     Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
>>>
>>> It is very similar to the issue that motivated the new KVMPPC_H_UPDATE_PHANDLE
>>> hcall (see patch 24 and 26 in this series):
>>>
>>> - QEMU creates an "interrupt-controller" node with a phandle property
>>>   with the value 0x1111
>>> - QEMU passes the FDT to SLOF
>>> - SLOF converts all references to the phandle to an SLOF internal value
>>>
>>> => from now on (ie, until the next machine reset), the guest only knows
>>>    the OF phandle.
>>>
>>> - during CAS, if we go XICS, we send back an updated FDT with the
>>>   phandle of the "interrupt-controller" node reverted to 0x1111
>>>
>>> => the guest complains because all cold-plugged devices nodes refer
>>>    to the OF phandle, not 0x1111
>>>
>>> A solution is to use the value set by KVMPPC_H_UPDATE_PHANDLE during CAS
>>> instead of 0x1111. I could verify it makes the guest warning disappear.
>>>
>>> I'll send a dedicated patchset to fix this in 2.10.
>>
>>
>> The SLOF I pushed for 2.10 does not have it though. And the rest of XIVE is
>> not targeted for 2.10 anyway. So imho the solution is reverting "spapr:
>> populate device tree depending on XIVE_EXPLOIT option" for 2.10.
> 
> I agree, I've applied the revert to ppc-for-2.10 now.

Sure. 

Greg, have you found a solution to get around this problem or do we need
to populate the device tree with the interrupt controller node for SLOF ? 

Thanks,

C.  

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[Thomas Huth]

[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]

On 28.07.2017 06:02, David Gibson wrote:
> On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:
>> The "phandle" property of the XICS node is referenced by the "interrupt-map"
>> property of each PHB node. This is used by the guest OS to setup IRQs for
>> all PCI devices.
>>
>> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
>> this value to a SLOF specific one, which is then presented to the guest OS.
>>
>> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
>> by SLOF to communicate the patched phandle value back to QEMU. This value
>> is then cached and preserved accross migration until machine reset.
>>
>> This is required to be able to support PHB hotplug.
>>
>> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
>> have to introduce its number even if QEMU currently doesn't implement it.
>>
>> Suggested-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Greg Kurz <groug@kaod.org>
> 
> Ugh.  I really, really hope we can avoid this, though I don't
> immediately see how.  Having to have two way communication between
> qemu and SLOF about the device tree contents just seems like opening
> the door to endless complexities.
> 
> This is basically a consequence of the fact that both qemu and partly
> responsible for constructing the device tree for the guest, and that's
> not easy to avoid.
> 
> Hrm.. Thomas, I know it's not really the OF way, but would it be
> feasible to change SLOF to use the phandles as supplied by qemu rather
> than creating its own?

I don't see a way to do this in an easy, clean, reasonable way. SLOF
uses pointers to internal structures as phandles all over the place. You
likely can't replace that so easily without rewriting half of the whole
device tree related code in SLOF, I guess...

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 00/26] spapr: add support for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 3902 bytes --]

On Fri, 28 Jul 2017 07:35:34 +0200
Cédric Le Goater <clg@kaod.org> wrote:

> On 07/28/2017 05:40 AM, David Gibson wrote:
> > On Fri, Jul 28, 2017 at 01:27:05PM +1000, Alexey Kardashevskiy wrote:  
> >> On 28/07/17 02:39, Greg Kurz wrote:  
> >>> On Wed, 26 Jul 2017 17:31:17 -0300
> >>> Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> wrote:
> >>>  
> >>>> I've tested the patch set using Greg's Github branch. It worked fine in 
> >>>> my tests
> >>>> using a Fedora 26 and an Ubuntu 17.04 guests. I have two observations
> >>>> though:
> >>>>
> >>>> 1 - This is not related to this patch set per se because it is 
> >>>> reproducible on master, but
> >>>> I think it is interfering with this new feature.  There is a 
> >>>> warning/error message in
> >>>> the kernel right after SLOF that goes:
> >>>>
> >>>> (...)  
> >>>>   -> smp_release_cpus()    
> >>>> spinning_secondaries = 0
> >>>>   <- smp_release_cpus()
> >>>> Linux ppc64le
> >>>> #1 SMP Mon Jul 1[    0.030450] pci 0000:00:02.0: of_irq_parse_pci: 
> >>>> failed with rc=-22
> >>>> [    0.030552] pci 0000:00:0f.0: of_irq_parse_pci: failed with rc=-22
> >>>> [  OK  ] Started Security Auditing Service.
> >>>> (...)
> >>>>  
> >>>
> >>> This is a regression in QEMU master introduced by this commit:
> >>>
> >>> commit b87680427e8a3ff682f66514e99a8344e7437247
> >>> Author: Cédric Le Goater <clg@kaod.org>
> >>> Date:   Wed Jul 5 19:13:15 2017 +0200
> >>>
> >>>     spapr: populate device tree depending on XIVE_EXPLOIT option
> >>>     
> >>>     When XIVE is supported, the device tree should be populated
> >>>     accordingly and the XIVE memory regions mapped to activate MMIOs.
> >>>     
> >>>     Depending on the design we choose, we could also allocate different
> >>>     ICS and ICP objects, or switch between objects. This needs to be
> >>>     discussed.
> >>>     
> >>>     Signed-off-by: Cédric Le Goater <clg@kaod.org>
> >>>     Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> >>>
> >>> It is very similar to the issue that motivated the new KVMPPC_H_UPDATE_PHANDLE
> >>> hcall (see patch 24 and 26 in this series):
> >>>
> >>> - QEMU creates an "interrupt-controller" node with a phandle property
> >>>   with the value 0x1111
> >>> - QEMU passes the FDT to SLOF
> >>> - SLOF converts all references to the phandle to an SLOF internal value
> >>>  
> >>> => from now on (ie, until the next machine reset), the guest only knows  
> >>>    the OF phandle.
> >>>
> >>> - during CAS, if we go XICS, we send back an updated FDT with the
> >>>   phandle of the "interrupt-controller" node reverted to 0x1111
> >>>  
> >>> => the guest complains because all cold-plugged devices nodes refer  
> >>>    to the OF phandle, not 0x1111
> >>>
> >>> A solution is to use the value set by KVMPPC_H_UPDATE_PHANDLE during CAS
> >>> instead of 0x1111. I could verify it makes the guest warning disappear.
> >>>
> >>> I'll send a dedicated patchset to fix this in 2.10.  
> >>
> >>
> >> The SLOF I pushed for 2.10 does not have it though. And the rest of XIVE is
> >> not targeted for 2.10 anyway. So imho the solution is reverting "spapr:
> >> populate device tree depending on XIVE_EXPLOIT option" for 2.10.  
> > 
> > I agree, I've applied the revert to ppc-for-2.10 now.  
> 
> Sure. 
> 
> Greg, have you found a solution to get around this problem or do we need
> to populate the device tree with the interrupt controller node for SLOF ? 
> 

With commit b87680427e8a, spapr_dt_xics() is called twice actually: first
time during machine reset and a second time during CAS.

The first call means we start in XICS mode by default. But I don't understand
why it is also called during CAS: if the guest didn't advertise XIVE, then
we don't have anything to do, do we ?

Cheers,

--
Greg

> Thanks,
> 
> C.  
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 5652 bytes --]

On Fri, 28 Jul 2017 13:49:25 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:
> > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > 
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > Signed-off-by: Greg Kurz <groug@kaod.org>  
> 
> 
> 
> > ---
> > Changes since RFC:
> > - rebased against ppc-for-2.10 (reset hooks registering already merged)
> > - added new DRC type for PHB
> > ---
> >  hw/ppc/spapr.c             |   15 +++++++++++++++
> >  hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
> >  include/hw/ppc/spapr_drc.h |    8 ++++++++
> >  3 files changed, 40 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 8dc505343c0f..5950c009ab7e 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -98,6 +98,9 @@
> >  
> >  #define PHANDLE_XICP            0x00001111
> >  
> > +/* maximum number of hotpluggable PHBs */
> > +#define SPAPR_DRC_MAX_PHB       256  
> 
> I wonder if we should actually make this a machine property.
> 

It makes sense.

Also, if all PHBs are instanciated with index != -1, we're limited to 31.
Maybe this could be the default value for the machine property instead of
256 then ?

> >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> >                                    const char *type_ics,
> >                                    int nr_irqs, Error **errp)
> > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> >  
> >      spapr->dr_phb_enabled = smc->dr_phb_enabled;
> >  
> > +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > +     * connectors (described in root DT node's "ibm,drc-types" property)
> > +     * are pre-initialized here. additional child connectors (such as
> > +     * connectors for a PHBs PCI slots) are added as needed during their
> > +     * parent's realization.
> > +     */
> > +    if (spapr->dr_phb_enabled) {
> > +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> > +        }
> > +    }
> > +
> >      /* Set up PCI */
> >      spapr_pci_rtas_init();
> >  
> > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > index eb8024d37c54..2e1049ce61c7 100644
> > --- a/hw/ppc/spapr_drc.c
> > +++ b/hw/ppc/spapr_drc.c
> > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
> >      drck->release = spapr_lmb_release;
> >  }
> >  
> > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > +{
> > +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > +
> > +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > +    drck->typename = "PHB";
> > +    drck->drc_name_prefix = "PHB ";
> > +}
> > +
> >  static const TypeInfo spapr_dr_connector_info = {
> >      .name          = TYPE_SPAPR_DR_CONNECTOR,
> >      .parent        = TYPE_DEVICE,
> > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> >      .class_init    = spapr_drc_lmb_class_init,
> >  };
> >  
> > +static const TypeInfo spapr_drc_phb_info = {
> > +    .name          = TYPE_SPAPR_DRC_PHB,
> > +    .parent        = TYPE_SPAPR_DRC_LOGICAL,  
> 
> I thought PHB DRCs were physical..
> 

My understanding is that only PCI IOAs need a physical DRC.

From LoPAPR v1.1 (March 24, 2016):

13.7 Logical Resource Dynamic Reconfiguration (LRDR)

The Logical Resource Dynamic Reconfiguration option allows a platform to make available and recover platform re-
sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots to/from its operating OS image(s).

...

The device tree contains logical resource DR connectors for the maximum number of resources that the platform can
allocate to the specific OS. In some cases such as for processors and PHBs...

and

Table 240. Currently Defined DR Connector Types

| PHB | Logical PCI Host Bridge |

> > +    .instance_size = sizeof(sPAPRDRConnector),
> > +    .class_init    = spapr_drc_phb_class_init,
> > +};
> > +
> >  /* helper functions for external users */
> >  
> >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> >      type_register_static(&spapr_drc_cpu_info);
> >      type_register_static(&spapr_drc_pci_info);
> >      type_register_static(&spapr_drc_lmb_info);
> > +    type_register_static(&spapr_drc_phb_info);
> >  
> >      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
> >                          rtas_set_indicator);
> > diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> > index a7958d0a8d14..535fc61b98a8 100644
> > --- a/include/hw/ppc/spapr_drc.h
> > +++ b/include/hw/ppc/spapr_drc.h
> > @@ -69,6 +69,14 @@
> >  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> >                                          TYPE_SPAPR_DRC_LMB)
> >  
> > +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> > +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> > +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> > +#define SPAPR_DRC_PHB_CLASS(klass) \
> > +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> > +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > +                                        TYPE_SPAPR_DRC_PHB)
> > +
> >  /*
> >   * Various hotplug types managed by sPAPRDRConnector
> >   *
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 6447 bytes --]

On Fri, Jul 28, 2017 at 12:30:35PM +0200, Greg Kurz wrote:
> On Fri, 28 Jul 2017 13:49:25 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:
> > > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > 
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > > Signed-off-by: Greg Kurz <groug@kaod.org>  
> > 
> > 
> > 
> > > ---
> > > Changes since RFC:
> > > - rebased against ppc-for-2.10 (reset hooks registering already merged)
> > > - added new DRC type for PHB
> > > ---
> > >  hw/ppc/spapr.c             |   15 +++++++++++++++
> > >  hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
> > >  include/hw/ppc/spapr_drc.h |    8 ++++++++
> > >  3 files changed, 40 insertions(+)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index 8dc505343c0f..5950c009ab7e 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -98,6 +98,9 @@
> > >  
> > >  #define PHANDLE_XICP            0x00001111
> > >  
> > > +/* maximum number of hotpluggable PHBs */
> > > +#define SPAPR_DRC_MAX_PHB       256  
> > 
> > I wonder if we should actually make this a machine property.
> > 
> 
> It makes sense.
> 
> Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> Maybe this could be the default value for the machine property instead of
> 256 then ?

Actually, if we're binding it back to index, which has a hard limit,
then it no longer makes sense to have it as a property and we should
go back to a constant (well, it could vary by machine type version).

> > >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> > >                                    const char *type_ics,
> > >                                    int nr_irqs, Error **errp)
> > > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> > >  
> > >      spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > >  
> > > +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > > +     * connectors (described in root DT node's "ibm,drc-types" property)
> > > +     * are pre-initialized here. additional child connectors (such as
> > > +     * connectors for a PHBs PCI slots) are added as needed during their
> > > +     * parent's realization.
> > > +     */
> > > +    if (spapr->dr_phb_enabled) {
> > > +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > > +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> > > +        }
> > > +    }
> > > +
> > >      /* Set up PCI */
> > >      spapr_pci_rtas_init();
> > >  
> > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > index eb8024d37c54..2e1049ce61c7 100644
> > > --- a/hw/ppc/spapr_drc.c
> > > +++ b/hw/ppc/spapr_drc.c
> > > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
> > >      drck->release = spapr_lmb_release;
> > >  }
> > >  
> > > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > +{
> > > +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > > +
> > > +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > +    drck->typename = "PHB";
> > > +    drck->drc_name_prefix = "PHB ";
> > > +}
> > > +
> > >  static const TypeInfo spapr_dr_connector_info = {
> > >      .name          = TYPE_SPAPR_DR_CONNECTOR,
> > >      .parent        = TYPE_DEVICE,
> > > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> > >      .class_init    = spapr_drc_lmb_class_init,
> > >  };
> > >  
> > > +static const TypeInfo spapr_drc_phb_info = {
> > > +    .name          = TYPE_SPAPR_DRC_PHB,
> > > +    .parent        = TYPE_SPAPR_DRC_LOGICAL,  
> > 
> > I thought PHB DRCs were physical..
> > 
> 
> My understanding is that only PCI IOAs need a physical DRC.
> 
> From LoPAPR v1.1 (March 24, 2016):
> 
> 13.7 Logical Resource Dynamic Reconfiguration (LRDR)
> 
> The Logical Resource Dynamic Reconfiguration option allows a platform to make available and recover platform re-
> sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots to/from its operating OS image(s).
> 
> ...
> 
> The device tree contains logical resource DR connectors for the maximum number of resources that the platform can
> allocate to the specific OS. In some cases such as for processors and PHBs...
> 
> and
> 
> Table 240. Currently Defined DR Connector Types
> 
> | PHB | Logical PCI Host Bridge |

Ah, my mistake.

> 
> > > +    .instance_size = sizeof(sPAPRDRConnector),
> > > +    .class_init    = spapr_drc_phb_class_init,
> > > +};
> > > +
> > >  /* helper functions for external users */
> > >  
> > >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> > >      type_register_static(&spapr_drc_cpu_info);
> > >      type_register_static(&spapr_drc_pci_info);
> > >      type_register_static(&spapr_drc_lmb_info);
> > > +    type_register_static(&spapr_drc_phb_info);
> > >  
> > >      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
> > >                          rtas_set_indicator);
> > > diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> > > index a7958d0a8d14..535fc61b98a8 100644
> > > --- a/include/hw/ppc/spapr_drc.h
> > > +++ b/include/hw/ppc/spapr_drc.h
> > > @@ -69,6 +69,14 @@
> > >  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > >                                          TYPE_SPAPR_DRC_LMB)
> > >  
> > > +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> > > +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> > > +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> > > +#define SPAPR_DRC_PHB_CLASS(klass) \
> > > +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> > > +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > +                                        TYPE_SPAPR_DRC_PHB)
> > > +
> > >  /*
> > >   * Various hotplug types managed by sPAPRDRConnector
> > >   *
> > >   
> > 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2347 bytes --]

On Fri, Jul 28, 2017 at 08:20:57AM +0200, Thomas Huth wrote:
> On 28.07.2017 06:02, David Gibson wrote:
> > On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:
> >> The "phandle" property of the XICS node is referenced by the "interrupt-map"
> >> property of each PHB node. This is used by the guest OS to setup IRQs for
> >> all PCI devices.
> >>
> >> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
> >> this value to a SLOF specific one, which is then presented to the guest OS.
> >>
> >> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
> >> by SLOF to communicate the patched phandle value back to QEMU. This value
> >> is then cached and preserved accross migration until machine reset.
> >>
> >> This is required to be able to support PHB hotplug.
> >>
> >> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> >> have to introduce its number even if QEMU currently doesn't implement it.
> >>
> >> Suggested-by: Thomas Huth <thuth@redhat.com>
> >> Signed-off-by: Greg Kurz <groug@kaod.org>
> > 
> > Ugh.  I really, really hope we can avoid this, though I don't
> > immediately see how.  Having to have two way communication between
> > qemu and SLOF about the device tree contents just seems like opening
> > the door to endless complexities.
> > 
> > This is basically a consequence of the fact that both qemu and partly
> > responsible for constructing the device tree for the guest, and that's
> > not easy to avoid.
> > 
> > Hrm.. Thomas, I know it's not really the OF way, but would it be
> > feasible to change SLOF to use the phandles as supplied by qemu rather
> > than creating its own?
> 
> I don't see a way to do this in an easy, clean, reasonable way. SLOF
> uses pointers to internal structures as phandles all over the place. You
> likely can't replace that so easily without rewriting half of the whole
> device tree related code in SLOF, I guess...

Dang, that's what I suspected.

Just to be clear the phandles are used directly as raw pointers?
There's not even some lookup macro we could change?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[Philippe Mathieu-Daudé]

Hi David,

On 07/26/2017 12:58 AM, David Gibson wrote:
> On Tue, Jul 25, 2017 at 07:58:53PM +0200, Greg Kurz wrote:
>> Passing a stack allocated buffer of arbitrary length to snprintf()
>> without checking the return value can cause the resultant strings
>> to be silently truncated.
>>
>> Signed-off-by: Greg Kurz <groug@kaod.org>
> 
> Applied to ppc-for-2.11.

Isn't it 2.10 material?

Regards,

Phil.

> 
>> ---
>>   hw/ppc/spapr_drc.c |   15 +++++++++------
>>   1 file changed, 9 insertions(+), 6 deletions(-)
>>
>> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
>> index 15bae5c216a9..e4e8383ec7b5 100644
>> --- a/hw/ppc/spapr_drc.c
>> +++ b/hw/ppc/spapr_drc.c
>> @@ -488,7 +488,7 @@ static void realize(DeviceState *d, Error **errp)
>>   {
>>       sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
>>       Object *root_container;
>> -    char link_name[256];
>> +    gchar *link_name;
>>       gchar *child_name;
>>       Error *err = NULL;
>>   
>> @@ -501,11 +501,12 @@ static void realize(DeviceState *d, Error **errp)
>>        * existing in the composition tree
>>        */
>>       root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
>> -    snprintf(link_name, sizeof(link_name), "%x", spapr_drc_index(drc));
>> +    link_name = g_strdup_printf("%x", spapr_drc_index(drc));
>>       child_name = object_get_canonical_path_component(OBJECT(drc));
>>       trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
>>       object_property_add_alias(root_container, link_name,
>>                                 drc->owner, child_name, &err);
>> +    g_free(link_name);
>>       if (err) {
>>           error_report_err(err);
>>           object_unref(OBJECT(drc));
>> @@ -521,13 +522,14 @@ static void unrealize(DeviceState *d, Error **errp)
>>   {
>>       sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
>>       Object *root_container;
>> -    char name[256];
>> +    gchar *name;
>>       Error *err = NULL;
>>   
>>       trace_spapr_drc_unrealize(spapr_drc_index(drc));
>>       root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
>> -    snprintf(name, sizeof(name), "%x", spapr_drc_index(drc));
>> +    name = g_strdup_printf("%x", spapr_drc_index(drc));
>>       object_property_del(root_container, name, &err);
>> +    g_free(name);
>>       if (err) {
>>           error_report_err(err);
>>           object_unref(OBJECT(drc));
>> @@ -729,10 +731,11 @@ static const TypeInfo spapr_drc_lmb_info = {
>>   sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
>>   {
>>       Object *obj;
>> -    char name[256];
>> +    gchar *name;
>>   
>> -    snprintf(name, sizeof(name), "%s/%x", DRC_CONTAINER_PATH, index);
>> +    name = g_strdup_printf("%s/%x", DRC_CONTAINER_PATH, index);
>>       obj = object_resolve_path(name, NULL);
>> +    g_free(name);
>>   
>>       return !obj ? NULL : SPAPR_DR_CONNECTOR(obj);
>>   }
>>
> 

Re: [Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 3575 bytes --]

On Mon, 31 Jul 2017 07:11:45 -0300
Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:

> Hi David,
> 
> On 07/26/2017 12:58 AM, David Gibson wrote:
> > On Tue, Jul 25, 2017 at 07:58:53PM +0200, Greg Kurz wrote:  
> >> Passing a stack allocated buffer of arbitrary length to snprintf()
> >> without checking the return value can cause the resultant strings
> >> to be silently truncated.
> >>
> >> Signed-off-by: Greg Kurz <groug@kaod.org>  
> > 
> > Applied to ppc-for-2.11.  
> 
> Isn't it 2.10 material?
> 

Hi Philippe,

Well... this patch doesn't fix any bug actually since the stack buffers
are large enough. It is more a question of coding style.

Something like below would have been more appropriate I guess:

"Building strings with g_strdup_printf() is a QEMU common practice."

No big deal.

Cheers,

--
Greg

> Regards,
> 
> Phil.
> 
> >   
> >> ---
> >>   hw/ppc/spapr_drc.c |   15 +++++++++------
> >>   1 file changed, 9 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> >> index 15bae5c216a9..e4e8383ec7b5 100644
> >> --- a/hw/ppc/spapr_drc.c
> >> +++ b/hw/ppc/spapr_drc.c
> >> @@ -488,7 +488,7 @@ static void realize(DeviceState *d, Error **errp)
> >>   {
> >>       sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
> >>       Object *root_container;
> >> -    char link_name[256];
> >> +    gchar *link_name;
> >>       gchar *child_name;
> >>       Error *err = NULL;
> >>   
> >> @@ -501,11 +501,12 @@ static void realize(DeviceState *d, Error **errp)
> >>        * existing in the composition tree
> >>        */
> >>       root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> >> -    snprintf(link_name, sizeof(link_name), "%x", spapr_drc_index(drc));
> >> +    link_name = g_strdup_printf("%x", spapr_drc_index(drc));
> >>       child_name = object_get_canonical_path_component(OBJECT(drc));
> >>       trace_spapr_drc_realize_child(spapr_drc_index(drc), child_name);
> >>       object_property_add_alias(root_container, link_name,
> >>                                 drc->owner, child_name, &err);
> >> +    g_free(link_name);
> >>       if (err) {
> >>           error_report_err(err);
> >>           object_unref(OBJECT(drc));
> >> @@ -521,13 +522,14 @@ static void unrealize(DeviceState *d, Error **errp)
> >>   {
> >>       sPAPRDRConnector *drc = SPAPR_DR_CONNECTOR(d);
> >>       Object *root_container;
> >> -    char name[256];
> >> +    gchar *name;
> >>       Error *err = NULL;
> >>   
> >>       trace_spapr_drc_unrealize(spapr_drc_index(drc));
> >>       root_container = container_get(object_get_root(), DRC_CONTAINER_PATH);
> >> -    snprintf(name, sizeof(name), "%x", spapr_drc_index(drc));
> >> +    name = g_strdup_printf("%x", spapr_drc_index(drc));
> >>       object_property_del(root_container, name, &err);
> >> +    g_free(name);
> >>       if (err) {
> >>           error_report_err(err);
> >>           object_unref(OBJECT(drc));
> >> @@ -729,10 +731,11 @@ static const TypeInfo spapr_drc_lmb_info = {
> >>   sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> >>   {
> >>       Object *obj;
> >> -    char name[256];
> >> +    gchar *name;
> >>   
> >> -    snprintf(name, sizeof(name), "%s/%x", DRC_CONTAINER_PATH, index);
> >> +    name = g_strdup_printf("%s/%x", DRC_CONTAINER_PATH, index);
> >>       obj = object_resolve_path(name, NULL);
> >> +    g_free(name);
> >>   
> >>       return !obj ? NULL : SPAPR_DR_CONNECTOR(obj);
> >>   }
> >>  
> >   


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

On Mon, Jul 31, 2017 at 12:34:41PM +0200, Greg Kurz wrote:
> On Mon, 31 Jul 2017 07:11:45 -0300
> Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> 
> > Hi David,
> > 
> > On 07/26/2017 12:58 AM, David Gibson wrote:
> > > On Tue, Jul 25, 2017 at 07:58:53PM +0200, Greg Kurz wrote:  
> > >> Passing a stack allocated buffer of arbitrary length to snprintf()
> > >> without checking the return value can cause the resultant strings
> > >> to be silently truncated.
> > >>
> > >> Signed-off-by: Greg Kurz <groug@kaod.org>  
> > > 
> > > Applied to ppc-for-2.11.  
> > 
> > Isn't it 2.10 material?
> > 
> 
> Hi Philippe,
> 
> Well... this patch doesn't fix any bug actually since the stack buffers
> are large enough. It is more a question of coding style.
> 
> Something like below would have been more appropriate I guess:
> 
> "Building strings with g_strdup_printf() is a QEMU common practice."
> 
> No big deal.

Exactly.  It's not a bugfix, so it doesn't go into 2.10 - we've passed
the hard freeze.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 04/26] spapr_drc: use g_strdup_printf() instead of snprintf()

[Philippe Mathieu-Daudé]

On 07/31/2017 09:53 AM, David Gibson wrote:
> On Mon, Jul 31, 2017 at 12:34:41PM +0200, Greg Kurz wrote:
>> On Mon, 31 Jul 2017 07:11:45 -0300
>> Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>>
>>> Hi David,
>>>
>>> On 07/26/2017 12:58 AM, David Gibson wrote:
>>>> On Tue, Jul 25, 2017 at 07:58:53PM +0200, Greg Kurz wrote:
>>>>> Passing a stack allocated buffer of arbitrary length to snprintf()
>>>>> without checking the return value can cause the resultant strings
>>>>> to be silently truncated.
>>>>>
>>>>> Signed-off-by: Greg Kurz <groug@kaod.org>
>>>>
>>>> Applied to ppc-for-2.11.
>>>
>>> Isn't it 2.10 material?
>>>
>>
>> Hi Philippe,
>>
>> Well... this patch doesn't fix any bug actually since the stack buffers
>> are large enough. It is more a question of coding style.
>>
>> Something like below would have been more appropriate I guess:
>>
>> "Building strings with g_strdup_printf() is a QEMU common practice."
>>
>> No big deal.
> 
> Exactly.  It's not a bugfix, so it doesn't go into 2.10 - we've passed
> the hard freeze.

Fair enough :)

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[Alexey Kardashevskiy]

[-- Attachment #1: Type: text/plain, Size: 2457 bytes --]

On 31/07/17 14:58, David Gibson wrote:
> On Fri, Jul 28, 2017 at 08:20:57AM +0200, Thomas Huth wrote:
>> On 28.07.2017 06:02, David Gibson wrote:
>>> On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:
>>>> The "phandle" property of the XICS node is referenced by the "interrupt-map"
>>>> property of each PHB node. This is used by the guest OS to setup IRQs for
>>>> all PCI devices.
>>>>
>>>> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
>>>> this value to a SLOF specific one, which is then presented to the guest OS.
>>>>
>>>> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
>>>> by SLOF to communicate the patched phandle value back to QEMU. This value
>>>> is then cached and preserved accross migration until machine reset.
>>>>
>>>> This is required to be able to support PHB hotplug.
>>>>
>>>> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
>>>> have to introduce its number even if QEMU currently doesn't implement it.
>>>>
>>>> Suggested-by: Thomas Huth <thuth@redhat.com>
>>>> Signed-off-by: Greg Kurz <groug@kaod.org>
>>>
>>> Ugh.  I really, really hope we can avoid this, though I don't
>>> immediately see how.  Having to have two way communication between
>>> qemu and SLOF about the device tree contents just seems like opening
>>> the door to endless complexities.
>>>
>>> This is basically a consequence of the fact that both qemu and partly
>>> responsible for constructing the device tree for the guest, and that's
>>> not easy to avoid.
>>>
>>> Hrm.. Thomas, I know it's not really the OF way, but would it be
>>> feasible to change SLOF to use the phandles as supplied by qemu rather
>>> than creating its own?
>>
>> I don't see a way to do this in an easy, clean, reasonable way. SLOF
>> uses pointers to internal structures as phandles all over the place. You
>> likely can't replace that so easily without rewriting half of the whole
>> device tree related code in SLOF, I guess...
> 
> Dang, that's what I suspected.
> 
> Just to be clear the phandles are used directly as raw pointers?
> There's not even some lookup macro we could change?
> 

We would need to rewrite
https://github.com/aik/SLOF/blob/master/slof/fs/node.fs

Since SLOF does not seem to use phandles as pointers directly anywhere but
in nofe.fs, this should be doable. Easier said than done though.


-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2784 bytes --]

On Tue, 1 Aug 2017 12:20:56 +1000
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 31/07/17 14:58, David Gibson wrote:
> > On Fri, Jul 28, 2017 at 08:20:57AM +0200, Thomas Huth wrote:  
> >> On 28.07.2017 06:02, David Gibson wrote:  
> >>> On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:  
> >>>> The "phandle" property of the XICS node is referenced by the "interrupt-map"
> >>>> property of each PHB node. This is used by the guest OS to setup IRQs for
> >>>> all PCI devices.
> >>>>
> >>>> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
> >>>> this value to a SLOF specific one, which is then presented to the guest OS.
> >>>>
> >>>> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
> >>>> by SLOF to communicate the patched phandle value back to QEMU. This value
> >>>> is then cached and preserved accross migration until machine reset.
> >>>>
> >>>> This is required to be able to support PHB hotplug.
> >>>>
> >>>> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> >>>> have to introduce its number even if QEMU currently doesn't implement it.
> >>>>
> >>>> Suggested-by: Thomas Huth <thuth@redhat.com>
> >>>> Signed-off-by: Greg Kurz <groug@kaod.org>  
> >>>
> >>> Ugh.  I really, really hope we can avoid this, though I don't
> >>> immediately see how.  Having to have two way communication between
> >>> qemu and SLOF about the device tree contents just seems like opening
> >>> the door to endless complexities.
> >>>
> >>> This is basically a consequence of the fact that both qemu and partly
> >>> responsible for constructing the device tree for the guest, and that's
> >>> not easy to avoid.
> >>>
> >>> Hrm.. Thomas, I know it's not really the OF way, but would it be
> >>> feasible to change SLOF to use the phandles as supplied by qemu rather
> >>> than creating its own?  
> >>
> >> I don't see a way to do this in an easy, clean, reasonable way. SLOF
> >> uses pointers to internal structures as phandles all over the place. You
> >> likely can't replace that so easily without rewriting half of the whole
> >> device tree related code in SLOF, I guess...  
> > 
> > Dang, that's what I suspected.
> > 
> > Just to be clear the phandles are used directly as raw pointers?
> > There's not even some lookup macro we could change?
> >   
> 
> We would need to rewrite
> https://github.com/aik/SLOF/blob/master/slof/fs/node.fs
> 
> Since SLOF does not seem to use phandles as pointers directly anywhere but
> in nofe.fs, this should be doable. Easier said than done though.
> 

Yikes... If we go that way, I'll definitely need some assistance and time.

Cc'ing Nikunj for some extra advices.

Cheers,

--
Greg

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 14260 bytes --]

On Thu, 27 Jul 2017 13:37:24 -0500
Michael Roth <mdroth@linux.vnet.ibm.com> wrote:

> Quoting Greg Kurz (2017-07-27 12:09:55)
> > On Thu, 27 Jul 2017 14:41:31 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> > > On 26/07/17 18:40, Greg Kurz wrote:  
> > > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > > main system bus, so we register spapr machine as the handler for the
> > > > main system bus.
> > > > 
> > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > > ---
> > > > - rebased against ppc-for-2.10
> > > > - converted to unplug_request
> > > > - handle drc_id at pre-plug
> > > > - reset hotplugged PHB at plug
> > > > - compatibility with older machine types
> > > > ---
> > > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > > >  hw/ppc/spapr_drc.c          |    1 
> > > >  hw/ppc/spapr_pci.c          |    2 -
> > > >  include/hw/pci-host/spapr.h |    2 +
> > > >  include/hw/ppc/spapr.h      |    1 
> > > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > index 90485054c2e7..589f76ef9fb8 100644
> > > > --- a/hw/ppc/spapr.c
> > > > +++ b/hw/ppc/spapr.c
> > > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > > >                           &savevm_htab_handlers, spapr);
> > > >  
> > > > +    if (spapr->dr_phb_enabled) {
> > > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > > +    }
> > > > +
> > > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > > >  
> > > >      if (kvm_enabled()) {
> > > > @@ -3238,6 +3242,103 @@ out:
> > > >      error_propagate(errp, local_err);
> > > >  }
> > > >  
> > > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > +                               Error **errp)
> > > > +{
> > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > +
> > > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > > +        sphb->drc_id = sphb->index;
> > > > +    }
> > > > +
> > > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > > +    }    
> > > 
> > > 
> > > sphb->index in considered 16bits in the existing code (even though it is
> > > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > > same limit for both, either 256 or 65536 is fine for me.
> > > 
> > > It is actually a bit weird - it is possible to completely configure few
> > > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > > this and just does (sphb->index << 16).  
> > 
> > You're right and this looks like a bug... I'll try to come up with a fix.  
> 
> Yup, that's a bug, and we can trigger it currently by adding 2
> additional PHBs that don't have an index specified. QOM catches
> and reports them as "attempt to add duplicate property", but it's
> just reported by spapr_dr_connector_new() and doesn't seem to be
> treated as fatal (and probably should be).
> 

Yeah, spapr_dr_connector_new() doesn't care for errors at all and
happily returns a pointer to an unrealized DRC. This causes weird
behavior. For example, when using mandatory props instead of index:

-device spapr-pci-host-bridge,id=pci1,\
                              buid=0x800000020000001,\
                              liobn=0x80000100,\
                              liobn64=0x80000101,\
                              mem_win_addr=0x200100000000,\
                              mem64_win_addr=0x220000000000,\
                              io_win_addr=0x200000010000 \
-device spapr-pci-host-bridge,id=pci2,\
                              buid=0x800000020000002,\
                              liobn=0x80000200,\
                              liobn64=0x80000201,\
                              mem_win_addr=0x200180000000,\
                              mem64_win_addr=0x230000000000,\
                              io_win_addr=0x200000020000 \
-device virtio-net,id=hp1,bus=pci1.0 \
-device virtio-net,id=hp2,bus=pci2.0 

QEMU complains and exits:

qemu-system-ppc64: -device virtio-net,id=hp2,bus=pci2.0: an attached device
 is still awaiting release

> Might also see this more in practice now with the multi-phb support
> in libvirt, though I'd imagine those would tend to rely on phb->index
> being set.
> 

I had confirmation from Shiva that libvirt passes the following to QEMU:

spapr-pci-host-bridge,index=X,id=pci.X<,numa_node=Z>

> Now that phb->drc_id is available though we can just use that instead.
> I agree it should be limited to 16-bit or smaller to avoid any
> possibility of overlap.
> 

Well, David changed his mind and now suggests we rather make index mandatory
instead of adding drc_id (even if it breaks existing setups that don't rely
on index).

> >   
> > > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > >   
> > 
> > This was how Mike did it in the original patchset but David suggested
> > to introduce drc_id (to preserve existing setups I guess):
> > 
> > http://patchwork.ozlabs.org/patch/466262/  
> 
> Althrough IIRC what David proposed was to handle it like the other
> properties which are filled in automatically when 'index' is specified,
> i.e. 'drc_id' would be set automatically by 'index' if index is
> specified, or we can set it explicitly if 'index' not specified, but
> if we try to set both 'index' and 'drc_id' we trigger the "Either "index"
> or other parameters must be specified for PAPR PHB, not both" error.
> 
> And it looks like 'index' is limited to 30, so it fits within our
> 16-bit limit for drc_id.
> 
> >   
> > > 
> > >   
> > > > +}
> > > > +
> > > > +static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > +                           Error **errp)
> > > > +{
> > > > +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > +    void *fdt = NULL;
> > > > +    int fdt_start_offset;
> > > > +    int fdt_size;
> > > > +    Error *local_err = NULL;
> > > > +    sPAPRDRConnector *drc;
> > > > +    uint32_t phandle;
> > > > +    int ret;
> > > > +    bool hotplugged = spapr_drc_hotplugged(dev);
> > > > +
> > > > +    if (!spapr->dr_phb_enabled) {
> > > > +        return;
> > > > +    }
> > > > +
> > > > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > > > +    /* hotplug hooks should check it's enabled before getting this far */
> > > > +    g_assert(drc);
> > > > +
> > > > +    if (hotplugged) {
> > > > +        if (spapr->xics_phandle == UINT32_MAX) {
> > > > +            error_setg(&local_err,
> > > > +                       "SLOF didn't update the XICS phandle. PHB hotplug cancelled");
> > > > +            goto out;
> > > > +        }
> > > > +        phandle = spapr->xics_phandle;
> > > > +
> > > > +        spapr_phb_reset(dev);    
> > > 
> > > 
> > > It could be DEVICE_GET_CLASS(dev)->reset(dev) without exporting
> > > spapr_phb_reset. Not sure how this fits into the current QEMU coding style
> > > though.
> > > 
> > > 
> > >   
> > > > +    } else {
> > > > +        phandle = PHANDLE_XICP;
> > > > +    }
> > > > +
> > > > +    fdt = create_device_tree(&fdt_size);
> > > > +    ret = spapr_populate_pci_dt(sphb, phandle, fdt, &fdt_start_offset);
> > > > +    if (ret < 0) {
> > > > +        error_setg(&local_err, "unable to create FDT for %sPHB",
> > > > +                   dev->hotplugged ? "hotplugged " : "");
> > > > +        goto out;
> > > > +    }
> > > > +
> > > > +    if (hotplugged) {
> > > > +        /* generally SLOF creates these, for hotplug it's up to QEMU */
> > > > +        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
> > > > +    }
> > > > +
> > > > +    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
> > > > +out:
> > > > +    if (local_err) {
> > > > +        error_propagate(errp, local_err);
> > > > +        g_free(fdt);
> > > > +        return;
> > > > +    }
> > > > +
> > > > +    if (hotplugged) {
> > > > +        spapr_hotplug_req_add_by_index(drc);
> > > > +    } else if (drc) {
> > > > +        spapr_drc_reset(drc);
> > > > +    }
> > > > +}
> > > > +
> > > > +void spapr_phb_release(DeviceState *dev)
> > > > +{
> > > > +    object_unparent(OBJECT(dev));
> > > > +}
> > > > +
> > > > +static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
> > > > +                                     DeviceState *dev, Error **errp)
> > > > +{
> > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > +    sPAPRDRConnector *drc;
> > > > +
> > > > +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->drc_id);
> > > > +    g_assert(drc);
> > > > +
> > > > +    if (!spapr_drc_unplug_requested(drc)) {
> > > > +        spapr_drc_detach(drc);
> > > > +        spapr_hotplug_req_remove_by_index(drc);
> > > > +    }
> > > > +}
> > > > +
> > > >  static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > > >                                        DeviceState *dev, Error **errp)
> > > >  {
> > > > @@ -3284,6 +3385,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
> > > >          spapr_memory_plug(hotplug_dev, dev, node, errp);
> > > >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > > >          spapr_core_plug(hotplug_dev, dev, errp);
> > > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > > +        spapr_phb_plug(hotplug_dev, dev, errp);
> > > >      }
> > > >  }
> > > >  
> > > > @@ -3311,6 +3414,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
> > > >              return;
> > > >          }
> > > >          spapr_core_unplug_request(hotplug_dev, dev, errp);
> > > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > > +        if (sms->dr_phb_enabled) {
> > > > +            spapr_phb_unplug_request(hotplug_dev, dev, errp);
> > > > +        } else {
> > > > +            error_setg(errp, "PHB hot unplug not supported on this machine");
> > > > +        }
> > > >      }
> > > >  }
> > > >  
> > > > @@ -3321,6 +3430,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
> > > >          spapr_memory_pre_plug(hotplug_dev, dev, errp);
> > > >      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > > >          spapr_core_pre_plug(hotplug_dev, dev, errp);
> > > > +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > > +        spapr_phb_pre_plug(hotplug_dev, dev, errp);
> > > >      }
> > > >  }
> > > >  
> > > > @@ -3328,7 +3439,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
> > > >                                                   DeviceState *dev)
> > > >  {
> > > >      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> > > > -        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> > > > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
> > > > +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> > > >          return HOTPLUG_HANDLER(machine);
> > > >      }
> > > >      return NULL;
> > > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > > index 2e1049ce61c7..845fcf70b932 100644
> > > > --- a/hw/ppc/spapr_drc.c
> > > > +++ b/hw/ppc/spapr_drc.c
> > > > @@ -704,6 +704,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > >      drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > >      drck->typename = "PHB";
> > > >      drck->drc_name_prefix = "PHB ";
> > > > +    drck->release = spapr_phb_release;
> > > >  }
> > > >  
> > > >  static const TypeInfo spapr_dr_connector_info = {
> > > > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > > > index 157867af8178..c12f71ae3e2d 100644
> > > > --- a/hw/ppc/spapr_pci.c
> > > > +++ b/hw/ppc/spapr_pci.c
> > > > @@ -1833,7 +1833,7 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb)
> > > >                             sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT);
> > > >  }
> > > >  
> > > > -static void spapr_phb_reset(DeviceState *qdev)
> > > > +void spapr_phb_reset(DeviceState *qdev)
> > > >  {
> > > >      sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(qdev);
> > > >  
> > > > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > > > index 7837fb0b1110..15799cee4280 100644
> > > > --- a/include/hw/pci-host/spapr.h
> > > > +++ b/include/hw/pci-host/spapr.h
> > > > @@ -120,6 +120,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb,
> > > >  
> > > >  void spapr_pci_rtas_init(void);
> > > >  
> > > > +void spapr_phb_reset(DeviceState *qdev);
> > > > +
> > > >  sPAPRPHBState *spapr_pci_find_phb(sPAPRMachineState *spapr, uint64_t buid);
> > > >  PCIDevice *spapr_pci_find_dev(sPAPRMachineState *spapr, uint64_t buid,
> > > >                                uint32_t config_addr);
> > > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > > > index f09c54d5bb94..a2f6782bdbbf 100644
> > > > --- a/include/hw/ppc/spapr.h
> > > > +++ b/include/hw/ppc/spapr.h
> > > > @@ -673,6 +673,7 @@ void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
> > > >  /* CPU and LMB DRC release callbacks. */
> > > >  void spapr_core_release(DeviceState *dev);
> > > >  void spapr_lmb_release(DeviceState *dev);
> > > > +void spapr_phb_release(DeviceState *dev);
> > > >  
> > > >  void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
> > > >  int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);
> > > > 
> > > >     
> > > 
> > >   
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 4315 bytes --]

On Fri, 28 Jul 2017 14:24:03 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Thu, Jul 27, 2017 at 07:09:55PM +0200, Greg Kurz wrote:
> > On Thu, 27 Jul 2017 14:41:31 +1000
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> > > On 26/07/17 18:40, Greg Kurz wrote:  
> > > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > > main system bus, so we register spapr machine as the handler for the
> > > > main system bus.
> > > > 
> > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > > ---
> > > > - rebased against ppc-for-2.10
> > > > - converted to unplug_request
> > > > - handle drc_id at pre-plug
> > > > - reset hotplugged PHB at plug
> > > > - compatibility with older machine types
> > > > ---
> > > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > > >  hw/ppc/spapr_drc.c          |    1 
> > > >  hw/ppc/spapr_pci.c          |    2 -
> > > >  include/hw/pci-host/spapr.h |    2 +
> > > >  include/hw/ppc/spapr.h      |    1 
> > > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > index 90485054c2e7..589f76ef9fb8 100644
> > > > --- a/hw/ppc/spapr.c
> > > > +++ b/hw/ppc/spapr.c
> > > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > > >                           &savevm_htab_handlers, spapr);
> > > >  
> > > > +    if (spapr->dr_phb_enabled) {
> > > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > > +    }
> > > > +
> > > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > > >  
> > > >      if (kvm_enabled()) {
> > > > @@ -3238,6 +3242,103 @@ out:
> > > >      error_propagate(errp, local_err);
> > > >  }
> > > >  
> > > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > +                               Error **errp)
> > > > +{
> > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > +
> > > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > > +        sphb->drc_id = sphb->index;
> > > > +    }
> > > > +
> > > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > > +    }    
> > > 
> > > 
> > > sphb->index in considered 16bits in the existing code (even though it is
> > > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > > same limit for both, either 256 or 65536 is fine for me.
> > > 
> > > It is actually a bit weird - it is possible to completely configure few
> > > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > > this and just does (sphb->index << 16).  
> > 
> > You're right and this looks like a bug... I'll try to come up with a fix.
> >   
> > > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > >   
> > 
> > This was how Mike did it in the original patchset but David suggested
> > to introduce drc_id (to preserve existing setups I guess):
> > 
> > http://patchwork.ozlabs.org/patch/466262/  
> 
> Huh.  So I did.  But.. sorry, I've changed my mind.
> 
> The fact that needing a DRC forces us to have a reasonable small id
> for each PHB seems like a good excuse to make index mandatory - I'm
> not convinced anyone was actually creating PHBs without index, and
> this does allow us to simplify a bunch of things.
> 
> I'd like to see that done as a preliminary cleanup patch, though.
> 

Just to be sure. I could verify that the weirdness reported by Alexey
causes QEMU to misbehave. Only the first "index-less" PHB has realized
DRCs:

=> subsequent "index-less" PHBs silently ignore hotplugging of PCI devices

=> QEMU won't even start with coldplugged devices in these "index-less"
   PHBs

This preliminary cleanup for hotpluggable PHBs is hence also a bug fix
for current PHBs.

Do we want to fix this long-standing bug in 2.10 ?

Do we want to preserve the current buggy behavior for older machine types ?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 24/26] spapr: allow guest to update the XICS phandle

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3407 bytes --]

On Tue, Aug 01, 2017 at 01:26:15PM +0200, Greg Kurz wrote:
> On Tue, 1 Aug 2017 12:20:56 +1000
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > On 31/07/17 14:58, David Gibson wrote:
> > > On Fri, Jul 28, 2017 at 08:20:57AM +0200, Thomas Huth wrote:  
> > >> On 28.07.2017 06:02, David Gibson wrote:  
> > >>> On Tue, Jul 25, 2017 at 08:03:06PM +0200, Greg Kurz wrote:  
> > >>>> The "phandle" property of the XICS node is referenced by the "interrupt-map"
> > >>>> property of each PHB node. This is used by the guest OS to setup IRQs for
> > >>>> all PCI devices.
> > >>>>
> > >>>> QEMU uses an arbitrary value (0x1111) for this phandle, but SLOF converts
> > >>>> this value to a SLOF specific one, which is then presented to the guest OS.
> > >>>>
> > >>>> This patches introduces the new KVMPPC_H_UPDATE_PHANDLE hcall, which is used
> > >>>> by SLOF to communicate the patched phandle value back to QEMU. This value
> > >>>> is then cached and preserved accross migration until machine reset.
> > >>>>
> > >>>> This is required to be able to support PHB hotplug.
> > >>>>
> > >>>> Note, that SLOF already has some code to call KVMPPC_H_RTAS_UPDATE, so we
> > >>>> have to introduce its number even if QEMU currently doesn't implement it.
> > >>>>
> > >>>> Suggested-by: Thomas Huth <thuth@redhat.com>
> > >>>> Signed-off-by: Greg Kurz <groug@kaod.org>  
> > >>>
> > >>> Ugh.  I really, really hope we can avoid this, though I don't
> > >>> immediately see how.  Having to have two way communication between
> > >>> qemu and SLOF about the device tree contents just seems like opening
> > >>> the door to endless complexities.
> > >>>
> > >>> This is basically a consequence of the fact that both qemu and partly
> > >>> responsible for constructing the device tree for the guest, and that's
> > >>> not easy to avoid.
> > >>>
> > >>> Hrm.. Thomas, I know it's not really the OF way, but would it be
> > >>> feasible to change SLOF to use the phandles as supplied by qemu rather
> > >>> than creating its own?  
> > >>
> > >> I don't see a way to do this in an easy, clean, reasonable way. SLOF
> > >> uses pointers to internal structures as phandles all over the place. You
> > >> likely can't replace that so easily without rewriting half of the whole
> > >> device tree related code in SLOF, I guess...  
> > > 
> > > Dang, that's what I suspected.
> > > 
> > > Just to be clear the phandles are used directly as raw pointers?
> > > There's not even some lookup macro we could change?
> > >   
> > 
> > We would need to rewrite
> > https://github.com/aik/SLOF/blob/master/slof/fs/node.fs
> > 
> > Since SLOF does not seem to use phandles as pointers directly anywhere but
> > in nofe.fs, this should be doable. Easier said than done though.
> > 
> 
> Yikes... If we go that way, I'll definitely need some assistance and time.
> 
> Cc'ing Nikunj for some extra advices.

Yeah, I hope we could get a feasibility idea from someone who knows
Forth - Thomas or Nikunj, maybe.

I'd be thinking of replacing the direct pointer dereferences for
phandles with a simple lookup table of phandle to pointer, populated
as we unflatten the tree.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5070 bytes --]

On Tue, Aug 01, 2017 at 05:30:46PM +0200, Greg Kurz wrote:
> On Fri, 28 Jul 2017 14:24:03 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Thu, Jul 27, 2017 at 07:09:55PM +0200, Greg Kurz wrote:
> > > On Thu, 27 Jul 2017 14:41:31 +1000
> > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > >   
> > > > On 26/07/17 18:40, Greg Kurz wrote:  
> > > > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > > > main system bus, so we register spapr machine as the handler for the
> > > > > main system bus.
> > > > > 
> > > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > > > ---
> > > > > - rebased against ppc-for-2.10
> > > > > - converted to unplug_request
> > > > > - handle drc_id at pre-plug
> > > > > - reset hotplugged PHB at plug
> > > > > - compatibility with older machine types
> > > > > ---
> > > > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > > > >  hw/ppc/spapr_drc.c          |    1 
> > > > >  hw/ppc/spapr_pci.c          |    2 -
> > > > >  include/hw/pci-host/spapr.h |    2 +
> > > > >  include/hw/ppc/spapr.h      |    1 
> > > > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > index 90485054c2e7..589f76ef9fb8 100644
> > > > > --- a/hw/ppc/spapr.c
> > > > > +++ b/hw/ppc/spapr.c
> > > > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > > > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > > > >                           &savevm_htab_handlers, spapr);
> > > > >  
> > > > > +    if (spapr->dr_phb_enabled) {
> > > > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > > > +    }
> > > > > +
> > > > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > > > >  
> > > > >      if (kvm_enabled()) {
> > > > > @@ -3238,6 +3242,103 @@ out:
> > > > >      error_propagate(errp, local_err);
> > > > >  }
> > > > >  
> > > > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > > +                               Error **errp)
> > > > > +{
> > > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > > +
> > > > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > > > +        sphb->drc_id = sphb->index;
> > > > > +    }
> > > > > +
> > > > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > > > +    }    
> > > > 
> > > > 
> > > > sphb->index in considered 16bits in the existing code (even though it is
> > > > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > > > same limit for both, either 256 or 65536 is fine for me.
> > > > 
> > > > It is actually a bit weird - it is possible to completely configure few
> > > > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > > > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > > > this and just does (sphb->index << 16).  
> > > 
> > > You're right and this looks like a bug... I'll try to come up with a fix.
> > >   
> > > > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > > >   
> > > 
> > > This was how Mike did it in the original patchset but David suggested
> > > to introduce drc_id (to preserve existing setups I guess):
> > > 
> > > http://patchwork.ozlabs.org/patch/466262/  
> > 
> > Huh.  So I did.  But.. sorry, I've changed my mind.
> > 
> > The fact that needing a DRC forces us to have a reasonable small id
> > for each PHB seems like a good excuse to make index mandatory - I'm
> > not convinced anyone was actually creating PHBs without index, and
> > this does allow us to simplify a bunch of things.
> > 
> > I'd like to see that done as a preliminary cleanup patch, though.
> > 
> 
> Just to be sure. I could verify that the weirdness reported by Alexey
> causes QEMU to misbehave. Only the first "index-less" PHB has realized
> DRCs:
> 
> => subsequent "index-less" PHBs silently ignore hotplugging of PCI devices
> 
> => QEMU won't even start with coldplugged devices in these "index-less"
>    PHBs
> 
> This preliminary cleanup for hotpluggable PHBs is hence also a bug fix
> for current PHBs.

Ok.

> Do we want to fix this long-standing bug in 2.10 ?

No, not worth pushing in this late.

> Do we want to preserve the current buggy behavior for older machine
> types ?

No, I don't think so.  I think the reasonable course here is to push
the new behaviour out.  Only if someone complains that they were
actually relying on the old behaviour in the wild do we try to
preserve it for the older machine types.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [for-2.11 PATCH 26/26] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 5357 bytes --]

On Wed, 2 Aug 2017 12:39:12 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Tue, Aug 01, 2017 at 05:30:46PM +0200, Greg Kurz wrote:
> > On Fri, 28 Jul 2017 14:24:03 +1000
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >   
> > > On Thu, Jul 27, 2017 at 07:09:55PM +0200, Greg Kurz wrote:  
> > > > On Thu, 27 Jul 2017 14:41:31 +1000
> > > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> > > >     
> > > > > On 26/07/17 18:40, Greg Kurz wrote:    
> > > > > > Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> > > > > > main system bus, so we register spapr machine as the handler for the
> > > > > > main system bus.
> > > > > > 
> > > > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > > > Signed-off-by: Greg Kurz <groug@kaod.org>
> > > > > > ---
> > > > > > - rebased against ppc-for-2.10
> > > > > > - converted to unplug_request
> > > > > > - handle drc_id at pre-plug
> > > > > > - reset hotplugged PHB at plug
> > > > > > - compatibility with older machine types
> > > > > > ---
> > > > > >  hw/ppc/spapr.c              |  114 +++++++++++++++++++++++++++++++++++++++++++
> > > > > >  hw/ppc/spapr_drc.c          |    1 
> > > > > >  hw/ppc/spapr_pci.c          |    2 -
> > > > > >  include/hw/pci-host/spapr.h |    2 +
> > > > > >  include/hw/ppc/spapr.h      |    1 
> > > > > >  5 files changed, 118 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > > index 90485054c2e7..589f76ef9fb8 100644
> > > > > > --- a/hw/ppc/spapr.c
> > > > > > +++ b/hw/ppc/spapr.c
> > > > > > @@ -2540,6 +2540,10 @@ static void ppc_spapr_init(MachineState *machine)
> > > > > >      register_savevm_live(NULL, "spapr/htab", -1, 1,
> > > > > >                           &savevm_htab_handlers, spapr);
> > > > > >  
> > > > > > +    if (spapr->dr_phb_enabled) {
> > > > > > +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine), NULL);
> > > > > > +    }
> > > > > > +
> > > > > >      qemu_register_boot_set(spapr_boot_set, spapr);
> > > > > >  
> > > > > >      if (kvm_enabled()) {
> > > > > > @@ -3238,6 +3242,103 @@ out:
> > > > > >      error_propagate(errp, local_err);
> > > > > >  }
> > > > > >  
> > > > > > +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> > > > > > +                               Error **errp)
> > > > > > +{
> > > > > > +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> > > > > > +
> > > > > > +    if (sphb->drc_id == (uint32_t)-1) {
> > > > > > +        sphb->drc_id = sphb->index;
> > > > > > +    }
> > > > > > +
> > > > > > +    if (sphb->drc_id >= SPAPR_DRC_MAX_PHB) {
> > > > > > +        error_setg(errp, "PHB id %d out of range", sphb->drc_id);
> > > > > > +    }      
> > > > > 
> > > > > 
> > > > > sphb->index in considered 16bits in the existing code (even though it is
> > > > > defined as 32bit) and SPAPR_DRC_MAX_PHB is just 256. I'd suggest using the
> > > > > same limit for both, either 256 or 65536 is fine for me.
> > > > > 
> > > > > It is actually a bit weird - it is possible to completely configure few
> > > > > PHBs in the command line so they will have index==-1 but PCI hotplug code -
> > > > > spapr_phb_get_pci_func_drc() and spapr_phb_realize() - does not check for
> > > > > this and just does (sphb->index << 16).    
> > > > 
> > > > You're right and this looks like a bug... I'll try to come up with a fix.
> > > >     
> > > > > May be just ditch drc_id, enforce index not to be -1 and use it as drc_id?
> > > > >     
> > > > 
> > > > This was how Mike did it in the original patchset but David suggested
> > > > to introduce drc_id (to preserve existing setups I guess):
> > > > 
> > > > http://patchwork.ozlabs.org/patch/466262/    
> > > 
> > > Huh.  So I did.  But.. sorry, I've changed my mind.
> > > 
> > > The fact that needing a DRC forces us to have a reasonable small id
> > > for each PHB seems like a good excuse to make index mandatory - I'm
> > > not convinced anyone was actually creating PHBs without index, and
> > > this does allow us to simplify a bunch of things.
> > > 
> > > I'd like to see that done as a preliminary cleanup patch, though.
> > >   
> > 
> > Just to be sure. I could verify that the weirdness reported by Alexey
> > causes QEMU to misbehave. Only the first "index-less" PHB has realized
> > DRCs:
> >   
> > => subsequent "index-less" PHBs silently ignore hotplugging of PCI devices  
> >   
> > => QEMU won't even start with coldplugged devices in these "index-less"  
> >    PHBs
> > 
> > This preliminary cleanup for hotpluggable PHBs is hence also a bug fix
> > for current PHBs.  
> 
> Ok.
> 
> > Do we want to fix this long-standing bug in 2.10 ?  
> 
> No, not worth pushing in this late.
> 

Cool. This will give us enough time to do it right.

> > Do we want to preserve the current buggy behavior for older machine
> > types ?  
> 
> No, I don't think so.  I think the reasonable course here is to push
> the new behaviour out.  Only if someone complains that they were
> actually relying on the old behaviour in the wild do we try to
> preserve it for the older machine types.
> 

Perfect ! I was hoping you'd say that. :)

Cheers,

--
Greg

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 7540 bytes --]

On Mon, 31 Jul 2017 12:58:25 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Fri, Jul 28, 2017 at 12:30:35PM +0200, Greg Kurz wrote:
> > On Fri, 28 Jul 2017 13:49:25 +1000
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >   
> > > On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:  
> > > > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > 
> > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > > > Signed-off-by: Greg Kurz <groug@kaod.org>    
> > > 
> > > 
> > >   
> > > > ---
> > > > Changes since RFC:
> > > > - rebased against ppc-for-2.10 (reset hooks registering already merged)
> > > > - added new DRC type for PHB
> > > > ---
> > > >  hw/ppc/spapr.c             |   15 +++++++++++++++
> > > >  hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
> > > >  include/hw/ppc/spapr_drc.h |    8 ++++++++
> > > >  3 files changed, 40 insertions(+)
> > > > 
> > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > index 8dc505343c0f..5950c009ab7e 100644
> > > > --- a/hw/ppc/spapr.c
> > > > +++ b/hw/ppc/spapr.c
> > > > @@ -98,6 +98,9 @@
> > > >  
> > > >  #define PHANDLE_XICP            0x00001111
> > > >  
> > > > +/* maximum number of hotpluggable PHBs */
> > > > +#define SPAPR_DRC_MAX_PHB       256    
> > > 
> > > I wonder if we should actually make this a machine property.
> > >   
> > 
> > It makes sense.
> > 
> > Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> > Maybe this could be the default value for the machine property instead of
> > 256 then ?  
> 
> Actually, if we're binding it back to index, which has a hard limit,
> then it no longer makes sense to have it as a property and we should
> go back to a constant (well, it could vary by machine type version).
> 

But I guess that the hard limit of 31 as described in the changelog of
commit 357d1e3bc7d2d80e5271bc4f3ac8537e30dc8046 still holds, doesn't it ?

    Because some guest versions (including most current distro kernels) can't
    access PCI MMIO above 64 TiB, we put all the PCI windows between 32 TiB and
    64 TiB.  This is broken into 1 TiB chunks.  The first 1 TiB contains the
    PIO (64 kiB) and 32-bit MMIO (2 GiB) windows for all of the PHBs.  Each
    subsequent TiB chunk contains a naturally aligned 64-bit MMIO window for
    one PHB each.
    
    This reduces the number of allowed PHBs (without full manual configuration
    of all the windows) from 256 to 31, but this should still be plenty in
    practice.

Not sure why a machine type version would have a different limit. Can
you think of a use case ?

> > > >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> > > >                                    const char *type_ics,
> > > >                                    int nr_irqs, Error **errp)
> > > > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> > > >  
> > > >      spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > > >  
> > > > +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > > > +     * connectors (described in root DT node's "ibm,drc-types" property)
> > > > +     * are pre-initialized here. additional child connectors (such as
> > > > +     * connectors for a PHBs PCI slots) are added as needed during their
> > > > +     * parent's realization.
> > > > +     */
> > > > +    if (spapr->dr_phb_enabled) {
> > > > +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > > > +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> > > > +        }
> > > > +    }
> > > > +
> > > >      /* Set up PCI */
> > > >      spapr_pci_rtas_init();
> > > >  
> > > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > > index eb8024d37c54..2e1049ce61c7 100644
> > > > --- a/hw/ppc/spapr_drc.c
> > > > +++ b/hw/ppc/spapr_drc.c
> > > > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
> > > >      drck->release = spapr_lmb_release;
> > > >  }
> > > >  
> > > > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > > +{
> > > > +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > > > +
> > > > +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > > +    drck->typename = "PHB";
> > > > +    drck->drc_name_prefix = "PHB ";
> > > > +}
> > > > +
> > > >  static const TypeInfo spapr_dr_connector_info = {
> > > >      .name          = TYPE_SPAPR_DR_CONNECTOR,
> > > >      .parent        = TYPE_DEVICE,
> > > > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> > > >      .class_init    = spapr_drc_lmb_class_init,
> > > >  };
> > > >  
> > > > +static const TypeInfo spapr_drc_phb_info = {
> > > > +    .name          = TYPE_SPAPR_DRC_PHB,
> > > > +    .parent        = TYPE_SPAPR_DRC_LOGICAL,    
> > > 
> > > I thought PHB DRCs were physical..
> > >   
> > 
> > My understanding is that only PCI IOAs need a physical DRC.
> > 
> > From LoPAPR v1.1 (March 24, 2016):
> > 
> > 13.7 Logical Resource Dynamic Reconfiguration (LRDR)
> > 
> > The Logical Resource Dynamic Reconfiguration option allows a platform to make available and recover platform re-
> > sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots to/from its operating OS image(s).
> > 
> > ...
> > 
> > The device tree contains logical resource DR connectors for the maximum number of resources that the platform can
> > allocate to the specific OS. In some cases such as for processors and PHBs...
> > 
> > and
> > 
> > Table 240. Currently Defined DR Connector Types
> > 
> > | PHB | Logical PCI Host Bridge |  
> 
> Ah, my mistake.
> 
> >   
> > > > +    .instance_size = sizeof(sPAPRDRConnector),
> > > > +    .class_init    = spapr_drc_phb_class_init,
> > > > +};
> > > > +
> > > >  /* helper functions for external users */
> > > >  
> > > >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > > > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> > > >      type_register_static(&spapr_drc_cpu_info);
> > > >      type_register_static(&spapr_drc_pci_info);
> > > >      type_register_static(&spapr_drc_lmb_info);
> > > > +    type_register_static(&spapr_drc_phb_info);
> > > >  
> > > >      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
> > > >                          rtas_set_indicator);
> > > > diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> > > > index a7958d0a8d14..535fc61b98a8 100644
> > > > --- a/include/hw/ppc/spapr_drc.h
> > > > +++ b/include/hw/ppc/spapr_drc.h
> > > > @@ -69,6 +69,14 @@
> > > >  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > >                                          TYPE_SPAPR_DRC_LMB)
> > > >  
> > > > +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> > > > +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> > > > +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> > > > +#define SPAPR_DRC_PHB_CLASS(klass) \
> > > > +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> > > > +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > > +                                        TYPE_SPAPR_DRC_PHB)
> > > > +
> > > >  /*
> > > >   * Various hotplug types managed by sPAPRDRConnector
> > > >   *
> > > >     
> > >   
> >   
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 8787 bytes --]

On Wed, Sep 06, 2017 at 01:32:09PM +0200, Greg Kurz wrote:
> On Mon, 31 Jul 2017 12:58:25 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Fri, Jul 28, 2017 at 12:30:35PM +0200, Greg Kurz wrote:
> > > On Fri, 28 Jul 2017 13:49:25 +1000
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > On Tue, Jul 25, 2017 at 08:01:50PM +0200, Greg Kurz wrote:  
> > > > > From: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > > 
> > > > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > > > Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> > > > > Signed-off-by: Greg Kurz <groug@kaod.org>    
> > > > 
> > > > 
> > > >   
> > > > > ---
> > > > > Changes since RFC:
> > > > > - rebased against ppc-for-2.10 (reset hooks registering already merged)
> > > > > - added new DRC type for PHB
> > > > > ---
> > > > >  hw/ppc/spapr.c             |   15 +++++++++++++++
> > > > >  hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
> > > > >  include/hw/ppc/spapr_drc.h |    8 ++++++++
> > > > >  3 files changed, 40 insertions(+)
> > > > > 
> > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > > > index 8dc505343c0f..5950c009ab7e 100644
> > > > > --- a/hw/ppc/spapr.c
> > > > > +++ b/hw/ppc/spapr.c
> > > > > @@ -98,6 +98,9 @@
> > > > >  
> > > > >  #define PHANDLE_XICP            0x00001111
> > > > >  
> > > > > +/* maximum number of hotpluggable PHBs */
> > > > > +#define SPAPR_DRC_MAX_PHB       256    
> > > > 
> > > > I wonder if we should actually make this a machine property.
> > > >   
> > > 
> > > It makes sense.
> > > 
> > > Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> > > Maybe this could be the default value for the machine property instead of
> > > 256 then ?  
> > 
> > Actually, if we're binding it back to index, which has a hard limit,
> > then it no longer makes sense to have it as a property and we should
> > go back to a constant (well, it could vary by machine type version).

Sorry I've taken so long to reply.

> But I guess that the hard limit of 31 as described in the changelog of
> commit 357d1e3bc7d2d80e5271bc4f3ac8537e30dc8046 still holds, doesn't
> it ?

That's right.  Note that that is a limit of *31* PHBs (numbered
0..30), not 32 PHBs numbered 0..31.

>     Because some guest versions (including most current distro kernels) can't
>     access PCI MMIO above 64 TiB, we put all the PCI windows between 32 TiB and
>     64 TiB.  This is broken into 1 TiB chunks.  The first 1 TiB contains the
>     PIO (64 kiB) and 32-bit MMIO (2 GiB) windows for all of the PHBs.  Each
>     subsequent TiB chunk contains a naturally aligned 64-bit MMIO window for
>     one PHB each.
>     
>     This reduces the number of allowed PHBs (without full manual configuration
>     of all the windows) from 256 to 31, but this should still be plenty in
>     practice.
> 
> Not sure why a machine type version would have a different limit. Can
> you think of a use case ?

Well, the older machine types had a different layout.  It allowed for
more indexes, but had smaller windows, which meant certain cards (e.g.
GPGPUs with huge BARs) didn't work properly.  It also had some weird
alignments that meant we were a bit wasteful of address space.

But we can't change the location of PHB windows on migration, so we
had to maintain that old layout for old machine types.  That's why
there's a different limit depending on machine type version.

> 
> > > > >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> > > > >                                    const char *type_ics,
> > > > >                                    int nr_irqs, Error **errp)
> > > > > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> > > > >  
> > > > >      spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > > > >  
> > > > > +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > > > > +     * connectors (described in root DT node's "ibm,drc-types" property)
> > > > > +     * are pre-initialized here. additional child connectors (such as
> > > > > +     * connectors for a PHBs PCI slots) are added as needed during their
> > > > > +     * parent's realization.
> > > > > +     */
> > > > > +    if (spapr->dr_phb_enabled) {
> > > > > +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > > > > +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> > > > > +        }
> > > > > +    }
> > > > > +
> > > > >      /* Set up PCI */
> > > > >      spapr_pci_rtas_init();
> > > > >  
> > > > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > > > index eb8024d37c54..2e1049ce61c7 100644
> > > > > --- a/hw/ppc/spapr_drc.c
> > > > > +++ b/hw/ppc/spapr_drc.c
> > > > > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
> > > > >      drck->release = spapr_lmb_release;
> > > > >  }
> > > > >  
> > > > > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > > > +{
> > > > > +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > > > > +
> > > > > +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > > > +    drck->typename = "PHB";
> > > > > +    drck->drc_name_prefix = "PHB ";
> > > > > +}
> > > > > +
> > > > >  static const TypeInfo spapr_dr_connector_info = {
> > > > >      .name          = TYPE_SPAPR_DR_CONNECTOR,
> > > > >      .parent        = TYPE_DEVICE,
> > > > > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> > > > >      .class_init    = spapr_drc_lmb_class_init,
> > > > >  };
> > > > >  
> > > > > +static const TypeInfo spapr_drc_phb_info = {
> > > > > +    .name          = TYPE_SPAPR_DRC_PHB,
> > > > > +    .parent        = TYPE_SPAPR_DRC_LOGICAL,    
> > > > 
> > > > I thought PHB DRCs were physical..
> > > >   
> > > 
> > > My understanding is that only PCI IOAs need a physical DRC.
> > > 
> > > From LoPAPR v1.1 (March 24, 2016):
> > > 
> > > 13.7 Logical Resource Dynamic Reconfiguration (LRDR)
> > > 
> > > The Logical Resource Dynamic Reconfiguration option allows a platform to make available and recover platform re-
> > > sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots to/from its operating OS image(s).
> > > 
> > > ...
> > > 
> > > The device tree contains logical resource DR connectors for the maximum number of resources that the platform can
> > > allocate to the specific OS. In some cases such as for processors and PHBs...
> > > 
> > > and
> > > 
> > > Table 240. Currently Defined DR Connector Types
> > > 
> > > | PHB | Logical PCI Host Bridge |  
> > 
> > Ah, my mistake.
> > 
> > >   
> > > > > +    .instance_size = sizeof(sPAPRDRConnector),
> > > > > +    .class_init    = spapr_drc_phb_class_init,
> > > > > +};
> > > > > +
> > > > >  /* helper functions for external users */
> > > > >  
> > > > >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > > > > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> > > > >      type_register_static(&spapr_drc_cpu_info);
> > > > >      type_register_static(&spapr_drc_pci_info);
> > > > >      type_register_static(&spapr_drc_lmb_info);
> > > > > +    type_register_static(&spapr_drc_phb_info);
> > > > >  
> > > > >      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
> > > > >                          rtas_set_indicator);
> > > > > diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> > > > > index a7958d0a8d14..535fc61b98a8 100644
> > > > > --- a/include/hw/ppc/spapr_drc.h
> > > > > +++ b/include/hw/ppc/spapr_drc.h
> > > > > @@ -69,6 +69,14 @@
> > > > >  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > > >                                          TYPE_SPAPR_DRC_LMB)
> > > > >  
> > > > > +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> > > > > +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> > > > > +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> > > > > +#define SPAPR_DRC_PHB_CLASS(klass) \
> > > > > +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> > > > > +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > > > +                                        TYPE_SPAPR_DRC_PHB)
> > > > > +
> > > > >  /*
> > > > >   * Various hotplug types managed by sPAPRDRConnector
> > > > >   *
> > > > >     
> > > >   
> > >   
> > 
> > 
> > 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 7697 bytes --]

On Wed, 13 Sep 2017 22:23:29 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

[...snip...]
> > > > Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> > > > Maybe this could be the default value for the machine property instead of
> > > > 256 then ?    
> > > 
> > > Actually, if we're binding it back to index, which has a hard limit,
> > > then it no longer makes sense to have it as a property and we should
> > > go back to a constant (well, it could vary by machine type version).  
> 
> Sorry I've taken so long to reply.
> 

Oh, don't mention it. :)

> > But I guess that the hard limit of 31 as described in the changelog of
> > commit 357d1e3bc7d2d80e5271bc4f3ac8537e30dc8046 still holds, doesn't
> > it ?  
> 
> That's right.  Note that that is a limit of *31* PHBs (numbered
> 0..30), not 32 PHBs numbered 0..31.
> 

Yeah I saw that.

> >     Because some guest versions (including most current distro kernels) can't
> >     access PCI MMIO above 64 TiB, we put all the PCI windows between 32 TiB and
> >     64 TiB.  This is broken into 1 TiB chunks.  The first 1 TiB contains the
> >     PIO (64 kiB) and 32-bit MMIO (2 GiB) windows for all of the PHBs.  Each
> >     subsequent TiB chunk contains a naturally aligned 64-bit MMIO window for
> >     one PHB each.
> >     
> >     This reduces the number of allowed PHBs (without full manual configuration
> >     of all the windows) from 256 to 31, but this should still be plenty in
> >     practice.
> > 
> > Not sure why a machine type version would have a different limit. Can
> > you think of a use case ?  
> 
> Well, the older machine types had a different layout.  It allowed for
> more indexes, but had smaller windows, which meant certain cards (e.g.
> GPGPUs with huge BARs) didn't work properly.  It also had some weird
> alignments that meant we were a bit wasteful of address space.
> 
> But we can't change the location of PHB windows on migration, so we
> had to maintain that old layout for old machine types.  That's why
> there's a different limit depending on machine type version.
> 

Ok, so we *just* have 2 different maximum number of PHBs:
- 256 for pseries <= 2.7
- 31 for newer machine types

I sent a RFC patch last monday:

https://lists.nongnu.org/archive/html/qemu-ppc/2017-09/msg00203.html

> >   
> > > > > >  static ICSState *spapr_ics_create(sPAPRMachineState *spapr,
> > > > > >                                    const char *type_ics,
> > > > > >                                    int nr_irqs, Error **errp)
> > > > > > @@ -2384,6 +2387,18 @@ static void ppc_spapr_init(MachineState *machine)
> > > > > >  
> > > > > >      spapr->dr_phb_enabled = smc->dr_phb_enabled;
> > > > > >  
> > > > > > +    /* Setup hotplug / dynamic-reconfiguration connectors. top-level
> > > > > > +     * connectors (described in root DT node's "ibm,drc-types" property)
> > > > > > +     * are pre-initialized here. additional child connectors (such as
> > > > > > +     * connectors for a PHBs PCI slots) are added as needed during their
> > > > > > +     * parent's realization.
> > > > > > +     */
> > > > > > +    if (spapr->dr_phb_enabled) {
> > > > > > +        for (i = 0; i < SPAPR_DRC_MAX_PHB; i++) {
> > > > > > +            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
> > > > > > +        }
> > > > > > +    }
> > > > > > +
> > > > > >      /* Set up PCI */
> > > > > >      spapr_pci_rtas_init();
> > > > > >  
> > > > > > diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> > > > > > index eb8024d37c54..2e1049ce61c7 100644
> > > > > > --- a/hw/ppc/spapr_drc.c
> > > > > > +++ b/hw/ppc/spapr_drc.c
> > > > > > @@ -697,6 +697,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
> > > > > >      drck->release = spapr_lmb_release;
> > > > > >  }
> > > > > >  
> > > > > > +static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
> > > > > > +{
> > > > > > +    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
> > > > > > +
> > > > > > +    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
> > > > > > +    drck->typename = "PHB";
> > > > > > +    drck->drc_name_prefix = "PHB ";
> > > > > > +}
> > > > > > +
> > > > > >  static const TypeInfo spapr_dr_connector_info = {
> > > > > >      .name          = TYPE_SPAPR_DR_CONNECTOR,
> > > > > >      .parent        = TYPE_DEVICE,
> > > > > > @@ -740,6 +749,13 @@ static const TypeInfo spapr_drc_lmb_info = {
> > > > > >      .class_init    = spapr_drc_lmb_class_init,
> > > > > >  };
> > > > > >  
> > > > > > +static const TypeInfo spapr_drc_phb_info = {
> > > > > > +    .name          = TYPE_SPAPR_DRC_PHB,
> > > > > > +    .parent        = TYPE_SPAPR_DRC_LOGICAL,      
> > > > > 
> > > > > I thought PHB DRCs were physical..
> > > > >     
> > > > 
> > > > My understanding is that only PCI IOAs need a physical DRC.
> > > > 
> > > > From LoPAPR v1.1 (March 24, 2016):
> > > > 
> > > > 13.7 Logical Resource Dynamic Reconfiguration (LRDR)
> > > > 
> > > > The Logical Resource Dynamic Reconfiguration option allows a platform to make available and recover platform re-
> > > > sources such as CPUs, Memory Regions, Processor Host Bridges, and I/O slots to/from its operating OS image(s).
> > > > 
> > > > ...
> > > > 
> > > > The device tree contains logical resource DR connectors for the maximum number of resources that the platform can
> > > > allocate to the specific OS. In some cases such as for processors and PHBs...
> > > > 
> > > > and
> > > > 
> > > > Table 240. Currently Defined DR Connector Types
> > > > 
> > > > | PHB | Logical PCI Host Bridge |    
> > > 
> > > Ah, my mistake.
> > >   
> > > >     
> > > > > > +    .instance_size = sizeof(sPAPRDRConnector),
> > > > > > +    .class_init    = spapr_drc_phb_class_init,
> > > > > > +};
> > > > > > +
> > > > > >  /* helper functions for external users */
> > > > > >  
> > > > > >  sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
> > > > > > @@ -1179,6 +1195,7 @@ static void spapr_drc_register_types(void)
> > > > > >      type_register_static(&spapr_drc_cpu_info);
> > > > > >      type_register_static(&spapr_drc_pci_info);
> > > > > >      type_register_static(&spapr_drc_lmb_info);
> > > > > > +    type_register_static(&spapr_drc_phb_info);
> > > > > >  
> > > > > >      spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
> > > > > >                          rtas_set_indicator);
> > > > > > diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
> > > > > > index a7958d0a8d14..535fc61b98a8 100644
> > > > > > --- a/include/hw/ppc/spapr_drc.h
> > > > > > +++ b/include/hw/ppc/spapr_drc.h
> > > > > > @@ -69,6 +69,14 @@
> > > > > >  #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > > > >                                          TYPE_SPAPR_DRC_LMB)
> > > > > >  
> > > > > > +#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
> > > > > > +#define SPAPR_DRC_PHB_GET_CLASS(obj) \
> > > > > > +        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
> > > > > > +#define SPAPR_DRC_PHB_CLASS(klass) \
> > > > > > +        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
> > > > > > +#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
> > > > > > +                                        TYPE_SPAPR_DRC_PHB)
> > > > > > +
> > > > > >  /*
> > > > > >   * Various hotplug types managed by sPAPRDRConnector
> > > > > >   *
> > > > > >       
> > > > >     
> > > >     
> > > 
> > > 
> > >   
> >   
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2793 bytes --]

On Wed, Sep 13, 2017 at 02:56:51PM +0200, Greg Kurz wrote:
> On Wed, 13 Sep 2017 22:23:29 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> [...snip...]
> > > > > Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> > > > > Maybe this could be the default value for the machine property instead of
> > > > > 256 then ?    
> > > > 
> > > > Actually, if we're binding it back to index, which has a hard limit,
> > > > then it no longer makes sense to have it as a property and we should
> > > > go back to a constant (well, it could vary by machine type version).  
> > 
> > Sorry I've taken so long to reply.
> > 
> 
> Oh, don't mention it. :)
> 
> > > But I guess that the hard limit of 31 as described in the changelog of
> > > commit 357d1e3bc7d2d80e5271bc4f3ac8537e30dc8046 still holds, doesn't
> > > it ?  
> > 
> > That's right.  Note that that is a limit of *31* PHBs (numbered
> > 0..30), not 32 PHBs numbered 0..31.
> > 
> 
> Yeah I saw that.
> 
> > >     Because some guest versions (including most current distro kernels) can't
> > >     access PCI MMIO above 64 TiB, we put all the PCI windows between 32 TiB and
> > >     64 TiB.  This is broken into 1 TiB chunks.  The first 1 TiB contains the
> > >     PIO (64 kiB) and 32-bit MMIO (2 GiB) windows for all of the PHBs.  Each
> > >     subsequent TiB chunk contains a naturally aligned 64-bit MMIO window for
> > >     one PHB each.
> > >     
> > >     This reduces the number of allowed PHBs (without full manual configuration
> > >     of all the windows) from 256 to 31, but this should still be plenty in
> > >     practice.
> > > 
> > > Not sure why a machine type version would have a different limit. Can
> > > you think of a use case ?  
> > 
> > Well, the older machine types had a different layout.  It allowed for
> > more indexes, but had smaller windows, which meant certain cards (e.g.
> > GPGPUs with huge BARs) didn't work properly.  It also had some weird
> > alignments that meant we were a bit wasteful of address space.
> > 
> > But we can't change the location of PHB windows on migration, so we
> > had to maintain that old layout for old machine types.  That's why
> > there's a different limit depending on machine type version.
> > 
> 
> Ok, so we *just* have 2 different maximum number of PHBs:
> - 256 for pseries <= 2.7
> - 31 for newer machine types

Yes, at least for now.  If we ever discover that 31 PHBs *isn't*
enough, we may have to come up with yet another layout, which could
have a different limit.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]