[Qemu-devel] [PATCH v3 00/19] spapr: Add support for PHB hotplug

[Qemu-devel] [PATCH v3 00/19] spapr: Add support for PHB hotplug

[Greg Kurz]

This allows to hotplug/unplug PHBs. I could successfully test:
- using in-kernel XICS, emulated XICS and XIVE
- using experimental in-kernel XIVE from Cedric
- hotplug/unplug with e1000 device to validate LSIs
- hotplug/unplug with virtio-net device to validate MSIs
- some simple migration scenarios

Change in v3:
- reworked phandle related code some more
- disintricate allocation/"type setting" of interrupts
- identify LSIs at machine init

Changes in v2:
- rebased on current ppc-for-4.0
- added some preliminary cleanup
- call unrealize from realize error path
- advertise PHB hotplug in last patch
- reworked phandle related code
- sync LSIs to KVM

Please comment.

--
Greg

---

Greg Kurz (11):
      spapr: Rename xics to intc in interrupt controller agnostic code
      xics: Disintricate allocation and type setting of interrupts
      spapr/xive: Don't set irq type in spapr_xive_irq_claim()
      spapr: Set irq type in a dedicated function
      spapr: Identify LSIs of all possible PHBs at machine init
      spapr_pci: add PHB unrealize
      spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend
      spapr_xive: Cache device tree nodename in sPAPRXive
      spapr: Expose the name of the interrupt controller node
      spapr_irq: Expose the phandle of the interrupt controller
      spapr: add hotplug hooks for PHB hotplug

Michael Roth (6):
      spapr: create DR connectors for PHBs
      spapr_events: add support for phb hotplug events
      qdev: pass an Object * to qbus_set_hotplug_handler()
      spapr_pci: provide node start offset via spapr_populate_pci_dt()
      spapr_pci: add ibm, my-drc-index property for PHB hotplug
      spapr: enable PHB hotplug for default pseries machine type

Nathan Fontenot (1):
      spapr: populate PHB DRC entries for root DT node

Thomas Huth (1):
      ppc: Move spapr-related prototypes from xics.h into a seperate header file


 hw/acpi/pcihp.c               |    2 -
 hw/acpi/piix4.c               |    2 -
 hw/char/virtio-serial-bus.c   |    2 -
 hw/core/bus.c                 |   11 +--
 hw/core/machine.c             |    4 +
 hw/intc/spapr_xive.c          |   35 ++++-----
 hw/intc/xics.c                |   28 ++++++-
 hw/intc/xics_kvm.c            |    1 
 hw/intc/xics_spapr.c          |   10 +--
 hw/pci/pcie.c                 |    2 -
 hw/pci/shpc.c                 |    2 -
 hw/ppc/pnv_psi.c              |    1 
 hw/ppc/spapr.c                |  163 +++++++++++++++++++++++++++++++++++++++--
 hw/ppc/spapr_drc.c            |   18 +++++
 hw/ppc/spapr_events.c         |    9 ++
 hw/ppc/spapr_irq.c            |  129 +++++++++++++++++++++++++++-----
 hw/ppc/spapr_pci.c            |  124 ++++++++++++++++++++++++-------
 hw/ppc/spapr_vio.c            |    4 +
 hw/s390x/css-bridge.c         |    2 -
 hw/s390x/s390-pci-bus.c       |    6 +-
 hw/scsi/virtio-scsi.c         |    2 -
 hw/scsi/vmw_pvscsi.c          |    2 -
 hw/usb/dev-smartcard-reader.c |    2 -
 include/hw/pci-host/spapr.h   |   11 ++-
 include/hw/ppc/spapr.h        |    4 +
 include/hw/ppc/spapr_drc.h    |    8 ++
 include/hw/ppc/spapr_irq.h    |   16 +++-
 include/hw/ppc/spapr_xive.h   |   13 +++
 include/hw/ppc/xics.h         |   15 ++--
 include/hw/ppc/xics_spapr.h   |   43 +++++++++++
 include/hw/qdev-core.h        |    3 -
 31 files changed, 545 insertions(+), 129 deletions(-)
 create mode 100644 include/hw/ppc/xics_spapr.h

[Qemu-devel] [PATCH v3 01/19] ppc: Move spapr-related prototypes from xics.h into a seperate header file

[Greg Kurz]

From: Thomas Huth <thuth@redhat.com>

When compiling with Clang in -std=gnu99 mode, there is a warning/error:

  CC      ppc64-softmmu/hw/intc/xics_spapr.o
In file included from /home/thuth/devel/qemu/hw/intc/xics_spapr.c:34:
/home/thuth/devel/qemu/include/hw/ppc/xics.h:203:34: error: redefinition of typedef 'sPAPRMachineState' is a C11 feature
      [-Werror,-Wtypedef-redefinition]
typedef struct sPAPRMachineState sPAPRMachineState;
                                 ^
/home/thuth/devel/qemu/include/hw/ppc/spapr_irq.h:25:34: note: previous definition is here
typedef struct sPAPRMachineState sPAPRMachineState;
                                 ^

We have to remove the duplicated typedef here and include "spapr.h" instead.
But "spapr.h" should not be included for the pnv machine files. So move
the spapr-related prototypes into a new file called "xics_spapr.h" instead.

Reviewed-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---

This patch should be merged through Thomas's tree soon. It is needed
because other patches in this series depend on it.
---
 hw/intc/xics_kvm.c          |    1 +
 hw/intc/xics_spapr.c        |    1 +
 hw/ppc/spapr_irq.c          |    1 +
 include/hw/ppc/xics.h       |    7 -------
 include/hw/ppc/xics_spapr.h |   37 +++++++++++++++++++++++++++++++++++++
 5 files changed, 40 insertions(+), 7 deletions(-)
 create mode 100644 include/hw/ppc/xics_spapr.h

diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c
index ac94594b1919..dff13300504c 100644
--- a/hw/intc/xics_kvm.c
+++ b/hw/intc/xics_kvm.c
@@ -34,6 +34,7 @@
 #include "sysemu/kvm.h"
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/xics.h"
+#include "hw/ppc/xics_spapr.h"
 #include "kvm_ppc.h"
 #include "qemu/config-file.h"
 #include "qemu/error-report.h"
diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
index 9c1a90d7094b..de6cc15b6474 100644
--- a/hw/intc/xics_spapr.c
+++ b/hw/intc/xics_spapr.c
@@ -32,6 +32,7 @@
 #include "qemu/timer.h"
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/xics.h"
+#include "hw/ppc/xics_spapr.h"
 #include "hw/ppc/fdt.h"
 #include "qapi/visitor.h"
 
diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index 5fce72fe0f6c..1da7a32348fc 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -14,6 +14,7 @@
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/spapr_xive.h"
 #include "hw/ppc/xics.h"
+#include "hw/ppc/xics_spapr.h"
 #include "sysemu/kvm.h"
 
 #include "trace.h"
diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
index 07508cbd217e..fad786e8b22d 100644
--- a/include/hw/ppc/xics.h
+++ b/include/hw/ppc/xics.h
@@ -200,13 +200,6 @@ void ics_pic_print_info(ICSState *ics, Monitor *mon);
 void ics_resend(ICSState *ics);
 void icp_resend(ICPState *ss);
 
-typedef struct sPAPRMachineState sPAPRMachineState;
-
-void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
-                   uint32_t phandle);
-int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
-void xics_spapr_init(sPAPRMachineState *spapr);
-
 Object *icp_create(Object *cpu, const char *type, XICSFabric *xi,
                    Error **errp);
 
diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
new file mode 100644
index 000000000000..b1ab27d022cf
--- /dev/null
+++ b/include/hw/ppc/xics_spapr.h
@@ -0,0 +1,37 @@
+/*
+ * QEMU PowerPC pSeries Logical Partition (aka sPAPR) hardware System Emulator
+ *
+ * PAPR Virtualized Interrupt System, aka ICS/ICP aka xics
+ *
+ * Copyright (c) 2010, 2011 David Gibson, IBM Corporation.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef XICS_SPAPR_H
+#define XICS_SPAPR_H
+
+#include "hw/ppc/spapr.h"
+
+void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
+                   uint32_t phandle);
+int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
+void xics_spapr_init(sPAPRMachineState *spapr);
+
+#endif /* XICS_SPAPR_H */

[Qemu-devel] [PATCH v3 02/19] spapr: Rename xics to intc in interrupt controller agnostic code

[Greg Kurz]

All this code is used with both the XICS and XIVE interrupt controllers.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
---
v3: - s/spapr_dt_intc_irq/spapr_dt_irq
---
 hw/ppc/spapr.c              |    6 +++---
 hw/ppc/spapr_events.c       |    2 +-
 hw/ppc/spapr_pci.c          |    6 +++---
 hw/ppc/spapr_vio.c          |    2 +-
 include/hw/pci-host/spapr.h |    2 +-
 include/hw/ppc/spapr.h      |    2 +-
 6 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 83081defde4e..26f8e55cc25e 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -96,7 +96,7 @@
 
 #define MIN_RMA_SLOF            128UL
 
-#define PHANDLE_XICP            0x00001111
+#define PHANDLE_INTC            0x00001111
 
 /* These two functions implement the VCPU id numbering: one to compute them
  * all and one to identify thread 0 of a VCORE. Any change to the first one
@@ -1276,7 +1276,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
 
     /* /interrupt controller */
     spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
-                          PHANDLE_XICP);
+                          PHANDLE_INTC);
 
     ret = spapr_populate_memory(spapr, fdt);
     if (ret < 0) {
@@ -1296,7 +1296,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
     }
 
     QLIST_FOREACH(phb, &spapr->phbs, list) {
-        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt,
+        ret = spapr_populate_pci_dt(phb, PHANDLE_INTC, fdt,
                                     spapr->irq->nr_msis);
         if (ret < 0) {
             error_report("couldn't setup PCI devices in fdt");
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 32719a1b72d0..b9c7ecb9e987 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -282,7 +282,7 @@ void spapr_dt_events(sPAPRMachineState *spapr, void *fdt)
             continue;
         }
 
-        spapr_dt_xics_irq(interrupts, source->irq, false);
+        spapr_dt_irq(interrupts, source->irq, false);
 
         _FDT(node_offset = fdt_add_subnode(fdt, event_sources, source_name));
         _FDT(fdt_setprop(fdt, node_offset, "interrupts", interrupts,
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index b74f2632ecc6..ccdaf2c9a606 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -2066,7 +2066,7 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
 
 }
 
-int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
+int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
                           uint32_t nr_msis)
 {
     int bus_off, i, j, ret;
@@ -2164,8 +2164,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
             irqmap[1] = 0;
             irqmap[2] = 0;
             irqmap[3] = cpu_to_be32(j+1);
-            irqmap[4] = cpu_to_be32(xics_phandle);
-            spapr_dt_xics_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
+            irqmap[4] = cpu_to_be32(intc_phandle);
+            spapr_dt_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
         }
     }
     /* Write interrupt map */
diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
index 7e8a9ad09337..f80b70a39c46 100644
--- a/hw/ppc/spapr_vio.c
+++ b/hw/ppc/spapr_vio.c
@@ -158,7 +158,7 @@ static int vio_make_devnode(VIOsPAPRDevice *dev,
     if (dev->irq) {
         uint32_t ints_prop[2];
 
-        spapr_dt_xics_irq(ints_prop, dev->irq, false);
+        spapr_dt_irq(ints_prop, dev->irq, false);
         ret = fdt_setprop(fdt, node_off, "interrupts", ints_prop,
                           sizeof(ints_prop));
         if (ret < 0) {
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 4eb3a2ce3eb8..e0e683c32469 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -113,7 +113,7 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
     return spapr_qirq(spapr, phb->lsi_table[pin].irq);
 }
 
-int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
+int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
                           uint32_t nr_msis);
 
 void spapr_pci_rtas_init(void);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 9e01a5a12e4a..f988dc6924b8 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -682,7 +682,7 @@ void spapr_load_rtas(sPAPRMachineState *spapr, void *fdt, hwaddr addr);
  * "interrupt-controller" node has its "#interrupt-cells" property set to 2 (ie,
  * VIO devices, RTAS event sources and PHBs).
  */
-static inline void spapr_dt_xics_irq(uint32_t *intspec, int irq, bool is_lsi)
+static inline void spapr_dt_irq(uint32_t *intspec, int irq, bool is_lsi)
 {
     intspec[0] = cpu_to_be32(irq);
     intspec[1] = is_lsi ? cpu_to_be32(1) : 0;

[Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[Greg Kurz]

The current code assumes that an interrupt is allocated as soon as its
type is set to MSI or LSI. PHB hotplug will require to be able to set
the type of an interrupt before actually allocating it.

Disintricate type setting from allocation by using another flag bit for
the latter. Introduce a new ics_claim_irq() function for allocation. The
behavior of aborting if the same irq gets allocated twice is kept.

ics_set_irq_type() now only sets the type to MSI or LSI. It doesn't bring
anything to abort if the type was already set before. Drop the assert
and XICS_FLAGS_IRQ_MASK on the way.

Older QEMUs don't know about XICS_FLAGS_IRQ_CLAIMED. In order to safely
handle incoming migration, we must fix the irq flags. This is done at
post load thanks to a compat property. We don't need to do anything for
backward migration since older machines only care for the irq type.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/core/machine.c     |    4 ++++
 hw/intc/xics.c        |   28 +++++++++++++++++++++++++---
 hw/ppc/pnv_psi.c      |    1 +
 hw/ppc/spapr_irq.c    |    4 +---
 include/hw/ppc/xics.h |    8 ++++++--
 5 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 1a0a9ab1117a..536a34092367 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -25,6 +25,10 @@
 
 GlobalProperty hw_compat_3_1[] = {
     {
+        .driver   = "ics-base",
+        .property = "has-claimed-flag",
+        .value    = "off",
+    },{
         .driver   = "pcie-root-port",
         .property = "x-speed",
         .value    = "2_5",
diff --git a/hw/intc/xics.c b/hw/intc/xics.c
index 16e8ffa2aaf7..82cf04548907 100644
--- a/hw/intc/xics.c
+++ b/hw/intc/xics.c
@@ -75,7 +75,7 @@ void ics_pic_print_info(ICSState *ics, Monitor *mon)
     for (i = 0; i < ics->nr_irqs; i++) {
         ICSIRQState *irq = ics->irqs + i;
 
-        if (!(irq->flags & XICS_FLAGS_IRQ_MASK)) {
+        if (!(irq->flags & XICS_FLAGS_IRQ_CLAIMED)) {
             continue;
         }
         monitor_printf(mon, "  %4x %s %02x %02x\n",
@@ -662,6 +662,22 @@ static int ics_base_dispatch_post_load(void *opaque, int version_id)
     ICSState *ics = opaque;
     ICSStateClass *info = ICS_BASE_GET_CLASS(ics);
 
+    if (!ics->has_claimed_flag) {
+        int i;
+
+        for (i = 0; i < ics->nr_irqs; i++) {
+            ICSIRQState *irq = ics->irqs + i;
+
+            /*
+             * For older machines, allocating the irq and setting its type is
+             * the same thing.
+             */
+            if (irq->flags & (XICS_FLAGS_IRQ_LSI| XICS_FLAGS_IRQ_MSI)) {
+                irq->flags |= XICS_FLAGS_IRQ_CLAIMED;
+            }
+        }
+    }
+
     if (info->post_load) {
         return info->post_load(ics, version_id);
     }
@@ -702,6 +718,7 @@ static const VMStateDescription vmstate_ics_base = {
 
 static Property ics_base_properties[] = {
     DEFINE_PROP_UINT32("nr-irqs", ICSState, nr_irqs, 0),
+    DEFINE_PROP_BOOL("has-claimed-flag", ICSState, has_claimed_flag, true),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -741,10 +758,15 @@ ICPState *xics_icp_get(XICSFabric *xi, int server)
     return xic->icp_get(xi, server);
 }
 
-void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
+void ics_claim_irq(ICSState *ics, int srcno)
 {
-    assert(!(ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MASK));
+    assert(ICS_IRQ_FREE(ics, srcno));
+
+    ics->irqs[srcno].flags |= XICS_FLAGS_IRQ_CLAIMED;
+}
 
+void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
+{
     ics->irqs[srcno].flags |=
         lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI;
 }
diff --git a/hw/ppc/pnv_psi.c b/hw/ppc/pnv_psi.c
index 8ced09506321..ced34e1119dc 100644
--- a/hw/ppc/pnv_psi.c
+++ b/hw/ppc/pnv_psi.c
@@ -488,6 +488,7 @@ static void pnv_psi_realize(DeviceState *dev, Error **errp)
 
     for (i = 0; i < ics->nr_irqs; i++) {
         ics_set_irq_type(ics, i, true);
+        ics_claim_irq(ics, i);
     }
 
     psi->qirqs = qemu_allocate_irqs(ics_simple_set_irq, ics, ics->nr_irqs);
diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index 1da7a32348fc..86c712d15382 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -125,9 +125,6 @@ error:
     error_propagate(errp, local_err);
 }
 
-#define ICS_IRQ_FREE(ics, srcno)   \
-    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_MASK)))
-
 static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
                                 Error **errp)
 {
@@ -146,6 +143,7 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
     }
 
     ics_set_irq_type(ics, irq - ics->offset, lsi);
+    ics_claim_irq(ics, irq - ics->offset);
     return 0;
 }
 
diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
index fad786e8b22d..e3be5cc663c5 100644
--- a/include/hw/ppc/xics.h
+++ b/include/hw/ppc/xics.h
@@ -131,6 +131,7 @@ struct ICSState {
     /*< public >*/
     uint32_t nr_irqs;
     uint32_t offset;
+    bool has_claimed_flag;
     ICSIRQState *irqs;
     XICSFabric *xics;
 };
@@ -153,13 +154,15 @@ struct ICSIRQState {
 #define XICS_STATUS_PRESENTED          0x10
 #define XICS_STATUS_QUEUED             0x20
     uint8_t status;
-/* (flags & XICS_FLAGS_IRQ_MASK) == 0 means the interrupt is not allocated */
 #define XICS_FLAGS_IRQ_LSI             0x1
 #define XICS_FLAGS_IRQ_MSI             0x2
-#define XICS_FLAGS_IRQ_MASK            0x3
+#define XICS_FLAGS_IRQ_CLAIMED         0x4
     uint8_t flags;
 };
 
+#define ICS_IRQ_FREE(ics, srcno)   \
+    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_CLAIMED)))
+
 struct XICSFabric {
     Object parent;
 };
@@ -193,6 +196,7 @@ void ics_simple_write_xive(ICSState *ics, int nr, int server,
 void ics_simple_set_irq(void *opaque, int srcno, int val);
 void ics_kvm_set_irq(void *opaque, int srcno, int val);
 
+void ics_claim_irq(ICSState *ics, int srcno);
 void ics_set_irq_type(ICSState *ics, int srcno, bool lsi);
 void icp_pic_print_info(ICPState *icp, Monitor *mon);
 void ics_pic_print_info(ICSState *ics, Monitor *mon);

[Qemu-devel] [PATCH v3 04/19] spapr/xive: Don't set irq type in spapr_xive_irq_claim()

[Greg Kurz]

PHB hotplug will need to set the type of all LSIs at machine init.
Prepare for that by calling xive_source_irq_set() in the callers
of spapr_xive_irq_claim().

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/intc/spapr_xive.c        |    5 +----
 hw/ppc/spapr_irq.c          |    6 ++++--
 include/hw/ppc/spapr_xive.h |    2 +-
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
index d391177ab81f..aab63cfd1178 100644
--- a/hw/intc/spapr_xive.c
+++ b/hw/intc/spapr_xive.c
@@ -471,16 +471,13 @@ static void spapr_xive_register_types(void)
 
 type_init(spapr_xive_register_types)
 
-bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi)
+bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn)
 {
-    XiveSource *xsrc = &xive->source;
-
     if (lisn >= xive->nr_irqs) {
         return false;
     }
 
     xive->eat[lisn].w |= cpu_to_be64(EAS_VALID);
-    xive_source_irq_set(xsrc, lisn, lsi);
     return true;
 }
 
diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index 86c712d15382..bcd816c5a5fb 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -285,7 +285,8 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
 
     /* Enable the CPU IPIs */
     for (i = 0; i < nr_servers; ++i) {
-        spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i, false);
+        spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i);
+        xive_source_irq_set(&spapr->xive->source, SPAPR_IRQ_IPI + i, false);
     }
 
     spapr_xive_hcall_init(spapr);
@@ -294,10 +295,11 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
 static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool lsi,
                                 Error **errp)
 {
-    if (!spapr_xive_irq_claim(spapr->xive, irq, lsi)) {
+    if (!spapr_xive_irq_claim(spapr->xive, irq)) {
         error_setg(errp, "IRQ %d is invalid", irq);
         return -1;
     }
+    xive_source_irq_set(&spapr->xive->source, irq, lsi);
     return 0;
 }
 
diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
index 7fdc25057420..f8854a4a6a18 100644
--- a/include/hw/ppc/spapr_xive.h
+++ b/include/hw/ppc/spapr_xive.h
@@ -37,7 +37,7 @@ typedef struct sPAPRXive {
     MemoryRegion  tm_mmio;
 } sPAPRXive;
 
-bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi);
+bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn);
 bool spapr_xive_irq_free(sPAPRXive *xive, uint32_t lisn);
 void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon);
 

[Qemu-devel] [PATCH v3 05/19] spapr: Set irq type in a dedicated function

[Greg Kurz]

PHB hotplug will need to set the type of all LSIs at machine init.

Prepare for that by moving the behaviour of setting the type from
spapr_irq_claim() to a separate spapr_irq_set_type() function. Have
all the callers of spapr_irq_claim() to also call the new function.

There's no such need of separating allocation and type setting for
MSIs, so this introduces a convenient spapr_irq_claim_msi() function
for MSI users.

Wire all this to the existing backends.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_events.c      |    4 +--
 hw/ppc/spapr_irq.c         |   59 ++++++++++++++++++++++++++++++++++++--------
 hw/ppc/spapr_pci.c         |    5 ++--
 hw/ppc/spapr_vio.c         |    2 +
 include/hw/ppc/spapr_irq.h |    7 ++++-
 5 files changed, 59 insertions(+), 18 deletions(-)

diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index b9c7ecb9e987..777a70842acc 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -713,7 +713,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
         epow_irq = spapr_irq_findone(spapr, &error_fatal);
     }
 
-    spapr_irq_claim(spapr, epow_irq, false, &error_fatal);
+    spapr_irq_claim(spapr, epow_irq, &error_fatal);
 
     QTAILQ_INIT(&spapr->pending_events);
 
@@ -737,7 +737,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
             hp_irq = spapr_irq_findone(spapr, &error_fatal);
         }
 
-        spapr_irq_claim(spapr, hp_irq, false, &error_fatal);
+        spapr_irq_claim_msi(spapr, hp_irq, &error_fatal);
 
         spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_HOT_PLUG,
                                      hp_irq);
diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index bcd816c5a5fb..396dd09bdbe0 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -125,8 +125,7 @@ error:
     error_propagate(errp, local_err);
 }
 
-static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
-                                Error **errp)
+static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, Error **errp)
 {
     ICSState *ics = spapr->ics;
 
@@ -142,11 +141,20 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
         return -1;
     }
 
-    ics_set_irq_type(ics, irq - ics->offset, lsi);
     ics_claim_irq(ics, irq - ics->offset);
     return 0;
 }
 
+static void spapr_irq_set_type_xics(sPAPRMachineState *spapr, int irq, bool lsi)
+{
+    ICSState *ics = spapr->ics;
+
+    assert(ics);
+    assert(ics_valid_irq(ics, irq));
+
+    ics_set_irq_type(ics, irq - ics->offset, lsi);
+}
+
 static void spapr_irq_free_xics(sPAPRMachineState *spapr, int irq, int num)
 {
     ICSState *ics = spapr->ics;
@@ -245,6 +253,7 @@ sPAPRIrq spapr_irq_xics = {
 
     .init        = spapr_irq_init_xics,
     .claim       = spapr_irq_claim_xics,
+    .set_type    = spapr_irq_set_type_xics,
     .free        = spapr_irq_free_xics,
     .qirq        = spapr_qirq_xics,
     .print_info  = spapr_irq_print_info_xics,
@@ -292,17 +301,20 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
     spapr_xive_hcall_init(spapr);
 }
 
-static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool lsi,
-                                Error **errp)
+static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, Error **errp)
 {
     if (!spapr_xive_irq_claim(spapr->xive, irq)) {
         error_setg(errp, "IRQ %d is invalid", irq);
         return -1;
     }
-    xive_source_irq_set(&spapr->xive->source, irq, lsi);
     return 0;
 }
 
+static void spapr_irq_set_type_xive(sPAPRMachineState *spapr, int irq, bool lsi)
+{
+    xive_source_irq_set(&spapr->xive->source, irq, lsi);
+}
+
 static void spapr_irq_free_xive(sPAPRMachineState *spapr, int irq, int num)
 {
     int i;
@@ -403,6 +415,7 @@ sPAPRIrq spapr_irq_xive = {
 
     .init        = spapr_irq_init_xive,
     .claim       = spapr_irq_claim_xive,
+    .set_type    = spapr_irq_set_type_xive,
     .free        = spapr_irq_free_xive,
     .qirq        = spapr_qirq_xive,
     .print_info  = spapr_irq_print_info_xive,
@@ -466,19 +479,19 @@ static void spapr_irq_init_dual(sPAPRMachineState *spapr, Error **errp)
     }
 }
 
-static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
+static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq,
                                 Error **errp)
 {
     Error *local_err = NULL;
     int ret;
 
-    ret = spapr_irq_xics.claim(spapr, irq, lsi, &local_err);
+    ret = spapr_irq_xics.claim(spapr, irq, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         return ret;
     }
 
-    ret = spapr_irq_xive.claim(spapr, irq, lsi, &local_err);
+    ret = spapr_irq_xive.claim(spapr, irq, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         return ret;
@@ -487,6 +500,12 @@ static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
     return ret;
 }
 
+static void spapr_irq_set_type_dual(sPAPRMachineState *spapr, int irq, bool lsi)
+{
+    spapr_irq_xics.set_type(spapr, irq, lsi);
+    spapr_irq_xive.set_type(spapr, irq, lsi);
+}
+
 static void spapr_irq_free_dual(sPAPRMachineState *spapr, int irq, int num)
 {
     spapr_irq_xics.free(spapr, irq, num);
@@ -582,6 +601,7 @@ sPAPRIrq spapr_irq_dual = {
 
     .init        = spapr_irq_init_dual,
     .claim       = spapr_irq_claim_dual,
+    .set_type    = spapr_irq_set_type_dual,
     .free        = spapr_irq_free_dual,
     .qirq        = spapr_qirq_dual,
     .print_info  = spapr_irq_print_info_dual,
@@ -608,9 +628,25 @@ void spapr_irq_init(sPAPRMachineState *spapr, Error **errp)
                                       spapr->irq->nr_irqs);
 }
 
-int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp)
+int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp)
+{
+    return spapr->irq->claim(spapr, irq, errp);
+}
+
+void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi)
 {
-    return spapr->irq->claim(spapr, irq, lsi, errp);
+    return spapr->irq->set_type(spapr, irq, lsi);
+}
+
+int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp)
+{
+    int ret = spapr_irq_claim(spapr, irq, errp);
+
+    if (!ret) {
+        spapr_irq_set_type(spapr, irq, false);
+    }
+
+    return ret;
 }
 
 void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num)
@@ -699,6 +735,7 @@ sPAPRIrq spapr_irq_xics_legacy = {
 
     .init        = spapr_irq_init_xics,
     .claim       = spapr_irq_claim_xics,
+    .set_type    = spapr_irq_set_type_xics,
     .free        = spapr_irq_free_xics,
     .qirq        = spapr_qirq_xics,
     .print_info  = spapr_irq_print_info_xics,
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index ccdaf2c9a606..ec9d4d28004c 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -391,7 +391,7 @@ static void rtas_ibm_change_msi(PowerPCCPU *cpu, sPAPRMachineState *spapr,
     }
 
     for (i = 0; i < req_num; i++) {
-        spapr_irq_claim(spapr, irq + i, false, &err);
+        spapr_irq_claim_msi(spapr, irq + i, &err);
         if (err) {
             error_reportf_err(err, "Can't allocate MSIs for device %x: ",
                               config_addr);
@@ -1738,11 +1738,12 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
             }
         }
 
-        spapr_irq_claim(spapr, irq, true, &local_err);
+        spapr_irq_claim(spapr, irq, &local_err);
         if (local_err) {
             error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
             return;
         }
+        spapr_irq_set_type(spapr, irq, true);
 
         sphb->lsi_table[i].irq = irq;
     }
diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
index f80b70a39c46..9ae4351d4414 100644
--- a/hw/ppc/spapr_vio.c
+++ b/hw/ppc/spapr_vio.c
@@ -546,7 +546,7 @@ static void spapr_vio_busdev_realize(DeviceState *qdev, Error **errp)
         }
     }
 
-    spapr_irq_claim(spapr, dev->irq, false, &local_err);
+    spapr_irq_claim_msi(spapr, dev->irq, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         return;
diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
index 14b02c3aca33..42b325eb51ba 100644
--- a/include/hw/ppc/spapr_irq.h
+++ b/include/hw/ppc/spapr_irq.h
@@ -36,7 +36,8 @@ typedef struct sPAPRIrq {
     uint8_t     ov5;
 
     void (*init)(sPAPRMachineState *spapr, Error **errp);
-    int (*claim)(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
+    int (*claim)(sPAPRMachineState *spapr, int irq, Error **errp);
+    void (*set_type)(sPAPRMachineState *spapr, int irq, bool lsi);
     void (*free)(sPAPRMachineState *spapr, int irq, int num);
     qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
     void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
@@ -55,7 +56,9 @@ extern sPAPRIrq spapr_irq_xive;
 extern sPAPRIrq spapr_irq_dual;
 
 void spapr_irq_init(sPAPRMachineState *spapr, Error **errp);
-int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
+int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp);
+void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi);
+int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp);
 void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
 qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
 int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);

[Qemu-devel] [PATCH v3 06/19] spapr: Identify LSIs of all possible PHBs at machine init

[Greg Kurz]

Every PHB needs to claim 4 LSIs to support legacy PCI devices. This is
currently done at PHB realize. When using in-kernel XICS (or upcoming
in-kernel XIVE), QEMU synchronizes the state of all irqs, including
these LSIs, later on at machine reset.

In order to support PHB hotplug, we need a way to tell KVM about the
LSIs that doesn't require a machine reset. Since these irq numbers are
fixed values derived from the PHB index, let's identify them all at
machine init. Older machines that don't have fixed irq numbers cannot
support PHB hotplug and keep the existing behavior.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c              |    6 ++++++
 hw/ppc/spapr_pci.c          |   20 ++++++++++++++++++--
 include/hw/pci-host/spapr.h |    2 ++
 3 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 26f8e55cc25e..9189b4d3a9d6 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2802,6 +2802,12 @@ static void spapr_machine_init(MachineState *machine)
 
     phb = spapr_create_default_phb();
 
+    if (!smc->legacy_irq_allocation) {
+        for (i = 0; i < SPAPR_MAX_PHBS; i++) {
+            spapr_phb_set_lsis(i, spapr);
+        }
+    }
+
     for (i = 0; i < nb_nics; i++) {
         NICInfo *nd = &nd_table[i];
 
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index ec9d4d28004c..f5f13a4d4816 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1559,6 +1559,22 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
     }
 }
 
+static uint32_t spapr_phb_index_to_lsi(int phb_index, int irq_index)
+{
+    return SPAPR_IRQ_PCI_LSI + phb_index * PCI_NUM_PINS + irq_index;
+}
+
+void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr)
+{
+    int i;
+
+    for (i = 0; i < PCI_NUM_PINS; i++) {
+        uint32_t irq = spapr_phb_index_to_lsi(index, i);
+
+        spapr_irq_set_type(spapr, irq, true);
+    }
+}
+
 static void spapr_phb_realize(DeviceState *dev, Error **errp)
 {
     /* We don't use SPAPR_MACHINE() in order to exit gracefully if the user
@@ -1726,7 +1742,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
 
     /* Initialize the LSI table */
     for (i = 0; i < PCI_NUM_PINS; i++) {
-        uint32_t irq = SPAPR_IRQ_PCI_LSI + sphb->index * PCI_NUM_PINS + i;
+        uint32_t irq = spapr_phb_index_to_lsi(sphb->index, i);
         Error *local_err = NULL;
 
         if (smc->legacy_irq_allocation) {
@@ -1736,6 +1752,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
                                         "can't allocate LSIs: ");
                 return;
             }
+            spapr_irq_set_type(spapr, irq, true);
         }
 
         spapr_irq_claim(spapr, irq, &local_err);
@@ -1743,7 +1760,6 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
             error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
             return;
         }
-        spapr_irq_set_type(spapr, irq, true);
 
         sphb->lsi_table[i].irq = irq;
     }
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index e0e683c32469..bb0ae7fdd41d 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -164,4 +164,6 @@ static inline void spapr_phb_vfio_reset(DeviceState *qdev)
 
 void spapr_phb_dma_reset(sPAPRPHBState *sphb);
 
+void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr);
+
 #endif /* PCI_HOST_SPAPR_H */

[Qemu-devel] [PATCH v3 07/19] spapr_pci: add PHB unrealize

[Greg Kurz]

To support PHB hotplug we need to clean up lingering references,
memory, child properties, etc. prior to the PHB object being
finalized. Generally this will be called as a result of calling
object_unparent() on the PHB object, which in turn would normally
be called as the result of an unplug() operation.

When the PHB is finalized, child objects will be unparented in
turn, and finalized if the PHB was the only reference holder. so
we don't bother to explicitly unparent child objects of the PHB
(spapr_iommu, spapr_drc, etc).

The formula that gives the number of DMA windows is moved to an
inline function in the hw/pci-host/spapr.h header because it
will have other users.

The unrealize function is able to cope with partially realized PHBs.
It is hence used to implement proper rollback on the realize error
path. Note that older machines that used the legacy irq allocation
cannot support PHB hotplug, so we don't care to free their LSIs in
the unrealize function.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
v3: - don't free LSIs at unrealize
v2: - implement rollback with unrealize function
---
 hw/ppc/spapr_pci.c          |   63 ++++++++++++++++++++++++++++++++++++++++---
 include/hw/pci-host/spapr.h |    5 +++
 2 files changed, 64 insertions(+), 4 deletions(-)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index f5f13a4d4816..0f6a173d5323 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1559,6 +1559,56 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
     }
 }
 
+static void spapr_phb_finalizefn(Object *obj)
+{
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(obj);
+
+    g_free(sphb->dtbusname);
+    sphb->dtbusname = NULL;
+}
+
+static void spapr_phb_unrealize(DeviceState *dev, Error **errp)
+{
+    SysBusDevice *s = SYS_BUS_DEVICE(dev);
+    PCIHostState *phb = PCI_HOST_BRIDGE(s);
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(phb);
+    sPAPRTCETable *tcet;
+    int i;
+    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
+
+    if (sphb->msi) {
+        g_hash_table_unref(sphb->msi);
+        sphb->msi = NULL;
+    }
+
+    /*
+     * Remove IO/MMIO subregions and aliases, rest should get cleaned
+     * via PHB's unrealize->object_finalize
+     */
+    for (i = windows_supported - 1; i >= 0; i--) {
+        tcet = spapr_tce_find_by_liobn(sphb->dma_liobn[i]);
+        if (tcet) {
+            memory_region_del_subregion(&sphb->iommu_root,
+                                        spapr_tce_get_iommu(tcet));
+        }
+    }
+
+    QLIST_REMOVE(sphb, list);
+
+    memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow);
+
+    address_space_destroy(&sphb->iommu_as);
+
+    qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort);
+    pci_unregister_root_bus(phb->bus);
+
+    memory_region_del_subregion(get_system_memory(), &sphb->iowindow);
+    if (sphb->mem64_win_pciaddr != (hwaddr)-1) {
+        memory_region_del_subregion(get_system_memory(), &sphb->mem64window);
+    }
+    memory_region_del_subregion(get_system_memory(), &sphb->mem32window);
+}
+
 static uint32_t spapr_phb_index_to_lsi(int phb_index, int irq_index)
 {
     return SPAPR_IRQ_PCI_LSI + phb_index * PCI_NUM_PINS + irq_index;
@@ -1592,8 +1642,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
     PCIBus *bus;
     uint64_t msi_window_size = 4096;
     sPAPRTCETable *tcet;
-    const unsigned windows_supported =
-        sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1;
+    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
 
     if (!spapr) {
         error_setg(errp, TYPE_SPAPR_PCI_HOST_BRIDGE " needs a pseries machine");
@@ -1750,7 +1799,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
             if (local_err) {
                 error_propagate_prepend(errp, local_err,
                                         "can't allocate LSIs: ");
-                return;
+                goto unrealize;
             }
             spapr_irq_set_type(spapr, irq, true);
         }
@@ -1778,13 +1827,17 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
         if (!tcet) {
             error_setg(errp, "Creating window#%d failed for %s",
                        i, sphb->dtbusname);
-            return;
+            goto unrealize;
         }
         memory_region_add_subregion(&sphb->iommu_root, 0,
                                     spapr_tce_get_iommu(tcet));
     }
 
     sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);
+    return;
+
+unrealize:
+    spapr_phb_unrealize(dev, NULL);
 }
 
 static int spapr_phb_children_reset(Object *child, void *opaque)
@@ -1983,6 +2036,7 @@ static void spapr_phb_class_init(ObjectClass *klass, void *data)
 
     hc->root_bus_path = spapr_phb_root_bus_path;
     dc->realize = spapr_phb_realize;
+    dc->unrealize = spapr_phb_unrealize;
     dc->props = spapr_phb_properties;
     dc->reset = spapr_phb_reset;
     dc->vmsd = &vmstate_spapr_pci;
@@ -1998,6 +2052,7 @@ static const TypeInfo spapr_phb_info = {
     .name          = TYPE_SPAPR_PCI_HOST_BRIDGE,
     .parent        = TYPE_PCI_HOST_BRIDGE,
     .instance_size = sizeof(sPAPRPHBState),
+    .instance_finalize = spapr_phb_finalizefn,
     .class_init    = spapr_phb_class_init,
     .interfaces    = (InterfaceInfo[]) {
         { TYPE_HOTPLUG_HANDLER },
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index bb0ae7fdd41d..2ab75e28dbb5 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -166,4 +166,9 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb);
 
 void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr);
 
+static inline unsigned spapr_phb_windows_supported(sPAPRPHBState *sphb)
+{
+    return sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1;
+}
+
 #endif /* PCI_HOST_SPAPR_H */

[Qemu-devel] [PATCH v3 08/19] spapr: create DR connectors for PHBs

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c             |   13 +++++++++++++
 hw/ppc/spapr_drc.c         |   17 +++++++++++++++++
 include/hw/ppc/spapr.h     |    1 +
 include/hw/ppc/spapr_drc.h |    8 ++++++++
 4 files changed, 39 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 9189b4d3a9d6..5be8f76351fd 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2797,6 +2797,19 @@ static void spapr_machine_init(MachineState *machine)
     /* We always have at least the nvram device on VIO */
     spapr_create_nvram(spapr);
 
+    /*
+     * Setup hotplug / dynamic-reconfiguration connectors. top-level
+     * connectors (described in root DT node's "ibm,drc-types" property)
+     * are pre-initialized here. additional child connectors (such as
+     * connectors for a PHBs PCI slots) are added as needed during their
+     * parent's realization.
+     */
+    if (smc->dr_phb_enabled) {
+        for (i = 0; i < SPAPR_MAX_PHBS; i++) {
+            spapr_dr_connector_new(OBJECT(machine), TYPE_SPAPR_DRC_PHB, i);
+        }
+    }
+
     /* Set up PCI */
     spapr_pci_rtas_init();
 
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 2edb7d1e9c8c..189ee681062a 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -696,6 +696,15 @@ static void spapr_drc_lmb_class_init(ObjectClass *k, void *data)
     drck->release = spapr_lmb_release;
 }
 
+static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
+{
+    sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_CLASS(k);
+
+    drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
+    drck->typename = "PHB";
+    drck->drc_name_prefix = "PHB ";
+}
+
 static const TypeInfo spapr_dr_connector_info = {
     .name          = TYPE_SPAPR_DR_CONNECTOR,
     .parent        = TYPE_DEVICE,
@@ -739,6 +748,13 @@ static const TypeInfo spapr_drc_lmb_info = {
     .class_init    = spapr_drc_lmb_class_init,
 };
 
+static const TypeInfo spapr_drc_phb_info = {
+    .name          = TYPE_SPAPR_DRC_PHB,
+    .parent        = TYPE_SPAPR_DRC_LOGICAL,
+    .instance_size = sizeof(sPAPRDRConnector),
+    .class_init    = spapr_drc_phb_class_init,
+};
+
 /* helper functions for external users */
 
 sPAPRDRConnector *spapr_drc_by_index(uint32_t index)
@@ -1189,6 +1205,7 @@ static void spapr_drc_register_types(void)
     type_register_static(&spapr_drc_cpu_info);
     type_register_static(&spapr_drc_pci_info);
     type_register_static(&spapr_drc_lmb_info);
+    type_register_static(&spapr_drc_phb_info);
 
     spapr_rtas_register(RTAS_SET_INDICATOR, "set-indicator",
                         rtas_set_indicator);
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index f988dc6924b8..bc8ebb072258 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -103,6 +103,7 @@ struct sPAPRMachineClass {
 
     /*< public >*/
     bool dr_lmb_enabled;       /* enable dynamic-reconfig/hotplug of LMBs */
+    bool dr_phb_enabled;       /* enable dynamic-reconfig/hotplug of PHBs */
     bool update_dt_enabled;    /* enable KVMPPC_H_UPDATE_DT */
     bool use_ohci_by_default;  /* use USB-OHCI instead of XHCI */
     bool pre_2_10_has_unused_icps;
diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h
index f6ff32e7e2f2..56bba36ad4da 100644
--- a/include/hw/ppc/spapr_drc.h
+++ b/include/hw/ppc/spapr_drc.h
@@ -70,6 +70,14 @@
 #define SPAPR_DRC_LMB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
                                         TYPE_SPAPR_DRC_LMB)
 
+#define TYPE_SPAPR_DRC_PHB "spapr-drc-phb"
+#define SPAPR_DRC_PHB_GET_CLASS(obj) \
+        OBJECT_GET_CLASS(sPAPRDRConnectorClass, obj, TYPE_SPAPR_DRC_PHB)
+#define SPAPR_DRC_PHB_CLASS(klass) \
+        OBJECT_CLASS_CHECK(sPAPRDRConnectorClass, klass, TYPE_SPAPR_DRC_PHB)
+#define SPAPR_DRC_PHB(obj) OBJECT_CHECK(sPAPRDRConnector, (obj), \
+                                        TYPE_SPAPR_DRC_PHB)
+
 /*
  * Various hotplug types managed by sPAPRDRConnector
  *

[Qemu-devel] [PATCH v3 09/19] spapr: populate PHB DRC entries for root DT node

[Greg Kurz]

From: Nathan Fontenot <nfont@linux.vnet.ibm.com>

This add entries to the root OF node to advertise our PHBs as being
DR-capable in accordance with PAPR specification.

Signed-off-by: Nathan Fontenot <nfont@linux.vnet.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 5be8f76351fd..83ec79caf48e 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1350,6 +1350,14 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
         exit(1);
     }
 
+    if (smc->dr_phb_enabled) {
+        ret = spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNECTOR_TYPE_PHB);
+        if (ret < 0) {
+            error_report("Couldn't set up PHB DR device tree properties");
+            exit(1);
+        }
+    }
+
     return fdt;
 }
 

[Qemu-devel] [PATCH v3 10/19] spapr_events: add support for phb hotplug events

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Extend the existing EPOW event format we use for PCI
devices to emit PHB plug/unplug events.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_events.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 777a70842acc..f1d1802c66f7 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -526,6 +526,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
     case SPAPR_DR_CONNECTOR_TYPE_CPU:
         hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU;
         break;
+    case SPAPR_DR_CONNECTOR_TYPE_PHB:
+        hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_PHB;
+        break;
     default:
         /* we shouldn't be signaling hotplug events for resources
          * that don't support them

[Qemu-devel] [PATCH v3 11/19] qdev: pass an Object * to qbus_set_hotplug_handler()

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

Certain devices types, like memory/CPU, are now being handled using a
hotplug interface provided by a top-level MachineClass. Hotpluggable
host bridges are another such device where it makes sense to use a
machine-level hotplug handler. However, unlike those devices,
host-bridges have a parent bus (the main system bus), and devices with
a parent bus use a different mechanism for registering their hotplug
handlers: qbus_set_hotplug_handler(). This interface currently expects
a handler to be a subclass of DeviceClass, but this is not the case
for MachineClass, which derives directly from ObjectClass.

Internally, the interface only requires an ObjectClass, so expose that
in qbus_set_hotplug_handler().

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Acked-by: Halil Pasic <pasic@linux.ibm.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/pcihp.c               |    2 +-
 hw/acpi/piix4.c               |    2 +-
 hw/char/virtio-serial-bus.c   |    2 +-
 hw/core/bus.c                 |   11 ++---------
 hw/pci/pcie.c                 |    2 +-
 hw/pci/shpc.c                 |    2 +-
 hw/ppc/spapr_pci.c            |    2 +-
 hw/s390x/css-bridge.c         |    2 +-
 hw/s390x/s390-pci-bus.c       |    6 +++---
 hw/scsi/virtio-scsi.c         |    2 +-
 hw/scsi/vmw_pvscsi.c          |    2 +-
 hw/usb/dev-smartcard-reader.c |    2 +-
 include/hw/qdev-core.h        |    3 +--
 13 files changed, 16 insertions(+), 24 deletions(-)

diff --git a/hw/acpi/pcihp.c b/hw/acpi/pcihp.c
index 7bc7a723407b..942918132376 100644
--- a/hw/acpi/pcihp.c
+++ b/hw/acpi/pcihp.c
@@ -251,7 +251,7 @@ void acpi_pcihp_device_plug_cb(HotplugHandler *hotplug_dev, AcpiPciHpState *s,
             object_dynamic_cast(OBJECT(dev), TYPE_PCI_BRIDGE)) {
             PCIBus *sec = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
 
-            qbus_set_hotplug_handler(BUS(sec), DEVICE(hotplug_dev),
+            qbus_set_hotplug_handler(BUS(sec), OBJECT(hotplug_dev),
                                      &error_abort);
             /* We don't have to overwrite any other hotplug handler yet */
             assert(QLIST_EMPTY(&sec->child));
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88f9a9ec0912..df8c0db909ce 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -536,7 +536,7 @@ static void piix4_pm_realize(PCIDevice *dev, Error **errp)
 
     piix4_acpi_system_hot_add_init(pci_address_space_io(dev),
                                    pci_get_bus(dev), s);
-    qbus_set_hotplug_handler(BUS(pci_get_bus(dev)), DEVICE(s), &error_abort);
+    qbus_set_hotplug_handler(BUS(pci_get_bus(dev)), OBJECT(s), &error_abort);
 
     piix4_pm_add_propeties(s);
 }
diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index 04e3ebe3526a..e4310c78f2dc 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -1052,7 +1052,7 @@ static void virtio_serial_device_realize(DeviceState *dev, Error **errp)
     /* Spawn a new virtio-serial bus on which the ports will ride as devices */
     qbus_create_inplace(&vser->bus, sizeof(vser->bus), TYPE_VIRTIO_SERIAL_BUS,
                         dev, vdev->bus_name);
-    qbus_set_hotplug_handler(BUS(&vser->bus), DEVICE(vser), errp);
+    qbus_set_hotplug_handler(BUS(&vser->bus), OBJECT(vser), errp);
     vser->bus.vser = vser;
     QTAILQ_INIT(&vser->ports);
 
diff --git a/hw/core/bus.c b/hw/core/bus.c
index 4651f244864c..e09843f6abea 100644
--- a/hw/core/bus.c
+++ b/hw/core/bus.c
@@ -22,22 +22,15 @@
 #include "hw/qdev.h"
 #include "qapi/error.h"
 
-static void qbus_set_hotplug_handler_internal(BusState *bus, Object *handler,
-                                              Error **errp)
+void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp)
 {
-
     object_property_set_link(OBJECT(bus), OBJECT(handler),
                              QDEV_HOTPLUG_HANDLER_PROPERTY, errp);
 }
 
-void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler, Error **errp)
-{
-    qbus_set_hotplug_handler_internal(bus, OBJECT(handler), errp);
-}
-
 void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp)
 {
-    qbus_set_hotplug_handler_internal(bus, OBJECT(bus), errp);
+    qbus_set_hotplug_handler(bus, OBJECT(bus), errp);
 }
 
 int qbus_walk_children(BusState *bus,
diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
index 2d3d8a047b83..a2287c3eb997 100644
--- a/hw/pci/pcie.c
+++ b/hw/pci/pcie.c
@@ -534,7 +534,7 @@ void pcie_cap_slot_init(PCIDevice *dev, uint16_t slot)
     dev->exp.hpev_notified = false;
 
     qbus_set_hotplug_handler(BUS(pci_bridge_get_sec_bus(PCI_BRIDGE(dev))),
-                             DEVICE(dev), NULL);
+                             OBJECT(dev), NULL);
 }
 
 void pcie_cap_slot_reset(PCIDevice *dev)
diff --git a/hw/pci/shpc.c b/hw/pci/shpc.c
index 45053b39b92c..52ccdc5ae3b9 100644
--- a/hw/pci/shpc.c
+++ b/hw/pci/shpc.c
@@ -648,7 +648,7 @@ int shpc_init(PCIDevice *d, PCIBus *sec_bus, MemoryRegion *bar,
     shpc_cap_update_dword(d);
     memory_region_add_subregion(bar, offset, &shpc->mmio);
 
-    qbus_set_hotplug_handler(BUS(sec_bus), DEVICE(d), NULL);
+    qbus_set_hotplug_handler(BUS(sec_bus), OBJECT(d), NULL);
 
     d->cap_present |= QEMU_PCI_CAP_SHPC;
     return 0;
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 0f6a173d5323..e8c9dcaf2250 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1745,7 +1745,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
                                 &sphb->memspace, &sphb->iospace,
                                 PCI_DEVFN(0, 0), PCI_NUM_PINS, TYPE_PCI_BUS);
     phb->bus = bus;
-    qbus_set_hotplug_handler(BUS(phb->bus), DEVICE(sphb), NULL);
+    qbus_set_hotplug_handler(BUS(phb->bus), OBJECT(sphb), NULL);
 
     /*
      * Initialize PHB address space.
diff --git a/hw/s390x/css-bridge.c b/hw/s390x/css-bridge.c
index 1bd6c8b45860..7573c40badbd 100644
--- a/hw/s390x/css-bridge.c
+++ b/hw/s390x/css-bridge.c
@@ -108,7 +108,7 @@ VirtualCssBus *virtual_css_bus_init(void)
     cbus = VIRTUAL_CSS_BUS(bus);
 
     /* Enable hotplugging */
-    qbus_set_hotplug_handler(bus, dev, &error_abort);
+    qbus_set_hotplug_handler(bus, OBJECT(dev), &error_abort);
 
     css_register_io_adapters(CSS_IO_ADAPTER_VIRTIO, true, false,
                              0, &error_abort);
diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
index 15759b6514a5..c01eed2bd35b 100644
--- a/hw/s390x/s390-pci-bus.c
+++ b/hw/s390x/s390-pci-bus.c
@@ -708,7 +708,7 @@ static void s390_pcihost_realize(DeviceState *dev, Error **errp)
     pci_setup_iommu(b, s390_pci_dma_iommu, s);
 
     bus = BUS(b);
-    qbus_set_hotplug_handler(bus, dev, &local_err);
+    qbus_set_hotplug_handler(bus, OBJECT(dev), &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         return;
@@ -716,7 +716,7 @@ static void s390_pcihost_realize(DeviceState *dev, Error **errp)
     phb->bus = b;
 
     s->bus = S390_PCI_BUS(qbus_create(TYPE_S390_PCI_BUS, dev, NULL));
-    qbus_set_hotplug_handler(BUS(s->bus), dev, &local_err);
+    qbus_set_hotplug_handler(BUS(s->bus), OBJECT(dev), &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         return;
@@ -844,7 +844,7 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
         pci_setup_iommu(&pb->sec_bus, s390_pci_dma_iommu, s);
 
         bus = BUS(&pb->sec_bus);
-        qbus_set_hotplug_handler(bus, DEVICE(s), errp);
+        qbus_set_hotplug_handler(bus, OBJECT(s), errp);
 
         if (dev->hotplugged) {
             pci_default_write_config(pdev, PCI_PRIMARY_BUS, s->bus_no, 1);
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 3aa99717e235..9cf6290c2986 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -893,7 +893,7 @@ static void virtio_scsi_device_realize(DeviceState *dev, Error **errp)
     scsi_bus_new(&s->bus, sizeof(s->bus), dev,
                  &virtio_scsi_scsi_info, vdev->bus_name);
     /* override default SCSI bus hotplug-handler, with virtio-scsi's one */
-    qbus_set_hotplug_handler(BUS(&s->bus), dev, &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
 
     virtio_scsi_dataplane_setup(s, errp);
 }
diff --git a/hw/scsi/vmw_pvscsi.c b/hw/scsi/vmw_pvscsi.c
index a3a019e30a74..584b4be07e79 100644
--- a/hw/scsi/vmw_pvscsi.c
+++ b/hw/scsi/vmw_pvscsi.c
@@ -1142,7 +1142,7 @@ pvscsi_realizefn(PCIDevice *pci_dev, Error **errp)
     scsi_bus_new(&s->bus, sizeof(s->bus), DEVICE(pci_dev),
                  &pvscsi_scsi_info, NULL);
     /* override default SCSI bus hotplug-handler, with pvscsi's one */
-    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(s), &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(s), &error_abort);
     pvscsi_reset_state(s);
 }
 
diff --git a/hw/usb/dev-smartcard-reader.c b/hw/usb/dev-smartcard-reader.c
index 8f716fc165a3..6b0137bb7699 100644
--- a/hw/usb/dev-smartcard-reader.c
+++ b/hw/usb/dev-smartcard-reader.c
@@ -1322,7 +1322,7 @@ static void ccid_realize(USBDevice *dev, Error **errp)
     usb_desc_init(dev);
     qbus_create_inplace(&s->bus, sizeof(s->bus), TYPE_CCID_BUS, DEVICE(dev),
                         NULL);
-    qbus_set_hotplug_handler(BUS(&s->bus), DEVICE(dev), &error_abort);
+    qbus_set_hotplug_handler(BUS(&s->bus), OBJECT(dev), &error_abort);
     s->intr = usb_ep_get(dev, USB_TOKEN_IN, CCID_INT_IN_EP);
     s->bulk = usb_ep_get(dev, USB_TOKEN_IN, CCID_BULK_IN_EP);
     s->card = NULL;
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index bc014c1c9fca..e11f6badcc2e 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -427,8 +427,7 @@ char *qdev_get_dev_path(DeviceState *dev);
 
 GSList *qdev_build_hotpluggable_device_list(Object *peripheral);
 
-void qbus_set_hotplug_handler(BusState *bus, DeviceState *handler,
-                              Error **errp);
+void qbus_set_hotplug_handler(BusState *bus, Object *handler, Error **errp);
 
 void qbus_set_bus_hotplug_handler(BusState *bus, Error **errp);
 

[Qemu-devel] [PATCH v3 12/19] spapr_pci: provide node start offset via spapr_populate_pci_dt()

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

PHB hotplug re-uses PHB device tree generation code and passes
it to a guest via RTAS. Doing this requires knowledge of where
exactly in the device tree the node describing the PHB begins.

Provide this via a new optional pointer that can be used to
store the PHB node's start offset.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c              |    2 +-
 hw/ppc/spapr_pci.c          |    5 ++++-
 include/hw/pci-host/spapr.h |    2 +-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 83ec79caf48e..2c8d0c0e149c 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1297,7 +1297,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
 
     QLIST_FOREACH(phb, &spapr->phbs, list) {
         ret = spapr_populate_pci_dt(phb, PHANDLE_INTC, fdt,
-                                    spapr->irq->nr_msis);
+                                    spapr->irq->nr_msis, NULL);
         if (ret < 0) {
             error_report("couldn't setup PCI devices in fdt");
             exit(1);
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index e8c9dcaf2250..e85cb31e0447 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -2139,7 +2139,7 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
 }
 
 int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
-                          uint32_t nr_msis)
+                          uint32_t nr_msis, int *node_offset)
 {
     int bus_off, i, j, ret;
     gchar *nodename;
@@ -2194,6 +2194,9 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
     nodename = g_strdup_printf("pci@%" PRIx64, phb->buid);
     _FDT(bus_off = fdt_add_subnode(fdt, 0, nodename));
     g_free(nodename);
+    if (node_offset) {
+        *node_offset = bus_off;
+    }
 
     /* Write PHB properties */
     _FDT(fdt_setprop_string(fdt, bus_off, "device_type", "pci"));
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 2ab75e28dbb5..e8ad1530d31d 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -114,7 +114,7 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
 }
 
 int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
-                          uint32_t nr_msis);
+                          uint32_t nr_msis, int *node_offset);
 
 void spapr_pci_rtas_init(void);
 

[Qemu-devel] [PATCH v3 13/19] spapr_pci: add ibm, my-drc-index property for PHB hotplug

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

This is needed to denote a boot-time PHB as being hot-pluggable.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_pci.c |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index e85cb31e0447..a0311560e47b 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -2189,6 +2189,7 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
     sPAPRTCETable *tcet;
     PCIBus *bus = PCI_HOST_BRIDGE(phb)->bus;
     sPAPRFDT s_fdt;
+    sPAPRDRConnector *drc;
 
     /* Start populating the FDT */
     nodename = g_strdup_printf("pci@%" PRIx64, phb->buid);
@@ -2255,6 +2256,14 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
                  tcet->liobn, tcet->bus_offset,
                  tcet->nb_table << tcet->page_shift);
 
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, phb->index);
+    if (drc) {
+        uint32_t drc_index = cpu_to_be32(spapr_drc_index(drc));
+
+        _FDT(fdt_setprop(fdt, bus_off, "ibm,my-drc-index", &drc_index,
+                         sizeof(drc_index)));
+    }
+
     /* Walk the bridges and program the bus numbers*/
     spapr_phb_pci_enumerate(phb);
     _FDT(fdt_setprop_cell(fdt, bus_off, "qemu,phb-enumerated", 0x1));

[Qemu-devel] [PATCH v3 14/19] spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend

[Greg Kurz]

The setting of the "phandle" and "linux,phandle" properties of the
interrupt controller node is duplicate code in XICS and XIVE. Move
it to a new spapr_irq_dt_populate() function.

Note that for XIVE, we set the "ibm,plat-res-int-priorities" machine
property earlier so that the offset returned by fdt_add_subnode()
can be safely returned to the caller.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/intc/spapr_xive.c        |   21 +++++++++------------
 hw/intc/xics_spapr.c        |    7 +++----
 hw/ppc/spapr.c              |    2 +-
 hw/ppc/spapr_irq.c          |   17 +++++++++++++----
 include/hw/ppc/spapr_irq.h  |    6 ++++--
 include/hw/ppc/spapr_xive.h |    3 +--
 include/hw/ppc/xics_spapr.h |    3 +--
 7 files changed, 32 insertions(+), 27 deletions(-)

diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
index aab63cfd1178..e542ae59d7fd 100644
--- a/hw/intc/spapr_xive.c
+++ b/hw/intc/spapr_xive.c
@@ -1411,8 +1411,7 @@ void spapr_xive_hcall_init(sPAPRMachineState *spapr)
     spapr_register_hypercall(H_INT_RESET, h_int_reset);
 }
 
-void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
-                   uint32_t phandle)
+int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
 {
     sPAPRXive *xive = spapr->xive;
     int node;
@@ -1439,6 +1438,13 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
     };
     gchar *nodename;
 
+    /*
+     * The "ibm,plat-res-int-priorities" property defines the priority
+     * ranges reserved by the hypervisor
+     */
+    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
+                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
+
     /* Thread Interrupt Management Area : User (ring 3) and OS (ring 2) */
     timas[0] = cpu_to_be64(xive->tm_base +
                            XIVE_TM_USER_PAGE * (1ull << TM_SHIFT));
@@ -1465,14 +1471,5 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
     _FDT(fdt_setprop(fdt, node, "interrupt-controller", NULL, 0));
     _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
 
-    /* For SLOF */
-    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
-    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
-
-    /*
-     * The "ibm,plat-res-int-priorities" property defines the priority
-     * ranges reserved by the hypervisor
-     */
-    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
-                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
+    return node;
 }
diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
index de6cc15b6474..375cb883c86d 100644
--- a/hw/intc/xics_spapr.c
+++ b/hw/intc/xics_spapr.c
@@ -245,8 +245,7 @@ void xics_spapr_init(sPAPRMachineState *spapr)
     spapr_register_hypercall(H_IPOLL, h_ipoll);
 }
 
-void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
-                   uint32_t phandle)
+int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
 {
     uint32_t interrupt_server_ranges_prop[] = {
         0, cpu_to_be32(nr_servers),
@@ -263,6 +262,6 @@ void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
                      interrupt_server_ranges_prop,
                      sizeof(interrupt_server_ranges_prop)));
     _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
-    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
-    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
+
+    return node;
 }
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2c8d0c0e149c..135136b4cf5f 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1275,7 +1275,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
     _FDT(fdt_setprop_cell(fdt, 0, "#size-cells", 2));
 
     /* /interrupt controller */
-    spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
+    spapr_irq_dt_populate(spapr, spapr_max_server_number(spapr), fdt,
                           PHANDLE_INTC);
 
     ret = spapr_populate_memory(spapr, fdt);
diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index 396dd09bdbe0..b3e0713428ed 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -11,6 +11,7 @@
 #include "qemu/log.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
+#include "hw/ppc/fdt.h"
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/spapr_xive.h"
 #include "hw/ppc/xics.h"
@@ -536,11 +537,10 @@ static void spapr_irq_print_info_dual(sPAPRMachineState *spapr, Monitor *mon)
     spapr_irq_current(spapr)->print_info(spapr, mon);
 }
 
-static void spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
-                                       uint32_t nr_servers, void *fdt,
-                                       uint32_t phandle)
+static int spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
+                                      uint32_t nr_servers, void *fdt)
 {
-    spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt, phandle);
+    return spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt);
 }
 
 static void spapr_irq_cpu_intc_create_dual(sPAPRMachineState *spapr,
@@ -671,6 +671,15 @@ void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp)
     }
 }
 
+void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
+                           void *fdt, uint32_t phandle)
+{
+    int node = spapr->irq->dt_populate(spapr, nr_servers, fdt);
+
+    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
+    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
+}
+
 /*
  * XICS legacy routines - to deprecate one day
  */
diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
index 42b325eb51ba..dfed6b7e9976 100644
--- a/include/hw/ppc/spapr_irq.h
+++ b/include/hw/ppc/spapr_irq.h
@@ -41,8 +41,8 @@ typedef struct sPAPRIrq {
     void (*free)(sPAPRMachineState *spapr, int irq, int num);
     qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
     void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
-    void (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
-                        void *fdt, uint32_t phandle);
+    int (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
+                       void *fdt);
     void (*cpu_intc_create)(sPAPRMachineState *spapr, PowerPCCPU *cpu,
                             Error **errp);
     int (*post_load)(sPAPRMachineState *spapr, int version_id);
@@ -63,6 +63,8 @@ void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
 qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
 int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
 void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
+void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
+                           void *fdt, uint32_t phandle);
 
 /*
  * XICS legacy routines
diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
index f8854a4a6a18..29dafead9ba0 100644
--- a/include/hw/ppc/spapr_xive.h
+++ b/include/hw/ppc/spapr_xive.h
@@ -44,8 +44,7 @@ void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon);
 typedef struct sPAPRMachineState sPAPRMachineState;
 
 void spapr_xive_hcall_init(sPAPRMachineState *spapr);
-void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
-                   uint32_t phandle);
+int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
 void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx);
 void spapr_xive_mmio_set_enabled(sPAPRXive *xive, bool enable);
 
diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
index b1ab27d022cf..9f51f8621f75 100644
--- a/include/hw/ppc/xics_spapr.h
+++ b/include/hw/ppc/xics_spapr.h
@@ -29,8 +29,7 @@
 
 #include "hw/ppc/spapr.h"
 
-void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
-                   uint32_t phandle);
+int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
 int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
 void xics_spapr_init(sPAPRMachineState *spapr);
 

[Qemu-devel] [PATCH v3 15/19] spapr_xive: Cache device tree nodename in sPAPRXive

[Greg Kurz]

PHB hotplug will need to know the name of the XIVE controller node. Since
it is an invariant for the machine lifetime, compute it at realize and
store it under the sPAPRXive structure.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/intc/spapr_xive.c        |    9 ++++-----
 include/hw/ppc/spapr_xive.h |    3 +++
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
index e542ae59d7fd..9732c3d89c77 100644
--- a/hw/intc/spapr_xive.c
+++ b/hw/intc/spapr_xive.c
@@ -316,6 +316,9 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
     /* Map all regions */
     spapr_xive_map_mmio(xive);
 
+    xive->nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
+                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
+
     qemu_register_reset(spapr_xive_reset, dev);
 }
 
@@ -1436,7 +1439,6 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
         cpu_to_be32(7),    /* start */
         cpu_to_be32(0xf8), /* count */
     };
-    gchar *nodename;
 
     /*
      * The "ibm,plat-res-int-priorities" property defines the priority
@@ -1453,10 +1455,7 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
                            XIVE_TM_OS_PAGE * (1ull << TM_SHIFT));
     timas[3] = cpu_to_be64(1ull << TM_SHIFT);
 
-    nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
-                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
-    _FDT(node = fdt_add_subnode(fdt, 0, nodename));
-    g_free(nodename);
+    _FDT(node = fdt_add_subnode(fdt, 0, xive->nodename));
 
     _FDT(fdt_setprop_string(fdt, node, "device_type", "power-ivpe"));
     _FDT(fdt_setprop(fdt, node, "reg", timas, sizeof(timas)));
diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
index 29dafead9ba0..deea34b03ee5 100644
--- a/include/hw/ppc/spapr_xive.h
+++ b/include/hw/ppc/spapr_xive.h
@@ -26,6 +26,9 @@ typedef struct sPAPRXive {
     XiveENDSource end_source;
     hwaddr        end_base;
 
+    /* DT */
+    gchar *nodename;
+
     /* Routing table */
     XiveEAS       *eat;
     uint32_t      nr_irqs;

[Qemu-devel] [PATCH v3 16/19] spapr: Expose the name of the interrupt controller node

[Greg Kurz]

This will be needed by PHB hotplug in order to access the "phandle"
property of the interrupt controller node.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/intc/xics_spapr.c        |    2 +-
 include/hw/ppc/spapr_xive.h |    5 +++++
 include/hw/ppc/xics_spapr.h |    7 +++++++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
index 375cb883c86d..ec50e21b8607 100644
--- a/hw/intc/xics_spapr.c
+++ b/hw/intc/xics_spapr.c
@@ -252,7 +252,7 @@ int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
     };
     int node;
 
-    _FDT(node = fdt_add_subnode(fdt, 0, "interrupt-controller"));
+    _FDT(node = fdt_add_subnode(fdt, 0, XICS_NODENAME));
 
     _FDT(fdt_setprop_string(fdt, node, "device_type",
                             "PowerPC-External-Interrupt-Presentation"));
diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
index deea34b03ee5..e769c2f9a018 100644
--- a/include/hw/ppc/spapr_xive.h
+++ b/include/hw/ppc/spapr_xive.h
@@ -51,4 +51,9 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
 void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx);
 void spapr_xive_mmio_set_enabled(sPAPRXive *xive, bool enable);
 
+static inline const char *spapr_xive_get_nodename(sPAPRXive *xive)
+{
+    return xive->nodename;
+}
+
 #endif /* PPC_SPAPR_XIVE_H */
diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
index 9f51f8621f75..fd66cf7bbba3 100644
--- a/include/hw/ppc/xics_spapr.h
+++ b/include/hw/ppc/xics_spapr.h
@@ -29,6 +29,13 @@
 
 #include "hw/ppc/spapr.h"
 
+#define XICS_NODENAME "interrupt-controller"
+
+static inline const char *xics_spapr_get_nodename(void)
+{
+    return XICS_NODENAME;
+}
+
 int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
 int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
 void xics_spapr_init(sPAPRMachineState *spapr);

[Qemu-devel] [PATCH v3 17/19] spapr_irq: Expose the phandle of the interrupt controller

[Greg Kurz]

This will be used by PHB hotplug in order to create the "interrupt-map"
property of the PHB node.

Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr_irq.c         |   44 +++++++++++++++++++++++++++++++++++++++++++-
 include/hw/ppc/spapr_irq.h |    3 +++
 2 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index b3e0713428ed..93243cfd0644 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -243,6 +243,11 @@ static void spapr_irq_reset_xics(sPAPRMachineState *spapr, Error **errp)
     /* TODO: create the KVM XICS device */
 }
 
+static const char *spapr_irq_get_nodename_xics(sPAPRMachineState *spapr)
+{
+    return xics_spapr_get_nodename();
+}
+
 #define SPAPR_IRQ_XICS_NR_IRQS     0x1000
 #define SPAPR_IRQ_XICS_NR_MSIS     \
     (XICS_IRQ_BASE + SPAPR_IRQ_XICS_NR_IRQS - SPAPR_IRQ_MSI)
@@ -263,6 +268,7 @@ sPAPRIrq spapr_irq_xics = {
     .post_load   = spapr_irq_post_load_xics,
     .reset       = spapr_irq_reset_xics,
     .set_irq     = spapr_irq_set_irq_xics,
+    .get_nodename = spapr_irq_get_nodename_xics,
 };
 
 /*
@@ -401,6 +407,11 @@ static void spapr_irq_set_irq_xive(void *opaque, int srcno, int val)
     xive_source_set_irq(&spapr->xive->source, srcno, val);
 }
 
+static const char *spapr_irq_get_nodename_xive(sPAPRMachineState *spapr)
+{
+    return spapr_xive_get_nodename(spapr->xive);
+}
+
 /*
  * XIVE uses the full IRQ number space. Set it to 8K to be compatible
  * with XICS.
@@ -425,6 +436,7 @@ sPAPRIrq spapr_irq_xive = {
     .post_load   = spapr_irq_post_load_xive,
     .reset       = spapr_irq_reset_xive,
     .set_irq     = spapr_irq_set_irq_xive,
+    .get_nodename = spapr_irq_get_nodename_xive,
 };
 
 /*
@@ -588,6 +600,11 @@ static void spapr_irq_set_irq_dual(void *opaque, int srcno, int val)
     spapr_irq_current(spapr)->set_irq(spapr, srcno, val);
 }
 
+static const char *spapr_irq_get_nodename_dual(sPAPRMachineState *spapr)
+{
+    return spapr_irq_current(spapr)->get_nodename(spapr);
+}
+
 /*
  * Define values in sync with the XIVE and XICS backend
  */
@@ -609,7 +626,8 @@ sPAPRIrq spapr_irq_dual = {
     .cpu_intc_create = spapr_irq_cpu_intc_create_dual,
     .post_load   = spapr_irq_post_load_dual,
     .reset       = spapr_irq_reset_dual,
-    .set_irq     = spapr_irq_set_irq_dual
+    .set_irq     = spapr_irq_set_irq_dual,
+    .get_nodename = spapr_irq_get_nodename_dual,
 };
 
 /*
@@ -680,6 +698,29 @@ void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
     _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
 }
 
+uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
+                               Error **errp)
+{
+    const char *nodename = spapr->irq->get_nodename(spapr);
+    int phandle, offset;
+
+    offset = fdt_subnode_offset(fdt, 0, nodename);
+    if (offset < 0) {
+        error_setg(errp, "Can't find node \"%s\": %s", nodename,
+                   fdt_strerror(offset));
+        return -1;
+    }
+
+    phandle = fdt_get_phandle(fdt, offset);
+    if (phandle < 0) {
+        error_setg(errp, "Can't get phandle of node \"%s\": %s",
+                   nodename, fdt_strerror(phandle));
+        return -1;
+    }
+
+    return phandle;
+}
+
 /*
  * XICS legacy routines - to deprecate one day
  */
@@ -752,4 +793,5 @@ sPAPRIrq spapr_irq_xics_legacy = {
     .cpu_intc_create = spapr_irq_cpu_intc_create_xics,
     .post_load   = spapr_irq_post_load_xics,
     .set_irq     = spapr_irq_set_irq_xics,
+    .get_nodename = spapr_irq_get_nodename_xics,
 };
diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
index dfed6b7e9976..d44d659231a6 100644
--- a/include/hw/ppc/spapr_irq.h
+++ b/include/hw/ppc/spapr_irq.h
@@ -48,6 +48,7 @@ typedef struct sPAPRIrq {
     int (*post_load)(sPAPRMachineState *spapr, int version_id);
     void (*reset)(sPAPRMachineState *spapr, Error **errp);
     void (*set_irq)(void *opaque, int srcno, int val);
+    const char *(*get_nodename)(sPAPRMachineState *spapr);
 } sPAPRIrq;
 
 extern sPAPRIrq spapr_irq_xics;
@@ -65,6 +66,8 @@ int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
 void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
 void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
                            void *fdt, uint32_t phandle);
+uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
+                               Error **errp);
 
 /*
  * XICS legacy routines

[Qemu-devel] [PATCH v3 18/19] spapr: add hotplug hooks for PHB hotplug

[Greg Kurz]

Hotplugging PHBs is a machine-level operation, but PHBs reside on the
main system bus, so we register spapr machine as the handler for the
main system bus.

We re-get the phandle of the interrupt controller systematically for
simplicity.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
v3: - reworked phandle handling some more
v2: - reworked phandle handling
    - sync LSIs to KVM
---
 hw/ppc/spapr.c         |  124 ++++++++++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_drc.c     |    1 
 hw/ppc/spapr_pci.c     |   16 ------
 include/hw/ppc/spapr.h |    1 
 4 files changed, 126 insertions(+), 16 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 135136b4cf5f..d94284f423be 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2936,6 +2936,11 @@ static void spapr_machine_init(MachineState *machine)
     register_savevm_live(NULL, "spapr/htab", -1, 1,
                          &savevm_htab_handlers, spapr);
 
+    if (smc->dr_phb_enabled) {
+        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine),
+                                 &error_fatal);
+    }
+
     qemu_register_boot_set(spapr_boot_set, spapr);
 
     if (kvm_enabled()) {
@@ -3733,6 +3738,111 @@ out:
     error_propagate(errp, local_err);
 }
 
+static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                               Error **errp)
+{
+    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
+    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
+
+    if (sphb->index == (uint32_t)-1) {
+        error_setg(errp, "\"index\" for PAPR PHB is mandatory");
+        return;
+    }
+
+    /*
+     * This will check that sphb->index doesn't exceed the maximum number of
+     * PHBs for the current machine type.
+     */
+    smc->phb_placement(spapr, sphb->index,
+                       &sphb->buid, &sphb->io_win_addr,
+                       &sphb->mem_win_addr, &sphb->mem64_win_addr,
+                       windows_supported, sphb->dma_liobn, errp);
+}
+
+static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
+                           Error **errp)
+{
+    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
+    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+    void *fdt = NULL;
+    int fdt_start_offset;
+    int fdt_size;
+    Error *local_err = NULL;
+    sPAPRDRConnector *drc;
+    int ret;
+    bool hotplugged = spapr_drc_hotplugged(dev);
+    int phandle;
+
+    if (!smc->dr_phb_enabled) {
+        return;
+    }
+
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->index);
+    /* hotplug hooks should check it's enabled before getting this far */
+    assert(drc);
+
+    if (hotplugged) {
+        phandle = spapr_irq_get_phandle(spapr, spapr->fdt_blob, &local_err);
+        if (local_err) {
+            goto out;
+        }
+    } else {
+        phandle = PHANDLE_INTC;
+    }
+
+    fdt = create_device_tree(&fdt_size);
+    ret = spapr_populate_pci_dt(sphb, phandle, fdt, spapr->irq->nr_msis,
+                                &fdt_start_offset);
+    if (ret < 0) {
+        error_setg(&local_err, "unable to create FDT for %sPHB",
+                   dev->hotplugged ? "hotplugged " : "");
+        goto out;
+    }
+
+    if (hotplugged) {
+        /* generally SLOF creates these, for hotplug it's up to QEMU */
+        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
+    }
+
+    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
+
+out:
+    if (local_err) {
+        error_propagate(errp, local_err);
+        g_free(fdt);
+        return;
+    }
+
+    if (hotplugged) {
+        spapr_hotplug_req_add_by_index(drc);
+    } else if (drc) {
+        spapr_drc_reset(drc);
+    }
+}
+
+void spapr_phb_release(DeviceState *dev)
+{
+    object_unparent(OBJECT(dev));
+}
+
+static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
+                                     DeviceState *dev, Error **errp)
+{
+    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
+    sPAPRDRConnector *drc;
+
+    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->index);
+    assert(drc);
+
+    if (!spapr_drc_unplug_requested(drc)) {
+        spapr_drc_detach(drc);
+        spapr_hotplug_req_remove_by_index(drc);
+    }
+}
+
 static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
                                       DeviceState *dev, Error **errp)
 {
@@ -3740,6 +3850,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
         spapr_memory_plug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
         spapr_core_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        spapr_phb_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -3758,6 +3870,7 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
 {
     sPAPRMachineState *sms = SPAPR_MACHINE(OBJECT(hotplug_dev));
     MachineClass *mc = MACHINE_GET_CLASS(sms);
+    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
 
     if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
         if (spapr_ovec_test(sms->ov5_cas, OV5_HP_EVT)) {
@@ -3777,6 +3890,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
             return;
         }
         spapr_core_unplug_request(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        if (!smc->dr_phb_enabled) {
+            error_setg(errp, "PHB hot unplug not supported on this machine");
+            return;
+        }
+        spapr_phb_unplug_request(hotplug_dev, dev, errp);
     }
 }
 
@@ -3787,6 +3906,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
         spapr_memory_pre_plug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
         spapr_core_pre_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
+        spapr_phb_pre_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -3794,7 +3915,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
                                                  DeviceState *dev)
 {
     if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
-        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
+        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
         return HOTPLUG_HANDLER(machine);
     }
     return NULL;
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 189ee681062a..7a2676716364 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -703,6 +703,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
     drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
     drck->typename = "PHB";
     drck->drc_name_prefix = "PHB ";
+    drck->release = spapr_phb_release;
 }
 
 static const TypeInfo spapr_dr_connector_info = {
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index a0311560e47b..b03bfcf3f446 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -1649,21 +1649,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (sphb->index != (uint32_t)-1) {
-        Error *local_err = NULL;
-
-        smc->phb_placement(spapr, sphb->index,
-                           &sphb->buid, &sphb->io_win_addr,
-                           &sphb->mem_win_addr, &sphb->mem64_win_addr,
-                           windows_supported, sphb->dma_liobn, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
-            return;
-        }
-    } else {
-        error_setg(errp, "\"index\" for PAPR PHB is mandatory");
-        return;
-    }
+    assert(sphb->index != (uint32_t)-1); /* checked in spapr_phb_pre_plug() */
 
     if (sphb->mem64_win_size != 0) {
         if (sphb->mem_win_size > SPAPR_PCI_MEM32_WIN_SIZE) {
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index bc8ebb072258..f78e44d58cef 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -766,6 +766,7 @@ int spapr_max_server_number(sPAPRMachineState *spapr);
 /* CPU and LMB DRC release callbacks. */
 void spapr_core_release(DeviceState *dev);
 void spapr_lmb_release(DeviceState *dev);
+void spapr_phb_release(DeviceState *dev);
 
 void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
 int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);

[Qemu-devel] [PATCH v3 19/19] spapr: enable PHB hotplug for default pseries machine type

[Greg Kurz]

From: Michael Roth <mdroth@linux.vnet.ibm.com>

The 'dr_phb_enabled' field of that class can be set as part of
machine-specific init code. It will be used to conditionally
enable creation of DRC objects and device-tree description to
facilitate hotplug of PHBs.

Since we can't migrate this state to older machine types,
default the option to true and disable it for older machine
types.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
---
 hw/ppc/spapr.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index d94284f423be..2687b65798d7 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4169,6 +4169,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF;
     spapr_caps_add_properties(smc, &error_abort);
     smc->irq = &spapr_irq_xics;
+    smc->dr_phb_enabled = true;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4234,6 +4235,7 @@ static void spapr_machine_3_1_class_options(MachineClass *mc)
     compat_props_add(mc->compat_props, hw_compat_3_1, hw_compat_3_1_len);
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
     smc->update_dt_enabled = false;
+    smc->dr_phb_enabled = false;
 }
 
 DEFINE_SPAPR_MACHINE(3_1, "3.1", false);

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[Greg Kurz]

On Thu, 17 Jan 2019 18:14:46 +0100
Greg Kurz <groug@kaod.org> wrote:

> The current code assumes that an interrupt is allocated as soon as its
> type is set to MSI or LSI. PHB hotplug will require to be able to set
> the type of an interrupt before actually allocating it.
> 
> Disintricate type setting from allocation by using another flag bit for
> the latter. Introduce a new ics_claim_irq() function for allocation. The
> behavior of aborting if the same irq gets allocated twice is kept.
> 
> ics_set_irq_type() now only sets the type to MSI or LSI. It doesn't bring
> anything to abort if the type was already set before. Drop the assert
> and XICS_FLAGS_IRQ_MASK on the way.
> 

With some more testing (and thinking :) I now realize that the type of
the irq should survive PHB unplug, which is not the case with this patch.

This calls for a v4.

> Older QEMUs don't know about XICS_FLAGS_IRQ_CLAIMED. In order to safely
> handle incoming migration, we must fix the irq flags. This is done at
> post load thanks to a compat property. We don't need to do anything for
> backward migration since older machines only care for the irq type.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/core/machine.c     |    4 ++++
>  hw/intc/xics.c        |   28 +++++++++++++++++++++++++---
>  hw/ppc/pnv_psi.c      |    1 +
>  hw/ppc/spapr_irq.c    |    4 +---
>  include/hw/ppc/xics.h |    8 ++++++--
>  5 files changed, 37 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 1a0a9ab1117a..536a34092367 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -25,6 +25,10 @@
>  
>  GlobalProperty hw_compat_3_1[] = {
>      {
> +        .driver   = "ics-base",
> +        .property = "has-claimed-flag",
> +        .value    = "off",
> +    },{
>          .driver   = "pcie-root-port",
>          .property = "x-speed",
>          .value    = "2_5",
> diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> index 16e8ffa2aaf7..82cf04548907 100644
> --- a/hw/intc/xics.c
> +++ b/hw/intc/xics.c
> @@ -75,7 +75,7 @@ void ics_pic_print_info(ICSState *ics, Monitor *mon)
>      for (i = 0; i < ics->nr_irqs; i++) {
>          ICSIRQState *irq = ics->irqs + i;
>  
> -        if (!(irq->flags & XICS_FLAGS_IRQ_MASK)) {
> +        if (!(irq->flags & XICS_FLAGS_IRQ_CLAIMED)) {
>              continue;
>          }
>          monitor_printf(mon, "  %4x %s %02x %02x\n",
> @@ -662,6 +662,22 @@ static int ics_base_dispatch_post_load(void *opaque, int version_id)
>      ICSState *ics = opaque;
>      ICSStateClass *info = ICS_BASE_GET_CLASS(ics);
>  
> +    if (!ics->has_claimed_flag) {
> +        int i;
> +
> +        for (i = 0; i < ics->nr_irqs; i++) {
> +            ICSIRQState *irq = ics->irqs + i;
> +
> +            /*
> +             * For older machines, allocating the irq and setting its type is
> +             * the same thing.
> +             */
> +            if (irq->flags & (XICS_FLAGS_IRQ_LSI| XICS_FLAGS_IRQ_MSI)) {
> +                irq->flags |= XICS_FLAGS_IRQ_CLAIMED;
> +            }
> +        }
> +    }
> +
>      if (info->post_load) {
>          return info->post_load(ics, version_id);
>      }
> @@ -702,6 +718,7 @@ static const VMStateDescription vmstate_ics_base = {
>  
>  static Property ics_base_properties[] = {
>      DEFINE_PROP_UINT32("nr-irqs", ICSState, nr_irqs, 0),
> +    DEFINE_PROP_BOOL("has-claimed-flag", ICSState, has_claimed_flag, true),
>      DEFINE_PROP_END_OF_LIST(),
>  };
>  
> @@ -741,10 +758,15 @@ ICPState *xics_icp_get(XICSFabric *xi, int server)
>      return xic->icp_get(xi, server);
>  }
>  
> -void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> +void ics_claim_irq(ICSState *ics, int srcno)
>  {
> -    assert(!(ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MASK));
> +    assert(ICS_IRQ_FREE(ics, srcno));
> +
> +    ics->irqs[srcno].flags |= XICS_FLAGS_IRQ_CLAIMED;
> +}
>  
> +void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> +{
>      ics->irqs[srcno].flags |=
>          lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI;
>  }
> diff --git a/hw/ppc/pnv_psi.c b/hw/ppc/pnv_psi.c
> index 8ced09506321..ced34e1119dc 100644
> --- a/hw/ppc/pnv_psi.c
> +++ b/hw/ppc/pnv_psi.c
> @@ -488,6 +488,7 @@ static void pnv_psi_realize(DeviceState *dev, Error **errp)
>  
>      for (i = 0; i < ics->nr_irqs; i++) {
>          ics_set_irq_type(ics, i, true);
> +        ics_claim_irq(ics, i);
>      }
>  
>      psi->qirqs = qemu_allocate_irqs(ics_simple_set_irq, ics, ics->nr_irqs);
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index 1da7a32348fc..86c712d15382 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -125,9 +125,6 @@ error:
>      error_propagate(errp, local_err);
>  }
>  
> -#define ICS_IRQ_FREE(ics, srcno)   \
> -    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_MASK)))
> -
>  static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
>                                  Error **errp)
>  {
> @@ -146,6 +143,7 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
>      }
>  
>      ics_set_irq_type(ics, irq - ics->offset, lsi);
> +    ics_claim_irq(ics, irq - ics->offset);
>      return 0;
>  }
>  
> diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
> index fad786e8b22d..e3be5cc663c5 100644
> --- a/include/hw/ppc/xics.h
> +++ b/include/hw/ppc/xics.h
> @@ -131,6 +131,7 @@ struct ICSState {
>      /*< public >*/
>      uint32_t nr_irqs;
>      uint32_t offset;
> +    bool has_claimed_flag;
>      ICSIRQState *irqs;
>      XICSFabric *xics;
>  };
> @@ -153,13 +154,15 @@ struct ICSIRQState {
>  #define XICS_STATUS_PRESENTED          0x10
>  #define XICS_STATUS_QUEUED             0x20
>      uint8_t status;
> -/* (flags & XICS_FLAGS_IRQ_MASK) == 0 means the interrupt is not allocated */
>  #define XICS_FLAGS_IRQ_LSI             0x1
>  #define XICS_FLAGS_IRQ_MSI             0x2
> -#define XICS_FLAGS_IRQ_MASK            0x3
> +#define XICS_FLAGS_IRQ_CLAIMED         0x4
>      uint8_t flags;
>  };
>  
> +#define ICS_IRQ_FREE(ics, srcno)   \
> +    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_CLAIMED)))
> +
>  struct XICSFabric {
>      Object parent;
>  };
> @@ -193,6 +196,7 @@ void ics_simple_write_xive(ICSState *ics, int nr, int server,
>  void ics_simple_set_irq(void *opaque, int srcno, int val);
>  void ics_kvm_set_irq(void *opaque, int srcno, int val);
>  
> +void ics_claim_irq(ICSState *ics, int srcno);
>  void ics_set_irq_type(ICSState *ics, int srcno, bool lsi);
>  void icp_pic_print_info(ICPState *icp, Monitor *mon);
>  void ics_pic_print_info(ICSState *ics, Monitor *mon);
> 

Re: [Qemu-devel] [PATCH v3 07/19] spapr_pci: add PHB unrealize

[Greg Kurz]

On Thu, 17 Jan 2019 18:15:12 +0100
Greg Kurz <groug@kaod.org> wrote:

> To support PHB hotplug we need to clean up lingering references,
> memory, child properties, etc. prior to the PHB object being
> finalized. Generally this will be called as a result of calling
> object_unparent() on the PHB object, which in turn would normally
> be called as the result of an unplug() operation.
> 
> When the PHB is finalized, child objects will be unparented in
> turn, and finalized if the PHB was the only reference holder. so
> we don't bother to explicitly unparent child objects of the PHB
> (spapr_iommu, spapr_drc, etc).
> 
> The formula that gives the number of DMA windows is moved to an
> inline function in the hw/pci-host/spapr.h header because it
> will have other users.
> 
> The unrealize function is able to cope with partially realized PHBs.
> It is hence used to implement proper rollback on the realize error
> path. Note that older machines that used the legacy irq allocation
> cannot support PHB hotplug, so we don't care to free their LSIs in
> the unrealize function.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> v3: - don't free LSIs at unrealize

Oops... I've mixed up with an intermediate version. We DO want to free LSIs
at unrealize.

Will fix in v4.

> v2: - implement rollback with unrealize function
> ---
>  hw/ppc/spapr_pci.c          |   63 ++++++++++++++++++++++++++++++++++++++++---
>  include/hw/pci-host/spapr.h |    5 +++
>  2 files changed, 64 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index f5f13a4d4816..0f6a173d5323 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1559,6 +1559,56 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
>      }
>  }
>  
> +static void spapr_phb_finalizefn(Object *obj)
> +{
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(obj);
> +
> +    g_free(sphb->dtbusname);
> +    sphb->dtbusname = NULL;
> +}
> +
> +static void spapr_phb_unrealize(DeviceState *dev, Error **errp)
> +{
> +    SysBusDevice *s = SYS_BUS_DEVICE(dev);
> +    PCIHostState *phb = PCI_HOST_BRIDGE(s);
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(phb);
> +    sPAPRTCETable *tcet;
> +    int i;
> +    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
> +
> +    if (sphb->msi) {
> +        g_hash_table_unref(sphb->msi);
> +        sphb->msi = NULL;
> +    }
> +
> +    /*
> +     * Remove IO/MMIO subregions and aliases, rest should get cleaned
> +     * via PHB's unrealize->object_finalize
> +     */
> +    for (i = windows_supported - 1; i >= 0; i--) {
> +        tcet = spapr_tce_find_by_liobn(sphb->dma_liobn[i]);
> +        if (tcet) {
> +            memory_region_del_subregion(&sphb->iommu_root,
> +                                        spapr_tce_get_iommu(tcet));
> +        }
> +    }
> +
> +    QLIST_REMOVE(sphb, list);
> +
> +    memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow);
> +
> +    address_space_destroy(&sphb->iommu_as);
> +
> +    qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort);
> +    pci_unregister_root_bus(phb->bus);
> +
> +    memory_region_del_subregion(get_system_memory(), &sphb->iowindow);
> +    if (sphb->mem64_win_pciaddr != (hwaddr)-1) {
> +        memory_region_del_subregion(get_system_memory(), &sphb->mem64window);
> +    }
> +    memory_region_del_subregion(get_system_memory(), &sphb->mem32window);
> +}
> +
>  static uint32_t spapr_phb_index_to_lsi(int phb_index, int irq_index)
>  {
>      return SPAPR_IRQ_PCI_LSI + phb_index * PCI_NUM_PINS + irq_index;
> @@ -1592,8 +1642,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>      PCIBus *bus;
>      uint64_t msi_window_size = 4096;
>      sPAPRTCETable *tcet;
> -    const unsigned windows_supported =
> -        sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1;
> +    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
>  
>      if (!spapr) {
>          error_setg(errp, TYPE_SPAPR_PCI_HOST_BRIDGE " needs a pseries machine");
> @@ -1750,7 +1799,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>              if (local_err) {
>                  error_propagate_prepend(errp, local_err,
>                                          "can't allocate LSIs: ");
> -                return;
> +                goto unrealize;
>              }
>              spapr_irq_set_type(spapr, irq, true);
>          }
> @@ -1778,13 +1827,17 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>          if (!tcet) {
>              error_setg(errp, "Creating window#%d failed for %s",
>                         i, sphb->dtbusname);
> -            return;
> +            goto unrealize;
>          }
>          memory_region_add_subregion(&sphb->iommu_root, 0,
>                                      spapr_tce_get_iommu(tcet));
>      }
>  
>      sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free);
> +    return;
> +
> +unrealize:
> +    spapr_phb_unrealize(dev, NULL);
>  }
>  
>  static int spapr_phb_children_reset(Object *child, void *opaque)
> @@ -1983,6 +2036,7 @@ static void spapr_phb_class_init(ObjectClass *klass, void *data)
>  
>      hc->root_bus_path = spapr_phb_root_bus_path;
>      dc->realize = spapr_phb_realize;
> +    dc->unrealize = spapr_phb_unrealize;
>      dc->props = spapr_phb_properties;
>      dc->reset = spapr_phb_reset;
>      dc->vmsd = &vmstate_spapr_pci;
> @@ -1998,6 +2052,7 @@ static const TypeInfo spapr_phb_info = {
>      .name          = TYPE_SPAPR_PCI_HOST_BRIDGE,
>      .parent        = TYPE_PCI_HOST_BRIDGE,
>      .instance_size = sizeof(sPAPRPHBState),
> +    .instance_finalize = spapr_phb_finalizefn,
>      .class_init    = spapr_phb_class_init,
>      .interfaces    = (InterfaceInfo[]) {
>          { TYPE_HOTPLUG_HANDLER },
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index bb0ae7fdd41d..2ab75e28dbb5 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -166,4 +166,9 @@ void spapr_phb_dma_reset(sPAPRPHBState *sphb);
>  
>  void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr);
>  
> +static inline unsigned spapr_phb_windows_supported(sPAPRPHBState *sphb)
> +{
> +    return sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1;
> +}
> +
>  #endif /* PCI_HOST_SPAPR_H */
> 

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[Cédric Le Goater]

On 1/17/19 6:14 PM, Greg Kurz wrote:
> The current code assumes that an interrupt is allocated as soon as its
> type is set to MSI or LSI. PHB hotplug will require to be able to set
> the type of an interrupt before actually allocating it.
> 
> Disintricate type setting from allocation by using another flag bit for
> the latter. Introduce a new ics_claim_irq() function for allocation. The
> behavior of aborting if the same irq gets allocated twice is kept.
> 
> ics_set_irq_type() now only sets the type to MSI or LSI. It doesn't bring
> anything to abort if the type was already set before. Drop the assert
> and XICS_FLAGS_IRQ_MASK on the way.
> 
> Older QEMUs don't know about XICS_FLAGS_IRQ_CLAIMED. In order to safely
> handle incoming migration, we must fix the irq flags. This is done at
> post load thanks to a compat property. We don't need to do anything for
> backward migration since older machines only care for the irq type.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>

 
Reviewed-by: Cédric Le Goater <clg@kaod.org>

In case of a respin, may be transform macro ICS_IRQ_FREE() in a static 
inline helper.

Thanks,

C.

> ---
>  hw/core/machine.c     |    4 ++++
>  hw/intc/xics.c        |   28 +++++++++++++++++++++++++---
>  hw/ppc/pnv_psi.c      |    1 +
>  hw/ppc/spapr_irq.c    |    4 +---
>  include/hw/ppc/xics.h |    8 ++++++--
>  5 files changed, 37 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 1a0a9ab1117a..536a34092367 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -25,6 +25,10 @@
>  
>  GlobalProperty hw_compat_3_1[] = {
>      {
> +        .driver   = "ics-base",
> +        .property = "has-claimed-flag",
> +        .value    = "off",
> +    },{
>          .driver   = "pcie-root-port",
>          .property = "x-speed",
>          .value    = "2_5",
> diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> index 16e8ffa2aaf7..82cf04548907 100644
> --- a/hw/intc/xics.c
> +++ b/hw/intc/xics.c
> @@ -75,7 +75,7 @@ void ics_pic_print_info(ICSState *ics, Monitor *mon)
>      for (i = 0; i < ics->nr_irqs; i++) {
>          ICSIRQState *irq = ics->irqs + i;
>  
> -        if (!(irq->flags & XICS_FLAGS_IRQ_MASK)) {
> +        if (!(irq->flags & XICS_FLAGS_IRQ_CLAIMED)) {
>              continue;
>          }
>          monitor_printf(mon, "  %4x %s %02x %02x\n",
> @@ -662,6 +662,22 @@ static int ics_base_dispatch_post_load(void *opaque, int version_id)
>      ICSState *ics = opaque;
>      ICSStateClass *info = ICS_BASE_GET_CLASS(ics);
>  
> +    if (!ics->has_claimed_flag) {
> +        int i;
> +
> +        for (i = 0; i < ics->nr_irqs; i++) {
> +            ICSIRQState *irq = ics->irqs + i;
> +
> +            /*
> +             * For older machines, allocating the irq and setting its type is
> +             * the same thing.
> +             */
> +            if (irq->flags & (XICS_FLAGS_IRQ_LSI| XICS_FLAGS_IRQ_MSI)) {
> +                irq->flags |= XICS_FLAGS_IRQ_CLAIMED;
> +            }
> +        }
> +    }
> +
>      if (info->post_load) {
>          return info->post_load(ics, version_id);
>      }
> @@ -702,6 +718,7 @@ static const VMStateDescription vmstate_ics_base = {
>  
>  static Property ics_base_properties[] = {
>      DEFINE_PROP_UINT32("nr-irqs", ICSState, nr_irqs, 0),
> +    DEFINE_PROP_BOOL("has-claimed-flag", ICSState, has_claimed_flag, true),
>      DEFINE_PROP_END_OF_LIST(),
>  };
>  
> @@ -741,10 +758,15 @@ ICPState *xics_icp_get(XICSFabric *xi, int server)
>      return xic->icp_get(xi, server);
>  }
>  
> -void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> +void ics_claim_irq(ICSState *ics, int srcno)
>  {
> -    assert(!(ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MASK));
> +    assert(ICS_IRQ_FREE(ics, srcno));
> +
> +    ics->irqs[srcno].flags |= XICS_FLAGS_IRQ_CLAIMED;
> +}
>  
> +void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> +{
>      ics->irqs[srcno].flags |=
>          lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI;
>  }
> diff --git a/hw/ppc/pnv_psi.c b/hw/ppc/pnv_psi.c
> index 8ced09506321..ced34e1119dc 100644
> --- a/hw/ppc/pnv_psi.c
> +++ b/hw/ppc/pnv_psi.c
> @@ -488,6 +488,7 @@ static void pnv_psi_realize(DeviceState *dev, Error **errp)
>  
>      for (i = 0; i < ics->nr_irqs; i++) {
>          ics_set_irq_type(ics, i, true);
> +        ics_claim_irq(ics, i);
>      }
>  
>      psi->qirqs = qemu_allocate_irqs(ics_simple_set_irq, ics, ics->nr_irqs);
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index 1da7a32348fc..86c712d15382 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -125,9 +125,6 @@ error:
>      error_propagate(errp, local_err);
>  }
>  
> -#define ICS_IRQ_FREE(ics, srcno)   \
> -    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_MASK)))
> -
>  static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
>                                  Error **errp)
>  {
> @@ -146,6 +143,7 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
>      }
>  
>      ics_set_irq_type(ics, irq - ics->offset, lsi);
> +    ics_claim_irq(ics, irq - ics->offset);
>      return 0;
>  }
>  
> diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
> index fad786e8b22d..e3be5cc663c5 100644
> --- a/include/hw/ppc/xics.h
> +++ b/include/hw/ppc/xics.h
> @@ -131,6 +131,7 @@ struct ICSState {
>      /*< public >*/
>      uint32_t nr_irqs;
>      uint32_t offset;
> +    bool has_claimed_flag;
>      ICSIRQState *irqs;
>      XICSFabric *xics;
>  };
> @@ -153,13 +154,15 @@ struct ICSIRQState {
>  #define XICS_STATUS_PRESENTED          0x10
>  #define XICS_STATUS_QUEUED             0x20
>      uint8_t status;
> -/* (flags & XICS_FLAGS_IRQ_MASK) == 0 means the interrupt is not allocated */
>  #define XICS_FLAGS_IRQ_LSI             0x1
>  #define XICS_FLAGS_IRQ_MSI             0x2
> -#define XICS_FLAGS_IRQ_MASK            0x3
> +#define XICS_FLAGS_IRQ_CLAIMED         0x4
>      uint8_t flags;
>  };
>  
> +#define ICS_IRQ_FREE(ics, srcno)   \
> +    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_CLAIMED)))
> +
>  struct XICSFabric {
>      Object parent;
>  };
> @@ -193,6 +196,7 @@ void ics_simple_write_xive(ICSState *ics, int nr, int server,
>  void ics_simple_set_irq(void *opaque, int srcno, int val);
>  void ics_kvm_set_irq(void *opaque, int srcno, int val);
>  
> +void ics_claim_irq(ICSState *ics, int srcno);
>  void ics_set_irq_type(ICSState *ics, int srcno, bool lsi);
>  void icp_pic_print_info(ICPState *icp, Monitor *mon);
>  void ics_pic_print_info(ICSState *ics, Monitor *mon);
> 

Re: [Qemu-devel] [PATCH v3 04/19] spapr/xive: Don't set irq type in spapr_xive_irq_claim()

[Cédric Le Goater]

On 1/17/19 6:14 PM, Greg Kurz wrote:
> PHB hotplug will need to set the type of all LSIs at machine init.
> Prepare for that by calling xive_source_irq_set() in the callers
> of spapr_xive_irq_claim().
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.

> ---
>  hw/intc/spapr_xive.c        |    5 +----
>  hw/ppc/spapr_irq.c          |    6 ++++--
>  include/hw/ppc/spapr_xive.h |    2 +-
>  3 files changed, 6 insertions(+), 7 deletions(-)
> 
> diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> index d391177ab81f..aab63cfd1178 100644
> --- a/hw/intc/spapr_xive.c
> +++ b/hw/intc/spapr_xive.c
> @@ -471,16 +471,13 @@ static void spapr_xive_register_types(void)
>  
>  type_init(spapr_xive_register_types)
>  
> -bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi)
> +bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn)
>  {
> -    XiveSource *xsrc = &xive->source;
> -
>      if (lisn >= xive->nr_irqs) {
>          return false;
>      }
>  
>      xive->eat[lisn].w |= cpu_to_be64(EAS_VALID);
> -    xive_source_irq_set(xsrc, lisn, lsi);
>      return true;
>  }
>  
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index 86c712d15382..bcd816c5a5fb 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -285,7 +285,8 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
>  
>      /* Enable the CPU IPIs */
>      for (i = 0; i < nr_servers; ++i) {
> -        spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i, false);
> +        spapr_xive_irq_claim(spapr->xive, SPAPR_IRQ_IPI + i);
> +        xive_source_irq_set(&spapr->xive->source, SPAPR_IRQ_IPI + i, false);
>      }
>  
>      spapr_xive_hcall_init(spapr);
> @@ -294,10 +295,11 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
>  static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool lsi,
>                                  Error **errp)
>  {
> -    if (!spapr_xive_irq_claim(spapr->xive, irq, lsi)) {
> +    if (!spapr_xive_irq_claim(spapr->xive, irq)) {
>          error_setg(errp, "IRQ %d is invalid", irq);
>          return -1;
>      }
> +    xive_source_irq_set(&spapr->xive->source, irq, lsi);
>      return 0;
>  }
>  
> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> index 7fdc25057420..f8854a4a6a18 100644
> --- a/include/hw/ppc/spapr_xive.h
> +++ b/include/hw/ppc/spapr_xive.h
> @@ -37,7 +37,7 @@ typedef struct sPAPRXive {
>      MemoryRegion  tm_mmio;
>  } sPAPRXive;
>  
> -bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn, bool lsi);
> +bool spapr_xive_irq_claim(sPAPRXive *xive, uint32_t lisn);
>  bool spapr_xive_irq_free(sPAPRXive *xive, uint32_t lisn);
>  void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon);
>  
> 

Re: [Qemu-devel] [PATCH v3 05/19] spapr: Set irq type in a dedicated function

[Cédric Le Goater]

On 1/17/19 6:14 PM, Greg Kurz wrote:
> PHB hotplug will need to set the type of all LSIs at machine init.
> 
> Prepare for that by moving the behaviour of setting the type from
> spapr_irq_claim() to a separate spapr_irq_set_type() function. Have
> all the callers of spapr_irq_claim() to also call the new function.
> 
> There's no such need of separating allocation and type setting for
> MSIs, so this introduces a convenient spapr_irq_claim_msi() function
> for MSI users.

I would make LSI the special case instead.

C.

 
> Wire all this to the existing backends.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr_events.c      |    4 +--
>  hw/ppc/spapr_irq.c         |   59 ++++++++++++++++++++++++++++++++++++--------
>  hw/ppc/spapr_pci.c         |    5 ++--
>  hw/ppc/spapr_vio.c         |    2 +
>  include/hw/ppc/spapr_irq.h |    7 ++++-
>  5 files changed, 59 insertions(+), 18 deletions(-)
> 
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index b9c7ecb9e987..777a70842acc 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -713,7 +713,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
>          epow_irq = spapr_irq_findone(spapr, &error_fatal);
>      }
>  
> -    spapr_irq_claim(spapr, epow_irq, false, &error_fatal);
> +    spapr_irq_claim(spapr, epow_irq, &error_fatal);
>  
>      QTAILQ_INIT(&spapr->pending_events);
>  
> @@ -737,7 +737,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
>              hp_irq = spapr_irq_findone(spapr, &error_fatal);
>          }
>  
> -        spapr_irq_claim(spapr, hp_irq, false, &error_fatal);
> +        spapr_irq_claim_msi(spapr, hp_irq, &error_fatal);
>  
>          spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_HOT_PLUG,
>                                       hp_irq);
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index bcd816c5a5fb..396dd09bdbe0 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -125,8 +125,7 @@ error:
>      error_propagate(errp, local_err);
>  }
>  
> -static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
> -                                Error **errp)
> +static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, Error **errp)
>  {
>      ICSState *ics = spapr->ics;
>  
> @@ -142,11 +141,20 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
>          return -1;
>      }
>  
> -    ics_set_irq_type(ics, irq - ics->offset, lsi);
>      ics_claim_irq(ics, irq - ics->offset);
>      return 0;
>  }
>  
> +static void spapr_irq_set_type_xics(sPAPRMachineState *spapr, int irq, bool lsi)
> +{
> +    ICSState *ics = spapr->ics;
> +
> +    assert(ics);
> +    assert(ics_valid_irq(ics, irq));
> +
> +    ics_set_irq_type(ics, irq - ics->offset, lsi);
> +}
> +
>  static void spapr_irq_free_xics(sPAPRMachineState *spapr, int irq, int num)
>  {
>      ICSState *ics = spapr->ics;
> @@ -245,6 +253,7 @@ sPAPRIrq spapr_irq_xics = {
>  
>      .init        = spapr_irq_init_xics,
>      .claim       = spapr_irq_claim_xics,
> +    .set_type    = spapr_irq_set_type_xics,
>      .free        = spapr_irq_free_xics,
>      .qirq        = spapr_qirq_xics,
>      .print_info  = spapr_irq_print_info_xics,
> @@ -292,17 +301,20 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
>      spapr_xive_hcall_init(spapr);
>  }
>  
> -static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool lsi,
> -                                Error **errp)
> +static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, Error **errp)
>  {
>      if (!spapr_xive_irq_claim(spapr->xive, irq)) {
>          error_setg(errp, "IRQ %d is invalid", irq);
>          return -1;
>      }
> -    xive_source_irq_set(&spapr->xive->source, irq, lsi);
>      return 0;
>  }
>  
> +static void spapr_irq_set_type_xive(sPAPRMachineState *spapr, int irq, bool lsi)
> +{
> +    xive_source_irq_set(&spapr->xive->source, irq, lsi);
> +}
> +
>  static void spapr_irq_free_xive(sPAPRMachineState *spapr, int irq, int num)
>  {
>      int i;
> @@ -403,6 +415,7 @@ sPAPRIrq spapr_irq_xive = {
>  
>      .init        = spapr_irq_init_xive,
>      .claim       = spapr_irq_claim_xive,
> +    .set_type    = spapr_irq_set_type_xive,
>      .free        = spapr_irq_free_xive,
>      .qirq        = spapr_qirq_xive,
>      .print_info  = spapr_irq_print_info_xive,
> @@ -466,19 +479,19 @@ static void spapr_irq_init_dual(sPAPRMachineState *spapr, Error **errp)
>      }
>  }
>  
> -static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
> +static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq,
>                                  Error **errp)
>  {
>      Error *local_err = NULL;
>      int ret;
>  
> -    ret = spapr_irq_xics.claim(spapr, irq, lsi, &local_err);
> +    ret = spapr_irq_xics.claim(spapr, irq, &local_err);
>      if (local_err) {
>          error_propagate(errp, local_err);
>          return ret;
>      }
>  
> -    ret = spapr_irq_xive.claim(spapr, irq, lsi, &local_err);
> +    ret = spapr_irq_xive.claim(spapr, irq, &local_err);
>      if (local_err) {
>          error_propagate(errp, local_err);
>          return ret;
> @@ -487,6 +500,12 @@ static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
>      return ret;
>  }
>  
> +static void spapr_irq_set_type_dual(sPAPRMachineState *spapr, int irq, bool lsi)
> +{
> +    spapr_irq_xics.set_type(spapr, irq, lsi);
> +    spapr_irq_xive.set_type(spapr, irq, lsi);
> +}
> +
>  static void spapr_irq_free_dual(sPAPRMachineState *spapr, int irq, int num)
>  {
>      spapr_irq_xics.free(spapr, irq, num);
> @@ -582,6 +601,7 @@ sPAPRIrq spapr_irq_dual = {
>  
>      .init        = spapr_irq_init_dual,
>      .claim       = spapr_irq_claim_dual,
> +    .set_type    = spapr_irq_set_type_dual,
>      .free        = spapr_irq_free_dual,
>      .qirq        = spapr_qirq_dual,
>      .print_info  = spapr_irq_print_info_dual,
> @@ -608,9 +628,25 @@ void spapr_irq_init(sPAPRMachineState *spapr, Error **errp)
>                                        spapr->irq->nr_irqs);
>  }
>  
> -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp)
> +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp)
> +{
> +    return spapr->irq->claim(spapr, irq, errp);
> +}
> +
> +void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi)
>  {
> -    return spapr->irq->claim(spapr, irq, lsi, errp);
> +    return spapr->irq->set_type(spapr, irq, lsi);
> +}
> +
> +int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp)
> +{
> +    int ret = spapr_irq_claim(spapr, irq, errp);
> +
> +    if (!ret) {
> +        spapr_irq_set_type(spapr, irq, false);
> +    }
> +
> +    return ret;
>  }
>  
>  void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num)
> @@ -699,6 +735,7 @@ sPAPRIrq spapr_irq_xics_legacy = {
>  
>      .init        = spapr_irq_init_xics,
>      .claim       = spapr_irq_claim_xics,
> +    .set_type    = spapr_irq_set_type_xics,
>      .free        = spapr_irq_free_xics,
>      .qirq        = spapr_qirq_xics,
>      .print_info  = spapr_irq_print_info_xics,
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index ccdaf2c9a606..ec9d4d28004c 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -391,7 +391,7 @@ static void rtas_ibm_change_msi(PowerPCCPU *cpu, sPAPRMachineState *spapr,
>      }
>  
>      for (i = 0; i < req_num; i++) {
> -        spapr_irq_claim(spapr, irq + i, false, &err);
> +        spapr_irq_claim_msi(spapr, irq + i, &err);
>          if (err) {
>              error_reportf_err(err, "Can't allocate MSIs for device %x: ",
>                                config_addr);
> @@ -1738,11 +1738,12 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>              }
>          }
>  
> -        spapr_irq_claim(spapr, irq, true, &local_err);
> +        spapr_irq_claim(spapr, irq, &local_err);
>          if (local_err) {
>              error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
>              return;
>          }
> +        spapr_irq_set_type(spapr, irq, true);
>  
>          sphb->lsi_table[i].irq = irq;
>      }
> diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
> index f80b70a39c46..9ae4351d4414 100644
> --- a/hw/ppc/spapr_vio.c
> +++ b/hw/ppc/spapr_vio.c
> @@ -546,7 +546,7 @@ static void spapr_vio_busdev_realize(DeviceState *qdev, Error **errp)
>          }
>      }
>  
> -    spapr_irq_claim(spapr, dev->irq, false, &local_err);
> +    spapr_irq_claim_msi(spapr, dev->irq, &local_err);
>      if (local_err) {
>          error_propagate(errp, local_err);
>          return;
> diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> index 14b02c3aca33..42b325eb51ba 100644
> --- a/include/hw/ppc/spapr_irq.h
> +++ b/include/hw/ppc/spapr_irq.h
> @@ -36,7 +36,8 @@ typedef struct sPAPRIrq {
>      uint8_t     ov5;
>  
>      void (*init)(sPAPRMachineState *spapr, Error **errp);
> -    int (*claim)(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
> +    int (*claim)(sPAPRMachineState *spapr, int irq, Error **errp);
> +    void (*set_type)(sPAPRMachineState *spapr, int irq, bool lsi);
>      void (*free)(sPAPRMachineState *spapr, int irq, int num);
>      qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
>      void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
> @@ -55,7 +56,9 @@ extern sPAPRIrq spapr_irq_xive;
>  extern sPAPRIrq spapr_irq_dual;
>  
>  void spapr_irq_init(sPAPRMachineState *spapr, Error **errp);
> -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
> +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp);
> +void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi);
> +int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp);
>  void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
>  qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
>  int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
> 

Re: [Qemu-devel] [PATCH v3 06/19] spapr: Identify LSIs of all possible PHBs at machine init

[Cédric Le Goater]

On 1/17/19 6:15 PM, Greg Kurz wrote:
> Every PHB needs to claim 4 LSIs to support legacy PCI devices. This is
> currently done at PHB realize. When using in-kernel XICS (or upcoming
> in-kernel XIVE), QEMU synchronizes the state of all irqs, including
> these LSIs, later on at machine reset.
> 
> In order to support PHB hotplug, we need a way to tell KVM about the
> LSIs that doesn't require a machine reset. Since these irq numbers are
> fixed values derived from the PHB index, let's identify them all at
> machine init. Older machines that don't have fixed irq numbers cannot
> support PHB hotplug and keep the existing behavior.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr.c              |    6 ++++++
>  hw/ppc/spapr_pci.c          |   20 ++++++++++++++++++--
>  include/hw/pci-host/spapr.h |    2 ++
>  3 files changed, 26 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 26f8e55cc25e..9189b4d3a9d6 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2802,6 +2802,12 @@ static void spapr_machine_init(MachineState *machine)
>  
>      phb = spapr_create_default_phb();
>  
> +    if (!smc->legacy_irq_allocation) {
> +        for (i = 0; i < SPAPR_MAX_PHBS; i++) {
> +            spapr_phb_set_lsis(i, spapr);
> +        }
> +    }
> +

Can't we be more brutal and do the LSI setting in spapr_irq_init() 
for the whole [ SPAPR_IRQ_PCI_LSI ... SPAPR_IRQ_MSI -1 ] range ? 

C. 

>      for (i = 0; i < nb_nics; i++) {
>          NICInfo *nd = &nd_table[i];
>  
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index ec9d4d28004c..f5f13a4d4816 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1559,6 +1559,22 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
>      }
>  }
>  
> +static uint32_t spapr_phb_index_to_lsi(int phb_index, int irq_index)
> +{
> +    return SPAPR_IRQ_PCI_LSI + phb_index * PCI_NUM_PINS + irq_index;
> +}
> +
> +void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr)
> +{
> +    int i;
> +
> +    for (i = 0; i < PCI_NUM_PINS; i++) {
> +        uint32_t irq = spapr_phb_index_to_lsi(index, i);
> +
> +        spapr_irq_set_type(spapr, irq, true);
> +    }
> +}
> +
>  static void spapr_phb_realize(DeviceState *dev, Error **errp)
>  {
>      /* We don't use SPAPR_MACHINE() in order to exit gracefully if the user
> @@ -1726,7 +1742,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>  
>      /* Initialize the LSI table */
>      for (i = 0; i < PCI_NUM_PINS; i++) {
> -        uint32_t irq = SPAPR_IRQ_PCI_LSI + sphb->index * PCI_NUM_PINS + i;
> +        uint32_t irq = spapr_phb_index_to_lsi(sphb->index, i);
>          Error *local_err = NULL;
>  
>          if (smc->legacy_irq_allocation) {
> @@ -1736,6 +1752,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>                                          "can't allocate LSIs: ");
>                  return;
>              }
> +            spapr_irq_set_type(spapr, irq, true);
>          }
>  
>          spapr_irq_claim(spapr, irq, &local_err);
> @@ -1743,7 +1760,6 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>              error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
>              return;
>          }
> -        spapr_irq_set_type(spapr, irq, true);
>  
>          sphb->lsi_table[i].irq = irq;
>      }
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index e0e683c32469..bb0ae7fdd41d 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -164,4 +164,6 @@ static inline void spapr_phb_vfio_reset(DeviceState *qdev)
>  
>  void spapr_phb_dma_reset(sPAPRPHBState *sphb);
>  
> +void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr);
> +
>  #endif /* PCI_HOST_SPAPR_H */
> 

Re: [Qemu-devel] [PATCH v3 14/19] spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend

[Cédric Le Goater]

On 1/17/19 6:15 PM, Greg Kurz wrote:
> The setting of the "phandle" and "linux,phandle" properties of the
> interrupt controller node is duplicate code in XICS and XIVE. Move
> it to a new spapr_irq_dt_populate() function.
> 
> Note that for XIVE, we set the "ibm,plat-res-int-priorities" machine
> property earlier so that the offset returned by fdt_add_subnode()
> can be safely returned to the caller.

Ah yes. This is a top level XIVE property but I don't think it belongs
to the machine.
 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/intc/spapr_xive.c        |   21 +++++++++------------
>  hw/intc/xics_spapr.c        |    7 +++----
>  hw/ppc/spapr.c              |    2 +-
>  hw/ppc/spapr_irq.c          |   17 +++++++++++++----
>  include/hw/ppc/spapr_irq.h  |    6 ++++--
>  include/hw/ppc/spapr_xive.h |    3 +--
>  include/hw/ppc/xics_spapr.h |    3 +--
>  7 files changed, 32 insertions(+), 27 deletions(-)
> 
> diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> index aab63cfd1178..e542ae59d7fd 100644
> --- a/hw/intc/spapr_xive.c
> +++ b/hw/intc/spapr_xive.c
> @@ -1411,8 +1411,7 @@ void spapr_xive_hcall_init(sPAPRMachineState *spapr)
>      spapr_register_hypercall(H_INT_RESET, h_int_reset);
>  }
>  
> -void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> -                   uint32_t phandle)
> +int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)

I am sure this is the correct prototype. This one returning an error and 
the phander looks better.

   int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
                      uint32_t *phandle)

But I won't make a fuss about it either.


Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.


>  {
>      sPAPRXive *xive = spapr->xive;
>      int node;
> @@ -1439,6 +1438,13 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
>      };
>      gchar *nodename;
>  
> +    /*
> +     * The "ibm,plat-res-int-priorities" property defines the priority
> +     * ranges reserved by the hypervisor
> +     */
> +    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
> +                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
> +
>      /* Thread Interrupt Management Area : User (ring 3) and OS (ring 2) */
>      timas[0] = cpu_to_be64(xive->tm_base +
>                             XIVE_TM_USER_PAGE * (1ull << TM_SHIFT));
> @@ -1465,14 +1471,5 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
>      _FDT(fdt_setprop(fdt, node, "interrupt-controller", NULL, 0));
>      _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
>  
> -    /* For SLOF */
> -    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> -    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> -
> -    /*
> -     * The "ibm,plat-res-int-priorities" property defines the priority
> -     * ranges reserved by the hypervisor
> -     */
> -    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
> -                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
> +    return node;
>  }
> diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
> index de6cc15b6474..375cb883c86d 100644
> --- a/hw/intc/xics_spapr.c
> +++ b/hw/intc/xics_spapr.c
> @@ -245,8 +245,7 @@ void xics_spapr_init(sPAPRMachineState *spapr)
>      spapr_register_hypercall(H_IPOLL, h_ipoll);
>  }
>  
> -void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> -                   uint32_t phandle)
> +int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>  {
>      uint32_t interrupt_server_ranges_prop[] = {
>          0, cpu_to_be32(nr_servers),
> @@ -263,6 +262,6 @@ void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
>                       interrupt_server_ranges_prop,
>                       sizeof(interrupt_server_ranges_prop)));
>      _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
> -    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> -    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> +
> +    return node;
>  }
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 2c8d0c0e149c..135136b4cf5f 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1275,7 +1275,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>      _FDT(fdt_setprop_cell(fdt, 0, "#size-cells", 2));
>  
>      /* /interrupt controller */
> -    spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> +    spapr_irq_dt_populate(spapr, spapr_max_server_number(spapr), fdt,
>                            PHANDLE_INTC);
>  
>      ret = spapr_populate_memory(spapr, fdt);
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index 396dd09bdbe0..b3e0713428ed 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -11,6 +11,7 @@
>  #include "qemu/log.h"
>  #include "qemu/error-report.h"
>  #include "qapi/error.h"
> +#include "hw/ppc/fdt.h"
>  #include "hw/ppc/spapr.h"
>  #include "hw/ppc/spapr_xive.h"
>  #include "hw/ppc/xics.h"
> @@ -536,11 +537,10 @@ static void spapr_irq_print_info_dual(sPAPRMachineState *spapr, Monitor *mon)
>      spapr_irq_current(spapr)->print_info(spapr, mon);
>  }
>  
> -static void spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
> -                                       uint32_t nr_servers, void *fdt,
> -                                       uint32_t phandle)
> +static int spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
> +                                      uint32_t nr_servers, void *fdt)
>  {
> -    spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt, phandle);
> +    return spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt);
>  }
>  
>  static void spapr_irq_cpu_intc_create_dual(sPAPRMachineState *spapr,
> @@ -671,6 +671,15 @@ void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp)
>      }
>  }
>  
> +void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> +                           void *fdt, uint32_t phandle)
> +{
> +    int node = spapr->irq->dt_populate(spapr, nr_servers, fdt);
> +
> +    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> +    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> +}
> +
>  /*
>   * XICS legacy routines - to deprecate one day
>   */
> diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> index 42b325eb51ba..dfed6b7e9976 100644
> --- a/include/hw/ppc/spapr_irq.h
> +++ b/include/hw/ppc/spapr_irq.h
> @@ -41,8 +41,8 @@ typedef struct sPAPRIrq {
>      void (*free)(sPAPRMachineState *spapr, int irq, int num);
>      qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
>      void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
> -    void (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
> -                        void *fdt, uint32_t phandle);
> +    int (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
> +                       void *fdt);
>      void (*cpu_intc_create)(sPAPRMachineState *spapr, PowerPCCPU *cpu,
>                              Error **errp);
>      int (*post_load)(sPAPRMachineState *spapr, int version_id);
> @@ -63,6 +63,8 @@ void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
>  qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
>  int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
>  void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
> +void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> +                           void *fdt, uint32_t phandle);
>  
>  /*
>   * XICS legacy routines
> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> index f8854a4a6a18..29dafead9ba0 100644
> --- a/include/hw/ppc/spapr_xive.h
> +++ b/include/hw/ppc/spapr_xive.h
> @@ -44,8 +44,7 @@ void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon);
>  typedef struct sPAPRMachineState sPAPRMachineState;
>  
>  void spapr_xive_hcall_init(sPAPRMachineState *spapr);
> -void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> -                   uint32_t phandle);
> +int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
>  void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx);
>  void spapr_xive_mmio_set_enabled(sPAPRXive *xive, bool enable);
>  
> diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
> index b1ab27d022cf..9f51f8621f75 100644
> --- a/include/hw/ppc/xics_spapr.h
> +++ b/include/hw/ppc/xics_spapr.h
> @@ -29,8 +29,7 @@
>  
>  #include "hw/ppc/spapr.h"
>  
> -void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> -                   uint32_t phandle);
> +int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
>  int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
>  void xics_spapr_init(sPAPRMachineState *spapr);
>  
> 

Re: [Qemu-devel] [PATCH v3 15/19] spapr_xive: Cache device tree nodename in sPAPRXive

[Cédric Le Goater]

On 1/17/19 6:16 PM, Greg Kurz wrote:
> PHB hotplug will need to know the name of the XIVE controller node. Since
> it is an invariant for the machine lifetime, compute it at realize and
> store it under the sPAPRXive structure.
Could you please gather all these changes [15-17] in one patch. It would 
be easier to track. 

> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/intc/spapr_xive.c        |    9 ++++-----
>  include/hw/ppc/spapr_xive.h |    3 +++
>  2 files changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> index e542ae59d7fd..9732c3d89c77 100644
> --- a/hw/intc/spapr_xive.c
> +++ b/hw/intc/spapr_xive.c
> @@ -316,6 +316,9 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
>      /* Map all regions */
>      spapr_xive_map_mmio(xive);
>  
> +    xive->nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> +                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));

I would use a helper routine instead.
 
>      qemu_register_reset(spapr_xive_reset, dev);
>  }
>  
> @@ -1436,7 +1439,6 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>          cpu_to_be32(7),    /* start */
>          cpu_to_be32(0xf8), /* count */
>      };
> -    gchar *nodename;
>  
>      /*
>       * The "ibm,plat-res-int-priorities" property defines the priority
> @@ -1453,10 +1455,7 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>                             XIVE_TM_OS_PAGE * (1ull << TM_SHIFT));
>      timas[3] = cpu_to_be64(1ull << TM_SHIFT);
>  
> -    nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> -                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
> -    _FDT(node = fdt_add_subnode(fdt, 0, nodename));
> -    g_free(nodename);
> +    _FDT(node = fdt_add_subnode(fdt, 0, xive->nodename));

and use the helper routine here.

>      _FDT(fdt_setprop_string(fdt, node, "device_type", "power-ivpe"));
>      _FDT(fdt_setprop(fdt, node, "reg", timas, sizeof(timas)));
> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> index 29dafead9ba0..deea34b03ee5 100644
> --- a/include/hw/ppc/spapr_xive.h
> +++ b/include/hw/ppc/spapr_xive.h
> @@ -26,6 +26,9 @@ typedef struct sPAPRXive {
>      XiveENDSource end_source;
>      hwaddr        end_base;
>  
> +    /* DT */
> +    gchar *nodename;

I don't think this is needed. See other patches for why.

Thanks,

C.
 
>
>      /* Routing table */
>      XiveEAS       *eat;
>      uint32_t      nr_irqs;
> 

Re: [Qemu-devel] [PATCH v3 16/19] spapr: Expose the name of the interrupt controller node

[Cédric Le Goater]

On 1/17/19 6:16 PM, Greg Kurz wrote:
> This will be needed by PHB hotplug in order to access the "phandle"
> property of the interrupt controller node.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/intc/xics_spapr.c        |    2 +-
>  include/hw/ppc/spapr_xive.h |    5 +++++
>  include/hw/ppc/xics_spapr.h |    7 +++++++
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
> index 375cb883c86d..ec50e21b8607 100644
> --- a/hw/intc/xics_spapr.c
> +++ b/hw/intc/xics_spapr.c
> @@ -252,7 +252,7 @@ int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>      };
>      int node;
>  
> -    _FDT(node = fdt_add_subnode(fdt, 0, "interrupt-controller"));
> +    _FDT(node = fdt_add_subnode(fdt, 0, XICS_NODENAME));
>  
>      _FDT(fdt_setprop_string(fdt, node, "device_type",
>                              "PowerPC-External-Interrupt-Presentation"));
> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> index deea34b03ee5..e769c2f9a018 100644
> --- a/include/hw/ppc/spapr_xive.h
> +++ b/include/hw/ppc/spapr_xive.h
> @@ -51,4 +51,9 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
>  void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx);
>  void spapr_xive_mmio_set_enabled(sPAPRXive *xive, bool enable);
>  
> +static inline const char *spapr_xive_get_nodename(sPAPRXive *xive)
> +{
> +    return xive->nodename;
> +}

There I would compute the nodename here in a static char * and return it. 

>  #endif /* PPC_SPAPR_XIVE_H */
> diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
> index 9f51f8621f75..fd66cf7bbba3 100644
> --- a/include/hw/ppc/xics_spapr.h
> +++ b/include/hw/ppc/xics_spapr.h
> @@ -29,6 +29,13 @@
>  
>  #include "hw/ppc/spapr.h"
>  
> +#define XICS_NODENAME "interrupt-controller"
> +
> +static inline const char *xics_spapr_get_nodename(void)
> +{
> +    return XICS_NODENAME;
> +}

I don't see much point in that helper. 

>  int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
>  int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
>  void xics_spapr_init(sPAPRMachineState *spapr);
> 

Re: [Qemu-devel] [PATCH v3 17/19] spapr_irq: Expose the phandle of the interrupt controller

[Cédric Le Goater]

On 1/17/19 6:16 PM, Greg Kurz wrote:
> This will be used by PHB hotplug in order to create the "interrupt-map"
> property of the PHB node.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
>  hw/ppc/spapr_irq.c         |   44 +++++++++++++++++++++++++++++++++++++++++++-
>  include/hw/ppc/spapr_irq.h |    3 +++
>  2 files changed, 46 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> index b3e0713428ed..93243cfd0644 100644
> --- a/hw/ppc/spapr_irq.c
> +++ b/hw/ppc/spapr_irq.c
> @@ -243,6 +243,11 @@ static void spapr_irq_reset_xics(sPAPRMachineState *spapr, Error **errp)
>      /* TODO: create the KVM XICS device */
>  }
>  
> +static const char *spapr_irq_get_nodename_xics(sPAPRMachineState *spapr)
> +{
> +    return xics_spapr_get_nodename();

can't we just return XICS_NODENAME ?

> +}
> +
>  #define SPAPR_IRQ_XICS_NR_IRQS     0x1000
>  #define SPAPR_IRQ_XICS_NR_MSIS     \
>      (XICS_IRQ_BASE + SPAPR_IRQ_XICS_NR_IRQS - SPAPR_IRQ_MSI)
> @@ -263,6 +268,7 @@ sPAPRIrq spapr_irq_xics = {
>      .post_load   = spapr_irq_post_load_xics,
>      .reset       = spapr_irq_reset_xics,
>      .set_irq     = spapr_irq_set_irq_xics,
> +    .get_nodename = spapr_irq_get_nodename_xics,
>  };
>  
>  /*
> @@ -401,6 +407,11 @@ static void spapr_irq_set_irq_xive(void *opaque, int srcno, int val)
>      xive_source_set_irq(&spapr->xive->source, srcno, val);
>  }
>  
> +static const char *spapr_irq_get_nodename_xive(sPAPRMachineState *spapr)
> +{
> +    return spapr_xive_get_nodename(spapr->xive);
> +}
> +
>  /*
>   * XIVE uses the full IRQ number space. Set it to 8K to be compatible
>   * with XICS.
> @@ -425,6 +436,7 @@ sPAPRIrq spapr_irq_xive = {
>      .post_load   = spapr_irq_post_load_xive,
>      .reset       = spapr_irq_reset_xive,
>      .set_irq     = spapr_irq_set_irq_xive,
> +    .get_nodename = spapr_irq_get_nodename_xive,
>  };
>  
>  /*
> @@ -588,6 +600,11 @@ static void spapr_irq_set_irq_dual(void *opaque, int srcno, int val)
>      spapr_irq_current(spapr)->set_irq(spapr, srcno, val);
>  }
>  
> +static const char *spapr_irq_get_nodename_dual(sPAPRMachineState *spapr)
> +{
> +    return spapr_irq_current(spapr)->get_nodename(spapr);
> +}
> +
>  /*
>   * Define values in sync with the XIVE and XICS backend
>   */
> @@ -609,7 +626,8 @@ sPAPRIrq spapr_irq_dual = {
>      .cpu_intc_create = spapr_irq_cpu_intc_create_dual,
>      .post_load   = spapr_irq_post_load_dual,
>      .reset       = spapr_irq_reset_dual,
> -    .set_irq     = spapr_irq_set_irq_dual
> +    .set_irq     = spapr_irq_set_irq_dual,
> +    .get_nodename = spapr_irq_get_nodename_dual,
>  };
>  
>  /*
> @@ -680,6 +698,29 @@ void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
>      _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
>  }
>  
> +uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
> +                               Error **errp)
> +{
> +    const char *nodename = spapr->irq->get_nodename(spapr);
> +    int phandle, offset;
> +
> +    offset = fdt_subnode_offset(fdt, 0, nodename);

if I am correct, you changed dt_populate() to return the same offset.

> +    if (offset < 0) {
> +        error_setg(errp, "Can't find node \"%s\": %s", nodename,
> +                   fdt_strerror(offset));
> +        return -1;
> +    }
> +
> +    phandle = fdt_get_phandle(fdt, offset);

if we stored the phandle when spapr_irq_dt_populate() is called, I think
things would be easier. May be under the backend ? 

C.


> +    if (phandle < 0) {
> +        error_setg(errp, "Can't get phandle of node \"%s\": %s",
> +                   nodename, fdt_strerror(phandle));
> +        return -1;
> +    }
> +
> +    return phandle;
> +}
> +
>  /*
>   * XICS legacy routines - to deprecate one day
>   */
> @@ -752,4 +793,5 @@ sPAPRIrq spapr_irq_xics_legacy = {
>      .cpu_intc_create = spapr_irq_cpu_intc_create_xics,
>      .post_load   = spapr_irq_post_load_xics,
>      .set_irq     = spapr_irq_set_irq_xics,
> +    .get_nodename = spapr_irq_get_nodename_xics,
>  };
> diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> index dfed6b7e9976..d44d659231a6 100644
> --- a/include/hw/ppc/spapr_irq.h
> +++ b/include/hw/ppc/spapr_irq.h
> @@ -48,6 +48,7 @@ typedef struct sPAPRIrq {
>      int (*post_load)(sPAPRMachineState *spapr, int version_id);
>      void (*reset)(sPAPRMachineState *spapr, Error **errp);
>      void (*set_irq)(void *opaque, int srcno, int val);
> +    const char *(*get_nodename)(sPAPRMachineState *spapr);
>  } sPAPRIrq;
>  
>  extern sPAPRIrq spapr_irq_xics;
> @@ -65,6 +66,8 @@ int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
>  void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
>  void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
>                             void *fdt, uint32_t phandle);
> +uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
> +                               Error **errp);
>  
>  /*
>   * XICS legacy routines
> 

Re: [Qemu-devel] [PATCH v3 18/19] spapr: add hotplug hooks for PHB hotplug

[Cédric Le Goater]

On 1/17/19 6:16 PM, Greg Kurz wrote:
> Hotplugging PHBs is a machine-level operation, but PHBs reside on the
> main system bus, so we register spapr machine as the handler for the
> main system bus.
> 
> We re-get the phandle of the interrupt controller systematically for
> simplicity.
> 
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Greg Kurz <groug@kaod.org>
> ---
> v3: - reworked phandle handling some more
> v2: - reworked phandle handling
>     - sync LSIs to KVM
> ---
>  hw/ppc/spapr.c         |  124 ++++++++++++++++++++++++++++++++++++++++++++++++
>  hw/ppc/spapr_drc.c     |    1 
>  hw/ppc/spapr_pci.c     |   16 ------
>  include/hw/ppc/spapr.h |    1 
>  4 files changed, 126 insertions(+), 16 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 135136b4cf5f..d94284f423be 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2936,6 +2936,11 @@ static void spapr_machine_init(MachineState *machine)
>      register_savevm_live(NULL, "spapr/htab", -1, 1,
>                           &savevm_htab_handlers, spapr);
>  
> +    if (smc->dr_phb_enabled) {
> +        qbus_set_hotplug_handler(sysbus_get_default(), OBJECT(machine),
> +                                 &error_fatal);
> +    }
> +
>      qemu_register_boot_set(spapr_boot_set, spapr);
>  
>      if (kvm_enabled()) {
> @@ -3733,6 +3738,111 @@ out:
>      error_propagate(errp, local_err);
>  }
>  
> +static void spapr_phb_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                               Error **errp)
> +{
> +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
> +    const unsigned windows_supported = spapr_phb_windows_supported(sphb);
> +
> +    if (sphb->index == (uint32_t)-1) {
> +        error_setg(errp, "\"index\" for PAPR PHB is mandatory");
> +        return;
> +    }
> +
> +    /*
> +     * This will check that sphb->index doesn't exceed the maximum number of
> +     * PHBs for the current machine type.
> +     */
> +    smc->phb_placement(spapr, sphb->index,
> +                       &sphb->buid, &sphb->io_win_addr,
> +                       &sphb->mem_win_addr, &sphb->mem64_win_addr,
> +                       windows_supported, sphb->dma_liobn, errp);
> +}
> +
> +static void spapr_phb_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
> +                           Error **errp)
> +{
> +    sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +    void *fdt = NULL;
> +    int fdt_start_offset;
> +    int fdt_size;
> +    Error *local_err = NULL;
> +    sPAPRDRConnector *drc;
> +    int ret;
> +    bool hotplugged = spapr_drc_hotplugged(dev);
> +    int phandle;
> +
> +    if (!smc->dr_phb_enabled) {
> +        return;
> +    }
> +
> +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->index);
> +    /* hotplug hooks should check it's enabled before getting this far */
> +    assert(drc);
> +
> +    if (hotplugged) {
> +        phandle = spapr_irq_get_phandle(spapr, spapr->fdt_blob, &local_err);
> +        if (local_err) {
> +            goto out;
> +        }
> +    } else {
> +        phandle = PHANDLE_INTC;
> +    }
>

I am starting to think we should have a spapr->irq->phandle.

C.

> +    fdt = create_device_tree(&fdt_size);
> +    ret = spapr_populate_pci_dt(sphb, phandle, fdt, spapr->irq->nr_msis,
> +                                &fdt_start_offset);
> +    if (ret < 0) {
> +        error_setg(&local_err, "unable to create FDT for %sPHB",
> +                   dev->hotplugged ? "hotplugged " : "");
> +        goto out;
> +    }
> +
> +    if (hotplugged) {
> +        /* generally SLOF creates these, for hotplug it's up to QEMU */
> +        _FDT(fdt_setprop_string(fdt, fdt_start_offset, "name", "pci"));
> +    }
> +
> +    spapr_drc_attach(drc, DEVICE(dev), fdt, fdt_start_offset, &local_err);
> +
> +out:
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        g_free(fdt);
> +        return;
> +    }
> +
> +    if (hotplugged) {
> +        spapr_hotplug_req_add_by_index(drc);
> +    } else if (drc) {
> +        spapr_drc_reset(drc);
> +    }
> +}
> +
> +void spapr_phb_release(DeviceState *dev)
> +{
> +    object_unparent(OBJECT(dev));
> +}
> +
> +static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev,
> +                                     DeviceState *dev, Error **errp)
> +{
> +    sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(dev);
> +    sPAPRDRConnector *drc;
> +
> +    drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PHB, sphb->index);
> +    assert(drc);
> +
> +    if (!spapr_drc_unplug_requested(drc)) {
> +        spapr_drc_detach(drc);
> +        spapr_hotplug_req_remove_by_index(drc);
> +    }
> +}
> +
>  static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
>                                        DeviceState *dev, Error **errp)
>  {
> @@ -3740,6 +3850,8 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
>          spapr_memory_plug(hotplug_dev, dev, errp);
>      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
>          spapr_core_plug(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        spapr_phb_plug(hotplug_dev, dev, errp);
>      }
>  }
>  
> @@ -3758,6 +3870,7 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
>  {
>      sPAPRMachineState *sms = SPAPR_MACHINE(OBJECT(hotplug_dev));
>      MachineClass *mc = MACHINE_GET_CLASS(sms);
> +    sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
>  
>      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
>          if (spapr_ovec_test(sms->ov5_cas, OV5_HP_EVT)) {
> @@ -3777,6 +3890,12 @@ static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
>              return;
>          }
>          spapr_core_unplug_request(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        if (!smc->dr_phb_enabled) {
> +            error_setg(errp, "PHB hot unplug not supported on this machine");
> +            return;
> +        }
> +        spapr_phb_unplug_request(hotplug_dev, dev, errp);
>      }
>  }
>  
> @@ -3787,6 +3906,8 @@ static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev,
>          spapr_memory_pre_plug(hotplug_dev, dev, errp);
>      } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
>          spapr_core_pre_plug(hotplug_dev, dev, errp);
> +    } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
> +        spapr_phb_pre_plug(hotplug_dev, dev, errp);
>      }
>  }
>  
> @@ -3794,7 +3915,8 @@ static HotplugHandler *spapr_get_hotplug_handler(MachineState *machine,
>                                                   DeviceState *dev)
>  {
>      if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
> -        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
> +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE) ||
> +        object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_PCI_HOST_BRIDGE)) {
>          return HOTPLUG_HANDLER(machine);
>      }
>      return NULL;
> diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
> index 189ee681062a..7a2676716364 100644
> --- a/hw/ppc/spapr_drc.c
> +++ b/hw/ppc/spapr_drc.c
> @@ -703,6 +703,7 @@ static void spapr_drc_phb_class_init(ObjectClass *k, void *data)
>      drck->typeshift = SPAPR_DR_CONNECTOR_TYPE_SHIFT_PHB;
>      drck->typename = "PHB";
>      drck->drc_name_prefix = "PHB ";
> +    drck->release = spapr_phb_release;
>  }
>  
>  static const TypeInfo spapr_dr_connector_info = {
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index a0311560e47b..b03bfcf3f446 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -1649,21 +1649,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
>          return;
>      }
>  
> -    if (sphb->index != (uint32_t)-1) {
> -        Error *local_err = NULL;
> -
> -        smc->phb_placement(spapr, sphb->index,
> -                           &sphb->buid, &sphb->io_win_addr,
> -                           &sphb->mem_win_addr, &sphb->mem64_win_addr,
> -                           windows_supported, sphb->dma_liobn, &local_err);
> -        if (local_err) {
> -            error_propagate(errp, local_err);
> -            return;
> -        }
> -    } else {
> -        error_setg(errp, "\"index\" for PAPR PHB is mandatory");
> -        return;
> -    }
> +    assert(sphb->index != (uint32_t)-1); /* checked in spapr_phb_pre_plug() */
>  
>      if (sphb->mem64_win_size != 0) {
>          if (sphb->mem_win_size > SPAPR_PCI_MEM32_WIN_SIZE) {
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index bc8ebb072258..f78e44d58cef 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -766,6 +766,7 @@ int spapr_max_server_number(sPAPRMachineState *spapr);
>  /* CPU and LMB DRC release callbacks. */
>  void spapr_core_release(DeviceState *dev);
>  void spapr_lmb_release(DeviceState *dev);
> +void spapr_phb_release(DeviceState *dev);
>  
>  void spapr_rtc_read(sPAPRRTCState *rtc, struct tm *tm, uint32_t *ns);
>  int spapr_rtc_import_offset(sPAPRRTCState *rtc, int64_t legacy_offset);
> 

Re: [Qemu-devel] [PATCH v3 02/19] spapr: Rename xics to intc in interrupt controller agnostic code

[Cédric Le Goater]

On 1/17/19 6:14 PM, Greg Kurz wrote:
> All this code is used with both the XICS and XIVE interrupt controllers.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> Reviewed-by: Cédric Le Goater <clg@kaod.org>
> ---
> v3: - s/spapr_dt_intc_irq/spapr_dt_irq
> ---
>  hw/ppc/spapr.c              |    6 +++---
>  hw/ppc/spapr_events.c       |    2 +-
>  hw/ppc/spapr_pci.c          |    6 +++---
>  hw/ppc/spapr_vio.c          |    2 +-
>  include/hw/pci-host/spapr.h |    2 +-
>  include/hw/ppc/spapr.h      |    2 +-
>  6 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 83081defde4e..26f8e55cc25e 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -96,7 +96,7 @@
>  
>  #define MIN_RMA_SLOF            128UL
>  
> -#define PHANDLE_XICP            0x00001111
> +#define PHANDLE_INTC            0x00001111

Shouln't that define be under a spapr->irq field ? 

>  
>  /* These two functions implement the VCPU id numbering: one to compute them
>   * all and one to identify thread 0 of a VCORE. Any change to the first one
> @@ -1276,7 +1276,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>  
>      /* /interrupt controller */
>      spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> -                          PHANDLE_XICP);
> +                          PHANDLE_INTC);
>  
>      ret = spapr_populate_memory(spapr, fdt);
>      if (ret < 0) {
> @@ -1296,7 +1296,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>      }
>  
>      QLIST_FOREACH(phb, &spapr->phbs, list) {
> -        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt,
> +        ret = spapr_populate_pci_dt(phb, PHANDLE_INTC, fdt,
>                                      spapr->irq->nr_msis);

and there we could pass spapr->irq.


>          if (ret < 0) {
>              error_report("couldn't setup PCI devices in fdt");
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index 32719a1b72d0..b9c7ecb9e987 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -282,7 +282,7 @@ void spapr_dt_events(sPAPRMachineState *spapr, void *fdt)
>              continue;
>          }
>  
> -        spapr_dt_xics_irq(interrupts, source->irq, false);
> +        spapr_dt_irq(interrupts, source->irq, false);
>  
>          _FDT(node_offset = fdt_add_subnode(fdt, event_sources, source_name));
>          _FDT(fdt_setprop(fdt, node_offset, "interrupts", interrupts,
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index b74f2632ecc6..ccdaf2c9a606 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -2066,7 +2066,7 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
>  
>  }
>  
> -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
>                            uint32_t nr_msis)
>  {
>      int bus_off, i, j, ret;
> @@ -2164,8 +2164,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
>              irqmap[1] = 0;
>              irqmap[2] = 0;
>              irqmap[3] = cpu_to_be32(j+1);
> -            irqmap[4] = cpu_to_be32(xics_phandle);
> -            spapr_dt_xics_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
> +            irqmap[4] = cpu_to_be32(intc_phandle);
> +            spapr_dt_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
>          }
>      }
>      /* Write interrupt map */
> diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
> index 7e8a9ad09337..f80b70a39c46 100644
> --- a/hw/ppc/spapr_vio.c
> +++ b/hw/ppc/spapr_vio.c
> @@ -158,7 +158,7 @@ static int vio_make_devnode(VIOsPAPRDevice *dev,
>      if (dev->irq) {
>          uint32_t ints_prop[2];
>  
> -        spapr_dt_xics_irq(ints_prop, dev->irq, false);
> +        spapr_dt_irq(ints_prop, dev->irq, false);
>          ret = fdt_setprop(fdt, node_off, "interrupts", ints_prop,
>                            sizeof(ints_prop));
>          if (ret < 0) {
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 4eb3a2ce3eb8..e0e683c32469 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -113,7 +113,7 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
>      return spapr_qirq(spapr, phb->lsi_table[pin].irq);
>  }
>  
> -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
>                            uint32_t nr_msis);
>  
>  void spapr_pci_rtas_init(void);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 9e01a5a12e4a..f988dc6924b8 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -682,7 +682,7 @@ void spapr_load_rtas(sPAPRMachineState *spapr, void *fdt, hwaddr addr);
>   * "interrupt-controller" node has its "#interrupt-cells" property set to 2 (ie,
>   * VIO devices, RTAS event sources and PHBs).
>   */
> -static inline void spapr_dt_xics_irq(uint32_t *intspec, int irq, bool is_lsi)
> +static inline void spapr_dt_irq(uint32_t *intspec, int irq, bool is_lsi)
>  {
>      intspec[0] = cpu_to_be32(irq);
>      intspec[1] = is_lsi ? cpu_to_be32(1) : 0;
> 

Re: [Qemu-devel] [PATCH v3 02/19] spapr: Rename xics to intc in interrupt controller agnostic code

[Greg Kurz]

On Fri, 18 Jan 2019 14:56:51 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:14 PM, Greg Kurz wrote:
> > All this code is used with both the XICS and XIVE interrupt controllers.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > Reviewed-by: Cédric Le Goater <clg@kaod.org>
> > ---
> > v3: - s/spapr_dt_intc_irq/spapr_dt_irq
> > ---
> >  hw/ppc/spapr.c              |    6 +++---
> >  hw/ppc/spapr_events.c       |    2 +-
> >  hw/ppc/spapr_pci.c          |    6 +++---
> >  hw/ppc/spapr_vio.c          |    2 +-
> >  include/hw/pci-host/spapr.h |    2 +-
> >  include/hw/ppc/spapr.h      |    2 +-
> >  6 files changed, 10 insertions(+), 10 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 83081defde4e..26f8e55cc25e 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -96,7 +96,7 @@
> >  
> >  #define MIN_RMA_SLOF            128UL
> >  
> > -#define PHANDLE_XICP            0x00001111
> > +#define PHANDLE_INTC            0x00001111  
> 
> Shouln't that define be under a spapr->irq field ? 
> 

You're probably right but that's not in the scope of this patch
which only does a global and trivial renaming. I'll have a closer
look at your suggestion anyway since there will be a v4 :)

> >  
> >  /* These two functions implement the VCPU id numbering: one to compute them
> >   * all and one to identify thread 0 of a VCORE. Any change to the first one
> > @@ -1276,7 +1276,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
> >  
> >      /* /interrupt controller */
> >      spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> > -                          PHANDLE_XICP);
> > +                          PHANDLE_INTC);
> >  
> >      ret = spapr_populate_memory(spapr, fdt);
> >      if (ret < 0) {
> > @@ -1296,7 +1296,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
> >      }
> >  
> >      QLIST_FOREACH(phb, &spapr->phbs, list) {
> > -        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt,
> > +        ret = spapr_populate_pci_dt(phb, PHANDLE_INTC, fdt,
> >                                      spapr->irq->nr_msis);  
> 
> and there we could pass spapr->irq.
> 

Yeah.

> 
> >          if (ret < 0) {
> >              error_report("couldn't setup PCI devices in fdt");
> > diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> > index 32719a1b72d0..b9c7ecb9e987 100644
> > --- a/hw/ppc/spapr_events.c
> > +++ b/hw/ppc/spapr_events.c
> > @@ -282,7 +282,7 @@ void spapr_dt_events(sPAPRMachineState *spapr, void *fdt)
> >              continue;
> >          }
> >  
> > -        spapr_dt_xics_irq(interrupts, source->irq, false);
> > +        spapr_dt_irq(interrupts, source->irq, false);
> >  
> >          _FDT(node_offset = fdt_add_subnode(fdt, event_sources, source_name));
> >          _FDT(fdt_setprop(fdt, node_offset, "interrupts", interrupts,
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index b74f2632ecc6..ccdaf2c9a606 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -2066,7 +2066,7 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
> >  
> >  }
> >  
> > -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> > +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
> >                            uint32_t nr_msis)
> >  {
> >      int bus_off, i, j, ret;
> > @@ -2164,8 +2164,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> >              irqmap[1] = 0;
> >              irqmap[2] = 0;
> >              irqmap[3] = cpu_to_be32(j+1);
> > -            irqmap[4] = cpu_to_be32(xics_phandle);
> > -            spapr_dt_xics_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
> > +            irqmap[4] = cpu_to_be32(intc_phandle);
> > +            spapr_dt_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
> >          }
> >      }
> >      /* Write interrupt map */
> > diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
> > index 7e8a9ad09337..f80b70a39c46 100644
> > --- a/hw/ppc/spapr_vio.c
> > +++ b/hw/ppc/spapr_vio.c
> > @@ -158,7 +158,7 @@ static int vio_make_devnode(VIOsPAPRDevice *dev,
> >      if (dev->irq) {
> >          uint32_t ints_prop[2];
> >  
> > -        spapr_dt_xics_irq(ints_prop, dev->irq, false);
> > +        spapr_dt_irq(ints_prop, dev->irq, false);
> >          ret = fdt_setprop(fdt, node_off, "interrupts", ints_prop,
> >                            sizeof(ints_prop));
> >          if (ret < 0) {
> > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > index 4eb3a2ce3eb8..e0e683c32469 100644
> > --- a/include/hw/pci-host/spapr.h
> > +++ b/include/hw/pci-host/spapr.h
> > @@ -113,7 +113,7 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
> >      return spapr_qirq(spapr, phb->lsi_table[pin].irq);
> >  }
> >  
> > -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> > +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
> >                            uint32_t nr_msis);
> >  
> >  void spapr_pci_rtas_init(void);
> > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> > index 9e01a5a12e4a..f988dc6924b8 100644
> > --- a/include/hw/ppc/spapr.h
> > +++ b/include/hw/ppc/spapr.h
> > @@ -682,7 +682,7 @@ void spapr_load_rtas(sPAPRMachineState *spapr, void *fdt, hwaddr addr);
> >   * "interrupt-controller" node has its "#interrupt-cells" property set to 2 (ie,
> >   * VIO devices, RTAS event sources and PHBs).
> >   */
> > -static inline void spapr_dt_xics_irq(uint32_t *intspec, int irq, bool is_lsi)
> > +static inline void spapr_dt_irq(uint32_t *intspec, int irq, bool is_lsi)
> >  {
> >      intspec[0] = cpu_to_be32(irq);
> >      intspec[1] = is_lsi ? cpu_to_be32(1) : 0;
> >   
> 

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[Greg Kurz]

On Fri, 18 Jan 2019 13:26:29 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:14 PM, Greg Kurz wrote:
> > The current code assumes that an interrupt is allocated as soon as its
> > type is set to MSI or LSI. PHB hotplug will require to be able to set
> > the type of an interrupt before actually allocating it.
> > 
> > Disintricate type setting from allocation by using another flag bit for
> > the latter. Introduce a new ics_claim_irq() function for allocation. The
> > behavior of aborting if the same irq gets allocated twice is kept.
> > 
> > ics_set_irq_type() now only sets the type to MSI or LSI. It doesn't bring
> > anything to abort if the type was already set before. Drop the assert
> > and XICS_FLAGS_IRQ_MASK on the way.
> > 
> > Older QEMUs don't know about XICS_FLAGS_IRQ_CLAIMED. In order to safely
> > handle incoming migration, we must fix the irq flags. This is done at
> > post load thanks to a compat property. We don't need to do anything for
> > backward migration since older machines only care for the irq type.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>  
> 
>  
> Reviewed-by: Cédric Le Goater <clg@kaod.org>
> 
> In case of a respin, may be transform macro ICS_IRQ_FREE() in a static 
> inline helper.
> 

Yeah, there will be a respin (see my other mail in this thread) and I'll
just do that.

> Thanks,
> 
> C.
> 
> > ---
> >  hw/core/machine.c     |    4 ++++
> >  hw/intc/xics.c        |   28 +++++++++++++++++++++++++---
> >  hw/ppc/pnv_psi.c      |    1 +
> >  hw/ppc/spapr_irq.c    |    4 +---
> >  include/hw/ppc/xics.h |    8 ++++++--
> >  5 files changed, 37 insertions(+), 8 deletions(-)
> > 
> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index 1a0a9ab1117a..536a34092367 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -25,6 +25,10 @@
> >  
> >  GlobalProperty hw_compat_3_1[] = {
> >      {
> > +        .driver   = "ics-base",
> > +        .property = "has-claimed-flag",
> > +        .value    = "off",
> > +    },{
> >          .driver   = "pcie-root-port",
> >          .property = "x-speed",
> >          .value    = "2_5",
> > diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> > index 16e8ffa2aaf7..82cf04548907 100644
> > --- a/hw/intc/xics.c
> > +++ b/hw/intc/xics.c
> > @@ -75,7 +75,7 @@ void ics_pic_print_info(ICSState *ics, Monitor *mon)
> >      for (i = 0; i < ics->nr_irqs; i++) {
> >          ICSIRQState *irq = ics->irqs + i;
> >  
> > -        if (!(irq->flags & XICS_FLAGS_IRQ_MASK)) {
> > +        if (!(irq->flags & XICS_FLAGS_IRQ_CLAIMED)) {
> >              continue;
> >          }
> >          monitor_printf(mon, "  %4x %s %02x %02x\n",
> > @@ -662,6 +662,22 @@ static int ics_base_dispatch_post_load(void *opaque, int version_id)
> >      ICSState *ics = opaque;
> >      ICSStateClass *info = ICS_BASE_GET_CLASS(ics);
> >  
> > +    if (!ics->has_claimed_flag) {
> > +        int i;
> > +
> > +        for (i = 0; i < ics->nr_irqs; i++) {
> > +            ICSIRQState *irq = ics->irqs + i;
> > +
> > +            /*
> > +             * For older machines, allocating the irq and setting its type is
> > +             * the same thing.
> > +             */
> > +            if (irq->flags & (XICS_FLAGS_IRQ_LSI| XICS_FLAGS_IRQ_MSI)) {
> > +                irq->flags |= XICS_FLAGS_IRQ_CLAIMED;
> > +            }
> > +        }
> > +    }
> > +
> >      if (info->post_load) {
> >          return info->post_load(ics, version_id);
> >      }
> > @@ -702,6 +718,7 @@ static const VMStateDescription vmstate_ics_base = {
> >  
> >  static Property ics_base_properties[] = {
> >      DEFINE_PROP_UINT32("nr-irqs", ICSState, nr_irqs, 0),
> > +    DEFINE_PROP_BOOL("has-claimed-flag", ICSState, has_claimed_flag, true),
> >      DEFINE_PROP_END_OF_LIST(),
> >  };
> >  
> > @@ -741,10 +758,15 @@ ICPState *xics_icp_get(XICSFabric *xi, int server)
> >      return xic->icp_get(xi, server);
> >  }
> >  
> > -void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> > +void ics_claim_irq(ICSState *ics, int srcno)
> >  {
> > -    assert(!(ics->irqs[srcno].flags & XICS_FLAGS_IRQ_MASK));
> > +    assert(ICS_IRQ_FREE(ics, srcno));
> > +
> > +    ics->irqs[srcno].flags |= XICS_FLAGS_IRQ_CLAIMED;
> > +}
> >  
> > +void ics_set_irq_type(ICSState *ics, int srcno, bool lsi)
> > +{
> >      ics->irqs[srcno].flags |=
> >          lsi ? XICS_FLAGS_IRQ_LSI : XICS_FLAGS_IRQ_MSI;
> >  }
> > diff --git a/hw/ppc/pnv_psi.c b/hw/ppc/pnv_psi.c
> > index 8ced09506321..ced34e1119dc 100644
> > --- a/hw/ppc/pnv_psi.c
> > +++ b/hw/ppc/pnv_psi.c
> > @@ -488,6 +488,7 @@ static void pnv_psi_realize(DeviceState *dev, Error **errp)
> >  
> >      for (i = 0; i < ics->nr_irqs; i++) {
> >          ics_set_irq_type(ics, i, true);
> > +        ics_claim_irq(ics, i);
> >      }
> >  
> >      psi->qirqs = qemu_allocate_irqs(ics_simple_set_irq, ics, ics->nr_irqs);
> > diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> > index 1da7a32348fc..86c712d15382 100644
> > --- a/hw/ppc/spapr_irq.c
> > +++ b/hw/ppc/spapr_irq.c
> > @@ -125,9 +125,6 @@ error:
> >      error_propagate(errp, local_err);
> >  }
> >  
> > -#define ICS_IRQ_FREE(ics, srcno)   \
> > -    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_MASK)))
> > -
> >  static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
> >                                  Error **errp)
> >  {
> > @@ -146,6 +143,7 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
> >      }
> >  
> >      ics_set_irq_type(ics, irq - ics->offset, lsi);
> > +    ics_claim_irq(ics, irq - ics->offset);
> >      return 0;
> >  }
> >  
> > diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h
> > index fad786e8b22d..e3be5cc663c5 100644
> > --- a/include/hw/ppc/xics.h
> > +++ b/include/hw/ppc/xics.h
> > @@ -131,6 +131,7 @@ struct ICSState {
> >      /*< public >*/
> >      uint32_t nr_irqs;
> >      uint32_t offset;
> > +    bool has_claimed_flag;
> >      ICSIRQState *irqs;
> >      XICSFabric *xics;
> >  };
> > @@ -153,13 +154,15 @@ struct ICSIRQState {
> >  #define XICS_STATUS_PRESENTED          0x10
> >  #define XICS_STATUS_QUEUED             0x20
> >      uint8_t status;
> > -/* (flags & XICS_FLAGS_IRQ_MASK) == 0 means the interrupt is not allocated */
> >  #define XICS_FLAGS_IRQ_LSI             0x1
> >  #define XICS_FLAGS_IRQ_MSI             0x2
> > -#define XICS_FLAGS_IRQ_MASK            0x3
> > +#define XICS_FLAGS_IRQ_CLAIMED         0x4
> >      uint8_t flags;
> >  };
> >  
> > +#define ICS_IRQ_FREE(ics, srcno)   \
> > +    (!((ics)->irqs[(srcno)].flags & (XICS_FLAGS_IRQ_CLAIMED)))
> > +
> >  struct XICSFabric {
> >      Object parent;
> >  };
> > @@ -193,6 +196,7 @@ void ics_simple_write_xive(ICSState *ics, int nr, int server,
> >  void ics_simple_set_irq(void *opaque, int srcno, int val);
> >  void ics_kvm_set_irq(void *opaque, int srcno, int val);
> >  
> > +void ics_claim_irq(ICSState *ics, int srcno);
> >  void ics_set_irq_type(ICSState *ics, int srcno, bool lsi);
> >  void icp_pic_print_info(ICPState *icp, Monitor *mon);
> >  void ics_pic_print_info(ICSState *ics, Monitor *mon);
> >   
> 

Re: [Qemu-devel] [PATCH v3 05/19] spapr: Set irq type in a dedicated function

[Greg Kurz]

On Fri, 18 Jan 2019 13:34:39 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:14 PM, Greg Kurz wrote:
> > PHB hotplug will need to set the type of all LSIs at machine init.
> > 
> > Prepare for that by moving the behaviour of setting the type from
> > spapr_irq_claim() to a separate spapr_irq_set_type() function. Have
> > all the callers of spapr_irq_claim() to also call the new function.
> > 
> > There's no such need of separating allocation and type setting for
> > MSIs, so this introduces a convenient spapr_irq_claim_msi() function
> > for MSI users.  
> 
> I would make LSI the special case instead.
> 

Well... LSI is the special case in the sense it calls two distinct
functions to have the job done instead of one for MSI. :)

But anyway I'm likely to try another approach in v4.

> C.
> 
>  
> > Wire all this to the existing backends.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr_events.c      |    4 +--
> >  hw/ppc/spapr_irq.c         |   59 ++++++++++++++++++++++++++++++++++++--------
> >  hw/ppc/spapr_pci.c         |    5 ++--
> >  hw/ppc/spapr_vio.c         |    2 +
> >  include/hw/ppc/spapr_irq.h |    7 ++++-
> >  5 files changed, 59 insertions(+), 18 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> > index b9c7ecb9e987..777a70842acc 100644
> > --- a/hw/ppc/spapr_events.c
> > +++ b/hw/ppc/spapr_events.c
> > @@ -713,7 +713,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
> >          epow_irq = spapr_irq_findone(spapr, &error_fatal);
> >      }
> >  
> > -    spapr_irq_claim(spapr, epow_irq, false, &error_fatal);
> > +    spapr_irq_claim(spapr, epow_irq, &error_fatal);
> >  
> >      QTAILQ_INIT(&spapr->pending_events);
> >  
> > @@ -737,7 +737,7 @@ void spapr_events_init(sPAPRMachineState *spapr)
> >              hp_irq = spapr_irq_findone(spapr, &error_fatal);
> >          }
> >  
> > -        spapr_irq_claim(spapr, hp_irq, false, &error_fatal);
> > +        spapr_irq_claim_msi(spapr, hp_irq, &error_fatal);
> >  
> >          spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_HOT_PLUG,
> >                                       hp_irq);
> > diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> > index bcd816c5a5fb..396dd09bdbe0 100644
> > --- a/hw/ppc/spapr_irq.c
> > +++ b/hw/ppc/spapr_irq.c
> > @@ -125,8 +125,7 @@ error:
> >      error_propagate(errp, local_err);
> >  }
> >  
> > -static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
> > -                                Error **errp)
> > +static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, Error **errp)
> >  {
> >      ICSState *ics = spapr->ics;
> >  
> > @@ -142,11 +141,20 @@ static int spapr_irq_claim_xics(sPAPRMachineState *spapr, int irq, bool lsi,
> >          return -1;
> >      }
> >  
> > -    ics_set_irq_type(ics, irq - ics->offset, lsi);
> >      ics_claim_irq(ics, irq - ics->offset);
> >      return 0;
> >  }
> >  
> > +static void spapr_irq_set_type_xics(sPAPRMachineState *spapr, int irq, bool lsi)
> > +{
> > +    ICSState *ics = spapr->ics;
> > +
> > +    assert(ics);
> > +    assert(ics_valid_irq(ics, irq));
> > +
> > +    ics_set_irq_type(ics, irq - ics->offset, lsi);
> > +}
> > +
> >  static void spapr_irq_free_xics(sPAPRMachineState *spapr, int irq, int num)
> >  {
> >      ICSState *ics = spapr->ics;
> > @@ -245,6 +253,7 @@ sPAPRIrq spapr_irq_xics = {
> >  
> >      .init        = spapr_irq_init_xics,
> >      .claim       = spapr_irq_claim_xics,
> > +    .set_type    = spapr_irq_set_type_xics,
> >      .free        = spapr_irq_free_xics,
> >      .qirq        = spapr_qirq_xics,
> >      .print_info  = spapr_irq_print_info_xics,
> > @@ -292,17 +301,20 @@ static void spapr_irq_init_xive(sPAPRMachineState *spapr, Error **errp)
> >      spapr_xive_hcall_init(spapr);
> >  }
> >  
> > -static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, bool lsi,
> > -                                Error **errp)
> > +static int spapr_irq_claim_xive(sPAPRMachineState *spapr, int irq, Error **errp)
> >  {
> >      if (!spapr_xive_irq_claim(spapr->xive, irq)) {
> >          error_setg(errp, "IRQ %d is invalid", irq);
> >          return -1;
> >      }
> > -    xive_source_irq_set(&spapr->xive->source, irq, lsi);
> >      return 0;
> >  }
> >  
> > +static void spapr_irq_set_type_xive(sPAPRMachineState *spapr, int irq, bool lsi)
> > +{
> > +    xive_source_irq_set(&spapr->xive->source, irq, lsi);
> > +}
> > +
> >  static void spapr_irq_free_xive(sPAPRMachineState *spapr, int irq, int num)
> >  {
> >      int i;
> > @@ -403,6 +415,7 @@ sPAPRIrq spapr_irq_xive = {
> >  
> >      .init        = spapr_irq_init_xive,
> >      .claim       = spapr_irq_claim_xive,
> > +    .set_type    = spapr_irq_set_type_xive,
> >      .free        = spapr_irq_free_xive,
> >      .qirq        = spapr_qirq_xive,
> >      .print_info  = spapr_irq_print_info_xive,
> > @@ -466,19 +479,19 @@ static void spapr_irq_init_dual(sPAPRMachineState *spapr, Error **errp)
> >      }
> >  }
> >  
> > -static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
> > +static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq,
> >                                  Error **errp)
> >  {
> >      Error *local_err = NULL;
> >      int ret;
> >  
> > -    ret = spapr_irq_xics.claim(spapr, irq, lsi, &local_err);
> > +    ret = spapr_irq_xics.claim(spapr, irq, &local_err);
> >      if (local_err) {
> >          error_propagate(errp, local_err);
> >          return ret;
> >      }
> >  
> > -    ret = spapr_irq_xive.claim(spapr, irq, lsi, &local_err);
> > +    ret = spapr_irq_xive.claim(spapr, irq, &local_err);
> >      if (local_err) {
> >          error_propagate(errp, local_err);
> >          return ret;
> > @@ -487,6 +500,12 @@ static int spapr_irq_claim_dual(sPAPRMachineState *spapr, int irq, bool lsi,
> >      return ret;
> >  }
> >  
> > +static void spapr_irq_set_type_dual(sPAPRMachineState *spapr, int irq, bool lsi)
> > +{
> > +    spapr_irq_xics.set_type(spapr, irq, lsi);
> > +    spapr_irq_xive.set_type(spapr, irq, lsi);
> > +}
> > +
> >  static void spapr_irq_free_dual(sPAPRMachineState *spapr, int irq, int num)
> >  {
> >      spapr_irq_xics.free(spapr, irq, num);
> > @@ -582,6 +601,7 @@ sPAPRIrq spapr_irq_dual = {
> >  
> >      .init        = spapr_irq_init_dual,
> >      .claim       = spapr_irq_claim_dual,
> > +    .set_type    = spapr_irq_set_type_dual,
> >      .free        = spapr_irq_free_dual,
> >      .qirq        = spapr_qirq_dual,
> >      .print_info  = spapr_irq_print_info_dual,
> > @@ -608,9 +628,25 @@ void spapr_irq_init(sPAPRMachineState *spapr, Error **errp)
> >                                        spapr->irq->nr_irqs);
> >  }
> >  
> > -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp)
> > +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp)
> > +{
> > +    return spapr->irq->claim(spapr, irq, errp);
> > +}
> > +
> > +void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi)
> >  {
> > -    return spapr->irq->claim(spapr, irq, lsi, errp);
> > +    return spapr->irq->set_type(spapr, irq, lsi);
> > +}
> > +
> > +int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp)
> > +{
> > +    int ret = spapr_irq_claim(spapr, irq, errp);
> > +
> > +    if (!ret) {
> > +        spapr_irq_set_type(spapr, irq, false);
> > +    }
> > +
> > +    return ret;
> >  }
> >  
> >  void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num)
> > @@ -699,6 +735,7 @@ sPAPRIrq spapr_irq_xics_legacy = {
> >  
> >      .init        = spapr_irq_init_xics,
> >      .claim       = spapr_irq_claim_xics,
> > +    .set_type    = spapr_irq_set_type_xics,
> >      .free        = spapr_irq_free_xics,
> >      .qirq        = spapr_qirq_xics,
> >      .print_info  = spapr_irq_print_info_xics,
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index ccdaf2c9a606..ec9d4d28004c 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -391,7 +391,7 @@ static void rtas_ibm_change_msi(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> >      }
> >  
> >      for (i = 0; i < req_num; i++) {
> > -        spapr_irq_claim(spapr, irq + i, false, &err);
> > +        spapr_irq_claim_msi(spapr, irq + i, &err);
> >          if (err) {
> >              error_reportf_err(err, "Can't allocate MSIs for device %x: ",
> >                                config_addr);
> > @@ -1738,11 +1738,12 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >              }
> >          }
> >  
> > -        spapr_irq_claim(spapr, irq, true, &local_err);
> > +        spapr_irq_claim(spapr, irq, &local_err);
> >          if (local_err) {
> >              error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
> >              return;
> >          }
> > +        spapr_irq_set_type(spapr, irq, true);
> >  
> >          sphb->lsi_table[i].irq = irq;
> >      }
> > diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
> > index f80b70a39c46..9ae4351d4414 100644
> > --- a/hw/ppc/spapr_vio.c
> > +++ b/hw/ppc/spapr_vio.c
> > @@ -546,7 +546,7 @@ static void spapr_vio_busdev_realize(DeviceState *qdev, Error **errp)
> >          }
> >      }
> >  
> > -    spapr_irq_claim(spapr, dev->irq, false, &local_err);
> > +    spapr_irq_claim_msi(spapr, dev->irq, &local_err);
> >      if (local_err) {
> >          error_propagate(errp, local_err);
> >          return;
> > diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> > index 14b02c3aca33..42b325eb51ba 100644
> > --- a/include/hw/ppc/spapr_irq.h
> > +++ b/include/hw/ppc/spapr_irq.h
> > @@ -36,7 +36,8 @@ typedef struct sPAPRIrq {
> >      uint8_t     ov5;
> >  
> >      void (*init)(sPAPRMachineState *spapr, Error **errp);
> > -    int (*claim)(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
> > +    int (*claim)(sPAPRMachineState *spapr, int irq, Error **errp);
> > +    void (*set_type)(sPAPRMachineState *spapr, int irq, bool lsi);
> >      void (*free)(sPAPRMachineState *spapr, int irq, int num);
> >      qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
> >      void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
> > @@ -55,7 +56,9 @@ extern sPAPRIrq spapr_irq_xive;
> >  extern sPAPRIrq spapr_irq_dual;
> >  
> >  void spapr_irq_init(sPAPRMachineState *spapr, Error **errp);
> > -int spapr_irq_claim(sPAPRMachineState *spapr, int irq, bool lsi, Error **errp);
> > +int spapr_irq_claim(sPAPRMachineState *spapr, int irq, Error **errp);
> > +void spapr_irq_set_type(sPAPRMachineState *spapr, int irq, bool lsi);
> > +int spapr_irq_claim_msi(sPAPRMachineState *spapr, int irq, Error **errp);
> >  void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
> >  qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
> >  int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
> >   
> 

Re: [Qemu-devel] [PATCH v3 06/19] spapr: Identify LSIs of all possible PHBs at machine init

[Greg Kurz]

On Fri, 18 Jan 2019 13:38:02 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:15 PM, Greg Kurz wrote:
> > Every PHB needs to claim 4 LSIs to support legacy PCI devices. This is
> > currently done at PHB realize. When using in-kernel XICS (or upcoming
> > in-kernel XIVE), QEMU synchronizes the state of all irqs, including
> > these LSIs, later on at machine reset.
> > 
> > In order to support PHB hotplug, we need a way to tell KVM about the
> > LSIs that doesn't require a machine reset. Since these irq numbers are
> > fixed values derived from the PHB index, let's identify them all at
> > machine init. Older machines that don't have fixed irq numbers cannot
> > support PHB hotplug and keep the existing behavior.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr.c              |    6 ++++++
> >  hw/ppc/spapr_pci.c          |   20 ++++++++++++++++++--
> >  include/hw/pci-host/spapr.h |    2 ++
> >  3 files changed, 26 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 26f8e55cc25e..9189b4d3a9d6 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2802,6 +2802,12 @@ static void spapr_machine_init(MachineState *machine)
> >  
> >      phb = spapr_create_default_phb();
> >  
> > +    if (!smc->legacy_irq_allocation) {
> > +        for (i = 0; i < SPAPR_MAX_PHBS; i++) {
> > +            spapr_phb_set_lsis(i, spapr);
> > +        }
> > +    }
> > +  
> 
> Can't we be more brutal and do the LSI setting in spapr_irq_init() 
> for the whole [ SPAPR_IRQ_PCI_LSI ... SPAPR_IRQ_MSI -1 ] range ? 
> 

I like this. And we should even set the type of all IRQs for the machine
lifetime since it is an invariant. I'll give it a try for v4.

> C. 
> 
> >      for (i = 0; i < nb_nics; i++) {
> >          NICInfo *nd = &nd_table[i];
> >  
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index ec9d4d28004c..f5f13a4d4816 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -1559,6 +1559,22 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
> >      }
> >  }
> >  
> > +static uint32_t spapr_phb_index_to_lsi(int phb_index, int irq_index)
> > +{
> > +    return SPAPR_IRQ_PCI_LSI + phb_index * PCI_NUM_PINS + irq_index;
> > +}
> > +
> > +void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr)
> > +{
> > +    int i;
> > +
> > +    for (i = 0; i < PCI_NUM_PINS; i++) {
> > +        uint32_t irq = spapr_phb_index_to_lsi(index, i);
> > +
> > +        spapr_irq_set_type(spapr, irq, true);
> > +    }
> > +}
> > +
> >  static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >  {
> >      /* We don't use SPAPR_MACHINE() in order to exit gracefully if the user
> > @@ -1726,7 +1742,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >  
> >      /* Initialize the LSI table */
> >      for (i = 0; i < PCI_NUM_PINS; i++) {
> > -        uint32_t irq = SPAPR_IRQ_PCI_LSI + sphb->index * PCI_NUM_PINS + i;
> > +        uint32_t irq = spapr_phb_index_to_lsi(sphb->index, i);
> >          Error *local_err = NULL;
> >  
> >          if (smc->legacy_irq_allocation) {
> > @@ -1736,6 +1752,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >                                          "can't allocate LSIs: ");
> >                  return;
> >              }
> > +            spapr_irq_set_type(spapr, irq, true);
> >          }
> >  
> >          spapr_irq_claim(spapr, irq, &local_err);
> > @@ -1743,7 +1760,6 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp)
> >              error_propagate_prepend(errp, local_err, "can't allocate LSIs: ");
> >              return;
> >          }
> > -        spapr_irq_set_type(spapr, irq, true);
> >  
> >          sphb->lsi_table[i].irq = irq;
> >      }
> > diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> > index e0e683c32469..bb0ae7fdd41d 100644
> > --- a/include/hw/pci-host/spapr.h
> > +++ b/include/hw/pci-host/spapr.h
> > @@ -164,4 +164,6 @@ static inline void spapr_phb_vfio_reset(DeviceState *qdev)
> >  
> >  void spapr_phb_dma_reset(sPAPRPHBState *sphb);
> >  
> > +void spapr_phb_set_lsis(int index, sPAPRMachineState *spapr);
> > +
> >  #endif /* PCI_HOST_SPAPR_H */
> >   
> 

Re: [Qemu-devel] [PATCH v3 14/19] spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend

[Greg Kurz]

On Fri, 18 Jan 2019 13:45:26 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:15 PM, Greg Kurz wrote:
> > The setting of the "phandle" and "linux,phandle" properties of the
> > interrupt controller node is duplicate code in XICS and XIVE. Move
> > it to a new spapr_irq_dt_populate() function.
> > 
> > Note that for XIVE, we set the "ibm,plat-res-int-priorities" machine
> > property earlier so that the offset returned by fdt_add_subnode()
> > can be safely returned to the caller.  
> 
> Ah yes. This is a top level XIVE property but I don't think it belongs
> to the machine.
>  

This property is described in "B.6.2.1 Root Node Properties" of LoPAPR.

> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/intc/spapr_xive.c        |   21 +++++++++------------
> >  hw/intc/xics_spapr.c        |    7 +++----
> >  hw/ppc/spapr.c              |    2 +-
> >  hw/ppc/spapr_irq.c          |   17 +++++++++++++----
> >  include/hw/ppc/spapr_irq.h  |    6 ++++--
> >  include/hw/ppc/spapr_xive.h |    3 +--
> >  include/hw/ppc/xics_spapr.h |    3 +--
> >  7 files changed, 32 insertions(+), 27 deletions(-)
> > 
> > diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> > index aab63cfd1178..e542ae59d7fd 100644
> > --- a/hw/intc/spapr_xive.c
> > +++ b/hw/intc/spapr_xive.c
> > @@ -1411,8 +1411,7 @@ void spapr_xive_hcall_init(sPAPRMachineState *spapr)
> >      spapr_register_hypercall(H_INT_RESET, h_int_reset);
> >  }
> >  
> > -void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> > -                   uint32_t phandle)
> > +int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)  
> 
> I am sure this is the correct prototype. This one returning an error and 
> the phander looks better.
> 
>    int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>                       uint32_t *phandle)
> 

This looks nicer. I'll do that in v4.

> But I won't make a fuss about it either.
> 
> 
> Reviewed-by: Cédric Le Goater <clg@kaod.org>
> 
> Thanks,
> 
> C.
> 
> 
> >  {
> >      sPAPRXive *xive = spapr->xive;
> >      int node;
> > @@ -1439,6 +1438,13 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> >      };
> >      gchar *nodename;
> >  
> > +    /*
> > +     * The "ibm,plat-res-int-priorities" property defines the priority
> > +     * ranges reserved by the hypervisor
> > +     */
> > +    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
> > +                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
> > +
> >      /* Thread Interrupt Management Area : User (ring 3) and OS (ring 2) */
> >      timas[0] = cpu_to_be64(xive->tm_base +
> >                             XIVE_TM_USER_PAGE * (1ull << TM_SHIFT));
> > @@ -1465,14 +1471,5 @@ void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> >      _FDT(fdt_setprop(fdt, node, "interrupt-controller", NULL, 0));
> >      _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
> >  
> > -    /* For SLOF */
> > -    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> > -    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> > -
> > -    /*
> > -     * The "ibm,plat-res-int-priorities" property defines the priority
> > -     * ranges reserved by the hypervisor
> > -     */
> > -    _FDT(fdt_setprop(fdt, 0, "ibm,plat-res-int-priorities",
> > -                     plat_res_int_priorities, sizeof(plat_res_int_priorities)));
> > +    return node;
> >  }
> > diff --git a/hw/intc/xics_spapr.c b/hw/intc/xics_spapr.c
> > index de6cc15b6474..375cb883c86d 100644
> > --- a/hw/intc/xics_spapr.c
> > +++ b/hw/intc/xics_spapr.c
> > @@ -245,8 +245,7 @@ void xics_spapr_init(sPAPRMachineState *spapr)
> >      spapr_register_hypercall(H_IPOLL, h_ipoll);
> >  }
> >  
> > -void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> > -                   uint32_t phandle)
> > +int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
> >  {
> >      uint32_t interrupt_server_ranges_prop[] = {
> >          0, cpu_to_be32(nr_servers),
> > @@ -263,6 +262,6 @@ void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> >                       interrupt_server_ranges_prop,
> >                       sizeof(interrupt_server_ranges_prop)));
> >      _FDT(fdt_setprop_cell(fdt, node, "#interrupt-cells", 2));
> > -    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> > -    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> > +
> > +    return node;
> >  }
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 2c8d0c0e149c..135136b4cf5f 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -1275,7 +1275,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
> >      _FDT(fdt_setprop_cell(fdt, 0, "#size-cells", 2));
> >  
> >      /* /interrupt controller */
> > -    spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> > +    spapr_irq_dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> >                            PHANDLE_INTC);
> >  
> >      ret = spapr_populate_memory(spapr, fdt);
> > diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> > index 396dd09bdbe0..b3e0713428ed 100644
> > --- a/hw/ppc/spapr_irq.c
> > +++ b/hw/ppc/spapr_irq.c
> > @@ -11,6 +11,7 @@
> >  #include "qemu/log.h"
> >  #include "qemu/error-report.h"
> >  #include "qapi/error.h"
> > +#include "hw/ppc/fdt.h"
> >  #include "hw/ppc/spapr.h"
> >  #include "hw/ppc/spapr_xive.h"
> >  #include "hw/ppc/xics.h"
> > @@ -536,11 +537,10 @@ static void spapr_irq_print_info_dual(sPAPRMachineState *spapr, Monitor *mon)
> >      spapr_irq_current(spapr)->print_info(spapr, mon);
> >  }
> >  
> > -static void spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
> > -                                       uint32_t nr_servers, void *fdt,
> > -                                       uint32_t phandle)
> > +static int spapr_irq_dt_populate_dual(sPAPRMachineState *spapr,
> > +                                      uint32_t nr_servers, void *fdt)
> >  {
> > -    spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt, phandle);
> > +    return spapr_irq_current(spapr)->dt_populate(spapr, nr_servers, fdt);
> >  }
> >  
> >  static void spapr_irq_cpu_intc_create_dual(sPAPRMachineState *spapr,
> > @@ -671,6 +671,15 @@ void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp)
> >      }
> >  }
> >  
> > +void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> > +                           void *fdt, uint32_t phandle)
> > +{
> > +    int node = spapr->irq->dt_populate(spapr, nr_servers, fdt);
> > +
> > +    _FDT(fdt_setprop_cell(fdt, node, "linux,phandle", phandle));
> > +    _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> > +}
> > +
> >  /*
> >   * XICS legacy routines - to deprecate one day
> >   */
> > diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> > index 42b325eb51ba..dfed6b7e9976 100644
> > --- a/include/hw/ppc/spapr_irq.h
> > +++ b/include/hw/ppc/spapr_irq.h
> > @@ -41,8 +41,8 @@ typedef struct sPAPRIrq {
> >      void (*free)(sPAPRMachineState *spapr, int irq, int num);
> >      qemu_irq (*qirq)(sPAPRMachineState *spapr, int irq);
> >      void (*print_info)(sPAPRMachineState *spapr, Monitor *mon);
> > -    void (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
> > -                        void *fdt, uint32_t phandle);
> > +    int (*dt_populate)(sPAPRMachineState *spapr, uint32_t nr_servers,
> > +                       void *fdt);
> >      void (*cpu_intc_create)(sPAPRMachineState *spapr, PowerPCCPU *cpu,
> >                              Error **errp);
> >      int (*post_load)(sPAPRMachineState *spapr, int version_id);
> > @@ -63,6 +63,8 @@ void spapr_irq_free(sPAPRMachineState *spapr, int irq, int num);
> >  qemu_irq spapr_qirq(sPAPRMachineState *spapr, int irq);
> >  int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
> >  void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
> > +void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> > +                           void *fdt, uint32_t phandle);
> >  
> >  /*
> >   * XICS legacy routines
> > diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> > index f8854a4a6a18..29dafead9ba0 100644
> > --- a/include/hw/ppc/spapr_xive.h
> > +++ b/include/hw/ppc/spapr_xive.h
> > @@ -44,8 +44,7 @@ void spapr_xive_pic_print_info(sPAPRXive *xive, Monitor *mon);
> >  typedef struct sPAPRMachineState sPAPRMachineState;
> >  
> >  void spapr_xive_hcall_init(sPAPRMachineState *spapr);
> > -void spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> > -                   uint32_t phandle);
> > +int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
> >  void spapr_xive_set_tctx_os_cam(XiveTCTX *tctx);
> >  void spapr_xive_mmio_set_enabled(sPAPRXive *xive, bool enable);
> >  
> > diff --git a/include/hw/ppc/xics_spapr.h b/include/hw/ppc/xics_spapr.h
> > index b1ab27d022cf..9f51f8621f75 100644
> > --- a/include/hw/ppc/xics_spapr.h
> > +++ b/include/hw/ppc/xics_spapr.h
> > @@ -29,8 +29,7 @@
> >  
> >  #include "hw/ppc/spapr.h"
> >  
> > -void spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt,
> > -                   uint32_t phandle);
> > +int spapr_dt_xics(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt);
> >  int xics_kvm_init(sPAPRMachineState *spapr, Error **errp);
> >  void xics_spapr_init(sPAPRMachineState *spapr);
> >  
> >   
> 

Re: [Qemu-devel] [PATCH v3 00/19] spapr: Add support for PHB hotplug

[Alexey Kardashevskiy]

On 18/01/2019 04:14, Greg Kurz wrote:
> This allows to hotplug/unplug PHBs. I could successfully test:
> - using in-kernel XICS, emulated XICS and XIVE
> - using experimental in-kernel XIVE from Cedric
> - hotplug/unplug with e1000 device to validate LSIs
> - hotplug/unplug with virtio-net device to validate MSIs
> - some simple migration scenarios

> 
> Change in v3:
> - reworked phandle related code some more
> - disintricate allocation/"type setting" of interrupts
> - identify LSIs at machine init
> 
> Changes in v2:
> - rebased on current ppc-for-4.0



Is the tree available anywhere in git? It does not apply on today's
ppc-for-4.0 :-/




> - added some preliminary cleanup
> - call unrealize from realize error path
> - advertise PHB hotplug in last patch
> - reworked phandle related code
> - sync LSIs to KVM
> 
> Please comment.
> 
> --
> Greg
> 
> ---
> 
> Greg Kurz (11):
>       spapr: Rename xics to intc in interrupt controller agnostic code
>       xics: Disintricate allocation and type setting of interrupts
>       spapr/xive: Don't set irq type in spapr_xive_irq_claim()
>       spapr: Set irq type in a dedicated function
>       spapr: Identify LSIs of all possible PHBs at machine init
>       spapr_pci: add PHB unrealize
>       spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend
>       spapr_xive: Cache device tree nodename in sPAPRXive
>       spapr: Expose the name of the interrupt controller node
>       spapr_irq: Expose the phandle of the interrupt controller
>       spapr: add hotplug hooks for PHB hotplug
> 
> Michael Roth (6):
>       spapr: create DR connectors for PHBs
>       spapr_events: add support for phb hotplug events
>       qdev: pass an Object * to qbus_set_hotplug_handler()
>       spapr_pci: provide node start offset via spapr_populate_pci_dt()
>       spapr_pci: add ibm, my-drc-index property for PHB hotplug
>       spapr: enable PHB hotplug for default pseries machine type
> 
> Nathan Fontenot (1):
>       spapr: populate PHB DRC entries for root DT node
> 
> Thomas Huth (1):
>       ppc: Move spapr-related prototypes from xics.h into a seperate header file
> 
> 
>  hw/acpi/pcihp.c               |    2 -
>  hw/acpi/piix4.c               |    2 -
>  hw/char/virtio-serial-bus.c   |    2 -
>  hw/core/bus.c                 |   11 +--
>  hw/core/machine.c             |    4 +
>  hw/intc/spapr_xive.c          |   35 ++++-----
>  hw/intc/xics.c                |   28 ++++++-
>  hw/intc/xics_kvm.c            |    1 
>  hw/intc/xics_spapr.c          |   10 +--
>  hw/pci/pcie.c                 |    2 -
>  hw/pci/shpc.c                 |    2 -
>  hw/ppc/pnv_psi.c              |    1 
>  hw/ppc/spapr.c                |  163 +++++++++++++++++++++++++++++++++++++++--
>  hw/ppc/spapr_drc.c            |   18 +++++
>  hw/ppc/spapr_events.c         |    9 ++
>  hw/ppc/spapr_irq.c            |  129 +++++++++++++++++++++++++++-----
>  hw/ppc/spapr_pci.c            |  124 ++++++++++++++++++++++++-------
>  hw/ppc/spapr_vio.c            |    4 +
>  hw/s390x/css-bridge.c         |    2 -
>  hw/s390x/s390-pci-bus.c       |    6 +-
>  hw/scsi/virtio-scsi.c         |    2 -
>  hw/scsi/vmw_pvscsi.c          |    2 -
>  hw/usb/dev-smartcard-reader.c |    2 -
>  include/hw/pci-host/spapr.h   |   11 ++-
>  include/hw/ppc/spapr.h        |    4 +
>  include/hw/ppc/spapr_drc.h    |    8 ++
>  include/hw/ppc/spapr_irq.h    |   16 +++-
>  include/hw/ppc/spapr_xive.h   |   13 +++
>  include/hw/ppc/xics.h         |   15 ++--
>  include/hw/ppc/xics_spapr.h   |   43 +++++++++++
>  include/hw/qdev-core.h        |    3 -
>  31 files changed, 545 insertions(+), 129 deletions(-)
>  create mode 100644 include/hw/ppc/xics_spapr.h
> 

-- 
Alexey

Re: [Qemu-devel] [PATCH v3 00/19] spapr: Add support for PHB hotplug

[Greg Kurz]

On Tue, 22 Jan 2019 16:44:02 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 18/01/2019 04:14, Greg Kurz wrote:
> > This allows to hotplug/unplug PHBs. I could successfully test:
> > - using in-kernel XICS, emulated XICS and XIVE
> > - using experimental in-kernel XIVE from Cedric
> > - hotplug/unplug with e1000 device to validate LSIs
> > - hotplug/unplug with virtio-net device to validate MSIs
> > - some simple migration scenarios  
> 
> > 
> > Change in v3:
> > - reworked phandle related code some more
> > - disintricate allocation/"type setting" of interrupts
> > - identify LSIs at machine init
> > 
> > Changes in v2:
> > - rebased on current ppc-for-4.0  
> 
> 
> 
> Is the tree available anywhere in git? It does not apply on today's
> ppc-for-4.0 :-/
> 

I'm currently working on v4. I'll push it to github when it's ready.

> 
> 
> 
> > - added some preliminary cleanup
> > - call unrealize from realize error path
> > - advertise PHB hotplug in last patch
> > - reworked phandle related code
> > - sync LSIs to KVM
> > 
> > Please comment.
> > 
> > --
> > Greg
> > 
> > ---
> > 
> > Greg Kurz (11):
> >       spapr: Rename xics to intc in interrupt controller agnostic code
> >       xics: Disintricate allocation and type setting of interrupts
> >       spapr/xive: Don't set irq type in spapr_xive_irq_claim()
> >       spapr: Set irq type in a dedicated function
> >       spapr: Identify LSIs of all possible PHBs at machine init
> >       spapr_pci: add PHB unrealize
> >       spapr: Factor out setting of "phandle" DT property to sPAPR irq frontend
> >       spapr_xive: Cache device tree nodename in sPAPRXive
> >       spapr: Expose the name of the interrupt controller node
> >       spapr_irq: Expose the phandle of the interrupt controller
> >       spapr: add hotplug hooks for PHB hotplug
> > 
> > Michael Roth (6):
> >       spapr: create DR connectors for PHBs
> >       spapr_events: add support for phb hotplug events
> >       qdev: pass an Object * to qbus_set_hotplug_handler()
> >       spapr_pci: provide node start offset via spapr_populate_pci_dt()
> >       spapr_pci: add ibm, my-drc-index property for PHB hotplug
> >       spapr: enable PHB hotplug for default pseries machine type
> > 
> > Nathan Fontenot (1):
> >       spapr: populate PHB DRC entries for root DT node
> > 
> > Thomas Huth (1):
> >       ppc: Move spapr-related prototypes from xics.h into a seperate header file
> > 
> > 
> >  hw/acpi/pcihp.c               |    2 -
> >  hw/acpi/piix4.c               |    2 -
> >  hw/char/virtio-serial-bus.c   |    2 -
> >  hw/core/bus.c                 |   11 +--
> >  hw/core/machine.c             |    4 +
> >  hw/intc/spapr_xive.c          |   35 ++++-----
> >  hw/intc/xics.c                |   28 ++++++-
> >  hw/intc/xics_kvm.c            |    1 
> >  hw/intc/xics_spapr.c          |   10 +--
> >  hw/pci/pcie.c                 |    2 -
> >  hw/pci/shpc.c                 |    2 -
> >  hw/ppc/pnv_psi.c              |    1 
> >  hw/ppc/spapr.c                |  163 +++++++++++++++++++++++++++++++++++++++--
> >  hw/ppc/spapr_drc.c            |   18 +++++
> >  hw/ppc/spapr_events.c         |    9 ++
> >  hw/ppc/spapr_irq.c            |  129 +++++++++++++++++++++++++++-----
> >  hw/ppc/spapr_pci.c            |  124 ++++++++++++++++++++++++-------
> >  hw/ppc/spapr_vio.c            |    4 +
> >  hw/s390x/css-bridge.c         |    2 -
> >  hw/s390x/s390-pci-bus.c       |    6 +-
> >  hw/scsi/virtio-scsi.c         |    2 -
> >  hw/scsi/vmw_pvscsi.c          |    2 -
> >  hw/usb/dev-smartcard-reader.c |    2 -
> >  include/hw/pci-host/spapr.h   |   11 ++-
> >  include/hw/ppc/spapr.h        |    4 +
> >  include/hw/ppc/spapr_drc.h    |    8 ++
> >  include/hw/ppc/spapr_irq.h    |   16 +++-
> >  include/hw/ppc/spapr_xive.h   |   13 +++
> >  include/hw/ppc/xics.h         |   15 ++--
> >  include/hw/ppc/xics_spapr.h   |   43 +++++++++++
> >  include/hw/qdev-core.h        |    3 -
> >  31 files changed, 545 insertions(+), 129 deletions(-)
> >  create mode 100644 include/hw/ppc/xics_spapr.h
> >   
> 

Re: [Qemu-devel] [PATCH v3 15/19] spapr_xive: Cache device tree nodename in sPAPRXive

[Greg Kurz]

On Fri, 18 Jan 2019 14:38:31 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:16 PM, Greg Kurz wrote:
> > PHB hotplug will need to know the name of the XIVE controller node. Since
> > it is an invariant for the machine lifetime, compute it at realize and
> > store it under the sPAPRXive structure.  
> Could you please gather all these changes [15-17] in one patch. It would 
> be easier to track. 
> 

I'll do that in v4.

> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/intc/spapr_xive.c        |    9 ++++-----
> >  include/hw/ppc/spapr_xive.h |    3 +++
> >  2 files changed, 7 insertions(+), 5 deletions(-)
> > 
> > diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> > index e542ae59d7fd..9732c3d89c77 100644
> > --- a/hw/intc/spapr_xive.c
> > +++ b/hw/intc/spapr_xive.c
> > @@ -316,6 +316,9 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
> >      /* Map all regions */
> >      spapr_xive_map_mmio(xive);
> >  
> > +    xive->nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> > +                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));  
> 
> I would use a helper routine instead.
>  

Ok, I see your suggestion to put the name in a static. Well, it would work
as long as we only have one interrupt controller around. It is the case now
but could this change in the future ? If not then I'm perfectly fine with
your suggestions, otherwise I guess the nodename should be under sPAPRXive.

> >      qemu_register_reset(spapr_xive_reset, dev);
> >  }
> >  
> > @@ -1436,7 +1439,6 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
> >          cpu_to_be32(7),    /* start */
> >          cpu_to_be32(0xf8), /* count */
> >      };
> > -    gchar *nodename;
> >  
> >      /*
> >       * The "ibm,plat-res-int-priorities" property defines the priority
> > @@ -1453,10 +1455,7 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
> >                             XIVE_TM_OS_PAGE * (1ull << TM_SHIFT));
> >      timas[3] = cpu_to_be64(1ull << TM_SHIFT);
> >  
> > -    nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> > -                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
> > -    _FDT(node = fdt_add_subnode(fdt, 0, nodename));
> > -    g_free(nodename);
> > +    _FDT(node = fdt_add_subnode(fdt, 0, xive->nodename));  
> 
> and use the helper routine here.
> 
> >      _FDT(fdt_setprop_string(fdt, node, "device_type", "power-ivpe"));
> >      _FDT(fdt_setprop(fdt, node, "reg", timas, sizeof(timas)));
> > diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> > index 29dafead9ba0..deea34b03ee5 100644
> > --- a/include/hw/ppc/spapr_xive.h
> > +++ b/include/hw/ppc/spapr_xive.h
> > @@ -26,6 +26,9 @@ typedef struct sPAPRXive {
> >      XiveENDSource end_source;
> >      hwaddr        end_base;
> >  
> > +    /* DT */
> > +    gchar *nodename;  
> 
> I don't think this is needed. See other patches for why.
> 
> Thanks,
> 
> C.
>  
> >
> >      /* Routing table */
> >      XiveEAS       *eat;
> >      uint32_t      nr_irqs;
> >   
> 

Re: [Qemu-devel] [PATCH v3 17/19] spapr_irq: Expose the phandle of the interrupt controller

[Greg Kurz]

On Fri, 18 Jan 2019 14:46:42 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/17/19 6:16 PM, Greg Kurz wrote:
> > This will be used by PHB hotplug in order to create the "interrupt-map"
> > property of the PHB node.
> > 
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/ppc/spapr_irq.c         |   44 +++++++++++++++++++++++++++++++++++++++++++-
> >  include/hw/ppc/spapr_irq.h |    3 +++
> >  2 files changed, 46 insertions(+), 1 deletion(-)
> > 
> > diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
> > index b3e0713428ed..93243cfd0644 100644
> > --- a/hw/ppc/spapr_irq.c
> > +++ b/hw/ppc/spapr_irq.c
> > @@ -243,6 +243,11 @@ static void spapr_irq_reset_xics(sPAPRMachineState *spapr, Error **errp)
> >      /* TODO: create the KVM XICS device */
> >  }
> >  
> > +static const char *spapr_irq_get_nodename_xics(sPAPRMachineState *spapr)
> > +{
> > +    return xics_spapr_get_nodename();  
> 
> can't we just return XICS_NODENAME ?
> 

Sure.

> > +}
> > +
> >  #define SPAPR_IRQ_XICS_NR_IRQS     0x1000
> >  #define SPAPR_IRQ_XICS_NR_MSIS     \
> >      (XICS_IRQ_BASE + SPAPR_IRQ_XICS_NR_IRQS - SPAPR_IRQ_MSI)
> > @@ -263,6 +268,7 @@ sPAPRIrq spapr_irq_xics = {
> >      .post_load   = spapr_irq_post_load_xics,
> >      .reset       = spapr_irq_reset_xics,
> >      .set_irq     = spapr_irq_set_irq_xics,
> > +    .get_nodename = spapr_irq_get_nodename_xics,
> >  };
> >  
> >  /*
> > @@ -401,6 +407,11 @@ static void spapr_irq_set_irq_xive(void *opaque, int srcno, int val)
> >      xive_source_set_irq(&spapr->xive->source, srcno, val);
> >  }
> >  
> > +static const char *spapr_irq_get_nodename_xive(sPAPRMachineState *spapr)
> > +{
> > +    return spapr_xive_get_nodename(spapr->xive);
> > +}
> > +
> >  /*
> >   * XIVE uses the full IRQ number space. Set it to 8K to be compatible
> >   * with XICS.
> > @@ -425,6 +436,7 @@ sPAPRIrq spapr_irq_xive = {
> >      .post_load   = spapr_irq_post_load_xive,
> >      .reset       = spapr_irq_reset_xive,
> >      .set_irq     = spapr_irq_set_irq_xive,
> > +    .get_nodename = spapr_irq_get_nodename_xive,
> >  };
> >  
> >  /*
> > @@ -588,6 +600,11 @@ static void spapr_irq_set_irq_dual(void *opaque, int srcno, int val)
> >      spapr_irq_current(spapr)->set_irq(spapr, srcno, val);
> >  }
> >  
> > +static const char *spapr_irq_get_nodename_dual(sPAPRMachineState *spapr)
> > +{
> > +    return spapr_irq_current(spapr)->get_nodename(spapr);
> > +}
> > +
> >  /*
> >   * Define values in sync with the XIVE and XICS backend
> >   */
> > @@ -609,7 +626,8 @@ sPAPRIrq spapr_irq_dual = {
> >      .cpu_intc_create = spapr_irq_cpu_intc_create_dual,
> >      .post_load   = spapr_irq_post_load_dual,
> >      .reset       = spapr_irq_reset_dual,
> > -    .set_irq     = spapr_irq_set_irq_dual
> > +    .set_irq     = spapr_irq_set_irq_dual,
> > +    .get_nodename = spapr_irq_get_nodename_dual,
> >  };
> >  
> >  /*
> > @@ -680,6 +698,29 @@ void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> >      _FDT(fdt_setprop_cell(fdt, node, "phandle", phandle));
> >  }
> >  
> > +uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
> > +                               Error **errp)
> > +{
> > +    const char *nodename = spapr->irq->get_nodename(spapr);
> > +    int phandle, offset;
> > +
> > +    offset = fdt_subnode_offset(fdt, 0, nodename);  
> 
> if I am correct, you changed dt_populate() to return the same offset.
> 
> > +    if (offset < 0) {
> > +        error_setg(errp, "Can't find node \"%s\": %s", nodename,
> > +                   fdt_strerror(offset));
> > +        return -1;
> > +    }
> > +
> > +    phandle = fdt_get_phandle(fdt, offset);  
> 
> if we stored the phandle when spapr_irq_dt_populate() is called, I think
> things would be easier. May be under the backend ? 
> 

Yes we can do that but SLOF may now update the fdt, so we need
to be able to update the phandle if it got changed. I'll look
how to do that cleanly in v4.

> C.
> 
> 
> > +    if (phandle < 0) {
> > +        error_setg(errp, "Can't get phandle of node \"%s\": %s",
> > +                   nodename, fdt_strerror(phandle));
> > +        return -1;
> > +    }
> > +
> > +    return phandle;
> > +}
> > +
> >  /*
> >   * XICS legacy routines - to deprecate one day
> >   */
> > @@ -752,4 +793,5 @@ sPAPRIrq spapr_irq_xics_legacy = {
> >      .cpu_intc_create = spapr_irq_cpu_intc_create_xics,
> >      .post_load   = spapr_irq_post_load_xics,
> >      .set_irq     = spapr_irq_set_irq_xics,
> > +    .get_nodename = spapr_irq_get_nodename_xics,
> >  };
> > diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
> > index dfed6b7e9976..d44d659231a6 100644
> > --- a/include/hw/ppc/spapr_irq.h
> > +++ b/include/hw/ppc/spapr_irq.h
> > @@ -48,6 +48,7 @@ typedef struct sPAPRIrq {
> >      int (*post_load)(sPAPRMachineState *spapr, int version_id);
> >      void (*reset)(sPAPRMachineState *spapr, Error **errp);
> >      void (*set_irq)(void *opaque, int srcno, int val);
> > +    const char *(*get_nodename)(sPAPRMachineState *spapr);
> >  } sPAPRIrq;
> >  
> >  extern sPAPRIrq spapr_irq_xics;
> > @@ -65,6 +66,8 @@ int spapr_irq_post_load(sPAPRMachineState *spapr, int version_id);
> >  void spapr_irq_reset(sPAPRMachineState *spapr, Error **errp);
> >  void spapr_irq_dt_populate(sPAPRMachineState *spapr, uint32_t nr_servers,
> >                             void *fdt, uint32_t phandle);
> > +uint32_t spapr_irq_get_phandle(sPAPRMachineState *spapr, void *fdt,
> > +                               Error **errp);
> >  
> >  /*
> >   * XICS legacy routines
> >   
> 

Re: [Qemu-devel] [PATCH v3 15/19] spapr_xive: Cache device tree nodename in sPAPRXive

[Cédric Le Goater]

On 1/22/19 2:27 PM, Greg Kurz wrote:
> On Fri, 18 Jan 2019 14:38:31 +0100
> Cédric Le Goater <clg@kaod.org> wrote:
> 
>> On 1/17/19 6:16 PM, Greg Kurz wrote:
>>> PHB hotplug will need to know the name of the XIVE controller node. Since
>>> it is an invariant for the machine lifetime, compute it at realize and
>>> store it under the sPAPRXive structure.  
>> Could you please gather all these changes [15-17] in one patch. It would 
>> be easier to track. 
>>
> 
> I'll do that in v4.
> 
>>> Signed-off-by: Greg Kurz <groug@kaod.org>
>>> ---
>>>  hw/intc/spapr_xive.c        |    9 ++++-----
>>>  include/hw/ppc/spapr_xive.h |    3 +++
>>>  2 files changed, 7 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
>>> index e542ae59d7fd..9732c3d89c77 100644
>>> --- a/hw/intc/spapr_xive.c
>>> +++ b/hw/intc/spapr_xive.c
>>> @@ -316,6 +316,9 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
>>>      /* Map all regions */
>>>      spapr_xive_map_mmio(xive);
>>>  
>>> +    xive->nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
>>> +                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));  
>>
>> I would use a helper routine instead.
>>  
> 
> Ok, I see your suggestion to put the name in a static. Well, it would work
> as long as we only have one interrupt controller around. It is the case now
> but could this change in the future ? If not then I'm perfectly fine with
> your suggestions, otherwise I guess the nodename should be under sPAPRXive.

Well, I am not sure it's worth extending SPAPRXive for a name that can 
be computed. Using a static is probably not a better solution. 

Maybe return the node offset instead. See pnv_chip_isa_offset() in pnv.c 
for a similar issue.

Thanks,

C. 

> 
>>>      qemu_register_reset(spapr_xive_reset, dev);
>>>  }
>>>  
>>> @@ -1436,7 +1439,6 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>>>          cpu_to_be32(7),    /* start */
>>>          cpu_to_be32(0xf8), /* count */
>>>      };
>>> -    gchar *nodename;
>>>  
>>>      /*
>>>       * The "ibm,plat-res-int-priorities" property defines the priority
>>> @@ -1453,10 +1455,7 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
>>>                             XIVE_TM_OS_PAGE * (1ull << TM_SHIFT));
>>>      timas[3] = cpu_to_be64(1ull << TM_SHIFT);
>>>  
>>> -    nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
>>> -                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
>>> -    _FDT(node = fdt_add_subnode(fdt, 0, nodename));
>>> -    g_free(nodename);
>>> +    _FDT(node = fdt_add_subnode(fdt, 0, xive->nodename));  
>>
>> and use the helper routine here.
>>
>>>      _FDT(fdt_setprop_string(fdt, node, "device_type", "power-ivpe"));
>>>      _FDT(fdt_setprop(fdt, node, "reg", timas, sizeof(timas)));
>>> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
>>> index 29dafead9ba0..deea34b03ee5 100644
>>> --- a/include/hw/ppc/spapr_xive.h
>>> +++ b/include/hw/ppc/spapr_xive.h
>>> @@ -26,6 +26,9 @@ typedef struct sPAPRXive {
>>>      XiveENDSource end_source;
>>>      hwaddr        end_base;
>>>  
>>> +    /* DT */
>>> +    gchar *nodename;  
>>
>> I don't think this is needed. See other patches for why.
>>
>> Thanks,
>>
>> C.
>>  
>>>
>>>      /* Routing table */
>>>      XiveEAS       *eat;
>>>      uint32_t      nr_irqs;
>>>   
>>
> 

Re: [Qemu-devel] [PATCH v3 15/19] spapr_xive: Cache device tree nodename in sPAPRXive

[Greg Kurz]

On Tue, 22 Jan 2019 15:26:45 +0100
Cédric Le Goater <clg@kaod.org> wrote:

> On 1/22/19 2:27 PM, Greg Kurz wrote:
> > On Fri, 18 Jan 2019 14:38:31 +0100
> > Cédric Le Goater <clg@kaod.org> wrote:
> >   
> >> On 1/17/19 6:16 PM, Greg Kurz wrote:  
> >>> PHB hotplug will need to know the name of the XIVE controller node. Since
> >>> it is an invariant for the machine lifetime, compute it at realize and
> >>> store it under the sPAPRXive structure.    
> >> Could you please gather all these changes [15-17] in one patch. It would 
> >> be easier to track. 
> >>  
> > 
> > I'll do that in v4.
> >   
> >>> Signed-off-by: Greg Kurz <groug@kaod.org>
> >>> ---
> >>>  hw/intc/spapr_xive.c        |    9 ++++-----
> >>>  include/hw/ppc/spapr_xive.h |    3 +++
> >>>  2 files changed, 7 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c
> >>> index e542ae59d7fd..9732c3d89c77 100644
> >>> --- a/hw/intc/spapr_xive.c
> >>> +++ b/hw/intc/spapr_xive.c
> >>> @@ -316,6 +316,9 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
> >>>      /* Map all regions */
> >>>      spapr_xive_map_mmio(xive);
> >>>  
> >>> +    xive->nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> >>> +                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));    
> >>
> >> I would use a helper routine instead.
> >>    
> > 
> > Ok, I see your suggestion to put the name in a static. Well, it would work
> > as long as we only have one interrupt controller around. It is the case now
> > but could this change in the future ? If not then I'm perfectly fine with
> > your suggestions, otherwise I guess the nodename should be under sPAPRXive.  
> 
> Well, I am not sure it's worth extending SPAPRXive for a name that can 
> be computed. Using a static is probably not a better solution. 
> 
> Maybe return the node offset instead. See pnv_chip_isa_offset() in pnv.c 

We can't with pseries because SLOF can update the machine's fdt, which
means the offset could be different from what it was when spapr_dt_xive()
was last called.

> for a similar issue.
> 
> Thanks,
> 
> C. 
> 
> >   
> >>>      qemu_register_reset(spapr_xive_reset, dev);
> >>>  }
> >>>  
> >>> @@ -1436,7 +1439,6 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
> >>>          cpu_to_be32(7),    /* start */
> >>>          cpu_to_be32(0xf8), /* count */
> >>>      };
> >>> -    gchar *nodename;
> >>>  
> >>>      /*
> >>>       * The "ibm,plat-res-int-priorities" property defines the priority
> >>> @@ -1453,10 +1455,7 @@ int spapr_dt_xive(sPAPRMachineState *spapr, uint32_t nr_servers, void *fdt)
> >>>                             XIVE_TM_OS_PAGE * (1ull << TM_SHIFT));
> >>>      timas[3] = cpu_to_be64(1ull << TM_SHIFT);
> >>>  
> >>> -    nodename = g_strdup_printf("interrupt-controller@%" PRIx64,
> >>> -                           xive->tm_base + XIVE_TM_USER_PAGE * (1 << TM_SHIFT));
> >>> -    _FDT(node = fdt_add_subnode(fdt, 0, nodename));
> >>> -    g_free(nodename);
> >>> +    _FDT(node = fdt_add_subnode(fdt, 0, xive->nodename));    
> >>
> >> and use the helper routine here.
> >>  
> >>>      _FDT(fdt_setprop_string(fdt, node, "device_type", "power-ivpe"));
> >>>      _FDT(fdt_setprop(fdt, node, "reg", timas, sizeof(timas)));
> >>> diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h
> >>> index 29dafead9ba0..deea34b03ee5 100644
> >>> --- a/include/hw/ppc/spapr_xive.h
> >>> +++ b/include/hw/ppc/spapr_xive.h
> >>> @@ -26,6 +26,9 @@ typedef struct sPAPRXive {
> >>>      XiveENDSource end_source;
> >>>      hwaddr        end_base;
> >>>  
> >>> +    /* DT */
> >>> +    gchar *nodename;    
> >>
> >> I don't think this is needed. See other patches for why.
> >>
> >> Thanks,
> >>
> >> C.
> >>    
> >>>
> >>>      /* Routing table */
> >>>      XiveEAS       *eat;
> >>>      uint32_t      nr_irqs;
> >>>     
> >>  
> >   
> 

Re: [Qemu-devel] [PATCH v3 02/19] spapr: Rename xics to intc in interrupt controller agnostic code

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 5562 bytes --]

On Thu, Jan 17, 2019 at 06:14:39PM +0100, Greg Kurz wrote:
> All this code is used with both the XICS and XIVE interrupt controllers.
> 
> Signed-off-by: Greg Kurz <groug@kaod.org>
> Reviewed-by: Cédric Le Goater <clg@kaod.org>

Applied to ppc-for-4.0, thanks.

> ---
> v3: - s/spapr_dt_intc_irq/spapr_dt_irq
> ---
>  hw/ppc/spapr.c              |    6 +++---
>  hw/ppc/spapr_events.c       |    2 +-
>  hw/ppc/spapr_pci.c          |    6 +++---
>  hw/ppc/spapr_vio.c          |    2 +-
>  include/hw/pci-host/spapr.h |    2 +-
>  include/hw/ppc/spapr.h      |    2 +-
>  6 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 83081defde4e..26f8e55cc25e 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -96,7 +96,7 @@
>  
>  #define MIN_RMA_SLOF            128UL
>  
> -#define PHANDLE_XICP            0x00001111
> +#define PHANDLE_INTC            0x00001111
>  
>  /* These two functions implement the VCPU id numbering: one to compute them
>   * all and one to identify thread 0 of a VCORE. Any change to the first one
> @@ -1276,7 +1276,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>  
>      /* /interrupt controller */
>      spapr->irq->dt_populate(spapr, spapr_max_server_number(spapr), fdt,
> -                          PHANDLE_XICP);
> +                          PHANDLE_INTC);
>  
>      ret = spapr_populate_memory(spapr, fdt);
>      if (ret < 0) {
> @@ -1296,7 +1296,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
>      }
>  
>      QLIST_FOREACH(phb, &spapr->phbs, list) {
> -        ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt,
> +        ret = spapr_populate_pci_dt(phb, PHANDLE_INTC, fdt,
>                                      spapr->irq->nr_msis);
>          if (ret < 0) {
>              error_report("couldn't setup PCI devices in fdt");
> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
> index 32719a1b72d0..b9c7ecb9e987 100644
> --- a/hw/ppc/spapr_events.c
> +++ b/hw/ppc/spapr_events.c
> @@ -282,7 +282,7 @@ void spapr_dt_events(sPAPRMachineState *spapr, void *fdt)
>              continue;
>          }
>  
> -        spapr_dt_xics_irq(interrupts, source->irq, false);
> +        spapr_dt_irq(interrupts, source->irq, false);
>  
>          _FDT(node_offset = fdt_add_subnode(fdt, event_sources, source_name));
>          _FDT(fdt_setprop(fdt, node_offset, "interrupts", interrupts,
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index b74f2632ecc6..ccdaf2c9a606 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -2066,7 +2066,7 @@ static void spapr_phb_pci_enumerate(sPAPRPHBState *phb)
>  
>  }
>  
> -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
>                            uint32_t nr_msis)
>  {
>      int bus_off, i, j, ret;
> @@ -2164,8 +2164,8 @@ int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
>              irqmap[1] = 0;
>              irqmap[2] = 0;
>              irqmap[3] = cpu_to_be32(j+1);
> -            irqmap[4] = cpu_to_be32(xics_phandle);
> -            spapr_dt_xics_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
> +            irqmap[4] = cpu_to_be32(intc_phandle);
> +            spapr_dt_irq(&irqmap[5], phb->lsi_table[lsi_num].irq, true);
>          }
>      }
>      /* Write interrupt map */
> diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
> index 7e8a9ad09337..f80b70a39c46 100644
> --- a/hw/ppc/spapr_vio.c
> +++ b/hw/ppc/spapr_vio.c
> @@ -158,7 +158,7 @@ static int vio_make_devnode(VIOsPAPRDevice *dev,
>      if (dev->irq) {
>          uint32_t ints_prop[2];
>  
> -        spapr_dt_xics_irq(ints_prop, dev->irq, false);
> +        spapr_dt_irq(ints_prop, dev->irq, false);
>          ret = fdt_setprop(fdt, node_off, "interrupts", ints_prop,
>                            sizeof(ints_prop));
>          if (ret < 0) {
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 4eb3a2ce3eb8..e0e683c32469 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -113,7 +113,7 @@ static inline qemu_irq spapr_phb_lsi_qirq(struct sPAPRPHBState *phb, int pin)
>      return spapr_qirq(spapr, phb->lsi_table[pin].irq);
>  }
>  
> -int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t xics_phandle, void *fdt,
> +int spapr_populate_pci_dt(sPAPRPHBState *phb, uint32_t intc_phandle, void *fdt,
>                            uint32_t nr_msis);
>  
>  void spapr_pci_rtas_init(void);
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index 9e01a5a12e4a..f988dc6924b8 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -682,7 +682,7 @@ void spapr_load_rtas(sPAPRMachineState *spapr, void *fdt, hwaddr addr);
>   * "interrupt-controller" node has its "#interrupt-cells" property set to 2 (ie,
>   * VIO devices, RTAS event sources and PHBs).
>   */
> -static inline void spapr_dt_xics_irq(uint32_t *intspec, int irq, bool is_lsi)
> +static inline void spapr_dt_irq(uint32_t *intspec, int irq, bool is_lsi)
>  {
>      intspec[0] = cpu_to_be32(irq);
>      intspec[1] = is_lsi ? cpu_to_be32(1) : 0;
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

On Thu, Jan 17, 2019 at 06:14:46PM +0100, Greg Kurz wrote:
> The current code assumes that an interrupt is allocated as soon as its
> type is set to MSI or LSI. PHB hotplug will require to be able to set
> the type of an interrupt before actually allocating it.

Hm.. why?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

On Tue, 5 Feb 2019 17:13:46 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Thu, Jan 17, 2019 at 06:14:46PM +0100, Greg Kurz wrote:
> > The current code assumes that an interrupt is allocated as soon as its
> > type is set to MSI or LSI. PHB hotplug will require to be able to set
> > the type of an interrupt before actually allocating it.  
> 
> Hm.. why?
> 

The justification for that is given in patch 6 actually:

Every PHB needs to claim 4 LSIs to support legacy PCI devices. This is
currently done at PHB realize. When using in-kernel XICS (or upcoming
in-kernel XIVE), QEMU synchronizes the state of all irqs, including
these LSIs, later on at machine reset.

In order to support PHB hotplug, we need a way to tell KVM about the
LSIs that doesn't require a machine reset. Since these irq numbers are
fixed values derived from the PHB index, let's identify them all at
machine init. Older machines that don't have fixed irq numbers cannot
support PHB hotplug and keep the existing behavior.



FYI, I'm currently reworking that part entirely. Maybe not worth wasting to
much time on reviewing this v3.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-devel] [PATCH v3 03/19] xics: Disintricate allocation and type setting of interrupts

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

On Tue, Feb 05, 2019 at 03:59:15PM +0100, Greg Kurz wrote:
> On Tue, 5 Feb 2019 17:13:46 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Thu, Jan 17, 2019 at 06:14:46PM +0100, Greg Kurz wrote:
> > > The current code assumes that an interrupt is allocated as soon as its
> > > type is set to MSI or LSI. PHB hotplug will require to be able to set
> > > the type of an interrupt before actually allocating it.  
> > 
> > Hm.. why?
> > 
> 
> The justification for that is given in patch 6 actually:
> 
> Every PHB needs to claim 4 LSIs to support legacy PCI devices. This is
> currently done at PHB realize. When using in-kernel XICS (or upcoming
> in-kernel XIVE), QEMU synchronizes the state of all irqs, including
> these LSIs, later on at machine reset.
> 
> In order to support PHB hotplug, we need a way to tell KVM about the
> LSIs that doesn't require a machine reset. Since these irq numbers are
> fixed values derived from the PHB index, let's identify them all at
> machine init. Older machines that don't have fixed irq numbers cannot
> support PHB hotplug and keep the existing behavior.

Sounds good.

> FYI, I'm currently reworking that part entirely. Maybe not worth wasting to
> much time on reviewing this v3.

Ok, I have plenty of other stuff to review, so I'll wait for the next spin.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]