[PATCH v2 00/16] powernv: introduce pnv-phb base/proxy devices

[PATCH v2 00/16] powernv: introduce pnv-phb base/proxy devices

[Daniel Henrique Barboza]

Hi,

This v2 is considerable different from the first version due to the
review provided by Mark Cave-Ayland.

We're now preserving all PnvPHB3/4/5 implementations already in place.
The PnvPHB device now acts as a base/proxy of the existing PHBs, which
are turned into backends of the base PnvPHB device.

QOM is being more used this time by passing through properties to the
PHB backends from the base device, and by setting the phb->version
via global machine properties in each machine.

The changes made impact both user creatable and default devices, meaning
that now the powernv machines are using the PnvPHB base device in all
circunstances.

The one thing that I didn't change from v1 is the root port revamp. I
didn't find enough reason to do the same thing we did with the PnvPHBs,
given that all that differs them is the reset() callback of
phb4_root_port. This means that patches 14-17 from v1 are still mostly
the same.


Changes from v1:
- lots of changes in patches 1-6 and 7 due to the change of direction
- patch 10 from v1: removed
- PnvPHB.version is now being removed in patch 16
- several other minor changes due to changes in the initial patches
- v1 link: https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg01410.html

Daniel Henrique Barboza (16):
  ppc/pnv: add PHB3 bus init helper
  ppc/pnv: add pnv_get_phb3_child()
  ppc/pnv: add PnvPHB base/proxy device
  ppc/pnv: change PnvPHB3 to be a PnvPHB backend
  ppc/pnv: user created pnv-phb for powernv8
  ppc/pnv: add PHB4 bus init helper
  ppc/pnv: change PnvPHB4 to be a PnvPHB backend
  ppc/pnv: user created pnv-phb for powernv9
  ppc/pnv: change pnv_phb4_get_pec() to also retrieve chip10->pecs
  ppc/pnv: user creatable pnv-phb for powernv10
  ppc/pnv: add pnv-phb-root-port device
  ppc/pnv: remove pnv-phb3-root-port
  ppc/pnv: remove pnv-phb4-root-port
  ppc/pnv: remove 'phb_rootport_typename' in pnv_phb_realize()
  ppc/pnv: remove pecc->rp_model
  ppc/pnv: remove PnvPHB4.version

 hw/pci-host/meson.build        |   3 +-
 hw/pci-host/pnv_phb.c          | 219 ++++++++++++++++++++++++++++++++
 hw/pci-host/pnv_phb.h          |  56 ++++++++
 hw/pci-host/pnv_phb3.c         | 144 ++++++++-------------
 hw/pci-host/pnv_phb4.c         | 226 ++++++++++++++-------------------
 hw/pci-host/pnv_phb4_pec.c     |  14 +-
 hw/ppc/pnv.c                   |  78 ++++++++++--
 include/hw/pci-host/pnv_phb3.h |  12 +-
 include/hw/pci-host/pnv_phb4.h |  18 +--
 include/hw/ppc/pnv.h           |   4 +-
 10 files changed, 512 insertions(+), 262 deletions(-)
 create mode 100644 hw/pci-host/pnv_phb.c
 create mode 100644 hw/pci-host/pnv_phb.h

-- 
2.36.1

[PATCH v2 01/16] ppc/pnv: add PHB3 bus init helper

[Daniel Henrique Barboza]

The PnvPHB3 bus init consists of initializing the pci_io and pci_mmio
regions, registering it via pci_register_root_bus() and then setup the
iommu.

We'll want to init the bus from outside pnv_phb3.c when the bus is
removed from the PnvPHB3 device and put into a new parent PnvPHB device.
The new pnv_phb3_bus_init() helper will be used by the parent to init
the bus when using the PHB3 backend.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb3.c         | 40 +++++++++++++++++++---------------
 include/hw/pci-host/pnv_phb3.h |  1 +
 2 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index 3f03467dde..60584e2aae 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -986,10 +986,31 @@ static void pnv_phb3_instance_init(Object *obj)
 
 }
 
+void pnv_phb3_bus_init(DeviceState *dev, PnvPHB3 *phb)
+{
+    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
+
+    /*
+     * PHB3 doesn't support IO space. However, qemu gets very upset if
+     * we don't have an IO region to anchor IO BARs onto so we just
+     * initialize one which we never hook up to anything
+     */
+    memory_region_init(&phb->pci_io, OBJECT(phb), "pci-io", 0x10000);
+    memory_region_init(&phb->pci_mmio, OBJECT(phb), "pci-mmio",
+                       PCI_MMIO_TOTAL_SIZE);
+
+    pci->bus = pci_register_root_bus(dev,
+                                     dev->id ? dev->id : NULL,
+                                     pnv_phb3_set_irq, pnv_phb3_map_irq, phb,
+                                     &phb->pci_mmio, &phb->pci_io,
+                                     0, 4, TYPE_PNV_PHB3_ROOT_BUS);
+
+    pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb);
+}
+
 static void pnv_phb3_realize(DeviceState *dev, Error **errp)
 {
     PnvPHB3 *phb = PNV_PHB3(dev);
-    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
     PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
     int i;
 
@@ -1035,22 +1056,7 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
     memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb3_reg_ops, phb,
                           "phb3-regs", 0x1000);
 
-    /*
-     * PHB3 doesn't support IO space. However, qemu gets very upset if
-     * we don't have an IO region to anchor IO BARs onto so we just
-     * initialize one which we never hook up to anything
-     */
-    memory_region_init(&phb->pci_io, OBJECT(phb), "pci-io", 0x10000);
-    memory_region_init(&phb->pci_mmio, OBJECT(phb), "pci-mmio",
-                       PCI_MMIO_TOTAL_SIZE);
-
-    pci->bus = pci_register_root_bus(dev,
-                                     dev->id ? dev->id : NULL,
-                                     pnv_phb3_set_irq, pnv_phb3_map_irq, phb,
-                                     &phb->pci_mmio, &phb->pci_io,
-                                     0, 4, TYPE_PNV_PHB3_ROOT_BUS);
-
-    pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb);
+    pnv_phb3_bus_init(dev, phb);
 
     pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
 }
diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
index af6ec83cf6..1375f18fc1 100644
--- a/include/hw/pci-host/pnv_phb3.h
+++ b/include/hw/pci-host/pnv_phb3.h
@@ -164,5 +164,6 @@ uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size);
 void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size);
 void pnv_phb3_update_regions(PnvPHB3 *phb);
 void pnv_phb3_remap_irqs(PnvPHB3 *phb);
+void pnv_phb3_bus_init(DeviceState *dev, PnvPHB3 *phb);
 
 #endif /* PCI_HOST_PNV_PHB3_H */
-- 
2.36.1

[PATCH v2 02/16] ppc/pnv: add pnv_get_phb3_child()

[Daniel Henrique Barboza]

The ics and pic related functions in pnv.c relies in the fact that the
child device of the chip is always a PnvPHB3 object. This will change in
the next patches, and the PHB3 will be a child of another device that
will be attached to the pnv8 chip.

To ease the amount of changes done later, on let's create a helper that
retrieves the PnvPHB3 pointer from the child object in these functions.
After that we'll have a single place to change when the chip starts
using the upcoming device instead.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/ppc/pnv.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 7c08a78d6c..aaf4d241c3 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -652,10 +652,15 @@ static ISABus *pnv_isa_create(PnvChip *chip, Error **errp)
     return PNV_CHIP_GET_CLASS(chip)->isa_create(chip, errp);
 }
 
+static PnvPHB3 *pnv_get_phb3_child(Object *child)
+{
+    return (PnvPHB3 *)object_dynamic_cast(child, TYPE_PNV_PHB3);
+}
+
 static int pnv_chip_power8_pic_print_info_child(Object *child, void *opaque)
 {
     Monitor *mon = opaque;
-    PnvPHB3 *phb3 = (PnvPHB3 *) object_dynamic_cast(child, TYPE_PNV_PHB3);
+    PnvPHB3 *phb3 = pnv_get_phb3_child(child);
 
     if (phb3) {
         pnv_phb3_msi_pic_print_info(&phb3->msis, mon);
@@ -1942,7 +1947,7 @@ typedef struct ForeachPhb3Args {
 static int pnv_ics_get_child(Object *child, void *opaque)
 {
     ForeachPhb3Args *args = opaque;
-    PnvPHB3 *phb3 = (PnvPHB3 *) object_dynamic_cast(child, TYPE_PNV_PHB3);
+    PnvPHB3 *phb3 = pnv_get_phb3_child(child);
 
     if (phb3) {
         if (ics_valid_irq(&phb3->lsis, args->irq)) {
@@ -1992,7 +1997,7 @@ PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id)
 
 static int pnv_ics_resend_child(Object *child, void *opaque)
 {
-    PnvPHB3 *phb3 = (PnvPHB3 *) object_dynamic_cast(child, TYPE_PNV_PHB3);
+    PnvPHB3 *phb3 = pnv_get_phb3_child(child);
 
     if (phb3) {
         ics_resend(&phb3->lsis);
-- 
2.36.1

[PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Daniel Henrique Barboza]

The PnvPHB device is going to be the base device for all other powernv
PHBs. It consists of a device that has the same user API as the other
PHB, namely being a PCIHostBridge and having chip-id and index
properties. It also has a 'backend' pointer that will be initialized
with the PHB implementation that the device is going to use.

The initialization of the PHB backend is done by checking the PHB
version via a 'version' attribute that can be set via a global machine
property.  The 'version' field will be used to make adjustments based on
the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
helper will be added later on.

For now let's add the basic logic of the PnvPHB object, which consists
mostly of pnv_phb_realize() doing all the work of checking the
phb->version set, initializing the proper backend, passing through its
attributes to the chosen backend, finalizing the backend realize and
adding a root port in the end.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/meson.build |   3 +-
 hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
 hw/pci-host/pnv_phb.h   |  39 +++++++++++++
 3 files changed, 164 insertions(+), 1 deletion(-)
 create mode 100644 hw/pci-host/pnv_phb.c
 create mode 100644 hw/pci-host/pnv_phb.h

diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
index c07596d0d1..e832babc9d 100644
--- a/hw/pci-host/meson.build
+++ b/hw/pci-host/meson.build
@@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
   'pnv_phb3_msi.c',
   'pnv_phb3_pbcq.c',
   'pnv_phb4.c',
-  'pnv_phb4_pec.c'
+  'pnv_phb4_pec.c',
+  'pnv_phb.c',
 ))
diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
new file mode 100644
index 0000000000..fa8472622f
--- /dev/null
+++ b/hw/pci-host/pnv_phb.c
@@ -0,0 +1,123 @@
+/*
+ * QEMU PowerPC PowerNV Proxy PHB model
+ *
+ * Copyright (c) 2022, IBM Corporation.
+ *
+ * This code is licensed under the GPL version 2 or later. See the
+ * COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/visitor.h"
+#include "qapi/error.h"
+#include "hw/pci-host/pnv_phb.h"
+#include "hw/pci-host/pnv_phb3.h"
+#include "hw/pci-host/pnv_phb4.h"
+#include "hw/ppc/pnv.h"
+#include "hw/qdev-properties.h"
+#include "qom/object.h"
+
+
+static void pnv_phb_realize(DeviceState *dev, Error **errp)
+{
+    PnvPHB *phb = PNV_PHB(dev);
+    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
+    g_autofree char *phb_typename = NULL;
+    g_autofree char *phb_rootport_typename = NULL;
+
+    if (!phb->version) {
+        error_setg(errp, "version not specified");
+        return;
+    }
+
+    switch (phb->version) {
+    case 3:
+        phb_typename = g_strdup(TYPE_PNV_PHB3);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
+        break;
+    case 4:
+        phb_typename = g_strdup(TYPE_PNV_PHB4);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
+        break;
+    case 5:
+        phb_typename = g_strdup(TYPE_PNV_PHB5);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    phb->backend = object_new(phb_typename);
+    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
+
+    /* Passthrough child device properties to the proxy device */
+    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
+    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
+    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
+
+    if (phb->version == 3) {
+        object_property_set_link(phb->backend, "chip",
+                                 OBJECT(phb->chip), errp);
+    } else {
+        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
+    }
+
+    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
+        return;
+    }
+
+    if (phb->version == 3) {
+        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
+    }
+
+    pnv_phb_attach_root_port(pci, phb_rootport_typename);
+}
+
+static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
+                                         PCIBus *rootbus)
+{
+    PnvPHB *phb = PNV_PHB(host_bridge);
+
+    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
+             phb->chip_id, phb->phb_id);
+    return phb->bus_path;
+}
+
+static Property pnv_phb_properties[] = {
+        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
+        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
+        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
+
+        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
+
+        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
+                         PnvPhb4PecState *),
+
+        DEFINE_PROP_END_OF_LIST(),
+};
+
+static void pnv_phb_class_init(ObjectClass *klass, void *data)
+{
+    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    hc->root_bus_path = pnv_phb_root_bus_path;
+    dc->realize = pnv_phb_realize;
+    device_class_set_props(dc, pnv_phb_properties);
+    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+    dc->user_creatable = true;
+}
+
+static void pnv_phb_register_type(void)
+{
+    static const TypeInfo pnv_phb_type_info = {
+        .name          = TYPE_PNV_PHB,
+        .parent        = TYPE_PCIE_HOST_BRIDGE,
+        .instance_size = sizeof(PnvPHB),
+        .class_init    = pnv_phb_class_init,
+    };
+
+    type_register_static(&pnv_phb_type_info);
+}
+type_init(pnv_phb_register_type)
diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
new file mode 100644
index 0000000000..a7cc8610e2
--- /dev/null
+++ b/hw/pci-host/pnv_phb.h
@@ -0,0 +1,39 @@
+/*
+ * QEMU PowerPC PowerNV Proxy PHB model
+ *
+ * Copyright (c) 2022, IBM Corporation.
+ *
+ * This code is licensed under the GPL version 2 or later. See the
+ * COPYING file in the top-level directory.
+ */
+
+#ifndef PCI_HOST_PNV_PHB_H
+#define PCI_HOST_PNV_PHB_H
+
+#include "hw/pci/pcie_host.h"
+#include "hw/pci/pcie_port.h"
+#include "qom/object.h"
+
+typedef struct PnvChip PnvChip;
+typedef struct PnvPhb4PecState PnvPhb4PecState;
+
+struct PnvPHB {
+    PCIExpressHost parent_obj;
+
+    uint32_t chip_id;
+    uint32_t phb_id;
+    uint32_t version;
+    char bus_path[8];
+
+    PnvChip *chip;
+
+    PnvPhb4PecState *pec;
+
+    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
+    Object *backend;
+};
+
+#define TYPE_PNV_PHB "pnv-phb"
+OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
+
+#endif /* PCI_HOST_PNV_PHB_H */
-- 
2.36.1

[PATCH v2 04/16] ppc/pnv: change PnvPHB3 to be a PnvPHB backend

[Daniel Henrique Barboza]

We need a handful of changes that needs to be done in a single swoop to
turn PnvPHB3 into a PnvPHB backend.

In the PnvPHB3, since the PnvPHB device implements PCIExpressHost and
will hold the PCI bus, change PnvPHB3 parent to TYPE_DEVICE. There are a
couple of instances in pnv_phb3.c that needs to access the PCI bus, so a
phb_base pointer is added to allow access to the parent PnvPHB. The
PnvPHB3 root port will now be connected to a PnvPHB object.

In pnv.c, the powernv8 machine chip8 will now hold an array of PnvPHB
objects.  pnv_get_phb3_child() needs to be adapted to return the PnvPHB3
backend from the PnvPHB child. A global property is added in
pnv_machine_power8_class_init() to ensure that all PnvPHBs are created
with phb->version = 3.

After all these changes we're still able to boot a powernv8 machine with
default settings. The real gain will come with user created PnvPHB
devices, coming up next.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb3.c         | 29 ++++++++---------------------
 hw/ppc/pnv.c                   | 21 +++++++++++++++++----
 include/hw/pci-host/pnv_phb3.h |  5 ++++-
 include/hw/ppc/pnv.h           |  3 ++-
 4 files changed, 31 insertions(+), 27 deletions(-)

diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index 60584e2aae..a39aa0e8c4 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -11,6 +11,7 @@
 #include "qapi/visitor.h"
 #include "qapi/error.h"
 #include "hw/pci-host/pnv_phb3_regs.h"
+#include "hw/pci-host/pnv_phb.h"
 #include "hw/pci-host/pnv_phb3.h"
 #include "hw/pci/pcie_host.h"
 #include "hw/pci/pcie_port.h"
@@ -26,7 +27,7 @@
 
 static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb)
 {
-    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
+    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
     uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
     uint8_t bus, devfn;
 
@@ -590,7 +591,7 @@ void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size)
 uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size)
 {
     PnvPHB3 *phb = opaque;
-    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
+    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
     uint64_t val;
 
     if ((off & 0xfffc) == PHB_CONFIG_DATA) {
@@ -1057,8 +1058,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
                           "phb3-regs", 0x1000);
 
     pnv_phb3_bus_init(dev, phb);
-
-    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
 }
 
 void pnv_phb3_update_regions(PnvPHB3 *phb)
@@ -1083,38 +1082,26 @@ void pnv_phb3_update_regions(PnvPHB3 *phb)
     pnv_phb3_check_all_m64s(phb);
 }
 
-static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge,
-                                          PCIBus *rootbus)
-{
-    PnvPHB3 *phb = PNV_PHB3(host_bridge);
-
-    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
-             phb->chip_id, phb->phb_id);
-    return phb->bus_path;
-}
-
 static Property pnv_phb3_properties[] = {
         DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0),
         DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0),
         DEFINE_PROP_LINK("chip", PnvPHB3, chip, TYPE_PNV_CHIP, PnvChip *),
+        DEFINE_PROP_LINK("phb-base", PnvPHB3, phb_base, TYPE_PNV_PHB, PnvPHB *),
         DEFINE_PROP_END_OF_LIST(),
 };
 
 static void pnv_phb3_class_init(ObjectClass *klass, void *data)
 {
-    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
-    hc->root_bus_path = pnv_phb3_root_bus_path;
     dc->realize = pnv_phb3_realize;
     device_class_set_props(dc, pnv_phb3_properties);
-    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
     dc->user_creatable = false;
 }
 
 static const TypeInfo pnv_phb3_type_info = {
     .name          = TYPE_PNV_PHB3,
-    .parent        = TYPE_PCIE_HOST_BRIDGE,
+    .parent = TYPE_DEVICE,
     .instance_size = sizeof(PnvPHB3),
     .class_init    = pnv_phb3_class_init,
     .instance_init = pnv_phb3_instance_init,
@@ -1146,11 +1133,11 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)
     PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
     PCIDevice *pci = PCI_DEVICE(dev);
     PCIBus *bus = pci_get_bus(pci);
-    PnvPHB3 *phb = NULL;
+    PnvPHB *phb = NULL;
     Error *local_err = NULL;
 
-    phb = (PnvPHB3 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
-                                          TYPE_PNV_PHB3);
+    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
+                                         TYPE_PNV_PHB);
 
     if (!phb) {
         error_setg(errp,
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index aaf4d241c3..6cd0af9adf 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -43,6 +43,7 @@
 #include "hw/ipmi/ipmi.h"
 #include "target/ppc/mmu-hash64.h"
 #include "hw/pci/msi.h"
+#include "hw/pci-host/pnv_phb.h"
 
 #include "hw/ppc/xics.h"
 #include "hw/qdev-properties.h"
@@ -654,7 +655,13 @@ static ISABus *pnv_isa_create(PnvChip *chip, Error **errp)
 
 static PnvPHB3 *pnv_get_phb3_child(Object *child)
 {
-    return (PnvPHB3 *)object_dynamic_cast(child, TYPE_PNV_PHB3);
+    PnvPHB *phb = (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
+
+    if (!phb) {
+        return NULL;
+    }
+
+    return (PnvPHB3 *)phb->backend;
 }
 
 static int pnv_chip_power8_pic_print_info_child(Object *child, void *opaque)
@@ -1160,7 +1167,7 @@ static void pnv_chip_power8_instance_init(Object *obj)
     chip8->num_phbs = pcc->num_phbs;
 
     for (i = 0; i < chip8->num_phbs; i++) {
-        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB3);
+        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB);
     }
 
 }
@@ -1282,9 +1289,9 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp)
     memory_region_add_subregion(get_system_memory(), PNV_HOMER_BASE(chip),
                                 &chip8->homer.regs);
 
-    /* PHB3 controllers */
+    /* PHB controllers */
     for (i = 0; i < chip8->num_phbs; i++) {
-        PnvPHB3 *phb = &chip8->phbs[i];
+        PnvPHB *phb = &chip8->phbs[i];
 
         object_property_set_int(OBJECT(phb), "index", i, &error_fatal);
         object_property_set_int(OBJECT(phb), "chip-id", chip->chip_id,
@@ -1957,6 +1964,7 @@ static int pnv_ics_get_child(Object *child, void *opaque)
             args->ics = ICS(&phb3->msis);
         }
     }
+
     return args->ics ? 1 : 0;
 }
 
@@ -2112,8 +2120,13 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
     PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
     static const char compat[] = "qemu,powernv8\0qemu,powernv\0ibm,powernv";
 
+    static GlobalProperty phb_compat[] = {
+        { TYPE_PNV_PHB, "version", "3" },
+    };
+
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
+    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
 
     xic->icp_get = pnv_icp_get;
     xic->ics_get = pnv_ics_get;
diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
index 1375f18fc1..3b9ff1096a 100644
--- a/include/hw/pci-host/pnv_phb3.h
+++ b/include/hw/pci-host/pnv_phb3.h
@@ -14,6 +14,7 @@
 #include "hw/pci/pcie_port.h"
 #include "hw/ppc/xics.h"
 #include "qom/object.h"
+#include "hw/pci-host/pnv_phb.h"
 
 typedef struct PnvPHB3 PnvPHB3;
 typedef struct PnvChip PnvChip;
@@ -127,7 +128,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB3, PNV_PHB3)
 #define PCI_MMIO_TOTAL_SIZE   (0x1ull << 60)
 
 struct PnvPHB3 {
-    PCIExpressHost parent_obj;
+    DeviceState parent;
+
+    PnvPHB *phb_base;
 
     uint32_t chip_id;
     uint32_t phb_id;
diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 86cb7d7f97..4595db418e 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -32,6 +32,7 @@
 #include "hw/ppc/pnv_core.h"
 #include "hw/pci-host/pnv_phb3.h"
 #include "hw/pci-host/pnv_phb4.h"
+#include "hw/pci-host/pnv_phb.h"
 #include "qom/object.h"
 
 #define TYPE_PNV_CHIP "pnv-chip"
@@ -80,7 +81,7 @@ struct Pnv8Chip {
     PnvHomer     homer;
 
 #define PNV8_CHIP_PHB3_MAX 4
-    PnvPHB3      phbs[PNV8_CHIP_PHB3_MAX];
+    PnvPHB       phbs[PNV8_CHIP_PHB3_MAX];
     uint32_t     num_phbs;
 
     XICSFabric    *xics;
-- 
2.36.1

[PATCH v2 05/16] ppc/pnv: user created pnv-phb for powernv8

[Daniel Henrique Barboza]

Let's reintroduce the powernv8 bits of the code what was removed in
commit 9c10d86fee "ppc/pnv: Remove user-created PHB{3,4,5} devices",
allowing us to enable user creatable pnv-phb devices for the powernv8
machine.

The difference is that this time we're adding support for a PnvPHB
device that is the same that will be used by the other powernv machines,
allowing the user to deal with a single PnvPHB device instead of
versioned PHBs for each one.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c  |  5 ++++-
 hw/pci-host/pnv_phb3.c | 26 +++++++++++++++++++++++++-
 hw/ppc/pnv.c           | 23 ++++++++++++++++++++++-
 include/hw/ppc/pnv.h   |  1 +
 4 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index fa8472622f..17532d25f0 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -17,6 +17,7 @@
 #include "hw/ppc/pnv.h"
 #include "hw/qdev-properties.h"
 #include "qom/object.h"
+#include "sysemu/sysemu.h"
 
 
 static void pnv_phb_realize(DeviceState *dev, Error **errp)
@@ -71,7 +72,9 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
         pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
     }
 
-    pnv_phb_attach_root_port(pci, phb_rootport_typename);
+    if (defaults_enabled()) {
+        pnv_phb_attach_root_port(pci, phb_rootport_typename);
+    }
 }
 
 static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index a39aa0e8c4..839c2dad00 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -1015,6 +1015,30 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
     PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
     int i;
 
+    /* User created devices */
+    if (!phb->chip) {
+        Error *local_err = NULL;
+        BusState *s;
+
+        phb->chip = pnv_get_chip(pnv, phb->chip_id);
+        if (!phb->chip) {
+            error_setg(errp, "invalid chip id: %d", phb->chip_id);
+            return;
+        }
+
+        /*
+         * Reparent user created devices to the chip to build
+         * correctly the device tree.
+         */
+        pnv_chip_parent_fixup(phb->chip, OBJECT(phb->phb_base), phb->phb_id);
+
+        s = qdev_get_parent_bus(DEVICE(phb->chip));
+        if (!qdev_set_parent_bus(DEVICE(phb->phb_base), s, &local_err)) {
+            error_propagate(errp, local_err);
+            return;
+        }
+    }
+
     if (phb->phb_id >= PNV_CHIP_GET_CLASS(phb->chip)->num_phbs) {
         error_setg(errp, "invalid PHB index: %d", phb->phb_id);
         return;
@@ -1167,7 +1191,7 @@ static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data)
 
     device_class_set_parent_realize(dc, pnv_phb3_root_port_realize,
                                     &rpc->parent_realize);
-    dc->user_creatable = false;
+    dc->user_creatable = true;
 
     k->vendor_id = PCI_VENDOR_ID_IBM;
     k->device_id = 0x03dc;
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 6cd0af9adf..081b6839cc 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -1164,7 +1164,9 @@ static void pnv_chip_power8_instance_init(Object *obj)
 
     object_initialize_child(obj, "homer", &chip8->homer, TYPE_PNV8_HOMER);
 
-    chip8->num_phbs = pcc->num_phbs;
+    if (defaults_enabled()) {
+        chip8->num_phbs = pcc->num_phbs;
+    }
 
     for (i = 0; i < chip8->num_phbs; i++) {
         object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB);
@@ -1990,6 +1992,23 @@ static ICSState *pnv_ics_get(XICSFabric *xi, int irq)
     return NULL;
 }
 
+void pnv_chip_parent_fixup(PnvChip *chip, Object *obj, int index)
+{
+    Object *parent = OBJECT(chip);
+    g_autofree char *default_id =
+        g_strdup_printf("%s[%d]", object_get_typename(obj), index);
+
+    if (obj->parent == parent) {
+        return;
+    }
+
+    object_ref(obj);
+    object_unparent(obj);
+    object_property_add_child(
+        parent, DEVICE(obj)->id ? DEVICE(obj)->id : default_id, obj);
+    object_unref(obj);
+}
+
 PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id)
 {
     int i;
@@ -2134,6 +2153,8 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
+
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
 }
 
 static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 4595db418e..fc95b8cfaa 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -191,6 +191,7 @@ DECLARE_INSTANCE_CHECKER(PnvChip, PNV_CHIP_POWER10,
 
 PowerPCCPU *pnv_chip_find_cpu(PnvChip *chip, uint32_t pir);
 void pnv_phb_attach_root_port(PCIHostState *pci, const char *name);
+void pnv_chip_parent_fixup(PnvChip *chip, Object *obj, int index);
 
 #define TYPE_PNV_MACHINE       MACHINE_TYPE_NAME("powernv")
 typedef struct PnvMachineClass PnvMachineClass;
-- 
2.36.1

[PATCH v2 06/16] ppc/pnv: add PHB4 bus init helper

[Daniel Henrique Barboza]

Similar to what we already did for the PnvPHB3 device, let's add a
helper to init the bus when using a PnvPHB4. This helper will be used by
PnvPHb when PnvPHB4 turns into a backend.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c          |  2 ++
 hw/pci-host/pnv_phb4.c         | 34 ++++++++++++++++++++--------------
 include/hw/pci-host/pnv_phb4.h |  1 +
 3 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index 17532d25f0..321c4e768a 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -70,6 +70,8 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
 
     if (phb->version == 3) {
         pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
+    } else {
+        pnv_phb4_bus_init(dev, (PnvPHB4 *)phb->backend);
     }
 
     if (defaults_enabled()) {
diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index 13ba9e45d8..ae5494fe72 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -1544,29 +1544,16 @@ static void pnv_phb4_instance_init(Object *obj)
     object_initialize_child(obj, "source", &phb->xsrc, TYPE_XIVE_SOURCE);
 }
 
-static void pnv_phb4_realize(DeviceState *dev, Error **errp)
+void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
 {
-    PnvPHB4 *phb = PNV_PHB4(dev);
     PCIHostState *pci = PCI_HOST_BRIDGE(dev);
-    XiveSource *xsrc = &phb->xsrc;
-    int nr_irqs;
     char name[32];
 
-    /* Set the "big_phb" flag */
-    phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
-
-    /* Controller Registers */
-    snprintf(name, sizeof(name), "phb4-%d.%d-regs", phb->chip_id,
-             phb->phb_id);
-    memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb4_reg_ops, phb,
-                          name, 0x2000);
-
     /*
      * PHB4 doesn't support IO space. However, qemu gets very upset if
      * we don't have an IO region to anchor IO BARs onto so we just
      * initialize one which we never hook up to anything
      */
-
     snprintf(name, sizeof(name), "phb4-%d.%d-pci-io", phb->chip_id,
              phb->phb_id);
     memory_region_init(&phb->pci_io, OBJECT(phb), name, 0x10000);
@@ -1582,6 +1569,25 @@ static void pnv_phb4_realize(DeviceState *dev, Error **errp)
                                      0, 4, TYPE_PNV_PHB4_ROOT_BUS);
     pci_setup_iommu(pci->bus, pnv_phb4_dma_iommu, phb);
     pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE;
+}
+
+static void pnv_phb4_realize(DeviceState *dev, Error **errp)
+{
+    PnvPHB4 *phb = PNV_PHB4(dev);
+    XiveSource *xsrc = &phb->xsrc;
+    int nr_irqs;
+    char name[32];
+
+    /* Set the "big_phb" flag */
+    phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
+
+    /* Controller Registers */
+    snprintf(name, sizeof(name), "phb4-%d.%d-regs", phb->chip_id,
+             phb->phb_id);
+    memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb4_reg_ops, phb,
+                          name, 0x2000);
+
+    pnv_phb4_bus_init(dev, phb);
 
     /* Setup XIVE Source */
     if (phb->big_phb) {
diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
index 19dcbd6f87..90843ac3a9 100644
--- a/include/hw/pci-host/pnv_phb4.h
+++ b/include/hw/pci-host/pnv_phb4.h
@@ -157,6 +157,7 @@ struct PnvPHB4 {
 
 void pnv_phb4_pic_print_info(PnvPHB4 *phb, Monitor *mon);
 int pnv_phb4_pec_get_phb_id(PnvPhb4PecState *pec, int stack_index);
+void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb);
 extern const MemoryRegionOps pnv_phb4_xscom_ops;
 
 /*
-- 
2.36.1

[PATCH v2 07/16] ppc/pnv: change PnvPHB4 to be a PnvPHB backend

[Daniel Henrique Barboza]

Change the parent type of the PnvPHB4 device to TYPE_PARENT since the
PCI bus is going to be initialized by the PnvPHB parent. Functions that
needs to access the bus via a PnvPHB4 object can do so via the
phb4->phb_base pointer.

pnv_phb4_pec now creates a PnvPHB object.

The powernv9 machine class will create PnvPHB devices with version '4'.
powernv10 will create using version '5'. Both are using global machine
properties in their class_init() to do that.

These changes will benefit us when adding PnvPHB user creatable devices
for powernv9 and powernv10.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb4.c         | 29 +++++++++--------------------
 hw/pci-host/pnv_phb4_pec.c     |  6 +-----
 hw/ppc/pnv.c                   | 20 +++++++++++++++++++-
 include/hw/pci-host/pnv_phb4.h |  5 ++++-
 4 files changed, 33 insertions(+), 27 deletions(-)

diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index ae5494fe72..22cf1c2a5e 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -49,7 +49,7 @@ static inline uint64_t SETFIELD(uint64_t mask, uint64_t word,
 
 static PCIDevice *pnv_phb4_find_cfg_dev(PnvPHB4 *phb)
 {
-    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
+    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
     uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
     uint8_t bus, devfn;
 
@@ -145,7 +145,7 @@ static uint64_t pnv_phb4_config_read(PnvPHB4 *phb, unsigned off,
 static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
                                      unsigned size, uint64_t val)
 {
-    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
+    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
     PCIDevice *pdev;
 
     if (size != 4) {
@@ -166,7 +166,7 @@ static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
 static uint64_t pnv_phb4_rc_config_read(PnvPHB4 *phb, unsigned off,
                                         unsigned size)
 {
-    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
+    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
     PCIDevice *pdev;
     uint64_t val;
 
@@ -1608,16 +1608,6 @@ static void pnv_phb4_realize(DeviceState *dev, Error **errp)
     pnv_phb4_xscom_realize(phb);
 }
 
-static const char *pnv_phb4_root_bus_path(PCIHostState *host_bridge,
-                                          PCIBus *rootbus)
-{
-    PnvPHB4 *phb = PNV_PHB4(host_bridge);
-
-    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
-             phb->chip_id, phb->phb_id);
-    return phb->bus_path;
-}
-
 /*
  * Address base trigger mode (POWER10)
  *
@@ -1702,19 +1692,18 @@ static Property pnv_phb4_properties[] = {
         DEFINE_PROP_UINT32("chip-id", PnvPHB4, chip_id, 0),
         DEFINE_PROP_LINK("pec", PnvPHB4, pec, TYPE_PNV_PHB4_PEC,
                          PnvPhb4PecState *),
+        DEFINE_PROP_LINK("phb-base", PnvPHB4, phb_base,
+                         TYPE_PNV_PHB, PnvPHB *),
         DEFINE_PROP_END_OF_LIST(),
 };
 
 static void pnv_phb4_class_init(ObjectClass *klass, void *data)
 {
-    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
     XiveNotifierClass *xfc = XIVE_NOTIFIER_CLASS(klass);
 
-    hc->root_bus_path   = pnv_phb4_root_bus_path;
     dc->realize         = pnv_phb4_realize;
     device_class_set_props(dc, pnv_phb4_properties);
-    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
     dc->user_creatable  = false;
 
     xfc->notify         = pnv_phb4_xive_notify;
@@ -1722,7 +1711,7 @@ static void pnv_phb4_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo pnv_phb4_type_info = {
     .name          = TYPE_PNV_PHB4,
-    .parent        = TYPE_PCIE_HOST_BRIDGE,
+    .parent        = TYPE_DEVICE,
     .instance_init = pnv_phb4_instance_init,
     .instance_size = sizeof(PnvPHB4),
     .class_init    = pnv_phb4_class_init,
@@ -1785,11 +1774,11 @@ static void pnv_phb4_root_port_realize(DeviceState *dev, Error **errp)
     PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
     PCIDevice *pci = PCI_DEVICE(dev);
     PCIBus *bus = pci_get_bus(pci);
-    PnvPHB4 *phb = NULL;
+    PnvPHB *phb = NULL;
     Error *local_err = NULL;
 
-    phb = (PnvPHB4 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
-                                          TYPE_PNV_PHB4);
+    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
+                                         TYPE_PNV_PHB);
 
     if (!phb) {
         error_setg(errp, "%s must be connected to pnv-phb4 buses", dev->id);
diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
index 61bc0b503e..888ecbe8f3 100644
--- a/hw/pci-host/pnv_phb4_pec.c
+++ b/hw/pci-host/pnv_phb4_pec.c
@@ -115,8 +115,7 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
                                         int stack_no,
                                         Error **errp)
 {
-    PnvPhb4PecClass *pecc = PNV_PHB4_PEC_GET_CLASS(pec);
-    PnvPHB4 *phb = PNV_PHB4(qdev_new(pecc->phb_type));
+    PnvPHB *phb = PNV_PHB(qdev_new(TYPE_PNV_PHB));
     int phb_id = pnv_phb4_pec_get_phb_id(pec, stack_no);
 
     object_property_add_child(OBJECT(pec), "phb[*]", OBJECT(phb));
@@ -130,9 +129,6 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
     if (!sysbus_realize(SYS_BUS_DEVICE(phb), errp)) {
         return;
     }
-
-    /* Add a single Root port if running with defaults */
-    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), pecc->rp_model);
 }
 
 static void pnv_pec_realize(DeviceState *dev, Error **errp)
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 081b6839cc..3b0b230e49 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -688,7 +688,14 @@ static void pnv_chip_power8_pic_print_info(PnvChip *chip, Monitor *mon)
 static int pnv_chip_power9_pic_print_info_child(Object *child, void *opaque)
 {
     Monitor *mon = opaque;
-    PnvPHB4 *phb4 = (PnvPHB4 *) object_dynamic_cast(child, TYPE_PNV_PHB4);
+    PnvPHB *phb =  (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
+    PnvPHB4 *phb4;
+
+    if (!phb) {
+        return 0;
+    }
+
+    phb4 = (PnvPHB4 *)phb->backend;
 
     if (phb4) {
         pnv_phb4_pic_print_info(phb4, mon);
@@ -2164,8 +2171,14 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
     PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
     static const char compat[] = "qemu,powernv9\0ibm,powernv";
 
+    static GlobalProperty phb_compat[] = {
+        { TYPE_PNV_PHB, "version", "4" },
+    };
+
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
+    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
+
     xfc->match_nvt = pnv_match_nvt;
 
     mc->alias = "powernv";
@@ -2182,8 +2195,13 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
     XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
     static const char compat[] = "qemu,powernv10\0ibm,powernv";
 
+    static GlobalProperty phb_compat[] = {
+        { TYPE_PNV_PHB, "version", "5" },
+    };
+
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
+    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
 
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
index 90843ac3a9..f22253358f 100644
--- a/include/hw/pci-host/pnv_phb4.h
+++ b/include/hw/pci-host/pnv_phb4.h
@@ -18,6 +18,7 @@
 typedef struct PnvPhb4PecState PnvPhb4PecState;
 typedef struct PnvPhb4PecStack PnvPhb4PecStack;
 typedef struct PnvPHB4 PnvPHB4;
+typedef struct PnvPHB PnvPHB;
 typedef struct PnvChip PnvChip;
 
 /*
@@ -78,7 +79,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB4, PNV_PHB4)
 #define PCI_MMIO_TOTAL_SIZE        (0x1ull << 60)
 
 struct PnvPHB4 {
-    PCIExpressHost parent_obj;
+    DeviceState parent;
+
+    PnvPHB *phb_base;
 
     uint32_t chip_id;
     uint32_t phb_id;
-- 
2.36.1

[PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Daniel Henrique Barboza]

To enable user creatable PnvPHB devices for powernv9 we'll revert the
powernv9 related changes made in 9c10d86fee "ppc/pnv: Remove
user-created PHB{3,4,5} devices".

This change alone isn't enough to enable user creatable devices for powernv10
due to how pnv_phb4_get_pec() currently works. For now let's just enable it
for powernv9 alone.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb4.c     | 58 +++++++++++++++++++++++++++++++++++++-
 hw/pci-host/pnv_phb4_pec.c |  6 ++--
 hw/ppc/pnv.c               |  2 ++
 3 files changed, 63 insertions(+), 3 deletions(-)

diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index 22cf1c2a5e..a5c8ae494b 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -1571,13 +1571,69 @@ void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
     pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE;
 }
 
+static PnvPhb4PecState *pnv_phb4_get_pec(PnvChip *chip, PnvPHB4 *phb,
+                                         Error **errp)
+{
+    Pnv9Chip *chip9 = PNV9_CHIP(chip);
+    int chip_id = phb->chip_id;
+    int index = phb->phb_id;
+    int i, j;
+
+    for (i = 0; i < chip->num_pecs; i++) {
+        /*
+         * For each PEC, check the amount of phbs it supports
+         * and see if the given phb4 index matches an index.
+         */
+        PnvPhb4PecState *pec = &chip9->pecs[i];
+
+        for (j = 0; j < pec->num_phbs; j++) {
+            if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
+                return pec;
+            }
+        }
+    }
+
+    error_setg(errp,
+               "pnv-phb4 chip-id %d index %d didn't match any existing PEC",
+               chip_id, index);
+
+    return NULL;
+}
+
 static void pnv_phb4_realize(DeviceState *dev, Error **errp)
 {
     PnvPHB4 *phb = PNV_PHB4(dev);
+    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+    PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
     XiveSource *xsrc = &phb->xsrc;
+    BusState *s;
+    Error *local_err = NULL;
     int nr_irqs;
     char name[32];
 
+    if (!chip) {
+        error_setg(errp, "invalid chip id: %d", phb->chip_id);
+        return;
+    }
+
+    /* User created PHBs need to be assigned to a PEC */
+    if (!phb->pec) {
+        phb->pec = pnv_phb4_get_pec(chip, phb, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            return;
+        }
+    }
+
+    /* Reparent the PHB to the chip to build the device tree */
+    pnv_chip_parent_fixup(chip, OBJECT(phb->phb_base), phb->phb_id);
+
+    s = qdev_get_parent_bus(DEVICE(chip));
+    if (!qdev_set_parent_bus(DEVICE(phb->phb_base), s, &local_err)) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
     /* Set the "big_phb" flag */
     phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
 
@@ -1803,7 +1859,7 @@ static void pnv_phb4_root_port_class_init(ObjectClass *klass, void *data)
     PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
 
     dc->desc     = "IBM PHB4 PCIE Root Port";
-    dc->user_creatable = false;
+    dc->user_creatable = true;
 
     device_class_set_parent_realize(dc, pnv_phb4_root_port_realize,
                                     &rpc->parent_realize);
diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
index 888ecbe8f3..0e67f3a338 100644
--- a/hw/pci-host/pnv_phb4_pec.c
+++ b/hw/pci-host/pnv_phb4_pec.c
@@ -146,8 +146,10 @@ static void pnv_pec_realize(DeviceState *dev, Error **errp)
     pec->num_phbs = pecc->num_phbs[pec->index];
 
     /* Create PHBs if running with defaults */
-    for (i = 0; i < pec->num_phbs; i++) {
-        pnv_pec_default_phb_realize(pec, i, errp);
+    if (defaults_enabled()) {
+        for (i = 0; i < pec->num_phbs; i++) {
+            pnv_pec_default_phb_realize(pec, i, errp);
+        }
     }
 
     /* Initialize the XSCOM regions for the PEC registers */
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 3b0b230e49..697a2b5302 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -2186,6 +2186,8 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
     pmc->dt_power_mgt = pnv_dt_power_mgt;
+
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
 }
 
 static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
-- 
2.36.1

[PATCH v2 09/16] ppc/pnv: change pnv_phb4_get_pec() to also retrieve chip10->pecs

[Daniel Henrique Barboza]

The function assumes that we're always dealing with a PNV9_CHIP()
object. This is not the case when the pnv-phb device belongs to a
powernv10 machine.

Change pnv_phb4_get_pec() to be able to work with PNV10_CHIP() if
necessary.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb4.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index a5c8ae494b..e953aabcbe 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -1574,17 +1574,30 @@ void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
 static PnvPhb4PecState *pnv_phb4_get_pec(PnvChip *chip, PnvPHB4 *phb,
                                          Error **errp)
 {
-    Pnv9Chip *chip9 = PNV9_CHIP(chip);
+    PnvPHB *phb_base = phb->phb_base;
+    PnvPhb4PecState *pecs = NULL;
     int chip_id = phb->chip_id;
     int index = phb->phb_id;
     int i, j;
 
+    if (phb_base->version == 4) {
+        Pnv9Chip *chip9 = PNV9_CHIP(chip);
+
+        pecs = chip9->pecs;
+    } else if (phb_base->version == 5) {
+        Pnv10Chip *chip10 = PNV10_CHIP(chip);
+
+        pecs = chip10->pecs;
+    } else {
+        return NULL;
+    }
+
     for (i = 0; i < chip->num_pecs; i++) {
         /*
          * For each PEC, check the amount of phbs it supports
          * and see if the given phb4 index matches an index.
          */
-        PnvPhb4PecState *pec = &chip9->pecs[i];
+        PnvPhb4PecState *pec = &pecs[i];
 
         for (j = 0; j < pec->num_phbs; j++) {
             if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
-- 
2.36.1

[PATCH v2 10/16] ppc/pnv: user creatable pnv-phb for powernv10

[Daniel Henrique Barboza]

Given that powernv9 and powernv10 uses the same pnv-phb backend, the
logic to allow user created pnv-phbs for powernv10 is already in place.
This patch just flips the switch.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb4.c | 2 +-
 hw/ppc/pnv.c           | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index e953aabcbe..8a907a77a7 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -1902,7 +1902,7 @@ static void pnv_phb5_root_port_class_init(ObjectClass *klass, void *data)
     PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
 
     dc->desc     = "IBM PHB5 PCIE Root Port";
-    dc->user_creatable = false;
+    dc->user_creatable = true;
 
     k->vendor_id = PCI_VENDOR_ID_IBM;
     k->device_id = PNV_PHB5_DEVICE_ID;
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 697a2b5302..4d2ea405db 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -2210,6 +2210,8 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     xfc->match_nvt = pnv10_xive_match_nvt;
+
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
 }
 
 static bool pnv_machine_get_hb(Object *obj, Error **errp)
-- 
2.36.1

[PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Daniel Henrique Barboza]

We have two very similar root-port devices, pnv-phb3-root-port and
pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
that, until now, has no additional attributes.

The main difference between the PHB3 and PHB4 root ports is that
pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
other differences can be merged in a single device without too much
trouble.

This patch introduces the unified pnv-phb-root-port that, in time, will
be used as the default root port for the pnv-phb device.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
 hw/pci-host/pnv_phb.h |  17 +++++++
 2 files changed, 116 insertions(+), 8 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index 321c4e768a..5047e90d3a 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
     dc->user_creatable = true;
 }
 
-static void pnv_phb_register_type(void)
+static void pnv_phb_root_port_reset(DeviceState *dev)
 {
-    static const TypeInfo pnv_phb_type_info = {
-        .name          = TYPE_PNV_PHB,
-        .parent        = TYPE_PCIE_HOST_BRIDGE,
-        .instance_size = sizeof(PnvPHB),
-        .class_init    = pnv_phb_class_init,
-    };
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
+    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
+    PCIDevice *d = PCI_DEVICE(dev);
+    uint8_t *conf = d->config;
 
+    rpc->parent_reset(dev);
+
+    if (rootport->version == 3) {
+        return;
+    }
+
+    /* PHB4 and later requires these extra reset steps */
+    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
+                               PCI_IO_RANGE_MASK & 0xff);
+    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
+                                 PCI_IO_RANGE_MASK & 0xff);
+    pci_set_word(conf + PCI_MEMORY_BASE, 0);
+    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
+    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
+    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
+    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
+    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
+    pci_config_set_interrupt_pin(conf, 0);
+}
+
+static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
+{
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
+    PCIDevice *pci = PCI_DEVICE(dev);
+    PCIBus *bus = pci_get_bus(pci);
+    PnvPHB *phb = NULL;
+    Error *local_err = NULL;
+
+    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
+                                          TYPE_PNV_PHB);
+
+    if (!phb) {
+        error_setg(errp,
+"pnv_phb_root_port devices must be connected to pnv-phb buses");
+        return;
+    }
+
+    /* Set unique chassis/slot values for the root port */
+    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
+    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
+
+    rpc->parent_realize(dev, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+    pci_config_set_interrupt_pin(pci->config, 0);
+}
+
+static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
+
+    dc->desc     = "IBM PHB PCIE Root Port";
+
+    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
+                                    &rpc->parent_realize);
+
+    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
+                                  &rpc->parent_reset);
+    dc->reset = &pnv_phb_root_port_reset;
+
+    dc->user_creatable = true;
+
+    k->vendor_id = PCI_VENDOR_ID_IBM;
+    /* device_id represents the latest PHB root port version supported */
+    k->device_id = PNV_PHB5_DEVICE_ID;
+    k->revision  = 0;
+
+    rpc->exp_offset = 0x48;
+    rpc->aer_offset = 0x100;
+}
+
+static const TypeInfo pnv_phb_type_info = {
+    .name          = TYPE_PNV_PHB,
+    .parent        = TYPE_PCIE_HOST_BRIDGE,
+    .instance_size = sizeof(PnvPHB),
+    .class_init    = pnv_phb_class_init,
+};
+
+static const TypeInfo pnv_phb_root_port_info = {
+    .name          = TYPE_PNV_PHB_ROOT_PORT,
+    .parent        = TYPE_PCIE_ROOT_PORT,
+    .instance_size = sizeof(PnvPHBRootPort),
+    .class_init    = pnv_phb_root_port_class_init,
+};
+
+static void pnv_phb_register_types(void)
+{
     type_register_static(&pnv_phb_type_info);
+    type_register_static(&pnv_phb_root_port_info);
 }
-type_init(pnv_phb_register_type)
+
+type_init(pnv_phb_register_types)
diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
index a7cc8610e2..c8eab4b767 100644
--- a/hw/pci-host/pnv_phb.h
+++ b/hw/pci-host/pnv_phb.h
@@ -36,4 +36,21 @@ struct PnvPHB {
 #define TYPE_PNV_PHB "pnv-phb"
 OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
 
+/*
+ * PHB PCIe Root port
+ */
+#define PNV_PHB3_DEVICE_ID         0x03dc
+#define PNV_PHB4_DEVICE_ID         0x04c1
+#define PNV_PHB5_DEVICE_ID         0x0652
+
+typedef struct PnvPHBRootPort {
+    PCIESlot parent_obj;
+
+    uint32_t version;
+} PnvPHBRootPort;
+
+#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
+#define PNV_PHB_ROOT_PORT(obj) \
+    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
+
 #endif /* PCI_HOST_PNV_PHB_H */
-- 
2.36.1

[PATCH v2 12/16] ppc/pnv: remove pnv-phb3-root-port

[Daniel Henrique Barboza]

The unified pnv-phb-root-port can be used in its place. There is no ABI
breakage in doing so because no official QEMU release introduced user
creatable pnv-phb3-root-port devices.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c          | 10 ++++--
 hw/pci-host/pnv_phb3.c         | 57 ----------------------------------
 hw/ppc/pnv.c                   |  1 +
 include/hw/pci-host/pnv_phb3.h |  6 ----
 4 files changed, 8 insertions(+), 66 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index 5047e90d3a..d1e8d28e97 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -35,7 +35,7 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
     switch (phb->version) {
     case 3:
         phb_typename = g_strdup(TYPE_PNV_PHB3);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     case 4:
         phb_typename = g_strdup(TYPE_PNV_PHB4);
@@ -170,6 +170,11 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
     pci_config_set_interrupt_pin(pci->config, 0);
 }
 
+static Property pnv_phb_root_port_properties[] = {
+    DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
 static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -180,11 +185,10 @@ static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
 
     device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
                                     &rpc->parent_realize);
-
     device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
                                   &rpc->parent_reset);
     dc->reset = &pnv_phb_root_port_reset;
-
+    device_class_set_props(dc, pnv_phb_root_port_properties);
     dc->user_creatable = true;
 
     k->vendor_id = PCI_VENDOR_ID_IBM;
diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index 839c2dad00..dc1068443a 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -1152,66 +1152,9 @@ static const TypeInfo pnv_phb3_root_bus_info = {
     },
 };
 
-static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)
-{
-    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
-    PCIDevice *pci = PCI_DEVICE(dev);
-    PCIBus *bus = pci_get_bus(pci);
-    PnvPHB *phb = NULL;
-    Error *local_err = NULL;
-
-    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
-                                         TYPE_PNV_PHB);
-
-    if (!phb) {
-        error_setg(errp,
-"pnv_phb3_root_port devices must be connected to pnv-phb3 buses");
-        return;
-    }
-
-    /* Set unique chassis/slot values for the root port */
-    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
-    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
-
-    rpc->parent_realize(dev, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
-        return;
-    }
-    pci_config_set_interrupt_pin(pci->config, 0);
-}
-
-static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data)
-{
-    DeviceClass *dc = DEVICE_CLASS(klass);
-    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
-    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
-
-    dc->desc     = "IBM PHB3 PCIE Root Port";
-
-    device_class_set_parent_realize(dc, pnv_phb3_root_port_realize,
-                                    &rpc->parent_realize);
-    dc->user_creatable = true;
-
-    k->vendor_id = PCI_VENDOR_ID_IBM;
-    k->device_id = 0x03dc;
-    k->revision  = 0;
-
-    rpc->exp_offset = 0x48;
-    rpc->aer_offset = 0x100;
-}
-
-static const TypeInfo pnv_phb3_root_port_info = {
-    .name          = TYPE_PNV_PHB3_ROOT_PORT,
-    .parent        = TYPE_PCIE_ROOT_PORT,
-    .instance_size = sizeof(PnvPHB3RootPort),
-    .class_init    = pnv_phb3_root_port_class_init,
-};
-
 static void pnv_phb3_register_types(void)
 {
     type_register_static(&pnv_phb3_root_bus_info);
-    type_register_static(&pnv_phb3_root_port_info);
     type_register_static(&pnv_phb3_type_info);
     type_register_static(&pnv_phb3_iommu_memory_region_info);
 }
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 4d2ea405db..5da5067b67 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -2148,6 +2148,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 
     static GlobalProperty phb_compat[] = {
         { TYPE_PNV_PHB, "version", "3" },
+        { TYPE_PNV_PHB_ROOT_PORT, "version", "3" },
     };
 
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
index 3b9ff1096a..bff69201d9 100644
--- a/include/hw/pci-host/pnv_phb3.h
+++ b/include/hw/pci-host/pnv_phb3.h
@@ -108,12 +108,6 @@ struct PnvPBCQState {
  */
 #define TYPE_PNV_PHB3_ROOT_BUS "pnv-phb3-root"
 
-#define TYPE_PNV_PHB3_ROOT_PORT "pnv-phb3-root-port"
-
-typedef struct PnvPHB3RootPort {
-    PCIESlot parent_obj;
-} PnvPHB3RootPort;
-
 /*
  * PHB3 PCIe Host Bridge for PowerNV machines (POWER8)
  */
-- 
2.36.1

[PATCH v2 13/16] ppc/pnv: remove pnv-phb4-root-port

[Daniel Henrique Barboza]

The unified pnv-phb-root-port can be used instead. The phb4-root-port
device isn't exposed to the user in any official QEMU release so there's
no ABI breakage in removing it.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c          |   4 +-
 hw/pci-host/pnv_phb4.c         | 100 ---------------------------------
 hw/pci-host/pnv_phb4_pec.c     |   4 +-
 include/hw/pci-host/pnv_phb4.h |   9 ---
 4 files changed, 4 insertions(+), 113 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index d1e8d28e97..f1c106edf1 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -39,11 +39,11 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
         break;
     case 4:
         phb_typename = g_strdup(TYPE_PNV_PHB4);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     case 5:
         phb_typename = g_strdup(TYPE_PNV_PHB5);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
+        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     default:
         g_assert_not_reached();
diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index 8a907a77a7..ba553ae25e 100644
--- a/hw/pci-host/pnv_phb4.c
+++ b/hw/pci-host/pnv_phb4.c
@@ -1817,109 +1817,9 @@ static const TypeInfo pnv_phb4_root_bus_info = {
     },
 };
 
-static void pnv_phb4_root_port_reset(DeviceState *dev)
-{
-    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
-    PCIDevice *d = PCI_DEVICE(dev);
-    uint8_t *conf = d->config;
-
-    rpc->parent_reset(dev);
-
-    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
-                               PCI_IO_RANGE_MASK & 0xff);
-    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
-                                 PCI_IO_RANGE_MASK & 0xff);
-    pci_set_word(conf + PCI_MEMORY_BASE, 0);
-    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
-    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
-    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
-    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
-    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
-    pci_config_set_interrupt_pin(conf, 0);
-}
-
-static void pnv_phb4_root_port_realize(DeviceState *dev, Error **errp)
-{
-    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
-    PCIDevice *pci = PCI_DEVICE(dev);
-    PCIBus *bus = pci_get_bus(pci);
-    PnvPHB *phb = NULL;
-    Error *local_err = NULL;
-
-    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
-                                         TYPE_PNV_PHB);
-
-    if (!phb) {
-        error_setg(errp, "%s must be connected to pnv-phb4 buses", dev->id);
-        return;
-    }
-
-    /* Set unique chassis/slot values for the root port */
-    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
-    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
-
-    rpc->parent_realize(dev, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
-        return;
-    }
-}
-
-static void pnv_phb4_root_port_class_init(ObjectClass *klass, void *data)
-{
-    DeviceClass *dc = DEVICE_CLASS(klass);
-    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
-    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
-
-    dc->desc     = "IBM PHB4 PCIE Root Port";
-    dc->user_creatable = true;
-
-    device_class_set_parent_realize(dc, pnv_phb4_root_port_realize,
-                                    &rpc->parent_realize);
-    device_class_set_parent_reset(dc, pnv_phb4_root_port_reset,
-                                  &rpc->parent_reset);
-
-    k->vendor_id = PCI_VENDOR_ID_IBM;
-    k->device_id = PNV_PHB4_DEVICE_ID;
-    k->revision  = 0;
-
-    rpc->exp_offset = 0x48;
-    rpc->aer_offset = 0x100;
-
-    dc->reset = &pnv_phb4_root_port_reset;
-}
-
-static const TypeInfo pnv_phb4_root_port_info = {
-    .name          = TYPE_PNV_PHB4_ROOT_PORT,
-    .parent        = TYPE_PCIE_ROOT_PORT,
-    .instance_size = sizeof(PnvPHB4RootPort),
-    .class_init    = pnv_phb4_root_port_class_init,
-};
-
-static void pnv_phb5_root_port_class_init(ObjectClass *klass, void *data)
-{
-    DeviceClass *dc = DEVICE_CLASS(klass);
-    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
-
-    dc->desc     = "IBM PHB5 PCIE Root Port";
-    dc->user_creatable = true;
-
-    k->vendor_id = PCI_VENDOR_ID_IBM;
-    k->device_id = PNV_PHB5_DEVICE_ID;
-}
-
-static const TypeInfo pnv_phb5_root_port_info = {
-    .name          = TYPE_PNV_PHB5_ROOT_PORT,
-    .parent        = TYPE_PNV_PHB4_ROOT_PORT,
-    .instance_size = sizeof(PnvPHB4RootPort),
-    .class_init    = pnv_phb5_root_port_class_init,
-};
-
 static void pnv_phb4_register_types(void)
 {
     type_register_static(&pnv_phb4_root_bus_info);
-    type_register_static(&pnv_phb5_root_port_info);
-    type_register_static(&pnv_phb4_root_port_info);
     type_register_static(&pnv_phb4_type_info);
     type_register_static(&pnv_phb5_type_info);
     type_register_static(&pnv_phb4_iommu_memory_region_info);
diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
index 0e67f3a338..785b778396 100644
--- a/hw/pci-host/pnv_phb4_pec.c
+++ b/hw/pci-host/pnv_phb4_pec.c
@@ -262,7 +262,7 @@ static void pnv_pec_class_init(ObjectClass *klass, void *data)
     pecc->version = PNV_PHB4_VERSION;
     pecc->phb_type = TYPE_PNV_PHB4;
     pecc->num_phbs = pnv_pec_num_phbs;
-    pecc->rp_model = TYPE_PNV_PHB4_ROOT_PORT;
+    pecc->rp_model = TYPE_PNV_PHB_ROOT_PORT;
 }
 
 static const TypeInfo pnv_pec_type_info = {
@@ -315,7 +315,7 @@ static void pnv_phb5_pec_class_init(ObjectClass *klass, void *data)
     pecc->version = PNV_PHB5_VERSION;
     pecc->phb_type = TYPE_PNV_PHB5;
     pecc->num_phbs = pnv_phb5_pec_num_stacks;
-    pecc->rp_model = TYPE_PNV_PHB5_ROOT_PORT;
+    pecc->rp_model = TYPE_PNV_PHB_ROOT_PORT;
 }
 
 static const TypeInfo pnv_phb5_pec_type_info = {
diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
index f22253358f..29c49ac79c 100644
--- a/include/hw/pci-host/pnv_phb4.h
+++ b/include/hw/pci-host/pnv_phb4.h
@@ -45,16 +45,7 @@ typedef struct PnvPhb4DMASpace {
     QLIST_ENTRY(PnvPhb4DMASpace) list;
 } PnvPhb4DMASpace;
 
-/*
- * PHB4 PCIe Root port
- */
 #define TYPE_PNV_PHB4_ROOT_BUS "pnv-phb4-root"
-#define TYPE_PNV_PHB4_ROOT_PORT "pnv-phb4-root-port"
-#define TYPE_PNV_PHB5_ROOT_PORT "pnv-phb5-root-port"
-
-typedef struct PnvPHB4RootPort {
-    PCIESlot parent_obj;
-} PnvPHB4RootPort;
 
 /*
  * PHB4 PCIe Host Bridge for PowerNV machines (POWER9)
-- 
2.36.1

[PATCH v2 14/16] ppc/pnv: remove 'phb_rootport_typename' in pnv_phb_realize()

[Daniel Henrique Barboza]

The var is being initialized using the TYPE_PNV_PHB_ROOT_PORT value for
all values of phb->version.

Remove it and call pnv_phb_attach_root_port() using
TYPE_PNV_PHB_ROOT_PORT directly.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index f1c106edf1..c33223d275 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -25,7 +25,6 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
     PnvPHB *phb = PNV_PHB(dev);
     PCIHostState *pci = PCI_HOST_BRIDGE(dev);
     g_autofree char *phb_typename = NULL;
-    g_autofree char *phb_rootport_typename = NULL;
 
     if (!phb->version) {
         error_setg(errp, "version not specified");
@@ -35,15 +34,12 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
     switch (phb->version) {
     case 3:
         phb_typename = g_strdup(TYPE_PNV_PHB3);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     case 4:
         phb_typename = g_strdup(TYPE_PNV_PHB4);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     case 5:
         phb_typename = g_strdup(TYPE_PNV_PHB5);
-        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
         break;
     default:
         g_assert_not_reached();
@@ -75,7 +71,7 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
     }
 
     if (defaults_enabled()) {
-        pnv_phb_attach_root_port(pci, phb_rootport_typename);
+        pnv_phb_attach_root_port(pci, TYPE_PNV_PHB_ROOT_PORT);
     }
 }
 
-- 
2.36.1

[PATCH v2 15/16] ppc/pnv: remove pecc->rp_model

[Daniel Henrique Barboza]

The attribute is unused.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 hw/pci-host/pnv_phb4_pec.c     | 2 --
 include/hw/pci-host/pnv_phb4.h | 1 -
 2 files changed, 3 deletions(-)

diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
index 785b778396..8f11e077c2 100644
--- a/hw/pci-host/pnv_phb4_pec.c
+++ b/hw/pci-host/pnv_phb4_pec.c
@@ -262,7 +262,6 @@ static void pnv_pec_class_init(ObjectClass *klass, void *data)
     pecc->version = PNV_PHB4_VERSION;
     pecc->phb_type = TYPE_PNV_PHB4;
     pecc->num_phbs = pnv_pec_num_phbs;
-    pecc->rp_model = TYPE_PNV_PHB_ROOT_PORT;
 }
 
 static const TypeInfo pnv_pec_type_info = {
@@ -315,7 +314,6 @@ static void pnv_phb5_pec_class_init(ObjectClass *klass, void *data)
     pecc->version = PNV_PHB5_VERSION;
     pecc->phb_type = TYPE_PNV_PHB5;
     pecc->num_phbs = pnv_phb5_pec_num_stacks;
-    pecc->rp_model = TYPE_PNV_PHB_ROOT_PORT;
 }
 
 static const TypeInfo pnv_phb5_pec_type_info = {
diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
index 29c49ac79c..61a0cb9989 100644
--- a/include/hw/pci-host/pnv_phb4.h
+++ b/include/hw/pci-host/pnv_phb4.h
@@ -200,7 +200,6 @@ struct PnvPhb4PecClass {
     uint64_t version;
     const char *phb_type;
     const uint32_t *num_phbs;
-    const char *rp_model;
 };
 
 /*
-- 
2.36.1

[PATCH v2 16/16] ppc/pnv: remove PnvPHB4.version

[Daniel Henrique Barboza]

It's unused.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
---
 include/hw/pci-host/pnv_phb4.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
index 61a0cb9989..20aa4819d3 100644
--- a/include/hw/pci-host/pnv_phb4.h
+++ b/include/hw/pci-host/pnv_phb4.h
@@ -77,8 +77,6 @@ struct PnvPHB4 {
     uint32_t chip_id;
     uint32_t phb_id;
 
-    uint64_t version;
-
     /* The owner PEC */
     PnvPhb4PecState *pec;
 
-- 
2.36.1

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Cédric Le Goater]

On 5/31/22 23:49, Daniel Henrique Barboza wrote:
> We have two very similar root-port devices, pnv-phb3-root-port and
> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
> that, until now, has no additional attributes.
> 
> The main difference between the PHB3 and PHB4 root ports is that
> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
> other differences can be merged in a single device without too much
> trouble.
> 
> This patch introduces the unified pnv-phb-root-port that, in time, will
> be used as the default root port for the pnv-phb device.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>   hw/pci-host/pnv_phb.h |  17 +++++++
>   2 files changed, 116 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> index 321c4e768a..5047e90d3a 100644
> --- a/hw/pci-host/pnv_phb.c
> +++ b/hw/pci-host/pnv_phb.c
> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>       dc->user_creatable = true;
>   }
>   
> -static void pnv_phb_register_type(void)
> +static void pnv_phb_root_port_reset(DeviceState *dev)
>   {
> -    static const TypeInfo pnv_phb_type_info = {
> -        .name          = TYPE_PNV_PHB,
> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
> -        .instance_size = sizeof(PnvPHB),
> -        .class_init    = pnv_phb_class_init,
> -    };
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
> +    PCIDevice *d = PCI_DEVICE(dev);
> +    uint8_t *conf = d->config;
>   
> +    rpc->parent_reset(dev);
> +
> +    if (rootport->version == 3) {
> +        return;
> +    }
> +
> +    /* PHB4 and later requires these extra reset steps */
> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
> +                               PCI_IO_RANGE_MASK & 0xff);
> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
> +                                 PCI_IO_RANGE_MASK & 0xff);
> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
> +    pci_config_set_interrupt_pin(conf, 0);
> +}
> +
> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
> +{
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
> +    PCIDevice *pci = PCI_DEVICE(dev);
> +    PCIBus *bus = pci_get_bus(pci);
> +    PnvPHB *phb = NULL;
> +    Error *local_err = NULL;
> +
> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> +                                          TYPE_PNV_PHB);
> +
> +    if (!phb) {
> +        error_setg(errp,
> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
> +        return;
> +    }
> +
> +    /* Set unique chassis/slot values for the root port */
> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
> +
> +    rpc->parent_realize(dev, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +    pci_config_set_interrupt_pin(pci->config, 0);
> +}
> +
> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
> +{
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
> +
> +    dc->desc     = "IBM PHB PCIE Root Port";
> +
> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
> +                                    &rpc->parent_realize);
> +
> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
> +                                  &rpc->parent_reset);
> +    dc->reset = &pnv_phb_root_port_reset;
> +
> +    dc->user_creatable = true;
> +
> +    k->vendor_id = PCI_VENDOR_ID_IBM;
> +    /* device_id represents the latest PHB root port version supported */
> +    k->device_id = PNV_PHB5_DEVICE_ID;

does that mean powernv8 machines will see phb devices as phb5 devices ?

C.

> +    k->revision  = 0;
> +
> +    rpc->exp_offset = 0x48;
> +    rpc->aer_offset = 0x100;
> +}
> +
> +static const TypeInfo pnv_phb_type_info = {
> +    .name          = TYPE_PNV_PHB,
> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
> +    .instance_size = sizeof(PnvPHB),
> +    .class_init    = pnv_phb_class_init,
> +};
> +
> +static const TypeInfo pnv_phb_root_port_info = {
> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
> +    .parent        = TYPE_PCIE_ROOT_PORT,
> +    .instance_size = sizeof(PnvPHBRootPort),
> +    .class_init    = pnv_phb_root_port_class_init,
> +};
> +
> +static void pnv_phb_register_types(void)
> +{
>       type_register_static(&pnv_phb_type_info);
> +    type_register_static(&pnv_phb_root_port_info);
>   }
> -type_init(pnv_phb_register_type)
> +
> +type_init(pnv_phb_register_types)
> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
> index a7cc8610e2..c8eab4b767 100644
> --- a/hw/pci-host/pnv_phb.h
> +++ b/hw/pci-host/pnv_phb.h
> @@ -36,4 +36,21 @@ struct PnvPHB {
>   #define TYPE_PNV_PHB "pnv-phb"
>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>   
> +/*
> + * PHB PCIe Root port
> + */
> +#define PNV_PHB3_DEVICE_ID         0x03dc
> +#define PNV_PHB4_DEVICE_ID         0x04c1
> +#define PNV_PHB5_DEVICE_ID         0x0652
> +
> +typedef struct PnvPHBRootPort {
> +    PCIESlot parent_obj;
> +
> +    uint32_t version;
> +} PnvPHBRootPort;
> +
> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
> +#define PNV_PHB_ROOT_PORT(obj) \
> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
> +
>   #endif /* PCI_HOST_PNV_PHB_H */

Re: [PATCH v2 12/16] ppc/pnv: remove pnv-phb3-root-port

[Daniel Henrique Barboza]

On 5/31/22 18:49, Daniel Henrique Barboza wrote:
> The unified pnv-phb-root-port can be used in its place. There is no ABI
> breakage in doing so because no official QEMU release introduced user
> creatable pnv-phb3-root-port devices.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb.c          | 10 ++++--
>   hw/pci-host/pnv_phb3.c         | 57 ----------------------------------
>   hw/ppc/pnv.c                   |  1 +
>   include/hw/pci-host/pnv_phb3.h |  6 ----
>   4 files changed, 8 insertions(+), 66 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> index 5047e90d3a..d1e8d28e97 100644
> --- a/hw/pci-host/pnv_phb.c
> +++ b/hw/pci-host/pnv_phb.c
> @@ -35,7 +35,7 @@ static void pnv_phb_realize(DeviceState *dev, Error **errp)
>       switch (phb->version) {
>       case 3:
>           phb_typename = g_strdup(TYPE_PNV_PHB3);
> -        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB_ROOT_PORT);
>           break;
>       case 4:
>           phb_typename = g_strdup(TYPE_PNV_PHB4);
> @@ -170,6 +170,11 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>       pci_config_set_interrupt_pin(pci->config, 0);
>   }
>   
> +static Property pnv_phb_root_port_properties[] = {
> +    DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
> +    DEFINE_PROP_END_OF_LIST(),
> +};
> +
>   static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>   {
>       DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -180,11 +185,10 @@ static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>   
>       device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>                                       &rpc->parent_realize);
> -
>       device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>                                     &rpc->parent_reset);
>       dc->reset = &pnv_phb_root_port_reset;
> -
> +    device_class_set_props(dc, pnv_phb_root_port_properties);
>       dc->user_creatable = true;
>   
>       k->vendor_id = PCI_VENDOR_ID_IBM;
> diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
> index 839c2dad00..dc1068443a 100644
> --- a/hw/pci-host/pnv_phb3.c
> +++ b/hw/pci-host/pnv_phb3.c
> @@ -1152,66 +1152,9 @@ static const TypeInfo pnv_phb3_root_bus_info = {
>       },
>   };
>   
> -static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)
> -{
> -    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
> -    PCIDevice *pci = PCI_DEVICE(dev);
> -    PCIBus *bus = pci_get_bus(pci);
> -    PnvPHB *phb = NULL;
> -    Error *local_err = NULL;
> -
> -    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> -                                         TYPE_PNV_PHB);
> -
> -    if (!phb) {
> -        error_setg(errp,
> -"pnv_phb3_root_port devices must be connected to pnv-phb3 buses");
> -        return;
> -    }
> -
> -    /* Set unique chassis/slot values for the root port */
> -    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
> -    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
> -
> -    rpc->parent_realize(dev, &local_err);
> -    if (local_err) {
> -        error_propagate(errp, local_err);
> -        return;
> -    }
> -    pci_config_set_interrupt_pin(pci->config, 0);
> -}
> -
> -static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data)
> -{
> -    DeviceClass *dc = DEVICE_CLASS(klass);
> -    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
> -    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
> -
> -    dc->desc     = "IBM PHB3 PCIE Root Port";
> -
> -    device_class_set_parent_realize(dc, pnv_phb3_root_port_realize,
> -                                    &rpc->parent_realize);
> -    dc->user_creatable = true;
> -
> -    k->vendor_id = PCI_VENDOR_ID_IBM;
> -    k->device_id = 0x03dc;
> -    k->revision  = 0;
> -
> -    rpc->exp_offset = 0x48;
> -    rpc->aer_offset = 0x100;
> -}
> -
> -static const TypeInfo pnv_phb3_root_port_info = {
> -    .name          = TYPE_PNV_PHB3_ROOT_PORT,
> -    .parent        = TYPE_PCIE_ROOT_PORT,
> -    .instance_size = sizeof(PnvPHB3RootPort),
> -    .class_init    = pnv_phb3_root_port_class_init,
> -};
> -
>   static void pnv_phb3_register_types(void)
>   {
>       type_register_static(&pnv_phb3_root_bus_info);
> -    type_register_static(&pnv_phb3_root_port_info);
>       type_register_static(&pnv_phb3_type_info);
>       type_register_static(&pnv_phb3_iommu_memory_region_info);
>   }
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 4d2ea405db..5da5067b67 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -2148,6 +2148,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>   
>       static GlobalProperty phb_compat[] = {
>           { TYPE_PNV_PHB, "version", "3" },
> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "3" },

Not sure how this ended up here. Consider it removed.



Daniel

>       };
>   
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
> diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
> index 3b9ff1096a..bff69201d9 100644
> --- a/include/hw/pci-host/pnv_phb3.h
> +++ b/include/hw/pci-host/pnv_phb3.h
> @@ -108,12 +108,6 @@ struct PnvPBCQState {
>    */
>   #define TYPE_PNV_PHB3_ROOT_BUS "pnv-phb3-root"
>   
> -#define TYPE_PNV_PHB3_ROOT_PORT "pnv-phb3-root-port"
> -
> -typedef struct PnvPHB3RootPort {
> -    PCIESlot parent_obj;
> -} PnvPHB3RootPort;
> -
>   /*
>    * PHB3 PCIe Host Bridge for PowerNV machines (POWER8)
>    */

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Daniel Henrique Barboza]

On 6/1/22 02:56, Cédric Le Goater wrote:
> On 5/31/22 23:49, Daniel Henrique Barboza wrote:
>> We have two very similar root-port devices, pnv-phb3-root-port and
>> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
>> that, until now, has no additional attributes.
>>
>> The main difference between the PHB3 and PHB4 root ports is that
>> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
>> other differences can be merged in a single device without too much
>> trouble.
>>
>> This patch introduces the unified pnv-phb-root-port that, in time, will
>> be used as the default root port for the pnv-phb device.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>>   hw/pci-host/pnv_phb.h |  17 +++++++
>>   2 files changed, 116 insertions(+), 8 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> index 321c4e768a..5047e90d3a 100644
>> --- a/hw/pci-host/pnv_phb.c
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>       dc->user_creatable = true;
>>   }
>> -static void pnv_phb_register_type(void)
>> +static void pnv_phb_root_port_reset(DeviceState *dev)
>>   {
>> -    static const TypeInfo pnv_phb_type_info = {
>> -        .name          = TYPE_PNV_PHB,
>> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
>> -        .instance_size = sizeof(PnvPHB),
>> -        .class_init    = pnv_phb_class_init,
>> -    };
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
>> +    PCIDevice *d = PCI_DEVICE(dev);
>> +    uint8_t *conf = d->config;
>> +    rpc->parent_reset(dev);
>> +
>> +    if (rootport->version == 3) {
>> +        return;
>> +    }
>> +
>> +    /* PHB4 and later requires these extra reset steps */
>> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
>> +                               PCI_IO_RANGE_MASK & 0xff);
>> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
>> +                                 PCI_IO_RANGE_MASK & 0xff);
>> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
>> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
>> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
>> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
>> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
>> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
>> +    pci_config_set_interrupt_pin(conf, 0);
>> +}
>> +
>> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>> +{
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>> +    PCIDevice *pci = PCI_DEVICE(dev);
>> +    PCIBus *bus = pci_get_bus(pci);
>> +    PnvPHB *phb = NULL;
>> +    Error *local_err = NULL;
>> +
>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> +                                          TYPE_PNV_PHB);
>> +
>> +    if (!phb) {
>> +        error_setg(errp,
>> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
>> +        return;
>> +    }
>> +
>> +    /* Set unique chassis/slot values for the root port */
>> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
>> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
>> +
>> +    rpc->parent_realize(dev, &local_err);
>> +    if (local_err) {
>> +        error_propagate(errp, local_err);
>> +        return;
>> +    }
>> +    pci_config_set_interrupt_pin(pci->config, 0);
>> +}
>> +
>> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>> +{
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>> +
>> +    dc->desc     = "IBM PHB PCIE Root Port";
>> +
>> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>> +                                    &rpc->parent_realize);
>> +
>> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>> +                                  &rpc->parent_reset);
>> +    dc->reset = &pnv_phb_root_port_reset;
>> +
>> +    dc->user_creatable = true;
>> +
>> +    k->vendor_id = PCI_VENDOR_ID_IBM;
>> +    /* device_id represents the latest PHB root port version supported */
>> +    k->device_id = PNV_PHB5_DEVICE_ID;
> 
> does that mean powernv8 machines will see phb devices as phb5 devices ?


I had something like this in this patch that would set device_id properly:


diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
index 5d66264a96..d468e8d44a 100644
--- a/hw/pci-host/pnv_phb.c
+++ b/hw/pci-host/pnv_phb.c
@@ -144,6 +144,22 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
      PCIBus *bus = pci_get_bus(pci);
      PnvPHB *phb = NULL;
      Error *local_err = NULL;
+    PnvPHBRootPort *phb_rp = PNV_PHB_ROOT_PORT(dev);
+    PCIDeviceClass *k = PCI_DEVICE_GET_CLASS(pci);
+
+    switch (phb_rp->version) {
+    case 3:
+        k->device_id = PNV_PHB3_DEVICE_ID;
+        break;
+    case 4:
+        k->device_id = PNV_PHB4_DEVICE_ID;
+        break;
+    case 5:
+        k->device_id = PNV_PHB5_DEVICE_ID;
+        break;
+    default:
+        g_assert_not_reached();
+    }
  
      phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
                                            TYPE_PNV_PHB);
@@ -166,6 +182,11 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
      pci_config_set_interrupt_pin(pci->config, 0);
  }
  
+static Property pnv_phb_root_port_properties[] = {
+    DEFINE_PROP_UINT32("version", PnvPHBRootPort, version, 0),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
  static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
  {
      DeviceClass *dc = DEVICE_CLASS(klass);
@@ -181,6 +202,7 @@ static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
                                    &rpc->parent_reset);
      dc->reset = &pnv_phb_root_port_reset;
  
+    device_class_set_props(dc, pnv_phb_root_port_properties);
      dc->user_creatable = true;
  
      k->vendor_id = PCI_VENDOR_ID_IBM;
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 4d2ea405db..13c8753eb2 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -2148,6 +2148,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
  
      static GlobalProperty phb_compat[] = {
          { TYPE_PNV_PHB, "version", "3" },
+        { TYPE_PNV_PHB_ROOT_PORT, "version", "3" },
      };
  
      mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
@@ -2173,6 +2174,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
  
      static GlobalProperty phb_compat[] = {
          { TYPE_PNV_PHB, "version", "4" },
+        { TYPE_PNV_PHB_ROOT_PORT, "version", "4" },
      };
  
      mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
@@ -2199,6 +2201,7 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
  
      static GlobalProperty phb_compat[] = {
          { TYPE_PNV_PHB, "version", "5" },
+        { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
      };
  
      mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";


The reason I didn't follow it through is because I wasn't sure if setting k->device_id
during realize() time was acceptable. Everyone else seems to set k->device_id during
class_init() or via an extra attribute 'device_id' that is written directly into
the PCI header.

If this is something that we can do then I'm fine with fixing this up in this patch.


Thanks,


Daniel

> 
> C.
> 
>> +    k->revision  = 0;
>> +
>> +    rpc->exp_offset = 0x48;
>> +    rpc->aer_offset = 0x100;
>> +}
>> +
>> +static const TypeInfo pnv_phb_type_info = {
>> +    .name          = TYPE_PNV_PHB,
>> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +    .instance_size = sizeof(PnvPHB),
>> +    .class_init    = pnv_phb_class_init,
>> +};
>> +
>> +static const TypeInfo pnv_phb_root_port_info = {
>> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
>> +    .parent        = TYPE_PCIE_ROOT_PORT,
>> +    .instance_size = sizeof(PnvPHBRootPort),
>> +    .class_init    = pnv_phb_root_port_class_init,
>> +};
>> +
>> +static void pnv_phb_register_types(void)
>> +{
>>       type_register_static(&pnv_phb_type_info);
>> +    type_register_static(&pnv_phb_root_port_info);
>>   }
>> -type_init(pnv_phb_register_type)
>> +
>> +type_init(pnv_phb_register_types)
>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>> index a7cc8610e2..c8eab4b767 100644
>> --- a/hw/pci-host/pnv_phb.h
>> +++ b/hw/pci-host/pnv_phb.h
>> @@ -36,4 +36,21 @@ struct PnvPHB {
>>   #define TYPE_PNV_PHB "pnv-phb"
>>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>> +/*
>> + * PHB PCIe Root port
>> + */
>> +#define PNV_PHB3_DEVICE_ID         0x03dc
>> +#define PNV_PHB4_DEVICE_ID         0x04c1
>> +#define PNV_PHB5_DEVICE_ID         0x0652
>> +
>> +typedef struct PnvPHBRootPort {
>> +    PCIESlot parent_obj;
>> +
>> +    uint32_t version;
>> +} PnvPHBRootPort;
>> +
>> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
>> +#define PNV_PHB_ROOT_PORT(obj) \
>> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
>> +
>>   #endif /* PCI_HOST_PNV_PHB_H */
> 

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Mark Cave-Ayland]

On 01/06/2022 09:04, Daniel Henrique Barboza wrote:

> On 6/1/22 02:56, Cédric Le Goater wrote:
>> On 5/31/22 23:49, Daniel Henrique Barboza wrote:
>>> We have two very similar root-port devices, pnv-phb3-root-port and
>>> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
>>> that, until now, has no additional attributes.
>>>
>>> The main difference between the PHB3 and PHB4 root ports is that
>>> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
>>> other differences can be merged in a single device without too much
>>> trouble.
>>>
>>> This patch introduces the unified pnv-phb-root-port that, in time, will
>>> be used as the default root port for the pnv-phb device.
>>>
>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>> ---
>>>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>>>   hw/pci-host/pnv_phb.h |  17 +++++++
>>>   2 files changed, 116 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>>> index 321c4e768a..5047e90d3a 100644
>>> --- a/hw/pci-host/pnv_phb.c
>>> +++ b/hw/pci-host/pnv_phb.c
>>> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>>       dc->user_creatable = true;
>>>   }
>>> -static void pnv_phb_register_type(void)
>>> +static void pnv_phb_root_port_reset(DeviceState *dev)
>>>   {
>>> -    static const TypeInfo pnv_phb_type_info = {
>>> -        .name          = TYPE_PNV_PHB,
>>> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> -        .instance_size = sizeof(PnvPHB),
>>> -        .class_init    = pnv_phb_class_init,
>>> -    };
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
>>> +    PCIDevice *d = PCI_DEVICE(dev);
>>> +    uint8_t *conf = d->config;
>>> +    rpc->parent_reset(dev);
>>> +
>>> +    if (rootport->version == 3) {
>>> +        return;
>>> +    }
>>> +
>>> +    /* PHB4 and later requires these extra reset steps */
>>> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
>>> +                               PCI_IO_RANGE_MASK & 0xff);
>>> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
>>> +                                 PCI_IO_RANGE_MASK & 0xff);
>>> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
>>> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
>>> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
>>> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
>>> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
>>> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
>>> +    pci_config_set_interrupt_pin(conf, 0);
>>> +}
>>> +
>>> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>>> +{
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>> +    PCIDevice *pci = PCI_DEVICE(dev);
>>> +    PCIBus *bus = pci_get_bus(pci);
>>> +    PnvPHB *phb = NULL;
>>> +    Error *local_err = NULL;
>>> +
>>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>> +                                          TYPE_PNV_PHB);
>>> +
>>> +    if (!phb) {
>>> +        error_setg(errp,
>>> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
>>> +        return;
>>> +    }
>>> +
>>> +    /* Set unique chassis/slot values for the root port */
>>> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
>>> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
>>> +
>>> +    rpc->parent_realize(dev, &local_err);
>>> +    if (local_err) {
>>> +        error_propagate(errp, local_err);
>>> +        return;
>>> +    }
>>> +    pci_config_set_interrupt_pin(pci->config, 0);
>>> +}
>>> +
>>> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>>> +{
>>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>>> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>>> +
>>> +    dc->desc     = "IBM PHB PCIE Root Port";
>>> +
>>> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>>> +                                    &rpc->parent_realize);
>>> +
>>> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>>> +                                  &rpc->parent_reset);
>>> +    dc->reset = &pnv_phb_root_port_reset;
>>> +
>>> +    dc->user_creatable = true;
>>> +
>>> +    k->vendor_id = PCI_VENDOR_ID_IBM;
>>> +    /* device_id represents the latest PHB root port version supported */
>>> +    k->device_id = PNV_PHB5_DEVICE_ID;
>>
>> does that mean powernv8 machines will see phb devices as phb5 devices ?
> 
> 
> I had something like this in this patch that would set device_id properly:
> 
> 
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> index 5d66264a96..d468e8d44a 100644
> --- a/hw/pci-host/pnv_phb.c
> +++ b/hw/pci-host/pnv_phb.c
> @@ -144,6 +144,22 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error 
> **errp)
>       PCIBus *bus = pci_get_bus(pci);
>       PnvPHB *phb = NULL;
>       Error *local_err = NULL;
> +    PnvPHBRootPort *phb_rp = PNV_PHB_ROOT_PORT(dev);
> +    PCIDeviceClass *k = PCI_DEVICE_GET_CLASS(pci);
> +
> +    switch (phb_rp->version) {
> +    case 3:
> +        k->device_id = PNV_PHB3_DEVICE_ID;
> +        break;
> +    case 4:
> +        k->device_id = PNV_PHB4_DEVICE_ID;
> +        break;
> +    case 5:
> +        k->device_id = PNV_PHB5_DEVICE_ID;
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> 
>       phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>                                             TYPE_PNV_PHB);
> @@ -166,6 +182,11 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error 
> **errp)
>       pci_config_set_interrupt_pin(pci->config, 0);
>   }
> 
> +static Property pnv_phb_root_port_properties[] = {
> +    DEFINE_PROP_UINT32("version", PnvPHBRootPort, version, 0),
> +    DEFINE_PROP_END_OF_LIST(),
> +};
> +
>   static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>   {
>       DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -181,6 +202,7 @@ static void pnv_phb_root_port_class_init(ObjectClass *klass, void 
> *data)
>                                     &rpc->parent_reset);
>       dc->reset = &pnv_phb_root_port_reset;
> 
> +    device_class_set_props(dc, pnv_phb_root_port_properties);
>       dc->user_creatable = true;
> 
>       k->vendor_id = PCI_VENDOR_ID_IBM;
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 4d2ea405db..13c8753eb2 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -2148,6 +2148,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void 
> *data)
> 
>       static GlobalProperty phb_compat[] = {
>           { TYPE_PNV_PHB, "version", "3" },
> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "3" },
>       };
> 
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
> @@ -2173,6 +2174,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void 
> *data)
> 
>       static GlobalProperty phb_compat[] = {
>           { TYPE_PNV_PHB, "version", "4" },
> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "4" },
>       };
> 
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
> @@ -2199,6 +2201,7 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, 
> void *data)
> 
>       static GlobalProperty phb_compat[] = {
>           { TYPE_PNV_PHB, "version", "5" },
> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
>       };
> 
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
> 
> 
> The reason I didn't follow it through is because I wasn't sure if setting k->device_id
> during realize() time was acceptable. Everyone else seems to set k->device_id during
> class_init() or via an extra attribute 'device_id' that is written directly into
> the PCI header.
> 
> If this is something that we can do then I'm fine with fixing this up in this patch.

Yeah I'd be slightly nervous about doing this outside of class_init() too. I think my 
preferred method would be that used by virtio, which is to write the value to config 
space (see virtio_pci_device_plugged()) and then set k->device_id to 0 with an 
appropriate comment as per 
https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost-user-rng-pci.c#L53. That 
makes it quite clear that the 0 value in class_init() is a placeholder for something 
else.

> Thanks,
>  
> Daniel
> 
>>
>> C.
>>
>>> +    k->revision  = 0;
>>> +
>>> +    rpc->exp_offset = 0x48;
>>> +    rpc->aer_offset = 0x100;
>>> +}
>>> +
>>> +static const TypeInfo pnv_phb_type_info = {
>>> +    .name          = TYPE_PNV_PHB,
>>> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> +    .instance_size = sizeof(PnvPHB),
>>> +    .class_init    = pnv_phb_class_init,
>>> +};
>>> +
>>> +static const TypeInfo pnv_phb_root_port_info = {
>>> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
>>> +    .parent        = TYPE_PCIE_ROOT_PORT,
>>> +    .instance_size = sizeof(PnvPHBRootPort),
>>> +    .class_init    = pnv_phb_root_port_class_init,
>>> +};
>>> +
>>> +static void pnv_phb_register_types(void)
>>> +{
>>>       type_register_static(&pnv_phb_type_info);
>>> +    type_register_static(&pnv_phb_root_port_info);
>>>   }
>>> -type_init(pnv_phb_register_type)
>>> +
>>> +type_init(pnv_phb_register_types)
>>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>>> index a7cc8610e2..c8eab4b767 100644
>>> --- a/hw/pci-host/pnv_phb.h
>>> +++ b/hw/pci-host/pnv_phb.h
>>> @@ -36,4 +36,21 @@ struct PnvPHB {
>>>   #define TYPE_PNV_PHB "pnv-phb"
>>>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>>> +/*
>>> + * PHB PCIe Root port
>>> + */
>>> +#define PNV_PHB3_DEVICE_ID         0x03dc
>>> +#define PNV_PHB4_DEVICE_ID         0x04c1
>>> +#define PNV_PHB5_DEVICE_ID         0x0652
>>> +
>>> +typedef struct PnvPHBRootPort {
>>> +    PCIESlot parent_obj;
>>> +
>>> +    uint32_t version;
>>> +} PnvPHBRootPort;
>>> +
>>> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
>>> +#define PNV_PHB_ROOT_PORT(obj) \
>>> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
>>> +
>>>   #endif /* PCI_HOST_PNV_PHB_H */

ATB,

Mark.

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Mark Cave-Ayland]

On 31/05/2022 22:49, Daniel Henrique Barboza wrote:

> The PnvPHB device is going to be the base device for all other powernv
> PHBs. It consists of a device that has the same user API as the other
> PHB, namely being a PCIHostBridge and having chip-id and index
> properties. It also has a 'backend' pointer that will be initialized
> with the PHB implementation that the device is going to use.
> 
> The initialization of the PHB backend is done by checking the PHB
> version via a 'version' attribute that can be set via a global machine
> property.  The 'version' field will be used to make adjustments based on
> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
> helper will be added later on.
> 
> For now let's add the basic logic of the PnvPHB object, which consists
> mostly of pnv_phb_realize() doing all the work of checking the
> phb->version set, initializing the proper backend, passing through its
> attributes to the chosen backend, finalizing the backend realize and
> adding a root port in the end.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/meson.build |   3 +-
>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>   3 files changed, 164 insertions(+), 1 deletion(-)
>   create mode 100644 hw/pci-host/pnv_phb.c
>   create mode 100644 hw/pci-host/pnv_phb.h
> 
> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
> index c07596d0d1..e832babc9d 100644
> --- a/hw/pci-host/meson.build
> +++ b/hw/pci-host/meson.build
> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>     'pnv_phb3_msi.c',
>     'pnv_phb3_pbcq.c',
>     'pnv_phb4.c',
> -  'pnv_phb4_pec.c'
> +  'pnv_phb4_pec.c',
> +  'pnv_phb.c',
>   ))
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> new file mode 100644
> index 0000000000..fa8472622f
> --- /dev/null
> +++ b/hw/pci-host/pnv_phb.c
> @@ -0,0 +1,123 @@
> +/*
> + * QEMU PowerPC PowerNV Proxy PHB model
> + *
> + * Copyright (c) 2022, IBM Corporation.
> + *
> + * This code is licensed under the GPL version 2 or later. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/log.h"
> +#include "qapi/visitor.h"
> +#include "qapi/error.h"
> +#include "hw/pci-host/pnv_phb.h"
> +#include "hw/pci-host/pnv_phb3.h"
> +#include "hw/pci-host/pnv_phb4.h"
> +#include "hw/ppc/pnv.h"
> +#include "hw/qdev-properties.h"
> +#include "qom/object.h"
> +
> +
> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
> +{
> +    PnvPHB *phb = PNV_PHB(dev);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
> +    g_autofree char *phb_typename = NULL;
> +    g_autofree char *phb_rootport_typename = NULL;
> +
> +    if (!phb->version) {
> +        error_setg(errp, "version not specified");
> +        return;
> +    }
> +
> +    switch (phb->version) {
> +    case 3:
> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
> +        break;
> +    case 4:
> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
> +        break;
> +    case 5:
> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> +
> +    phb->backend = object_new(phb_typename);
> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
> +
> +    /* Passthrough child device properties to the proxy device */
> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
> +
> +    if (phb->version == 3) {
> +        object_property_set_link(phb->backend, "chip",
> +                                 OBJECT(phb->chip), errp);
> +    } else {
> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
> +    }
> +
> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
> +        return;
> +    }
> +
> +    if (phb->version == 3) {
> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
> +    }
> +
> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
> +}
> +
> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
> +                                         PCIBus *rootbus)
> +{
> +    PnvPHB *phb = PNV_PHB(host_bridge);
> +
> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
> +             phb->chip_id, phb->phb_id);
> +    return phb->bus_path;
> +}
> +
> +static Property pnv_phb_properties[] = {
> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
> +
> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
> +
> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
> +                         PnvPhb4PecState *),
> +
> +        DEFINE_PROP_END_OF_LIST(),
> +};

Looks like some odd spacing has crept into the pnv_phb_properties[] array above?

> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
> +{
> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +
> +    hc->root_bus_path = pnv_phb_root_bus_path;
> +    dc->realize = pnv_phb_realize;
> +    device_class_set_props(dc, pnv_phb_properties);
> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
> +    dc->user_creatable = true;
> +}
> +
> +static void pnv_phb_register_type(void)
> +{
> +    static const TypeInfo pnv_phb_type_info = {
> +        .name          = TYPE_PNV_PHB,
> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
> +        .instance_size = sizeof(PnvPHB),
> +        .class_init    = pnv_phb_class_init,
> +    };
> +
> +    type_register_static(&pnv_phb_type_info);
> +}
> +type_init(pnv_phb_register_type)
> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
> new file mode 100644
> index 0000000000..a7cc8610e2
> --- /dev/null
> +++ b/hw/pci-host/pnv_phb.h
> @@ -0,0 +1,39 @@
> +/*
> + * QEMU PowerPC PowerNV Proxy PHB model
> + *
> + * Copyright (c) 2022, IBM Corporation.
> + *
> + * This code is licensed under the GPL version 2 or later. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#ifndef PCI_HOST_PNV_PHB_H
> +#define PCI_HOST_PNV_PHB_H
> +
> +#include "hw/pci/pcie_host.h"
> +#include "hw/pci/pcie_port.h"
> +#include "qom/object.h"
> +
> +typedef struct PnvChip PnvChip;
> +typedef struct PnvPhb4PecState PnvPhb4PecState;
> +
> +struct PnvPHB {
> +    PCIExpressHost parent_obj;
> +
> +    uint32_t chip_id;
> +    uint32_t phb_id;
> +    uint32_t version;
> +    char bus_path[8];
> +
> +    PnvChip *chip;
> +
> +    PnvPhb4PecState *pec;
> +
> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
> +    Object *backend;
> +};
> +
> +#define TYPE_PNV_PHB "pnv-phb"
> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
> +
> +#endif /* PCI_HOST_PNV_PHB_H */


ATB,

Mark.

Re: [PATCH v2 04/16] ppc/pnv: change PnvPHB3 to be a PnvPHB backend

[Mark Cave-Ayland]

On 31/05/2022 22:49, Daniel Henrique Barboza wrote:

> We need a handful of changes that needs to be done in a single swoop to
> turn PnvPHB3 into a PnvPHB backend.
> 
> In the PnvPHB3, since the PnvPHB device implements PCIExpressHost and
> will hold the PCI bus, change PnvPHB3 parent to TYPE_DEVICE. There are a
> couple of instances in pnv_phb3.c that needs to access the PCI bus, so a
> phb_base pointer is added to allow access to the parent PnvPHB. The
> PnvPHB3 root port will now be connected to a PnvPHB object.
> 
> In pnv.c, the powernv8 machine chip8 will now hold an array of PnvPHB
> objects.  pnv_get_phb3_child() needs to be adapted to return the PnvPHB3
> backend from the PnvPHB child. A global property is added in
> pnv_machine_power8_class_init() to ensure that all PnvPHBs are created
> with phb->version = 3.
> 
> After all these changes we're still able to boot a powernv8 machine with
> default settings. The real gain will come with user created PnvPHB
> devices, coming up next.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb3.c         | 29 ++++++++---------------------
>   hw/ppc/pnv.c                   | 21 +++++++++++++++++----
>   include/hw/pci-host/pnv_phb3.h |  5 ++++-
>   include/hw/ppc/pnv.h           |  3 ++-
>   4 files changed, 31 insertions(+), 27 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
> index 60584e2aae..a39aa0e8c4 100644
> --- a/hw/pci-host/pnv_phb3.c
> +++ b/hw/pci-host/pnv_phb3.c
> @@ -11,6 +11,7 @@
>   #include "qapi/visitor.h"
>   #include "qapi/error.h"
>   #include "hw/pci-host/pnv_phb3_regs.h"
> +#include "hw/pci-host/pnv_phb.h"
>   #include "hw/pci-host/pnv_phb3.h"
>   #include "hw/pci/pcie_host.h"
>   #include "hw/pci/pcie_port.h"
> @@ -26,7 +27,7 @@
>   
>   static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>       uint8_t bus, devfn;
>   
> @@ -590,7 +591,7 @@ void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size)
>   uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size)
>   {
>       PnvPHB3 *phb = opaque;
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       uint64_t val;
>   
>       if ((off & 0xfffc) == PHB_CONFIG_DATA) {
> @@ -1057,8 +1058,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
>                             "phb3-regs", 0x1000);
>   
>       pnv_phb3_bus_init(dev, phb);
> -
> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
>   }
>   
>   void pnv_phb3_update_regions(PnvPHB3 *phb)
> @@ -1083,38 +1082,26 @@ void pnv_phb3_update_regions(PnvPHB3 *phb)
>       pnv_phb3_check_all_m64s(phb);
>   }
>   
> -static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge,
> -                                          PCIBus *rootbus)
> -{
> -    PnvPHB3 *phb = PNV_PHB3(host_bridge);
> -
> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
> -             phb->chip_id, phb->phb_id);
> -    return phb->bus_path;
> -}
> -
>   static Property pnv_phb3_properties[] = {
>           DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0),
>           DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0),
>           DEFINE_PROP_LINK("chip", PnvPHB3, chip, TYPE_PNV_CHIP, PnvChip *),
> +        DEFINE_PROP_LINK("phb-base", PnvPHB3, phb_base, TYPE_PNV_PHB, PnvPHB *),
>           DEFINE_PROP_END_OF_LIST(),
>   };
>   
>   static void pnv_phb3_class_init(ObjectClass *klass, void *data)
>   {
> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>       DeviceClass *dc = DEVICE_CLASS(klass);
>   
> -    hc->root_bus_path = pnv_phb3_root_bus_path;
>       dc->realize = pnv_phb3_realize;
>       device_class_set_props(dc, pnv_phb3_properties);
> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>       dc->user_creatable = false;
>   }
>   
>   static const TypeInfo pnv_phb3_type_info = {
>       .name          = TYPE_PNV_PHB3,
> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
> +    .parent = TYPE_DEVICE,
>       .instance_size = sizeof(PnvPHB3),
>       .class_init    = pnv_phb3_class_init,
>       .instance_init = pnv_phb3_instance_init,
> @@ -1146,11 +1133,11 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)
>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>       PCIDevice *pci = PCI_DEVICE(dev);
>       PCIBus *bus = pci_get_bus(pci);
> -    PnvPHB3 *phb = NULL;
> +    PnvPHB *phb = NULL;
>       Error *local_err = NULL;
>   
> -    phb = (PnvPHB3 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> -                                          TYPE_PNV_PHB3);
> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> +                                         TYPE_PNV_PHB);

I realize that this has come from existing code, however these days there generally 
isn't a good reason for anything to access bus->qbus directly (and C casts without a 
QOM macro often need a closer look). I'm also not convinced by the use of 
object_dynamic_cast() here either. Could this be rewritten as something like:

     phb = PNV_PHB(PNV_PHB3_ROOT_PORT(dev)->phb_base);

>       if (!phb) {
>           error_setg(errp,
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index aaf4d241c3..6cd0af9adf 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -43,6 +43,7 @@
>   #include "hw/ipmi/ipmi.h"
>   #include "target/ppc/mmu-hash64.h"
>   #include "hw/pci/msi.h"
> +#include "hw/pci-host/pnv_phb.h"
>   
>   #include "hw/ppc/xics.h"
>   #include "hw/qdev-properties.h"
> @@ -654,7 +655,13 @@ static ISABus *pnv_isa_create(PnvChip *chip, Error **errp)
>   
>   static PnvPHB3 *pnv_get_phb3_child(Object *child)
>   {
> -    return (PnvPHB3 *)object_dynamic_cast(child, TYPE_PNV_PHB3);
> +    PnvPHB *phb = (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);

And here, assuming child is of type TYPE_PNV_PHB then as it is already known at 
compile time so I think this could become just:

     PnvPHB *phb = PNV_PHB(child);

> +    if (!phb) {
> +        return NULL;
> +    }
> +
> +    return (PnvPHB3 *)phb->backend;
>   }
>   
>   static int pnv_chip_power8_pic_print_info_child(Object *child, void *opaque)
> @@ -1160,7 +1167,7 @@ static void pnv_chip_power8_instance_init(Object *obj)
>       chip8->num_phbs = pcc->num_phbs;
>   
>       for (i = 0; i < chip8->num_phbs; i++) {
> -        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB3);
> +        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB);
>       }
>   
>   }
> @@ -1282,9 +1289,9 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp)
>       memory_region_add_subregion(get_system_memory(), PNV_HOMER_BASE(chip),
>                                   &chip8->homer.regs);
>   
> -    /* PHB3 controllers */
> +    /* PHB controllers */
>       for (i = 0; i < chip8->num_phbs; i++) {
> -        PnvPHB3 *phb = &chip8->phbs[i];
> +        PnvPHB *phb = &chip8->phbs[i];
>   
>           object_property_set_int(OBJECT(phb), "index", i, &error_fatal);
>           object_property_set_int(OBJECT(phb), "chip-id", chip->chip_id,
> @@ -1957,6 +1964,7 @@ static int pnv_ics_get_child(Object *child, void *opaque)
>               args->ics = ICS(&phb3->msis);
>           }
>       }
> +
>       return args->ics ? 1 : 0;
>   }
>   
> @@ -2112,8 +2120,13 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>       static const char compat[] = "qemu,powernv8\0qemu,powernv\0ibm,powernv";
>   
> +    static GlobalProperty phb_compat[] = {
> +        { TYPE_PNV_PHB, "version", "3" },
> +    };
> +
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   
>       xic->icp_get = pnv_icp_get;
>       xic->ics_get = pnv_ics_get;
> diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
> index 1375f18fc1..3b9ff1096a 100644
> --- a/include/hw/pci-host/pnv_phb3.h
> +++ b/include/hw/pci-host/pnv_phb3.h
> @@ -14,6 +14,7 @@
>   #include "hw/pci/pcie_port.h"
>   #include "hw/ppc/xics.h"
>   #include "qom/object.h"
> +#include "hw/pci-host/pnv_phb.h"
>   
>   typedef struct PnvPHB3 PnvPHB3;
>   typedef struct PnvChip PnvChip;
> @@ -127,7 +128,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB3, PNV_PHB3)
>   #define PCI_MMIO_TOTAL_SIZE   (0x1ull << 60)
>   
>   struct PnvPHB3 {
> -    PCIExpressHost parent_obj;
> +    DeviceState parent;
> +
> +    PnvPHB *phb_base;
>   
>       uint32_t chip_id;
>       uint32_t phb_id;
> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> index 86cb7d7f97..4595db418e 100644
> --- a/include/hw/ppc/pnv.h
> +++ b/include/hw/ppc/pnv.h
> @@ -32,6 +32,7 @@
>   #include "hw/ppc/pnv_core.h"
>   #include "hw/pci-host/pnv_phb3.h"
>   #include "hw/pci-host/pnv_phb4.h"
> +#include "hw/pci-host/pnv_phb.h"
>   #include "qom/object.h"
>   
>   #define TYPE_PNV_CHIP "pnv-chip"
> @@ -80,7 +81,7 @@ struct Pnv8Chip {
>       PnvHomer     homer;
>   
>   #define PNV8_CHIP_PHB3_MAX 4
> -    PnvPHB3      phbs[PNV8_CHIP_PHB3_MAX];
> +    PnvPHB       phbs[PNV8_CHIP_PHB3_MAX];
>       uint32_t     num_phbs;
>   
>       XICSFabric    *xics;


ATB,

Mark.

Re: [PATCH v2 07/16] ppc/pnv: change PnvPHB4 to be a PnvPHB backend

[Mark Cave-Ayland]

On 31/05/2022 22:49, Daniel Henrique Barboza wrote:

> Change the parent type of the PnvPHB4 device to TYPE_PARENT since the
> PCI bus is going to be initialized by the PnvPHB parent. Functions that
> needs to access the bus via a PnvPHB4 object can do so via the
> phb4->phb_base pointer.
> 
> pnv_phb4_pec now creates a PnvPHB object.
> 
> The powernv9 machine class will create PnvPHB devices with version '4'.
> powernv10 will create using version '5'. Both are using global machine
> properties in their class_init() to do that.
> 
> These changes will benefit us when adding PnvPHB user creatable devices
> for powernv9 and powernv10.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb4.c         | 29 +++++++++--------------------
>   hw/pci-host/pnv_phb4_pec.c     |  6 +-----
>   hw/ppc/pnv.c                   | 20 +++++++++++++++++++-
>   include/hw/pci-host/pnv_phb4.h |  5 ++++-
>   4 files changed, 33 insertions(+), 27 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
> index ae5494fe72..22cf1c2a5e 100644
> --- a/hw/pci-host/pnv_phb4.c
> +++ b/hw/pci-host/pnv_phb4.c
> @@ -49,7 +49,7 @@ static inline uint64_t SETFIELD(uint64_t mask, uint64_t word,
>   
>   static PCIDevice *pnv_phb4_find_cfg_dev(PnvPHB4 *phb)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>       uint8_t bus, devfn;
>   
> @@ -145,7 +145,7 @@ static uint64_t pnv_phb4_config_read(PnvPHB4 *phb, unsigned off,
>   static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>                                        unsigned size, uint64_t val)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       PCIDevice *pdev;
>   
>       if (size != 4) {
> @@ -166,7 +166,7 @@ static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>   static uint64_t pnv_phb4_rc_config_read(PnvPHB4 *phb, unsigned off,
>                                           unsigned size)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       PCIDevice *pdev;
>       uint64_t val;
>   
> @@ -1608,16 +1608,6 @@ static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>       pnv_phb4_xscom_realize(phb);
>   }
>   
> -static const char *pnv_phb4_root_bus_path(PCIHostState *host_bridge,
> -                                          PCIBus *rootbus)
> -{
> -    PnvPHB4 *phb = PNV_PHB4(host_bridge);
> -
> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
> -             phb->chip_id, phb->phb_id);
> -    return phb->bus_path;
> -}
> -
>   /*
>    * Address base trigger mode (POWER10)
>    *
> @@ -1702,19 +1692,18 @@ static Property pnv_phb4_properties[] = {
>           DEFINE_PROP_UINT32("chip-id", PnvPHB4, chip_id, 0),
>           DEFINE_PROP_LINK("pec", PnvPHB4, pec, TYPE_PNV_PHB4_PEC,
>                            PnvPhb4PecState *),
> +        DEFINE_PROP_LINK("phb-base", PnvPHB4, phb_base,
> +                         TYPE_PNV_PHB, PnvPHB *),
>           DEFINE_PROP_END_OF_LIST(),
>   };
>   
>   static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>   {
> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>       DeviceClass *dc = DEVICE_CLASS(klass);
>       XiveNotifierClass *xfc = XIVE_NOTIFIER_CLASS(klass);
>   
> -    hc->root_bus_path   = pnv_phb4_root_bus_path;
>       dc->realize         = pnv_phb4_realize;
>       device_class_set_props(dc, pnv_phb4_properties);
> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>       dc->user_creatable  = false;
>   
>       xfc->notify         = pnv_phb4_xive_notify;
> @@ -1722,7 +1711,7 @@ static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>   
>   static const TypeInfo pnv_phb4_type_info = {
>       .name          = TYPE_PNV_PHB4,
> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
> +    .parent        = TYPE_DEVICE,
>       .instance_init = pnv_phb4_instance_init,
>       .instance_size = sizeof(PnvPHB4),
>       .class_init    = pnv_phb4_class_init,
> @@ -1785,11 +1774,11 @@ static void pnv_phb4_root_port_realize(DeviceState *dev, Error **errp)
>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>       PCIDevice *pci = PCI_DEVICE(dev);
>       PCIBus *bus = pci_get_bus(pci);
> -    PnvPHB4 *phb = NULL;
> +    PnvPHB *phb = NULL;
>       Error *local_err = NULL;
>   
> -    phb = (PnvPHB4 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> -                                          TYPE_PNV_PHB4);
> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> +                                         TYPE_PNV_PHB);

Same comment here re: accessing bus->qbus directly.

>       if (!phb) {
>           error_setg(errp, "%s must be connected to pnv-phb4 buses", dev->id);
> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
> index 61bc0b503e..888ecbe8f3 100644
> --- a/hw/pci-host/pnv_phb4_pec.c
> +++ b/hw/pci-host/pnv_phb4_pec.c
> @@ -115,8 +115,7 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>                                           int stack_no,
>                                           Error **errp)
>   {
> -    PnvPhb4PecClass *pecc = PNV_PHB4_PEC_GET_CLASS(pec);
> -    PnvPHB4 *phb = PNV_PHB4(qdev_new(pecc->phb_type));
> +    PnvPHB *phb = PNV_PHB(qdev_new(TYPE_PNV_PHB));
>       int phb_id = pnv_phb4_pec_get_phb_id(pec, stack_no);
>   
>       object_property_add_child(OBJECT(pec), "phb[*]", OBJECT(phb));
> @@ -130,9 +129,6 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>       if (!sysbus_realize(SYS_BUS_DEVICE(phb), errp)) {
>           return;
>       }
> -
> -    /* Add a single Root port if running with defaults */
> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), pecc->rp_model);
>   }
>   
>   static void pnv_pec_realize(DeviceState *dev, Error **errp)
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 081b6839cc..3b0b230e49 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -688,7 +688,14 @@ static void pnv_chip_power8_pic_print_info(PnvChip *chip, Monitor *mon)
>   static int pnv_chip_power9_pic_print_info_child(Object *child, void *opaque)
>   {
>       Monitor *mon = opaque;
> -    PnvPHB4 *phb4 = (PnvPHB4 *) object_dynamic_cast(child, TYPE_PNV_PHB4);
> +    PnvPHB *phb =  (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);

I'm sure this could just be:

     PnvPHB *phb = PNV_PHB(child);

> +    PnvPHB4 *phb4;
> +
> +    if (!phb) {
> +        return 0;
> +    }
> +
> +    phb4 = (PnvPHB4 *)phb->backend;

and you should be able to do:

     phb4 = PNV_PHB4(phb->backend);

My preference for using the QOM macros where possible is that they also include a 
type check that assert()s if you assign the wrong type of QOM object, which a direct 
C cast would miss.

>       if (phb4) {
>           pnv_phb4_pic_print_info(phb4, mon);
> @@ -2164,8 +2171,14 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>       static const char compat[] = "qemu,powernv9\0ibm,powernv";
>   
> +    static GlobalProperty phb_compat[] = {
> +        { TYPE_PNV_PHB, "version", "4" },
> +    };
> +
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
> +
>       xfc->match_nvt = pnv_match_nvt;
>   
>       mc->alias = "powernv";
> @@ -2182,8 +2195,13 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>       XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
>       static const char compat[] = "qemu,powernv10\0ibm,powernv";
>   
> +    static GlobalProperty phb_compat[] = {
> +        { TYPE_PNV_PHB, "version", "5" },
> +    };
> +
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   
>       pmc->compat = compat;
>       pmc->compat_size = sizeof(compat);
> diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
> index 90843ac3a9..f22253358f 100644
> --- a/include/hw/pci-host/pnv_phb4.h
> +++ b/include/hw/pci-host/pnv_phb4.h
> @@ -18,6 +18,7 @@
>   typedef struct PnvPhb4PecState PnvPhb4PecState;
>   typedef struct PnvPhb4PecStack PnvPhb4PecStack;
>   typedef struct PnvPHB4 PnvPHB4;
> +typedef struct PnvPHB PnvPHB;
>   typedef struct PnvChip PnvChip;
>   
>   /*
> @@ -78,7 +79,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB4, PNV_PHB4)
>   #define PCI_MMIO_TOTAL_SIZE        (0x1ull << 60)
>   
>   struct PnvPHB4 {
> -    PCIExpressHost parent_obj;
> +    DeviceState parent;
> +
> +    PnvPHB *phb_base;
>   
>       uint32_t chip_id;
>       uint32_t phb_id;


ATB,

Mark.

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Mark Cave-Ayland]

On 31/05/2022 22:49, Daniel Henrique Barboza wrote:

> We have two very similar root-port devices, pnv-phb3-root-port and
> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
> that, until now, has no additional attributes.
> 
> The main difference between the PHB3 and PHB4 root ports is that
> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
> other differences can be merged in a single device without too much
> trouble.
> 
> This patch introduces the unified pnv-phb-root-port that, in time, will
> be used as the default root port for the pnv-phb device.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>   hw/pci-host/pnv_phb.h |  17 +++++++
>   2 files changed, 116 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> index 321c4e768a..5047e90d3a 100644
> --- a/hw/pci-host/pnv_phb.c
> +++ b/hw/pci-host/pnv_phb.c
> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>       dc->user_creatable = true;
>   }
>   
> -static void pnv_phb_register_type(void)
> +static void pnv_phb_root_port_reset(DeviceState *dev)
>   {
> -    static const TypeInfo pnv_phb_type_info = {
> -        .name          = TYPE_PNV_PHB,
> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
> -        .instance_size = sizeof(PnvPHB),
> -        .class_init    = pnv_phb_class_init,
> -    };
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
> +    PCIDevice *d = PCI_DEVICE(dev);
> +    uint8_t *conf = d->config;
>   
> +    rpc->parent_reset(dev);
> +
> +    if (rootport->version == 3) {
> +        return;
> +    }
> +
> +    /* PHB4 and later requires these extra reset steps */
> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
> +                               PCI_IO_RANGE_MASK & 0xff);
> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
> +                                 PCI_IO_RANGE_MASK & 0xff);
> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
> +    pci_config_set_interrupt_pin(conf, 0);
> +}
> +
> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
> +{
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
> +    PCIDevice *pci = PCI_DEVICE(dev);
> +    PCIBus *bus = pci_get_bus(pci);
> +    PnvPHB *phb = NULL;
> +    Error *local_err = NULL;
> +
> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> +                                          TYPE_PNV_PHB);

Same here too.

> +    if (!phb) {
> +        error_setg(errp,
> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
> +        return;
> +    }
> +
> +    /* Set unique chassis/slot values for the root port */
> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);

Again this is from older code, but we already have dev so these could be:

     qdev_prop_set_uint8(dev, "chassis", phb->chip_id);
     qdev_prop_set_uint16(dev, "slot", phb->phb_id);

> +    rpc->parent_realize(dev, &local_err);
> +    if (local_err) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }
> +    pci_config_set_interrupt_pin(pci->config, 0);
> +}
> +
> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
> +{
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
> +
> +    dc->desc     = "IBM PHB PCIE Root Port";
> +
> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
> +                                    &rpc->parent_realize);
> +
> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
> +                                  &rpc->parent_reset);
> +    dc->reset = &pnv_phb_root_port_reset;
> +
> +    dc->user_creatable = true;
> +
> +    k->vendor_id = PCI_VENDOR_ID_IBM;
> +    /* device_id represents the latest PHB root port version supported */
> +    k->device_id = PNV_PHB5_DEVICE_ID;
> +    k->revision  = 0;
> +
> +    rpc->exp_offset = 0x48;
> +    rpc->aer_offset = 0x100;
> +}
> +
> +static const TypeInfo pnv_phb_type_info = {
> +    .name          = TYPE_PNV_PHB,
> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
> +    .instance_size = sizeof(PnvPHB),
> +    .class_init    = pnv_phb_class_init,
> +};
> +
> +static const TypeInfo pnv_phb_root_port_info = {
> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
> +    .parent        = TYPE_PCIE_ROOT_PORT,
> +    .instance_size = sizeof(PnvPHBRootPort),
> +    .class_init    = pnv_phb_root_port_class_init,
> +};
> +
> +static void pnv_phb_register_types(void)
> +{
>       type_register_static(&pnv_phb_type_info);
> +    type_register_static(&pnv_phb_root_port_info);
>   }
> -type_init(pnv_phb_register_type)
> +
> +type_init(pnv_phb_register_types)
> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
> index a7cc8610e2..c8eab4b767 100644
> --- a/hw/pci-host/pnv_phb.h
> +++ b/hw/pci-host/pnv_phb.h
> @@ -36,4 +36,21 @@ struct PnvPHB {
>   #define TYPE_PNV_PHB "pnv-phb"
>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>   
> +/*
> + * PHB PCIe Root port
> + */
> +#define PNV_PHB3_DEVICE_ID         0x03dc
> +#define PNV_PHB4_DEVICE_ID         0x04c1
> +#define PNV_PHB5_DEVICE_ID         0x0652
> +
> +typedef struct PnvPHBRootPort {
> +    PCIESlot parent_obj;
> +
> +    uint32_t version;
> +} PnvPHBRootPort;
> +
> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
> +#define PNV_PHB_ROOT_PORT(obj) \
> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
> +
>   #endif /* PCI_HOST_PNV_PHB_H */


ATB,

Mark.

Re: [PATCH v2 00/16] powernv: introduce pnv-phb base/proxy devices

[Mark Cave-Ayland]

On 31/05/2022 22:49, Daniel Henrique Barboza wrote:

> Hi,
> 
> This v2 is considerable different from the first version due to the
> review provided by Mark Cave-Ayland.
> 
> We're now preserving all PnvPHB3/4/5 implementations already in place.
> The PnvPHB device now acts as a base/proxy of the existing PHBs, which
> are turned into backends of the base PnvPHB device.
> 
> QOM is being more used this time by passing through properties to the
> PHB backends from the base device, and by setting the phb->version
> via global machine properties in each machine.
> 
> The changes made impact both user creatable and default devices, meaning
> that now the powernv machines are using the PnvPHB base device in all
> circunstances.
> 
> The one thing that I didn't change from v1 is the root port revamp. I
> didn't find enough reason to do the same thing we did with the PnvPHBs,
> given that all that differs them is the reset() callback of
> phb4_root_port. This means that patches 14-17 from v1 are still mostly
> the same.
> 
> 
> Changes from v1:
> - lots of changes in patches 1-6 and 7 due to the change of direction
> - patch 10 from v1: removed
> - PnvPHB.version is now being removed in patch 16
> - several other minor changes due to changes in the initial patches
> - v1 link: https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg01410.html
> 
> Daniel Henrique Barboza (16):
>    ppc/pnv: add PHB3 bus init helper
>    ppc/pnv: add pnv_get_phb3_child()
>    ppc/pnv: add PnvPHB base/proxy device
>    ppc/pnv: change PnvPHB3 to be a PnvPHB backend
>    ppc/pnv: user created pnv-phb for powernv8
>    ppc/pnv: add PHB4 bus init helper
>    ppc/pnv: change PnvPHB4 to be a PnvPHB backend
>    ppc/pnv: user created pnv-phb for powernv9
>    ppc/pnv: change pnv_phb4_get_pec() to also retrieve chip10->pecs
>    ppc/pnv: user creatable pnv-phb for powernv10
>    ppc/pnv: add pnv-phb-root-port device
>    ppc/pnv: remove pnv-phb3-root-port
>    ppc/pnv: remove pnv-phb4-root-port
>    ppc/pnv: remove 'phb_rootport_typename' in pnv_phb_realize()
>    ppc/pnv: remove pecc->rp_model
>    ppc/pnv: remove PnvPHB4.version
> 
>   hw/pci-host/meson.build        |   3 +-
>   hw/pci-host/pnv_phb.c          | 219 ++++++++++++++++++++++++++++++++
>   hw/pci-host/pnv_phb.h          |  56 ++++++++
>   hw/pci-host/pnv_phb3.c         | 144 ++++++++-------------
>   hw/pci-host/pnv_phb4.c         | 226 ++++++++++++++-------------------
>   hw/pci-host/pnv_phb4_pec.c     |  14 +-
>   hw/ppc/pnv.c                   |  78 ++++++++++--
>   include/hw/pci-host/pnv_phb3.h |  12 +-
>   include/hw/pci-host/pnv_phb4.h |  18 +--
>   include/hw/ppc/pnv.h           |   4 +-
>   10 files changed, 512 insertions(+), 262 deletions(-)
>   create mode 100644 hw/pci-host/pnv_phb.c
>   create mode 100644 hw/pci-host/pnv_phb.h

I've had a quick look over the series, and whilst I'm not overly familiar with the 
PMV PHB side, this looks much better to me from a QOM/qdev modelling perspective. 
I've added a few comments but in general I'm fairly happy with the way this is going.


ATB,

Mark.

Re: [PATCH v2 00/16] powernv: introduce pnv-phb base/proxy devices

[Frederic Barrat]

On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
> Hi,
> 
> This v2 is considerable different from the first version due to the
> review provided by Mark Cave-Ayland.
> 
> We're now preserving all PnvPHB3/4/5 implementations already in place.
> The PnvPHB device now acts as a base/proxy of the existing PHBs, which
> are turned into backends of the base PnvPHB device.
> 
> QOM is being more used this time by passing through properties to the
> PHB backends from the base device, and by setting the phb->version
> via global machine properties in each machine.
> 
> The changes made impact both user creatable and default devices, meaning
> that now the powernv machines are using the PnvPHB base device in all
> circunstances.
> 
> The one thing that I didn't change from v1 is the root port revamp. I
> didn't find enough reason to do the same thing we did with the PnvPHBs,
> given that all that differs them is the reset() callback of
> phb4_root_port. This means that patches 14-17 from v1 are still mostly
> the same.
> 
> 
> Changes from v1:
> - lots of changes in patches 1-6 and 7 due to the change of direction
> - patch 10 from v1: removed
> - PnvPHB.version is now being removed in patch 16
> - several other minor changes due to changes in the initial patches
> - v1 link: https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg01410.html


Hi Daniel,

I have a few mostly minor comments coming but that series looks pretty 
good already! And you did a pretty good job splitting it, making the 
review easier. Thanks!

   Fred



> Daniel Henrique Barboza (16):
>    ppc/pnv: add PHB3 bus init helper
>    ppc/pnv: add pnv_get_phb3_child()
>    ppc/pnv: add PnvPHB base/proxy device
>    ppc/pnv: change PnvPHB3 to be a PnvPHB backend
>    ppc/pnv: user created pnv-phb for powernv8
>    ppc/pnv: add PHB4 bus init helper
>    ppc/pnv: change PnvPHB4 to be a PnvPHB backend
>    ppc/pnv: user created pnv-phb for powernv9
>    ppc/pnv: change pnv_phb4_get_pec() to also retrieve chip10->pecs
>    ppc/pnv: user creatable pnv-phb for powernv10
>    ppc/pnv: add pnv-phb-root-port device
>    ppc/pnv: remove pnv-phb3-root-port
>    ppc/pnv: remove pnv-phb4-root-port
>    ppc/pnv: remove 'phb_rootport_typename' in pnv_phb_realize()
>    ppc/pnv: remove pecc->rp_model
>    ppc/pnv: remove PnvPHB4.version
> 
>   hw/pci-host/meson.build        |   3 +-
>   hw/pci-host/pnv_phb.c          | 219 ++++++++++++++++++++++++++++++++
>   hw/pci-host/pnv_phb.h          |  56 ++++++++
>   hw/pci-host/pnv_phb3.c         | 144 ++++++++-------------
>   hw/pci-host/pnv_phb4.c         | 226 ++++++++++++++-------------------
>   hw/pci-host/pnv_phb4_pec.c     |  14 +-
>   hw/ppc/pnv.c                   |  78 ++++++++++--
>   include/hw/pci-host/pnv_phb3.h |  12 +-
>   include/hw/pci-host/pnv_phb4.h |  18 +--
>   include/hw/ppc/pnv.h           |   4 +-
>   10 files changed, 512 insertions(+), 262 deletions(-)
>   create mode 100644 hw/pci-host/pnv_phb.c
>   create mode 100644 hw/pci-host/pnv_phb.h
> 

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Frederic Barrat]

On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
> The PnvPHB device is going to be the base device for all other powernv
> PHBs. It consists of a device that has the same user API as the other
> PHB, namely being a PCIHostBridge and having chip-id and index
> properties. It also has a 'backend' pointer that will be initialized
> with the PHB implementation that the device is going to use.
> 
> The initialization of the PHB backend is done by checking the PHB
> version via a 'version' attribute that can be set via a global machine
> property.  The 'version' field will be used to make adjustments based on
> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
> helper will be added later on.
> 
> For now let's add the basic logic of the PnvPHB object, which consists
> mostly of pnv_phb_realize() doing all the work of checking the
> phb->version set, initializing the proper backend, passing through its
> attributes to the chosen backend, finalizing the backend realize and
> adding a root port in the end.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/meson.build |   3 +-
>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>   3 files changed, 164 insertions(+), 1 deletion(-)
>   create mode 100644 hw/pci-host/pnv_phb.c
>   create mode 100644 hw/pci-host/pnv_phb.h
> 
> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
> index c07596d0d1..e832babc9d 100644
> --- a/hw/pci-host/meson.build
> +++ b/hw/pci-host/meson.build
> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>     'pnv_phb3_msi.c',
>     'pnv_phb3_pbcq.c',
>     'pnv_phb4.c',
> -  'pnv_phb4_pec.c'
> +  'pnv_phb4_pec.c',
> +  'pnv_phb.c',
>   ))
> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
> new file mode 100644
> index 0000000000..fa8472622f
> --- /dev/null
> +++ b/hw/pci-host/pnv_phb.c
> @@ -0,0 +1,123 @@
> +/*
> + * QEMU PowerPC PowerNV Proxy PHB model
> + *
> + * Copyright (c) 2022, IBM Corporation.
> + *
> + * This code is licensed under the GPL version 2 or later. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/log.h"
> +#include "qapi/visitor.h"
> +#include "qapi/error.h"
> +#include "hw/pci-host/pnv_phb.h"
> +#include "hw/pci-host/pnv_phb3.h"
> +#include "hw/pci-host/pnv_phb4.h"
> +#include "hw/ppc/pnv.h"
> +#include "hw/qdev-properties.h"
> +#include "qom/object.h"
> +
> +
> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
> +{
> +    PnvPHB *phb = PNV_PHB(dev);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
> +    g_autofree char *phb_typename = NULL;
> +    g_autofree char *phb_rootport_typename = NULL;
> +
> +    if (!phb->version) {
> +        error_setg(errp, "version not specified");
> +        return;
> +    }
> +
> +    switch (phb->version) {
> +    case 3:
> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
> +        break;
> +    case 4:
> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
> +        break;
> +    case 5:
> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> +
> +    phb->backend = object_new(phb_typename);
> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
> +
> +    /* Passthrough child device properties to the proxy device */
> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
> +
> +    if (phb->version == 3) {
> +        object_property_set_link(phb->backend, "chip",
> +                                 OBJECT(phb->chip), errp);
> +    } else {
> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
> +    }


The patch is fine, but it just highlights that we're doing something 
wrong. I don't believe there's any reason for the chip/pec/phb 
relationship to be different between P8 and P9/P10. One day, a brave 
soul could try to unify the models, it would avoid test like that.
It would be a good cleanup series to do if we ever extend the model with 
yet another version :-)



> +
> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
> +        return;
> +    }
> +
> +    if (phb->version == 3) {
> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
> +    }
> +
> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);



After we've removed the other instances (done in later patches), we 
could move pnv_phb_attach_root_port() to pnv_phb.c instead of pnv.c. It 
would be the perfect home for it as it starts looking off in pnv.c

   Fred



> +}
> +
> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
> +                                         PCIBus *rootbus)
> +{
> +    PnvPHB *phb = PNV_PHB(host_bridge);
> +
> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
> +             phb->chip_id, phb->phb_id);
> +    return phb->bus_path;
> +}
> +
> +static Property pnv_phb_properties[] = {
> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
> +
> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
> +
> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
> +                         PnvPhb4PecState *),
> +
> +        DEFINE_PROP_END_OF_LIST(),
> +};
> +
> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
> +{
> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +
> +    hc->root_bus_path = pnv_phb_root_bus_path;
> +    dc->realize = pnv_phb_realize;
> +    device_class_set_props(dc, pnv_phb_properties);
> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
> +    dc->user_creatable = true;
> +}
> +
> +static void pnv_phb_register_type(void)
> +{
> +    static const TypeInfo pnv_phb_type_info = {
> +        .name          = TYPE_PNV_PHB,
> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
> +        .instance_size = sizeof(PnvPHB),
> +        .class_init    = pnv_phb_class_init,
> +    };
> +
> +    type_register_static(&pnv_phb_type_info);
> +}
> +type_init(pnv_phb_register_type)
> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
> new file mode 100644
> index 0000000000..a7cc8610e2
> --- /dev/null
> +++ b/hw/pci-host/pnv_phb.h
> @@ -0,0 +1,39 @@
> +/*
> + * QEMU PowerPC PowerNV Proxy PHB model
> + *
> + * Copyright (c) 2022, IBM Corporation.
> + *
> + * This code is licensed under the GPL version 2 or later. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#ifndef PCI_HOST_PNV_PHB_H
> +#define PCI_HOST_PNV_PHB_H
> +
> +#include "hw/pci/pcie_host.h"
> +#include "hw/pci/pcie_port.h"
> +#include "qom/object.h"
> +
> +typedef struct PnvChip PnvChip;
> +typedef struct PnvPhb4PecState PnvPhb4PecState;
> +
> +struct PnvPHB {
> +    PCIExpressHost parent_obj;
> +
> +    uint32_t chip_id;
> +    uint32_t phb_id;
> +    uint32_t version;
> +    char bus_path[8];
> +
> +    PnvChip *chip;
> +
> +    PnvPhb4PecState *pec;
> +
> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
> +    Object *backend;
> +};
> +
> +#define TYPE_PNV_PHB "pnv-phb"
> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
> +
> +#endif /* PCI_HOST_PNV_PHB_H */

Re: [PATCH v2 07/16] ppc/pnv: change PnvPHB4 to be a PnvPHB backend

[Frederic Barrat]

On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
> Change the parent type of the PnvPHB4 device to TYPE_PARENT since the


s/TYPE_PARENT/TYPE_DEVICE

   Fred


> PCI bus is going to be initialized by the PnvPHB parent. Functions that
> needs to access the bus via a PnvPHB4 object can do so via the
> phb4->phb_base pointer.
> 
> pnv_phb4_pec now creates a PnvPHB object.
> 
> The powernv9 machine class will create PnvPHB devices with version '4'.
> powernv10 will create using version '5'. Both are using global machine
> properties in their class_init() to do that.
> 
> These changes will benefit us when adding PnvPHB user creatable devices
> for powernv9 and powernv10.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb4.c         | 29 +++++++++--------------------
>   hw/pci-host/pnv_phb4_pec.c     |  6 +-----
>   hw/ppc/pnv.c                   | 20 +++++++++++++++++++-
>   include/hw/pci-host/pnv_phb4.h |  5 ++++-
>   4 files changed, 33 insertions(+), 27 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
> index ae5494fe72..22cf1c2a5e 100644
> --- a/hw/pci-host/pnv_phb4.c
> +++ b/hw/pci-host/pnv_phb4.c
> @@ -49,7 +49,7 @@ static inline uint64_t SETFIELD(uint64_t mask, uint64_t word,
>   
>   static PCIDevice *pnv_phb4_find_cfg_dev(PnvPHB4 *phb)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>       uint8_t bus, devfn;
>   
> @@ -145,7 +145,7 @@ static uint64_t pnv_phb4_config_read(PnvPHB4 *phb, unsigned off,
>   static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>                                        unsigned size, uint64_t val)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       PCIDevice *pdev;
>   
>       if (size != 4) {
> @@ -166,7 +166,7 @@ static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>   static uint64_t pnv_phb4_rc_config_read(PnvPHB4 *phb, unsigned off,
>                                           unsigned size)
>   {
> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>       PCIDevice *pdev;
>       uint64_t val;
>   
> @@ -1608,16 +1608,6 @@ static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>       pnv_phb4_xscom_realize(phb);
>   }
>   
> -static const char *pnv_phb4_root_bus_path(PCIHostState *host_bridge,
> -                                          PCIBus *rootbus)
> -{
> -    PnvPHB4 *phb = PNV_PHB4(host_bridge);
> -
> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
> -             phb->chip_id, phb->phb_id);
> -    return phb->bus_path;
> -}
> -
>   /*
>    * Address base trigger mode (POWER10)
>    *
> @@ -1702,19 +1692,18 @@ static Property pnv_phb4_properties[] = {
>           DEFINE_PROP_UINT32("chip-id", PnvPHB4, chip_id, 0),
>           DEFINE_PROP_LINK("pec", PnvPHB4, pec, TYPE_PNV_PHB4_PEC,
>                            PnvPhb4PecState *),
> +        DEFINE_PROP_LINK("phb-base", PnvPHB4, phb_base,
> +                         TYPE_PNV_PHB, PnvPHB *),
>           DEFINE_PROP_END_OF_LIST(),
>   };
>   
>   static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>   {
> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>       DeviceClass *dc = DEVICE_CLASS(klass);
>       XiveNotifierClass *xfc = XIVE_NOTIFIER_CLASS(klass);
>   
> -    hc->root_bus_path   = pnv_phb4_root_bus_path;
>       dc->realize         = pnv_phb4_realize;
>       device_class_set_props(dc, pnv_phb4_properties);
> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>       dc->user_creatable  = false;
>   
>       xfc->notify         = pnv_phb4_xive_notify;
> @@ -1722,7 +1711,7 @@ static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>   
>   static const TypeInfo pnv_phb4_type_info = {
>       .name          = TYPE_PNV_PHB4,
> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
> +    .parent        = TYPE_DEVICE,
>       .instance_init = pnv_phb4_instance_init,
>       .instance_size = sizeof(PnvPHB4),
>       .class_init    = pnv_phb4_class_init,
> @@ -1785,11 +1774,11 @@ static void pnv_phb4_root_port_realize(DeviceState *dev, Error **errp)
>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>       PCIDevice *pci = PCI_DEVICE(dev);
>       PCIBus *bus = pci_get_bus(pci);
> -    PnvPHB4 *phb = NULL;
> +    PnvPHB *phb = NULL;
>       Error *local_err = NULL;
>   
> -    phb = (PnvPHB4 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> -                                          TYPE_PNV_PHB4);
> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
> +                                         TYPE_PNV_PHB);
>   
>       if (!phb) {
>           error_setg(errp, "%s must be connected to pnv-phb4 buses", dev->id);
> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
> index 61bc0b503e..888ecbe8f3 100644
> --- a/hw/pci-host/pnv_phb4_pec.c
> +++ b/hw/pci-host/pnv_phb4_pec.c
> @@ -115,8 +115,7 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>                                           int stack_no,
>                                           Error **errp)
>   {
> -    PnvPhb4PecClass *pecc = PNV_PHB4_PEC_GET_CLASS(pec);
> -    PnvPHB4 *phb = PNV_PHB4(qdev_new(pecc->phb_type));
> +    PnvPHB *phb = PNV_PHB(qdev_new(TYPE_PNV_PHB));
>       int phb_id = pnv_phb4_pec_get_phb_id(pec, stack_no);
>   
>       object_property_add_child(OBJECT(pec), "phb[*]", OBJECT(phb));
> @@ -130,9 +129,6 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>       if (!sysbus_realize(SYS_BUS_DEVICE(phb), errp)) {
>           return;
>       }
> -
> -    /* Add a single Root port if running with defaults */
> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), pecc->rp_model);
>   }
>   
>   static void pnv_pec_realize(DeviceState *dev, Error **errp)
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 081b6839cc..3b0b230e49 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -688,7 +688,14 @@ static void pnv_chip_power8_pic_print_info(PnvChip *chip, Monitor *mon)
>   static int pnv_chip_power9_pic_print_info_child(Object *child, void *opaque)
>   {
>       Monitor *mon = opaque;
> -    PnvPHB4 *phb4 = (PnvPHB4 *) object_dynamic_cast(child, TYPE_PNV_PHB4);
> +    PnvPHB *phb =  (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
> +    PnvPHB4 *phb4;
> +
> +    if (!phb) {
> +        return 0;
> +    }
> +
> +    phb4 = (PnvPHB4 *)phb->backend;
>   
>       if (phb4) {
>           pnv_phb4_pic_print_info(phb4, mon);
> @@ -2164,8 +2171,14 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>       static const char compat[] = "qemu,powernv9\0ibm,powernv";
>   
> +    static GlobalProperty phb_compat[] = {
> +        { TYPE_PNV_PHB, "version", "4" },
> +    };
> +
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
> +
>       xfc->match_nvt = pnv_match_nvt;
>   
>       mc->alias = "powernv";
> @@ -2182,8 +2195,13 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>       XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
>       static const char compat[] = "qemu,powernv10\0ibm,powernv";
>   
> +    static GlobalProperty phb_compat[] = {
> +        { TYPE_PNV_PHB, "version", "5" },
> +    };
> +
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   
>       pmc->compat = compat;
>       pmc->compat_size = sizeof(compat);
> diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
> index 90843ac3a9..f22253358f 100644
> --- a/include/hw/pci-host/pnv_phb4.h
> +++ b/include/hw/pci-host/pnv_phb4.h
> @@ -18,6 +18,7 @@
>   typedef struct PnvPhb4PecState PnvPhb4PecState;
>   typedef struct PnvPhb4PecStack PnvPhb4PecStack;
>   typedef struct PnvPHB4 PnvPHB4;
> +typedef struct PnvPHB PnvPHB;
>   typedef struct PnvChip PnvChip;
>   
>   /*
> @@ -78,7 +79,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB4, PNV_PHB4)
>   #define PCI_MMIO_TOTAL_SIZE        (0x1ull << 60)
>   
>   struct PnvPHB4 {
> -    PCIExpressHost parent_obj;
> +    DeviceState parent;
> +
> +    PnvPHB *phb_base;
>   
>       uint32_t chip_id;
>       uint32_t phb_id;

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Frederic Barrat]

On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
> To enable user creatable PnvPHB devices for powernv9 we'll revert the
> powernv9 related changes made in 9c10d86fee "ppc/pnv: Remove
> user-created PHB{3,4,5} devices".
> 
> This change alone isn't enough to enable user creatable devices for powernv10
> due to how pnv_phb4_get_pec() currently works. For now let's just enable it
> for powernv9 alone.
> 
> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> ---
>   hw/pci-host/pnv_phb4.c     | 58 +++++++++++++++++++++++++++++++++++++-
>   hw/pci-host/pnv_phb4_pec.c |  6 ++--
>   hw/ppc/pnv.c               |  2 ++
>   3 files changed, 63 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
> index 22cf1c2a5e..a5c8ae494b 100644
> --- a/hw/pci-host/pnv_phb4.c
> +++ b/hw/pci-host/pnv_phb4.c
> @@ -1571,13 +1571,69 @@ void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
>       pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE;
>   }
>   
> +static PnvPhb4PecState *pnv_phb4_get_pec(PnvChip *chip, PnvPHB4 *phb,
> +                                         Error **errp)
> +{
> +    Pnv9Chip *chip9 = PNV9_CHIP(chip);
> +    int chip_id = phb->chip_id;
> +    int index = phb->phb_id;
> +    int i, j;
> +
> +    for (i = 0; i < chip->num_pecs; i++) {
> +        /*
> +         * For each PEC, check the amount of phbs it supports
> +         * and see if the given phb4 index matches an index.
> +         */
> +        PnvPhb4PecState *pec = &chip9->pecs[i];
> +
> +        for (j = 0; j < pec->num_phbs; j++) {
> +            if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
> +                return pec;
> +            }
> +        }
> +    }
> +
> +    error_setg(errp,
> +               "pnv-phb4 chip-id %d index %d didn't match any existing PEC",
> +               chip_id, index);
> +
> +    return NULL;
> +}
> +
>   static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>   {
>       PnvPHB4 *phb = PNV_PHB4(dev);
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
> +    PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
>       XiveSource *xsrc = &phb->xsrc;
> +    BusState *s;
> +    Error *local_err = NULL;
>       int nr_irqs;
>       char name[32];
>   
> +    if (!chip) {
> +        error_setg(errp, "invalid chip id: %d", phb->chip_id);
> +        return;
> +    }
> +
> +    /* User created PHBs need to be assigned to a PEC */
> +    if (!phb->pec) {
> +        phb->pec = pnv_phb4_get_pec(chip, phb, &local_err);
> +        if (local_err) {
> +            error_propagate(errp, local_err);
> +            return;
> +        }
> +    }
> +
> +    /* Reparent the PHB to the chip to build the device tree */
> +    pnv_chip_parent_fixup(chip, OBJECT(phb->phb_base), phb->phb_id);


Didn't you mean to do that only for the user-created case? And why not 
attaching the PHB to the PEC instead of the chip, like in 
pnv_pec_default_phb_realize() ? I think it makes more sense to keep the 
chip->PEC->PHB parenting in the qom tree (and incidentally, that's where 
I'd prefer to have the phb3 model move to).
Also, the comment seems wrong to me. The qom parenting doesn't matter 
when building the device tree. We only iterate over the PHBs found in 
the array of the PEC object (cf. pnv_pec_dt_xscom())



> +    s = qdev_get_parent_bus(DEVICE(chip));
> +    if (!qdev_set_parent_bus(DEVICE(phb->phb_base), s, &local_err)) {
> +        error_propagate(errp, local_err);
> +        return;
> +    }


Same comment, I think that's only desirable for user-created devices. 
We're already calling sysbus_realize() for the default case.


Silly question: where does it break if a user tries to create 2 PHBs 
with the same index?


   Fred




>       /* Set the "big_phb" flag */
>       phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
>   
> @@ -1803,7 +1859,7 @@ static void pnv_phb4_root_port_class_init(ObjectClass *klass, void *data)
>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>   
>       dc->desc     = "IBM PHB4 PCIE Root Port";
> -    dc->user_creatable = false;
> +    dc->user_creatable = true;
>   
>       device_class_set_parent_realize(dc, pnv_phb4_root_port_realize,
>                                       &rpc->parent_realize);
> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
> index 888ecbe8f3..0e67f3a338 100644
> --- a/hw/pci-host/pnv_phb4_pec.c
> +++ b/hw/pci-host/pnv_phb4_pec.c
> @@ -146,8 +146,10 @@ static void pnv_pec_realize(DeviceState *dev, Error **errp)
>       pec->num_phbs = pecc->num_phbs[pec->index];
>   
>       /* Create PHBs if running with defaults */
> -    for (i = 0; i < pec->num_phbs; i++) {
> -        pnv_pec_default_phb_realize(pec, i, errp);
> +    if (defaults_enabled()) {
> +        for (i = 0; i < pec->num_phbs; i++) {
> +            pnv_pec_default_phb_realize(pec, i, errp);
> +        }
>       }
>   
>       /* Initialize the XSCOM regions for the PEC registers */
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 3b0b230e49..697a2b5302 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -2186,6 +2186,8 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       pmc->compat = compat;
>       pmc->compat_size = sizeof(compat);
>       pmc->dt_power_mgt = pnv_dt_power_mgt;
> +
> +    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
>   }
>   
>   static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Daniel Henrique Barboza]

On 6/2/22 04:18, Mark Cave-Ayland wrote:
> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
> 
>> The PnvPHB device is going to be the base device for all other powernv
>> PHBs. It consists of a device that has the same user API as the other
>> PHB, namely being a PCIHostBridge and having chip-id and index
>> properties. It also has a 'backend' pointer that will be initialized
>> with the PHB implementation that the device is going to use.
>>
>> The initialization of the PHB backend is done by checking the PHB
>> version via a 'version' attribute that can be set via a global machine
>> property.  The 'version' field will be used to make adjustments based on
>> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
>> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
>> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
>> helper will be added later on.
>>
>> For now let's add the basic logic of the PnvPHB object, which consists
>> mostly of pnv_phb_realize() doing all the work of checking the
>> phb->version set, initializing the proper backend, passing through its
>> attributes to the chosen backend, finalizing the backend realize and
>> adding a root port in the end.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/meson.build |   3 +-
>>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>>   3 files changed, 164 insertions(+), 1 deletion(-)
>>   create mode 100644 hw/pci-host/pnv_phb.c
>>   create mode 100644 hw/pci-host/pnv_phb.h
>>
>> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
>> index c07596d0d1..e832babc9d 100644
>> --- a/hw/pci-host/meson.build
>> +++ b/hw/pci-host/meson.build
>> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>>     'pnv_phb3_msi.c',
>>     'pnv_phb3_pbcq.c',
>>     'pnv_phb4.c',
>> -  'pnv_phb4_pec.c'
>> +  'pnv_phb4_pec.c',
>> +  'pnv_phb.c',
>>   ))
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> new file mode 100644
>> index 0000000000..fa8472622f
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -0,0 +1,123 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/log.h"
>> +#include "qapi/visitor.h"
>> +#include "qapi/error.h"
>> +#include "hw/pci-host/pnv_phb.h"
>> +#include "hw/pci-host/pnv_phb3.h"
>> +#include "hw/pci-host/pnv_phb4.h"
>> +#include "hw/ppc/pnv.h"
>> +#include "hw/qdev-properties.h"
>> +#include "qom/object.h"
>> +
>> +
>> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
>> +{
>> +    PnvPHB *phb = PNV_PHB(dev);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>> +    g_autofree char *phb_typename = NULL;
>> +    g_autofree char *phb_rootport_typename = NULL;
>> +
>> +    if (!phb->version) {
>> +        error_setg(errp, "version not specified");
>> +        return;
>> +    }
>> +
>> +    switch (phb->version) {
>> +    case 3:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
>> +        break;
>> +    case 4:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
>> +        break;
>> +    case 5:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
>> +        break;
>> +    default:
>> +        g_assert_not_reached();
>> +    }
>> +
>> +    phb->backend = object_new(phb_typename);
>> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
>> +
>> +    /* Passthrough child device properties to the proxy device */
>> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
>> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
>> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
>> +
>> +    if (phb->version == 3) {
>> +        object_property_set_link(phb->backend, "chip",
>> +                                 OBJECT(phb->chip), errp);
>> +    } else {
>> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
>> +    }
>> +
>> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (phb->version == 3) {
>> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
>> +    }
>> +
>> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
>> +}
>> +
>> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
>> +                                         PCIBus *rootbus)
>> +{
>> +    PnvPHB *phb = PNV_PHB(host_bridge);
>> +
>> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>> +             phb->chip_id, phb->phb_id);
>> +    return phb->bus_path;
>> +}
>> +
>> +static Property pnv_phb_properties[] = {
>> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
>> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
>> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
>> +
>> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
>> +
>> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
>> +                         PnvPhb4PecState *),
>> +
>> +        DEFINE_PROP_END_OF_LIST(),
>> +};
> 
> Looks like some odd spacing has crept into the pnv_phb_properties[] array above?

Hm, I'm pretty sure there's some other instances of these spaces around
the pnv code.


I'll fix this up and go after them.


Daniel

> 
>> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
>> +{
>> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +
>> +    hc->root_bus_path = pnv_phb_root_bus_path;
>> +    dc->realize = pnv_phb_realize;
>> +    device_class_set_props(dc, pnv_phb_properties);
>> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>> +    dc->user_creatable = true;
>> +}
>> +
>> +static void pnv_phb_register_type(void)
>> +{
>> +    static const TypeInfo pnv_phb_type_info = {
>> +        .name          = TYPE_PNV_PHB,
>> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +        .instance_size = sizeof(PnvPHB),
>> +        .class_init    = pnv_phb_class_init,
>> +    };
>> +
>> +    type_register_static(&pnv_phb_type_info);
>> +}
>> +type_init(pnv_phb_register_type)
>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>> new file mode 100644
>> index 0000000000..a7cc8610e2
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.h
>> @@ -0,0 +1,39 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef PCI_HOST_PNV_PHB_H
>> +#define PCI_HOST_PNV_PHB_H
>> +
>> +#include "hw/pci/pcie_host.h"
>> +#include "hw/pci/pcie_port.h"
>> +#include "qom/object.h"
>> +
>> +typedef struct PnvChip PnvChip;
>> +typedef struct PnvPhb4PecState PnvPhb4PecState;
>> +
>> +struct PnvPHB {
>> +    PCIExpressHost parent_obj;
>> +
>> +    uint32_t chip_id;
>> +    uint32_t phb_id;
>> +    uint32_t version;
>> +    char bus_path[8];
>> +
>> +    PnvChip *chip;
>> +
>> +    PnvPhb4PecState *pec;
>> +
>> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
>> +    Object *backend;
>> +};
>> +
>> +#define TYPE_PNV_PHB "pnv-phb"
>> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>> +
>> +#endif /* PCI_HOST_PNV_PHB_H */
> 
> 
> ATB,
> 
> Mark.

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Daniel Henrique Barboza]

On 6/2/22 13:16, Frederic Barrat wrote:
> 
> 
> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>> The PnvPHB device is going to be the base device for all other powernv
>> PHBs. It consists of a device that has the same user API as the other
>> PHB, namely being a PCIHostBridge and having chip-id and index
>> properties. It also has a 'backend' pointer that will be initialized
>> with the PHB implementation that the device is going to use.
>>
>> The initialization of the PHB backend is done by checking the PHB
>> version via a 'version' attribute that can be set via a global machine
>> property.  The 'version' field will be used to make adjustments based on
>> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
>> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
>> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
>> helper will be added later on.
>>
>> For now let's add the basic logic of the PnvPHB object, which consists
>> mostly of pnv_phb_realize() doing all the work of checking the
>> phb->version set, initializing the proper backend, passing through its
>> attributes to the chosen backend, finalizing the backend realize and
>> adding a root port in the end.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/meson.build |   3 +-
>>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>>   3 files changed, 164 insertions(+), 1 deletion(-)
>>   create mode 100644 hw/pci-host/pnv_phb.c
>>   create mode 100644 hw/pci-host/pnv_phb.h
>>
>> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
>> index c07596d0d1..e832babc9d 100644
>> --- a/hw/pci-host/meson.build
>> +++ b/hw/pci-host/meson.build
>> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>>     'pnv_phb3_msi.c',
>>     'pnv_phb3_pbcq.c',
>>     'pnv_phb4.c',
>> -  'pnv_phb4_pec.c'
>> +  'pnv_phb4_pec.c',
>> +  'pnv_phb.c',
>>   ))
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> new file mode 100644
>> index 0000000000..fa8472622f
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -0,0 +1,123 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/log.h"
>> +#include "qapi/visitor.h"
>> +#include "qapi/error.h"
>> +#include "hw/pci-host/pnv_phb.h"
>> +#include "hw/pci-host/pnv_phb3.h"
>> +#include "hw/pci-host/pnv_phb4.h"
>> +#include "hw/ppc/pnv.h"
>> +#include "hw/qdev-properties.h"
>> +#include "qom/object.h"
>> +
>> +
>> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
>> +{
>> +    PnvPHB *phb = PNV_PHB(dev);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>> +    g_autofree char *phb_typename = NULL;
>> +    g_autofree char *phb_rootport_typename = NULL;
>> +
>> +    if (!phb->version) {
>> +        error_setg(errp, "version not specified");
>> +        return;
>> +    }
>> +
>> +    switch (phb->version) {
>> +    case 3:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
>> +        break;
>> +    case 4:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
>> +        break;
>> +    case 5:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
>> +        break;
>> +    default:
>> +        g_assert_not_reached();
>> +    }
>> +
>> +    phb->backend = object_new(phb_typename);
>> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
>> +
>> +    /* Passthrough child device properties to the proxy device */
>> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
>> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
>> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
>> +
>> +    if (phb->version == 3) {
>> +        object_property_set_link(phb->backend, "chip",
>> +                                 OBJECT(phb->chip), errp);
>> +    } else {
>> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
>> +    }
> 
> 
> The patch is fine, but it just highlights that we're doing something wrong. I don't believe there's any reason for the chip/pec/phb relationship to be different between P8 and P9/P10. One day, a brave soul could try to unify the models, it would avoid test like that.

Not a bad idea, especially if we can cut more complexity out of the code.
I'll give it some thought.


Daniel



> It would be a good cleanup series to do if we ever extend the model with yet another version :-)
> 
> 
> 
>> +
>> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (phb->version == 3) {
>> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
>> +    }
>> +
>> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
> 
> 
> 
> After we've removed the other instances (done in later patches), we could move pnv_phb_attach_root_port() to pnv_phb.c instead of pnv.c. It would be the perfect home for it as it starts looking off in pnv.c
> 
>    Fred
> 
> 
> 
>> +}
>> +
>> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
>> +                                         PCIBus *rootbus)
>> +{
>> +    PnvPHB *phb = PNV_PHB(host_bridge);
>> +
>> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>> +             phb->chip_id, phb->phb_id);
>> +    return phb->bus_path;
>> +}
>> +
>> +static Property pnv_phb_properties[] = {
>> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
>> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
>> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
>> +
>> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
>> +
>> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
>> +                         PnvPhb4PecState *),
>> +
>> +        DEFINE_PROP_END_OF_LIST(),
>> +};
>> +
>> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
>> +{
>> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +
>> +    hc->root_bus_path = pnv_phb_root_bus_path;
>> +    dc->realize = pnv_phb_realize;
>> +    device_class_set_props(dc, pnv_phb_properties);
>> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>> +    dc->user_creatable = true;
>> +}
>> +
>> +static void pnv_phb_register_type(void)
>> +{
>> +    static const TypeInfo pnv_phb_type_info = {
>> +        .name          = TYPE_PNV_PHB,
>> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +        .instance_size = sizeof(PnvPHB),
>> +        .class_init    = pnv_phb_class_init,
>> +    };
>> +
>> +    type_register_static(&pnv_phb_type_info);
>> +}
>> +type_init(pnv_phb_register_type)
>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>> new file mode 100644
>> index 0000000000..a7cc8610e2
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.h
>> @@ -0,0 +1,39 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef PCI_HOST_PNV_PHB_H
>> +#define PCI_HOST_PNV_PHB_H
>> +
>> +#include "hw/pci/pcie_host.h"
>> +#include "hw/pci/pcie_port.h"
>> +#include "qom/object.h"
>> +
>> +typedef struct PnvChip PnvChip;
>> +typedef struct PnvPhb4PecState PnvPhb4PecState;
>> +
>> +struct PnvPHB {
>> +    PCIExpressHost parent_obj;
>> +
>> +    uint32_t chip_id;
>> +    uint32_t phb_id;
>> +    uint32_t version;
>> +    char bus_path[8];
>> +
>> +    PnvChip *chip;
>> +
>> +    PnvPhb4PecState *pec;
>> +
>> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
>> +    Object *backend;
>> +};
>> +
>> +#define TYPE_PNV_PHB "pnv-phb"
>> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>> +
>> +#endif /* PCI_HOST_PNV_PHB_H */

Re: [PATCH v2 04/16] ppc/pnv: change PnvPHB3 to be a PnvPHB backend

[Daniel Henrique Barboza]

On 6/2/22 04:56, Mark Cave-Ayland wrote:
> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
> 
>> We need a handful of changes that needs to be done in a single swoop to
>> turn PnvPHB3 into a PnvPHB backend.
>>
>> In the PnvPHB3, since the PnvPHB device implements PCIExpressHost and
>> will hold the PCI bus, change PnvPHB3 parent to TYPE_DEVICE. There are a
>> couple of instances in pnv_phb3.c that needs to access the PCI bus, so a
>> phb_base pointer is added to allow access to the parent PnvPHB. The
>> PnvPHB3 root port will now be connected to a PnvPHB object.
>>
>> In pnv.c, the powernv8 machine chip8 will now hold an array of PnvPHB
>> objects.  pnv_get_phb3_child() needs to be adapted to return the PnvPHB3
>> backend from the PnvPHB child. A global property is added in
>> pnv_machine_power8_class_init() to ensure that all PnvPHBs are created
>> with phb->version = 3.
>>
>> After all these changes we're still able to boot a powernv8 machine with
>> default settings. The real gain will come with user created PnvPHB
>> devices, coming up next.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb3.c         | 29 ++++++++---------------------
>>   hw/ppc/pnv.c                   | 21 +++++++++++++++++----
>>   include/hw/pci-host/pnv_phb3.h |  5 ++++-
>>   include/hw/ppc/pnv.h           |  3 ++-
>>   4 files changed, 31 insertions(+), 27 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
>> index 60584e2aae..a39aa0e8c4 100644
>> --- a/hw/pci-host/pnv_phb3.c
>> +++ b/hw/pci-host/pnv_phb3.c
>> @@ -11,6 +11,7 @@
>>   #include "qapi/visitor.h"
>>   #include "qapi/error.h"
>>   #include "hw/pci-host/pnv_phb3_regs.h"
>> +#include "hw/pci-host/pnv_phb.h"
>>   #include "hw/pci-host/pnv_phb3.h"
>>   #include "hw/pci/pcie_host.h"
>>   #include "hw/pci/pcie_port.h"
>> @@ -26,7 +27,7 @@
>>   static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb)
>>   {
>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>>       uint8_t bus, devfn;
>> @@ -590,7 +591,7 @@ void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size)
>>   uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size)
>>   {
>>       PnvPHB3 *phb = opaque;
>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>       uint64_t val;
>>       if ((off & 0xfffc) == PHB_CONFIG_DATA) {
>> @@ -1057,8 +1058,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
>>                             "phb3-regs", 0x1000);
>>       pnv_phb3_bus_init(dev, phb);
>> -
>> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
>>   }
>>   void pnv_phb3_update_regions(PnvPHB3 *phb)
>> @@ -1083,38 +1082,26 @@ void pnv_phb3_update_regions(PnvPHB3 *phb)
>>       pnv_phb3_check_all_m64s(phb);
>>   }
>> -static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge,
>> -                                          PCIBus *rootbus)
>> -{
>> -    PnvPHB3 *phb = PNV_PHB3(host_bridge);
>> -
>> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>> -             phb->chip_id, phb->phb_id);
>> -    return phb->bus_path;
>> -}
>> -
>>   static Property pnv_phb3_properties[] = {
>>           DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0),
>>           DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0),
>>           DEFINE_PROP_LINK("chip", PnvPHB3, chip, TYPE_PNV_CHIP, PnvChip *),
>> +        DEFINE_PROP_LINK("phb-base", PnvPHB3, phb_base, TYPE_PNV_PHB, PnvPHB *),
>>           DEFINE_PROP_END_OF_LIST(),
>>   };
>>   static void pnv_phb3_class_init(ObjectClass *klass, void *data)
>>   {
>> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>>       DeviceClass *dc = DEVICE_CLASS(klass);
>> -    hc->root_bus_path = pnv_phb3_root_bus_path;
>>       dc->realize = pnv_phb3_realize;
>>       device_class_set_props(dc, pnv_phb3_properties);
>> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>>       dc->user_creatable = false;
>>   }
>>   static const TypeInfo pnv_phb3_type_info = {
>>       .name          = TYPE_PNV_PHB3,
>> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +    .parent = TYPE_DEVICE,
>>       .instance_size = sizeof(PnvPHB3),
>>       .class_init    = pnv_phb3_class_init,
>>       .instance_init = pnv_phb3_instance_init,
>> @@ -1146,11 +1133,11 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)
>>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>       PCIDevice *pci = PCI_DEVICE(dev);
>>       PCIBus *bus = pci_get_bus(pci);
>> -    PnvPHB3 *phb = NULL;
>> +    PnvPHB *phb = NULL;
>>       Error *local_err = NULL;
>> -    phb = (PnvPHB3 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> -                                          TYPE_PNV_PHB3);
>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> +                                         TYPE_PNV_PHB);
> 
> I realize that this has come from existing code, however these days there generally isn't a good reason for anything to access bus->qbus directly (and C casts without a QOM macro often need a closer look). I'm also not convinced by the use of object_dynamic_cast() here either. Could this be rewritten as something like:
> 
>      phb = PNV_PHB(PNV_PHB3_ROOT_PORT(dev)->phb_base);

Just checked. At this moment we can't because we don't declare a PNV_PHB3_ROOT_PORT()
macro for this device. Yes, we're missing a OBJECT_DECLARE_SIMPLE_TYPE() for it. Same
thing for PnvPHB4RootPort.

I can add the missing PNV_PHB3_ROOT_PORT() macro and do this change. Or I can make
a comment mentioning that this code can be improved but it's going to be removed
shortly and this will be amended appropriately in the pnv-phb-root-port code.


What do you think? I'm fine with both alternatives.


Thanks,


Daniel

> 
>>       if (!phb) {
>>           error_setg(errp,
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index aaf4d241c3..6cd0af9adf 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -43,6 +43,7 @@
>>   #include "hw/ipmi/ipmi.h"
>>   #include "target/ppc/mmu-hash64.h"
>>   #include "hw/pci/msi.h"
>> +#include "hw/pci-host/pnv_phb.h"
>>   #include "hw/ppc/xics.h"
>>   #include "hw/qdev-properties.h"
>> @@ -654,7 +655,13 @@ static ISABus *pnv_isa_create(PnvChip *chip, Error **errp)
>>   static PnvPHB3 *pnv_get_phb3_child(Object *child)
>>   {
>> -    return (PnvPHB3 *)object_dynamic_cast(child, TYPE_PNV_PHB3);
>> +    PnvPHB *phb = (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
> 
> And here, assuming child is of type TYPE_PNV_PHB then as it is already known at compile time so I think this could become just:
> 
>      PnvPHB *phb = PNV_PHB(child);
> 
>> +    if (!phb) {
>> +        return NULL;
>> +    }
>> +
>> +    return (PnvPHB3 *)phb->backend;
>>   }
>>   static int pnv_chip_power8_pic_print_info_child(Object *child, void *opaque)
>> @@ -1160,7 +1167,7 @@ static void pnv_chip_power8_instance_init(Object *obj)
>>       chip8->num_phbs = pcc->num_phbs;
>>       for (i = 0; i < chip8->num_phbs; i++) {
>> -        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB3);
>> +        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB);
>>       }
>>   }
>> @@ -1282,9 +1289,9 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp)
>>       memory_region_add_subregion(get_system_memory(), PNV_HOMER_BASE(chip),
>>                                   &chip8->homer.regs);
>> -    /* PHB3 controllers */
>> +    /* PHB controllers */
>>       for (i = 0; i < chip8->num_phbs; i++) {
>> -        PnvPHB3 *phb = &chip8->phbs[i];
>> +        PnvPHB *phb = &chip8->phbs[i];
>>           object_property_set_int(OBJECT(phb), "index", i, &error_fatal);
>>           object_property_set_int(OBJECT(phb), "chip-id", chip->chip_id,
>> @@ -1957,6 +1964,7 @@ static int pnv_ics_get_child(Object *child, void *opaque)
>>               args->ics = ICS(&phb3->msis);
>>           }
>>       }
>> +
>>       return args->ics ? 1 : 0;
>>   }
>> @@ -2112,8 +2120,13 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>>       static const char compat[] = "qemu,powernv8\0qemu,powernv\0ibm,powernv";
>> +    static GlobalProperty phb_compat[] = {
>> +        { TYPE_PNV_PHB, "version", "3" },
>> +    };
>> +
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
>> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>>       xic->icp_get = pnv_icp_get;
>>       xic->ics_get = pnv_ics_get;
>> diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
>> index 1375f18fc1..3b9ff1096a 100644
>> --- a/include/hw/pci-host/pnv_phb3.h
>> +++ b/include/hw/pci-host/pnv_phb3.h
>> @@ -14,6 +14,7 @@
>>   #include "hw/pci/pcie_port.h"
>>   #include "hw/ppc/xics.h"
>>   #include "qom/object.h"
>> +#include "hw/pci-host/pnv_phb.h"
>>   typedef struct PnvPHB3 PnvPHB3;
>>   typedef struct PnvChip PnvChip;
>> @@ -127,7 +128,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB3, PNV_PHB3)
>>   #define PCI_MMIO_TOTAL_SIZE   (0x1ull << 60)
>>   struct PnvPHB3 {
>> -    PCIExpressHost parent_obj;
>> +    DeviceState parent;
>> +
>> +    PnvPHB *phb_base;
>>       uint32_t chip_id;
>>       uint32_t phb_id;
>> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
>> index 86cb7d7f97..4595db418e 100644
>> --- a/include/hw/ppc/pnv.h
>> +++ b/include/hw/ppc/pnv.h
>> @@ -32,6 +32,7 @@
>>   #include "hw/ppc/pnv_core.h"
>>   #include "hw/pci-host/pnv_phb3.h"
>>   #include "hw/pci-host/pnv_phb4.h"
>> +#include "hw/pci-host/pnv_phb.h"
>>   #include "qom/object.h"
>>   #define TYPE_PNV_CHIP "pnv-chip"
>> @@ -80,7 +81,7 @@ struct Pnv8Chip {
>>       PnvHomer     homer;
>>   #define PNV8_CHIP_PHB3_MAX 4
>> -    PnvPHB3      phbs[PNV8_CHIP_PHB3_MAX];
>> +    PnvPHB       phbs[PNV8_CHIP_PHB3_MAX];
>>       uint32_t     num_phbs;
>>       XICSFabric    *xics;
> 
> 
> ATB,
> 
> Mark.

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Daniel Henrique Barboza]

On 6/2/22 05:12, Mark Cave-Ayland wrote:
> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
> 
>> We have two very similar root-port devices, pnv-phb3-root-port and
>> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
>> that, until now, has no additional attributes.
>>
>> The main difference between the PHB3 and PHB4 root ports is that
>> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
>> other differences can be merged in a single device without too much
>> trouble.
>>
>> This patch introduces the unified pnv-phb-root-port that, in time, will
>> be used as the default root port for the pnv-phb device.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>>   hw/pci-host/pnv_phb.h |  17 +++++++
>>   2 files changed, 116 insertions(+), 8 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> index 321c4e768a..5047e90d3a 100644
>> --- a/hw/pci-host/pnv_phb.c
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>       dc->user_creatable = true;
>>   }
>> -static void pnv_phb_register_type(void)
>> +static void pnv_phb_root_port_reset(DeviceState *dev)
>>   {
>> -    static const TypeInfo pnv_phb_type_info = {
>> -        .name          = TYPE_PNV_PHB,
>> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
>> -        .instance_size = sizeof(PnvPHB),
>> -        .class_init    = pnv_phb_class_init,
>> -    };
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
>> +    PCIDevice *d = PCI_DEVICE(dev);
>> +    uint8_t *conf = d->config;
>> +    rpc->parent_reset(dev);
>> +
>> +    if (rootport->version == 3) {
>> +        return;
>> +    }
>> +
>> +    /* PHB4 and later requires these extra reset steps */
>> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
>> +                               PCI_IO_RANGE_MASK & 0xff);
>> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
>> +                                 PCI_IO_RANGE_MASK & 0xff);
>> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
>> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
>> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
>> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
>> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
>> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
>> +    pci_config_set_interrupt_pin(conf, 0);
>> +}
>> +
>> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>> +{
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>> +    PCIDevice *pci = PCI_DEVICE(dev);
>> +    PCIBus *bus = pci_get_bus(pci);
>> +    PnvPHB *phb = NULL;
>> +    Error *local_err = NULL;
>> +
>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> +                                          TYPE_PNV_PHB);
> 
> Same here too.


This case is a bit different than the previous because there's no ->phb_base in
this device. Doing

PnvPHB *phb = PNV_PHB(PNV_PHB_ROOT_PORT(dev));

Will somehow retrieve a PnvPHB pointer but with zeroed properties. This makes
the slot/chassis code down below to fail when adding a second port.

I checked how other parts of the code handles that and the closer I found so
far is doing something like this:

     PCIDevice *pci = PCI_DEVICE(dev);
     PnvPHB *phb = PNV_PHB(pci_device_root_bus(pci)->qbus.parent);

This is how pci_bus_bypass_iommu() from hw/pci/pci.c retrieves a PCIHostState
pointer of a given bus.

Another idea is to simply add a pointer to the parent PnvPHB device that the
root port is attached to. This would trivialize this part and I could also
use the already set PnvPHB->version for the root port logic as well since they're
set to be the same.


Thanks,


Daniel



> 
>> +    if (!phb) {
>> +        error_setg(errp,
>> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
>> +        return;
>> +    }
>> +
>> +    /* Set unique chassis/slot values for the root port */
>> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
>> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
> 
> Again this is from older code, but we already have dev so these could be:
> 
>      qdev_prop_set_uint8(dev, "chassis", phb->chip_id);
>      qdev_prop_set_uint16(dev, "slot", phb->phb_id);
> 
>> +    rpc->parent_realize(dev, &local_err);
>> +    if (local_err) {
>> +        error_propagate(errp, local_err);
>> +        return;
>> +    }
>> +    pci_config_set_interrupt_pin(pci->config, 0);
>> +}
>> +
>> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>> +{
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>> +
>> +    dc->desc     = "IBM PHB PCIE Root Port";
>> +
>> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>> +                                    &rpc->parent_realize);
>> +
>> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>> +                                  &rpc->parent_reset);
>> +    dc->reset = &pnv_phb_root_port_reset;
>> +
>> +    dc->user_creatable = true;
>> +
>> +    k->vendor_id = PCI_VENDOR_ID_IBM;
>> +    /* device_id represents the latest PHB root port version supported */
>> +    k->device_id = PNV_PHB5_DEVICE_ID;
>> +    k->revision  = 0;
>> +
>> +    rpc->exp_offset = 0x48;
>> +    rpc->aer_offset = 0x100;
>> +}
>> +
>> +static const TypeInfo pnv_phb_type_info = {
>> +    .name          = TYPE_PNV_PHB,
>> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +    .instance_size = sizeof(PnvPHB),
>> +    .class_init    = pnv_phb_class_init,
>> +};
>> +
>> +static const TypeInfo pnv_phb_root_port_info = {
>> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
>> +    .parent        = TYPE_PCIE_ROOT_PORT,
>> +    .instance_size = sizeof(PnvPHBRootPort),
>> +    .class_init    = pnv_phb_root_port_class_init,
>> +};
>> +
>> +static void pnv_phb_register_types(void)
>> +{
>>       type_register_static(&pnv_phb_type_info);
>> +    type_register_static(&pnv_phb_root_port_info);
>>   }
>> -type_init(pnv_phb_register_type)
>> +
>> +type_init(pnv_phb_register_types)
>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>> index a7cc8610e2..c8eab4b767 100644
>> --- a/hw/pci-host/pnv_phb.h
>> +++ b/hw/pci-host/pnv_phb.h
>> @@ -36,4 +36,21 @@ struct PnvPHB {
>>   #define TYPE_PNV_PHB "pnv-phb"
>>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>> +/*
>> + * PHB PCIe Root port
>> + */
>> +#define PNV_PHB3_DEVICE_ID         0x03dc
>> +#define PNV_PHB4_DEVICE_ID         0x04c1
>> +#define PNV_PHB5_DEVICE_ID         0x0652
>> +
>> +typedef struct PnvPHBRootPort {
>> +    PCIESlot parent_obj;
>> +
>> +    uint32_t version;
>> +} PnvPHBRootPort;
>> +
>> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
>> +#define PNV_PHB_ROOT_PORT(obj) \
>> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
>> +
>>   #endif /* PCI_HOST_PNV_PHB_H */
> 
> 
> ATB,
> 
> Mark.

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Daniel Henrique Barboza]

On 6/2/22 13:33, Frederic Barrat wrote:
> 
> 
> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>> To enable user creatable PnvPHB devices for powernv9 we'll revert the
>> powernv9 related changes made in 9c10d86fee "ppc/pnv: Remove
>> user-created PHB{3,4,5} devices".
>>
>> This change alone isn't enough to enable user creatable devices for powernv10
>> due to how pnv_phb4_get_pec() currently works. For now let's just enable it
>> for powernv9 alone.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb4.c     | 58 +++++++++++++++++++++++++++++++++++++-
>>   hw/pci-host/pnv_phb4_pec.c |  6 ++--
>>   hw/ppc/pnv.c               |  2 ++
>>   3 files changed, 63 insertions(+), 3 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
>> index 22cf1c2a5e..a5c8ae494b 100644
>> --- a/hw/pci-host/pnv_phb4.c
>> +++ b/hw/pci-host/pnv_phb4.c
>> @@ -1571,13 +1571,69 @@ void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
>>       pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE;
>>   }
>> +static PnvPhb4PecState *pnv_phb4_get_pec(PnvChip *chip, PnvPHB4 *phb,
>> +                                         Error **errp)
>> +{
>> +    Pnv9Chip *chip9 = PNV9_CHIP(chip);
>> +    int chip_id = phb->chip_id;
>> +    int index = phb->phb_id;
>> +    int i, j;
>> +
>> +    for (i = 0; i < chip->num_pecs; i++) {
>> +        /*
>> +         * For each PEC, check the amount of phbs it supports
>> +         * and see if the given phb4 index matches an index.
>> +         */
>> +        PnvPhb4PecState *pec = &chip9->pecs[i];
>> +
>> +        for (j = 0; j < pec->num_phbs; j++) {
>> +            if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
>> +                return pec;
>> +            }
>> +        }
>> +    }
>> +
>> +    error_setg(errp,
>> +               "pnv-phb4 chip-id %d index %d didn't match any existing PEC",
>> +               chip_id, index);
>> +
>> +    return NULL;
>> +}
>> +
>>   static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>>   {
>>       PnvPHB4 *phb = PNV_PHB4(dev);
>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>> +    PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
>>       XiveSource *xsrc = &phb->xsrc;
>> +    BusState *s;
>> +    Error *local_err = NULL;
>>       int nr_irqs;
>>       char name[32];
>> +    if (!chip) {
>> +        error_setg(errp, "invalid chip id: %d", phb->chip_id);
>> +        return;
>> +    }
>> +
>> +    /* User created PHBs need to be assigned to a PEC */
>> +    if (!phb->pec) {
>> +        phb->pec = pnv_phb4_get_pec(chip, phb, &local_err);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            return;
>> +        }
>> +    }
>> +
>> +    /* Reparent the PHB to the chip to build the device tree */
>> +    pnv_chip_parent_fixup(chip, OBJECT(phb->phb_base), phb->phb_id);
> 
> 
> Didn't you mean to do that only for the user-created case? 

It's done for both because the default generated PHBs are being created loosely
from the chip starting on 3f4c369ea63e ("ppc/pnv: make PECs create and realize
PHB4s"). We had to amend the code after that commit to fix the QOM hierarchy
of these devices. 6e7b96750359 ("ppc/pnv: fix default PHB4 QOM hierarchy") has
more details.


> And why not attaching the PHB to the PEC instead of the chip, like in pnv_pec_default_phb_realize() ? I think it makes more sense to keep the chip->PEC->PHB parenting in the qom tree (and incidentally, that's where I'd prefer to have the phb3 model move to).

I can only speculate since I didn't participate in the initial design. My
educated guess is that we didn't want to show PECs in qtree, hence we
made the PHB a child of the chip instead. I'll also guess that this can be
due to how PHB3 worked and we just copied the existing design.

> Also, the comment seems wrong to me. The qom parenting doesn't matter when building the device tree. We only iterate over the PHBs found in the array of the PEC object (cf. pnv_pec_dt_xscom())

I believe it refers to the QOM tree, a.k.a qtree. This has no relation to the
actual device tree the kernel uses. This comment can be clearer though.


Thanks,


Daniel

> 
> 
> 
>> +    s = qdev_get_parent_bus(DEVICE(chip));
>> +    if (!qdev_set_parent_bus(DEVICE(phb->phb_base), s, &local_err)) {
>> +        error_propagate(errp, local_err);
>> +        return;
>> +    }
> 
> 
> Same comment, I think that's only desirable for user-created devices. We're already calling sysbus_realize() for the default case.
> 
> 
> Silly question: where does it break if a user tries to create 2 PHBs with the same index?
> 
> 
>    Fred
> 
> 
> 
> 
>>       /* Set the "big_phb" flag */
>>       phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
>> @@ -1803,7 +1859,7 @@ static void pnv_phb4_root_port_class_init(ObjectClass *klass, void *data)
>>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>>       dc->desc     = "IBM PHB4 PCIE Root Port";
>> -    dc->user_creatable = false;
>> +    dc->user_creatable = true;
>>       device_class_set_parent_realize(dc, pnv_phb4_root_port_realize,
>>                                       &rpc->parent_realize);
>> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
>> index 888ecbe8f3..0e67f3a338 100644
>> --- a/hw/pci-host/pnv_phb4_pec.c
>> +++ b/hw/pci-host/pnv_phb4_pec.c
>> @@ -146,8 +146,10 @@ static void pnv_pec_realize(DeviceState *dev, Error **errp)
>>       pec->num_phbs = pecc->num_phbs[pec->index];
>>       /* Create PHBs if running with defaults */
>> -    for (i = 0; i < pec->num_phbs; i++) {
>> -        pnv_pec_default_phb_realize(pec, i, errp);
>> +    if (defaults_enabled()) {
>> +        for (i = 0; i < pec->num_phbs; i++) {
>> +            pnv_pec_default_phb_realize(pec, i, errp);
>> +        }
>>       }
>>       /* Initialize the XSCOM regions for the PEC registers */
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index 3b0b230e49..697a2b5302 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -2186,6 +2186,8 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>       pmc->compat = compat;
>>       pmc->compat_size = sizeof(compat);
>>       pmc->dt_power_mgt = pnv_dt_power_mgt;
>> +
>> +    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
>>   }
>>   static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)

Re: [PATCH v2 04/16] ppc/pnv: change PnvPHB3 to be a PnvPHB backend

[Mark Cave-Ayland]

On 03/06/2022 21:30, Daniel Henrique Barboza wrote:

> On 6/2/22 04:56, Mark Cave-Ayland wrote:
>> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
>>
>>> We need a handful of changes that needs to be done in a single swoop to
>>> turn PnvPHB3 into a PnvPHB backend.
>>>
>>> In the PnvPHB3, since the PnvPHB device implements PCIExpressHost and
>>> will hold the PCI bus, change PnvPHB3 parent to TYPE_DEVICE. There are a
>>> couple of instances in pnv_phb3.c that needs to access the PCI bus, so a
>>> phb_base pointer is added to allow access to the parent PnvPHB. The
>>> PnvPHB3 root port will now be connected to a PnvPHB object.
>>>
>>> In pnv.c, the powernv8 machine chip8 will now hold an array of PnvPHB
>>> objects.  pnv_get_phb3_child() needs to be adapted to return the PnvPHB3
>>> backend from the PnvPHB child. A global property is added in
>>> pnv_machine_power8_class_init() to ensure that all PnvPHBs are created
>>> with phb->version = 3.
>>>
>>> After all these changes we're still able to boot a powernv8 machine with
>>> default settings. The real gain will come with user created PnvPHB
>>> devices, coming up next.
>>>
>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>> ---
>>>   hw/pci-host/pnv_phb3.c         | 29 ++++++++---------------------
>>>   hw/ppc/pnv.c                   | 21 +++++++++++++++++----
>>>   include/hw/pci-host/pnv_phb3.h |  5 ++++-
>>>   include/hw/ppc/pnv.h           |  3 ++-
>>>   4 files changed, 31 insertions(+), 27 deletions(-)
>>>
>>> diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
>>> index 60584e2aae..a39aa0e8c4 100644
>>> --- a/hw/pci-host/pnv_phb3.c
>>> +++ b/hw/pci-host/pnv_phb3.c
>>> @@ -11,6 +11,7 @@
>>>   #include "qapi/visitor.h"
>>>   #include "qapi/error.h"
>>>   #include "hw/pci-host/pnv_phb3_regs.h"
>>> +#include "hw/pci-host/pnv_phb.h"
>>>   #include "hw/pci-host/pnv_phb3.h"
>>>   #include "hw/pci/pcie_host.h"
>>>   #include "hw/pci/pcie_port.h"
>>> @@ -26,7 +27,7 @@
>>>   static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb)
>>>   {
>>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>>>       uint8_t bus, devfn;
>>> @@ -590,7 +591,7 @@ void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t 
>>> val, unsigned size)
>>>   uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size)
>>>   {
>>>       PnvPHB3 *phb = opaque;
>>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>>       uint64_t val;
>>>       if ((off & 0xfffc) == PHB_CONFIG_DATA) {
>>> @@ -1057,8 +1058,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error **errp)
>>>                             "phb3-regs", 0x1000);
>>>       pnv_phb3_bus_init(dev, phb);
>>> -
>>> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
>>>   }
>>>   void pnv_phb3_update_regions(PnvPHB3 *phb)
>>> @@ -1083,38 +1082,26 @@ void pnv_phb3_update_regions(PnvPHB3 *phb)
>>>       pnv_phb3_check_all_m64s(phb);
>>>   }
>>> -static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge,
>>> -                                          PCIBus *rootbus)
>>> -{
>>> -    PnvPHB3 *phb = PNV_PHB3(host_bridge);
>>> -
>>> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>>> -             phb->chip_id, phb->phb_id);
>>> -    return phb->bus_path;
>>> -}
>>> -
>>>   static Property pnv_phb3_properties[] = {
>>>           DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0),
>>>           DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0),
>>>           DEFINE_PROP_LINK("chip", PnvPHB3, chip, TYPE_PNV_CHIP, PnvChip *),
>>> +        DEFINE_PROP_LINK("phb-base", PnvPHB3, phb_base, TYPE_PNV_PHB, PnvPHB *),
>>>           DEFINE_PROP_END_OF_LIST(),
>>>   };
>>>   static void pnv_phb3_class_init(ObjectClass *klass, void *data)
>>>   {
>>> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>>>       DeviceClass *dc = DEVICE_CLASS(klass);
>>> -    hc->root_bus_path = pnv_phb3_root_bus_path;
>>>       dc->realize = pnv_phb3_realize;
>>>       device_class_set_props(dc, pnv_phb3_properties);
>>> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>>>       dc->user_creatable = false;
>>>   }
>>>   static const TypeInfo pnv_phb3_type_info = {
>>>       .name          = TYPE_PNV_PHB3,
>>> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> +    .parent = TYPE_DEVICE,
>>>       .instance_size = sizeof(PnvPHB3),
>>>       .class_init    = pnv_phb3_class_init,
>>>       .instance_init = pnv_phb3_instance_init,
>>> @@ -1146,11 +1133,11 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, 
>>> Error **errp)
>>>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>>       PCIDevice *pci = PCI_DEVICE(dev);
>>>       PCIBus *bus = pci_get_bus(pci);
>>> -    PnvPHB3 *phb = NULL;
>>> +    PnvPHB *phb = NULL;
>>>       Error *local_err = NULL;
>>> -    phb = (PnvPHB3 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>> -                                          TYPE_PNV_PHB3);
>>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>> +                                         TYPE_PNV_PHB);
>>
>> I realize that this has come from existing code, however these days there generally 
>> isn't a good reason for anything to access bus->qbus directly (and C casts without 
>> a QOM macro often need a closer look). I'm also not convinced by the use of 
>> object_dynamic_cast() here either. Could this be rewritten as something like:
>>
>>      phb = PNV_PHB(PNV_PHB3_ROOT_PORT(dev)->phb_base);
> 
> Just checked. At this moment we can't because we don't declare a PNV_PHB3_ROOT_PORT()
> macro for this device. Yes, we're missing a OBJECT_DECLARE_SIMPLE_TYPE() for it. Same
> thing for PnvPHB4RootPort.
> 
> I can add the missing PNV_PHB3_ROOT_PORT() macro and do this change. Or I can make
> a comment mentioning that this code can be improved but it's going to be removed
> shortly and this will be amended appropriately in the pnv-phb-root-port code.
> 
> 
> What do you think? I'm fine with both alternatives.

I see there are some queries about the way these devices are modelled, so if that 
requires more than trivial changes it may make sense to integrate these changes into 
v3, particularly if it helps with type safety.

Otherwise I'd be simply inclined to add these changes to the end of your existing 
series as a final tidy-up and post that as v3.

> Thanks,
> 
> 
> Daniel
> 
>>
>>>       if (!phb) {
>>>           error_setg(errp,
>>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>>> index aaf4d241c3..6cd0af9adf 100644
>>> --- a/hw/ppc/pnv.c
>>> +++ b/hw/ppc/pnv.c
>>> @@ -43,6 +43,7 @@
>>>   #include "hw/ipmi/ipmi.h"
>>>   #include "target/ppc/mmu-hash64.h"
>>>   #include "hw/pci/msi.h"
>>> +#include "hw/pci-host/pnv_phb.h"
>>>   #include "hw/ppc/xics.h"
>>>   #include "hw/qdev-properties.h"
>>> @@ -654,7 +655,13 @@ static ISABus *pnv_isa_create(PnvChip *chip, Error **errp)
>>>   static PnvPHB3 *pnv_get_phb3_child(Object *child)
>>>   {
>>> -    return (PnvPHB3 *)object_dynamic_cast(child, TYPE_PNV_PHB3);
>>> +    PnvPHB *phb = (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
>>
>> And here, assuming child is of type TYPE_PNV_PHB then as it is already known at 
>> compile time so I think this could become just:
>>
>>      PnvPHB *phb = PNV_PHB(child);
>>
>>> +    if (!phb) {
>>> +        return NULL;
>>> +    }
>>> +
>>> +    return (PnvPHB3 *)phb->backend;
>>>   }
>>>   static int pnv_chip_power8_pic_print_info_child(Object *child, void *opaque)
>>> @@ -1160,7 +1167,7 @@ static void pnv_chip_power8_instance_init(Object *obj)
>>>       chip8->num_phbs = pcc->num_phbs;
>>>       for (i = 0; i < chip8->num_phbs; i++) {
>>> -        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB3);
>>> +        object_initialize_child(obj, "phb[*]", &chip8->phbs[i], TYPE_PNV_PHB);
>>>       }
>>>   }
>>> @@ -1282,9 +1289,9 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error 
>>> **errp)
>>>       memory_region_add_subregion(get_system_memory(), PNV_HOMER_BASE(chip),
>>>                                   &chip8->homer.regs);
>>> -    /* PHB3 controllers */
>>> +    /* PHB controllers */
>>>       for (i = 0; i < chip8->num_phbs; i++) {
>>> -        PnvPHB3 *phb = &chip8->phbs[i];
>>> +        PnvPHB *phb = &chip8->phbs[i];
>>>           object_property_set_int(OBJECT(phb), "index", i, &error_fatal);
>>>           object_property_set_int(OBJECT(phb), "chip-id", chip->chip_id,
>>> @@ -1957,6 +1964,7 @@ static int pnv_ics_get_child(Object *child, void *opaque)
>>>               args->ics = ICS(&phb3->msis);
>>>           }
>>>       }
>>> +
>>>       return args->ics ? 1 : 0;
>>>   }
>>> @@ -2112,8 +2120,13 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, 
>>> void *data)
>>>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>>>       static const char compat[] = "qemu,powernv8\0qemu,powernv\0ibm,powernv";
>>> +    static GlobalProperty phb_compat[] = {
>>> +        { TYPE_PNV_PHB, "version", "3" },
>>> +    };
>>> +
>>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
>>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
>>> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>>>       xic->icp_get = pnv_icp_get;
>>>       xic->ics_get = pnv_ics_get;
>>> diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h
>>> index 1375f18fc1..3b9ff1096a 100644
>>> --- a/include/hw/pci-host/pnv_phb3.h
>>> +++ b/include/hw/pci-host/pnv_phb3.h
>>> @@ -14,6 +14,7 @@
>>>   #include "hw/pci/pcie_port.h"
>>>   #include "hw/ppc/xics.h"
>>>   #include "qom/object.h"
>>> +#include "hw/pci-host/pnv_phb.h"
>>>   typedef struct PnvPHB3 PnvPHB3;
>>>   typedef struct PnvChip PnvChip;
>>> @@ -127,7 +128,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB3, PNV_PHB3)
>>>   #define PCI_MMIO_TOTAL_SIZE   (0x1ull << 60)
>>>   struct PnvPHB3 {
>>> -    PCIExpressHost parent_obj;
>>> +    DeviceState parent;
>>> +
>>> +    PnvPHB *phb_base;
>>>       uint32_t chip_id;
>>>       uint32_t phb_id;
>>> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
>>> index 86cb7d7f97..4595db418e 100644
>>> --- a/include/hw/ppc/pnv.h
>>> +++ b/include/hw/ppc/pnv.h
>>> @@ -32,6 +32,7 @@
>>>   #include "hw/ppc/pnv_core.h"
>>>   #include "hw/pci-host/pnv_phb3.h"
>>>   #include "hw/pci-host/pnv_phb4.h"
>>> +#include "hw/pci-host/pnv_phb.h"
>>>   #include "qom/object.h"
>>>   #define TYPE_PNV_CHIP "pnv-chip"
>>> @@ -80,7 +81,7 @@ struct Pnv8Chip {
>>>       PnvHomer     homer;
>>>   #define PNV8_CHIP_PHB3_MAX 4
>>> -    PnvPHB3      phbs[PNV8_CHIP_PHB3_MAX];
>>> +    PnvPHB       phbs[PNV8_CHIP_PHB3_MAX];
>>>       uint32_t     num_phbs;
>>>       XICSFabric    *xics;


ATB,

Mark.

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Mark Cave-Ayland]

On 03/06/2022 21:47, Daniel Henrique Barboza wrote:

> On 6/2/22 05:12, Mark Cave-Ayland wrote:
>> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
>>
>>> We have two very similar root-port devices, pnv-phb3-root-port and
>>> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
>>> that, until now, has no additional attributes.
>>>
>>> The main difference between the PHB3 and PHB4 root ports is that
>>> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
>>> other differences can be merged in a single device without too much
>>> trouble.
>>>
>>> This patch introduces the unified pnv-phb-root-port that, in time, will
>>> be used as the default root port for the pnv-phb device.
>>>
>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>> ---
>>>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>>>   hw/pci-host/pnv_phb.h |  17 +++++++
>>>   2 files changed, 116 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>>> index 321c4e768a..5047e90d3a 100644
>>> --- a/hw/pci-host/pnv_phb.c
>>> +++ b/hw/pci-host/pnv_phb.c
>>> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>>       dc->user_creatable = true;
>>>   }
>>> -static void pnv_phb_register_type(void)
>>> +static void pnv_phb_root_port_reset(DeviceState *dev)
>>>   {
>>> -    static const TypeInfo pnv_phb_type_info = {
>>> -        .name          = TYPE_PNV_PHB,
>>> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> -        .instance_size = sizeof(PnvPHB),
>>> -        .class_init    = pnv_phb_class_init,
>>> -    };
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
>>> +    PCIDevice *d = PCI_DEVICE(dev);
>>> +    uint8_t *conf = d->config;
>>> +    rpc->parent_reset(dev);
>>> +
>>> +    if (rootport->version == 3) {
>>> +        return;
>>> +    }
>>> +
>>> +    /* PHB4 and later requires these extra reset steps */
>>> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
>>> +                               PCI_IO_RANGE_MASK & 0xff);
>>> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
>>> +                                 PCI_IO_RANGE_MASK & 0xff);
>>> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
>>> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
>>> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
>>> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
>>> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
>>> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
>>> +    pci_config_set_interrupt_pin(conf, 0);
>>> +}
>>> +
>>> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>>> +{
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>> +    PCIDevice *pci = PCI_DEVICE(dev);
>>> +    PCIBus *bus = pci_get_bus(pci);
>>> +    PnvPHB *phb = NULL;
>>> +    Error *local_err = NULL;
>>> +
>>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>> +                                          TYPE_PNV_PHB);
>>
>> Same here too.
> 
> 
> This case is a bit different than the previous because there's no ->phb_base in
> this device. Doing
> 
> PnvPHB *phb = PNV_PHB(PNV_PHB_ROOT_PORT(dev));
> 
> Will somehow retrieve a PnvPHB pointer but with zeroed properties. This makes
> the slot/chassis code down below to fail when adding a second port.

Hmmm that's odd.

> I checked how other parts of the code handles that and the closer I found so
> far is doing something like this:
> 
>      PCIDevice *pci = PCI_DEVICE(dev);
>      PnvPHB *phb = PNV_PHB(pci_device_root_bus(pci)->qbus.parent);
> 
> This is how pci_bus_bypass_iommu() from hw/pci/pci.c retrieves a PCIHostState
> pointer of a given bus.
> 
> Another idea is to simply add a pointer to the parent PnvPHB device that the
> root port is attached to. This would trivialize this part and I could also
> use the already set PnvPHB->version for the root port logic as well since they're
> set to be the same.

I think what should happen is that the parent device should instantiate the pnv_phb 
root port via object_initialize_child() so it is embedded within it, at which point 
the pnv_phb root port device can just use container_of() to obtain the reference to 
the parent device state.

But certainly dereferencing the internal qbus object is not the right long-term solution.

>>> +    if (!phb) {
>>> +        error_setg(errp,
>>> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
>>> +        return;
>>> +    }
>>> +
>>> +    /* Set unique chassis/slot values for the root port */
>>> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
>>> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
>>
>> Again this is from older code, but we already have dev so these could be:
>>
>>      qdev_prop_set_uint8(dev, "chassis", phb->chip_id);
>>      qdev_prop_set_uint16(dev, "slot", phb->phb_id);
>>
>>> +    rpc->parent_realize(dev, &local_err);
>>> +    if (local_err) {
>>> +        error_propagate(errp, local_err);
>>> +        return;
>>> +    }
>>> +    pci_config_set_interrupt_pin(pci->config, 0);
>>> +}
>>> +
>>> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>>> +{
>>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>>> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>>> +
>>> +    dc->desc     = "IBM PHB PCIE Root Port";
>>> +
>>> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>>> +                                    &rpc->parent_realize);
>>> +
>>> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>>> +                                  &rpc->parent_reset);
>>> +    dc->reset = &pnv_phb_root_port_reset;
>>> +
>>> +    dc->user_creatable = true;
>>> +
>>> +    k->vendor_id = PCI_VENDOR_ID_IBM;
>>> +    /* device_id represents the latest PHB root port version supported */
>>> +    k->device_id = PNV_PHB5_DEVICE_ID;
>>> +    k->revision  = 0;
>>> +
>>> +    rpc->exp_offset = 0x48;
>>> +    rpc->aer_offset = 0x100;
>>> +}
>>> +
>>> +static const TypeInfo pnv_phb_type_info = {
>>> +    .name          = TYPE_PNV_PHB,
>>> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> +    .instance_size = sizeof(PnvPHB),
>>> +    .class_init    = pnv_phb_class_init,
>>> +};
>>> +
>>> +static const TypeInfo pnv_phb_root_port_info = {
>>> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
>>> +    .parent        = TYPE_PCIE_ROOT_PORT,
>>> +    .instance_size = sizeof(PnvPHBRootPort),
>>> +    .class_init    = pnv_phb_root_port_class_init,
>>> +};
>>> +
>>> +static void pnv_phb_register_types(void)
>>> +{
>>>       type_register_static(&pnv_phb_type_info);
>>> +    type_register_static(&pnv_phb_root_port_info);
>>>   }
>>> -type_init(pnv_phb_register_type)
>>> +
>>> +type_init(pnv_phb_register_types)
>>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>>> index a7cc8610e2..c8eab4b767 100644
>>> --- a/hw/pci-host/pnv_phb.h
>>> +++ b/hw/pci-host/pnv_phb.h
>>> @@ -36,4 +36,21 @@ struct PnvPHB {
>>>   #define TYPE_PNV_PHB "pnv-phb"
>>>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>>> +/*
>>> + * PHB PCIe Root port
>>> + */
>>> +#define PNV_PHB3_DEVICE_ID         0x03dc
>>> +#define PNV_PHB4_DEVICE_ID         0x04c1
>>> +#define PNV_PHB5_DEVICE_ID         0x0652
>>> +
>>> +typedef struct PnvPHBRootPort {
>>> +    PCIESlot parent_obj;
>>> +
>>> +    uint32_t version;
>>> +} PnvPHBRootPort;
>>> +
>>> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
>>> +#define PNV_PHB_ROOT_PORT(obj) \
>>> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
>>> +
>>>   #endif /* PCI_HOST_PNV_PHB_H */


ATB,

Mark.

Re: [PATCH v2 07/16] ppc/pnv: change PnvPHB4 to be a PnvPHB backend

[Cédric Le Goater]

On 6/2/22 10:02, Mark Cave-Ayland wrote:
> On 31/05/2022 22:49, Daniel Henrique Barboza wrote:
> 
>> Change the parent type of the PnvPHB4 device to TYPE_PARENT since the
>> PCI bus is going to be initialized by the PnvPHB parent. Functions that
>> needs to access the bus via a PnvPHB4 object can do so via the
>> phb4->phb_base pointer.
>>
>> pnv_phb4_pec now creates a PnvPHB object.
>>
>> The powernv9 machine class will create PnvPHB devices with version '4'.
>> powernv10 will create using version '5'. Both are using global machine
>> properties in their class_init() to do that.
>>
>> These changes will benefit us when adding PnvPHB user creatable devices
>> for powernv9 and powernv10.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb4.c         | 29 +++++++++--------------------
>>   hw/pci-host/pnv_phb4_pec.c     |  6 +-----
>>   hw/ppc/pnv.c                   | 20 +++++++++++++++++++-
>>   include/hw/pci-host/pnv_phb4.h |  5 ++++-
>>   4 files changed, 33 insertions(+), 27 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
>> index ae5494fe72..22cf1c2a5e 100644
>> --- a/hw/pci-host/pnv_phb4.c
>> +++ b/hw/pci-host/pnv_phb4.c
>> @@ -49,7 +49,7 @@ static inline uint64_t SETFIELD(uint64_t mask, uint64_t word,
>>   static PCIDevice *pnv_phb4_find_cfg_dev(PnvPHB4 *phb)
>>   {
>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>       uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3];
>>       uint8_t bus, devfn;
>> @@ -145,7 +145,7 @@ static uint64_t pnv_phb4_config_read(PnvPHB4 *phb, unsigned off,
>>   static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>>                                        unsigned size, uint64_t val)
>>   {
>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>       PCIDevice *pdev;
>>       if (size != 4) {
>> @@ -166,7 +166,7 @@ static void pnv_phb4_rc_config_write(PnvPHB4 *phb, unsigned off,
>>   static uint64_t pnv_phb4_rc_config_read(PnvPHB4 *phb, unsigned off,
>>                                           unsigned size)
>>   {
>> -    PCIHostState *pci = PCI_HOST_BRIDGE(phb);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(phb->phb_base);
>>       PCIDevice *pdev;
>>       uint64_t val;
>> @@ -1608,16 +1608,6 @@ static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>>       pnv_phb4_xscom_realize(phb);
>>   }
>> -static const char *pnv_phb4_root_bus_path(PCIHostState *host_bridge,
>> -                                          PCIBus *rootbus)
>> -{
>> -    PnvPHB4 *phb = PNV_PHB4(host_bridge);
>> -
>> -    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>> -             phb->chip_id, phb->phb_id);
>> -    return phb->bus_path;
>> -}
>> -
>>   /*
>>    * Address base trigger mode (POWER10)
>>    *
>> @@ -1702,19 +1692,18 @@ static Property pnv_phb4_properties[] = {
>>           DEFINE_PROP_UINT32("chip-id", PnvPHB4, chip_id, 0),
>>           DEFINE_PROP_LINK("pec", PnvPHB4, pec, TYPE_PNV_PHB4_PEC,
>>                            PnvPhb4PecState *),
>> +        DEFINE_PROP_LINK("phb-base", PnvPHB4, phb_base,
>> +                         TYPE_PNV_PHB, PnvPHB *),
>>           DEFINE_PROP_END_OF_LIST(),
>>   };
>>   static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>>   {
>> -    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>>       DeviceClass *dc = DEVICE_CLASS(klass);
>>       XiveNotifierClass *xfc = XIVE_NOTIFIER_CLASS(klass);
>> -    hc->root_bus_path   = pnv_phb4_root_bus_path;
>>       dc->realize         = pnv_phb4_realize;
>>       device_class_set_props(dc, pnv_phb4_properties);
>> -    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>>       dc->user_creatable  = false;
>>       xfc->notify         = pnv_phb4_xive_notify;
>> @@ -1722,7 +1711,7 @@ static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>>   static const TypeInfo pnv_phb4_type_info = {
>>       .name          = TYPE_PNV_PHB4,
>> -    .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +    .parent        = TYPE_DEVICE,
>>       .instance_init = pnv_phb4_instance_init,
>>       .instance_size = sizeof(PnvPHB4),
>>       .class_init    = pnv_phb4_class_init,
>> @@ -1785,11 +1774,11 @@ static void pnv_phb4_root_port_realize(DeviceState *dev, Error **errp)
>>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>       PCIDevice *pci = PCI_DEVICE(dev);
>>       PCIBus *bus = pci_get_bus(pci);
>> -    PnvPHB4 *phb = NULL;
>> +    PnvPHB *phb = NULL;
>>       Error *local_err = NULL;
>> -    phb = (PnvPHB4 *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> -                                          TYPE_PNV_PHB4);
>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>> +                                         TYPE_PNV_PHB);
> 
> Same comment here re: accessing bus->qbus directly.
> 
>>       if (!phb) {
>>           error_setg(errp, "%s must be connected to pnv-phb4 buses", dev->id);
>> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
>> index 61bc0b503e..888ecbe8f3 100644
>> --- a/hw/pci-host/pnv_phb4_pec.c
>> +++ b/hw/pci-host/pnv_phb4_pec.c
>> @@ -115,8 +115,7 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>>                                           int stack_no,
>>                                           Error **errp)
>>   {
>> -    PnvPhb4PecClass *pecc = PNV_PHB4_PEC_GET_CLASS(pec);
>> -    PnvPHB4 *phb = PNV_PHB4(qdev_new(pecc->phb_type));
>> +    PnvPHB *phb = PNV_PHB(qdev_new(TYPE_PNV_PHB));
>>       int phb_id = pnv_phb4_pec_get_phb_id(pec, stack_no);
>>       object_property_add_child(OBJECT(pec), "phb[*]", OBJECT(phb));
>> @@ -130,9 +129,6 @@ static void pnv_pec_default_phb_realize(PnvPhb4PecState *pec,
>>       if (!sysbus_realize(SYS_BUS_DEVICE(phb), errp)) {
>>           return;
>>       }
>> -
>> -    /* Add a single Root port if running with defaults */
>> -    pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), pecc->rp_model);
>>   }
>>   static void pnv_pec_realize(DeviceState *dev, Error **errp)
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index 081b6839cc..3b0b230e49 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -688,7 +688,14 @@ static void pnv_chip_power8_pic_print_info(PnvChip *chip, Monitor *mon)
>>   static int pnv_chip_power9_pic_print_info_child(Object *child, void *opaque)
>>   {
>>       Monitor *mon = opaque;
>> -    PnvPHB4 *phb4 = (PnvPHB4 *) object_dynamic_cast(child, TYPE_PNV_PHB4);
>> +    PnvPHB *phb =  (PnvPHB *) object_dynamic_cast(child, TYPE_PNV_PHB);
> 
> I'm sure this could just be:
> 
>      PnvPHB *phb = PNV_PHB(child);

But it would assert if child is not of the correct type. The routine
above is called from a object_child_foreach() which loops on all
children.

I think it could be improved by using directly &chip*->phbs[i].

C.

> 
>> +    PnvPHB4 *phb4;
>> +
>> +    if (!phb) {
>> +        return 0;
>> +    }
>> +
>> +    phb4 = (PnvPHB4 *)phb->backend;
> 
> and you should be able to do:
> 
>      phb4 = PNV_PHB4(phb->backend);
> 
> My preference for using the QOM macros where possible is that they also include a type check that assert()s if you assign the wrong type of QOM object, which a direct C cast would miss.
> 
>>       if (phb4) {
>>           pnv_phb4_pic_print_info(phb4, mon);
>> @@ -2164,8 +2171,14 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>       PnvMachineClass *pmc = PNV_MACHINE_CLASS(oc);
>>       static const char compat[] = "qemu,powernv9\0ibm,powernv";
>> +    static GlobalProperty phb_compat[] = {
>> +        { TYPE_PNV_PHB, "version", "4" },
>> +    };
>> +
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
>> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>> +
>>       xfc->match_nvt = pnv_match_nvt;
>>       mc->alias = "powernv";
>> @@ -2182,8 +2195,13 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>>       XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
>>       static const char compat[] = "qemu,powernv10\0ibm,powernv";
>> +    static GlobalProperty phb_compat[] = {
>> +        { TYPE_PNV_PHB, "version", "5" },
>> +    };
>> +
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
>> +    compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>>       pmc->compat = compat;
>>       pmc->compat_size = sizeof(compat);
>> diff --git a/include/hw/pci-host/pnv_phb4.h b/include/hw/pci-host/pnv_phb4.h
>> index 90843ac3a9..f22253358f 100644
>> --- a/include/hw/pci-host/pnv_phb4.h
>> +++ b/include/hw/pci-host/pnv_phb4.h
>> @@ -18,6 +18,7 @@
>>   typedef struct PnvPhb4PecState PnvPhb4PecState;
>>   typedef struct PnvPhb4PecStack PnvPhb4PecStack;
>>   typedef struct PnvPHB4 PnvPHB4;
>> +typedef struct PnvPHB PnvPHB;
>>   typedef struct PnvChip PnvChip;
>>   /*
>> @@ -78,7 +79,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB4, PNV_PHB4)
>>   #define PCI_MMIO_TOTAL_SIZE        (0x1ull << 60)
>>   struct PnvPHB4 {
>> -    PCIExpressHost parent_obj;
>> +    DeviceState parent;
>> +
>> +    PnvPHB *phb_base;
>>       uint32_t chip_id;
>>       uint32_t phb_id;
> 
> 
> ATB,
> 
> Mark.

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Cédric Le Goater]

On 6/3/22 23:00, Daniel Henrique Barboza wrote:
> 
> 
> On 6/2/22 13:33, Frederic Barrat wrote:
>>
>>
>> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>>> To enable user creatable PnvPHB devices for powernv9 we'll revert the
>>> powernv9 related changes made in 9c10d86fee "ppc/pnv: Remove
>>> user-created PHB{3,4,5} devices".
>>>
>>> This change alone isn't enough to enable user creatable devices for powernv10
>>> due to how pnv_phb4_get_pec() currently works. For now let's just enable it
>>> for powernv9 alone.
>>>
>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>> ---
>>>   hw/pci-host/pnv_phb4.c     | 58 +++++++++++++++++++++++++++++++++++++-
>>>   hw/pci-host/pnv_phb4_pec.c |  6 ++--
>>>   hw/ppc/pnv.c               |  2 ++
>>>   3 files changed, 63 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
>>> index 22cf1c2a5e..a5c8ae494b 100644
>>> --- a/hw/pci-host/pnv_phb4.c
>>> +++ b/hw/pci-host/pnv_phb4.c
>>> @@ -1571,13 +1571,69 @@ void pnv_phb4_bus_init(DeviceState *dev, PnvPHB4 *phb)
>>>       pci->bus->flags |= PCI_BUS_EXTENDED_CONFIG_SPACE;
>>>   }
>>> +static PnvPhb4PecState *pnv_phb4_get_pec(PnvChip *chip, PnvPHB4 *phb,
>>> +                                         Error **errp)
>>> +{
>>> +    Pnv9Chip *chip9 = PNV9_CHIP(chip);
>>> +    int chip_id = phb->chip_id;
>>> +    int index = phb->phb_id;
>>> +    int i, j;
>>> +
>>> +    for (i = 0; i < chip->num_pecs; i++) {
>>> +        /*
>>> +         * For each PEC, check the amount of phbs it supports
>>> +         * and see if the given phb4 index matches an index.
>>> +         */
>>> +        PnvPhb4PecState *pec = &chip9->pecs[i];
>>> +
>>> +        for (j = 0; j < pec->num_phbs; j++) {
>>> +            if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
>>> +                return pec;
>>> +            }
>>> +        }
>>> +    }
>>> +
>>> +    error_setg(errp,
>>> +               "pnv-phb4 chip-id %d index %d didn't match any existing PEC",
>>> +               chip_id, index);
>>> +
>>> +    return NULL;
>>> +}
>>> +
>>>   static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>>>   {
>>>       PnvPHB4 *phb = PNV_PHB4(dev);
>>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>>> +    PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
>>>       XiveSource *xsrc = &phb->xsrc;
>>> +    BusState *s;
>>> +    Error *local_err = NULL;
>>>       int nr_irqs;
>>>       char name[32];
>>> +    if (!chip) {
>>> +        error_setg(errp, "invalid chip id: %d", phb->chip_id);
>>> +        return;
>>> +    }
>>> +
>>> +    /* User created PHBs need to be assigned to a PEC */
>>> +    if (!phb->pec) {
>>> +        phb->pec = pnv_phb4_get_pec(chip, phb, &local_err);
>>> +        if (local_err) {
>>> +            error_propagate(errp, local_err);
>>> +            return;
>>> +        }
>>> +    }
>>> +
>>> +    /* Reparent the PHB to the chip to build the device tree */
>>> +    pnv_chip_parent_fixup(chip, OBJECT(phb->phb_base), phb->phb_id);
>>
>>
>> Didn't you mean to do that only for the user-created case? 
> 
> It's done for both because the default generated PHBs are being created loosely
> from the chip starting on 3f4c369ea63e ("ppc/pnv: make PECs create and realize
> PHB4s"). We had to amend the code after that commit to fix the QOM hierarchy
> of these devices. 6e7b96750359 ("ppc/pnv: fix default PHB4 QOM hierarchy") has
> more details.
> 
> 
>> And why not attaching the PHB to the PEC instead of the chip, like in pnv_pec_default_phb_realize() ? I think it makes more sense to keep the chip->PEC->PHB parenting in the qom tree (and incidentally, that's where I'd prefer to have the phb3 model move to).
>
> I can only speculate since I didn't participate in the initial design. My
> educated guess is that we didn't want to show PECs in qtree, 

PECs are low level pieces of logic, that don't need to be exposed
under the monitor. We need the models for the FW though.

> hence we made the PHB a child of the chip instead. 

Did we ? see below.

> I'll also guess that this can be
> due to how PHB3 worked and we just copied the existing design.


hmm, I am not following what you are saying. Here is what we have :

P8:

/machine (powernv8-machine)
   /chip[0] (power8_v2.0-pnv-chip)
     /phb[0] (pnv-phb3)
       /lsi (ics)
       /msi (phb3-msi)
       /pbcq (pnv-pbcq)
         /xscom-pbcq-nest-0.0[0] (memory-region)
         /xscom-pbcq-pci-0.0[0] (memory-region)
         /xscom-pbcq-spci-0.0[0] (memory-region)
       /pnv-phb3-root.0 (pnv-phb3-root)

   /unattached (container)
     /device[1] (pnv-phb3-root-port)

P9:

/machine (powernv9-machine)
   /chip[0] (power9_v2.0-pnv-chip)
     /pec[0] (pnv-phb4-pec)
       /phb[0] (pnv-phb4)
         /pnv-phb4-root.0 (pnv-phb4-root)
         /source (xive-source)
         /xscom-pec-0.0-nest-phb-0[0] (memory-region)
         /xscom-pec-0.0-pci-phb-0[0] (memory-region)
         /xscom-pec-0.0-pci-phb-0[1] (memory-region)
     ...

   /unattached (container)
     /device[1] (pnv-phb4-root-port)


The RP was detached for some libvirt reason I think.

We would like to keep the same QOM hierarchy and possibly fixed
the modeling of PHB3 which is reversed.


> 
>> Also, the comment seems wrong to me. The qom parenting doesn't matter when building the device tree. 

it does. See pnv_dt_xscom()

Thanks,

C.


>> We only iterate over the PHBs found in the array of the PEC object (cf. pnv_pec_dt_xscom())
> 
> I believe it refers to the QOM tree, a.k.a qtree. This has no relation to the
> actual device tree the kernel uses. This comment can be clearer though.
> 
> 
> Thanks,
> 
> 
> Daniel
> 
>>
>>
>>
>>> +    s = qdev_get_parent_bus(DEVICE(chip));
>>> +    if (!qdev_set_parent_bus(DEVICE(phb->phb_base), s, &local_err)) {
>>> +        error_propagate(errp, local_err);
>>> +        return;
>>> +    }
>>
>>
>> Same comment, I think that's only desirable for user-created devices. We're already calling sysbus_realize() for the default case.
>>
>>
>> Silly question: where does it break if a user tries to create 2 PHBs with the same index?
>>
>>
>>    Fred
>>
>>
>>
>>
>>>       /* Set the "big_phb" flag */
>>>       phb->big_phb = phb->phb_id == 0 || phb->phb_id == 3;
>>> @@ -1803,7 +1859,7 @@ static void pnv_phb4_root_port_class_init(ObjectClass *klass, void *data)
>>>       PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>>>       dc->desc     = "IBM PHB4 PCIE Root Port";
>>> -    dc->user_creatable = false;
>>> +    dc->user_creatable = true;
>>>       device_class_set_parent_realize(dc, pnv_phb4_root_port_realize,
>>>                                       &rpc->parent_realize);
>>> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
>>> index 888ecbe8f3..0e67f3a338 100644
>>> --- a/hw/pci-host/pnv_phb4_pec.c
>>> +++ b/hw/pci-host/pnv_phb4_pec.c
>>> @@ -146,8 +146,10 @@ static void pnv_pec_realize(DeviceState *dev, Error **errp)
>>>       pec->num_phbs = pecc->num_phbs[pec->index];
>>>       /* Create PHBs if running with defaults */
>>> -    for (i = 0; i < pec->num_phbs; i++) {
>>> -        pnv_pec_default_phb_realize(pec, i, errp);
>>> +    if (defaults_enabled()) {
>>> +        for (i = 0; i < pec->num_phbs; i++) {
>>> +            pnv_pec_default_phb_realize(pec, i, errp);
>>> +        }
>>>       }
>>>       /* Initialize the XSCOM regions for the PEC registers */
>>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>>> index 3b0b230e49..697a2b5302 100644
>>> --- a/hw/ppc/pnv.c
>>> +++ b/hw/ppc/pnv.c
>>> @@ -2186,6 +2186,8 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>>       pmc->compat = compat;
>>>       pmc->compat_size = sizeof(compat);
>>>       pmc->dt_power_mgt = pnv_dt_power_mgt;
>>> +
>>> +    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
>>>   }
>>>   static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)

Re: [PATCH v2 11/16] ppc/pnv: add pnv-phb-root-port device

[Cédric Le Goater]

On 6/2/22 09:06, Mark Cave-Ayland wrote:
> On 01/06/2022 09:04, Daniel Henrique Barboza wrote:
> 
>> On 6/1/22 02:56, Cédric Le Goater wrote:
>>> On 5/31/22 23:49, Daniel Henrique Barboza wrote:
>>>> We have two very similar root-port devices, pnv-phb3-root-port and
>>>> pnv-phb4-root-port. Both consist of a wrapper around the PCIESlot device
>>>> that, until now, has no additional attributes.
>>>>
>>>> The main difference between the PHB3 and PHB4 root ports is that
>>>> pnv-phb4-root-port has the pnv_phb4_root_port_reset() callback. All
>>>> other differences can be merged in a single device without too much
>>>> trouble.
>>>>
>>>> This patch introduces the unified pnv-phb-root-port that, in time, will
>>>> be used as the default root port for the pnv-phb device.
>>>>
>>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>>> ---
>>>>   hw/pci-host/pnv_phb.c | 107 ++++++++++++++++++++++++++++++++++++++----
>>>>   hw/pci-host/pnv_phb.h |  17 +++++++
>>>>   2 files changed, 116 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>>>> index 321c4e768a..5047e90d3a 100644
>>>> --- a/hw/pci-host/pnv_phb.c
>>>> +++ b/hw/pci-host/pnv_phb.c
>>>> @@ -114,15 +114,106 @@ static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>>>       dc->user_creatable = true;
>>>>   }
>>>> -static void pnv_phb_register_type(void)
>>>> +static void pnv_phb_root_port_reset(DeviceState *dev)
>>>>   {
>>>> -    static const TypeInfo pnv_phb_type_info = {
>>>> -        .name          = TYPE_PNV_PHB,
>>>> -        .parent        = TYPE_PCIE_HOST_BRIDGE,
>>>> -        .instance_size = sizeof(PnvPHB),
>>>> -        .class_init    = pnv_phb_class_init,
>>>> -    };
>>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>>> +    PnvPHBRootPort *rootport = PNV_PHB_ROOT_PORT(dev);
>>>> +    PCIDevice *d = PCI_DEVICE(dev);
>>>> +    uint8_t *conf = d->config;
>>>> +    rpc->parent_reset(dev);
>>>> +
>>>> +    if (rootport->version == 3) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    /* PHB4 and later requires these extra reset steps */
>>>> +    pci_byte_test_and_set_mask(conf + PCI_IO_BASE,
>>>> +                               PCI_IO_RANGE_MASK & 0xff);
>>>> +    pci_byte_test_and_clear_mask(conf + PCI_IO_LIMIT,
>>>> +                                 PCI_IO_RANGE_MASK & 0xff);
>>>> +    pci_set_word(conf + PCI_MEMORY_BASE, 0);
>>>> +    pci_set_word(conf + PCI_MEMORY_LIMIT, 0xfff0);
>>>> +    pci_set_word(conf + PCI_PREF_MEMORY_BASE, 0x1);
>>>> +    pci_set_word(conf + PCI_PREF_MEMORY_LIMIT, 0xfff1);
>>>> +    pci_set_long(conf + PCI_PREF_BASE_UPPER32, 0x1); /* Hack */
>>>> +    pci_set_long(conf + PCI_PREF_LIMIT_UPPER32, 0xffffffff);
>>>> +    pci_config_set_interrupt_pin(conf, 0);
>>>> +}
>>>> +
>>>> +static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>>>> +{
>>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev);
>>>> +    PCIDevice *pci = PCI_DEVICE(dev);
>>>> +    PCIBus *bus = pci_get_bus(pci);
>>>> +    PnvPHB *phb = NULL;
>>>> +    Error *local_err = NULL;
>>>> +
>>>> +    phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>>> +                                          TYPE_PNV_PHB);
>>>> +
>>>> +    if (!phb) {
>>>> +        error_setg(errp,
>>>> +"pnv_phb_root_port devices must be connected to pnv-phb buses");
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    /* Set unique chassis/slot values for the root port */
>>>> +    qdev_prop_set_uint8(&pci->qdev, "chassis", phb->chip_id);
>>>> +    qdev_prop_set_uint16(&pci->qdev, "slot", phb->phb_id);
>>>> +
>>>> +    rpc->parent_realize(dev, &local_err);
>>>> +    if (local_err) {
>>>> +        error_propagate(errp, local_err);
>>>> +        return;
>>>> +    }
>>>> +    pci_config_set_interrupt_pin(pci->config, 0);
>>>> +}
>>>> +
>>>> +static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>>>> +{
>>>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>>>> +    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
>>>> +    PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass);
>>>> +
>>>> +    dc->desc     = "IBM PHB PCIE Root Port";
>>>> +
>>>> +    device_class_set_parent_realize(dc, pnv_phb_root_port_realize,
>>>> +                                    &rpc->parent_realize);
>>>> +
>>>> +    device_class_set_parent_reset(dc, pnv_phb_root_port_reset,
>>>> +                                  &rpc->parent_reset);
>>>> +    dc->reset = &pnv_phb_root_port_reset;
>>>> +
>>>> +    dc->user_creatable = true;
>>>> +
>>>> +    k->vendor_id = PCI_VENDOR_ID_IBM;
>>>> +    /* device_id represents the latest PHB root port version supported */
>>>> +    k->device_id = PNV_PHB5_DEVICE_ID;
>>>
>>> does that mean powernv8 machines will see phb devices as phb5 devices ?
>>
>>
>> I had something like this in this patch that would set device_id properly:
>>
>>
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> index 5d66264a96..d468e8d44a 100644
>> --- a/hw/pci-host/pnv_phb.c
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -144,6 +144,22 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>>       PCIBus *bus = pci_get_bus(pci);
>>       PnvPHB *phb = NULL;
>>       Error *local_err = NULL;
>> +    PnvPHBRootPort *phb_rp = PNV_PHB_ROOT_PORT(dev);
>> +    PCIDeviceClass *k = PCI_DEVICE_GET_CLASS(pci);
>> +
>> +    switch (phb_rp->version) {
>> +    case 3:
>> +        k->device_id = PNV_PHB3_DEVICE_ID;
>> +        break;
>> +    case 4:
>> +        k->device_id = PNV_PHB4_DEVICE_ID;
>> +        break;
>> +    case 5:
>> +        k->device_id = PNV_PHB5_DEVICE_ID;
>> +        break;
>> +    default:
>> +        g_assert_not_reached();
>> +    }
>>
>>       phb = (PnvPHB *) object_dynamic_cast(OBJECT(bus->qbus.parent),
>>                                             TYPE_PNV_PHB);
>> @@ -166,6 +182,11 @@ static void pnv_phb_root_port_realize(DeviceState *dev, Error **errp)
>>       pci_config_set_interrupt_pin(pci->config, 0);
>>   }
>>
>> +static Property pnv_phb_root_port_properties[] = {
>> +    DEFINE_PROP_UINT32("version", PnvPHBRootPort, version, 0),
>> +    DEFINE_PROP_END_OF_LIST(),
>> +};
>> +
>>   static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>>   {
>>       DeviceClass *dc = DEVICE_CLASS(klass);
>> @@ -181,6 +202,7 @@ static void pnv_phb_root_port_class_init(ObjectClass *klass, void *data)
>>                                     &rpc->parent_reset);
>>       dc->reset = &pnv_phb_root_port_reset;
>>
>> +    device_class_set_props(dc, pnv_phb_root_port_properties);
>>       dc->user_creatable = true;
>>
>>       k->vendor_id = PCI_VENDOR_ID_IBM;
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index 4d2ea405db..13c8753eb2 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -2148,6 +2148,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>>
>>       static GlobalProperty phb_compat[] = {
>>           { TYPE_PNV_PHB, "version", "3" },
>> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "3" },
>>       };
>>
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
>> @@ -2173,6 +2174,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>
>>       static GlobalProperty phb_compat[] = {
>>           { TYPE_PNV_PHB, "version", "4" },
>> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "4" },
>>       };
>>
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
>> @@ -2199,6 +2201,7 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>>
>>       static GlobalProperty phb_compat[] = {
>>           { TYPE_PNV_PHB, "version", "5" },
>> +        { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
>>       };
>>
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
>>
>>
>> The reason I didn't follow it through is because I wasn't sure if setting k->device_id
>> during realize() time was acceptable. Everyone else seems to set k->device_id during
>> class_init() or via an extra attribute 'device_id' that is written directly into
>> the PCI header.
>>
>> If this is something that we can do then I'm fine with fixing this up in this patch.
> 
> Yeah I'd be slightly nervous about doing this outside of class_init() too. I think my preferred method would be that used by virtio, which is to write the value to config space (see virtio_pci_device_plugged()) and then set k->device_id to 0 with an appropriate comment as per https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost-user-rng-pci.c#L53. That makes it quite clear that the 0 value in class_init() is a placeholder for something else.


Ah yes. This is a nice trick. It would simplify a lot the models.

Thanks,

C.


> 
>> Thanks,
>>
>> Daniel
>>
>>>
>>> C.
>>>
>>>> +    k->revision  = 0;
>>>> +
>>>> +    rpc->exp_offset = 0x48;
>>>> +    rpc->aer_offset = 0x100;
>>>> +}
>>>> +
>>>> +static const TypeInfo pnv_phb_type_info = {
>>>> +    .name          = TYPE_PNV_PHB,
>>>> +    .parent        = TYPE_PCIE_HOST_BRIDGE,
>>>> +    .instance_size = sizeof(PnvPHB),
>>>> +    .class_init    = pnv_phb_class_init,
>>>> +};
>>>> +
>>>> +static const TypeInfo pnv_phb_root_port_info = {
>>>> +    .name          = TYPE_PNV_PHB_ROOT_PORT,
>>>> +    .parent        = TYPE_PCIE_ROOT_PORT,
>>>> +    .instance_size = sizeof(PnvPHBRootPort),
>>>> +    .class_init    = pnv_phb_root_port_class_init,
>>>> +};
>>>> +
>>>> +static void pnv_phb_register_types(void)
>>>> +{
>>>>       type_register_static(&pnv_phb_type_info);
>>>> +    type_register_static(&pnv_phb_root_port_info);
>>>>   }
>>>> -type_init(pnv_phb_register_type)
>>>> +
>>>> +type_init(pnv_phb_register_types)
>>>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>>>> index a7cc8610e2..c8eab4b767 100644
>>>> --- a/hw/pci-host/pnv_phb.h
>>>> +++ b/hw/pci-host/pnv_phb.h
>>>> @@ -36,4 +36,21 @@ struct PnvPHB {
>>>>   #define TYPE_PNV_PHB "pnv-phb"
>>>>   OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>>>> +/*
>>>> + * PHB PCIe Root port
>>>> + */
>>>> +#define PNV_PHB3_DEVICE_ID         0x03dc
>>>> +#define PNV_PHB4_DEVICE_ID         0x04c1
>>>> +#define PNV_PHB5_DEVICE_ID         0x0652
>>>> +
>>>> +typedef struct PnvPHBRootPort {
>>>> +    PCIESlot parent_obj;
>>>> +
>>>> +    uint32_t version;
>>>> +} PnvPHBRootPort;
>>>> +
>>>> +#define TYPE_PNV_PHB_ROOT_PORT "pnv-phb-root-port"
>>>> +#define PNV_PHB_ROOT_PORT(obj) \
>>>> +    OBJECT_CHECK(PnvPHBRootPort, obj, TYPE_PNV_PHB_ROOT_PORT)
>>>> +
>>>>   #endif /* PCI_HOST_PNV_PHB_H */
> 
> ATB,
> 
> Mark.

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Cédric Le Goater]

On 6/2/22 18:16, Frederic Barrat wrote:
> 
> 
> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>> The PnvPHB device is going to be the base device for all other powernv
>> PHBs. It consists of a device that has the same user API as the other
>> PHB, namely being a PCIHostBridge and having chip-id and index
>> properties. It also has a 'backend' pointer that will be initialized
>> with the PHB implementation that the device is going to use.
>>
>> The initialization of the PHB backend is done by checking the PHB
>> version via a 'version' attribute that can be set via a global machine
>> property.  The 'version' field will be used to make adjustments based on
>> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
>> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
>> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
>> helper will be added later on.
>>
>> For now let's add the basic logic of the PnvPHB object, which consists
>> mostly of pnv_phb_realize() doing all the work of checking the
>> phb->version set, initializing the proper backend, passing through its
>> attributes to the chosen backend, finalizing the backend realize and
>> adding a root port in the end.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/meson.build |   3 +-
>>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>>   3 files changed, 164 insertions(+), 1 deletion(-)
>>   create mode 100644 hw/pci-host/pnv_phb.c
>>   create mode 100644 hw/pci-host/pnv_phb.h
>>
>> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
>> index c07596d0d1..e832babc9d 100644
>> --- a/hw/pci-host/meson.build
>> +++ b/hw/pci-host/meson.build
>> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>>     'pnv_phb3_msi.c',
>>     'pnv_phb3_pbcq.c',
>>     'pnv_phb4.c',
>> -  'pnv_phb4_pec.c'
>> +  'pnv_phb4_pec.c',
>> +  'pnv_phb.c',
>>   ))
>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>> new file mode 100644
>> index 0000000000..fa8472622f
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.c
>> @@ -0,0 +1,123 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/log.h"
>> +#include "qapi/visitor.h"
>> +#include "qapi/error.h"
>> +#include "hw/pci-host/pnv_phb.h"
>> +#include "hw/pci-host/pnv_phb3.h"
>> +#include "hw/pci-host/pnv_phb4.h"
>> +#include "hw/ppc/pnv.h"
>> +#include "hw/qdev-properties.h"
>> +#include "qom/object.h"
>> +
>> +
>> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
>> +{
>> +    PnvPHB *phb = PNV_PHB(dev);
>> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>> +    g_autofree char *phb_typename = NULL;
>> +    g_autofree char *phb_rootport_typename = NULL;
>> +
>> +    if (!phb->version) {
>> +        error_setg(errp, "version not specified");
>> +        return;
>> +    }
>> +
>> +    switch (phb->version) {
>> +    case 3:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
>> +        break;
>> +    case 4:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
>> +        break;
>> +    case 5:
>> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
>> +        break;
>> +    default:
>> +        g_assert_not_reached();
>> +    }
>> +
>> +    phb->backend = object_new(phb_typename);
>> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
>> +
>> +    /* Passthrough child device properties to the proxy device */
>> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
>> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
>> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
>> +
>> +    if (phb->version == 3) {
>> +        object_property_set_link(phb->backend, "chip",
>> +                                 OBJECT(phb->chip), errp);
>> +    } else {
>> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
>> +    }
> 
> 
> The patch is fine, but it just highlights that we're doing something wrong. I don't believe there's any reason for the chip/pec/phb relationship to be different between P8 and P9/P10. One day, a brave soul could try to unify the models, it would avoid test like that.

I think this is the first thing to do: introduce a PEC model to match
P8 HW and unify PHB3/4/5. We have been dragging this for too long (2014 ...)

It shouldn't be that complex with all the cleanups that have been done.

Thanks,

C.


> It would be a good cleanup series to do if we ever extend the model with yet another version :-)
>
> 
> 
> 
>> +
>> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (phb->version == 3) {
>> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
>> +    }
>> +
>> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
> 
> 
> 
> After we've removed the other instances (done in later patches), we could move pnv_phb_attach_root_port() to pnv_phb.c instead of pnv.c. It would be the perfect home for it as it starts looking off in pnv.c
> 
>    Fred
> 
> 
> 
>> +}
>> +
>> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
>> +                                         PCIBus *rootbus)
>> +{
>> +    PnvPHB *phb = PNV_PHB(host_bridge);
>> +
>> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>> +             phb->chip_id, phb->phb_id);
>> +    return phb->bus_path;
>> +}
>> +
>> +static Property pnv_phb_properties[] = {
>> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
>> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
>> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
>> +
>> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
>> +
>> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
>> +                         PnvPhb4PecState *),
>> +
>> +        DEFINE_PROP_END_OF_LIST(),
>> +};
>> +
>> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
>> +{
>> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +
>> +    hc->root_bus_path = pnv_phb_root_bus_path;
>> +    dc->realize = pnv_phb_realize;
>> +    device_class_set_props(dc, pnv_phb_properties);
>> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>> +    dc->user_creatable = true;
>> +}
>> +
>> +static void pnv_phb_register_type(void)
>> +{
>> +    static const TypeInfo pnv_phb_type_info = {
>> +        .name          = TYPE_PNV_PHB,
>> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
>> +        .instance_size = sizeof(PnvPHB),
>> +        .class_init    = pnv_phb_class_init,
>> +    };
>> +
>> +    type_register_static(&pnv_phb_type_info);
>> +}
>> +type_init(pnv_phb_register_type)
>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>> new file mode 100644
>> index 0000000000..a7cc8610e2
>> --- /dev/null
>> +++ b/hw/pci-host/pnv_phb.h
>> @@ -0,0 +1,39 @@
>> +/*
>> + * QEMU PowerPC PowerNV Proxy PHB model
>> + *
>> + * Copyright (c) 2022, IBM Corporation.
>> + *
>> + * This code is licensed under the GPL version 2 or later. See the
>> + * COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef PCI_HOST_PNV_PHB_H
>> +#define PCI_HOST_PNV_PHB_H
>> +
>> +#include "hw/pci/pcie_host.h"
>> +#include "hw/pci/pcie_port.h"
>> +#include "qom/object.h"
>> +
>> +typedef struct PnvChip PnvChip;
>> +typedef struct PnvPhb4PecState PnvPhb4PecState;
>> +
>> +struct PnvPHB {
>> +    PCIExpressHost parent_obj;
>> +
>> +    uint32_t chip_id;
>> +    uint32_t phb_id;
>> +    uint32_t version;
>> +    char bus_path[8];
>> +
>> +    PnvChip *chip;
>> +
>> +    PnvPhb4PecState *pec;
>> +
>> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
>> +    Object *backend;
>> +};
>> +
>> +#define TYPE_PNV_PHB "pnv-phb"
>> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>> +
>> +#endif /* PCI_HOST_PNV_PHB_H */

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Cédric Le Goater]

>>> Also, the comment seems wrong to me. The qom parenting doesn't matter when building the device tree. 
> 
> it does. See pnv_dt_xscom()

And this is the root cause of many headaches for user-created devices.
Could it be done differently ?

Thanks,

C.
  

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Frederic Barrat]

On 03/06/2022 23:00, Daniel Henrique Barboza wrote:
>>>   static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>>>   {
>>>       PnvPHB4 *phb = PNV_PHB4(dev);
>>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>>> +    PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
>>>       XiveSource *xsrc = &phb->xsrc;
>>> +    BusState *s;
>>> +    Error *local_err = NULL;
>>>       int nr_irqs;
>>>       char name[32];
>>> +    if (!chip) {
>>> +        error_setg(errp, "invalid chip id: %d", phb->chip_id);
>>> +        return;
>>> +    }
>>> +
>>> +    /* User created PHBs need to be assigned to a PEC */
>>> +    if (!phb->pec) {
>>> +        phb->pec = pnv_phb4_get_pec(chip, phb, &local_err);
>>> +        if (local_err) {
>>> +            error_propagate(errp, local_err);
>>> +            return;
>>> +        }
>>> +    }
>>> +
>>> +    /* Reparent the PHB to the chip to build the device tree */
>>> +    pnv_chip_parent_fixup(chip, OBJECT(phb->phb_base), phb->phb_id);
>>
>>
>> Didn't you mean to do that only for the user-created case? 
> 
> It's done for both because the default generated PHBs are being created 
> loosely
> from the chip starting on 3f4c369ea63e ("ppc/pnv: make PECs create and 
> realize
> PHB4s"). We had to amend the code after that commit to fix the QOM 
> hierarchy
> of these devices. 6e7b96750359 ("ppc/pnv: fix default PHB4 QOM 
> hierarchy") has
> more details.
> 
> 
>> And why not attaching the PHB to the PEC instead of the chip, like in 
>> pnv_pec_default_phb_realize() ? I think it makes more sense to keep 
>> the chip->PEC->PHB parenting in the qom tree (and incidentally, that's 
>> where I'd prefer to have the phb3 model move to).
> 
> I can only speculate since I didn't participate in the initial design. My
> educated guess is that we didn't want to show PECs in qtree, hence we
> made the PHB a child of the chip instead. I'll also guess that this can be
> due to how PHB3 worked and we just copied the existing design.


I don't believe that's correct though. As Cedric replied, the PECs show 
up in the qom tree on P9/P10, with chip->PEC->PHB, in that order. And I 
think that's fine, that's the model we should tend to (and which would 
require a fix on P8, since there we have chip->phb->pbcq, which is 
backwards. The pbcq object is the equivalent of the PEC).

So I think we should keep the qom relationship as it currently is on 
P9/P10, since it looks good. On P8, we can keep the current status for 
now and, as discussed privately, have another series later to clean 
things up.

   Fred

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Frederic Barrat]

On 07/06/2022 08:42, Cédric Le Goater wrote:
> On 6/2/22 18:16, Frederic Barrat wrote:
>>
>>
>> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>>> The PnvPHB device is going to be the base device for all other powernv
>>> PHBs. It consists of a device that has the same user API as the other
>>> PHB, namely being a PCIHostBridge and having chip-id and index
>>> properties. It also has a 'backend' pointer that will be initialized
>>> with the PHB implementation that the device is going to use.
>>>
>>> The initialization of the PHB backend is done by checking the PHB
>>> version via a 'version' attribute that can be set via a global machine
>>> property.  The 'version' field will be used to make adjustments based on
>>> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
>>> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
>>> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
>>> helper will be added later on.
>>>
>>> For now let's add the basic logic of the PnvPHB object, which consists
>>> mostly of pnv_phb_realize() doing all the work of checking the
>>> phb->version set, initializing the proper backend, passing through its
>>> attributes to the chosen backend, finalizing the backend realize and
>>> adding a root port in the end.
>>>
>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>> ---
>>>   hw/pci-host/meson.build |   3 +-
>>>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>>>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>>>   3 files changed, 164 insertions(+), 1 deletion(-)
>>>   create mode 100644 hw/pci-host/pnv_phb.c
>>>   create mode 100644 hw/pci-host/pnv_phb.h
>>>
>>> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
>>> index c07596d0d1..e832babc9d 100644
>>> --- a/hw/pci-host/meson.build
>>> +++ b/hw/pci-host/meson.build
>>> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', 
>>> if_true: files(
>>>     'pnv_phb3_msi.c',
>>>     'pnv_phb3_pbcq.c',
>>>     'pnv_phb4.c',
>>> -  'pnv_phb4_pec.c'
>>> +  'pnv_phb4_pec.c',
>>> +  'pnv_phb.c',
>>>   ))
>>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>>> new file mode 100644
>>> index 0000000000..fa8472622f
>>> --- /dev/null
>>> +++ b/hw/pci-host/pnv_phb.c
>>> @@ -0,0 +1,123 @@
>>> +/*
>>> + * QEMU PowerPC PowerNV Proxy PHB model
>>> + *
>>> + * Copyright (c) 2022, IBM Corporation.
>>> + *
>>> + * This code is licensed under the GPL version 2 or later. See the
>>> + * COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#include "qemu/osdep.h"
>>> +#include "qemu/log.h"
>>> +#include "qapi/visitor.h"
>>> +#include "qapi/error.h"
>>> +#include "hw/pci-host/pnv_phb.h"
>>> +#include "hw/pci-host/pnv_phb3.h"
>>> +#include "hw/pci-host/pnv_phb4.h"
>>> +#include "hw/ppc/pnv.h"
>>> +#include "hw/qdev-properties.h"
>>> +#include "qom/object.h"
>>> +
>>> +
>>> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
>>> +{
>>> +    PnvPHB *phb = PNV_PHB(dev);
>>> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>>> +    g_autofree char *phb_typename = NULL;
>>> +    g_autofree char *phb_rootport_typename = NULL;
>>> +
>>> +    if (!phb->version) {
>>> +        error_setg(errp, "version not specified");
>>> +        return;
>>> +    }
>>> +
>>> +    switch (phb->version) {
>>> +    case 3:
>>> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
>>> +        break;
>>> +    case 4:
>>> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
>>> +        break;
>>> +    case 5:
>>> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
>>> +        break;
>>> +    default:
>>> +        g_assert_not_reached();
>>> +    }
>>> +
>>> +    phb->backend = object_new(phb_typename);
>>> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
>>> +
>>> +    /* Passthrough child device properties to the proxy device */
>>> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
>>> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, 
>>> errp);
>>> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), 
>>> errp);
>>> +
>>> +    if (phb->version == 3) {
>>> +        object_property_set_link(phb->backend, "chip",
>>> +                                 OBJECT(phb->chip), errp);
>>> +    } else {
>>> +        object_property_set_link(phb->backend, "pec", 
>>> OBJECT(phb->pec), errp);
>>> +    }
>>
>>
>> The patch is fine, but it just highlights that we're doing something 
>> wrong. I don't believe there's any reason for the chip/pec/phb 
>> relationship to be different between P8 and P9/P10. One day, a brave 
>> soul could try to unify the models, it would avoid test like that.
> 
> I think this is the first thing to do: introduce a PEC model to match
> P8 HW and unify PHB3/4/5. We have been dragging this for too long (2014 
> ...)


I agree it's needed, but it's also orthogonal to this series. And I 
wouldn't force it on Daniel since we're getting into PCI territory.

   Fred


> 
> It shouldn't be that complex with all the cleanups that have been done.
> 
> Thanks,
> 
> C.
> 
> 
>> It would be a good cleanup series to do if we ever extend the model 
>> with yet another version :-)
>>
>>
>>
>>
>>> +
>>> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
>>> +        return;
>>> +    }
>>> +
>>> +    if (phb->version == 3) {
>>> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
>>> +    }
>>> +
>>> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
>>
>>
>>
>> After we've removed the other instances (done in later patches), we 
>> could move pnv_phb_attach_root_port() to pnv_phb.c instead of pnv.c. 
>> It would be the perfect home for it as it starts looking off in pnv.c
>>
>>    Fred
>>
>>
>>
>>> +}
>>> +
>>> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
>>> +                                         PCIBus *rootbus)
>>> +{
>>> +    PnvPHB *phb = PNV_PHB(host_bridge);
>>> +
>>> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>>> +             phb->chip_id, phb->phb_id);
>>> +    return phb->bus_path;
>>> +}
>>> +
>>> +static Property pnv_phb_properties[] = {
>>> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
>>> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
>>> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
>>> +
>>> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, 
>>> PnvChip *),
>>> +
>>> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
>>> +                         PnvPhb4PecState *),
>>> +
>>> +        DEFINE_PROP_END_OF_LIST(),
>>> +};
>>> +
>>> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>> +{
>>> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>>> +
>>> +    hc->root_bus_path = pnv_phb_root_bus_path;
>>> +    dc->realize = pnv_phb_realize;
>>> +    device_class_set_props(dc, pnv_phb_properties);
>>> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>>> +    dc->user_creatable = true;
>>> +}
>>> +
>>> +static void pnv_phb_register_type(void)
>>> +{
>>> +    static const TypeInfo pnv_phb_type_info = {
>>> +        .name          = TYPE_PNV_PHB,
>>> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
>>> +        .instance_size = sizeof(PnvPHB),
>>> +        .class_init    = pnv_phb_class_init,
>>> +    };
>>> +
>>> +    type_register_static(&pnv_phb_type_info);
>>> +}
>>> +type_init(pnv_phb_register_type)
>>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>>> new file mode 100644
>>> index 0000000000..a7cc8610e2
>>> --- /dev/null
>>> +++ b/hw/pci-host/pnv_phb.h
>>> @@ -0,0 +1,39 @@
>>> +/*
>>> + * QEMU PowerPC PowerNV Proxy PHB model
>>> + *
>>> + * Copyright (c) 2022, IBM Corporation.
>>> + *
>>> + * This code is licensed under the GPL version 2 or later. See the
>>> + * COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#ifndef PCI_HOST_PNV_PHB_H
>>> +#define PCI_HOST_PNV_PHB_H
>>> +
>>> +#include "hw/pci/pcie_host.h"
>>> +#include "hw/pci/pcie_port.h"
>>> +#include "qom/object.h"
>>> +
>>> +typedef struct PnvChip PnvChip;
>>> +typedef struct PnvPhb4PecState PnvPhb4PecState;
>>> +
>>> +struct PnvPHB {
>>> +    PCIExpressHost parent_obj;
>>> +
>>> +    uint32_t chip_id;
>>> +    uint32_t phb_id;
>>> +    uint32_t version;
>>> +    char bus_path[8];
>>> +
>>> +    PnvChip *chip;
>>> +
>>> +    PnvPhb4PecState *pec;
>>> +
>>> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
>>> +    Object *backend;
>>> +};
>>> +
>>> +#define TYPE_PNV_PHB "pnv-phb"
>>> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>>> +
>>> +#endif /* PCI_HOST_PNV_PHB_H */
> 

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Frederic Barrat]

On 07/06/2022 08:35, Cédric Le Goater wrote:
>>
>>> Also, the comment seems wrong to me. The qom parenting doesn't matter 
>>> when building the device tree. 
> 
> it does. See pnv_dt_xscom()


Yeah, what I meant is that on P9, there's no "dt_scom" method for the 
PHB. The PHBs are added by the dt_scom() of the PEC. So the parenting of 
the PHB doesn't really matter.

I was actually wondering why it was done that way. If we have a clean 
qom tree (again, only on P9/P10 because P8 is wrong), then the PEC could 
add the "pbcq@xxxxxx" layer in the device tree, then call the qom 
children, i.e. the PHBs, and they would each add themselves (each phb 
adds the 'stack@xxxxxx' entry in the device tree).

But then I see your comment about giving headaches for user-created 
devices. So something else to discuss...

   Fred

Re: [PATCH v2 03/16] ppc/pnv: add PnvPHB base/proxy device

[Daniel Henrique Barboza]

On 6/7/22 05:44, Frederic Barrat wrote:
> 
> 
> On 07/06/2022 08:42, Cédric Le Goater wrote:
>> On 6/2/22 18:16, Frederic Barrat wrote:
>>>
>>>
>>> On 31/05/2022 23:49, Daniel Henrique Barboza wrote:
>>>> The PnvPHB device is going to be the base device for all other powernv
>>>> PHBs. It consists of a device that has the same user API as the other
>>>> PHB, namely being a PCIHostBridge and having chip-id and index
>>>> properties. It also has a 'backend' pointer that will be initialized
>>>> with the PHB implementation that the device is going to use.
>>>>
>>>> The initialization of the PHB backend is done by checking the PHB
>>>> version via a 'version' attribute that can be set via a global machine
>>>> property.  The 'version' field will be used to make adjustments based on
>>>> the running version, e.g. PHB3 uses a 'chip' reference while PHB4 uses
>>>> 'pec'. To init the PnvPHB bus we'll rely on helpers for each version.
>>>> The version 3 helper is already added (pnv_phb3_bus_init), the PHB4
>>>> helper will be added later on.
>>>>
>>>> For now let's add the basic logic of the PnvPHB object, which consists
>>>> mostly of pnv_phb_realize() doing all the work of checking the
>>>> phb->version set, initializing the proper backend, passing through its
>>>> attributes to the chosen backend, finalizing the backend realize and
>>>> adding a root port in the end.
>>>>
>>>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>>> ---
>>>>   hw/pci-host/meson.build |   3 +-
>>>>   hw/pci-host/pnv_phb.c   | 123 ++++++++++++++++++++++++++++++++++++++++
>>>>   hw/pci-host/pnv_phb.h   |  39 +++++++++++++
>>>>   3 files changed, 164 insertions(+), 1 deletion(-)
>>>>   create mode 100644 hw/pci-host/pnv_phb.c
>>>>   create mode 100644 hw/pci-host/pnv_phb.h
>>>>
>>>> diff --git a/hw/pci-host/meson.build b/hw/pci-host/meson.build
>>>> index c07596d0d1..e832babc9d 100644
>>>> --- a/hw/pci-host/meson.build
>>>> +++ b/hw/pci-host/meson.build
>>>> @@ -35,5 +35,6 @@ specific_ss.add(when: 'CONFIG_PCI_POWERNV', if_true: files(
>>>>     'pnv_phb3_msi.c',
>>>>     'pnv_phb3_pbcq.c',
>>>>     'pnv_phb4.c',
>>>> -  'pnv_phb4_pec.c'
>>>> +  'pnv_phb4_pec.c',
>>>> +  'pnv_phb.c',
>>>>   ))
>>>> diff --git a/hw/pci-host/pnv_phb.c b/hw/pci-host/pnv_phb.c
>>>> new file mode 100644
>>>> index 0000000000..fa8472622f
>>>> --- /dev/null
>>>> +++ b/hw/pci-host/pnv_phb.c
>>>> @@ -0,0 +1,123 @@
>>>> +/*
>>>> + * QEMU PowerPC PowerNV Proxy PHB model
>>>> + *
>>>> + * Copyright (c) 2022, IBM Corporation.
>>>> + *
>>>> + * This code is licensed under the GPL version 2 or later. See the
>>>> + * COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#include "qemu/osdep.h"
>>>> +#include "qemu/log.h"
>>>> +#include "qapi/visitor.h"
>>>> +#include "qapi/error.h"
>>>> +#include "hw/pci-host/pnv_phb.h"
>>>> +#include "hw/pci-host/pnv_phb3.h"
>>>> +#include "hw/pci-host/pnv_phb4.h"
>>>> +#include "hw/ppc/pnv.h"
>>>> +#include "hw/qdev-properties.h"
>>>> +#include "qom/object.h"
>>>> +
>>>> +
>>>> +static void pnv_phb_realize(DeviceState *dev, Error **errp)
>>>> +{
>>>> +    PnvPHB *phb = PNV_PHB(dev);
>>>> +    PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>>>> +    g_autofree char *phb_typename = NULL;
>>>> +    g_autofree char *phb_rootport_typename = NULL;
>>>> +
>>>> +    if (!phb->version) {
>>>> +        error_setg(errp, "version not specified");
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    switch (phb->version) {
>>>> +    case 3:
>>>> +        phb_typename = g_strdup(TYPE_PNV_PHB3);
>>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB3_ROOT_PORT);
>>>> +        break;
>>>> +    case 4:
>>>> +        phb_typename = g_strdup(TYPE_PNV_PHB4);
>>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB4_ROOT_PORT);
>>>> +        break;
>>>> +    case 5:
>>>> +        phb_typename = g_strdup(TYPE_PNV_PHB5);
>>>> +        phb_rootport_typename = g_strdup(TYPE_PNV_PHB5_ROOT_PORT);
>>>> +        break;
>>>> +    default:
>>>> +        g_assert_not_reached();
>>>> +    }
>>>> +
>>>> +    phb->backend = object_new(phb_typename);
>>>> +    object_property_add_child(OBJECT(dev), "phb-device", phb->backend);
>>>> +
>>>> +    /* Passthrough child device properties to the proxy device */
>>>> +    object_property_set_uint(phb->backend, "index", phb->phb_id, errp);
>>>> +    object_property_set_uint(phb->backend, "chip-id", phb->chip_id, errp);
>>>> +    object_property_set_link(phb->backend, "phb-base", OBJECT(phb), errp);
>>>> +
>>>> +    if (phb->version == 3) {
>>>> +        object_property_set_link(phb->backend, "chip",
>>>> +                                 OBJECT(phb->chip), errp);
>>>> +    } else {
>>>> +        object_property_set_link(phb->backend, "pec", OBJECT(phb->pec), errp);
>>>> +    }
>>>
>>>
>>> The patch is fine, but it just highlights that we're doing something wrong. I don't believe there's any reason for the chip/pec/phb relationship to be different between P8 and P9/P10. One day, a brave soul could try to unify the models, it would avoid test like that.
>>
>> I think this is the first thing to do: introduce a PEC model to match
>> P8 HW and unify PHB3/4/5. We have been dragging this for too long (2014 ...)
> 
> 
> I agree it's needed, but it's also orthogonal to this series. And I wouldn't force it on Daniel since we're getting into PCI territory.

I don't mind doing it after this series lands.


Thanks,


Daniel

> 
>    Fred
> 
> 
>>
>> It shouldn't be that complex with all the cleanups that have been done.
>>
>> Thanks,
>>
>> C.
>>
>>
>>> It would be a good cleanup series to do if we ever extend the model with yet another version :-)
>>>
>>>
>>>
>>>
>>>> +
>>>> +    if (!qdev_realize(DEVICE(phb->backend), NULL, errp)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (phb->version == 3) {
>>>> +        pnv_phb3_bus_init(dev, (PnvPHB3 *)phb->backend);
>>>> +    }
>>>> +
>>>> +    pnv_phb_attach_root_port(pci, phb_rootport_typename);
>>>
>>>
>>>
>>> After we've removed the other instances (done in later patches), we could move pnv_phb_attach_root_port() to pnv_phb.c instead of pnv.c. It would be the perfect home for it as it starts looking off in pnv.c
>>>
>>>    Fred
>>>
>>>
>>>
>>>> +}
>>>> +
>>>> +static const char *pnv_phb_root_bus_path(PCIHostState *host_bridge,
>>>> +                                         PCIBus *rootbus)
>>>> +{
>>>> +    PnvPHB *phb = PNV_PHB(host_bridge);
>>>> +
>>>> +    snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x",
>>>> +             phb->chip_id, phb->phb_id);
>>>> +    return phb->bus_path;
>>>> +}
>>>> +
>>>> +static Property pnv_phb_properties[] = {
>>>> +        DEFINE_PROP_UINT32("index", PnvPHB, phb_id, 0),
>>>> +        DEFINE_PROP_UINT32("chip-id", PnvPHB, chip_id, 0),
>>>> +        DEFINE_PROP_UINT32("version", PnvPHB, version, 0),
>>>> +
>>>> +        DEFINE_PROP_LINK("chip", PnvPHB, chip, TYPE_PNV_CHIP, PnvChip *),
>>>> +
>>>> +        DEFINE_PROP_LINK("pec", PnvPHB, pec, TYPE_PNV_PHB4_PEC,
>>>> +                         PnvPhb4PecState *),
>>>> +
>>>> +        DEFINE_PROP_END_OF_LIST(),
>>>> +};
>>>> +
>>>> +static void pnv_phb_class_init(ObjectClass *klass, void *data)
>>>> +{
>>>> +    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass);
>>>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>>>> +
>>>> +    hc->root_bus_path = pnv_phb_root_bus_path;
>>>> +    dc->realize = pnv_phb_realize;
>>>> +    device_class_set_props(dc, pnv_phb_properties);
>>>> +    set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>>>> +    dc->user_creatable = true;
>>>> +}
>>>> +
>>>> +static void pnv_phb_register_type(void)
>>>> +{
>>>> +    static const TypeInfo pnv_phb_type_info = {
>>>> +        .name          = TYPE_PNV_PHB,
>>>> +        .parent        = TYPE_PCIE_HOST_BRIDGE,
>>>> +        .instance_size = sizeof(PnvPHB),
>>>> +        .class_init    = pnv_phb_class_init,
>>>> +    };
>>>> +
>>>> +    type_register_static(&pnv_phb_type_info);
>>>> +}
>>>> +type_init(pnv_phb_register_type)
>>>> diff --git a/hw/pci-host/pnv_phb.h b/hw/pci-host/pnv_phb.h
>>>> new file mode 100644
>>>> index 0000000000..a7cc8610e2
>>>> --- /dev/null
>>>> +++ b/hw/pci-host/pnv_phb.h
>>>> @@ -0,0 +1,39 @@
>>>> +/*
>>>> + * QEMU PowerPC PowerNV Proxy PHB model
>>>> + *
>>>> + * Copyright (c) 2022, IBM Corporation.
>>>> + *
>>>> + * This code is licensed under the GPL version 2 or later. See the
>>>> + * COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#ifndef PCI_HOST_PNV_PHB_H
>>>> +#define PCI_HOST_PNV_PHB_H
>>>> +
>>>> +#include "hw/pci/pcie_host.h"
>>>> +#include "hw/pci/pcie_port.h"
>>>> +#include "qom/object.h"
>>>> +
>>>> +typedef struct PnvChip PnvChip;
>>>> +typedef struct PnvPhb4PecState PnvPhb4PecState;
>>>> +
>>>> +struct PnvPHB {
>>>> +    PCIExpressHost parent_obj;
>>>> +
>>>> +    uint32_t chip_id;
>>>> +    uint32_t phb_id;
>>>> +    uint32_t version;
>>>> +    char bus_path[8];
>>>> +
>>>> +    PnvChip *chip;
>>>> +
>>>> +    PnvPhb4PecState *pec;
>>>> +
>>>> +    /* The PHB backend (PnvPHB3, PnvPHB4 ...) being used */
>>>> +    Object *backend;
>>>> +};
>>>> +
>>>> +#define TYPE_PNV_PHB "pnv-phb"
>>>> +OBJECT_DECLARE_SIMPLE_TYPE(PnvPHB, PNV_PHB)
>>>> +
>>>> +#endif /* PCI_HOST_PNV_PHB_H */
>>

Re: [PATCH v2 08/16] ppc/pnv: user created pnv-phb for powernv9

[Daniel Henrique Barboza]

On 6/7/22 03:44, Cédric Le Goater wrote:
>>>> Also, the comment seems wrong to me. The qom parenting doesn't matter when building the device tree. 
>>
>> it does. See pnv_dt_xscom()
> 
> And this is the root cause of many headaches for user-created devices.
> Could it be done differently ?

Just tried to do a related change based on the review you gave in patch 07:

----
But it would assert if child is not of the correct type. The routine
above is called from a object_child_foreach() which loops on all
children.

I think it could be improved by using directly &chip*->phbs[i].

C.
-----

This doesn't work out of the gate because, for user creatable devices, we
are not setting the phbs back into the chip->phbs[] array. In fact we're
not even incrementing chip->num_phbs. We were getting away with all of it
because we are parenting the PHBs to the chip during realize.

Considering that, and also how pnv_dt_xscom() works, I´d rather stick with
most of the the QOM usage we already have. Otherwise we would need, for example,
to change xscom_dt_child to go through each device we want in the DT. And
there's no clear benefit aside from using less QOM, but that can be amended
by adding more documentation here and there.

I can make an exception for powernv8 and pnv_ics_get_child()/pnv_ics_resend_child(),
where we're cycling through all child elements every time. For those cases it's worth
to access the phbs directly via chip->phbs[], and for user creatable phb3s I'll add
the created phb in the array.

I also believe that I can do more to make the current handling of default phb3/phb4
closer to what user creatable devices does. This will ease the work to be done
by this series and will also make the design easier to understand. I might also do
some changes that Mark pointed out in the phb3/4 root ports as well. This series can
then be more about the PnvPHB proxy.


Thanks,


Daniel




> 
> Thanks,
> 
> C.
> 
>