[Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Marcel Apfelbaum]

Notes:
 - Sorry for the late submission, I was waiting for dynamic ACPI series to get merged
   in order to submit - my bad.
 - The prev version (v2) was wrongfully tagged by me as RFC, it was actually ready but not
   rebased. V3 only rebases with no actual functionality changed.
 - This series is not that big, patches 1-8 are really small and can be submitted
   separately, however I preferred to keep them here to get the whole picture.

The series is fully functional.
 - Limitations:
   - Pxb's bus does not support hotplug. It will be addressed on top of this series
     because is already getting to big.
 - Depends on:
    - [SeaBIOS] [PATCH V3 0/2] fw/pci: better support for multiple host bridges
      - It was already reviewed/accepted by the maintainers, it will be merged
        after this series gets accepted.
 - You are more than welcome to try using:
       -device pxb-device,id=pxb,bus_nr=4,numa_node=1 -device e1000,bus=pxb,addr=0x1 -bios <patched with the above series>

v4->v5:
 - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
 - Added PXB documentation (patch 28/28)
 - Addressed Gerd Hoffmann's review:
   - fix PXB behaviour if used with unsupported BIOS (patch 27/28)
 - Addressed Michael S. Tsirkin's review:
   - Removed assert in aml_index (patch 5/28)
   - Renamed pci_ functions to crs_ (patch 12/28)
   - used uint64_t variables instead of signed ones (patch 12/28)
   - Emit MEM/IO AML only for PXBs and i440fx (patch 26/28)
 - Addressed Shannon Zhao's review:
   - Changed build_append_int to  build_append_byte in aml_or (patch 2/25)
 - Thanks to Igor and Kevin for reviews

v3->v4:
 - Addressed Michael S. Tsirkin's review:
   - refactored build_prt method (patch 11/25) 
     hw/apci: add _PRT method for extra PCI root busses
 - Addressed Igor Mammedov's reiew
   - add assert to aml_index (patch 5/25)
 - Fixed aml_equal implementation (patch 1/25)

v2->v3:
 - Rebased on Michael S. Tsirkin's pci branch (that includes now all the dependencies)
 - Refactored acpi terms patch into multiple patches to match Igor's design.

v1->v2:
 - Add support for multiple pxb devices.
 - Attach pxb's bus to specific NUMA node.
 - Got rid of the hacks from prev version.
 - Tested also for Win7 and Fedora 20, and for virtio blk devices.
 - Several bug-fixes resulting in a stable version ready for submission.

Reasoning:
We need multiple primary busess for a few reasons, the most important one
is to be able to associate a pass-trough device with a guest NUMA node.
The OS-es are able to associate a NUMA node only to a primary bus, not to
a specific PCI device or a pci-2-pci bridge.
PC machines support multiple NUMA nodes for CPUs and memory, however the IO
was not yet supported.

patch 1-9 adds the necessary acpi constructs based on Igor's series
patch 10-13 implements acpi code needed to expose the pxb's primary bus to guests
patch 14 separates the pci_bus code into a designated file
patch 15-19 handles the implicit assumptions in code that only one primary bus can exist
patch 20 handles the actual implementation of the PXB devices
patch 21-22 enables the device
patch 23 implements PXB map_irq function, (can be squashed into the actual PXB)
patch 24-25 adds NUMA support
patch 26 Emit MEM/IO AML only for PXBs and i440fx
patch 27 Backward compatibility with SeaBIOS
patch 28 PXB Documentation

Marcel Apfelbaum (28):
  acpi: fix aml_equal term implementation
  acpi: add aml_or() term
  acpi: add aml_add() term
  acpi: add aml_lless() term
  acpi: add aml_index() term
  acpi: add aml_shiftleft() term
  acpi: add aml_shiftright() term
  acpi: add aml_increment() term
  acpi: add aml_while() term
  hw/acpi: add support for multiple root busses
  hw/apci: add _PRT method for extra PCI root busses
  hw/acpi: add _CRS method for extra root busses
  hw/acpi: remove from root bus 0 the crs resources used by other
    busses.
  hw/pci: move pci bus related code to separate files
  hw/pci: made pci_bus_is_root a PCIBusClass method
  hw/pci: made pci_bus_num a PCIBusClass method
  hw/pci: introduce TYPE_PCI_MAIN_HOST_BRIDGE interface
  hw/pci: removed 'rootbus nr is 0' assumption from qmp_pci_query
  hw/pci: implement iteration over multiple host bridges
  hw/pci: introduce PCI Expander Bridge (PXB)
  hw/pci: inform bios if the system has more than one pci bridge
  hw/pci: piix - suport multiple host bridges
  hw/pxb: add map_irq func
  hw/pci_bus: add support for NUMA nodes
  hw/pxb: add numa_node parameter
  acpi: restrict the aml emission to PXB host bridges
  apci: fix PXB behaviour if used with unsupported BIOS
  docs: Add PXB documentation

 arch_init.c                         |   1 +
 docs/pci_expander_bridge.txt        |  52 ++++
 hw/acpi/aml-build.c                 |  76 +++++-
 hw/alpha/typhoon.c                  |   1 +
 hw/i386/acpi-build.c                | 367 ++++++++++++++++++++++++-
 hw/i386/kvm/pci-assign.c            |   1 +
 hw/i386/pc.c                        |  13 +
 hw/mips/gt64xxx_pci.c               |   1 +
 hw/pci-bridge/Makefile.objs         |   1 +
 hw/pci-bridge/pci_expander_bridge.c | 207 +++++++++++++++
 hw/pci-host/bonito.c                |   1 +
 hw/pci-host/grackle.c               |   1 +
 hw/pci-host/piix.c                  |  63 ++++-
 hw/pci-host/ppce500.c               |   1 +
 hw/pci-host/q35.c                   |   5 +
 hw/pci-host/uninorth.c              |   1 +
 hw/pci/Makefile.objs                |   2 +-
 hw/pci/pci.c                        | 501 +---------------------------------
 hw/pci/pci_bus.c                    | 517 ++++++++++++++++++++++++++++++++++++
 hw/pci/pci_host.c                   |   6 +
 hw/ppc/ppc4xx_pci.c                 |   1 +
 hw/scsi/megasas.c                   |   1 +
 hw/sh4/r2d.c                        |   1 +
 hw/sh4/sh_pci.c                     |   1 +
 hw/vfio/pci.c                       |   1 +
 hw/xen/xen_pt.c                     |   1 +
 include/hw/acpi/aml-build.h         |   8 +
 include/hw/pci/pci.h                |   6 +-
 include/hw/pci/pci_bus.h            |  35 +++
 include/hw/pci/pci_host.h           |  13 +
 include/sysemu/sysemu.h             |   1 +
 31 files changed, 1377 insertions(+), 510 deletions(-)
 create mode 100644 docs/pci_expander_bridge.txt
 create mode 100644 hw/pci-bridge/pci_expander_bridge.c
 create mode 100644 hw/pci/pci_bus.c

-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 01/28] acpi: fix aml_equal term implementation

[Marcel Apfelbaum]

The DefLEqual op does not have a target operand. Remove it.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index e01b8c2..ace180b 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -546,7 +546,6 @@ Aml *aml_equal(Aml *arg1, Aml *arg2)
     Aml *var = aml_opcode(0x93 /* LequalOp */);
     aml_append(var, arg1);
     aml_append(var, arg2);
-    build_append_byte(var->buf, 0x00); /* NullNameOp */
     return var;
 }
 
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 02/28] acpi: add aml_or() term

[Marcel Apfelbaum]

Add encoding for ACPI DefOr Opcode.

Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 10 ++++++++++
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index ace180b..db8d346 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -452,6 +452,16 @@ Aml *aml_and(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefOr */
+Aml *aml_or(Aml *arg1, Aml *arg2)
+{
+    Aml *var = aml_opcode(0x7D /* OrOp */);
+    aml_append(var, arg1);
+    aml_append(var, arg2);
+    build_append_int(var->buf, 0x00); /* NullNameOp */
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefNotify */
 Aml *aml_notify(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 17d3beb..c0eb691 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -137,6 +137,7 @@ Aml *aml_int(const uint64_t val);
 Aml *aml_arg(int pos);
 Aml *aml_store(Aml *val, Aml *target);
 Aml *aml_and(Aml *arg1, Aml *arg2);
+Aml *aml_or(Aml *arg1, Aml *arg2);
 Aml *aml_notify(Aml *arg1, Aml *arg2);
 Aml *aml_call1(const char *method, Aml *arg1);
 Aml *aml_call2(const char *method, Aml *arg1, Aml *arg2);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 03/28] acpi: add aml_add() term

[Marcel Apfelbaum]

Add encoding for ACPI DefAdd Opcode.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 10 ++++++++++
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index db8d346..967798b 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -462,6 +462,16 @@ Aml *aml_or(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefAdd */
+Aml *aml_add(Aml *arg1, Aml *arg2)
+{
+    Aml *var = aml_opcode(0x72 /* AddOp */);
+    aml_append(var, arg1);
+    aml_append(var, arg2);
+    build_append_int(var->buf, 0x00 /* NullNameOp */);
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefNotify */
 Aml *aml_notify(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index c0eb691..d7a3d0f 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -138,6 +138,7 @@ Aml *aml_arg(int pos);
 Aml *aml_store(Aml *val, Aml *target);
 Aml *aml_and(Aml *arg1, Aml *arg2);
 Aml *aml_or(Aml *arg1, Aml *arg2);
+Aml *aml_add(Aml *arg1, Aml *arg2);
 Aml *aml_notify(Aml *arg1, Aml *arg2);
 Aml *aml_call1(const char *method, Aml *arg1);
 Aml *aml_call2(const char *method, Aml *arg1, Aml *arg2);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 04/28] acpi: add aml_lless() term

[Marcel Apfelbaum]

Add encoding for ACPI DefLLess Opcode.

Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 9 +++++++++
 include/hw/acpi/aml-build.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 967798b..be73d30 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -462,6 +462,15 @@ Aml *aml_or(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefLLess */
+Aml *aml_lless(Aml *arg1, Aml *arg2)
+{
+    Aml *var = aml_opcode(0x95 /* LLessOp */);
+    aml_append(var, arg1);
+    aml_append(var, arg2);
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefAdd */
 Aml *aml_add(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index d7a3d0f..8a79e7f 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -138,6 +138,7 @@ Aml *aml_arg(int pos);
 Aml *aml_store(Aml *val, Aml *target);
 Aml *aml_and(Aml *arg1, Aml *arg2);
 Aml *aml_or(Aml *arg1, Aml *arg2);
+Aml *aml_lless(Aml *arg1, Aml *arg2);
 Aml *aml_add(Aml *arg1, Aml *arg2);
 Aml *aml_notify(Aml *arg1, Aml *arg2);
 Aml *aml_call1(const char *method, Aml *arg1);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 05/28] acpi: add aml_index() term

[Marcel Apfelbaum]

Add encoding for ACPI DefIndex Opcode.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 10 ++++++++++
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index be73d30..aeaa8f6 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -481,6 +481,16 @@ Aml *aml_add(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefIndex */
+Aml *aml_index(Aml *arg1, Aml *idx)
+{
+    Aml *var = aml_opcode(0x88 /* IndexOp */);
+    aml_append(var, arg1);
+    aml_append(var, idx);
+    build_append_int(var->buf, 0x00 /* NullNameOp */);
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefNotify */
 Aml *aml_notify(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 8a79e7f..d5a2697 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -140,6 +140,7 @@ Aml *aml_and(Aml *arg1, Aml *arg2);
 Aml *aml_or(Aml *arg1, Aml *arg2);
 Aml *aml_lless(Aml *arg1, Aml *arg2);
 Aml *aml_add(Aml *arg1, Aml *arg2);
+Aml *aml_index(Aml *arg1, Aml *idx);
 Aml *aml_notify(Aml *arg1, Aml *arg2);
 Aml *aml_call1(const char *method, Aml *arg1);
 Aml *aml_call2(const char *method, Aml *arg1, Aml *arg2);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 06/28] acpi: add aml_shiftleft() term

[Marcel Apfelbaum]

Add encoding for ACPI DefShiftLeft Opcode.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 10 ++++++++++
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index aeaa8f6..f945ea9 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -462,6 +462,16 @@ Aml *aml_or(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefShiftLeft */
+Aml *aml_shiftleft(Aml *arg1, Aml *count)
+{
+    Aml *var = aml_opcode(0x79 /* ShiftLeftOp */);
+    aml_append(var, arg1);
+    aml_append(var, count);
+    build_append_int(var->buf, 0x00); /* NullNameOp */
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefLLess */
 Aml *aml_lless(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index d5a2697..26eff03 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -138,6 +138,7 @@ Aml *aml_arg(int pos);
 Aml *aml_store(Aml *val, Aml *target);
 Aml *aml_and(Aml *arg1, Aml *arg2);
 Aml *aml_or(Aml *arg1, Aml *arg2);
+Aml *aml_shiftleft(Aml *arg1, Aml *count);
 Aml *aml_lless(Aml *arg1, Aml *arg2);
 Aml *aml_add(Aml *arg1, Aml *arg2);
 Aml *aml_index(Aml *arg1, Aml *idx);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 07/28] acpi: add aml_shiftright() term

[Marcel Apfelbaum]

Add encoding for ACPI DefShiftRight Opcode.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 10 ++++++++++
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index f945ea9..58c3c38 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -472,6 +472,16 @@ Aml *aml_shiftleft(Aml *arg1, Aml *count)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefShiftRight */
+Aml *aml_shiftright(Aml *arg1, Aml *count)
+{
+    Aml *var = aml_opcode(0x7A /* ShiftRightOp */);
+    aml_append(var, arg1);
+    aml_append(var, count);
+    build_append_int(var->buf, 0x00); /* NullNameOp */
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefLLess */
 Aml *aml_lless(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 26eff03..ead0cc9 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -139,6 +139,7 @@ Aml *aml_store(Aml *val, Aml *target);
 Aml *aml_and(Aml *arg1, Aml *arg2);
 Aml *aml_or(Aml *arg1, Aml *arg2);
 Aml *aml_shiftleft(Aml *arg1, Aml *count);
+Aml *aml_shiftright(Aml *arg1, Aml *count);
 Aml *aml_lless(Aml *arg1, Aml *arg2);
 Aml *aml_add(Aml *arg1, Aml *arg2);
 Aml *aml_index(Aml *arg1, Aml *idx);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 08/28] acpi: add aml_increment() term

[Marcel Apfelbaum]

Add encoding for ACPI DefIncrement Opcode.

Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 8 ++++++++
 include/hw/acpi/aml-build.h | 1 +
 2 files changed, 9 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 58c3c38..b056165 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -501,6 +501,14 @@ Aml *aml_add(Aml *arg1, Aml *arg2)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefIncrement */
+Aml *aml_increment(Aml *arg)
+{
+    Aml *var = aml_opcode(0x75 /* IncrementOp */);
+    aml_append(var, arg);
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefIndex */
 Aml *aml_index(Aml *arg1, Aml *idx)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index ead0cc9..6c68125 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -142,6 +142,7 @@ Aml *aml_shiftleft(Aml *arg1, Aml *count);
 Aml *aml_shiftright(Aml *arg1, Aml *count);
 Aml *aml_lless(Aml *arg1, Aml *arg2);
 Aml *aml_add(Aml *arg1, Aml *arg2);
+Aml *aml_increment(Aml *arg);
 Aml *aml_index(Aml *arg1, Aml *idx);
 Aml *aml_notify(Aml *arg1, Aml *arg2);
 Aml *aml_call1(const char *method, Aml *arg1);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 09/28] acpi: add aml_while() term

[Marcel Apfelbaum]

Add encoding for ACPI DefWhile Opcode.

Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/acpi/aml-build.c         | 8 ++++++++
 include/hw/acpi/aml-build.h | 1 +
 2 files changed, 9 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index b056165..6f7a5ff 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -624,6 +624,14 @@ Aml *aml_if(Aml *predicate)
     return var;
 }
 
+/* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefWhile */
+Aml *aml_while(Aml *predicate)
+{
+    Aml *var = aml_bundle(0xA2 /* WhileOp */, AML_PACKAGE);
+    aml_append(var, predicate);
+    return var;
+}
+
 /* ACPI 1.0b: 16.2.5.2 Named Objects Encoding: DefMethod */
 Aml *aml_method(const char *name, int arg_count)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 6c68125..7bb72c0 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -189,6 +189,7 @@ Aml *aml_scope(const char *name_format, ...) GCC_FMT_ATTR(1, 2);
 Aml *aml_device(const char *name_format, ...) GCC_FMT_ATTR(1, 2);
 Aml *aml_method(const char *name, int arg_count);
 Aml *aml_if(Aml *predicate);
+Aml *aml_while(Aml *predicate);
 Aml *aml_package(uint8_t num_elements);
 Aml *aml_buffer(void);
 Aml *aml_resource_template(void);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 10/28] hw/acpi: add support for multiple root busses

[Marcel Apfelbaum]

If the machine has several root busses, we need to add them to
acpi in order to be properly detected by guests.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index d0a5c85..3e2701b 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -60,6 +60,8 @@
 #include "qom/qom-qobject.h"
 #include "exec/ram_addr.h"
 
+#include "qmp-commands.h"
+
 /* These are used to size the ACPI tables for -M pc-i440fx-1.7 and
  * -M pc-i440fx-2.0.  Even if the actual amount of AML generated grows
  * a little bit, there should be plenty of free space since the DSDT
@@ -670,6 +672,36 @@ build_ssdt(GArray *table_data, GArray *linker,
     /* Reserve space for header */
     acpi_data_push(ssdt->buf, sizeof(AcpiTableHeader));
 
+    {
+        PciInfoList *info_list, *info;
+        Error *err = NULL;
+
+        info_list = qmp_query_pci(&err);
+        if (err) {
+            error_free(err);
+            return;
+        }
+
+        for (info = info_list; info; info = info->next) {
+            PciInfo *bus_info = info->value;
+
+            if (bus_info->bus == 0) {
+                continue;
+            }
+
+            scope = aml_scope("\\_SB");
+            dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
+            aml_append(dev, aml_name_decl("_UID",
+                aml_string("PC%.02X", (uint8_t)bus_info->bus)));
+            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
+            aml_append(dev,
+                aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
+            aml_append(scope, dev);
+            aml_append(ssdt, scope);
+        }
+        qapi_free_PciInfoList(info_list);
+    }
+
     scope = aml_scope("\\_SB.PCI0");
     /* build PCI0._CRS */
     crs = aml_resource_template();
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 11/28] hw/apci: add _PRT method for extra PCI root busses

[Marcel Apfelbaum]

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 3e2701b..513fd6b 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -652,6 +652,70 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
     aml_append(parent_scope, method);
 }
 
+static Aml *build_link(Aml *lnk, Aml *lnk_pkg, const char *link_name, int idx)
+{
+    Aml *if_ctx, *pkg;
+
+    if_ctx = aml_if(aml_equal(lnk, aml_int(idx)));
+    pkg = aml_package(4);
+    aml_append(pkg, aml_int(0));
+    aml_append(pkg, aml_int(0));
+    aml_append(pkg, aml_name(link_name, ""));
+    aml_append(pkg, aml_int(0));
+    aml_append(if_ctx, aml_store(pkg, lnk_pkg));
+
+    return if_ctx;
+}
+
+static Aml *build_prt(void)
+{
+    Aml *method, *while_ctx, *pin, *slot, *lnk, *pkg, *res;
+
+    method = aml_method("_PRT", 0);
+    res = aml_local(0);
+    pin = aml_local(1);
+    slot = aml_local(2);
+    lnk = aml_local(3);
+    pkg = aml_local(4);
+
+    aml_append(method, aml_store(aml_package(128), res));
+    aml_append(method, aml_store(aml_int(0), pin));
+
+    /* while (pin < 128) */
+    while_ctx = aml_while(aml_lless(pin, aml_int(128)));
+    {
+        /* slot = pin >> 2 */
+        aml_append(while_ctx,
+                   aml_store(aml_shiftright(pin, aml_int(2)), slot));
+        /* lnk = (slot + pin) & 3 */
+        aml_append(while_ctx,
+                   aml_store(aml_and(aml_add(pin, slot), aml_int(3)), lnk));
+
+        /* pkg[2] = "LNK[A|B|C|D]", selection based on lnk % 3  */
+        aml_append(while_ctx, build_link(lnk, pkg, "LNKD", 0));
+        aml_append(while_ctx, build_link(lnk, pkg, "LNKA", 1));
+        aml_append(while_ctx, build_link(lnk, pkg, "LNKB", 2));
+        aml_append(while_ctx, build_link(lnk, pkg, "LNKC", 3));
+
+        /* pkg[0] = 0x[slot]FFFF */
+        aml_append(while_ctx,
+            aml_store(aml_or(aml_shiftleft(slot, aml_int(16)), aml_int(0xFFFF)),
+                      aml_index(pkg, aml_int(0))));
+        /* pkg[1] = pin & 3 */
+        aml_append(while_ctx,
+            aml_store(aml_and(pin, aml_int(3)), aml_index(pkg, aml_int(1))));
+        /* res[pin] = pkg */
+        aml_append(while_ctx, aml_store(pkg, aml_index(res, pin)));
+        /* pin++ */
+        aml_append(while_ctx, aml_increment(pin));
+    }
+    aml_append(method, while_ctx);
+    /* return res*/
+    aml_append(method, aml_return(res));
+
+    return method;
+}
+
 static void
 build_ssdt(GArray *table_data, GArray *linker,
            AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
@@ -696,6 +760,7 @@ build_ssdt(GArray *table_data, GArray *linker,
             aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
             aml_append(dev,
                 aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
+            aml_append(dev, build_prt());
             aml_append(scope, dev);
             aml_append(ssdt, scope);
         }
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 12/28] hw/acpi: add _CRS method for extra root busses

[Marcel Apfelbaum]

Save the IO/mem/bus numbers ranges assigned to the extra root busses
to be removed from the root bus 0 range.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 152 insertions(+)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 513fd6b..5a00f14 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -716,6 +716,151 @@ static Aml *build_prt(void)
     return method;
 }
 
+typedef struct CrsRangeEntry {
+    QLIST_ENTRY(CrsRangeEntry) entry;
+    uint64_t base;
+    uint64_t limit;
+} CrsRangeEntry;
+
+typedef QLIST_HEAD(CrsRangeQ, CrsRangeEntry) CrsRangeQ;
+
+static void crs_range_insert(CrsRangeQ *list, uint64_t base, uint64_t limit)
+{
+    CrsRangeEntry *entry, *next, *e;
+
+    if (!base) {
+        return;
+    }
+
+    e = g_malloc(sizeof(*entry));
+    e->base = base;
+    e->limit = limit;
+
+    if (QLIST_EMPTY(list)) {
+        QLIST_INSERT_HEAD(list, e, entry);
+    } else {
+        QLIST_FOREACH_SAFE(entry, list, entry, next) {
+            if (base < entry->base) {
+                QLIST_INSERT_BEFORE(entry, e, entry);
+                break;
+            } else if (!next) {
+                QLIST_INSERT_AFTER(entry, e, entry);
+                break;
+            }
+        }
+    }
+}
+
+static void crs_range_list_free(CrsRangeQ *list)
+{
+    CrsRangeEntry *entry, *next;
+
+    QLIST_FOREACH_SAFE(entry, list, entry, next) {
+        QLIST_REMOVE(entry, entry);
+        g_free(entry);
+    }
+}
+
+static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
+                      CrsRangeQ *io_ranges, CrsRangeQ *mem_ranges)
+{
+    PciDeviceInfoList *dev_list;
+    uint64_t range_base, range_limit;
+    uint8_t max_bus;
+    Aml *crs;
+
+    crs = aml_resource_template();
+    max_bus = bus_info->bus;
+
+    for (dev_list = bus_info->devices; dev_list; dev_list = dev_list->next) {
+        PciMemoryRegionList *region;
+
+        for (region = dev_list->value->regions; region; region = region->next) {
+            range_base = region->value->address;
+            range_limit = region->value->address + region->value->size - 1;
+
+            if (!strcmp(region->value->type, "io")) {
+                aml_append(crs,
+                    aml_word_io(aml_min_fixed, aml_max_fixed,
+                                aml_pos_decode, aml_entire_range,
+                                0,
+                                range_base,
+                                range_limit,
+                                0,
+                                range_limit - range_base + 1));
+                crs_range_insert(io_ranges, range_base, range_limit);
+            } else { /* "memory" */
+                aml_append(crs,
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
+                                     aml_max_fixed, aml_non_cacheable,
+                                     aml_ReadWrite,
+                                     0,
+                                     range_base,
+                                     range_limit,
+                                     0,
+                                     range_limit - range_base + 1));
+                crs_range_insert(mem_ranges, range_base, range_limit);
+            }
+        }
+
+        if (dev_list->value->has_pci_bridge) {
+            PciBridgeInfo *bridge_info = dev_list->value->pci_bridge;
+
+            if (bridge_info->bus.subordinate > max_bus) {
+                max_bus = bridge_info->bus.subordinate;
+            }
+
+            range_base = bridge_info->bus.io_range->base;
+            range_limit = bridge_info->bus.io_range->limit;
+            aml_append(crs,
+                aml_word_io(aml_min_fixed, aml_max_fixed,
+                            aml_pos_decode, aml_entire_range,
+                            0,
+                            range_base,
+                            range_limit,
+                            0,
+                            range_limit - range_base + 1));
+            crs_range_insert(io_ranges, range_base, range_limit);
+
+            range_base = bridge_info->bus.memory_range->base;
+            range_limit = bridge_info->bus.memory_range->limit;
+             aml_append(crs,
+                aml_dword_memory(aml_pos_decode, aml_min_fixed,
+                                 aml_max_fixed, aml_non_cacheable,
+                                 aml_ReadWrite,
+                                 0,
+                                 range_base,
+                                 range_limit,
+                                 0,
+                                 range_limit - range_base + 1));
+            crs_range_insert(mem_ranges, range_base, range_limit);
+
+            range_base = bridge_info->bus.prefetchable_range->base;
+            range_limit = bridge_info->bus.prefetchable_range->limit;
+            aml_append(crs,
+                aml_dword_memory(aml_pos_decode, aml_min_fixed,
+                                 aml_max_fixed, aml_non_cacheable,
+                                 aml_ReadWrite,
+                                 0,
+                                 range_base,
+                                 range_limit,
+                                 0,
+                                 range_limit - range_base + 1));
+            crs_range_insert(mem_ranges, range_base, range_limit);
+        }
+    }
+
+    aml_append(crs,
+        aml_word_bus_number(aml_min_fixed, aml_max_fixed, aml_pos_decode,
+                            0,
+                            bus_info->bus,
+                            max_bus,
+                            0,
+                            max_bus - bus_info->bus + 1));
+
+    return crs;
+}
+
 static void
 build_ssdt(GArray *table_data, GArray *linker,
            AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
@@ -725,6 +870,8 @@ build_ssdt(GArray *table_data, GArray *linker,
     uint32_t nr_mem = machine->ram_slots;
     unsigned acpi_cpus = guest_info->apic_id_limit;
     Aml *ssdt, *sb_scope, *scope, *pkg, *dev, *method, *crs, *field, *ifctx;
+    CrsRangeQ io_ranges = QLIST_HEAD_INITIALIZER(io_ranges);
+    CrsRangeQ mem_ranges = QLIST_HEAD_INITIALIZER(mem_ranges);
     int i;
 
     ssdt = init_aml_allocator();
@@ -761,9 +908,14 @@ build_ssdt(GArray *table_data, GArray *linker,
             aml_append(dev,
                 aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
             aml_append(dev, build_prt());
+            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
+            aml_append(dev, aml_name_decl("_CRS", crs));
             aml_append(scope, dev);
             aml_append(ssdt, scope);
         }
+
+        crs_range_list_free(&io_ranges);
+        crs_range_list_free(&mem_ranges);
         qapi_free_PciInfoList(info_list);
     }
 
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 13/28] hw/acpi: remove from root bus 0 the crs resources used by other busses.

[Marcel Apfelbaum]

If multiple root busses are used, root bus 0 cannot use all the
pci holes ranges. Remove the IO/mem ranges used by the other
primary busses.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c | 84 ++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 72 insertions(+), 12 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 5a00f14..28d7a43 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -872,6 +872,9 @@ build_ssdt(GArray *table_data, GArray *linker,
     Aml *ssdt, *sb_scope, *scope, *pkg, *dev, *method, *crs, *field, *ifctx;
     CrsRangeQ io_ranges = QLIST_HEAD_INITIALIZER(io_ranges);
     CrsRangeQ mem_ranges = QLIST_HEAD_INITIALIZER(mem_ranges);
+    uint64_t range_base, range_limit;
+    CrsRangeEntry *entry;
+    int root_bus_limit = 0xFF;
     int i;
 
     ssdt = init_aml_allocator();
@@ -900,6 +903,10 @@ build_ssdt(GArray *table_data, GArray *linker,
                 continue;
             }
 
+            if (bus_info->bus < root_bus_limit) {
+                root_bus_limit = bus_info->bus - 1;
+            }
+
             scope = aml_scope("\\_SB");
             dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
             aml_append(dev, aml_name_decl("_UID",
@@ -914,8 +921,6 @@ build_ssdt(GArray *table_data, GArray *linker,
             aml_append(ssdt, scope);
         }
 
-        crs_range_list_free(&io_ranges);
-        crs_range_list_free(&mem_ranges);
         qapi_free_PciInfoList(info_list);
     }
 
@@ -924,26 +929,78 @@ build_ssdt(GArray *table_data, GArray *linker,
     crs = aml_resource_template();
     aml_append(crs,
         aml_word_bus_number(aml_min_fixed, aml_max_fixed, aml_pos_decode,
-                            0x0000, 0x0000, 0x00FF, 0x0000, 0x0100));
+                            0x0000, 0x0, root_bus_limit,
+                            0x0000, root_bus_limit + 1));
     aml_append(crs, aml_io(aml_decode16, 0x0CF8, 0x0CF8, 0x01, 0x08));
 
     aml_append(crs,
         aml_word_io(aml_min_fixed, aml_max_fixed,
                     aml_pos_decode, aml_entire_range,
                     0x0000, 0x0000, 0x0CF7, 0x0000, 0x0CF8));
-    aml_append(crs,
-        aml_word_io(aml_min_fixed, aml_max_fixed,
-                    aml_pos_decode, aml_entire_range,
-                    0x0000, 0x0D00, 0xFFFF, 0x0000, 0xF300));
+
+    /* prepare PCI IO ranges */
+    range_base = 0x0D00;
+    range_limit = 0xFFFF;
+    if (QLIST_EMPTY(&io_ranges)) {
+        aml_append(crs,
+            aml_word_io(aml_min_fixed, aml_max_fixed,
+                        aml_pos_decode, aml_entire_range,
+                        0x0000, range_base, range_limit,
+                        0x0000, range_limit - range_base + 1));
+    } else {
+        QLIST_FOREACH(entry, &io_ranges, entry) {
+            if (range_base < entry->base) {
+                aml_append(crs,
+                    aml_word_io(aml_min_fixed, aml_max_fixed,
+                                aml_pos_decode, aml_entire_range,
+                                0x0000, range_base, entry->base - 1,
+                                0x0000, entry->base - range_base));
+            }
+            range_base = entry->limit + 1;
+            if (!QLIST_NEXT(entry, entry)) {
+                aml_append(crs,
+                    aml_word_io(aml_min_fixed, aml_max_fixed,
+                                aml_pos_decode, aml_entire_range,
+                                0x0000, range_base, range_limit,
+                                0x0000, range_limit - range_base + 1));
+            }
+        }
+    }
+
     aml_append(crs,
         aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
                          aml_cacheable, aml_ReadWrite,
                          0, 0x000A0000, 0x000BFFFF, 0, 0x00020000));
-    aml_append(crs,
-        aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
-                         aml_non_cacheable, aml_ReadWrite,
-                         0, pci->w32.begin, pci->w32.end - 1, 0,
-                         pci->w32.end - pci->w32.begin));
+
+    /* prepare PCI memory ranges */
+    range_base = pci->w32.begin;
+    range_limit = pci->w32.end - 1;
+    if (QLIST_EMPTY(&mem_ranges)) {
+        aml_append(crs,
+            aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
+                             aml_non_cacheable, aml_ReadWrite,
+                             0, range_base, range_limit,
+                             0, range_limit - range_base + 1));
+    } else {
+        QLIST_FOREACH(entry, &mem_ranges, entry) {
+            if (range_base < entry->base) {
+                aml_append(crs,
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
+                                     aml_non_cacheable, aml_ReadWrite,
+                                     0, range_base, entry->base - 1,
+                                     0, entry->base - range_base));
+            }
+            range_base = entry->limit + 1;
+            if (!QLIST_NEXT(entry, entry)) {
+                aml_append(crs,
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
+                                     aml_non_cacheable, aml_ReadWrite,
+                                     0, range_base, range_limit,
+                                     0, range_base - range_limit + 1));
+            }
+        }
+    }
+
     if (pci->w64.begin) {
         aml_append(crs,
             aml_qword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
@@ -966,6 +1023,9 @@ build_ssdt(GArray *table_data, GArray *linker,
     aml_append(dev, aml_name_decl("_CRS", crs));
     aml_append(scope, dev);
 
+    crs_range_list_free(&io_ranges);
+    crs_range_list_free(&mem_ranges);
+
     /* reserve PCIHP resources */
     if (pm->pcihp_io_len) {
         dev = aml_device("PHPR");
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 14/28] hw/pci: move pci bus related code to separate files

[Marcel Apfelbaum]

This refactoring moves all the code needed (recursively)
to register TYPE_PCI_BUS type to a new file hw/pci/pci_bus.c .
This allows to properly add new functionality to the pci bus class.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 arch_init.c              |   1 +
 hw/alpha/typhoon.c       |   1 +
 hw/mips/gt64xxx_pci.c    |   1 +
 hw/pci-host/bonito.c     |   1 +
 hw/pci-host/grackle.c    |   1 +
 hw/pci-host/piix.c       |   1 +
 hw/pci-host/ppce500.c    |   1 +
 hw/pci-host/q35.c        |   1 +
 hw/pci-host/uninorth.c   |   1 +
 hw/pci/Makefile.objs     |   2 +-
 hw/pci/pci.c             | 472 +--------------------------------------------
 hw/pci/pci_bus.c         | 491 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/ppc/ppc4xx_pci.c      |   1 +
 hw/sh4/r2d.c             |   1 +
 hw/sh4/sh_pci.c          |   1 +
 include/hw/pci/pci.h     |   3 +-
 include/hw/pci/pci_bus.h |   8 +
 17 files changed, 514 insertions(+), 474 deletions(-)
 create mode 100644 hw/pci/pci_bus.c

diff --git a/arch_init.c b/arch_init.c
index 691b5e2..cbebf6d 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -37,6 +37,7 @@
 #include "audio/audio.h"
 #include "hw/i386/pc.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/audio/audio.h"
 #include "sysemu/kvm.h"
 #include "migration/migration.h"
diff --git a/hw/alpha/typhoon.c b/hw/alpha/typhoon.c
index 62af946..d3fad5d 100644
--- a/hw/alpha/typhoon.c
+++ b/hw/alpha/typhoon.c
@@ -8,6 +8,7 @@
 
 #include "cpu.h"
 #include "hw/hw.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/devices.h"
 #include "sysemu/sysemu.h"
 #include "alpha_sys.h"
diff --git a/hw/mips/gt64xxx_pci.c b/hw/mips/gt64xxx_pci.c
index 10fcca3..c4ca6f3 100644
--- a/hw/mips/gt64xxx_pci.c
+++ b/hw/mips/gt64xxx_pci.c
@@ -25,6 +25,7 @@
 #include "hw/hw.h"
 #include "hw/mips/mips.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 #include "hw/i386/pc.h"
 #include "exec/address-spaces.h"
diff --git a/hw/pci-host/bonito.c b/hw/pci-host/bonito.c
index 8bdd569..49e1122 100644
--- a/hw/pci-host/bonito.c
+++ b/hw/pci-host/bonito.c
@@ -41,6 +41,7 @@
 
 #include "hw/hw.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/i386/pc.h"
 #include "hw/mips/mips.h"
 #include "hw/pci/pci_host.h"
diff --git a/hw/pci-host/grackle.c b/hw/pci-host/grackle.c
index bfe707a..89745cd 100644
--- a/hw/pci-host/grackle.c
+++ b/hw/pci-host/grackle.c
@@ -26,6 +26,7 @@
 #include "hw/pci/pci_host.h"
 #include "hw/ppc/mac.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 
 /* debug Grackle */
 //#define DEBUG_GRACKLE
diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
index 723836f..292b6e9 100644
--- a/hw/pci-host/piix.c
+++ b/hw/pci-host/piix.c
@@ -25,6 +25,7 @@
 #include "hw/hw.h"
 #include "hw/i386/pc.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 #include "hw/isa/isa.h"
 #include "hw/sysbus.h"
diff --git a/hw/pci-host/ppce500.c b/hw/pci-host/ppce500.c
index 613ba73..4363951 100644
--- a/hw/pci-host/ppce500.c
+++ b/hw/pci-host/ppce500.c
@@ -17,6 +17,7 @@
 #include "hw/hw.h"
 #include "hw/ppc/e500-ccsr.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 #include "qemu/bswap.h"
 #include "hw/pci-host/ppce500.h"
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index df60e61..a42d0fb 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -28,6 +28,7 @@
  * THE SOFTWARE.
  */
 #include "hw/hw.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci-host/q35.h"
 #include "qapi/visitor.h"
 
diff --git a/hw/pci-host/uninorth.c b/hw/pci-host/uninorth.c
index 53f2b59..1406b42 100644
--- a/hw/pci-host/uninorth.c
+++ b/hw/pci-host/uninorth.c
@@ -24,6 +24,7 @@
 #include "hw/hw.h"
 #include "hw/ppc/mac.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 
 /* debug UniNorth */
diff --git a/hw/pci/Makefile.objs b/hw/pci/Makefile.objs
index 9f905e6..a05cca0 100644
--- a/hw/pci/Makefile.objs
+++ b/hw/pci/Makefile.objs
@@ -1,4 +1,4 @@
-common-obj-$(CONFIG_PCI) += pci.o pci_bridge.o
+common-obj-$(CONFIG_PCI) += pci.o pci_bridge.o pci_bus.o
 common-obj-$(CONFIG_PCI) += msix.o msi.o
 common-obj-$(CONFIG_PCI) += shpc.o
 common-obj-$(CONFIG_PCI) += slotid_cap.o
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index cc5d946..442d209 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -45,11 +45,6 @@
 # define PCI_DPRINTF(format, ...)       do { } while (0)
 #endif
 
-static void pcibus_dev_print(Monitor *mon, DeviceState *dev, int indent);
-static char *pcibus_get_dev_path(DeviceState *dev);
-static char *pcibus_get_fw_dev_path(DeviceState *dev);
-static void pcibus_reset(BusState *qbus);
-
 static Property pci_props[] = {
     DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
     DEFINE_PROP_STRING("romfile", PCIDevice, romfile),
@@ -61,58 +56,11 @@ static Property pci_props[] = {
     DEFINE_PROP_END_OF_LIST()
 };
 
-static const VMStateDescription vmstate_pcibus = {
-    .name = "PCIBUS",
-    .version_id = 1,
-    .minimum_version_id = 1,
-    .fields = (VMStateField[]) {
-        VMSTATE_INT32_EQUAL(nirq, PCIBus),
-        VMSTATE_VARRAY_INT32(irq_count, PCIBus,
-                             nirq, 0, vmstate_info_int32,
-                             int32_t),
-        VMSTATE_END_OF_LIST()
-    }
-};
-
-static void pci_bus_realize(BusState *qbus, Error **errp)
-{
-    PCIBus *bus = PCI_BUS(qbus);
-
-    vmstate_register(NULL, -1, &vmstate_pcibus, bus);
-}
-
-static void pci_bus_unrealize(BusState *qbus, Error **errp)
-{
-    PCIBus *bus = PCI_BUS(qbus);
-
-    vmstate_unregister(NULL, &vmstate_pcibus, bus);
-}
-
-static void pci_bus_class_init(ObjectClass *klass, void *data)
-{
-    BusClass *k = BUS_CLASS(klass);
-
-    k->print_dev = pcibus_dev_print;
-    k->get_dev_path = pcibus_get_dev_path;
-    k->get_fw_dev_path = pcibus_get_fw_dev_path;
-    k->realize = pci_bus_realize;
-    k->unrealize = pci_bus_unrealize;
-    k->reset = pcibus_reset;
-}
-
-static const TypeInfo pci_bus_info = {
-    .name = TYPE_PCI_BUS,
-    .parent = TYPE_BUS,
-    .instance_size = sizeof(PCIBus),
-    .class_init = pci_bus_class_init,
-};
-
 static const TypeInfo pcie_bus_info = {
     .name = TYPE_PCIE_BUS,
     .parent = TYPE_PCI_BUS,
 };
 
-static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num);
 static void pci_update_mappings(PCIDevice *d);
 static void pci_irq_handler(void *opaque, int irq_num, int level);
 static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **);
@@ -185,7 +133,7 @@ void pci_device_deassert_intx(PCIDevice *dev)
     }
 }
 
-static void pci_do_device_reset(PCIDevice *dev)
+void pci_do_device_reset(PCIDevice *dev)
 {
     int r;
 
@@ -230,27 +178,6 @@ void pci_device_reset(PCIDevice *dev)
     pci_do_device_reset(dev);
 }
 
-/*
- * Trigger pci bus reset under a given bus.
- * Called via qbus_reset_all on RST# assert, after the devices
- * have been reset qdev_reset_all-ed already.
- */
-static void pcibus_reset(BusState *qbus)
-{
-    PCIBus *bus = DO_UPCAST(PCIBus, qbus, qbus);
-    int i;
-
-    for (i = 0; i < ARRAY_SIZE(bus->devices); ++i) {
-        if (bus->devices[i]) {
-            pci_do_device_reset(bus->devices[i]);
-        }
-    }
-
-    for (i = 0; i < bus->nirq; i++) {
-        assert(bus->irq_count[i] == 0);
-    }
-}
-
 static void pci_host_bus_register(PCIBus *bus, DeviceState *parent)
 {
     PCIHostState *host_bridge = PCI_HOST_BRIDGE(parent);
@@ -1301,74 +1228,6 @@ int pci_swizzle_map_irq_fn(PCIDevice *pci_dev, int pin)
     return (pin + PCI_SLOT(pci_dev->devfn)) % PCI_NUM_PINS;
 }
 
-/***********************************************************/
-/* monitor info on PCI */
-
-typedef struct {
-    uint16_t class;
-    const char *desc;
-    const char *fw_name;
-    uint16_t fw_ign_bits;
-} pci_class_desc;
-
-static const pci_class_desc pci_class_descriptions[] =
-{
-    { 0x0001, "VGA controller", "display"},
-    { 0x0100, "SCSI controller", "scsi"},
-    { 0x0101, "IDE controller", "ide"},
-    { 0x0102, "Floppy controller", "fdc"},
-    { 0x0103, "IPI controller", "ipi"},
-    { 0x0104, "RAID controller", "raid"},
-    { 0x0106, "SATA controller"},
-    { 0x0107, "SAS controller"},
-    { 0x0180, "Storage controller"},
-    { 0x0200, "Ethernet controller", "ethernet"},
-    { 0x0201, "Token Ring controller", "token-ring"},
-    { 0x0202, "FDDI controller", "fddi"},
-    { 0x0203, "ATM controller", "atm"},
-    { 0x0280, "Network controller"},
-    { 0x0300, "VGA controller", "display", 0x00ff},
-    { 0x0301, "XGA controller"},
-    { 0x0302, "3D controller"},
-    { 0x0380, "Display controller"},
-    { 0x0400, "Video controller", "video"},
-    { 0x0401, "Audio controller", "sound"},
-    { 0x0402, "Phone"},
-    { 0x0403, "Audio controller", "sound"},
-    { 0x0480, "Multimedia controller"},
-    { 0x0500, "RAM controller", "memory"},
-    { 0x0501, "Flash controller", "flash"},
-    { 0x0580, "Memory controller"},
-    { 0x0600, "Host bridge", "host"},
-    { 0x0601, "ISA bridge", "isa"},
-    { 0x0602, "EISA bridge", "eisa"},
-    { 0x0603, "MC bridge", "mca"},
-    { 0x0604, "PCI bridge", "pci-bridge"},
-    { 0x0605, "PCMCIA bridge", "pcmcia"},
-    { 0x0606, "NUBUS bridge", "nubus"},
-    { 0x0607, "CARDBUS bridge", "cardbus"},
-    { 0x0608, "RACEWAY bridge"},
-    { 0x0680, "Bridge"},
-    { 0x0700, "Serial port", "serial"},
-    { 0x0701, "Parallel port", "parallel"},
-    { 0x0800, "Interrupt controller", "interrupt-controller"},
-    { 0x0801, "DMA controller", "dma-controller"},
-    { 0x0802, "Timer", "timer"},
-    { 0x0803, "RTC", "rtc"},
-    { 0x0900, "Keyboard", "keyboard"},
-    { 0x0901, "Pen", "pen"},
-    { 0x0902, "Mouse", "mouse"},
-    { 0x0A00, "Dock station", "dock", 0x00ff},
-    { 0x0B00, "i386 cpu", "cpu", 0x00ff},
-    { 0x0c00, "Fireware contorller", "fireware"},
-    { 0x0c01, "Access bus controller", "access-bus"},
-    { 0x0c02, "SSA controller", "ssa"},
-    { 0x0c03, "USB controller", "usb"},
-    { 0x0c04, "Fibre channel controller", "fibre-channel"},
-    { 0x0c05, "SMBus"},
-    { 0, NULL}
-};
-
 static void pci_for_each_device_under_bus(PCIBus *bus,
                                           void (*fn)(PCIBus *b, PCIDevice *d,
                                                      void *opaque),
@@ -1396,161 +1255,6 @@ void pci_for_each_device(PCIBus *bus, int bus_num,
     }
 }
 
-static const pci_class_desc *get_class_desc(int class)
-{
-    const pci_class_desc *desc;
-
-    desc = pci_class_descriptions;
-    while (desc->desc && class != desc->class) {
-        desc++;
-    }
-
-    return desc;
-}
-
-static PciDeviceInfoList *qmp_query_pci_devices(PCIBus *bus, int bus_num);
-
-static PciMemoryRegionList *qmp_query_pci_regions(const PCIDevice *dev)
-{
-    PciMemoryRegionList *head = NULL, *cur_item = NULL;
-    int i;
-
-    for (i = 0; i < PCI_NUM_REGIONS; i++) {
-        const PCIIORegion *r = &dev->io_regions[i];
-        PciMemoryRegionList *region;
-
-        if (!r->size) {
-            continue;
-        }
-
-        region = g_malloc0(sizeof(*region));
-        region->value = g_malloc0(sizeof(*region->value));
-
-        if (r->type & PCI_BASE_ADDRESS_SPACE_IO) {
-            region->value->type = g_strdup("io");
-        } else {
-            region->value->type = g_strdup("memory");
-            region->value->has_prefetch = true;
-            region->value->prefetch = !!(r->type & PCI_BASE_ADDRESS_MEM_PREFETCH);
-            region->value->has_mem_type_64 = true;
-            region->value->mem_type_64 = !!(r->type & PCI_BASE_ADDRESS_MEM_TYPE_64);
-        }
-
-        region->value->bar = i;
-        region->value->address = r->addr;
-        region->value->size = r->size;
-
-        /* XXX: waiting for the qapi to support GSList */
-        if (!cur_item) {
-            head = cur_item = region;
-        } else {
-            cur_item->next = region;
-            cur_item = region;
-        }
-    }
-
-    return head;
-}
-
-static PciBridgeInfo *qmp_query_pci_bridge(PCIDevice *dev, PCIBus *bus,
-                                           int bus_num)
-{
-    PciBridgeInfo *info;
-
-    info = g_malloc0(sizeof(*info));
-
-    info->bus.number = dev->config[PCI_PRIMARY_BUS];
-    info->bus.secondary = dev->config[PCI_SECONDARY_BUS];
-    info->bus.subordinate = dev->config[PCI_SUBORDINATE_BUS];
-
-    info->bus.io_range = g_malloc0(sizeof(*info->bus.io_range));
-    info->bus.io_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_IO);
-    info->bus.io_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
-
-    info->bus.memory_range = g_malloc0(sizeof(*info->bus.memory_range));
-    info->bus.memory_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
-    info->bus.memory_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
-
-    info->bus.prefetchable_range = g_malloc0(sizeof(*info->bus.prefetchable_range));
-    info->bus.prefetchable_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
-    info->bus.prefetchable_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
-
-    if (dev->config[PCI_SECONDARY_BUS] != 0) {
-        PCIBus *child_bus = pci_find_bus_nr(bus, dev->config[PCI_SECONDARY_BUS]);
-        if (child_bus) {
-            info->has_devices = true;
-            info->devices = qmp_query_pci_devices(child_bus, dev->config[PCI_SECONDARY_BUS]);
-        }
-    }
-
-    return info;
-}
-
-static PciDeviceInfo *qmp_query_pci_device(PCIDevice *dev, PCIBus *bus,
-                                           int bus_num)
-{
-    const pci_class_desc *desc;
-    PciDeviceInfo *info;
-    uint8_t type;
-    int class;
-
-    info = g_malloc0(sizeof(*info));
-    info->bus = bus_num;
-    info->slot = PCI_SLOT(dev->devfn);
-    info->function = PCI_FUNC(dev->devfn);
-
-    class = pci_get_word(dev->config + PCI_CLASS_DEVICE);
-    info->class_info.q_class = class;
-    desc = get_class_desc(class);
-    if (desc->desc) {
-        info->class_info.has_desc = true;
-        info->class_info.desc = g_strdup(desc->desc);
-    }
-
-    info->id.vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
-    info->id.device = pci_get_word(dev->config + PCI_DEVICE_ID);
-    info->regions = qmp_query_pci_regions(dev);
-    info->qdev_id = g_strdup(dev->qdev.id ? dev->qdev.id : "");
-
-    if (dev->config[PCI_INTERRUPT_PIN] != 0) {
-        info->has_irq = true;
-        info->irq = dev->config[PCI_INTERRUPT_LINE];
-    }
-
-    type = dev->config[PCI_HEADER_TYPE] & ~PCI_HEADER_TYPE_MULTI_FUNCTION;
-    if (type == PCI_HEADER_TYPE_BRIDGE) {
-        info->has_pci_bridge = true;
-        info->pci_bridge = qmp_query_pci_bridge(dev, bus, bus_num);
-    }
-
-    return info;
-}
-
-static PciDeviceInfoList *qmp_query_pci_devices(PCIBus *bus, int bus_num)
-{
-    PciDeviceInfoList *info, *head = NULL, *cur_item = NULL;
-    PCIDevice *dev;
-    int devfn;
-
-    for (devfn = 0; devfn < ARRAY_SIZE(bus->devices); devfn++) {
-        dev = bus->devices[devfn];
-        if (dev) {
-            info = g_malloc0(sizeof(*info));
-            info->value = qmp_query_pci_device(dev, bus, bus_num);
-
-            /* XXX: waiting for the qapi to support GSList */
-            if (!cur_item) {
-                head = cur_item = info;
-            } else {
-                cur_item->next = info;
-                cur_item = info;
-            }
-        }
-    }
-
-    return head;
-}
-
 static PciInfo *qmp_query_pci_bus(PCIBus *bus, int bus_num)
 {
     PciInfo *info = NULL;
@@ -1674,50 +1378,6 @@ PCIDevice *pci_vga_init(PCIBus *bus)
     }
 }
 
-/* Whether a given bus number is in range of the secondary
- * bus of the given bridge device. */
-static bool pci_secondary_bus_in_range(PCIDevice *dev, int bus_num)
-{
-    return !(pci_get_word(dev->config + PCI_BRIDGE_CONTROL) &
-             PCI_BRIDGE_CTL_BUS_RESET) /* Don't walk the bus if it's reset. */ &&
-        dev->config[PCI_SECONDARY_BUS] < bus_num &&
-        bus_num <= dev->config[PCI_SUBORDINATE_BUS];
-}
-
-static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num)
-{
-    PCIBus *sec;
-
-    if (!bus) {
-        return NULL;
-    }
-
-    if (pci_bus_num(bus) == bus_num) {
-        return bus;
-    }
-
-    /* Consider all bus numbers in range for the host pci bridge. */
-    if (!pci_bus_is_root(bus) &&
-        !pci_secondary_bus_in_range(bus->parent_dev, bus_num)) {
-        return NULL;
-    }
-
-    /* try child bus */
-    for (; bus; bus = sec) {
-        QLIST_FOREACH(sec, &bus->child, sibling) {
-            assert(!pci_bus_is_root(sec));
-            if (sec->parent_dev->config[PCI_SECONDARY_BUS] == bus_num) {
-                return sec;
-            }
-            if (pci_secondary_bus_in_range(sec->parent_dev, bus_num)) {
-                break;
-            }
-        }
-    }
-
-    return NULL;
-}
-
 void pci_for_each_bus_depth_first(PCIBus *bus,
                                   void *(*begin)(PCIBus *bus, void *parent_state),
                                   void (*end)(PCIBus *bus, void *state),
@@ -2130,135 +1790,6 @@ uint8_t pci_find_capability(PCIDevice *pdev, uint8_t cap_id)
     return pci_find_capability_list(pdev, cap_id, NULL);
 }
 
-static void pcibus_dev_print(Monitor *mon, DeviceState *dev, int indent)
-{
-    PCIDevice *d = (PCIDevice *)dev;
-    const pci_class_desc *desc;
-    char ctxt[64];
-    PCIIORegion *r;
-    int i, class;
-
-    class = pci_get_word(d->config + PCI_CLASS_DEVICE);
-    desc = pci_class_descriptions;
-    while (desc->desc && class != desc->class)
-        desc++;
-    if (desc->desc) {
-        snprintf(ctxt, sizeof(ctxt), "%s", desc->desc);
-    } else {
-        snprintf(ctxt, sizeof(ctxt), "Class %04x", class);
-    }
-
-    monitor_printf(mon, "%*sclass %s, addr %02x:%02x.%x, "
-                   "pci id %04x:%04x (sub %04x:%04x)\n",
-                   indent, "", ctxt, pci_bus_num(d->bus),
-                   PCI_SLOT(d->devfn), PCI_FUNC(d->devfn),
-                   pci_get_word(d->config + PCI_VENDOR_ID),
-                   pci_get_word(d->config + PCI_DEVICE_ID),
-                   pci_get_word(d->config + PCI_SUBSYSTEM_VENDOR_ID),
-                   pci_get_word(d->config + PCI_SUBSYSTEM_ID));
-    for (i = 0; i < PCI_NUM_REGIONS; i++) {
-        r = &d->io_regions[i];
-        if (!r->size)
-            continue;
-        monitor_printf(mon, "%*sbar %d: %s at 0x%"FMT_PCIBUS
-                       " [0x%"FMT_PCIBUS"]\n",
-                       indent, "",
-                       i, r->type & PCI_BASE_ADDRESS_SPACE_IO ? "i/o" : "mem",
-                       r->addr, r->addr + r->size - 1);
-    }
-}
-
-static char *pci_dev_fw_name(DeviceState *dev, char *buf, int len)
-{
-    PCIDevice *d = (PCIDevice *)dev;
-    const char *name = NULL;
-    const pci_class_desc *desc =  pci_class_descriptions;
-    int class = pci_get_word(d->config + PCI_CLASS_DEVICE);
-
-    while (desc->desc &&
-          (class & ~desc->fw_ign_bits) !=
-          (desc->class & ~desc->fw_ign_bits)) {
-        desc++;
-    }
-
-    if (desc->desc) {
-        name = desc->fw_name;
-    }
-
-    if (name) {
-        pstrcpy(buf, len, name);
-    } else {
-        snprintf(buf, len, "pci%04x,%04x",
-                 pci_get_word(d->config + PCI_VENDOR_ID),
-                 pci_get_word(d->config + PCI_DEVICE_ID));
-    }
-
-    return buf;
-}
-
-static char *pcibus_get_fw_dev_path(DeviceState *dev)
-{
-    PCIDevice *d = (PCIDevice *)dev;
-    char path[50], name[33];
-    int off;
-
-    off = snprintf(path, sizeof(path), "%s@%x",
-                   pci_dev_fw_name(dev, name, sizeof name),
-                   PCI_SLOT(d->devfn));
-    if (PCI_FUNC(d->devfn))
-        snprintf(path + off, sizeof(path) + off, ",%x", PCI_FUNC(d->devfn));
-    return g_strdup(path);
-}
-
-static char *pcibus_get_dev_path(DeviceState *dev)
-{
-    PCIDevice *d = container_of(dev, PCIDevice, qdev);
-    PCIDevice *t;
-    int slot_depth;
-    /* Path format: Domain:00:Slot.Function:Slot.Function....:Slot.Function.
-     * 00 is added here to make this format compatible with
-     * domain:Bus:Slot.Func for systems without nested PCI bridges.
-     * Slot.Function list specifies the slot and function numbers for all
-     * devices on the path from root to the specific device. */
-    const char *root_bus_path;
-    int root_bus_len;
-    char slot[] = ":SS.F";
-    int slot_len = sizeof slot - 1 /* For '\0' */;
-    int path_len;
-    char *path, *p;
-    int s;
-
-    root_bus_path = pci_root_bus_path(d);
-    root_bus_len = strlen(root_bus_path);
-
-    /* Calculate # of slots on path between device and root. */;
-    slot_depth = 0;
-    for (t = d; t; t = t->bus->parent_dev) {
-        ++slot_depth;
-    }
-
-    path_len = root_bus_len + slot_len * slot_depth;
-
-    /* Allocate memory, fill in the terminating null byte. */
-    path = g_malloc(path_len + 1 /* For '\0' */);
-    path[path_len] = '\0';
-
-    memcpy(path, root_bus_path, root_bus_len);
-
-    /* Fill in slot numbers. We walk up from device to root, so need to print
-     * them in the reverse order, last to first. */
-    p = path + path_len;
-    for (t = d; t; t = t->bus->parent_dev) {
-        p -= slot_len;
-        s = snprintf(slot, sizeof slot, ":%02x.%x",
-                     PCI_SLOT(t->devfn), PCI_FUNC(t->devfn));
-        assert(s == slot_len);
-        memcpy(p, slot, slot_len);
-    }
-
-    return path;
-}
-
 static int pci_qdev_find_recursive(PCIBus *bus,
                                    const char *id, PCIDevice **pdev)
 {
@@ -2404,7 +1935,6 @@ static const TypeInfo pci_device_type_info = {
 
 static void pci_register_types(void)
 {
-    type_register_static(&pci_bus_info);
     type_register_static(&pcie_bus_info);
     type_register_static(&pci_device_type_info);
 }
diff --git a/hw/pci/pci_bus.c b/hw/pci/pci_bus.c
new file mode 100644
index 0000000..d156194
--- /dev/null
+++ b/hw/pci/pci_bus.c
@@ -0,0 +1,491 @@
+/*
+ * PCI Bus
+ *
+ * Copyright (C) 2014 Red Hat Inc
+ *
+ * Authors:
+ *   Marcel Apfelbaum <marcel.a@redhat.com> (split out from pci.c)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
+#include "hw/pci/pci_bridge.h"
+#include "monitor/monitor.h"
+
+typedef struct {
+    uint16_t class;
+    const char *desc;
+    const char *fw_name;
+    uint16_t fw_ign_bits;
+} pci_class_desc;
+
+static const pci_class_desc pci_class_descriptions[] = {
+    { 0x0001, "VGA controller", "display"},
+    { 0x0100, "SCSI controller", "scsi"},
+    { 0x0101, "IDE controller", "ide"},
+    { 0x0102, "Floppy controller", "fdc"},
+    { 0x0103, "IPI controller", "ipi"},
+    { 0x0104, "RAID controller", "raid"},
+    { 0x0106, "SATA controller"},
+    { 0x0107, "SAS controller"},
+    { 0x0180, "Storage controller"},
+    { 0x0200, "Ethernet controller", "ethernet"},
+    { 0x0201, "Token Ring controller", "token-ring"},
+    { 0x0202, "FDDI controller", "fddi"},
+    { 0x0203, "ATM controller", "atm"},
+    { 0x0280, "Network controller"},
+    { 0x0300, "VGA controller", "display", 0x00ff},
+    { 0x0301, "XGA controller"},
+    { 0x0302, "3D controller"},
+    { 0x0380, "Display controller"},
+    { 0x0400, "Video controller", "video"},
+    { 0x0401, "Audio controller", "sound"},
+    { 0x0402, "Phone"},
+    { 0x0403, "Audio controller", "sound"},
+    { 0x0480, "Multimedia controller"},
+    { 0x0500, "RAM controller", "memory"},
+    { 0x0501, "Flash controller", "flash"},
+    { 0x0580, "Memory controller"},
+    { 0x0600, "Host bridge", "host"},
+    { 0x0601, "ISA bridge", "isa"},
+    { 0x0602, "EISA bridge", "eisa"},
+    { 0x0603, "MC bridge", "mca"},
+    { 0x0604, "PCI bridge", "pci-bridge"},
+    { 0x0605, "PCMCIA bridge", "pcmcia"},
+    { 0x0606, "NUBUS bridge", "nubus"},
+    { 0x0607, "CARDBUS bridge", "cardbus"},
+    { 0x0608, "RACEWAY bridge"},
+    { 0x0680, "Bridge"},
+    { 0x0700, "Serial port", "serial"},
+    { 0x0701, "Parallel port", "parallel"},
+    { 0x0800, "Interrupt controller", "interrupt-controller"},
+    { 0x0801, "DMA controller", "dma-controller"},
+    { 0x0802, "Timer", "timer"},
+    { 0x0803, "RTC", "rtc"},
+    { 0x0900, "Keyboard", "keyboard"},
+    { 0x0901, "Pen", "pen"},
+    { 0x0902, "Mouse", "mouse"},
+    { 0x0A00, "Dock station", "dock", 0x00ff},
+    { 0x0B00, "i386 cpu", "cpu", 0x00ff},
+    { 0x0c00, "Fireware contorller", "fireware"},
+    { 0x0c01, "Access bus controller", "access-bus"},
+    { 0x0c02, "SSA controller", "ssa"},
+    { 0x0c03, "USB controller", "usb"},
+    { 0x0c04, "Fibre channel controller", "fibre-channel"},
+    { 0x0c05, "SMBus"},
+    { 0, NULL}
+};
+
+/* Whether a given bus number is in range of the secondary
+ * bus of the given bridge device. */
+static bool pci_secondary_bus_in_range(PCIDevice *dev, int bus_num)
+{
+    return !(pci_get_word(dev->config + PCI_BRIDGE_CONTROL) &
+             PCI_BRIDGE_CTL_BUS_RESET) /* Don't walk the bus if it's reset. */ &&
+        dev->config[PCI_SECONDARY_BUS] < bus_num &&
+        bus_num <= dev->config[PCI_SUBORDINATE_BUS];
+}
+
+PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num)
+{
+    PCIBus *sec;
+
+    if (!bus) {
+        return NULL;
+    }
+
+    if (pci_bus_num(bus) == bus_num) {
+        return bus;
+    }
+
+    /* Consider all bus numbers in range for the host pci bridge. */
+    if (!pci_bus_is_root(bus) &&
+        !pci_secondary_bus_in_range(bus->parent_dev, bus_num)) {
+        return NULL;
+    }
+
+    /* try child bus */
+    for (; bus; bus = sec) {
+        QLIST_FOREACH(sec, &bus->child, sibling) {
+            assert(!pci_bus_is_root(sec));
+            if (sec->parent_dev->config[PCI_SECONDARY_BUS] == bus_num) {
+                return sec;
+            }
+            if (pci_secondary_bus_in_range(sec->parent_dev, bus_num)) {
+                break;
+            }
+        }
+    }
+
+    return NULL;
+}
+
+static const pci_class_desc *get_class_desc(int class)
+{
+    const pci_class_desc *desc;
+
+    desc = pci_class_descriptions;
+    while (desc->desc && class != desc->class) {
+        desc++;
+    }
+
+    return desc;
+}
+
+static PciMemoryRegionList *qmp_query_pci_regions(const PCIDevice *dev)
+{
+    PciMemoryRegionList *head = NULL, *cur_item = NULL;
+    int i;
+
+    for (i = 0; i < PCI_NUM_REGIONS; i++) {
+        const PCIIORegion *r = &dev->io_regions[i];
+        PciMemoryRegionList *region;
+
+        if (!r->size) {
+            continue;
+        }
+
+        region = g_malloc0(sizeof(*region));
+        region->value = g_malloc0(sizeof(*region->value));
+
+        if (r->type & PCI_BASE_ADDRESS_SPACE_IO) {
+            region->value->type = g_strdup("io");
+        } else {
+            region->value->type = g_strdup("memory");
+            region->value->has_prefetch = true;
+            region->value->prefetch = !!(r->type & PCI_BASE_ADDRESS_MEM_PREFETCH);
+            region->value->has_mem_type_64 = true;
+            region->value->mem_type_64 = !!(r->type & PCI_BASE_ADDRESS_MEM_TYPE_64);
+        }
+
+        region->value->bar = i;
+        region->value->address = r->addr;
+        region->value->size = r->size;
+
+        /* XXX: waiting for the qapi to support GSList */
+        if (!cur_item) {
+            head = cur_item = region;
+        } else {
+            cur_item->next = region;
+            cur_item = region;
+        }
+    }
+
+    return head;
+}
+
+static PciBridgeInfo *qmp_query_pci_bridge(PCIDevice *dev, PCIBus *bus,
+                                           int bus_num)
+{
+    PciBridgeInfo *info;
+
+    info = g_malloc0(sizeof(*info));
+
+    info->bus.number = dev->config[PCI_PRIMARY_BUS];
+    info->bus.secondary = dev->config[PCI_SECONDARY_BUS];
+    info->bus.subordinate = dev->config[PCI_SUBORDINATE_BUS];
+
+    info->bus.io_range = g_malloc0(sizeof(*info->bus.io_range));
+    info->bus.io_range->base = pci_bridge_get_base(dev,
+                    PCI_BASE_ADDRESS_SPACE_IO);
+    info->bus.io_range->limit =
+                    pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
+
+    info->bus.memory_range = g_malloc0(sizeof(*info->bus.memory_range));
+    info->bus.memory_range->base =
+                    pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+    info->bus.memory_range->limit =
+                    pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+
+    info->bus.prefetchable_range =
+                    g_malloc0(sizeof(*info->bus.prefetchable_range));
+    info->bus.prefetchable_range->base =
+                    pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+    info->bus.prefetchable_range->limit =
+                    pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+
+    if (dev->config[PCI_SECONDARY_BUS] != 0) {
+        PCIBus *child_bus = pci_find_bus_nr(bus, dev->config[PCI_SECONDARY_BUS]);
+        if (child_bus) {
+            info->has_devices = true;
+            info->devices = qmp_query_pci_devices(child_bus,
+                                                  dev->config[PCI_SECONDARY_BUS]);
+        }
+    }
+
+    return info;
+}
+
+static PciDeviceInfo *qmp_query_pci_device(PCIDevice *dev, PCIBus *bus,
+                                           int bus_num)
+{
+    const pci_class_desc *desc;
+    PciDeviceInfo *info;
+    uint8_t type;
+    int class;
+
+    info = g_malloc0(sizeof(*info));
+    info->bus = bus_num;
+    info->slot = PCI_SLOT(dev->devfn);
+    info->function = PCI_FUNC(dev->devfn);
+
+    class = pci_get_word(dev->config + PCI_CLASS_DEVICE);
+    info->class_info.q_class = class;
+    desc = get_class_desc(class);
+    if (desc->desc) {
+        info->class_info.has_desc = true;
+        info->class_info.desc = g_strdup(desc->desc);
+    }
+
+    info->id.vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
+    info->id.device = pci_get_word(dev->config + PCI_DEVICE_ID);
+    info->regions = qmp_query_pci_regions(dev);
+    info->qdev_id = g_strdup(dev->qdev.id ? dev->qdev.id : "");
+
+    if (dev->config[PCI_INTERRUPT_PIN] != 0) {
+        info->has_irq = true;
+        info->irq = dev->config[PCI_INTERRUPT_LINE];
+    }
+
+    type = dev->config[PCI_HEADER_TYPE] & ~PCI_HEADER_TYPE_MULTI_FUNCTION;
+    if (type == PCI_HEADER_TYPE_BRIDGE) {
+        info->has_pci_bridge = true;
+        info->pci_bridge = qmp_query_pci_bridge(dev, bus, bus_num);
+    }
+
+    return info;
+}
+
+PciDeviceInfoList *qmp_query_pci_devices(PCIBus *bus, int bus_num)
+{
+    PciDeviceInfoList *info, *head = NULL, *cur_item = NULL;
+    PCIDevice *dev;
+    int devfn;
+
+    for (devfn = 0; devfn < ARRAY_SIZE(bus->devices); devfn++) {
+        dev = bus->devices[devfn];
+        if (dev) {
+            info = g_malloc0(sizeof(*info));
+            info->value = qmp_query_pci_device(dev, bus, bus_num);
+
+            /* XXX: waiting for the qapi to support GSList */
+            if (!cur_item) {
+                head = cur_item = info;
+            } else {
+                cur_item->next = info;
+                cur_item = info;
+            }
+        }
+    }
+
+    return head;
+}
+
+
+static void pcibus_dev_print(Monitor *mon, DeviceState *dev, int indent)
+{
+    PCIDevice *d = (PCIDevice *)dev;
+    const pci_class_desc *desc;
+    char ctxt[64];
+    PCIIORegion *r;
+    int i, class;
+
+    class = pci_get_word(d->config + PCI_CLASS_DEVICE);
+    desc = pci_class_descriptions;
+    while (desc->desc && class != desc->class) {
+        desc++;
+    }
+    if (desc->desc) {
+        snprintf(ctxt, sizeof(ctxt), "%s", desc->desc);
+    } else {
+        snprintf(ctxt, sizeof(ctxt), "Class %04x", class);
+    }
+
+    monitor_printf(mon, "%*sclass %s, addr %02x:%02x.%x, "
+                   "pci id %04x:%04x (sub %04x:%04x)\n",
+                   indent, "", ctxt, pci_bus_num(d->bus),
+                   PCI_SLOT(d->devfn), PCI_FUNC(d->devfn),
+                   pci_get_word(d->config + PCI_VENDOR_ID),
+                   pci_get_word(d->config + PCI_DEVICE_ID),
+                   pci_get_word(d->config + PCI_SUBSYSTEM_VENDOR_ID),
+                   pci_get_word(d->config + PCI_SUBSYSTEM_ID));
+    for (i = 0; i < PCI_NUM_REGIONS; i++) {
+        r = &d->io_regions[i];
+        if (!r->size) {
+            continue;
+        }
+        monitor_printf(mon, "%*sbar %d: %s at 0x%"FMT_PCIBUS
+                       " [0x%"FMT_PCIBUS"]\n",
+                       indent, "",
+                       i, r->type & PCI_BASE_ADDRESS_SPACE_IO ? "i/o" : "mem",
+                       r->addr, r->addr + r->size - 1);
+    }
+}
+
+static char *pcibus_get_dev_path(DeviceState *dev)
+{
+    PCIDevice *d = container_of(dev, PCIDevice, qdev);
+    PCIDevice *t;
+    int slot_depth;
+    /* Path format: Domain:00:Slot.Function:Slot.Function....:Slot.Function.
+     * 00 is added here to make this format compatible with
+     * domain:Bus:Slot.Func for systems without nested PCI bridges.
+     * Slot.Function list specifies the slot and function numbers for all
+     * devices on the path from root to the specific device. */
+    const char *root_bus_path;
+    int root_bus_len;
+    char slot[] = ":SS.F";
+    int slot_len = sizeof slot - 1 /* For '\0' */;
+    int path_len;
+    char *path, *p;
+    int s;
+
+    root_bus_path = pci_root_bus_path(d);
+    root_bus_len = strlen(root_bus_path);
+
+    /* Calculate # of slots on path between device and root. */;
+    slot_depth = 0;
+    for (t = d; t; t = t->bus->parent_dev) {
+        ++slot_depth;
+    }
+
+    path_len = root_bus_len + slot_len * slot_depth;
+
+    /* Allocate memory, fill in the terminating null byte. */
+    path = g_malloc(path_len + 1 /* For '\0' */);
+    path[path_len] = '\0';
+
+    memcpy(path, root_bus_path, root_bus_len);
+
+    /* Fill in slot numbers. We walk up from device to root, so need to print
+     * them in the reverse order, last to first. */
+    p = path + path_len;
+    for (t = d; t; t = t->bus->parent_dev) {
+        p -= slot_len;
+        s = snprintf(slot, sizeof slot, ":%02x.%x",
+                     PCI_SLOT(t->devfn), PCI_FUNC(t->devfn));
+        assert(s == slot_len);
+        memcpy(p, slot, slot_len);
+    }
+
+    return path;
+}
+
+static char *pci_dev_fw_name(DeviceState *dev, char *buf, int len)
+{
+    PCIDevice *d = (PCIDevice *)dev;
+    const char *name = NULL;
+    const pci_class_desc *desc =  pci_class_descriptions;
+    int class = pci_get_word(d->config + PCI_CLASS_DEVICE);
+
+    while (desc->desc &&
+          (class & ~desc->fw_ign_bits) !=
+          (desc->class & ~desc->fw_ign_bits)) {
+        desc++;
+    }
+
+    if (desc->desc) {
+        name = desc->fw_name;
+    }
+
+    if (name) {
+        pstrcpy(buf, len, name);
+    } else {
+        snprintf(buf, len, "pci%04x,%04x",
+                 pci_get_word(d->config + PCI_VENDOR_ID),
+                 pci_get_word(d->config + PCI_DEVICE_ID));
+    }
+
+    return buf;
+}
+
+static char *pcibus_get_fw_dev_path(DeviceState *dev)
+{
+    PCIDevice *d = (PCIDevice *)dev;
+    char path[50], name[33];
+    int off;
+
+    off = snprintf(path, sizeof(path), "%s@%x",
+                   pci_dev_fw_name(dev, name, sizeof name),
+                   PCI_SLOT(d->devfn));
+    if (PCI_FUNC(d->devfn)) {
+        snprintf(path + off, sizeof(path) + off, ",%x", PCI_FUNC(d->devfn));
+    }
+    return g_strdup(path);
+}
+
+static const VMStateDescription vmstate_pcibus = {
+    .name = "PCIBUS",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32_EQUAL(nirq, PCIBus),
+        VMSTATE_VARRAY_INT32(irq_count, PCIBus,
+                             nirq, 0, vmstate_info_int32,
+                             int32_t),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void pci_bus_realize(BusState *qbus, Error **errp)
+{
+    PCIBus *bus = PCI_BUS(qbus);
+
+    vmstate_register(NULL, -1, &vmstate_pcibus, bus);
+}
+
+static void pci_bus_unrealize(BusState *qbus, Error **errp)
+{
+    PCIBus *bus = PCI_BUS(qbus);
+
+    vmstate_unregister(NULL, &vmstate_pcibus, bus);
+}
+
+/*
+ * Trigger pci bus reset under a given bus.
+ * Called via qbus_reset_all on RST# assert, after the devices
+ * have been reset qdev_reset_all-ed already.
+ */
+static void pcibus_reset(BusState *qbus)
+{
+    PCIBus *bus = DO_UPCAST(PCIBus, qbus, qbus);
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(bus->devices); ++i) {
+        if (bus->devices[i]) {
+            pci_do_device_reset(bus->devices[i]);
+        }
+    }
+
+    for (i = 0; i < bus->nirq; i++) {
+        assert(bus->irq_count[i] == 0);
+    }
+}
+
+static void pci_bus_class_init(ObjectClass *klass, void *data)
+{
+    BusClass *k = BUS_CLASS(klass);
+
+    k->print_dev = pcibus_dev_print;
+    k->get_dev_path = pcibus_get_dev_path;
+    k->get_fw_dev_path = pcibus_get_fw_dev_path;
+    k->realize = pci_bus_realize;
+    k->unrealize = pci_bus_unrealize;
+    k->reset = pcibus_reset;
+}
+
+static const TypeInfo pci_bus_info = {
+    .name = TYPE_PCI_BUS,
+    .parent = TYPE_BUS,
+    .instance_size = sizeof(PCIBus),
+    .class_init = pci_bus_class_init,
+};
+
+static void pci_bus_register_types(void)
+{
+    type_register_static(&pci_bus_info);
+}
+
+type_init(pci_bus_register_types)
diff --git a/hw/ppc/ppc4xx_pci.c b/hw/ppc/ppc4xx_pci.c
index 0bb3cdb..f5847bc 100644
--- a/hw/ppc/ppc4xx_pci.c
+++ b/hw/ppc/ppc4xx_pci.c
@@ -23,6 +23,7 @@
 #include "hw/ppc/ppc.h"
 #include "hw/ppc/ppc4xx.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 #include "exec/address-spaces.h"
 
diff --git a/hw/sh4/r2d.c b/hw/sh4/r2d.c
index d1d0847..8dd2ce3 100644
--- a/hw/sh4/r2d.c
+++ b/hw/sh4/r2d.c
@@ -30,6 +30,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/boards.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "net/net.h"
 #include "sh7750_regs.h"
 #include "hw/ide.h"
diff --git a/hw/sh4/sh_pci.c b/hw/sh4/sh_pci.c
index a2f6d9e..f02e998 100644
--- a/hw/sh4/sh_pci.c
+++ b/hw/sh4/sh_pci.c
@@ -24,6 +24,7 @@
 #include "hw/sysbus.h"
 #include "hw/sh4/sh.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_host.h"
 #include "qemu/bswap.h"
 #include "exec/address-spaces.h"
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index be2d9b8..47077cd 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -337,8 +337,6 @@ typedef void (*pci_set_irq_fn)(void *opaque, int irq_num, int level);
 typedef int (*pci_map_irq_fn)(PCIDevice *pci_dev, int irq_num);
 typedef PCIINTxRoute (*pci_route_irq_fn)(void *opaque, int pin);
 
-#define TYPE_PCI_BUS "PCI"
-#define PCI_BUS(obj) OBJECT_CHECK(PCIBus, (obj), TYPE_PCI_BUS)
 #define TYPE_PCIE_BUS "PCIE"
 
 bool pci_bus_is_express(PCIBus *bus);
@@ -370,6 +368,7 @@ void pci_bus_fire_intx_routing_notifier(PCIBus *bus);
 void pci_device_set_intx_routing_notifier(PCIDevice *dev,
                                           PCIINTxRoutingNotifier notifier);
 void pci_device_reset(PCIDevice *dev);
+void pci_do_device_reset(PCIDevice *dev);
 
 PCIDevice *pci_nic_init_nofail(NICInfo *nd, PCIBus *rootbus,
                                const char *default_model,
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index fabaeee..ea427a3 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -1,6 +1,8 @@
 #ifndef QEMU_PCI_BUS_H
 #define QEMU_PCI_BUS_H
 
+#include "hw/pci/pci.h"
+
 /*
  * PCI Bus and Bridge datastructures.
  *
@@ -8,6 +10,12 @@
  * use accessor functions in pci.h, pci_bridge.h
  */
 
+PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num);
+PciDeviceInfoList *qmp_query_pci_devices(PCIBus *bus, int bus_num);
+
+#define TYPE_PCI_BUS "PCI"
+#define PCI_BUS(obj) OBJECT_CHECK(PCIBus, (obj), TYPE_PCI_BUS)
+
 struct PCIBus {
     BusState qbus;
     PCIIOMMUFunc iommu_fn;
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 15/28] hw/pci: made pci_bus_is_root a PCIBusClass method

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

Refactoring it as a method of PCIBusClass will allow
different implementations for subclasses.

Removed the assumption that the root bus does not
have a parent device because is specific only
to the default class implementation.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci/pci.c             | 11 ++++-------
 hw/pci/pci_bus.c         |  9 +++++++++
 hw/vfio/pci.c            |  1 +
 include/hw/pci/pci.h     |  1 -
 include/hw/pci/pci_bus.h | 15 +++++++++++++++
 5 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 442d209..196989f 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -205,7 +205,10 @@ PCIBus *pci_device_root_bus(const PCIDevice *d)
 {
     PCIBus *bus = d->bus;
 
-    while ((d = bus->parent_dev) != NULL) {
+    while (!pci_bus_is_root(bus)) {
+        d = bus->parent_dev;
+        assert(d != NULL);
+
         bus = d->bus;
     }
 
@@ -218,7 +221,6 @@ const char *pci_root_bus_path(PCIDevice *dev)
     PCIHostState *host_bridge = PCI_HOST_BRIDGE(rootbus->qbus.parent);
     PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_GET_CLASS(host_bridge);
 
-    assert(!rootbus->parent_dev);
     assert(host_bridge->bus == rootbus);
 
     if (hc->root_bus_path) {
@@ -250,11 +252,6 @@ bool pci_bus_is_express(PCIBus *bus)
     return object_dynamic_cast(OBJECT(bus), TYPE_PCIE_BUS);
 }
 
-bool pci_bus_is_root(PCIBus *bus)
-{
-    return !bus->parent_dev;
-}
-
 void pci_bus_new_inplace(PCIBus *bus, size_t bus_size, DeviceState *parent,
                          const char *name,
                          MemoryRegion *address_space_mem,
diff --git a/hw/pci/pci_bus.c b/hw/pci/pci_bus.c
index d156194..0922a75 100644
--- a/hw/pci/pci_bus.c
+++ b/hw/pci/pci_bus.c
@@ -464,9 +464,15 @@ static void pcibus_reset(BusState *qbus)
     }
 }
 
+static bool pcibus_is_root(PCIBus *bus)
+{
+    return !bus->parent_dev;
+}
+
 static void pci_bus_class_init(ObjectClass *klass, void *data)
 {
     BusClass *k = BUS_CLASS(klass);
+    PCIBusClass *pbc = PCI_BUS_CLASS(klass);
 
     k->print_dev = pcibus_dev_print;
     k->get_dev_path = pcibus_get_dev_path;
@@ -474,11 +480,14 @@ static void pci_bus_class_init(ObjectClass *klass, void *data)
     k->realize = pci_bus_realize;
     k->unrealize = pci_bus_unrealize;
     k->reset = pcibus_reset;
+
+    pbc->is_root = pcibus_is_root;
 }
 
 static const TypeInfo pci_bus_info = {
     .name = TYPE_PCI_BUS,
     .parent = TYPE_BUS,
+    .class_size = sizeof(PCIBusClass),
     .instance_size = sizeof(PCIBus),
     .class_init = pci_bus_class_init,
 };
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 84e9d99..e8057db 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -32,6 +32,7 @@
 #include "hw/pci/msi.h"
 #include "hw/pci/msix.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "qemu-common.h"
 #include "qemu/error-report.h"
 #include "qemu/event_notifier.h"
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index 47077cd..ae2c4a5 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -340,7 +340,6 @@ typedef PCIINTxRoute (*pci_route_irq_fn)(void *opaque, int pin);
 #define TYPE_PCIE_BUS "PCIE"
 
 bool pci_bus_is_express(PCIBus *bus);
-bool pci_bus_is_root(PCIBus *bus);
 void pci_bus_new_inplace(PCIBus *bus, size_t bus_size, DeviceState *parent,
                          const char *name,
                          MemoryRegion *address_space_mem,
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index ea427a3..306ef10 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -15,6 +15,16 @@ PciDeviceInfoList *qmp_query_pci_devices(PCIBus *bus, int bus_num);
 
 #define TYPE_PCI_BUS "PCI"
 #define PCI_BUS(obj) OBJECT_CHECK(PCIBus, (obj), TYPE_PCI_BUS)
+#define PCI_BUS_CLASS(klass) OBJECT_CLASS_CHECK(PCIBusClass, (klass), TYPE_PCI_BUS)
+#define PCI_BUS_GET_CLASS(obj) OBJECT_GET_CLASS(PCIBusClass, (obj), TYPE_PCI_BUS)
+
+typedef struct PCIBusClass {
+    /*< private >*/
+    BusClass parent_class;
+    /*< public >*/
+
+    bool (*is_root)(PCIBus *bus);
+} PCIBusClass;
 
 struct PCIBus {
     BusState qbus;
@@ -39,6 +49,11 @@ struct PCIBus {
     int *irq_count;
 };
 
+static inline bool pci_bus_is_root(PCIBus *bus)
+{
+    return PCI_BUS_GET_CLASS(bus)->is_root(bus);
+}
+
 typedef struct PCIBridgeWindows PCIBridgeWindows;
 
 /*
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 16/28] hw/pci: made pci_bus_num a PCIBusClass method

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

Refactoring it as a method of PCIBusClass will allow
different implementations for subclasses.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/kvm/pci-assign.c |  1 +
 hw/pci/pci.c             |  7 -------
 hw/pci/pci_bus.c         | 10 ++++++++++
 hw/scsi/megasas.c        |  1 +
 hw/xen/xen_pt.c          |  1 +
 include/hw/pci/pci.h     |  1 -
 include/hw/pci/pci_bus.h |  6 ++++++
 7 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index 9db7c77..ad573ec 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -35,6 +35,7 @@
 #include "qemu/range.h"
 #include "sysemu/sysemu.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/pci/msi.h"
 #include "kvm_i386.h"
 
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 196989f..e386f2c 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -301,13 +301,6 @@ PCIBus *pci_register_bus(DeviceState *parent, const char *name,
     return bus;
 }
 
-int pci_bus_num(PCIBus *s)
-{
-    if (pci_bus_is_root(s))
-        return 0;       /* pci host bridge */
-    return s->parent_dev->config[PCI_SECONDARY_BUS];
-}
-
 static int get_pci_config_device(QEMUFile *f, void *pv, size_t size)
 {
     PCIDevice *s = container_of(pv, PCIDevice, config);
diff --git a/hw/pci/pci_bus.c b/hw/pci/pci_bus.c
index 0922a75..ed99208 100644
--- a/hw/pci/pci_bus.c
+++ b/hw/pci/pci_bus.c
@@ -469,6 +469,15 @@ static bool pcibus_is_root(PCIBus *bus)
     return !bus->parent_dev;
 }
 
+static int pcibus_num(PCIBus *bus)
+{
+    if (pcibus_is_root(bus)) {
+        return 0;       /* pci host bridge */
+    }
+
+    return bus->parent_dev->config[PCI_SECONDARY_BUS];
+}
+
 static void pci_bus_class_init(ObjectClass *klass, void *data)
 {
     BusClass *k = BUS_CLASS(klass);
@@ -482,6 +491,7 @@ static void pci_bus_class_init(ObjectClass *klass, void *data)
     k->reset = pcibus_reset;
 
     pbc->is_root = pcibus_is_root;
+    pbc->bus_num = pcibus_num;
 }
 
 static const TypeInfo pci_bus_info = {
diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c
index 4852237..fa4e3d0 100644
--- a/hw/scsi/megasas.c
+++ b/hw/scsi/megasas.c
@@ -20,6 +20,7 @@
 
 #include "hw/hw.h"
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "sysemu/dma.h"
 #include "sysemu/block-backend.h"
 #include "hw/pci/msi.h"
diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index f2893b2..cf56a48 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -55,6 +55,7 @@
 #include <sys/ioctl.h>
 
 #include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/xen/xen.h"
 #include "hw/xen/xen_backend.h"
 #include "xen_pt.h"
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index ae2c4a5..a69cf94 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -375,7 +375,6 @@ PCIDevice *pci_nic_init_nofail(NICInfo *nd, PCIBus *rootbus,
 
 PCIDevice *pci_vga_init(PCIBus *bus);
 
-int pci_bus_num(PCIBus *s);
 void pci_for_each_device(PCIBus *bus, int bus_num,
                          void (*fn)(PCIBus *bus, PCIDevice *d, void *opaque),
                          void *opaque);
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index 306ef10..553814e 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -24,6 +24,7 @@ typedef struct PCIBusClass {
     /*< public >*/
 
     bool (*is_root)(PCIBus *bus);
+    int (*bus_num)(PCIBus *bus);
 } PCIBusClass;
 
 struct PCIBus {
@@ -54,6 +55,11 @@ static inline bool pci_bus_is_root(PCIBus *bus)
     return PCI_BUS_GET_CLASS(bus)->is_root(bus);
 }
 
+static inline int pci_bus_num(PCIBus *bus)
+{
+    return PCI_BUS_GET_CLASS(bus)->bus_num(bus);
+}
+
 typedef struct PCIBridgeWindows PCIBridgeWindows;
 
 /*
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 17/28] hw/pci: introduce TYPE_PCI_MAIN_HOST_BRIDGE interface

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

This is a marker interface used to differentiate the
"default" host bridge on a system with multiple host bridges.
This differentiation is required only for pc machines for now
by the ACPI subsystem.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c      | 9 ++++++---
 hw/pci-host/piix.c        | 5 +++++
 hw/pci-host/q35.c         | 4 ++++
 hw/pci/pci_host.c         | 6 ++++++
 include/hw/pci/pci_host.h | 7 +++++++
 5 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 28d7a43..b78a73a 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -251,7 +251,8 @@ static void acpi_get_pci_info(PcPciInfo *info)
     Object *pci_host;
     bool ambiguous;
 
-    pci_host = object_resolve_path_type("", TYPE_PCI_HOST_BRIDGE, &ambiguous);
+    pci_host = object_resolve_path_type("", TYPE_PCI_MAIN_HOST_BRIDGE,
+                                        &ambiguous);
     g_assert(!ambiguous);
     g_assert(pci_host);
 
@@ -1302,7 +1303,8 @@ build_ssdt(GArray *table_data, GArray *linker,
             PCIBus *bus = NULL;
             bool ambiguous;
 
-            pci_host = object_resolve_path_type("", TYPE_PCI_HOST_BRIDGE, &ambiguous);
+            pci_host = object_resolve_path_type("", TYPE_PCI_MAIN_HOST_BRIDGE,
+                                                &ambiguous);
             if (!ambiguous && pci_host) {
                 bus = PCI_HOST_BRIDGE(pci_host)->bus;
             }
@@ -1647,7 +1649,8 @@ static bool acpi_get_mcfg(AcpiMcfgInfo *mcfg)
     QObject *o;
     bool ambiguous;
 
-    pci_host = object_resolve_path_type("", TYPE_PCI_HOST_BRIDGE, &ambiguous);
+    pci_host = object_resolve_path_type("", TYPE_PCI_MAIN_HOST_BRIDGE,
+                                        &ambiguous);
     g_assert(!ambiguous);
     g_assert(pci_host);
 
diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
index 292b6e9..0033ab4 100644
--- a/hw/pci-host/piix.c
+++ b/hw/pci-host/piix.c
@@ -766,6 +766,11 @@ static const TypeInfo i440fx_pcihost_info = {
     .instance_size = sizeof(I440FXState),
     .instance_init = i440fx_pcihost_initfn,
     .class_init    = i440fx_pcihost_class_init,
+    .interfaces    = (InterfaceInfo[]) {
+        { TYPE_PCI_MAIN_HOST_BRIDGE },
+        { }
+    }
+
 };
 
 static void i440fx_register_types(void)
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index a42d0fb..70a86af 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -193,6 +193,10 @@ static const TypeInfo q35_host_info = {
     .instance_size = sizeof(Q35PCIHost),
     .instance_init = q35_host_initfn,
     .class_init = q35_host_class_init,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_PCI_MAIN_HOST_BRIDGE },
+        { }
+    }
 };
 
 /****************************************************************************
diff --git a/hw/pci/pci_host.c b/hw/pci/pci_host.c
index 3e26f92..87180c8 100644
--- a/hw/pci/pci_host.c
+++ b/hw/pci/pci_host.c
@@ -175,6 +175,11 @@ const MemoryRegionOps pci_host_data_be_ops = {
     .endianness = DEVICE_BIG_ENDIAN,
 };
 
+static const TypeInfo pci_main_host_interface_info = {
+    .name          = TYPE_PCI_MAIN_HOST_BRIDGE,
+    .parent        = TYPE_INTERFACE,
+};
+
 static const TypeInfo pci_host_type_info = {
     .name = TYPE_PCI_HOST_BRIDGE,
     .parent = TYPE_SYS_BUS_DEVICE,
@@ -185,6 +190,7 @@ static const TypeInfo pci_host_type_info = {
 
 static void pci_host_register_types(void)
 {
+    type_register_static(&pci_main_host_interface_info);
     type_register_static(&pci_host_type_info);
 }
 
diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
index ba31595..3c72e26 100644
--- a/include/hw/pci/pci_host.h
+++ b/include/hw/pci/pci_host.h
@@ -30,6 +30,13 @@
 
 #include "hw/sysbus.h"
 
+/**
+ * Marker interface for classes whose instances can
+ * be main host bridges. It is intended to be used
+ * when the QOM tree includes multiple host bridges.
+ */
+#define TYPE_PCI_MAIN_HOST_BRIDGE "pci-main-host-bridge"
+
 #define TYPE_PCI_HOST_BRIDGE "pci-host-bridge"
 #define PCI_HOST_BRIDGE(obj) \
     OBJECT_CHECK(PCIHostState, (obj), TYPE_PCI_HOST_BRIDGE)
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 18/28] hw/pci: removed 'rootbus nr is 0' assumption from qmp_pci_query

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

Use the newer pci_bus_num to correctly get the root bus number.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index e386f2c..53598bd 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -1266,7 +1266,8 @@ PciInfoList *qmp_query_pci(Error **errp)
 
     QLIST_FOREACH(host_bridge, &pci_host_bridges, next) {
         info = g_malloc0(sizeof(*info));
-        info->value = qmp_query_pci_bus(host_bridge->bus, 0);
+        info->value = qmp_query_pci_bus(host_bridge->bus,
+                                        pci_bus_num(host_bridge->bus));
 
         /* XXX: waiting for the qapi to support GSList */
         if (!cur_item) {
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 19/28] hw/pci: implement iteration over multiple host bridges

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci/pci.c              | 8 ++++----
 include/hw/pci/pci_host.h | 4 ++++
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 53598bd..f0cf752 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -69,7 +69,7 @@ static void pci_del_option_rom(PCIDevice *pdev);
 static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET;
 static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU;
 
-static QLIST_HEAD(, PCIHostState) pci_host_bridges;
+struct PCIHostQ pci_host_bridges = QLIST_HEAD_INITIALIZER(pci_host_bridges);
 
 static int pci_bar(PCIDevice *d, int reg)
 {
@@ -190,7 +190,7 @@ PCIBus *pci_find_primary_bus(void)
     PCIBus *primary_bus = NULL;
     PCIHostState *host;
 
-    QLIST_FOREACH(host, &pci_host_bridges, next) {
+    HOST_BRIDGE_FOREACH(host) {
         if (primary_bus) {
             /* We have multiple root buses, refuse to select a primary */
             return NULL;
@@ -1264,7 +1264,7 @@ PciInfoList *qmp_query_pci(Error **errp)
     PciInfoList *info, *head = NULL, *cur_item = NULL;
     PCIHostState *host_bridge;
 
-    QLIST_FOREACH(host_bridge, &pci_host_bridges, next) {
+    HOST_BRIDGE_FOREACH(host_bridge) {
         info = g_malloc0(sizeof(*info));
         info->value = qmp_query_pci_bus(host_bridge->bus,
                                         pci_bus_num(host_bridge->bus));
@@ -1802,7 +1802,7 @@ int pci_qdev_find_device(const char *id, PCIDevice **pdev)
     PCIHostState *host_bridge;
     int rc = -ENODEV;
 
-    QLIST_FOREACH(host_bridge, &pci_host_bridges, next) {
+    HOST_BRIDGE_FOREACH(host_bridge) {
         int tmp = pci_qdev_find_recursive(host_bridge->bus, id, pdev);
         if (!tmp) {
             rc = 0;
diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
index 3c72e26..ba5272f 100644
--- a/include/hw/pci/pci_host.h
+++ b/include/hw/pci/pci_host.h
@@ -63,6 +63,10 @@ typedef struct PCIHostBridgeClass {
     const char *(*root_bus_path)(PCIHostState *, PCIBus *);
 } PCIHostBridgeClass;
 
+QLIST_HEAD(PCIHostQ, PCIHostState);
+extern struct PCIHostQ pci_host_bridges;
+#define HOST_BRIDGE_FOREACH(host) QLIST_FOREACH(host, &pci_host_bridges, next)
+
 /* common internal helpers for PCI/PCIe hosts, cut off overflows */
 void pci_host_config_write_common(PCIDevice *pci_dev, uint32_t addr,
                                   uint32_t limit, uint32_t val, uint32_t len);
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 20/28] hw/pci: introduce PCI Expander Bridge (PXB)

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

PXB is a "light-weight" host bridge whose purpose is to enable
the main host bridge to support multiple PCI root buses.

As oposed to PCI-2-PCI bridge's secondary bus, PXB's bus
is a primary bus and can be associated with a NUMA node
(different from the main host bridge) allowing the guest OS
to recognize the proximity of a pass-through device to
other resources as RAM and CPUs.

The PXB is composed from:
 - A primary PCI bus (can be associated with a NUMA node)
   Acts like a normal pci bus and from the functionality point
   of view is an "expansion" of the bus behind the
   main host bridge.
 - A pci-2-pci bridge behind the primary PCI bus where the actual
   devices will be attached.
 - A host-bridge PCI device
   Situated on the bus behind the main host bridge, allows
   the BIOS to configure the bus number and IO/mem resources.
   It does not have its own config/data register for configuration
   cycles, this being handled by the main host bridge.
-  A host-bridge sysbus to comply with QEMU current design.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci-bridge/Makefile.objs         |   1 +
 hw/pci-bridge/pci_expander_bridge.c | 173 ++++++++++++++++++++++++++++++++++++
 include/hw/pci/pci.h                |   1 +
 3 files changed, 175 insertions(+)
 create mode 100644 hw/pci-bridge/pci_expander_bridge.c

diff --git a/hw/pci-bridge/Makefile.objs b/hw/pci-bridge/Makefile.objs
index 968b369..632e442 100644
--- a/hw/pci-bridge/Makefile.objs
+++ b/hw/pci-bridge/Makefile.objs
@@ -1,4 +1,5 @@
 common-obj-y += pci_bridge_dev.o
+common-obj-y += pci_expander_bridge.o
 common-obj-y += ioh3420.o xio3130_upstream.o xio3130_downstream.o
 common-obj-y += i82801b11.o
 # NewWorld PowerMac
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
new file mode 100644
index 0000000..941f3c8
--- /dev/null
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -0,0 +1,173 @@
+/*
+ * PCI Expander Bridge Device Emulation
+ *
+ * Copyright (C) 2014 Red Hat Inc
+ *
+ * Authors:
+ *   Marcel Apfelbaum <marcel.a@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "hw/pci/pci.h"
+#include "hw/pci/pci_bus.h"
+#include "hw/pci/pci_host.h"
+#include "hw/pci/pci_bus.h"
+#include "qemu/range.h"
+#include "qemu/error-report.h"
+
+#define TYPE_PXB_BUS "pxb-bus"
+#define PXB_BUS(obj) OBJECT_CHECK(PXBBus, (obj), TYPE_PXB_BUS)
+
+typedef struct PXBBus {
+    /*< private >*/
+    PCIBus parent_obj;
+    /*< public >*/
+
+    char bus_path[8];
+} PXBBus;
+
+#define TYPE_PXB_DEVICE "pxb-device"
+#define PXB_DEV(obj) OBJECT_CHECK(PXBDev, (obj), TYPE_PXB_DEVICE)
+
+typedef struct PXBDev {
+    /*< private >*/
+    PCIDevice parent_obj;
+    /*< public >*/
+
+    uint8_t bus_nr;
+} PXBDev;
+
+#define TYPE_PXB_HOST "pxb-host"
+
+static int pxb_bus_num(PCIBus *bus)
+{
+    PXBDev *pxb = PXB_DEV(bus->parent_dev);
+
+    return pxb->bus_nr;
+}
+
+static bool pxb_is_root(PCIBus *bus)
+{
+    return true; /* by definition */
+}
+
+static void pxb_bus_class_init(ObjectClass *class, void *data)
+{
+    PCIBusClass *pbc = PCI_BUS_CLASS(class);
+
+    pbc->bus_num = pxb_bus_num;
+    pbc->is_root = pxb_is_root;
+}
+
+static const TypeInfo pxb_bus_info = {
+    .name          = TYPE_PXB_BUS,
+    .parent        = TYPE_PCI_BUS,
+    .instance_size = sizeof(PXBBus),
+    .class_init    = pxb_bus_class_init,
+};
+
+static const char *pxb_host_root_bus_path(PCIHostState *host_bridge,
+                                          PCIBus *rootbus)
+{
+    PXBBus *bus = PXB_BUS(rootbus);
+
+    snprintf(bus->bus_path, 8, "0000:%02x", pxb_bus_num(rootbus));
+    return bus->bus_path;
+}
+
+static void pxb_host_class_init(ObjectClass *class, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(class);
+    PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(class);
+
+    dc->fw_name = "pci";
+    hc->root_bus_path = pxb_host_root_bus_path;
+}
+
+static const TypeInfo pxb_host_info = {
+    .name          = TYPE_PXB_HOST,
+    .parent        = TYPE_PCI_HOST_BRIDGE,
+    .class_init    = pxb_host_class_init,
+};
+
+static int pxb_dev_initfn(PCIDevice *dev)
+{
+    PXBDev *pxb = PXB_DEV(dev);
+    DeviceState *ds, *bds;
+    PCIHostState *phs;
+    PCIBus *bus;
+    const char *dev_name = NULL;
+
+    HOST_BRIDGE_FOREACH(phs) {
+        if (pxb->bus_nr == pci_bus_num(phs->bus)) {
+            error_report("Bus nr %d is already used by %s.",
+                         pxb->bus_nr, phs->bus->qbus.name);
+            return -EINVAL;
+        }
+    }
+
+    if (dev->qdev.id && *dev->qdev.id) {
+        dev_name = dev->qdev.id;
+    }
+
+    ds = qdev_create(NULL, TYPE_PXB_HOST);
+    bus = pci_bus_new(ds, "pxb-internal", NULL, NULL, 0, TYPE_PXB_BUS);
+
+    bus->parent_dev = dev;
+    bus->address_space_mem = dev->bus->address_space_mem;
+    bus->address_space_io = dev->bus->address_space_io;
+    bus->map_irq = pci_swizzle_map_irq_fn;
+
+    bds = qdev_create(BUS(bus), "pci-bridge");
+    bds->id = dev_name;
+    qdev_prop_set_uint8(bds, "chassis_nr", pxb->bus_nr);
+
+    PCI_HOST_BRIDGE(ds)->bus = bus;
+
+    qdev_init_nofail(ds);
+    qdev_init_nofail(bds);
+
+    pci_word_test_and_set_mask(dev->config + PCI_STATUS,
+                               PCI_STATUS_66MHZ | PCI_STATUS_FAST_BACK);
+    pci_config_set_class(dev->config, PCI_CLASS_BRIDGE_HOST);
+
+    return 0;
+}
+
+static Property pxb_dev_properties[] = {
+    /* Note: 0 is not a legal a PXB bus number. */
+    DEFINE_PROP_UINT8("bus_nr", PXBDev, bus_nr, 0),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void pxb_dev_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+
+    k->init = pxb_dev_initfn;
+    k->vendor_id = PCI_VENDOR_ID_REDHAT;
+    k->device_id = PCI_DEVICE_ID_REDHAT_PXB;
+    k->class_id = PCI_CLASS_BRIDGE_HOST;
+
+    dc->desc = "PCI Expander Bridge";
+    dc->props = pxb_dev_properties;
+}
+
+static const TypeInfo pxb_dev_info = {
+    .name          = TYPE_PXB_DEVICE,
+    .parent        = TYPE_PCI_DEVICE,
+    .instance_size = sizeof(PXBDev),
+    .class_init    = pxb_dev_class_init,
+};
+
+static void pxb_register_types(void)
+{
+    type_register_static(&pxb_bus_info);
+    type_register_static(&pxb_host_info);
+    type_register_static(&pxb_dev_info);
+}
+
+type_init(pxb_register_types)
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index a69cf94..4325784 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -90,6 +90,7 @@
 #define PCI_DEVICE_ID_REDHAT_TEST        0x0005
 #define PCI_DEVICE_ID_REDHAT_SDHCI       0x0007
 #define PCI_DEVICE_ID_REDHAT_PCIE_HOST   0x0008
+#define PCI_DEVICE_ID_REDHAT_PXB         0x0009
 #define PCI_DEVICE_ID_REDHAT_QXL         0x0100
 
 #define FMT_PCIBUS                      PRIx64
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 21/28] hw/pci: inform bios if the system has more than one pci bridge

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

The bios looks for 'etc/extra-pci-roots' to decide if
is going to scan further buses after bus 0 tree.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/pc.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index ae3ef0a..71d2f5b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1072,9 +1072,22 @@ typedef struct PcGuestInfoState {
 static
 void pc_guest_info_machine_done(Notifier *notifier, void *data)
 {
+    PCIHostState *host;
+    int hosts = 0;
     PcGuestInfoState *guest_info_state = container_of(notifier,
                                                       PcGuestInfoState,
                                                       machine_done);
+    HOST_BRIDGE_FOREACH(host) {
+        hosts++;
+    }
+
+    if (hosts && guest_info_state->info.fw_cfg) {
+        uint64_t *val = g_malloc(sizeof(*val));
+        *val = cpu_to_le64(hosts - 1);
+        fw_cfg_add_file(guest_info_state->info.fw_cfg,
+                        "etc/extra-pci-roots", val, sizeof(*val));
+    }
+
     acpi_setup(&guest_info_state->info);
 }
 
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 22/28] hw/pci: piix - suport multiple host bridges

[Marcel Apfelbaum]

From: Marcel Apfelbaum <marcel.a@redhat.com>

Instead of assuming it has only one bus, it
enumerates all the host bridges until it finds
the one with bus number corresponding with the
config register.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci-host/piix.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)

diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
index 0033ab4..3c3a192 100644
--- a/hw/pci-host/piix.c
+++ b/hw/pci-host/piix.c
@@ -255,6 +255,61 @@ static void i440fx_pcihost_get_pci_hole64_end(Object *obj, Visitor *v,
     visit_type_uint64(v, &w64.end, name, errp);
 }
 
+static PCIBus *i440fx_find_primary_bus(int bus_num)
+{
+    PCIHostState *host;
+    PCIBus *bus = NULL;
+    int current = -1;
+
+    HOST_BRIDGE_FOREACH(host) {
+        int b = pci_bus_num(host->bus);
+        if (b <= bus_num && b > current) {
+            current = b;
+            bus = host->bus;
+        }
+    }
+
+    return bus;
+}
+
+static void i440fx_pcihost_data_write(void *opaque, hwaddr addr,
+                                      uint64_t val, unsigned len)
+{
+    uint32_t config_reg = PCI_HOST_BRIDGE(opaque)->config_reg;
+
+    if (config_reg & (1u << 31)) {
+        int bus_num = (config_reg >> 16) & 0xFF;
+        PCIBus *bus = i440fx_find_primary_bus(bus_num);
+
+        if (bus) {
+            pci_data_write(bus, config_reg | (addr & 3), val, len);
+        }
+    }
+}
+
+static uint64_t i440fx_pcihost_data_read(void *opaque,
+                                         hwaddr addr, unsigned len)
+{
+    uint32_t config_reg = PCI_HOST_BRIDGE(opaque)->config_reg;
+
+    if (config_reg & (1U << 31)) {
+        int bus_num = (config_reg >> 16) & 0xFF;
+        PCIBus *bus = i440fx_find_primary_bus(bus_num);
+
+        if (bus) {
+            return pci_data_read(bus, config_reg | (addr & 3), len);
+        }
+    }
+
+    return 0xffffffff;
+}
+
+const MemoryRegionOps i440fx_pcihost_data_le_ops = {
+    .read = i440fx_pcihost_data_read,
+    .write = i440fx_pcihost_data_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static void i440fx_pcihost_initfn(Object *obj)
 {
     PCIHostState *s = PCI_HOST_BRIDGE(obj);
@@ -262,7 +317,7 @@ static void i440fx_pcihost_initfn(Object *obj)
 
     memory_region_init_io(&s->conf_mem, obj, &pci_host_conf_le_ops, s,
                           "pci-conf-idx", 4);
-    memory_region_init_io(&s->data_mem, obj, &pci_host_data_le_ops, s,
+    memory_region_init_io(&s->data_mem, obj, &i440fx_pcihost_data_le_ops, s,
                           "pci-conf-data", 4);
 
     object_property_add(obj, PCI_HOST_PROP_PCI_HOLE_START, "int",
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 23/28] hw/pxb: add map_irq func

[Marcel Apfelbaum]

The bios does not index the pxb slot number when
it computes the IRQ because it resides on bus 0
and not on the current bus.
However Qemu routes the irq through bus 0 and adds
the pxb slot to the IRQ computation.

Synchronize between bios and Qemu by canceling
pxb's effect.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci-bridge/pci_expander_bridge.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 941f3c8..87515c1 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -92,6 +92,24 @@ static const TypeInfo pxb_host_info = {
     .class_init    = pxb_host_class_init,
 };
 
+
+static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
+{
+    PCIDevice *pxb = pci_dev->bus->parent_dev;
+
+    /*
+     * The bios does not index the pxb slot number when
+     * it computes the IRQ because it resides on bus 0
+     * and not on the current bus.
+     * However QEMU routes the irq through bus 0 and adds
+     * the pxb slot to the IRQ computation.
+     *
+     * Synchronize between bios and QEMU by canceling
+     * pxb's effect.
+     */
+    return pin - PCI_SLOT(pxb->devfn);
+}
+
 static int pxb_dev_initfn(PCIDevice *dev)
 {
     PXBDev *pxb = PXB_DEV(dev);
@@ -118,7 +136,7 @@ static int pxb_dev_initfn(PCIDevice *dev)
     bus->parent_dev = dev;
     bus->address_space_mem = dev->bus->address_space_mem;
     bus->address_space_io = dev->bus->address_space_io;
-    bus->map_irq = pci_swizzle_map_irq_fn;
+    bus->map_irq = pxb_map_irq_fn;
 
     bds = qdev_create(BUS(bus), "pci-bridge");
     bds->id = dev_name;
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 24/28] hw/pci_bus: add support for NUMA nodes

[Marcel Apfelbaum]

PCI root buses can be attached to a specific NUMA node.
PCI buses are not attached be default to a NUMA node.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/pci/pci_bus.c         | 7 +++++++
 include/hw/pci/pci_bus.h | 6 ++++++
 include/sysemu/sysemu.h  | 1 +
 3 files changed, 14 insertions(+)

diff --git a/hw/pci/pci_bus.c b/hw/pci/pci_bus.c
index ed99208..15882a7 100644
--- a/hw/pci/pci_bus.c
+++ b/hw/pci/pci_bus.c
@@ -13,6 +13,7 @@
 #include "hw/pci/pci_bus.h"
 #include "hw/pci/pci_bridge.h"
 #include "monitor/monitor.h"
+#include "sysemu/sysemu.h"
 
 typedef struct {
     uint16_t class;
@@ -478,6 +479,11 @@ static int pcibus_num(PCIBus *bus)
     return bus->parent_dev->config[PCI_SECONDARY_BUS];
 }
 
+static uint16_t pcibus_numa_node(PCIBus *bus)
+{
+    return NUMA_NODE_UNASSIGNED;
+}
+
 static void pci_bus_class_init(ObjectClass *klass, void *data)
 {
     BusClass *k = BUS_CLASS(klass);
@@ -492,6 +498,7 @@ static void pci_bus_class_init(ObjectClass *klass, void *data)
 
     pbc->is_root = pcibus_is_root;
     pbc->bus_num = pcibus_num;
+    pbc->numa_node = pcibus_numa_node;
 }
 
 static const TypeInfo pci_bus_info = {
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index 553814e..75cd1fa 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -25,6 +25,7 @@ typedef struct PCIBusClass {
 
     bool (*is_root)(PCIBus *bus);
     int (*bus_num)(PCIBus *bus);
+    uint16_t (*numa_node)(PCIBus *bus);
 } PCIBusClass;
 
 struct PCIBus {
@@ -60,6 +61,11 @@ static inline int pci_bus_num(PCIBus *bus)
     return PCI_BUS_GET_CLASS(bus)->bus_num(bus);
 }
 
+static inline int pci_bus_numa_node(PCIBus *bus)
+{
+    return PCI_BUS_GET_CLASS(bus)->numa_node(bus);
+}
+
 typedef struct PCIBridgeWindows PCIBridgeWindows;
 
 /*
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index e7135e1..934eb5d 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -136,6 +136,7 @@ extern const char *mem_path;
 extern int mem_prealloc;
 
 #define MAX_NODES 128
+#define NUMA_NODE_UNASSIGNED MAX_NODES
 
 /* The following shall be true for all CPUs:
  *   cpu->cpu_index < max_cpus <= MAX_CPUMASK_BITS
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 25/28] hw/pxb: add numa_node parameter

[Marcel Apfelbaum]

The pxb can be attach to and existing numa node by specifying
numa_node option that equals the desired numa nodeid.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c                | 12 ++++++++++++
 hw/pci-bridge/pci_expander_bridge.c | 17 +++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index b78a73a..ff18a07 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -899,6 +899,7 @@ build_ssdt(GArray *table_data, GArray *linker,
 
         for (info = info_list; info; info = info->next) {
             PciInfo *bus_info = info->value;
+            PCIHostState *host;
 
             if (bus_info->bus == 0) {
                 continue;
@@ -915,6 +916,17 @@ build_ssdt(GArray *table_data, GArray *linker,
             aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
             aml_append(dev,
                 aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
+
+            HOST_BRIDGE_FOREACH(host) {
+                if (pci_bus_num(host->bus) == bus_info->bus) {
+                    int numa_node = pci_bus_numa_node(host->bus);
+                    if (numa_node != NUMA_NODE_UNASSIGNED) {
+                        aml_append(dev,
+                            aml_name_decl("_PXM", aml_int(numa_node)));
+                    }
+                }
+            }
+
             aml_append(dev, build_prt());
             crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
             aml_append(dev, aml_name_decl("_CRS", crs));
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 87515c1..9329aab 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -16,6 +16,7 @@
 #include "hw/pci/pci_bus.h"
 #include "qemu/range.h"
 #include "qemu/error-report.h"
+#include "sysemu/sysemu.h"
 
 #define TYPE_PXB_BUS "pxb-bus"
 #define PXB_BUS(obj) OBJECT_CHECK(PXBBus, (obj), TYPE_PXB_BUS)
@@ -37,6 +38,7 @@ typedef struct PXBDev {
     /*< public >*/
 
     uint8_t bus_nr;
+    uint16_t numa_node;
 } PXBDev;
 
 #define TYPE_PXB_HOST "pxb-host"
@@ -53,12 +55,20 @@ static bool pxb_is_root(PCIBus *bus)
     return true; /* by definition */
 }
 
+static uint16_t pxb_bus_numa_node(PCIBus *bus)
+{
+    PXBDev *pxb = PXB_DEV(bus->parent_dev);
+
+    return pxb->numa_node;
+}
+
 static void pxb_bus_class_init(ObjectClass *class, void *data)
 {
     PCIBusClass *pbc = PCI_BUS_CLASS(class);
 
     pbc->bus_num = pxb_bus_num;
     pbc->is_root = pxb_is_root;
+    pbc->numa_node = pxb_bus_numa_node;
 }
 
 static const TypeInfo pxb_bus_info = {
@@ -126,6 +136,12 @@ static int pxb_dev_initfn(PCIDevice *dev)
         }
     }
 
+    if (pxb->numa_node != NUMA_NODE_UNASSIGNED &&
+        pxb->numa_node >= nb_numa_nodes) {
+        error_report("Illegal numa node %d.", pxb->numa_node);
+        return -EINVAL;
+    }
+
     if (dev->qdev.id && *dev->qdev.id) {
         dev_name = dev->qdev.id;
     }
@@ -157,6 +173,7 @@ static int pxb_dev_initfn(PCIDevice *dev)
 static Property pxb_dev_properties[] = {
     /* Note: 0 is not a legal a PXB bus number. */
     DEFINE_PROP_UINT8("bus_nr", PXBDev, bus_nr, 0),
+    DEFINE_PROP_UINT16("numa_node", PXBDev, numa_node, NUMA_NODE_UNASSIGNED),
     DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 26/28] acpi: restrict the aml emission to PXB host bridges

[Marcel Apfelbaum]

Initial implementation assumed that the aml used for
any extra root buses would be generic, however this
is not always true. Restrict aml emission only to i440fx and
PXB because is the only supported combination for now.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c                | 56 ++++++++++++++++++++-----------------
 hw/pci-bridge/pci_expander_bridge.c |  1 -
 include/hw/pci/pci_host.h           |  2 ++
 3 files changed, 32 insertions(+), 27 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index ff18a07..2bc8a80 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -887,7 +887,7 @@ build_ssdt(GArray *table_data, GArray *linker,
     /* Reserve space for header */
     acpi_data_push(ssdt->buf, sizeof(AcpiTableHeader));
 
-    {
+    if (find_i440fx()) {
         PciInfoList *info_list, *info;
         Error *err = NULL;
 
@@ -901,37 +901,41 @@ build_ssdt(GArray *table_data, GArray *linker,
             PciInfo *bus_info = info->value;
             PCIHostState *host;
 
-            if (bus_info->bus == 0) {
-                continue;
-            }
+            HOST_BRIDGE_FOREACH(host) {
+                int numa_node;
 
-            if (bus_info->bus < root_bus_limit) {
-                root_bus_limit = bus_info->bus - 1;
-            }
+                if (!(pci_bus_num(host->bus) == bus_info->bus)) {
+                    continue;
+                }
 
-            scope = aml_scope("\\_SB");
-            dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
-            aml_append(dev, aml_name_decl("_UID",
-                aml_string("PC%.02X", (uint8_t)bus_info->bus)));
-            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
-            aml_append(dev,
-                aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
+                if (!object_dynamic_cast(OBJECT(host), TYPE_PXB_HOST)) {
+                    break;
+                }
 
-            HOST_BRIDGE_FOREACH(host) {
-                if (pci_bus_num(host->bus) == bus_info->bus) {
-                    int numa_node = pci_bus_numa_node(host->bus);
-                    if (numa_node != NUMA_NODE_UNASSIGNED) {
-                        aml_append(dev,
+                if (bus_info->bus < root_bus_limit) {
+                    root_bus_limit = bus_info->bus - 1;
+                }
+
+                scope = aml_scope("\\_SB");
+                dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
+                aml_append(dev, aml_name_decl("_UID",
+                            aml_string("PC%.02X", (uint8_t)bus_info->bus)));
+                aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
+                aml_append(dev,
+                        aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
+
+                numa_node = pci_bus_numa_node(host->bus);
+                if (numa_node != NUMA_NODE_UNASSIGNED) {
+                    aml_append(dev,
                             aml_name_decl("_PXM", aml_int(numa_node)));
-                    }
                 }
-            }
 
-            aml_append(dev, build_prt());
-            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
-            aml_append(dev, aml_name_decl("_CRS", crs));
-            aml_append(scope, dev);
-            aml_append(ssdt, scope);
+                aml_append(dev, build_prt());
+                crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
+                aml_append(dev, aml_name_decl("_CRS", crs));
+                aml_append(scope, dev);
+                aml_append(ssdt, scope);
+            }
         }
 
         qapi_free_PciInfoList(info_list);
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 9329aab..8e6740e 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -41,7 +41,6 @@ typedef struct PXBDev {
     uint16_t numa_node;
 } PXBDev;
 
-#define TYPE_PXB_HOST "pxb-host"
 
 static int pxb_bus_num(PCIBus *bus)
 {
diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
index ba5272f..9a35389 100644
--- a/include/hw/pci/pci_host.h
+++ b/include/hw/pci/pci_host.h
@@ -30,6 +30,8 @@
 
 #include "hw/sysbus.h"
 
+#define TYPE_PXB_HOST "pxb-host"
+
 /**
  * Marker interface for classes whose instances can
  * be main host bridges. It is intended to be used
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 27/28] apci: fix PXB behaviour if used with unsupported BIOS

[Marcel Apfelbaum]

PXB does not work with an unsupported BIOS, but should
not interfere with normal OS operation.

Fix this by not adding PXB mem/IO chunks to _CRS
if they weren't configured by BIOS.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 hw/i386/acpi-build.c | 71 +++++++++++++++++++++++++++++++---------------------
 1 file changed, 42 insertions(+), 29 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2bc8a80..416972c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -780,6 +780,10 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
             range_base = region->value->address;
             range_limit = region->value->address + region->value->size - 1;
 
+            if (range_base == PCI_BAR_UNMAPPED) {
+                continue;
+            }
+
             if (!strcmp(region->value->type, "io")) {
                 aml_append(crs,
                     aml_word_io(aml_min_fixed, aml_max_fixed,
@@ -813,41 +817,50 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
 
             range_base = bridge_info->bus.io_range->base;
             range_limit = bridge_info->bus.io_range->limit;
-            aml_append(crs,
-                aml_word_io(aml_min_fixed, aml_max_fixed,
-                            aml_pos_decode, aml_entire_range,
-                            0,
-                            range_base,
-                            range_limit,
-                            0,
-                            range_limit - range_base + 1));
-            crs_range_insert(io_ranges, range_base, range_limit);
+            /* PCI Bridge I/O limit is aligned to 4K */
+            if (range_limit >> 12) {
+                aml_append(crs,
+                    aml_word_io(aml_min_fixed, aml_max_fixed,
+                                aml_pos_decode, aml_entire_range,
+                                0,
+                                range_base,
+                                range_limit,
+                                0,
+                                range_limit - range_base + 1));
+                crs_range_insert(io_ranges, range_base, range_limit);
+            }
 
             range_base = bridge_info->bus.memory_range->base;
             range_limit = bridge_info->bus.memory_range->limit;
-             aml_append(crs,
-                aml_dword_memory(aml_pos_decode, aml_min_fixed,
-                                 aml_max_fixed, aml_non_cacheable,
-                                 aml_ReadWrite,
-                                 0,
-                                 range_base,
-                                 range_limit,
-                                 0,
-                                 range_limit - range_base + 1));
-            crs_range_insert(mem_ranges, range_base, range_limit);
+            /* PCI Bridge MEM limit is aligned to 1M */
+            if (range_limit >> 20) {
+                aml_append(crs,
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
+                                     aml_max_fixed, aml_non_cacheable,
+                                     aml_ReadWrite,
+                                     0,
+                                     range_base,
+                                     range_limit,
+                                     0,
+                                     range_limit - range_base + 1));
+                    crs_range_insert(mem_ranges, range_base, range_limit);
+            }
 
             range_base = bridge_info->bus.prefetchable_range->base;
             range_limit = bridge_info->bus.prefetchable_range->limit;
-            aml_append(crs,
-                aml_dword_memory(aml_pos_decode, aml_min_fixed,
-                                 aml_max_fixed, aml_non_cacheable,
-                                 aml_ReadWrite,
-                                 0,
-                                 range_base,
-                                 range_limit,
-                                 0,
-                                 range_limit - range_base + 1));
-            crs_range_insert(mem_ranges, range_base, range_limit);
+            /* PCI Bridge Prefetch MEM limit is aligned to 1M */
+            if (range_limit >> 20) {
+                aml_append(crs,
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
+                                     aml_max_fixed, aml_non_cacheable,
+                                     aml_ReadWrite,
+                                     0,
+                                     range_base,
+                                     range_limit,
+                                     0,
+                                     range_limit - range_base + 1));
+                crs_range_insert(mem_ranges, range_base, range_limit);
+            }
         }
     }
 
-- 
2.1.0

[Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Marcel Apfelbaum]

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
---
 docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 docs/pci_expander_bridge.txt

diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
new file mode 100644
index 0000000..58bf7a8
--- /dev/null
+++ b/docs/pci_expander_bridge.txt
@@ -0,0 +1,52 @@
+PCI EXPANDER BRIDGE (PXB)
+=========================
+
+Description
+===========
+PXB is a "light-weight" host bridge in the same PCI domain
+as the main host bridge whose purpose is to enable
+the main host bridge to support multiple PCI root buses.
+It is implemented only for i440fx.
+
+As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
+is a primary bus and can be associated with a NUMA node
+(different from the main host bridge) allowing the guest OS
+to recognize the proximity of a pass-through device to
+other resources as RAM and CPUs.
+
+Usage
+=====
+A detailed command line would be:
+
+[qemu-bin + storage options]
+-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
+-m 2G
+-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
+-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
+-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
+-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
+-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
+
+Here you have:
+ - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
+ - a pxb host bridge attached to NUMA 1 with an e1000 behind it
+ - a pxb host bridge attached to NUMA 0 with an e1000 behind it
+ - a pxb host bridge not attached to any NUMA with a hard drive behind it.
+
+Implementation
+==============
+The PXB is composed by:
+- HostBridge (TYPE_PXB_HOST)
+  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
+- PXBDev(TYPE_PXB_DEVICE)
+  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
+  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
+  The interrupts from devices behind the PXB are routed through this device the same as if it were a
+  PCI-2-PCI bridge. The _PRT follows the i440fx model.
+- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
+  Created automatically as part of init sequence.
+  When adding a device to PXB it is attached to the bridge for two reasons:
+  - Using the bridge will enable hotplug support
+  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
+    the PCI address space.
+
-- 
2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 12/28] hw/acpi: add _CRS method for extra root busses

[Michael S. Tsirkin]

Comment from 20150308162743.GC31757@redhat.com not addressed yet I think.

On Tue, Mar 10, 2015 at 05:31:58PM +0200, Marcel Apfelbaum wrote:
> Save the IO/mem/bus numbers ranges assigned to the extra root busses
> to be removed from the root bus 0 range.
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  hw/i386/acpi-build.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 152 insertions(+)
> 
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 513fd6b..5a00f14 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -716,6 +716,151 @@ static Aml *build_prt(void)
>      return method;
>  }
>  
> +typedef struct CrsRangeEntry {
> +    QLIST_ENTRY(CrsRangeEntry) entry;
> +    uint64_t base;
> +    uint64_t limit;
> +} CrsRangeEntry;
> +
> +typedef QLIST_HEAD(CrsRangeQ, CrsRangeEntry) CrsRangeQ;
> +
> +static void crs_range_insert(CrsRangeQ *list, uint64_t base, uint64_t limit)
> +{
> +    CrsRangeEntry *entry, *next, *e;
> +
> +    if (!base) {
> +        return;
> +    }
> +
> +    e = g_malloc(sizeof(*entry));
> +    e->base = base;
> +    e->limit = limit;
> +
> +    if (QLIST_EMPTY(list)) {
> +        QLIST_INSERT_HEAD(list, e, entry);
> +    } else {
> +        QLIST_FOREACH_SAFE(entry, list, entry, next) {
> +            if (base < entry->base) {
> +                QLIST_INSERT_BEFORE(entry, e, entry);
> +                break;
> +            } else if (!next) {
> +                QLIST_INSERT_AFTER(entry, e, entry);
> +                break;
> +            }
> +        }
> +    }
> +}
> +
> +static void crs_range_list_free(CrsRangeQ *list)
> +{
> +    CrsRangeEntry *entry, *next;
> +
> +    QLIST_FOREACH_SAFE(entry, list, entry, next) {
> +        QLIST_REMOVE(entry, entry);
> +        g_free(entry);
> +    }
> +}
> +
> +static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
> +                      CrsRangeQ *io_ranges, CrsRangeQ *mem_ranges)
> +{
> +    PciDeviceInfoList *dev_list;
> +    uint64_t range_base, range_limit;
> +    uint8_t max_bus;
> +    Aml *crs;
> +
> +    crs = aml_resource_template();
> +    max_bus = bus_info->bus;
> +
> +    for (dev_list = bus_info->devices; dev_list; dev_list = dev_list->next) {
> +        PciMemoryRegionList *region;
> +
> +        for (region = dev_list->value->regions; region; region = region->next) {
> +            range_base = region->value->address;
> +            range_limit = region->value->address + region->value->size - 1;
> +
> +            if (!strcmp(region->value->type, "io")) {
> +                aml_append(crs,
> +                    aml_word_io(aml_min_fixed, aml_max_fixed,
> +                                aml_pos_decode, aml_entire_range,
> +                                0,
> +                                range_base,
> +                                range_limit,
> +                                0,
> +                                range_limit - range_base + 1));
> +                crs_range_insert(io_ranges, range_base, range_limit);
> +            } else { /* "memory" */
> +                aml_append(crs,
> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> +                                     aml_max_fixed, aml_non_cacheable,
> +                                     aml_ReadWrite,
> +                                     0,
> +                                     range_base,
> +                                     range_limit,
> +                                     0,
> +                                     range_limit - range_base + 1));
> +                crs_range_insert(mem_ranges, range_base, range_limit);
> +            }
> +        }
> +
> +        if (dev_list->value->has_pci_bridge) {
> +            PciBridgeInfo *bridge_info = dev_list->value->pci_bridge;
> +
> +            if (bridge_info->bus.subordinate > max_bus) {
> +                max_bus = bridge_info->bus.subordinate;
> +            }
> +
> +            range_base = bridge_info->bus.io_range->base;
> +            range_limit = bridge_info->bus.io_range->limit;
> +            aml_append(crs,
> +                aml_word_io(aml_min_fixed, aml_max_fixed,
> +                            aml_pos_decode, aml_entire_range,
> +                            0,
> +                            range_base,
> +                            range_limit,
> +                            0,
> +                            range_limit - range_base + 1));
> +            crs_range_insert(io_ranges, range_base, range_limit);
> +
> +            range_base = bridge_info->bus.memory_range->base;
> +            range_limit = bridge_info->bus.memory_range->limit;
> +             aml_append(crs,
> +                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> +                                 aml_max_fixed, aml_non_cacheable,
> +                                 aml_ReadWrite,
> +                                 0,
> +                                 range_base,
> +                                 range_limit,
> +                                 0,
> +                                 range_limit - range_base + 1));
> +            crs_range_insert(mem_ranges, range_base, range_limit);
> +
> +            range_base = bridge_info->bus.prefetchable_range->base;
> +            range_limit = bridge_info->bus.prefetchable_range->limit;
> +            aml_append(crs,
> +                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> +                                 aml_max_fixed, aml_non_cacheable,
> +                                 aml_ReadWrite,
> +                                 0,
> +                                 range_base,
> +                                 range_limit,
> +                                 0,
> +                                 range_limit - range_base + 1));
> +            crs_range_insert(mem_ranges, range_base, range_limit);
> +        }
> +    }
> +
> +    aml_append(crs,
> +        aml_word_bus_number(aml_min_fixed, aml_max_fixed, aml_pos_decode,
> +                            0,
> +                            bus_info->bus,
> +                            max_bus,
> +                            0,
> +                            max_bus - bus_info->bus + 1));
> +
> +    return crs;
> +}
> +
>  static void
>  build_ssdt(GArray *table_data, GArray *linker,
>             AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
> @@ -725,6 +870,8 @@ build_ssdt(GArray *table_data, GArray *linker,
>      uint32_t nr_mem = machine->ram_slots;
>      unsigned acpi_cpus = guest_info->apic_id_limit;
>      Aml *ssdt, *sb_scope, *scope, *pkg, *dev, *method, *crs, *field, *ifctx;
> +    CrsRangeQ io_ranges = QLIST_HEAD_INITIALIZER(io_ranges);
> +    CrsRangeQ mem_ranges = QLIST_HEAD_INITIALIZER(mem_ranges);
>      int i;
>  
>      ssdt = init_aml_allocator();
> @@ -761,9 +908,14 @@ build_ssdt(GArray *table_data, GArray *linker,
>              aml_append(dev,
>                  aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
>              aml_append(dev, build_prt());
> +            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
> +            aml_append(dev, aml_name_decl("_CRS", crs));
>              aml_append(scope, dev);
>              aml_append(ssdt, scope);
>          }
> +
> +        crs_range_list_free(&io_ranges);
> +        crs_range_list_free(&mem_ranges);
>          qapi_free_PciInfoList(info_list);
>      }
>  
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 26/28] acpi: restrict the aml emission to PXB host bridges

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:12PM +0200, Marcel Apfelbaum wrote:
> Initial implementation assumed that the aml used for
> any extra root buses would be generic, however this
> is not always true. Restrict aml emission only to i440fx and
> PXB because is the only supported combination for now.
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>

I was wondering about this too.
Please split this up and squash into appropriate patches.
I know it's more work but it's worth it.


> ---
>  hw/i386/acpi-build.c                | 56 ++++++++++++++++++++-----------------
>  hw/pci-bridge/pci_expander_bridge.c |  1 -
>  include/hw/pci/pci_host.h           |  2 ++
>  3 files changed, 32 insertions(+), 27 deletions(-)
> 
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index ff18a07..2bc8a80 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -887,7 +887,7 @@ build_ssdt(GArray *table_data, GArray *linker,
>      /* Reserve space for header */
>      acpi_data_push(ssdt->buf, sizeof(AcpiTableHeader));
>  
> -    {
> +    if (find_i440fx()) {
>          PciInfoList *info_list, *info;
>          Error *err = NULL;
>  
> @@ -901,37 +901,41 @@ build_ssdt(GArray *table_data, GArray *linker,
>              PciInfo *bus_info = info->value;
>              PCIHostState *host;
>  
> -            if (bus_info->bus == 0) {
> -                continue;
> -            }
> +            HOST_BRIDGE_FOREACH(host) {
> +                int numa_node;
>  
> -            if (bus_info->bus < root_bus_limit) {
> -                root_bus_limit = bus_info->bus - 1;
> -            }
> +                if (!(pci_bus_num(host->bus) == bus_info->bus)) {
> +                    continue;
> +                }
>  
> -            scope = aml_scope("\\_SB");
> -            dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
> -            aml_append(dev, aml_name_decl("_UID",
> -                aml_string("PC%.02X", (uint8_t)bus_info->bus)));
> -            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
> -            aml_append(dev,
> -                aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
> +                if (!object_dynamic_cast(OBJECT(host), TYPE_PXB_HOST)) {
> +                    break;
> +                }

Or you can check the device/vendor id if that's easier.

>  
> -            HOST_BRIDGE_FOREACH(host) {
> -                if (pci_bus_num(host->bus) == bus_info->bus) {
> -                    int numa_node = pci_bus_numa_node(host->bus);
> -                    if (numa_node != NUMA_NODE_UNASSIGNED) {
> -                        aml_append(dev,
> +                if (bus_info->bus < root_bus_limit) {
> +                    root_bus_limit = bus_info->bus - 1;
> +                }
> +
> +                scope = aml_scope("\\_SB");
> +                dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
> +                aml_append(dev, aml_name_decl("_UID",
> +                            aml_string("PC%.02X", (uint8_t)bus_info->bus)));
> +                aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
> +                aml_append(dev,
> +                        aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
> +
> +                numa_node = pci_bus_numa_node(host->bus);
> +                if (numa_node != NUMA_NODE_UNASSIGNED) {
> +                    aml_append(dev,
>                              aml_name_decl("_PXM", aml_int(numa_node)));
> -                    }
>                  }
> -            }
>  
> -            aml_append(dev, build_prt());
> -            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
> -            aml_append(dev, aml_name_decl("_CRS", crs));
> -            aml_append(scope, dev);
> -            aml_append(ssdt, scope);
> +                aml_append(dev, build_prt());
> +                crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
> +                aml_append(dev, aml_name_decl("_CRS", crs));
> +                aml_append(scope, dev);
> +                aml_append(ssdt, scope);
> +            }
>          }
>  
>          qapi_free_PciInfoList(info_list);
> diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
> index 9329aab..8e6740e 100644
> --- a/hw/pci-bridge/pci_expander_bridge.c
> +++ b/hw/pci-bridge/pci_expander_bridge.c
> @@ -41,7 +41,6 @@ typedef struct PXBDev {
>      uint16_t numa_node;
>  } PXBDev;
>  
> -#define TYPE_PXB_HOST "pxb-host"
>  
>  static int pxb_bus_num(PCIBus *bus)
>  {
> diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
> index ba5272f..9a35389 100644
> --- a/include/hw/pci/pci_host.h
> +++ b/include/hw/pci/pci_host.h
> @@ -30,6 +30,8 @@
>  
>  #include "hw/sysbus.h"
>  
> +#define TYPE_PXB_HOST "pxb-host"
> +

That's a wrong place for it I think.

>  /**
>   * Marker interface for classes whose instances can
>   * be main host bridges. It is intended to be used
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 27/28] apci: fix PXB behaviour if used with unsupported BIOS

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:13PM +0200, Marcel Apfelbaum wrote:
> PXB does not work with an unsupported BIOS, but should
> not interfere with normal OS operation.
> 
> Fix this by not adding PXB mem/IO chunks to _CRS
> if they weren't configured by BIOS.
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  hw/i386/acpi-build.c | 71 +++++++++++++++++++++++++++++++---------------------
>  1 file changed, 42 insertions(+), 29 deletions(-)
> 
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 2bc8a80..416972c 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -780,6 +780,10 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
>              range_base = region->value->address;
>              range_limit = region->value->address + region->value->size - 1;
>  
> +            if (range_base == PCI_BAR_UNMAPPED) {
> +                continue;
> +            }
> +
>              if (!strcmp(region->value->type, "io")) {
>                  aml_append(crs,
>                      aml_word_io(aml_min_fixed, aml_max_fixed,
> @@ -813,41 +817,50 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
>  
>              range_base = bridge_info->bus.io_range->base;
>              range_limit = bridge_info->bus.io_range->limit;
> -            aml_append(crs,
> -                aml_word_io(aml_min_fixed, aml_max_fixed,
> -                            aml_pos_decode, aml_entire_range,
> -                            0,
> -                            range_base,
> -                            range_limit,
> -                            0,
> -                            range_limit - range_base + 1));
> -            crs_range_insert(io_ranges, range_base, range_limit);
> +            /* PCI Bridge I/O limit is aligned to 4K */

Can't say this comment explains anything.
So it's aligned - thus if (range_limit >> 12) is same as
if (range_limit).

Also, generally you should test limit>base for bridges.

> +            if (range_limit >> 12) {
> +                aml_append(crs,
> +                    aml_word_io(aml_min_fixed, aml_max_fixed,
> +                                aml_pos_decode, aml_entire_range,
> +                                0,
> +                                range_base,
> +                                range_limit,
> +                                0,
> +                                range_limit - range_base + 1));
> +                crs_range_insert(io_ranges, range_base, range_limit);
> +            }
>  
>              range_base = bridge_info->bus.memory_range->base;
>              range_limit = bridge_info->bus.memory_range->limit;
> -             aml_append(crs,
> -                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> -                                 aml_max_fixed, aml_non_cacheable,
> -                                 aml_ReadWrite,
> -                                 0,
> -                                 range_base,
> -                                 range_limit,
> -                                 0,
> -                                 range_limit - range_base + 1));
> -            crs_range_insert(mem_ranges, range_base, range_limit);
> +            /* PCI Bridge MEM limit is aligned to 1M */

same comment here.

> +            if (range_limit >> 20) {
> +                aml_append(crs,
> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> +                                     aml_max_fixed, aml_non_cacheable,
> +                                     aml_ReadWrite,
> +                                     0,
> +                                     range_base,
> +                                     range_limit,
> +                                     0,
> +                                     range_limit - range_base + 1));
> +                    crs_range_insert(mem_ranges, range_base, range_limit);
> +            }
>  
>              range_base = bridge_info->bus.prefetchable_range->base;
>              range_limit = bridge_info->bus.prefetchable_range->limit;
> -            aml_append(crs,
> -                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> -                                 aml_max_fixed, aml_non_cacheable,
> -                                 aml_ReadWrite,
> -                                 0,
> -                                 range_base,
> -                                 range_limit,
> -                                 0,
> -                                 range_limit - range_base + 1));
> -            crs_range_insert(mem_ranges, range_base, range_limit);
> +            /* PCI Bridge Prefetch MEM limit is aligned to 1M */
> +            if (range_limit >> 20) {
> +                aml_append(crs,
> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> +                                     aml_max_fixed, aml_non_cacheable,
> +                                     aml_ReadWrite,
> +                                     0,
> +                                     range_base,
> +                                     range_limit,
> +                                     0,
> +                                     range_limit - range_base + 1));
> +                crs_range_insert(mem_ranges, range_base, range_limit);
> +            }
>          }
>      }
>  
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
>  create mode 100644 docs/pci_expander_bridge.txt
> 
> diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
> new file mode 100644
> index 0000000..58bf7a8
> --- /dev/null
> +++ b/docs/pci_expander_bridge.txt
> @@ -0,0 +1,52 @@
> +PCI EXPANDER BRIDGE (PXB)
> +=========================
> +
> +Description
> +===========
> +PXB is a "light-weight" host bridge in the same PCI domain
> +as the main host bridge whose purpose is to enable
> +the main host bridge to support multiple PCI root buses.
> +It is implemented only for i440fx.

BTW what makes it i440fx specific?
Also, what happens if you try to use it
with a different machine type?

> +
> +As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
> +is a primary bus and can be associated with a NUMA node
> +(different from the main host bridge) allowing the guest OS
> +to recognize the proximity of a pass-through device to
> +other resources as RAM and CPUs.
> +
> +Usage
> +=====
> +A detailed command line would be:
> +
> +[qemu-bin + storage options]
> +-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
> +-m 2G
> +-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
> +-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
> +-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
> +-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
> +-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
> +
> +Here you have:
> + - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
> + - a pxb host bridge attached to NUMA 1 with an e1000 behind it
> + - a pxb host bridge attached to NUMA 0 with an e1000 behind it
> + - a pxb host bridge not attached to any NUMA with a hard drive behind it.
> +
> +Implementation
> +==============
> +The PXB is composed by:
> +- HostBridge (TYPE_PXB_HOST)
> +  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
> +- PXBDev(TYPE_PXB_DEVICE)
> +  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
> +  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
> +  The interrupts from devices behind the PXB are routed through this device the same as if it were a
> +  PCI-2-PCI bridge. The _PRT follows the i440fx model.
> +- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
> +  Created automatically as part of init sequence.
> +  When adding a device to PXB it is attached to the bridge for two reasons:
> +  - Using the bridge will enable hotplug support
> +  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
> +    the PCI address space.
> +
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 12/28] hw/acpi: add _CRS method for extra root busses

[Marcel Apfelbaum]

On 03/10/2015 05:38 PM, Michael S. Tsirkin wrote:
> Comment from 20150308162743.GC31757@redhat.com not addressed yet I think.
I'll try to use g_array, let's see how it will look.

Thanks,
Marcel

>
> On Tue, Mar 10, 2015 at 05:31:58PM +0200, Marcel Apfelbaum wrote:
>> Save the IO/mem/bus numbers ranges assigned to the extra root busses
>> to be removed from the root bus 0 range.
>>
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>> ---
>>   hw/i386/acpi-build.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 152 insertions(+)
>>
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index 513fd6b..5a00f14 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -716,6 +716,151 @@ static Aml *build_prt(void)
>>       return method;
>>   }
>>
>> +typedef struct CrsRangeEntry {
>> +    QLIST_ENTRY(CrsRangeEntry) entry;
>> +    uint64_t base;
>> +    uint64_t limit;
>> +} CrsRangeEntry;
>> +
>> +typedef QLIST_HEAD(CrsRangeQ, CrsRangeEntry) CrsRangeQ;
>> +
>> +static void crs_range_insert(CrsRangeQ *list, uint64_t base, uint64_t limit)
>> +{
>> +    CrsRangeEntry *entry, *next, *e;
>> +
>> +    if (!base) {
>> +        return;
>> +    }
>> +
>> +    e = g_malloc(sizeof(*entry));
>> +    e->base = base;
>> +    e->limit = limit;
>> +
>> +    if (QLIST_EMPTY(list)) {
>> +        QLIST_INSERT_HEAD(list, e, entry);
>> +    } else {
>> +        QLIST_FOREACH_SAFE(entry, list, entry, next) {
>> +            if (base < entry->base) {
>> +                QLIST_INSERT_BEFORE(entry, e, entry);
>> +                break;
>> +            } else if (!next) {
>> +                QLIST_INSERT_AFTER(entry, e, entry);
>> +                break;
>> +            }
>> +        }
>> +    }
>> +}
>> +
>> +static void crs_range_list_free(CrsRangeQ *list)
>> +{
>> +    CrsRangeEntry *entry, *next;
>> +
>> +    QLIST_FOREACH_SAFE(entry, list, entry, next) {
>> +        QLIST_REMOVE(entry, entry);
>> +        g_free(entry);
>> +    }
>> +}
>> +
>> +static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
>> +                      CrsRangeQ *io_ranges, CrsRangeQ *mem_ranges)
>> +{
>> +    PciDeviceInfoList *dev_list;
>> +    uint64_t range_base, range_limit;
>> +    uint8_t max_bus;
>> +    Aml *crs;
>> +
>> +    crs = aml_resource_template();
>> +    max_bus = bus_info->bus;
>> +
>> +    for (dev_list = bus_info->devices; dev_list; dev_list = dev_list->next) {
>> +        PciMemoryRegionList *region;
>> +
>> +        for (region = dev_list->value->regions; region; region = region->next) {
>> +            range_base = region->value->address;
>> +            range_limit = region->value->address + region->value->size - 1;
>> +
>> +            if (!strcmp(region->value->type, "io")) {
>> +                aml_append(crs,
>> +                    aml_word_io(aml_min_fixed, aml_max_fixed,
>> +                                aml_pos_decode, aml_entire_range,
>> +                                0,
>> +                                range_base,
>> +                                range_limit,
>> +                                0,
>> +                                range_limit - range_base + 1));
>> +                crs_range_insert(io_ranges, range_base, range_limit);
>> +            } else { /* "memory" */
>> +                aml_append(crs,
>> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> +                                     aml_max_fixed, aml_non_cacheable,
>> +                                     aml_ReadWrite,
>> +                                     0,
>> +                                     range_base,
>> +                                     range_limit,
>> +                                     0,
>> +                                     range_limit - range_base + 1));
>> +                crs_range_insert(mem_ranges, range_base, range_limit);
>> +            }
>> +        }
>> +
>> +        if (dev_list->value->has_pci_bridge) {
>> +            PciBridgeInfo *bridge_info = dev_list->value->pci_bridge;
>> +
>> +            if (bridge_info->bus.subordinate > max_bus) {
>> +                max_bus = bridge_info->bus.subordinate;
>> +            }
>> +
>> +            range_base = bridge_info->bus.io_range->base;
>> +            range_limit = bridge_info->bus.io_range->limit;
>> +            aml_append(crs,
>> +                aml_word_io(aml_min_fixed, aml_max_fixed,
>> +                            aml_pos_decode, aml_entire_range,
>> +                            0,
>> +                            range_base,
>> +                            range_limit,
>> +                            0,
>> +                            range_limit - range_base + 1));
>> +            crs_range_insert(io_ranges, range_base, range_limit);
>> +
>> +            range_base = bridge_info->bus.memory_range->base;
>> +            range_limit = bridge_info->bus.memory_range->limit;
>> +             aml_append(crs,
>> +                aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> +                                 aml_max_fixed, aml_non_cacheable,
>> +                                 aml_ReadWrite,
>> +                                 0,
>> +                                 range_base,
>> +                                 range_limit,
>> +                                 0,
>> +                                 range_limit - range_base + 1));
>> +            crs_range_insert(mem_ranges, range_base, range_limit);
>> +
>> +            range_base = bridge_info->bus.prefetchable_range->base;
>> +            range_limit = bridge_info->bus.prefetchable_range->limit;
>> +            aml_append(crs,
>> +                aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> +                                 aml_max_fixed, aml_non_cacheable,
>> +                                 aml_ReadWrite,
>> +                                 0,
>> +                                 range_base,
>> +                                 range_limit,
>> +                                 0,
>> +                                 range_limit - range_base + 1));
>> +            crs_range_insert(mem_ranges, range_base, range_limit);
>> +        }
>> +    }
>> +
>> +    aml_append(crs,
>> +        aml_word_bus_number(aml_min_fixed, aml_max_fixed, aml_pos_decode,
>> +                            0,
>> +                            bus_info->bus,
>> +                            max_bus,
>> +                            0,
>> +                            max_bus - bus_info->bus + 1));
>> +
>> +    return crs;
>> +}
>> +
>>   static void
>>   build_ssdt(GArray *table_data, GArray *linker,
>>              AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
>> @@ -725,6 +870,8 @@ build_ssdt(GArray *table_data, GArray *linker,
>>       uint32_t nr_mem = machine->ram_slots;
>>       unsigned acpi_cpus = guest_info->apic_id_limit;
>>       Aml *ssdt, *sb_scope, *scope, *pkg, *dev, *method, *crs, *field, *ifctx;
>> +    CrsRangeQ io_ranges = QLIST_HEAD_INITIALIZER(io_ranges);
>> +    CrsRangeQ mem_ranges = QLIST_HEAD_INITIALIZER(mem_ranges);
>>       int i;
>>
>>       ssdt = init_aml_allocator();
>> @@ -761,9 +908,14 @@ build_ssdt(GArray *table_data, GArray *linker,
>>               aml_append(dev,
>>                   aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
>>               aml_append(dev, build_prt());
>> +            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
>> +            aml_append(dev, aml_name_decl("_CRS", crs));
>>               aml_append(scope, dev);
>>               aml_append(ssdt, scope);
>>           }
>> +
>> +        crs_range_list_free(&io_ranges);
>> +        crs_range_list_free(&mem_ranges);
>>           qapi_free_PciInfoList(info_list);
>>       }
>>
>> --
>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 26/28] acpi: restrict the aml emission to PXB host bridges

[Marcel Apfelbaum]

On 03/10/2015 05:41 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 05:32:12PM +0200, Marcel Apfelbaum wrote:
>> Initial implementation assumed that the aml used for
>> any extra root buses would be generic, however this
>> is not always true. Restrict aml emission only to i440fx and
>> PXB because is the only supported combination for now.
>>
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>
> I was wondering about this too.
> Please split this up and squash into appropriate patches.
> I know it's more work but it's worth it.
Sure,

Thanks,
Marcel
>
>
>> ---
>>   hw/i386/acpi-build.c                | 56 ++++++++++++++++++++-----------------
>>   hw/pci-bridge/pci_expander_bridge.c |  1 -
>>   include/hw/pci/pci_host.h           |  2 ++
>>   3 files changed, 32 insertions(+), 27 deletions(-)
>>
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index ff18a07..2bc8a80 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -887,7 +887,7 @@ build_ssdt(GArray *table_data, GArray *linker,
>>       /* Reserve space for header */
>>       acpi_data_push(ssdt->buf, sizeof(AcpiTableHeader));
>>
>> -    {
>> +    if (find_i440fx()) {
>>           PciInfoList *info_list, *info;
>>           Error *err = NULL;
>>
>> @@ -901,37 +901,41 @@ build_ssdt(GArray *table_data, GArray *linker,
>>               PciInfo *bus_info = info->value;
>>               PCIHostState *host;
>>
>> -            if (bus_info->bus == 0) {
>> -                continue;
>> -            }
>> +            HOST_BRIDGE_FOREACH(host) {
>> +                int numa_node;
>>
>> -            if (bus_info->bus < root_bus_limit) {
>> -                root_bus_limit = bus_info->bus - 1;
>> -            }
>> +                if (!(pci_bus_num(host->bus) == bus_info->bus)) {
>> +                    continue;
>> +                }
>>
>> -            scope = aml_scope("\\_SB");
>> -            dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
>> -            aml_append(dev, aml_name_decl("_UID",
>> -                aml_string("PC%.02X", (uint8_t)bus_info->bus)));
>> -            aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
>> -            aml_append(dev,
>> -                aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
>> +                if (!object_dynamic_cast(OBJECT(host), TYPE_PXB_HOST)) {
>> +                    break;
>> +                }
>
> Or you can check the device/vendor id if that's easier.
>
>>
>> -            HOST_BRIDGE_FOREACH(host) {
>> -                if (pci_bus_num(host->bus) == bus_info->bus) {
>> -                    int numa_node = pci_bus_numa_node(host->bus);
>> -                    if (numa_node != NUMA_NODE_UNASSIGNED) {
>> -                        aml_append(dev,
>> +                if (bus_info->bus < root_bus_limit) {
>> +                    root_bus_limit = bus_info->bus - 1;
>> +                }
>> +
>> +                scope = aml_scope("\\_SB");
>> +                dev = aml_device("PC%.02X", (uint8_t)bus_info->bus);
>> +                aml_append(dev, aml_name_decl("_UID",
>> +                            aml_string("PC%.02X", (uint8_t)bus_info->bus)));
>> +                aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
>> +                aml_append(dev,
>> +                        aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
>> +
>> +                numa_node = pci_bus_numa_node(host->bus);
>> +                if (numa_node != NUMA_NODE_UNASSIGNED) {
>> +                    aml_append(dev,
>>                               aml_name_decl("_PXM", aml_int(numa_node)));
>> -                    }
>>                   }
>> -            }
>>
>> -            aml_append(dev, build_prt());
>> -            crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
>> -            aml_append(dev, aml_name_decl("_CRS", crs));
>> -            aml_append(scope, dev);
>> -            aml_append(ssdt, scope);
>> +                aml_append(dev, build_prt());
>> +                crs = build_crs(pci, bus_info, &io_ranges, &mem_ranges);
>> +                aml_append(dev, aml_name_decl("_CRS", crs));
>> +                aml_append(scope, dev);
>> +                aml_append(ssdt, scope);
>> +            }
>>           }
>>
>>           qapi_free_PciInfoList(info_list);
>> diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
>> index 9329aab..8e6740e 100644
>> --- a/hw/pci-bridge/pci_expander_bridge.c
>> +++ b/hw/pci-bridge/pci_expander_bridge.c
>> @@ -41,7 +41,6 @@ typedef struct PXBDev {
>>       uint16_t numa_node;
>>   } PXBDev;
>>
>> -#define TYPE_PXB_HOST "pxb-host"
>>
>>   static int pxb_bus_num(PCIBus *bus)
>>   {
>> diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
>> index ba5272f..9a35389 100644
>> --- a/include/hw/pci/pci_host.h
>> +++ b/include/hw/pci/pci_host.h
>> @@ -30,6 +30,8 @@
>>
>>   #include "hw/sysbus.h"
>>
>> +#define TYPE_PXB_HOST "pxb-host"
>> +
>
> That's a wrong place for it I think.
>
>>   /**
>>    * Marker interface for classes whose instances can
>>    * be main host bridges. It is intended to be used
>> --
>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 27/28] apci: fix PXB behaviour if used with unsupported BIOS

[Marcel Apfelbaum]

On 03/10/2015 05:44 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 05:32:13PM +0200, Marcel Apfelbaum wrote:
>> PXB does not work with an unsupported BIOS, but should
>> not interfere with normal OS operation.
>>
>> Fix this by not adding PXB mem/IO chunks to _CRS
>> if they weren't configured by BIOS.
>>
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>> ---
>>   hw/i386/acpi-build.c | 71 +++++++++++++++++++++++++++++++---------------------
>>   1 file changed, 42 insertions(+), 29 deletions(-)
>>
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index 2bc8a80..416972c 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -780,6 +780,10 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
>>               range_base = region->value->address;
>>               range_limit = region->value->address + region->value->size - 1;
>>
>> +            if (range_base == PCI_BAR_UNMAPPED) {
>> +                continue;
>> +            }
>> +
>>               if (!strcmp(region->value->type, "io")) {
>>                   aml_append(crs,
>>                       aml_word_io(aml_min_fixed, aml_max_fixed,
>> @@ -813,41 +817,50 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
>>
>>               range_base = bridge_info->bus.io_range->base;
>>               range_limit = bridge_info->bus.io_range->limit;
>> -            aml_append(crs,
>> -                aml_word_io(aml_min_fixed, aml_max_fixed,
>> -                            aml_pos_decode, aml_entire_range,
>> -                            0,
>> -                            range_base,
>> -                            range_limit,
>> -                            0,
>> -                            range_limit - range_base + 1));
>> -            crs_range_insert(io_ranges, range_base, range_limit);
>> +            /* PCI Bridge I/O limit is aligned to 4K */
>
> Can't say this comment explains anything.
> So it's aligned - thus if (range_limit >> 12) is same as
> if (range_limit).
I'll use range_base == 0 as invalid value.

Thanks,
Marcel

>
> Also, generally you should test limit>base for bridges.

>
>> +            if (range_limit >> 12) {
>> +                aml_append(crs,
>> +                    aml_word_io(aml_min_fixed, aml_max_fixed,
>> +                                aml_pos_decode, aml_entire_range,
>> +                                0,
>> +                                range_base,
>> +                                range_limit,
>> +                                0,
>> +                                range_limit - range_base + 1));
>> +                crs_range_insert(io_ranges, range_base, range_limit);
>> +            }
>>
>>               range_base = bridge_info->bus.memory_range->base;
>>               range_limit = bridge_info->bus.memory_range->limit;
>> -             aml_append(crs,
>> -                aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> -                                 aml_max_fixed, aml_non_cacheable,
>> -                                 aml_ReadWrite,
>> -                                 0,
>> -                                 range_base,
>> -                                 range_limit,
>> -                                 0,
>> -                                 range_limit - range_base + 1));
>> -            crs_range_insert(mem_ranges, range_base, range_limit);
>> +            /* PCI Bridge MEM limit is aligned to 1M */
>
> same comment here.
>
>> +            if (range_limit >> 20) {
>> +                aml_append(crs,
>> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> +                                     aml_max_fixed, aml_non_cacheable,
>> +                                     aml_ReadWrite,
>> +                                     0,
>> +                                     range_base,
>> +                                     range_limit,
>> +                                     0,
>> +                                     range_limit - range_base + 1));
>> +                    crs_range_insert(mem_ranges, range_base, range_limit);
>> +            }
>>
>>               range_base = bridge_info->bus.prefetchable_range->base;
>>               range_limit = bridge_info->bus.prefetchable_range->limit;
>> -            aml_append(crs,
>> -                aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> -                                 aml_max_fixed, aml_non_cacheable,
>> -                                 aml_ReadWrite,
>> -                                 0,
>> -                                 range_base,
>> -                                 range_limit,
>> -                                 0,
>> -                                 range_limit - range_base + 1));
>> -            crs_range_insert(mem_ranges, range_base, range_limit);
>> +            /* PCI Bridge Prefetch MEM limit is aligned to 1M */
>> +            if (range_limit >> 20) {
>> +                aml_append(crs,
>> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
>> +                                     aml_max_fixed, aml_non_cacheable,
>> +                                     aml_ReadWrite,
>> +                                     0,
>> +                                     range_base,
>> +                                     range_limit,
>> +                                     0,
>> +                                     range_limit - range_base + 1));
>> +                crs_range_insert(mem_ranges, range_base, range_limit);
>> +            }
>>           }
>>       }
>>
>> --
>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 27/28] apci: fix PXB behaviour if used with unsupported BIOS

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 06:19:43PM +0200, Marcel Apfelbaum wrote:
> On 03/10/2015 05:44 PM, Michael S. Tsirkin wrote:
> >On Tue, Mar 10, 2015 at 05:32:13PM +0200, Marcel Apfelbaum wrote:
> >>PXB does not work with an unsupported BIOS, but should
> >>not interfere with normal OS operation.
> >>
> >>Fix this by not adding PXB mem/IO chunks to _CRS
> >>if they weren't configured by BIOS.
> >>
> >>Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> >>---
> >>  hw/i386/acpi-build.c | 71 +++++++++++++++++++++++++++++++---------------------
> >>  1 file changed, 42 insertions(+), 29 deletions(-)
> >>
> >>diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> >>index 2bc8a80..416972c 100644
> >>--- a/hw/i386/acpi-build.c
> >>+++ b/hw/i386/acpi-build.c
> >>@@ -780,6 +780,10 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
> >>              range_base = region->value->address;
> >>              range_limit = region->value->address + region->value->size - 1;
> >>
> >>+            if (range_base == PCI_BAR_UNMAPPED) {
> >>+                continue;
> >>+            }
> >>+
> >>              if (!strcmp(region->value->type, "io")) {
> >>                  aml_append(crs,
> >>                      aml_word_io(aml_min_fixed, aml_max_fixed,
> >>@@ -813,41 +817,50 @@ static Aml *build_crs(PcPciInfo *pci, PciInfo *bus_info,
> >>
> >>              range_base = bridge_info->bus.io_range->base;
> >>              range_limit = bridge_info->bus.io_range->limit;
> >>-            aml_append(crs,
> >>-                aml_word_io(aml_min_fixed, aml_max_fixed,
> >>-                            aml_pos_decode, aml_entire_range,
> >>-                            0,
> >>-                            range_base,
> >>-                            range_limit,
> >>-                            0,
> >>-                            range_limit - range_base + 1));
> >>-            crs_range_insert(io_ranges, range_base, range_limit);
> >>+            /* PCI Bridge I/O limit is aligned to 4K */
> >
> >Can't say this comment explains anything.
> >So it's aligned - thus if (range_limit >> 12) is same as
> >if (range_limit).
> I'll use range_base == 0 as invalid value.
> 
> Thanks,
> Marcel

It's also just a work-around for old broken bioses.
We don't ship them anymore, but it's reasonable
to keep around until we update the bios in qemu.
However, pls add a comment to this end.

> >
> >Also, generally you should test limit>base for bridges.
> 
> >
> >>+            if (range_limit >> 12) {
> >>+                aml_append(crs,
> >>+                    aml_word_io(aml_min_fixed, aml_max_fixed,
> >>+                                aml_pos_decode, aml_entire_range,
> >>+                                0,
> >>+                                range_base,
> >>+                                range_limit,
> >>+                                0,
> >>+                                range_limit - range_base + 1));
> >>+                crs_range_insert(io_ranges, range_base, range_limit);
> >>+            }
> >>
> >>              range_base = bridge_info->bus.memory_range->base;
> >>              range_limit = bridge_info->bus.memory_range->limit;
> >>-             aml_append(crs,
> >>-                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >>-                                 aml_max_fixed, aml_non_cacheable,
> >>-                                 aml_ReadWrite,
> >>-                                 0,
> >>-                                 range_base,
> >>-                                 range_limit,
> >>-                                 0,
> >>-                                 range_limit - range_base + 1));
> >>-            crs_range_insert(mem_ranges, range_base, range_limit);
> >>+            /* PCI Bridge MEM limit is aligned to 1M */
> >
> >same comment here.
> >
> >>+            if (range_limit >> 20) {
> >>+                aml_append(crs,
> >>+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >>+                                     aml_max_fixed, aml_non_cacheable,
> >>+                                     aml_ReadWrite,
> >>+                                     0,
> >>+                                     range_base,
> >>+                                     range_limit,
> >>+                                     0,
> >>+                                     range_limit - range_base + 1));
> >>+                    crs_range_insert(mem_ranges, range_base, range_limit);
> >>+            }
> >>
> >>              range_base = bridge_info->bus.prefetchable_range->base;
> >>              range_limit = bridge_info->bus.prefetchable_range->limit;
> >>-            aml_append(crs,
> >>-                aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >>-                                 aml_max_fixed, aml_non_cacheable,
> >>-                                 aml_ReadWrite,
> >>-                                 0,
> >>-                                 range_base,
> >>-                                 range_limit,
> >>-                                 0,
> >>-                                 range_limit - range_base + 1));
> >>-            crs_range_insert(mem_ranges, range_base, range_limit);
> >>+            /* PCI Bridge Prefetch MEM limit is aligned to 1M */
> >>+            if (range_limit >> 20) {
> >>+                aml_append(crs,
> >>+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >>+                                     aml_max_fixed, aml_non_cacheable,
> >>+                                     aml_ReadWrite,
> >>+                                     0,
> >>+                                     range_base,
> >>+                                     range_limit,
> >>+                                     0,
> >>+                                     range_limit - range_base + 1));
> >>+                crs_range_insert(mem_ranges, range_base, range_limit);
> >>+            }
> >>          }
> >>      }
> >>
> >>--
> >>2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Marcel Apfelbaum]

On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>> ---
>>   docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 52 insertions(+)
>>   create mode 100644 docs/pci_expander_bridge.txt
>>
>> diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
>> new file mode 100644
>> index 0000000..58bf7a8
>> --- /dev/null
>> +++ b/docs/pci_expander_bridge.txt
>> @@ -0,0 +1,52 @@
>> +PCI EXPANDER BRIDGE (PXB)
>> +=========================
>> +
>> +Description
>> +===========
>> +PXB is a "light-weight" host bridge in the same PCI domain
>> +as the main host bridge whose purpose is to enable
>> +the main host bridge to support multiple PCI root buses.
>> +It is implemented only for i440fx.
>
> BTW what makes it i440fx specific?
> Also, what happens if you try to use it
> with a different machine type?
Is is i440fx specific, please look at patch 22/28.
Also we have a specific check for i440fx, so CRS
will not be emitted for other machine types.

Thanks,
Marcel

>
>> +
>> +As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
>> +is a primary bus and can be associated with a NUMA node
>> +(different from the main host bridge) allowing the guest OS
>> +to recognize the proximity of a pass-through device to
>> +other resources as RAM and CPUs.
>> +
>> +Usage
>> +=====
>> +A detailed command line would be:
>> +
>> +[qemu-bin + storage options]
>> +-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
>> +-m 2G
>> +-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
>> +-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
>> +-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
>> +-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
>> +-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
>> +
>> +Here you have:
>> + - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
>> + - a pxb host bridge attached to NUMA 1 with an e1000 behind it
>> + - a pxb host bridge attached to NUMA 0 with an e1000 behind it
>> + - a pxb host bridge not attached to any NUMA with a hard drive behind it.
>> +
>> +Implementation
>> +==============
>> +The PXB is composed by:
>> +- HostBridge (TYPE_PXB_HOST)
>> +  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
>> +- PXBDev(TYPE_PXB_DEVICE)
>> +  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
>> +  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
>> +  The interrupts from devices behind the PXB are routed through this device the same as if it were a
>> +  PCI-2-PCI bridge. The _PRT follows the i440fx model.
>> +- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
>> +  Created automatically as part of init sequence.
>> +  When adding a device to PXB it is attached to the bridge for two reasons:
>> +  - Using the bridge will enable hotplug support
>> +  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
>> +    the PCI address space.
>> +
>> --
>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 22/28] hw/pci: piix - suport multiple host bridges

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:08PM +0200, Marcel Apfelbaum wrote:
> From: Marcel Apfelbaum <marcel.a@redhat.com>
> 
> Instead of assuming it has only one bus, it
> enumerates all the host bridges until it finds
> the one with bus number corresponding with the
> config register.

This really only works for PXB since that listens
in on config cycles on the main bus.
Best limit to that.

One also wonders what will happen when multiple
PXBs are present on separate root complexes.

> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  hw/pci-host/piix.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 56 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
> index 0033ab4..3c3a192 100644
> --- a/hw/pci-host/piix.c
> +++ b/hw/pci-host/piix.c
> @@ -255,6 +255,61 @@ static void i440fx_pcihost_get_pci_hole64_end(Object *obj, Visitor *v,
>      visit_type_uint64(v, &w64.end, name, errp);
>  }
>  
> +static PCIBus *i440fx_find_primary_bus(int bus_num)
> +{
> +    PCIHostState *host;
> +    PCIBus *bus = NULL;
> +    int current = -1;
> +
> +    HOST_BRIDGE_FOREACH(host) {
> +        int b = pci_bus_num(host->bus);
> +        if (b <= bus_num && b > current) {
> +            current = b;
> +            bus = host->bus;
> +        }
> +    }
> +
> +    return bus;
> +}
> +
> +static void i440fx_pcihost_data_write(void *opaque, hwaddr addr,
> +                                      uint64_t val, unsigned len)
> +{
> +    uint32_t config_reg = PCI_HOST_BRIDGE(opaque)->config_reg;
> +
> +    if (config_reg & (1u << 31)) {
> +        int bus_num = (config_reg >> 16) & 0xFF;
> +        PCIBus *bus = i440fx_find_primary_bus(bus_num);
> +
> +        if (bus) {
> +            pci_data_write(bus, config_reg | (addr & 3), val, len);
> +        }
> +    }
> +}
> +
> +static uint64_t i440fx_pcihost_data_read(void *opaque,
> +                                         hwaddr addr, unsigned len)
> +{
> +    uint32_t config_reg = PCI_HOST_BRIDGE(opaque)->config_reg;
> +
> +    if (config_reg & (1U << 31)) {
> +        int bus_num = (config_reg >> 16) & 0xFF;
> +        PCIBus *bus = i440fx_find_primary_bus(bus_num);
> +
> +        if (bus) {
> +            return pci_data_read(bus, config_reg | (addr & 3), len);
> +        }
> +    }
> +
> +    return 0xffffffff;
> +}
> +
> +const MemoryRegionOps i440fx_pcihost_data_le_ops = {
> +    .read = i440fx_pcihost_data_read,
> +    .write = i440fx_pcihost_data_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +};
> +
>  static void i440fx_pcihost_initfn(Object *obj)
>  {
>      PCIHostState *s = PCI_HOST_BRIDGE(obj);
> @@ -262,7 +317,7 @@ static void i440fx_pcihost_initfn(Object *obj)
>  
>      memory_region_init_io(&s->conf_mem, obj, &pci_host_conf_le_ops, s,
>                            "pci-conf-idx", 4);
> -    memory_region_init_io(&s->data_mem, obj, &pci_host_data_le_ops, s,
> +    memory_region_init_io(&s->data_mem, obj, &i440fx_pcihost_data_le_ops, s,
>                            "pci-conf-data", 4);
>  
>      object_property_add(obj, PCI_HOST_PROP_PCI_HOLE_START, "int",
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 21/28] hw/pci: inform bios if the system has more than one pci bridge

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:07PM +0200, Marcel Apfelbaum wrote:
> From: Marcel Apfelbaum <marcel.a@redhat.com>
> 
> The bios looks for 'etc/extra-pci-roots' to decide if
> is going to scan further buses after bus 0 tree.
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>

Again it's not the right thing to do for extra root
complexes, only works for PXB.

> ---
>  hw/i386/pc.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index ae3ef0a..71d2f5b 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1072,9 +1072,22 @@ typedef struct PcGuestInfoState {
>  static
>  void pc_guest_info_machine_done(Notifier *notifier, void *data)
>  {
> +    PCIHostState *host;
> +    int hosts = 0;
>      PcGuestInfoState *guest_info_state = container_of(notifier,
>                                                        PcGuestInfoState,
>                                                        machine_done);
> +    HOST_BRIDGE_FOREACH(host) {
> +        hosts++;
> +    }
> +
> +    if (hosts && guest_info_state->info.fw_cfg) {
> +        uint64_t *val = g_malloc(sizeof(*val));
> +        *val = cpu_to_le64(hosts - 1);
> +        fw_cfg_add_file(guest_info_state->info.fw_cfg,
> +                        "etc/extra-pci-roots", val, sizeof(*val));
> +    }
> +
>      acpi_setup(&guest_info_state->info);
>  }
>  
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 19/28] hw/pci: implement iteration over multiple host bridges

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:05PM +0200, Marcel Apfelbaum wrote:
> From: Marcel Apfelbaum <marcel.a@redhat.com>
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>

This isn't a good way to split patches.
You add an API with no documentation,
it's use is also in a separate file.

This patch really can't reasonably be reviewed
in isolation.

In this case, I think a better way is to
scan from root complex down, and find PXBs.
This will scale nicely to multiple root complexes.
You can do this using PCI or QOM methods.



> ---
>  hw/pci/pci.c              | 8 ++++----
>  include/hw/pci/pci_host.h | 4 ++++
>  2 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/pci/pci.c b/hw/pci/pci.c
> index 53598bd..f0cf752 100644
> --- a/hw/pci/pci.c
> +++ b/hw/pci/pci.c
> @@ -69,7 +69,7 @@ static void pci_del_option_rom(PCIDevice *pdev);
>  static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET;
>  static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU;
>  
> -static QLIST_HEAD(, PCIHostState) pci_host_bridges;
> +struct PCIHostQ pci_host_bridges = QLIST_HEAD_INITIALIZER(pci_host_bridges);
>  
>  static int pci_bar(PCIDevice *d, int reg)
>  {
> @@ -190,7 +190,7 @@ PCIBus *pci_find_primary_bus(void)
>      PCIBus *primary_bus = NULL;
>      PCIHostState *host;
>  
> -    QLIST_FOREACH(host, &pci_host_bridges, next) {
> +    HOST_BRIDGE_FOREACH(host) {
>          if (primary_bus) {
>              /* We have multiple root buses, refuse to select a primary */
>              return NULL;
> @@ -1264,7 +1264,7 @@ PciInfoList *qmp_query_pci(Error **errp)
>      PciInfoList *info, *head = NULL, *cur_item = NULL;
>      PCIHostState *host_bridge;
>  
> -    QLIST_FOREACH(host_bridge, &pci_host_bridges, next) {
> +    HOST_BRIDGE_FOREACH(host_bridge) {
>          info = g_malloc0(sizeof(*info));
>          info->value = qmp_query_pci_bus(host_bridge->bus,
>                                          pci_bus_num(host_bridge->bus));
> @@ -1802,7 +1802,7 @@ int pci_qdev_find_device(const char *id, PCIDevice **pdev)
>      PCIHostState *host_bridge;
>      int rc = -ENODEV;
>  
> -    QLIST_FOREACH(host_bridge, &pci_host_bridges, next) {
> +    HOST_BRIDGE_FOREACH(host_bridge) {
>          int tmp = pci_qdev_find_recursive(host_bridge->bus, id, pdev);
>          if (!tmp) {
>              rc = 0;
> diff --git a/include/hw/pci/pci_host.h b/include/hw/pci/pci_host.h
> index 3c72e26..ba5272f 100644
> --- a/include/hw/pci/pci_host.h
> +++ b/include/hw/pci/pci_host.h
> @@ -63,6 +63,10 @@ typedef struct PCIHostBridgeClass {
>      const char *(*root_bus_path)(PCIHostState *, PCIBus *);
>  } PCIHostBridgeClass;
>  
> +QLIST_HEAD(PCIHostQ, PCIHostState);
> +extern struct PCIHostQ pci_host_bridges;
> +#define HOST_BRIDGE_FOREACH(host) QLIST_FOREACH(host, &pci_host_bridges, next)
> +
>  /* common internal helpers for PCI/PCIe hosts, cut off overflows */
>  void pci_host_config_write_common(PCIDevice *pci_dev, uint32_t addr,
>                                    uint32_t limit, uint32_t val, uint32_t len);
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 11/28] hw/apci: add _PRT method for extra PCI root busses

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:31:57PM +0200, Marcel Apfelbaum wrote:
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>

You seem to have ignored some comments on this patch.
Pls dig them up and address.

> ---
>  hw/i386/acpi-build.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 65 insertions(+)
> 
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 3e2701b..513fd6b 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -652,6 +652,70 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>      aml_append(parent_scope, method);
>  }
>  
> +static Aml *build_link(Aml *lnk, Aml *lnk_pkg, const char *link_name, int idx)
> +{
> +    Aml *if_ctx, *pkg;
> +
> +    if_ctx = aml_if(aml_equal(lnk, aml_int(idx)));
> +    pkg = aml_package(4);
> +    aml_append(pkg, aml_int(0));
> +    aml_append(pkg, aml_int(0));
> +    aml_append(pkg, aml_name(link_name, ""));
> +    aml_append(pkg, aml_int(0));
> +    aml_append(if_ctx, aml_store(pkg, lnk_pkg));
> +
> +    return if_ctx;
> +}
> +
> +static Aml *build_prt(void)
> +{
> +    Aml *method, *while_ctx, *pin, *slot, *lnk, *pkg, *res;
> +
> +    method = aml_method("_PRT", 0);
> +    res = aml_local(0);
> +    pin = aml_local(1);
> +    slot = aml_local(2);
> +    lnk = aml_local(3);
> +    pkg = aml_local(4);
> +
> +    aml_append(method, aml_store(aml_package(128), res));
> +    aml_append(method, aml_store(aml_int(0), pin));
> +
> +    /* while (pin < 128) */
> +    while_ctx = aml_while(aml_lless(pin, aml_int(128)));
> +    {
> +        /* slot = pin >> 2 */
> +        aml_append(while_ctx,
> +                   aml_store(aml_shiftright(pin, aml_int(2)), slot));
> +        /* lnk = (slot + pin) & 3 */
> +        aml_append(while_ctx,
> +                   aml_store(aml_and(aml_add(pin, slot), aml_int(3)), lnk));
> +
> +        /* pkg[2] = "LNK[A|B|C|D]", selection based on lnk % 3  */
> +        aml_append(while_ctx, build_link(lnk, pkg, "LNKD", 0));
> +        aml_append(while_ctx, build_link(lnk, pkg, "LNKA", 1));
> +        aml_append(while_ctx, build_link(lnk, pkg, "LNKB", 2));
> +        aml_append(while_ctx, build_link(lnk, pkg, "LNKC", 3));
> +
> +        /* pkg[0] = 0x[slot]FFFF */
> +        aml_append(while_ctx,
> +            aml_store(aml_or(aml_shiftleft(slot, aml_int(16)), aml_int(0xFFFF)),
> +                      aml_index(pkg, aml_int(0))));
> +        /* pkg[1] = pin & 3 */
> +        aml_append(while_ctx,
> +            aml_store(aml_and(pin, aml_int(3)), aml_index(pkg, aml_int(1))));
> +        /* res[pin] = pkg */
> +        aml_append(while_ctx, aml_store(pkg, aml_index(res, pin)));
> +        /* pin++ */
> +        aml_append(while_ctx, aml_increment(pin));
> +    }
> +    aml_append(method, while_ctx);
> +    /* return res*/
> +    aml_append(method, aml_return(res));
> +
> +    return method;
> +}
> +
>  static void
>  build_ssdt(GArray *table_data, GArray *linker,
>             AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
> @@ -696,6 +760,7 @@ build_ssdt(GArray *table_data, GArray *linker,
>              aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A03")));
>              aml_append(dev,
>                  aml_name_decl("_BBN", aml_int((uint8_t)bus_info->bus)));
> +            aml_append(dev, build_prt());
>              aml_append(scope, dev);
>              aml_append(ssdt, scope);
>          }
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 23/28] hw/pxb: add map_irq func

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 05:32:09PM +0200, Marcel Apfelbaum wrote:
> The bios does not index the pxb slot number when
> it computes the IRQ because it resides on bus 0
> and not on the current bus.
> However Qemu routes the irq through bus 0 and adds
> the pxb slot to the IRQ computation.
> 
> Synchronize between bios and Qemu by canceling
> pxb's effect.
> 
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  hw/pci-bridge/pci_expander_bridge.c | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
> index 941f3c8..87515c1 100644
> --- a/hw/pci-bridge/pci_expander_bridge.c
> +++ b/hw/pci-bridge/pci_expander_bridge.c
> @@ -92,6 +92,24 @@ static const TypeInfo pxb_host_info = {
>      .class_init    = pxb_host_class_init,
>  };
>  
> +
> +static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
> +{
> +    PCIDevice *pxb = pci_dev->bus->parent_dev;
> +
> +    /*
> +     * The bios does not index the pxb slot number when
> +     * it computes the IRQ because it resides on bus 0
> +     * and not on the current bus.
> +     * However QEMU routes the irq through bus 0 and adds
> +     * the pxb slot to the IRQ computation.

I know it's QEMU but which function exactly?

> +     *
> +     * Synchronize between bios and QEMU by canceling
> +     * pxb's effect.
> +     */
> +    return pin - PCI_SLOT(pxb->devfn);
> +}
> +
>  static int pxb_dev_initfn(PCIDevice *dev)
>  {
>      PXBDev *pxb = PXB_DEV(dev);
> @@ -118,7 +136,7 @@ static int pxb_dev_initfn(PCIDevice *dev)
>      bus->parent_dev = dev;
>      bus->address_space_mem = dev->bus->address_space_mem;
>      bus->address_space_io = dev->bus->address_space_io;
> -    bus->map_irq = pci_swizzle_map_irq_fn;
> +    bus->map_irq = pxb_map_irq_fn;
>  
>      bds = qdev_create(BUS(bus), "pci-bridge");
>      bds->id = dev_name;
> -- 
> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Michael S. Tsirkin]

On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
> On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
> >On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
> >>Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> >>---
> >>  docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 52 insertions(+)
> >>  create mode 100644 docs/pci_expander_bridge.txt
> >>
> >>diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
> >>new file mode 100644
> >>index 0000000..58bf7a8
> >>--- /dev/null
> >>+++ b/docs/pci_expander_bridge.txt
> >>@@ -0,0 +1,52 @@
> >>+PCI EXPANDER BRIDGE (PXB)
> >>+=========================
> >>+
> >>+Description
> >>+===========
> >>+PXB is a "light-weight" host bridge in the same PCI domain
> >>+as the main host bridge whose purpose is to enable
> >>+the main host bridge to support multiple PCI root buses.
> >>+It is implemented only for i440fx.
> >
> >BTW what makes it i440fx specific?
> >Also, what happens if you try to use it
> >with a different machine type?
> Is is i440fx specific, please look at patch 22/28.
> Also we have a specific check for i440fx, so CRS
> will not be emitted for other machine types.
> 
> Thanks,
> Marcel

In fact it won't work at all. Need to think about it,
maybe we can make it work more generally.
For CRS, should be possible to emit for q35 too?

> >
> >>+
> >>+As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
> >>+is a primary bus and can be associated with a NUMA node
> >>+(different from the main host bridge) allowing the guest OS
> >>+to recognize the proximity of a pass-through device to
> >>+other resources as RAM and CPUs.
> >>+
> >>+Usage
> >>+=====
> >>+A detailed command line would be:
> >>+
> >>+[qemu-bin + storage options]
> >>+-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
> >>+-m 2G
> >>+-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
> >>+-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
> >>+-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
> >>+-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
> >>+-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
> >>+
> >>+Here you have:
> >>+ - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
> >>+ - a pxb host bridge attached to NUMA 1 with an e1000 behind it
> >>+ - a pxb host bridge attached to NUMA 0 with an e1000 behind it
> >>+ - a pxb host bridge not attached to any NUMA with a hard drive behind it.
> >>+
> >>+Implementation
> >>+==============
> >>+The PXB is composed by:
> >>+- HostBridge (TYPE_PXB_HOST)
> >>+  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
> >>+- PXBDev(TYPE_PXB_DEVICE)
> >>+  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
> >>+  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
> >>+  The interrupts from devices behind the PXB are routed through this device the same as if it were a
> >>+  PCI-2-PCI bridge. The _PRT follows the i440fx model.
> >>+- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
> >>+  Created automatically as part of init sequence.
> >>+  When adding a device to PXB it is attached to the bridge for two reasons:
> >>+  - Using the bridge will enable hotplug support
> >>+  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
> >>+    the PCI address space.
> >>+
> >>--
> >>2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 02/28] acpi: add aml_or() term

[Shannon Zhao]

On 2015/3/10 23:31, Marcel Apfelbaum wrote:
> Add encoding for ACPI DefOr Opcode.
> 
> Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
> Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> ---
>  hw/acpi/aml-build.c         | 10 ++++++++++
>  include/hw/acpi/aml-build.h |  1 +
>  2 files changed, 11 insertions(+)
> 
> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> index ace180b..db8d346 100644
> --- a/hw/acpi/aml-build.c
> +++ b/hw/acpi/aml-build.c
> @@ -452,6 +452,16 @@ Aml *aml_and(Aml *arg1, Aml *arg2)
>      return var;
>  }
>  
> +/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefOr */
> +Aml *aml_or(Aml *arg1, Aml *arg2)
> +{
> +    Aml *var = aml_opcode(0x7D /* OrOp */);
> +    aml_append(var, arg1);
> +    aml_append(var, arg2);
> +    build_append_int(var->buf, 0x00); /* NullNameOp */

Maybe you forgot to fix this. Same with patch 03, 05, 06, 07.

Thanks,
Shannon

> +    return var;
> +}
> +
>  /* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefNotify */
>  Aml *aml_notify(Aml *arg1, Aml *arg2)
>  {
> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> index 17d3beb..c0eb691 100644
> --- a/include/hw/acpi/aml-build.h
> +++ b/include/hw/acpi/aml-build.h
> @@ -137,6 +137,7 @@ Aml *aml_int(const uint64_t val);
>  Aml *aml_arg(int pos);
>  Aml *aml_store(Aml *val, Aml *target);
>  Aml *aml_and(Aml *arg1, Aml *arg2);
> +Aml *aml_or(Aml *arg1, Aml *arg2);
>  Aml *aml_notify(Aml *arg1, Aml *arg2);
>  Aml *aml_call1(const char *method, Aml *arg1);
>  Aml *aml_call2(const char *method, Aml *arg1, Aml *arg2);
> 

Re: [Qemu-devel] [PATCH v5 for-2.3 02/28] acpi: add aml_or() term

[Marcel Apfelbaum]

On 03/11/2015 03:17 AM, Shannon Zhao wrote:
> On 2015/3/10 23:31, Marcel Apfelbaum wrote:
>> Add encoding for ACPI DefOr Opcode.
>>
>> Reviewed-by: Shannon Zhao <zhaoshenglong@huawei.com>
>> Reviewed-by: Igor Mammedov <imammedo@redhat.com>
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>> ---
>>   hw/acpi/aml-build.c         | 10 ++++++++++
>>   include/hw/acpi/aml-build.h |  1 +
>>   2 files changed, 11 insertions(+)
>>
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index ace180b..db8d346 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -452,6 +452,16 @@ Aml *aml_and(Aml *arg1, Aml *arg2)
>>       return var;
>>   }
>>
>> +/* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefOr */
>> +Aml *aml_or(Aml *arg1, Aml *arg2)
>> +{
>> +    Aml *var = aml_opcode(0x7D /* OrOp */);
>> +    aml_append(var, arg1);
>> +    aml_append(var, arg2);
>> +    build_append_int(var->buf, 0x00); /* NullNameOp */
>
> Maybe you forgot to fix this. Same with patch 03, 05, 06, 07.
Strange, I was sure I took care of it.

Thanks for bringing this up again!
Marcel

>
> Thanks,
> Shannon
>
>> +    return var;
>> +}
>> +
>>   /* ACPI 1.0b: 16.2.5.3 Type 1 Opcodes Encoding: DefNotify */
>>   Aml *aml_notify(Aml *arg1, Aml *arg2)
>>   {
>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>> index 17d3beb..c0eb691 100644
>> --- a/include/hw/acpi/aml-build.h
>> +++ b/include/hw/acpi/aml-build.h
>> @@ -137,6 +137,7 @@ Aml *aml_int(const uint64_t val);
>>   Aml *aml_arg(int pos);
>>   Aml *aml_store(Aml *val, Aml *target);
>>   Aml *aml_and(Aml *arg1, Aml *arg2);
>> +Aml *aml_or(Aml *arg1, Aml *arg2);
>>   Aml *aml_notify(Aml *arg1, Aml *arg2);
>>   Aml *aml_call1(const char *method, Aml *arg1);
>>   Aml *aml_call2(const char *method, Aml *arg1, Aml *arg2);
>>
>

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Gerd Hoffmann]

  Hi,

> v4->v5:
>  - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git

Have again trouble applying this.  Which commit hash?  Or do you have a
git tree somewhere with this?

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Marcel Apfelbaum]

On 03/11/2015 03:32 PM, Gerd Hoffmann wrote:
>    Hi,
>
>> v4->v5:
>>   - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
>
> Have again trouble applying this.  Which commit hash?  Or do you have a
> git tree somewhere with this?
Yes, it is changing too fast :(
This version works with commit: a3b66ab

If you have problem with this commit, tell me and I'll push my branch to github.

Thanks,
Marcel

>
> cheers,
>    Gerd
>
>

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Gerd Hoffmann]

On Mi, 2015-03-11 at 15:44 +0200, Marcel Apfelbaum wrote:
> On 03/11/2015 03:32 PM, Gerd Hoffmann wrote:
> >    Hi,
> >
> >> v4->v5:
> >>   - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
> >
> > Have again trouble applying this.  Which commit hash?  Or do you have a
> > git tree somewhere with this?
> Yes, it is changing too fast :(
> This version works with commit: a3b66ab

Applies fine, but throws a checkpatch warning:

Applying: hw/acpi: remove from root bus 0 the crs resources used by
other busses.
=== checkpatch complains ===
WARNING: line over 80 characters
#106: FILE: hw/i386/acpi-build.c:988:
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
aml_max_fixed,

WARNING: line over 80 characters
#114: FILE: hw/i386/acpi-build.c:996:
+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
aml_max_fixed,

total: 0 errors, 2 warnings, 124 lines checked

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Marcel Apfelbaum]

On 03/11/2015 03:51 PM, Gerd Hoffmann wrote:
> On Mi, 2015-03-11 at 15:44 +0200, Marcel Apfelbaum wrote:
>> On 03/11/2015 03:32 PM, Gerd Hoffmann wrote:
>>>     Hi,
>>>
>>>> v4->v5:
>>>>    - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
>>>
>>> Have again trouble applying this.  Which commit hash?  Or do you have a
>>> git tree somewhere with this?
>> Yes, it is changing too fast :(
>> This version works with commit: a3b66ab
>
> Applies fine, but throws a checkpatch warning:
"Line over 80 ch", I try not to exaggerate with this kind of warnings, but
I left a few.

If you think is worth it, I'll take care of it.
Thanks,
Marcel

>
> Applying: hw/acpi: remove from root bus 0 the crs resources used by
> other busses.
> === checkpatch complains ===
> WARNING: line over 80 characters
> #106: FILE: hw/i386/acpi-build.c:988:
> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> aml_max_fixed,
>
> WARNING: line over 80 characters
> #114: FILE: hw/i386/acpi-build.c:996:
> +                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> aml_max_fixed,
>
> total: 0 errors, 2 warnings, 124 lines checked
>
> cheers,
>    Gerd
>
>

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Michael S. Tsirkin]

On Wed, Mar 11, 2015 at 04:01:30PM +0200, Marcel Apfelbaum wrote:
> On 03/11/2015 03:51 PM, Gerd Hoffmann wrote:
> >On Mi, 2015-03-11 at 15:44 +0200, Marcel Apfelbaum wrote:
> >>On 03/11/2015 03:32 PM, Gerd Hoffmann wrote:
> >>>    Hi,
> >>>
> >>>>v4->v5:
> >>>>   - Rebased on pci branch, tree: git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
> >>>
> >>>Have again trouble applying this.  Which commit hash?  Or do you have a
> >>>git tree somewhere with this?
> >>Yes, it is changing too fast :(
> >>This version works with commit: a3b66ab
> >
> >Applies fine, but throws a checkpatch warning:
> "Line over 80 ch", I try not to exaggerate with this kind of warnings, but
> I left a few.
> 
> If you think is worth it, I'll take care of it.
> Thanks,
> Marcel

Probably easier than argue about it.

> >
> >Applying: hw/acpi: remove from root bus 0 the crs resources used by
> >other busses.
> >=== checkpatch complains ===
> >WARNING: line over 80 characters
> >#106: FILE: hw/i386/acpi-build.c:988:
> >+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >aml_max_fixed,
> >
> >WARNING: line over 80 characters
> >#114: FILE: hw/i386/acpi-build.c:996:
> >+                    aml_dword_memory(aml_pos_decode, aml_min_fixed,
> >aml_max_fixed,
> >
> >total: 0 errors, 2 warnings, 124 lines checked
> >
> >cheers,
> >   Gerd
> >
> >

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Gerd Hoffmann]

On Di, 2015-03-10 at 17:31 +0200, Marcel Apfelbaum wrote:
> v4->v5:
>  - Rebased on pci branch, tree:
> git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
>  - Added PXB documentation (patch 28/28)
>  - Addressed Gerd Hoffmann's review:
>    - fix PXB behaviour if used with unsupported BIOS (patch 27/28)
>  - Addressed Michael S. Tsirkin's review:
>    - Removed assert in aml_index (patch 5/28)
>    - Renamed pci_ functions to crs_ (patch 12/28)
>    - used uint64_t variables instead of signed ones (patch 12/28)
>    - Emit MEM/IO AML only for PXBs and i440fx (patch 26/28)
>  - Addressed Shannon Zhao's review:
>    - Changed build_append_int to  build_append_byte in aml_or (patch
> 2/25)
>  - Thanks to Igor and Kevin for reviews

Tested-by: Gerd Hoffmann <kraxel@gmail.com>

Possible improvement:  When you figure the devices behind the pxb are
not initialized by the firmware (i.e. unmapped), you can try grab some
address space not used by devices under root bus 0 and assign it to the
pxb.  Then the linux kernel can initialize the devices even if the
firmware did not.

[ Surely should be done incremental like hotplug support to not delay
  this series even more ]

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v5 for-2.3 00/28] hw/pc: implement multiple primary busses for pc machines

[Marcel Apfelbaum]

On 03/11/2015 04:12 PM, Gerd Hoffmann wrote:
> On Di, 2015-03-10 at 17:31 +0200, Marcel Apfelbaum wrote:
>> v4->v5:
>>   - Rebased on pci branch, tree:
>> git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git
>>   - Added PXB documentation (patch 28/28)
>>   - Addressed Gerd Hoffmann's review:
>>     - fix PXB behaviour if used with unsupported BIOS (patch 27/28)
>>   - Addressed Michael S. Tsirkin's review:
>>     - Removed assert in aml_index (patch 5/28)
>>     - Renamed pci_ functions to crs_ (patch 12/28)
>>     - used uint64_t variables instead of signed ones (patch 12/28)
>>     - Emit MEM/IO AML only for PXBs and i440fx (patch 26/28)
>>   - Addressed Shannon Zhao's review:
>>     - Changed build_append_int to  build_append_byte in aml_or (patch
>> 2/25)
>>   - Thanks to Igor and Kevin for reviews
>
Hi Gerd,

> Tested-by: Gerd Hoffmann <kraxel@gmail.com>
Appreciated!

>
> Possible improvement:  When you figure the devices behind the pxb are
> not initialized by the firmware (i.e. unmapped), you can try grab some
> address space not used by devices under root bus 0 and assign it to the
> pxb.  Then the linux kernel can initialize the devices even if the
> firmware did not.


Thanks for the idea, it is on my todo list after I'll make hotplug work for PXB buses.
Marcel

>
> [ Surely should be done incremental like hotplug support to not delay
>    this series even more ]
>
> cheers,
>    Gerd
>
>

Re: [Qemu-devel] [PATCH v5 for-2.3 23/28] hw/pxb: add map_irq func

[Marcel Apfelbaum]

On 03/10/2015 06:43 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 05:32:09PM +0200, Marcel Apfelbaum wrote:
>> The bios does not index the pxb slot number when
>> it computes the IRQ because it resides on bus 0
>> and not on the current bus.
>> However Qemu routes the irq through bus 0 and adds
>> the pxb slot to the IRQ computation.
>>
>> Synchronize between bios and Qemu by canceling
>> pxb's effect.
>>
>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>> ---
>>   hw/pci-bridge/pci_expander_bridge.c | 20 +++++++++++++++++++-
>>   1 file changed, 19 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
>> index 941f3c8..87515c1 100644
>> --- a/hw/pci-bridge/pci_expander_bridge.c
>> +++ b/hw/pci-bridge/pci_expander_bridge.c
>> @@ -92,6 +92,24 @@ static const TypeInfo pxb_host_info = {
>>       .class_init    = pxb_host_class_init,
>>   };
>>
>> +
>> +static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
>> +{
>> +    PCIDevice *pxb = pci_dev->bus->parent_dev;
>> +
>> +    /*
>> +     * The bios does not index the pxb slot number when
>> +     * it computes the IRQ because it resides on bus 0
>> +     * and not on the current bus.
>> +     * However QEMU routes the irq through bus 0 and adds
>> +     * the pxb slot to the IRQ computation.
>
> I know it's QEMU but which function exactly?
PXB device with is a PCI host-bridge device.

Thanks,
Marcel

>
>> +     *
>> +     * Synchronize between bios and QEMU by canceling
>> +     * pxb's effect.
>> +     */
>> +    return pin - PCI_SLOT(pxb->devfn);
>> +}
>> +
>>   static int pxb_dev_initfn(PCIDevice *dev)
>>   {
>>       PXBDev *pxb = PXB_DEV(dev);
>> @@ -118,7 +136,7 @@ static int pxb_dev_initfn(PCIDevice *dev)
>>       bus->parent_dev = dev;
>>       bus->address_space_mem = dev->bus->address_space_mem;
>>       bus->address_space_io = dev->bus->address_space_io;
>> -    bus->map_irq = pci_swizzle_map_irq_fn;
>> +    bus->map_irq = pxb_map_irq_fn;
>>
>>       bds = qdev_create(BUS(bus), "pci-bridge");
>>       bds->id = dev_name;
>> --
>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Marcel Apfelbaum]

On 03/10/2015 07:42 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
>> On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
>>> On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
>>>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>>>> ---
>>>>   docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
>>>>   1 file changed, 52 insertions(+)
>>>>   create mode 100644 docs/pci_expander_bridge.txt
>>>>
>>>> diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
>>>> new file mode 100644
>>>> index 0000000..58bf7a8
>>>> --- /dev/null
>>>> +++ b/docs/pci_expander_bridge.txt
>>>> @@ -0,0 +1,52 @@
>>>> +PCI EXPANDER BRIDGE (PXB)
>>>> +=========================
>>>> +
>>>> +Description
>>>> +===========
>>>> +PXB is a "light-weight" host bridge in the same PCI domain
>>>> +as the main host bridge whose purpose is to enable
>>>> +the main host bridge to support multiple PCI root buses.
>>>> +It is implemented only for i440fx.
>>>
>>> BTW what makes it i440fx specific?
>>> Also, what happens if you try to use it
>>> with a different machine type?
>> Is is i440fx specific, please look at patch 22/28.
>> Also we have a specific check for i440fx, so CRS
>> will not be emitted for other machine types.
>>
>> Thanks,
>> Marcel
>
> In fact it won't work at all. Need to think about it,
> maybe we can make it work more generally.
> For CRS, should be possible to emit for q35 too?
We can make it work, but not on the scope of this series.
However, I'll add a IHostBridgeSnoop interface that will
make the device work only with associated bus and
this will make it less general.

Thanks,
Marcel

>
>>>
>>>> +
>>>> +As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
>>>> +is a primary bus and can be associated with a NUMA node
>>>> +(different from the main host bridge) allowing the guest OS
>>>> +to recognize the proximity of a pass-through device to
>>>> +other resources as RAM and CPUs.
>>>> +
>>>> +Usage
>>>> +=====
>>>> +A detailed command line would be:
>>>> +
>>>> +[qemu-bin + storage options]
>>>> +-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
>>>> +-m 2G
>>>> +-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
>>>> +-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
>>>> +-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
>>>> +-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
>>>> +-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
>>>> +
>>>> +Here you have:
>>>> + - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
>>>> + - a pxb host bridge attached to NUMA 1 with an e1000 behind it
>>>> + - a pxb host bridge attached to NUMA 0 with an e1000 behind it
>>>> + - a pxb host bridge not attached to any NUMA with a hard drive behind it.
>>>> +
>>>> +Implementation
>>>> +==============
>>>> +The PXB is composed by:
>>>> +- HostBridge (TYPE_PXB_HOST)
>>>> +  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
>>>> +- PXBDev(TYPE_PXB_DEVICE)
>>>> +  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
>>>> +  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
>>>> +  The interrupts from devices behind the PXB are routed through this device the same as if it were a
>>>> +  PCI-2-PCI bridge. The _PRT follows the i440fx model.
>>>> +- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
>>>> +  Created automatically as part of init sequence.
>>>> +  When adding a device to PXB it is attached to the bridge for two reasons:
>>>> +  - Using the bridge will enable hotplug support
>>>> +  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
>>>> +    the PCI address space.
>>>> +
>>>> --
>>>> 2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 23/28] hw/pxb: add map_irq func

[Michael S. Tsirkin]

On Mon, Mar 16, 2015 at 02:11:45PM +0200, Marcel Apfelbaum wrote:
> On 03/10/2015 06:43 PM, Michael S. Tsirkin wrote:
> >On Tue, Mar 10, 2015 at 05:32:09PM +0200, Marcel Apfelbaum wrote:
> >>The bios does not index the pxb slot number when
> >>it computes the IRQ because it resides on bus 0
> >>and not on the current bus.
> >>However Qemu routes the irq through bus 0 and adds
> >>the pxb slot to the IRQ computation.
> >>
> >>Synchronize between bios and Qemu by canceling
> >>pxb's effect.
> >>
> >>Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> >>---
> >>  hw/pci-bridge/pci_expander_bridge.c | 20 +++++++++++++++++++-
> >>  1 file changed, 19 insertions(+), 1 deletion(-)
> >>
> >>diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
> >>index 941f3c8..87515c1 100644
> >>--- a/hw/pci-bridge/pci_expander_bridge.c
> >>+++ b/hw/pci-bridge/pci_expander_bridge.c
> >>@@ -92,6 +92,24 @@ static const TypeInfo pxb_host_info = {
> >>      .class_init    = pxb_host_class_init,
> >>  };
> >>
> >>+
> >>+static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
> >>+{
> >>+    PCIDevice *pxb = pci_dev->bus->parent_dev;
> >>+
> >>+    /*
> >>+     * The bios does not index the pxb slot number when
> >>+     * it computes the IRQ because it resides on bus 0
> >>+     * and not on the current bus.
> >>+     * However QEMU routes the irq through bus 0 and adds
> >>+     * the pxb slot to the IRQ computation.
> >
> >I know it's QEMU but which function exactly?
> PXB device with is a PCI host-bridge device.
> 
> Thanks,
> Marcel

right, so pls tweak the comment to say so.

> >
> >>+     *
> >>+     * Synchronize between bios and QEMU by canceling
> >>+     * pxb's effect.
> >>+     */
> >>+    return pin - PCI_SLOT(pxb->devfn);
> >>+}
> >>+
> >>  static int pxb_dev_initfn(PCIDevice *dev)
> >>  {
> >>      PXBDev *pxb = PXB_DEV(dev);
> >>@@ -118,7 +136,7 @@ static int pxb_dev_initfn(PCIDevice *dev)
> >>      bus->parent_dev = dev;
> >>      bus->address_space_mem = dev->bus->address_space_mem;
> >>      bus->address_space_io = dev->bus->address_space_io;
> >>-    bus->map_irq = pci_swizzle_map_irq_fn;
> >>+    bus->map_irq = pxb_map_irq_fn;
> >>
> >>      bds = qdev_create(BUS(bus), "pci-bridge");
> >>      bds->id = dev_name;
> >>--
> >>2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Michael S. Tsirkin]

On Mon, Mar 16, 2015 at 02:16:40PM +0200, Marcel Apfelbaum wrote:
> On 03/10/2015 07:42 PM, Michael S. Tsirkin wrote:
> >On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
> >>On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
> >>>On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
> >>>>Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
> >>>>---
> >>>>  docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
> >>>>  1 file changed, 52 insertions(+)
> >>>>  create mode 100644 docs/pci_expander_bridge.txt
> >>>>
> >>>>diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
> >>>>new file mode 100644
> >>>>index 0000000..58bf7a8
> >>>>--- /dev/null
> >>>>+++ b/docs/pci_expander_bridge.txt
> >>>>@@ -0,0 +1,52 @@
> >>>>+PCI EXPANDER BRIDGE (PXB)
> >>>>+=========================
> >>>>+
> >>>>+Description
> >>>>+===========
> >>>>+PXB is a "light-weight" host bridge in the same PCI domain
> >>>>+as the main host bridge whose purpose is to enable
> >>>>+the main host bridge to support multiple PCI root buses.
> >>>>+It is implemented only for i440fx.
> >>>
> >>>BTW what makes it i440fx specific?
> >>>Also, what happens if you try to use it
> >>>with a different machine type?
> >>Is is i440fx specific, please look at patch 22/28.
> >>Also we have a specific check for i440fx, so CRS
> >>will not be emitted for other machine types.
> >>
> >>Thanks,
> >>Marcel
> >
> >In fact it won't work at all. Need to think about it,
> >maybe we can make it work more generally.
> >For CRS, should be possible to emit for q35 too?
> We can make it work, but not on the scope of this series.
> However, I'll add a IHostBridgeSnoop interface that will
> make the device work only with associated bus and
> this will make it less general.
> 
> Thanks,
> Marcel

OK, this works too. Do you mean PCIHostBridgeSnoop?

> >
> >>>
> >>>>+
> >>>>+As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
> >>>>+is a primary bus and can be associated with a NUMA node
> >>>>+(different from the main host bridge) allowing the guest OS
> >>>>+to recognize the proximity of a pass-through device to
> >>>>+other resources as RAM and CPUs.
> >>>>+
> >>>>+Usage
> >>>>+=====
> >>>>+A detailed command line would be:
> >>>>+
> >>>>+[qemu-bin + storage options]
> >>>>+-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
> >>>>+-m 2G
> >>>>+-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
> >>>>+-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
> >>>>+-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
> >>>>+-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
> >>>>+-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
> >>>>+
> >>>>+Here you have:
> >>>>+ - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
> >>>>+ - a pxb host bridge attached to NUMA 1 with an e1000 behind it
> >>>>+ - a pxb host bridge attached to NUMA 0 with an e1000 behind it
> >>>>+ - a pxb host bridge not attached to any NUMA with a hard drive behind it.
> >>>>+
> >>>>+Implementation
> >>>>+==============
> >>>>+The PXB is composed by:
> >>>>+- HostBridge (TYPE_PXB_HOST)
> >>>>+  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
> >>>>+- PXBDev(TYPE_PXB_DEVICE)
> >>>>+  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
> >>>>+  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
> >>>>+  The interrupts from devices behind the PXB are routed through this device the same as if it were a
> >>>>+  PCI-2-PCI bridge. The _PRT follows the i440fx model.
> >>>>+- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
> >>>>+  Created automatically as part of init sequence.
> >>>>+  When adding a device to PXB it is attached to the bridge for two reasons:
> >>>>+  - Using the bridge will enable hotplug support
> >>>>+  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
> >>>>+    the PCI address space.
> >>>>+
> >>>>--
> >>>>2.1.0

Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Marcel Apfelbaum]

On 03/16/2015 05:28 PM, Michael S. Tsirkin wrote:
> On Mon, Mar 16, 2015 at 02:16:40PM +0200, Marcel Apfelbaum wrote:
>> On 03/10/2015 07:42 PM, Michael S. Tsirkin wrote:
>>> On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
>>>> On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
>>>>> On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
>>>>>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>>>>>> ---
>>>>>>   docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
>>>>>>   1 file changed, 52 insertions(+)
>>>>>>   create mode 100644 docs/pci_expander_bridge.txt
>>>>>>
>>>>>> diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
>>>>>> new file mode 100644
>>>>>> index 0000000..58bf7a8
>>>>>> --- /dev/null
>>>>>> +++ b/docs/pci_expander_bridge.txt
>>>>>> @@ -0,0 +1,52 @@
>>>>>> +PCI EXPANDER BRIDGE (PXB)
>>>>>> +=========================
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +PXB is a "light-weight" host bridge in the same PCI domain
>>>>>> +as the main host bridge whose purpose is to enable
>>>>>> +the main host bridge to support multiple PCI root buses.
>>>>>> +It is implemented only for i440fx.
>>>>>
>>>>> BTW what makes it i440fx specific?
>>>>> Also, what happens if you try to use it
>>>>> with a different machine type?
>>>> Is is i440fx specific, please look at patch 22/28.
>>>> Also we have a specific check for i440fx, so CRS
>>>> will not be emitted for other machine types.
>>>>
>>>> Thanks,
>>>> Marcel
>>>
>>> In fact it won't work at all. Need to think about it,
>>> maybe we can make it work more generally.
>>> For CRS, should be possible to emit for q35 too?
>> We can make it work, but not on the scope of this series.
>> However, I'll add a IHostBridgeSnoop interface that will
>> make the device work only with associated bus and
>> this will make it less general.
>>
>> Thanks,
>> Marcel
>
> OK, this works too. Do you mean PCIHostBridgeSnoop?
Sure.

Thanks,
Marcel

>
>>>
>>>>>
>>>>>> +
>>>>>> +As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
>>>>>> +is a primary bus and can be associated with a NUMA node
>>>>>> +(different from the main host bridge) allowing the guest OS
>>>>>> +to recognize the proximity of a pass-through device to
>>>>>> +other resources as RAM and CPUs.
>>>>>> +
>>>>>> +Usage
>>>>>> +=====
>>>>>> +A detailed command line would be:
>>>>>> +
>>>>>> +[qemu-bin + storage options]
>>>>>> +-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
>>>>>> +-m 2G
>>>>>> +-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
>>>>>> +-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
>>>>>> +-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
>>>>>> +-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
>>>>>> +-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
>>>>>> +
>>>>>> +Here you have:
>>>>>> + - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
>>>>>> + - a pxb host bridge attached to NUMA 1 with an e1000 behind it
>>>>>> + - a pxb host bridge attached to NUMA 0 with an e1000 behind it
>>>>>> + - a pxb host bridge not attached to any NUMA with a hard drive behind it.
>>>>>> +
>>>>>> +Implementation
>>>>>> +==============
>>>>>> +The PXB is composed by:
>>>>>> +- HostBridge (TYPE_PXB_HOST)
>>>>>> +  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
>>>>>> +- PXBDev(TYPE_PXB_DEVICE)
>>>>>> +  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
>>>>>> +  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
>>>>>> +  The interrupts from devices behind the PXB are routed through this device the same as if it were a
>>>>>> +  PCI-2-PCI bridge. The _PRT follows the i440fx model.
>>>>>> +- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
>>>>>> +  Created automatically as part of init sequence.
>>>>>> +  When adding a device to PXB it is attached to the bridge for two reasons:
>>>>>> +  - Using the bridge will enable hotplug support
>>>>>> +  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
>>>>>> +    the PCI address space.
>>>>>> +
>>>>>> --
>>>>>> 2.1.0