All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling
@ 2016-10-21  1:19 David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 01/11] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
                   ` (11 more replies)
  0 siblings, 12 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

This series contains a number of cleanups to the libqos code for
accessing PCI devices, and to tests which use it.

The general aim is to improve the consistency of semantics across
functions, and reduce the amount of intimate knowledge of the libqos
PCI layer needed by tests.

This should make it easier to write PCI tests which will be portable
to different guest machines with different PCI host bridge
arrangements.

This series is on top of my ppc-for-2.8 branch, since it contains
patches enabling the virtio tests on ppc, which would otherwise
conflict with the changes here.

Changes since v3:
  * Fixed another endian bug introduced in ide-test

Changes since v2:
  * Fixed build bugs in virtio-9p-test, which I didn't find earlier
    due to not having the right libraries installed
  * Fixed an endian bug I accidentally introduced in ide-test
  * Better handling of invalid BAR tokens

Changes since v1:
  * Split out updates to tco-test into separate patch
  * Split out updates to ide-test into separate patch
  * Neater and more general handling of legacy PIO addresses
  * Removed now-redundant fields from platform specific bus structures
  * Introduced CONFIG_BASE() macro to virtio-pci to remove many
    similar assignments
  * Fixed handling of two guest testcasesin ivshmem
  * Added 64-bit accessors
  * Rebase on ppc-for-2.8 to avoid conflict with Laurent's series in
    the same area


*** BLURB HERE ***

David Gibson (11):
  libqos: Give qvirtio_config_read*() consistent semantics
  libqos: Handle PCI IO de-multiplexing in common code
  libqos: Move BAR assignment to common code
  libqos: Better handling of PCI legacy IO
  tests: Adjust tco-test to use qpci_legacy_iomap()
  libqos: Add streaming accessors for PCI MMIO
  libqos: Implement mmio accessors in terms of mem{read,write}
  tests: Clean up IO handling in ide-test
  libqos: Add 64-bit PCI IO accessors
  tests: Use qpci_mem{read,write} in ivshmem-test
  libqos: Change PCI accessors to take opaque BAR handle

 tests/ahci-test.c          |   4 +-
 tests/e1000e-test.c        |   7 +-
 tests/ide-test.c           | 177 ++++++++++++++++++++++++++--------------
 tests/ivshmem-test.c       |  47 +++++++----
 tests/libqos/ahci.c        |   3 +-
 tests/libqos/ahci.h        |   6 +-
 tests/libqos/pci-pc.c      | 187 ++++++++++--------------------------------
 tests/libqos/pci-spapr.c   | 194 ++++++++++++--------------------------------
 tests/libqos/pci.c         | 196 +++++++++++++++++++++++++++++++++++++--------
 tests/libqos/pci.h         |  70 +++++++++++-----
 tests/libqos/usb.c         |   6 +-
 tests/libqos/usb.h         |   2 +-
 tests/libqos/virtio-mmio.c |  16 ++--
 tests/libqos/virtio-pci.c  | 123 ++++++++++++++--------------
 tests/libqos/virtio-pci.h  |   2 +-
 tests/rtl8139-test.c       |  10 +--
 tests/tco-test.c           |  80 +++++++++---------
 tests/usb-hcd-ehci-test.c  |   5 +-
 tests/virtio-9p-test.c     |   8 +-
 tests/virtio-blk-test.c    |  42 +++-------
 tests/virtio-scsi-test.c   |   4 +-
 21 files changed, 598 insertions(+), 591 deletions(-)

-- 
2.7.4

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 01/11] libqos: Give qvirtio_config_read*() consistent semantics
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 02/11] libqos: Handle PCI IO de-multiplexing in common code David Gibson
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The 'addr' parameter to qvirtio_config_read*() doesn't have a consistent
meaning: when using the virtio-pci versions, it's a full PCI space address,
but for virtio-mmio, it's an offset from the device's base mmio address.

This means that the callers need to do different things to calculate the
addresses in the two cases, which rather defeats the purpose of function
pointer backends.

All the current users of these functions are using them to retrieve
variables from the device specific portion of the virtio config space.
So, this patch alters the semantics to always be an offset into that
device specific config area.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/virtio-mmio.c | 16 ++++++++--------
 tests/libqos/virtio-pci.c  | 25 ++++++++++++++-----------
 tests/virtio-9p-test.c     |  8 ++------
 tests/virtio-blk-test.c    | 42 +++++++++++-------------------------------
 tests/virtio-scsi-test.c   |  4 +---
 5 files changed, 36 insertions(+), 59 deletions(-)

diff --git a/tests/libqos/virtio-mmio.c b/tests/libqos/virtio-mmio.c
index bced680..7aa8383 100644
--- a/tests/libqos/virtio-mmio.c
+++ b/tests/libqos/virtio-mmio.c
@@ -15,28 +15,28 @@
 #include "libqos/malloc-generic.h"
 #include "standard-headers/linux/virtio_ring.h"
 
-static uint8_t qvirtio_mmio_config_readb(QVirtioDevice *d, uint64_t addr)
+static uint8_t qvirtio_mmio_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readb(dev->addr + addr);
+    return readb(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint16_t qvirtio_mmio_config_readw(QVirtioDevice *d, uint64_t addr)
+static uint16_t qvirtio_mmio_config_readw(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readw(dev->addr + addr);
+    return readw(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint32_t qvirtio_mmio_config_readl(QVirtioDevice *d, uint64_t addr)
+static uint32_t qvirtio_mmio_config_readl(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readl(dev->addr + addr);
+    return readl(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint64_t qvirtio_mmio_config_readq(QVirtioDevice *d, uint64_t addr)
+static uint64_t qvirtio_mmio_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readq(dev->addr + addr);
+    return readq(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
 static uint32_t qvirtio_mmio_get_features(QVirtioDevice *d)
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index 7e60b3a..fa82132 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -62,10 +62,13 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
     *vpcidev = (QVirtioPCIDevice *)d;
 }
 
-static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t addr)
+#define CONFIG_BASE(dev) \
+    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
+
+static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, (void *)(uintptr_t)addr);
+    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
 }
 
 /* PCI is always read in little-endian order
@@ -76,31 +79,31 @@ static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t addr)
  * case will be managed inside qvirtio_is_big_endian()
  */
 
-static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t addr)
+static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint16_t value;
 
-    value = qpci_io_readw(dev->pdev, (void *)(uintptr_t)addr);
+    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap16(value);
     }
     return value;
 }
 
-static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t addr)
+static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint32_t value;
 
-    value = qpci_io_readl(dev->pdev, (void *)(uintptr_t)addr);
+    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap32(value);
     }
     return value;
 }
 
-static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t addr)
+static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     int i;
@@ -108,13 +111,13 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t addr)
 
     if (qvirtio_is_big_endian(d)) {
         for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev,
-                                (void *)(uintptr_t)addr + i) << (7 - i) * 8;
+            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
+                                           + off + i) << (7 - i) * 8;
         }
     } else {
         for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev,
-                                (void *)(uintptr_t)addr + i) << i * 8;
+            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
+                                           + off + i) << i * 8;
         }
     }
 
diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c
index 693920a..9c4f6cb 100644
--- a/tests/virtio-9p-test.c
+++ b/tests/virtio-9p-test.c
@@ -95,7 +95,6 @@ static void qvirtio_9p_pci_free(QVirtIO9P *v9p)
 static void pci_basic_config(void)
 {
     QVirtIO9P *v9p;
-    void *addr;
     size_t tag_len;
     char *tag;
     int i;
@@ -104,15 +103,12 @@ static void pci_basic_config(void)
     qs = qvirtio_9p_start();
     v9p = qvirtio_9p_pci_init(qs);
 
-    addr = ((QVirtioPCIDevice *) v9p->dev)->addr + VIRTIO_PCI_CONFIG_OFF(false);
-    tag_len = qvirtio_config_readw(v9p->dev,
-                                   (uint64_t)(uintptr_t)addr);
+    tag_len = qvirtio_config_readw(v9p->dev, 0);
     g_assert_cmpint(tag_len, ==, strlen(mount_tag));
-    addr += sizeof(uint16_t);
 
     tag = g_malloc(tag_len);
     for (i = 0; i < tag_len; i++) {
-        tag[i] = qvirtio_config_readb(v9p->dev, (uint64_t)(uintptr_t)addr + i);
+        tag[i] = qvirtio_config_readb(v9p->dev, i + 2);
     }
     g_assert_cmpmem(tag, tag_len, mount_tag, tag_len);
     g_free(tag);
diff --git a/tests/virtio-blk-test.c b/tests/virtio-blk-test.c
index f737c40..0e32e41 100644
--- a/tests/virtio-blk-test.c
+++ b/tests/virtio-blk-test.c
@@ -155,7 +155,7 @@ static uint64_t virtio_blk_request(QGuestAllocator *alloc, QVirtioDevice *d,
 }
 
 static void test_basic(QVirtioDevice *dev, QGuestAllocator *alloc,
-                       QVirtQueue *vq, uint64_t device_specific)
+                       QVirtQueue *vq)
 {
     QVirtioBlkReq req;
     uint64_t req_addr;
@@ -165,7 +165,7 @@ static void test_basic(QVirtioDevice *dev, QGuestAllocator *alloc,
     uint8_t status;
     char *data;
 
-    capacity = qvirtio_config_readq(dev, device_specific);
+    capacity = qvirtio_config_readq(dev, 0);
 
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
@@ -285,17 +285,13 @@ static void pci_basic(void)
     QVirtioPCIDevice *dev;
     QOSState *qs;
     QVirtQueuePCI *vqpci;
-    void *addr;
 
     qs = pci_test_start();
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
     vqpci = (QVirtQueuePCI *)qvirtqueue_setup(&dev->vdev, qs->alloc, 0);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    test_basic(&dev->vdev, qs->alloc, &vqpci->vq, (uint64_t)(uintptr_t)addr);
+    test_basic(&dev->vdev, qs->alloc, &vqpci->vq);
 
     /* End test */
     qvirtqueue_cleanup(dev->vdev.bus, &vqpci->vq, qs->alloc);
@@ -311,7 +307,6 @@ static void pci_indirect(void)
     QOSState *qs;
     QVirtioBlkReq req;
     QVRingIndirectDesc *indirect;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -323,10 +318,7 @@ static void pci_indirect(void)
 
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -406,17 +398,13 @@ static void pci_config(void)
     QVirtioPCIDevice *dev;
     QOSState *qs;
     int n_size = TEST_IMAGE_SIZE / 2;
-    void *addr;
     uint64_t capacity;
 
     qs = pci_test_start();
 
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     qvirtio_set_driver_ok(&dev->vdev);
@@ -425,7 +413,7 @@ static void pci_config(void)
                                                     " 'size': %d } }", n_size);
     qvirtio_wait_config_isr(&dev->vdev, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     qvirtio_pci_device_disable(dev);
@@ -441,7 +429,6 @@ static void pci_msix(void)
     QVirtQueuePCI *vqpci;
     QVirtioBlkReq req;
     int n_size = TEST_IMAGE_SIZE / 2;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -456,10 +443,7 @@ static void pci_msix(void)
 
     qvirtio_pci_set_msix_configuration_vector(dev, qs->alloc, 0);
 
-    /* MSI-X is enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(true);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -479,7 +463,7 @@ static void pci_msix(void)
 
     qvirtio_wait_config_isr(&dev->vdev, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, (uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     /* Write request */
@@ -550,7 +534,6 @@ static void pci_idx(void)
     QOSState *qs;
     QVirtQueuePCI *vqpci;
     QVirtioBlkReq req;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -565,10 +548,7 @@ static void pci_idx(void)
 
     qvirtio_pci_set_msix_configuration_vector(dev, qs->alloc, 0);
 
-    /* MSI-X is enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(true);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -709,14 +689,14 @@ static void mmio_basic(void)
     alloc = generic_alloc_init(MMIO_RAM_ADDR, MMIO_RAM_SIZE, MMIO_PAGE_SIZE);
     vq = qvirtqueue_setup(&dev->vdev, alloc, 0);
 
-    test_basic(&dev->vdev, alloc, vq, QVIRTIO_MMIO_DEVICE_SPECIFIC);
+    test_basic(&dev->vdev, alloc, vq);
 
     qmp("{ 'execute': 'block_resize', 'arguments': { 'device': 'drive0', "
                                                     " 'size': %d } }", n_size);
 
     qvirtio_wait_queue_isr(&dev->vdev, vq, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, QVIRTIO_MMIO_DEVICE_SPECIFIC);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     /* End test */
diff --git a/tests/virtio-scsi-test.c b/tests/virtio-scsi-test.c
index 60dc9ab..69220ef 100644
--- a/tests/virtio-scsi-test.c
+++ b/tests/virtio-scsi-test.c
@@ -143,7 +143,6 @@ static QVirtIOSCSI *qvirtio_scsi_pci_init(int slot)
     QVirtIOSCSI *vs;
     QVirtioPCIDevice *dev;
     struct virtio_scsi_cmd_resp resp;
-    void *addr;
     int i;
 
     vs = g_new0(QVirtIOSCSI, 1);
@@ -161,8 +160,7 @@ static QVirtIOSCSI *qvirtio_scsi_pci_init(int slot)
     qvirtio_set_acknowledge(vs->dev);
     qvirtio_set_driver(vs->dev);
 
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-    vs->num_queues = qvirtio_config_readl(vs->dev, (uint64_t)(uintptr_t)addr);
+    vs->num_queues = qvirtio_config_readl(vs->dev, 0);
 
     g_assert_cmpint(vs->num_queues, <, MAX_NUM_QUEUES);
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 02/11] libqos: Handle PCI IO de-multiplexing in common code
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 01/11] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 03/11] libqos: Move BAR assignment to " David Gibson
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The PCI IO space (aka PIO, aka legacy IO) and PCI memory space (aka MMIO)
are distinct address spaces by the PCI spec (although parts of one might be
aliased to parts of the other in some cases).

However, qpci_io_read*() and qpci_io_write*() can perform accesses to
either space depending on parameter.  That's convenient for test case
drivers, since there are a fair few devices which can be controlled via
either a PIO or MMIO BAR but with an otherwise identical driver.

This is implemented by having addresses below 64kiB treated as PIO, and
those above treated as MMIO.  This works because low addresses in memory
space are generally reserved for DMA rather than MMIO.

At the moment, this demultiplexing must be handled by each PCI backend
(pc and spapr, so far).  There's no real reason for this - the current
encoding is likely to work for all platforms, and even if it doesn't we
can still use a more complex common encoding since the value returned from
iomap are semi-opaque.

This patch moves the demultiplexing into the common part of the libqos PCI
code, with the backends having simpler, separate accessors for PIO and
MMIO space.  This also means we have a way of explicitly accessing either
space if it's necessary for some special case.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 107 ++++++++++++++++++++----------------------
 tests/libqos/pci-spapr.c | 118 +++++++++++++++++++++++++----------------------
 tests/libqos/pci.c       |  49 +++++++++++++++++---
 tests/libqos/pci.h       |  22 ++++++---
 4 files changed, 170 insertions(+), 126 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 9600ed6..51dff8a 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -36,79 +36,64 @@ typedef struct QPCIBusPC
     uint16_t pci_iohole_alloc;
 } QPCIBusPC;
 
-static uint8_t qpci_pc_io_readb(QPCIBus *bus, void *addr)
+static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint8_t value;
-
-    if (port < 0x10000) {
-        value = inb(port);
-    } else {
-        value = readb(port);
-    }
-
-    return value;
+    return inb(addr);
 }
 
-static uint16_t qpci_pc_io_readw(QPCIBus *bus, void *addr)
+static uint8_t qpci_pc_mmio_readb(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint16_t value;
-
-    if (port < 0x10000) {
-        value = inw(port);
-    } else {
-        value = readw(port);
-    }
+    return readb(addr);
+}
 
-    return value;
+static void qpci_pc_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
+{
+    outb(addr, val);
 }
 
-static uint32_t qpci_pc_io_readl(QPCIBus *bus, void *addr)
+static void qpci_pc_mmio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint32_t value;
+    writeb(addr, val);
+}
 
-    if (port < 0x10000) {
-        value = inl(port);
-    } else {
-        value = readl(port);
-    }
+static uint16_t qpci_pc_pio_readw(QPCIBus *bus, uint32_t addr)
+{
+    return inw(addr);
+}
 
-    return value;
+static uint16_t qpci_pc_mmio_readw(QPCIBus *bus, uint32_t addr)
+{
+    return readw(addr);
 }
 
-static void qpci_pc_io_writeb(QPCIBus *bus, void *addr, uint8_t value)
+static void qpci_pc_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
+    outw(addr, val);
+}
 
-    if (port < 0x10000) {
-        outb(port, value);
-    } else {
-        writeb(port, value);
-    }
+static void qpci_pc_mmio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    writew(addr, val);
 }
 
-static void qpci_pc_io_writew(QPCIBus *bus, void *addr, uint16_t value)
+static uint32_t qpci_pc_pio_readl(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
+    return inl(addr);
+}
 
-    if (port < 0x10000) {
-        outw(port, value);
-    } else {
-        writew(port, value);
-    }
+static uint32_t qpci_pc_mmio_readl(QPCIBus *bus, uint32_t addr)
+{
+    return readl(addr);
 }
 
-static void qpci_pc_io_writel(QPCIBus *bus, void *addr, uint32_t value)
+static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
+    outl(addr, val);
+}
 
-    if (port < 0x10000) {
-        outl(port, value);
-    } else {
-        writel(port, value);
-    }
+static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    writel(addr, val);
 }
 
 static uint8_t qpci_pc_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
@@ -218,13 +203,21 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
 
     ret = g_malloc(sizeof(*ret));
 
-    ret->bus.io_readb = qpci_pc_io_readb;
-    ret->bus.io_readw = qpci_pc_io_readw;
-    ret->bus.io_readl = qpci_pc_io_readl;
+    ret->bus.pio_readb = qpci_pc_pio_readb;
+    ret->bus.pio_readw = qpci_pc_pio_readw;
+    ret->bus.pio_readl = qpci_pc_pio_readl;
+
+    ret->bus.pio_writeb = qpci_pc_pio_writeb;
+    ret->bus.pio_writew = qpci_pc_pio_writew;
+    ret->bus.pio_writel = qpci_pc_pio_writel;
+
+    ret->bus.mmio_readb = qpci_pc_mmio_readb;
+    ret->bus.mmio_readw = qpci_pc_mmio_readw;
+    ret->bus.mmio_readl = qpci_pc_mmio_readl;
 
-    ret->bus.io_writeb = qpci_pc_io_writeb;
-    ret->bus.io_writew = qpci_pc_io_writew;
-    ret->bus.io_writel = qpci_pc_io_writel;
+    ret->bus.mmio_writeb = qpci_pc_mmio_writeb;
+    ret->bus.mmio_writew = qpci_pc_mmio_writew;
+    ret->bus.mmio_writel = qpci_pc_mmio_writel;
 
     ret->bus.config_readb = qpci_pc_config_readb;
     ret->bus.config_readw = qpci_pc_config_readw;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 2eaaf91..2d26a94 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -50,78 +50,76 @@ typedef struct QPCIBusSPAPR {
  * so PCI accessors need to swap data endianness
  */
 
-static uint8_t qpci_spapr_io_readb(QPCIBus *bus, void *addr)
+static uint8_t qpci_spapr_pio_readb(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint8_t v;
-    if (port < s->pio.size) {
-        v = readb(s->pio_cpu_base + port);
-    } else {
-        v = readb(s->mmio32_cpu_base + port);
-    }
-    return v;
+    return readb(s->pio_cpu_base + addr);
 }
 
-static uint16_t qpci_spapr_io_readw(QPCIBus *bus, void *addr)
+static uint8_t qpci_spapr_mmio32_readb(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint16_t v;
-    if (port < s->pio.size) {
-        v = readw(s->pio_cpu_base + port);
-    } else {
-        v = readw(s->mmio32_cpu_base + port);
-    }
-    return bswap16(v);
+    return readb(s->mmio32_cpu_base + addr);
 }
 
-static uint32_t qpci_spapr_io_readl(QPCIBus *bus, void *addr)
+static void qpci_spapr_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint32_t v;
-    if (port < s->pio.size) {
-        v = readl(s->pio_cpu_base + port);
-    } else {
-        v = readl(s->mmio32_cpu_base + port);
-    }
-    return bswap32(v);
+    writeb(s->pio_cpu_base + addr, val);
 }
 
-static void qpci_spapr_io_writeb(QPCIBus *bus, void *addr, uint8_t value)
+static void qpci_spapr_mmio32_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    if (port < s->pio.size) {
-        writeb(s->pio_cpu_base + port, value);
-    } else {
-        writeb(s->mmio32_cpu_base + port, value);
-    }
+    writeb(s->mmio32_cpu_base + addr, val);
 }
 
-static void qpci_spapr_io_writew(QPCIBus *bus, void *addr, uint16_t value)
+static uint16_t qpci_spapr_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    value = bswap16(value);
-    if (port < s->pio.size) {
-        writew(s->pio_cpu_base + port, value);
-    } else {
-        writew(s->mmio32_cpu_base + port, value);
-    }
+    return bswap16(readw(s->pio_cpu_base + addr));
 }
 
-static void qpci_spapr_io_writel(QPCIBus *bus, void *addr, uint32_t value)
+static uint16_t qpci_spapr_mmio32_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    value = bswap32(value);
-    if (port < s->pio.size) {
-        writel(s->pio_cpu_base + port, value);
-    } else {
-        writel(s->mmio32_cpu_base + port, value);
-    }
+    return bswap16(readw(s->mmio32_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writew(s->pio_cpu_base + addr, bswap16(val));
+}
+
+static void qpci_spapr_mmio32_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writew(s->mmio32_cpu_base + addr, bswap16(val));
+}
+
+static uint32_t qpci_spapr_pio_readl(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap32(readl(s->pio_cpu_base + addr));
+}
+
+static uint32_t qpci_spapr_mmio32_readl(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap32(readl(s->mmio32_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writel(s->pio_cpu_base + addr, bswap32(val));
+}
+
+static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writel(s->mmio32_cpu_base + addr, bswap32(val));
 }
 
 static uint8_t qpci_spapr_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
@@ -248,13 +246,21 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
 
     ret->alloc = alloc;
 
-    ret->bus.io_readb = qpci_spapr_io_readb;
-    ret->bus.io_readw = qpci_spapr_io_readw;
-    ret->bus.io_readl = qpci_spapr_io_readl;
+    ret->bus.pio_readb = qpci_spapr_pio_readb;
+    ret->bus.pio_readw = qpci_spapr_pio_readw;
+    ret->bus.pio_readl = qpci_spapr_pio_readl;
+
+    ret->bus.pio_writeb = qpci_spapr_pio_writeb;
+    ret->bus.pio_writew = qpci_spapr_pio_writew;
+    ret->bus.pio_writel = qpci_spapr_pio_writel;
+
+    ret->bus.mmio_readb = qpci_spapr_mmio32_readb;
+    ret->bus.mmio_readw = qpci_spapr_mmio32_readw;
+    ret->bus.mmio_readl = qpci_spapr_mmio32_readl;
 
-    ret->bus.io_writeb = qpci_spapr_io_writeb;
-    ret->bus.io_writew = qpci_spapr_io_writew;
-    ret->bus.io_writel = qpci_spapr_io_writel;
+    ret->bus.mmio_writeb = qpci_spapr_mmio32_writeb;
+    ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
+    ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
 
     ret->bus.config_readb = qpci_spapr_config_readb;
     ret->bus.config_readw = qpci_spapr_config_readw;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index c3f3382..55b01df 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -224,33 +224,68 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
 
 uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readb(dev->bus, data);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readb(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readb(dev->bus, addr);
+    }
 }
 
 uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readw(dev->bus, data);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readw(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readw(dev->bus, addr);
+    }
 }
 
 uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readl(dev->bus, data);
-}
+    uintptr_t addr = (uintptr_t)data;
 
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readl(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readl(dev->bus, addr);
+    }
+}
 
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
 {
-    dev->bus->io_writeb(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeb(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writeb(dev->bus, addr, value);
+    }
 }
 
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
 {
-    dev->bus->io_writew(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writew(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writew(dev->bus, addr, value);
+    }
 }
 
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
 {
-    dev->bus->io_writel(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writel(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writel(dev->bus, addr, value);
+    }
 }
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index c06add8..72a2245 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -15,6 +15,8 @@
 
 #include "libqtest.h"
 
+#define QPCI_PIO_LIMIT    0x10000
+
 #define QPCI_DEVFN(dev, fn) (((dev) << 3) | (fn))
 
 typedef struct QPCIDevice QPCIDevice;
@@ -22,13 +24,21 @@ typedef struct QPCIBus QPCIBus;
 
 struct QPCIBus
 {
-    uint8_t (*io_readb)(QPCIBus *bus, void *addr);
-    uint16_t (*io_readw)(QPCIBus *bus, void *addr);
-    uint32_t (*io_readl)(QPCIBus *bus, void *addr);
+    uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
+    uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
+    uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
+
+    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
+    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
+    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
+
+    void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
+    void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
+    void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
-    void (*io_writeb)(QPCIBus *bus, void *addr, uint8_t value);
-    void (*io_writew)(QPCIBus *bus, void *addr, uint16_t value);
-    void (*io_writel)(QPCIBus *bus, void *addr, uint32_t value);
+    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
+    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
+    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
     uint8_t (*config_readb)(QPCIBus *bus, int devfn, uint8_t offset);
     uint16_t (*config_readw)(QPCIBus *bus, int devfn, uint8_t offset);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 03/11] libqos: Move BAR assignment to common code
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 01/11] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 02/11] libqos: Handle PCI IO de-multiplexing in common code David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO David Gibson
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The PCI backends in libqos each supply an iomap() and iounmap() function
which is used to set up a specified PCI BAR.  But PCI BAR allocation takes
place entirely within PCI space, so doesn't really need per-backend
versions.  For example, Linux includes generic BAR allocation code used on
platforms where that isn't done by firmware.

This patch merges the BAR allocation from the two existing backends into a
single simplified copy.  The back ends just need to set up some parameters
describing the window of PCI IO and PCI memory addresses which are
available for allocation.  Like both the existing versions the new one uses
a simple bump allocator.

Note that (again like the existing versions) this doesn't really handle
64-bit memory BARs properly.  It is actually used for such a BAR by the
ivshmem test, and apparently the 32-bit MMIO BAR logic is close enough to
work, as long as the BAR isn't too big.  Fixing that to properly handle
64-bit BAR allocation is a problem for another time.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 87 ++--------------------------------------------
 tests/libqos/pci-spapr.c | 89 ++----------------------------------------------
 tests/libqos/pci.c       | 56 ++++++++++++++++++++++++++++--
 tests/libqos/pci.h       |  7 ++--
 4 files changed, 63 insertions(+), 176 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 51dff8a..d0eb84a 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -17,7 +17,6 @@
 #include "hw/pci/pci_regs.h"
 
 #include "qemu-common.h"
-#include "qemu/host-utils.h"
 
 
 #define ACPI_PCIHP_ADDR         0xae00
@@ -26,14 +25,6 @@
 typedef struct QPCIBusPC
 {
     QPCIBus bus;
-
-    uint32_t pci_hole_start;
-    uint32_t pci_hole_size;
-    uint32_t pci_hole_alloc;
-
-    uint16_t pci_iohole_start;
-    uint16_t pci_iohole_size;
-    uint16_t pci_iohole_alloc;
 } QPCIBusPC;
 
 static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
@@ -132,71 +123,6 @@ static void qpci_pc_config_writel(QPCIBus *bus, int devfn, uint8_t offset, uint3
     outl(0xcfc, value);
 }
 
-static void *qpci_pc_iomap(QPCIBus *bus, QPCIDevice *dev, int barno, uint64_t *sizeptr)
-{
-    QPCIBusPC *s = container_of(bus, QPCIBusPC, bus);
-    static const int bar_reg_map[] = {
-        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
-        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
-    };
-    int bar_reg;
-    uint32_t addr;
-    uint64_t size;
-    uint32_t io_type;
-
-    g_assert(barno >= 0 && barno <= 5);
-    bar_reg = bar_reg_map[barno];
-
-    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
-    addr = qpci_config_readl(dev, bar_reg);
-
-    io_type = addr & PCI_BASE_ADDRESS_SPACE;
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        addr &= PCI_BASE_ADDRESS_IO_MASK;
-    } else {
-        addr &= PCI_BASE_ADDRESS_MEM_MASK;
-    }
-
-    size = (1ULL << ctzl(addr));
-    if (size == 0) {
-        return NULL;
-    }
-    if (sizeptr) {
-        *sizeptr = size;
-    }
-
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        uint16_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_iohole_alloc, size) + size
-                 <= s->pci_iohole_size);
-        s->pci_iohole_alloc = QEMU_ALIGN_UP(s->pci_iohole_alloc, size);
-        loc = s->pci_iohole_start + s->pci_iohole_alloc;
-        s->pci_iohole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
-
-        return (void *)(intptr_t)loc;
-    } else {
-        uint64_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_hole_alloc, size) + size
-                 <= s->pci_hole_size);
-        s->pci_hole_alloc = QEMU_ALIGN_UP(s->pci_hole_alloc, size);
-        loc = s->pci_hole_start + s->pci_hole_alloc;
-        s->pci_hole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc);
-
-        return (void *)(intptr_t)loc;
-    }
-}
-
-static void qpci_pc_iounmap(QPCIBus *bus, void *data)
-{
-    /* FIXME */
-}
-
 QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
 {
     QPCIBusPC *ret;
@@ -227,16 +153,9 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.config_writew = qpci_pc_config_writew;
     ret->bus.config_writel = qpci_pc_config_writel;
 
-    ret->bus.iomap = qpci_pc_iomap;
-    ret->bus.iounmap = qpci_pc_iounmap;
-
-    ret->pci_hole_start = 0xE0000000;
-    ret->pci_hole_size = 0x20000000;
-    ret->pci_hole_alloc = 0;
-
-    ret->pci_iohole_start = 0xc000;
-    ret->pci_iohole_size = 0x4000;
-    ret->pci_iohole_alloc = 0;
+    ret->bus.pio_alloc_ptr = 0xc000;
+    ret->bus.mmio_alloc_ptr = 0xE0000000;
+    ret->bus.mmio_limit = 0x100000000ULL;
 
     return &ret->bus;
 }
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 2d26a94..70a24b5 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -34,14 +34,6 @@ typedef struct QPCIBusSPAPR {
 
     uint64_t mmio32_cpu_base;
     QPCIWindow mmio32;
-
-    uint64_t pci_hole_start;
-    uint64_t pci_hole_size;
-    uint64_t pci_hole_alloc;
-
-    uint32_t pci_iohole_start;
-    uint32_t pci_iohole_size;
-    uint32_t pci_iohole_alloc;
 } QPCIBusSPAPR;
 
 /*
@@ -167,72 +159,6 @@ static void qpci_spapr_config_writel(QPCIBus *bus, int devfn, uint8_t offset,
     qrtas_ibm_write_pci_config(s->alloc, s->buid, config_addr, 4, value);
 }
 
-static void *qpci_spapr_iomap(QPCIBus *bus, QPCIDevice *dev, int barno,
-                              uint64_t *sizeptr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    static const int bar_reg_map[] = {
-        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
-        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
-    };
-    int bar_reg;
-    uint32_t addr;
-    uint64_t size;
-    uint32_t io_type;
-
-    g_assert(barno >= 0 && barno <= 5);
-    bar_reg = bar_reg_map[barno];
-
-    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
-    addr = qpci_config_readl(dev, bar_reg);
-
-    io_type = addr & PCI_BASE_ADDRESS_SPACE;
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        addr &= PCI_BASE_ADDRESS_IO_MASK;
-    } else {
-        addr &= PCI_BASE_ADDRESS_MEM_MASK;
-    }
-
-    size = (1ULL << ctzl(addr));
-    if (size == 0) {
-        return NULL;
-    }
-    if (sizeptr) {
-        *sizeptr = size;
-    }
-
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        uint16_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_iohole_alloc, size) + size
-                 <= s->pci_iohole_size);
-        s->pci_iohole_alloc = QEMU_ALIGN_UP(s->pci_iohole_alloc, size);
-        loc = s->pci_iohole_start + s->pci_iohole_alloc;
-        s->pci_iohole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
-
-        return (void *)(unsigned long)loc;
-    } else {
-        uint64_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_hole_alloc, size) + size
-                 <= s->pci_hole_size);
-        s->pci_hole_alloc = QEMU_ALIGN_UP(s->pci_hole_alloc, size);
-        loc = s->pci_hole_start + s->pci_hole_alloc;
-        s->pci_hole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc);
-
-        return (void *)(unsigned long)loc;
-    }
-}
-
-static void qpci_spapr_iounmap(QPCIBus *bus, void *data)
-{
-    /* FIXME */
-}
-
 #define SPAPR_PCI_BASE               (1ULL << 45)
 
 #define SPAPR_PCI_MMIO32_WIN_SIZE    0x80000000 /* 2 GiB */
@@ -270,9 +196,6 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.config_writew = qpci_spapr_config_writew;
     ret->bus.config_writel = qpci_spapr_config_writel;
 
-    ret->bus.iomap = qpci_spapr_iomap;
-    ret->bus.iounmap = qpci_spapr_iounmap;
-
     /* FIXME: We assume the default location of the PHB for now.
      * Ideally we'd parse the device tree deposited in the guest to
      * get the window locations */
@@ -287,15 +210,9 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->mmio32.pci_base = 0x80000000; /* 2 GiB */
     ret->mmio32.size = SPAPR_PCI_MMIO32_WIN_SIZE;
 
-    ret->pci_hole_start = 0xC0000000;
-    ret->pci_hole_size =
-        ret->mmio32.pci_base + ret->mmio32.size - ret->pci_hole_start;
-    ret->pci_hole_alloc = 0;
-
-    ret->pci_iohole_start = 0xc000;
-    ret->pci_iohole_size =
-        ret->pio.pci_base + ret->pio.size - ret->pci_iohole_start;
-    ret->pci_iohole_alloc = 0;
+    ret->bus.pio_alloc_ptr = 0xc000;
+    ret->bus.mmio_alloc_ptr = ret->mmio32.pci_base;
+    ret->bus.mmio_limit = ret->mmio32.pci_base + ret->mmio32.size;
 
     return &ret->bus;
 }
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 55b01df..bf1c532 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -14,6 +14,7 @@
 #include "libqos/pci.h"
 
 #include "hw/pci/pci_regs.h"
+#include "qemu/host-utils.h"
 
 void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
                          void (*func)(QPCIDevice *dev, int devfn, void *data),
@@ -290,12 +291,63 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
-    return dev->bus->iomap(dev->bus, dev, barno, sizeptr);
+    QPCIBus *bus = dev->bus;
+    static const int bar_reg_map[] = {
+        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
+        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
+    };
+    int bar_reg;
+    uint32_t addr, size;
+    uint32_t io_type;
+    uint64_t loc;
+
+    g_assert(barno >= 0 && barno <= 5);
+    bar_reg = bar_reg_map[barno];
+
+    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
+    addr = qpci_config_readl(dev, bar_reg);
+
+    io_type = addr & PCI_BASE_ADDRESS_SPACE;
+    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
+        addr &= PCI_BASE_ADDRESS_IO_MASK;
+    } else {
+        addr &= PCI_BASE_ADDRESS_MEM_MASK;
+    }
+
+    g_assert(addr); /* Must have *some* size bits */
+
+    size = 1U << ctz32(addr);
+    if (sizeptr) {
+        *sizeptr = size;
+    }
+
+    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
+        loc = QEMU_ALIGN_UP(bus->pio_alloc_ptr, size);
+
+        g_assert(loc >= bus->pio_alloc_ptr);
+        g_assert(loc + size <= QPCI_PIO_LIMIT); /* Keep PIO below 64kiB */
+
+        bus->pio_alloc_ptr = loc + size;
+
+        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
+    } else {
+        loc = QEMU_ALIGN_UP(bus->mmio_alloc_ptr, size);
+
+        /* Check for space */
+        g_assert(loc >= bus->mmio_alloc_ptr);
+        g_assert(loc + size <= bus->mmio_limit);
+
+        bus->mmio_alloc_ptr = loc + size;
+
+        qpci_config_writel(dev, bar_reg, loc);
+    }
+
+    return (void *)(uintptr_t)loc;
 }
 
 void qpci_iounmap(QPCIDevice *dev, void *data)
 {
-    dev->bus->iounmap(dev->bus, data);
+    /* FIXME */
 }
 
 void qpci_plug_device_test(const char *driver, const char *id,
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 72a2245..f6f916d 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -22,8 +22,7 @@
 typedef struct QPCIDevice QPCIDevice;
 typedef struct QPCIBus QPCIBus;
 
-struct QPCIBus
-{
+struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
@@ -51,8 +50,8 @@ struct QPCIBus
     void (*config_writel)(QPCIBus *bus, int devfn,
                           uint8_t offset, uint32_t value);
 
-    void *(*iomap)(QPCIBus *bus, QPCIDevice *dev, int barno, uint64_t *sizeptr);
-    void (*iounmap)(QPCIBus *bus, void *data);
+    uint16_t pio_alloc_ptr;
+    uint64_t mmio_alloc_ptr, mmio_limit;
 };
 
 struct QPCIDevice
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (2 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 03/11] libqos: Move BAR assignment to " David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21 10:05   ` Greg Kurz
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 05/11] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
                   ` (7 subsequent siblings)
  11 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The usual model for PCI IO with libqos is to use qpci_iomap() to map a
specific BAR for a PCI device, then perform IOs within that BAR using
qpci_io_{read,write}*().

However, certain devices also have legacy PCI IO.  In this case, instead of
(or as well as) being accessed via PCI BARs, the device can be accessed
via certain well-known, fixed addresses in PCI IO space.

Two existing tests use legacy PCI IO, and take different flawed approaches
to it:
    * tco-test manually constructs a tco_io_base value instead of calling
      qpci_iomap(), which assumes internal knowledge of the structure of
      the value it shouldn't have
    * ide-test uses direct in*() and out*() calls instead of using
      qpci_io_*() accessors, meaning it's not portable to non-x86 machine
      types.

This patch implements a new qpci_iomap_legacy() interface which gets a
handle in the same format as qpci_iomap() but refers to a region in
the legacy PIO space.  For a device which has the same registers
available both in a BAR and in legacy space (quite common), this
allows the same test code to test both options with just a different
iomap() at the beginning.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/libqos/pci.c | 5 +++++
 tests/libqos/pci.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index bf1c532..98a2e56 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -350,6 +350,11 @@ void qpci_iounmap(QPCIDevice *dev, void *data)
     /* FIXME */
 }
 
+void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
+{
+    return (void *)(uintptr_t)addr;
+}
+
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts)
 {
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index f6f916d..b6f855e 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -94,6 +94,7 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
 void qpci_iounmap(QPCIDevice *dev, void *data);
+void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
 
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 05/11] tests: Adjust tco-test to use qpci_legacy_iomap()
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (3 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 06/11] libqos: Add streaming accessors for PCI MMIO David Gibson
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Avoid tco-test making assumptions about the internal format of the address
tokens passed to PCI IO accessors, by using the new qpci_legacy_iomap()
function.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/tco-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/tco-test.c b/tests/tco-test.c
index 0d201b1..129577d 100644
--- a/tests/tco-test.c
+++ b/tests/tco-test.c
@@ -70,7 +70,7 @@ static void test_init(TestData *d)
     /* set Root Complex BAR */
     qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
 
-    d->tco_io_base = (void *)((uintptr_t)PM_IO_BASE_ADDR + 0x60);
+    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
 }
 
 static void stop_tco(const TestData *d)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 06/11] libqos: Add streaming accessors for PCI MMIO
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (4 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 05/11] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Currently PCI memory (aka MMIO) space is accessed via a set of readb/writeb
style accessors.  This is what we want for accessing discrete registers of
a certain size.  However, there are a few cases where we instead need a
"bag of bytes" style streaming interface to PCI MMIO space.  This can be
either for streaming data style registers or when there's actual memory
rather than registers in PCI space, for example frame buffers or ivshmem.

This patch adds backend callbacks, and libqos wrappers for this type of
byte address order preserving accesses.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 14 ++++++++++++++
 tests/libqos/pci-spapr.c | 17 +++++++++++++++++
 tests/libqos/pci.c       | 16 ++++++++++++++++
 tests/libqos/pci.h       |  6 ++++++
 4 files changed, 53 insertions(+)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index d0eb84a..84aee25 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -87,6 +87,17 @@ static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(addr, val);
 }
 
+static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
+{
+    memread(addr, buf, len);
+}
+
+static void qpci_pc_memwrite(QPCIBus *bus, uint32_t addr,
+                             const void *buf, size_t len)
+{
+    memwrite(addr, buf, len);
+}
+
 static uint8_t qpci_pc_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
 {
     outl(0xcf8, (1U << 31) | (devfn << 8) | offset);
@@ -145,6 +156,9 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.mmio_writew = qpci_pc_mmio_writew;
     ret->bus.mmio_writel = qpci_pc_mmio_writel;
 
+    ret->bus.memread = qpci_pc_memread;
+    ret->bus.memwrite = qpci_pc_memwrite;
+
     ret->bus.config_readb = qpci_pc_config_readb;
     ret->bus.config_readw = qpci_pc_config_readw;
     ret->bus.config_readl = qpci_pc_config_readl;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 70a24b5..ad12c2e 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -114,6 +114,20 @@ static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(s->mmio32_cpu_base + addr, bswap32(val));
 }
 
+static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
+                               void *buf, size_t len)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    memread(s->mmio32_cpu_base + addr, buf, len);
+}
+
+static void qpci_spapr_memwrite(QPCIBus *bus, uint32_t addr,
+                                const void *buf, size_t len)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    memwrite(s->mmio32_cpu_base + addr, buf, len);
+}
+
 static uint8_t qpci_spapr_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
@@ -188,6 +202,9 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
     ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
 
+    ret->bus.memread = qpci_spapr_memread;
+    ret->bus.memwrite = qpci_spapr_memwrite;
+
     ret->bus.config_readb = qpci_spapr_config_readb;
     ret->bus.config_readw = qpci_spapr_config_readw;
     ret->bus.config_readl = qpci_spapr_config_readl;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 98a2e56..146b063 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -289,6 +289,22 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     }
 }
 
+void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    g_assert(addr >= QPCI_PIO_LIMIT);
+    dev->bus->memread(dev->bus, addr, buf, len);
+}
+
+void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    g_assert(addr >= QPCI_PIO_LIMIT);
+    dev->bus->memwrite(dev->bus, addr, buf, len);
+}
+
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
     QPCIBus *bus = dev->bus;
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index b6f855e..2b08362 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -39,6 +39,9 @@ struct QPCIBus {
     void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
+    void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
+    void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
+
     uint8_t (*config_readb)(QPCIBus *bus, int devfn, uint8_t offset);
     uint16_t (*config_readw)(QPCIBus *bus, int devfn, uint8_t offset);
     uint32_t (*config_readl)(QPCIBus *bus, int devfn, uint8_t offset);
@@ -92,6 +95,9 @@ void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
 
+void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
+void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
+
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
 void qpci_iounmap(QPCIDevice *dev, void *data);
 void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (5 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 06/11] libqos: Add streaming accessors for PCI MMIO David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21 10:08   ` Greg Kurz
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test David Gibson
                   ` (4 subsequent siblings)
  11 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

In the libqos PCI code we now have accessors both for registers (byte
significance preserving) and for streaming data (byte address order
preserving).  These exist in both the interface for qtest drivers and in
the machine specific backends.

However, the register-style accessors aren't actually necessary in the
backend.  They can be implemented in terms of the byte address order
preserving accessors by the libqos wrappers.  This works because PCI is
always little endian.

This does assume that the back end byte address order preserving accessors
will perform the equivalent of a single bus transaction for short lengths.
This is the case, and in fact they currently end up using the same
cpu_physical_memory_rw() implementation within the qtest accelerator.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/libqos/pci-pc.c    | 38 --------------------------------------
 tests/libqos/pci-spapr.c | 44 --------------------------------------------
 tests/libqos/pci.c       | 20 ++++++++++++++------
 tests/libqos/pci.h       |  8 --------
 4 files changed, 14 insertions(+), 96 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 84aee25..849ea56 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -32,61 +32,31 @@ static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
     return inb(addr);
 }
 
-static uint8_t qpci_pc_mmio_readb(QPCIBus *bus, uint32_t addr)
-{
-    return readb(addr);
-}
-
 static void qpci_pc_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     outb(addr, val);
 }
 
-static void qpci_pc_mmio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
-{
-    writeb(addr, val);
-}
-
 static uint16_t qpci_pc_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     return inw(addr);
 }
 
-static uint16_t qpci_pc_mmio_readw(QPCIBus *bus, uint32_t addr)
-{
-    return readw(addr);
-}
-
 static void qpci_pc_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
     outw(addr, val);
 }
 
-static void qpci_pc_mmio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
-{
-    writew(addr, val);
-}
-
 static uint32_t qpci_pc_pio_readl(QPCIBus *bus, uint32_t addr)
 {
     return inl(addr);
 }
 
-static uint32_t qpci_pc_mmio_readl(QPCIBus *bus, uint32_t addr)
-{
-    return readl(addr);
-}
-
 static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
     outl(addr, val);
 }
 
-static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
-{
-    writel(addr, val);
-}
-
 static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
 {
     memread(addr, buf, len);
@@ -148,14 +118,6 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.pio_writew = qpci_pc_pio_writew;
     ret->bus.pio_writel = qpci_pc_pio_writel;
 
-    ret->bus.mmio_readb = qpci_pc_mmio_readb;
-    ret->bus.mmio_readw = qpci_pc_mmio_readw;
-    ret->bus.mmio_readl = qpci_pc_mmio_readl;
-
-    ret->bus.mmio_writeb = qpci_pc_mmio_writeb;
-    ret->bus.mmio_writew = qpci_pc_mmio_writew;
-    ret->bus.mmio_writel = qpci_pc_mmio_writel;
-
     ret->bus.memread = qpci_pc_memread;
     ret->bus.memwrite = qpci_pc_memwrite;
 
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index ad12c2e..f26488a 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -48,72 +48,36 @@ static uint8_t qpci_spapr_pio_readb(QPCIBus *bus, uint32_t addr)
     return readb(s->pio_cpu_base + addr);
 }
 
-static uint8_t qpci_spapr_mmio32_readb(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return readb(s->mmio32_cpu_base + addr);
-}
-
 static void qpci_spapr_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writeb(s->pio_cpu_base + addr, val);
 }
 
-static void qpci_spapr_mmio32_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writeb(s->mmio32_cpu_base + addr, val);
-}
-
 static uint16_t qpci_spapr_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     return bswap16(readw(s->pio_cpu_base + addr));
 }
 
-static uint16_t qpci_spapr_mmio32_readw(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return bswap16(readw(s->mmio32_cpu_base + addr));
-}
-
 static void qpci_spapr_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writew(s->pio_cpu_base + addr, bswap16(val));
 }
 
-static void qpci_spapr_mmio32_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writew(s->mmio32_cpu_base + addr, bswap16(val));
-}
-
 static uint32_t qpci_spapr_pio_readl(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     return bswap32(readl(s->pio_cpu_base + addr));
 }
 
-static uint32_t qpci_spapr_mmio32_readl(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return bswap32(readl(s->mmio32_cpu_base + addr));
-}
-
 static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writel(s->pio_cpu_base + addr, bswap32(val));
 }
 
-static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writel(s->mmio32_cpu_base + addr, bswap32(val));
-}
-
 static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
                                void *buf, size_t len)
 {
@@ -194,14 +158,6 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.pio_writew = qpci_spapr_pio_writew;
     ret->bus.pio_writel = qpci_spapr_pio_writel;
 
-    ret->bus.mmio_readb = qpci_spapr_mmio32_readb;
-    ret->bus.mmio_readw = qpci_spapr_mmio32_readw;
-    ret->bus.mmio_readl = qpci_spapr_mmio32_readl;
-
-    ret->bus.mmio_writeb = qpci_spapr_mmio32_writeb;
-    ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
-    ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
-
     ret->bus.memread = qpci_spapr_memread;
     ret->bus.memwrite = qpci_spapr_memwrite;
 
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 146b063..bdffeb3 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -230,7 +230,9 @@ uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readb(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readb(dev->bus, addr);
+        uint8_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return val;
     }
 }
 
@@ -241,7 +243,9 @@ uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readw(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readw(dev->bus, addr);
+        uint16_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le16_to_cpu(val);
     }
 }
 
@@ -252,7 +256,9 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readl(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readl(dev->bus, addr);
+        uint32_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le32_to_cpu(val);
     }
 }
 
@@ -263,7 +269,7 @@ void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writeb(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writeb(dev->bus, addr, value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
@@ -274,7 +280,8 @@ void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writew(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writew(dev->bus, addr, value);
+        value = cpu_to_le16(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
@@ -285,7 +292,8 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writel(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writel(dev->bus, addr, value);
+        value = cpu_to_le32(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 2b08362..ce6ed08 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -27,18 +27,10 @@ struct QPCIBus {
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
 
-    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
-    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
-    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
-
     void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
     void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
-    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
-    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
-    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
-
     void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
     void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (6 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  8:31   ` Laurent Vivier
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors David Gibson
                   ` (3 subsequent siblings)
  11 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

ide-test uses many explicit inb() / outb() operations for its IO, which
means it's not portable to non-x86 platforms.  This cleans it up to use
the libqos PCI accessors instead.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 118 insertions(+), 61 deletions(-)

diff --git a/tests/ide-test.c b/tests/ide-test.c
index a8a4081..86c4373 100644
--- a/tests/ide-test.c
+++ b/tests/ide-test.c
@@ -137,7 +137,7 @@ static void ide_test_quit(void)
     qtest_end();
 }
 
-static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
+static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
 {
     QPCIDevice *dev;
     uint16_t vendor_id, device_id;
@@ -156,7 +156,9 @@ static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
     g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
 
     /* Map bmdma BAR */
-    *bmdma_base = (uint16_t)(uintptr_t) qpci_iomap(dev, 4, NULL);
+    *bmdma_base = qpci_iomap(dev, 4, NULL);
+
+    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
 
     qpci_device_enable(dev);
 
@@ -179,17 +181,19 @@ typedef struct PrdtEntry {
 
 static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
                             PrdtEntry *prdt, int prdt_entries,
-                            void(*post_exec)(uint64_t sector, int nb_sectors))
+                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
+                                             uint64_t sector, int nb_sectors))
 {
     QPCIDevice *dev;
-    uint16_t bmdma_base;
+    void *bmdma_base;
+    void *ide_base;
     uintptr_t guest_prdt;
     size_t len;
     bool from_dev;
     uint8_t status;
     int flags;
 
-    dev = get_pci_device(&bmdma_base);
+    dev = get_pci_device(&bmdma_base, &ide_base);
 
     flags = cmd & ~0xff;
     cmd &= 0xff;
@@ -214,59 +218,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
     }
 
     /* Select device 0 */
-    outb(IDE_BASE + reg_device, 0 | LBA);
+    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
 
     /* Stop any running transfer, clear any pending interrupt */
-    outb(bmdma_base + bmreg_cmd, 0);
-    outb(bmdma_base + bmreg_status, BM_STS_INTR);
+    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
 
     /* Setup PRDT */
     len = sizeof(*prdt) * prdt_entries;
     guest_prdt = guest_alloc(guest_malloc, len);
     memwrite(guest_prdt, prdt, len);
-    outl(bmdma_base + bmreg_prdt, guest_prdt);
+    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
 
     /* ATA DMA command */
     if (cmd == CMD_PACKET) {
         /* Enables ATAPI DMA; otherwise PIO is attempted */
-        outb(IDE_BASE + reg_feature, 0x01);
+        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
     } else {
-        outb(IDE_BASE + reg_nsectors, nb_sectors);
-        outb(IDE_BASE + reg_lba_low,    sector & 0xff);
-        outb(IDE_BASE + reg_lba_middle, (sector >> 8) & 0xff);
-        outb(IDE_BASE + reg_lba_high,   (sector >> 16) & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
+        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
     }
 
-    outb(IDE_BASE + reg_command, cmd);
+    qpci_io_writeb(dev, ide_base + reg_command, cmd);
 
     if (post_exec) {
-        post_exec(sector, nb_sectors);
+        post_exec(dev, ide_base, sector, nb_sectors);
     }
 
     /* Start DMA transfer */
-    outb(bmdma_base + bmreg_cmd, BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
+    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
+                   BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
 
     if (flags & CMDF_ABORT) {
-        outb(bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
     }
 
     /* Wait for the DMA transfer to complete */
     do {
-        status = inb(bmdma_base + bmreg_status);
+        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
     } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
 
     g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
 
     /* Check IDE status code */
-    assert_bit_set(inb(IDE_BASE + reg_status), DRDY);
-    assert_bit_clear(inb(IDE_BASE + reg_status), BSY | DRQ);
+    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
 
     /* Reading the status register clears the IRQ */
     g_assert(!get_irq(IDE_PRIMARY_IRQ));
 
     /* Stop DMA transfer if still active */
     if (status & BM_STS_ACTIVE) {
-        outb(bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
     }
 
     free_pci_device(dev);
@@ -276,6 +281,8 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
 
 static void test_bmdma_simple_rw(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
     uint8_t *buf;
     uint8_t *cmpbuf;
@@ -289,6 +296,8 @@ static void test_bmdma_simple_rw(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     buf = g_malloc(len);
     cmpbuf = g_malloc(len);
 
@@ -299,7 +308,7 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Write 0xaa pattern to sector 1 */
     memset(buf, 0xaa, len);
@@ -308,14 +317,14 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Read and verify 0x55 pattern in sector 0 */
     memset(cmpbuf, 0x55, len);
 
     status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -325,7 +334,7 @@ static void test_bmdma_simple_rw(void)
 
     status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -337,6 +346,8 @@ static void test_bmdma_simple_rw(void)
 
 static void test_bmdma_short_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -346,21 +357,25 @@ static void test_bmdma_short_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_one_sector_short_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     /* Read 2 sectors but only give 1 sector in PRDT */
@@ -371,21 +386,25 @@ static void test_bmdma_one_sector_short_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_long_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -395,23 +414,29 @@ static void test_bmdma_long_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_no_busmaster(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
      * able to access it anyway because the Bus Master bit in the PCI command
      * register isn't set. This is complete nonsense, but it used to be pretty
@@ -424,7 +449,7 @@ static void test_bmdma_no_busmaster(void)
     /* Not entirely clear what the expected result is, but this is what we get
      * in practice. At least we want to be aware of any changes. */
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_setup(void)
@@ -454,6 +479,8 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
 
 static void test_identify(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     uint16_t buf[256];
     int i;
@@ -464,23 +491,25 @@ static void test_identify(void)
         "-global ide-hd.ver=%s",
         tmp_path, "testdisk", "version");
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* IDENTIFY command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_IDENTIFY);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
 
     /* Read in the IDENTIFY buffer and check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     for (i = 0; i < 256; i++) {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
         assert_bit_set(data, DRDY | DRQ);
         assert_bit_clear(data, BSY | DF | ERR);
 
-        ((uint16_t*) buf)[i] = inw(IDE_BASE + reg_data);
+        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
     }
 
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, DRDY);
     assert_bit_clear(data, BSY | DF | ERR | DRQ);
 
@@ -505,11 +534,15 @@ static void test_identify(void)
  */
 static void make_dirty(uint8_t device)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
     size_t len = 512;
     uintptr_t guest_buf;
     void* buf;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     guest_buf = guest_alloc(guest_malloc, len);
     buf = g_malloc(len);
     g_assert(guest_buf);
@@ -527,19 +560,23 @@ static void make_dirty(uint8_t device)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     g_free(buf);
 }
 
 static void test_flush(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
 
     ide_test_start(
         "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
         tmp_path);
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
@@ -549,11 +586,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
 
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -561,11 +598,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
 
     /* Check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -576,6 +613,8 @@ static void test_flush(void)
 
 static void test_retry_flush(const char *machine)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     const char *s;
 
@@ -587,17 +626,19 @@ static void test_retry_flush(const char *machine)
         "rerror=stop,werror=stop",
         debug_path, tmp_path);
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
     make_dirty(0);
 
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -608,11 +649,11 @@ static void test_retry_flush(const char *machine)
     qmp_discard_response(s);
 
     /* Check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -623,11 +664,16 @@ static void test_retry_flush(const char *machine)
 
 static void test_flush_nodev(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
+
     ide_test_start("");
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Just testing that qemu doesn't crash... */
 
@@ -654,7 +700,8 @@ typedef struct Read10CDB {
     uint16_t padding;
 } __attribute__((__packed__)) Read10CDB;
 
-static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
+static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
+                                 uint64_t lba, int nblocks)
 {
     Read10CDB pkt = { .padding = 0 };
     int i;
@@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
 
     /* Send Packet */
     for (i = 0; i < sizeof(Read10CDB)/2; i++) {
-        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
+        qpci_io_writew(dev, ide_base + reg_data,
+                       le16_to_cpu(((uint16_t *)&pkt)[i]));
     }
 }
 
@@ -683,13 +731,17 @@ static void nsleep(int64_t nsecs)
 
 static uint8_t ide_wait_clear(uint8_t flag)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     time_t st;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Wait with a 5 second timeout */
     time(&st);
     while (true) {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
         if (!(data & flag)) {
             return data;
         }
@@ -723,6 +775,8 @@ static void ide_wait_intr(int irq)
 
 static void cdrom_pio_impl(int nblocks)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     FILE *fh;
     int patt_blocks = MAX(16, nblocks);
     size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
@@ -741,13 +795,15 @@ static void cdrom_pio_impl(int nblocks)
 
     ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
                    "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
+    dev = get_pci_device(&bmdma_base, &ide_base);
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* PACKET command on device 0 */
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
-    outb(IDE_BASE + reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
-    outb(IDE_BASE + reg_command, CMD_PACKET);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
+    qpci_io_writeb(dev, ide_base + reg_lba_high,
+                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
     /* HP0: Check_Status_A State */
     nsleep(400);
     data = ide_wait_clear(BSY);
@@ -756,7 +812,7 @@ static void cdrom_pio_impl(int nblocks)
     assert_bit_clear(data, ERR | DF | BSY);
 
     /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
-    send_scsi_cdb_read10(0, nblocks);
+    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
 
     /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
      * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
@@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
 
         /* HP4: Transfer_Data */
         for (j = 0; j < MIN((limit / 2), rem); j++) {
-            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
+            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
+                                                       ide_base + reg_data));
         }
     }
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (7 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21 10:31   ` Greg Kurz
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 10/11] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
                   ` (2 subsequent siblings)
  11 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Currently the libqos PCI layer includes accessor helpers for 8, 16 and 32
bit reads and writes.  It's likely that we'll want 64-bit accesses in the
future (plenty of modern peripherals will have 64-bit reigsters).  This
adds them.

For PIO (not MMIO) accesses on the PC backend, this is implemented as two
32-bit ins or outs.  That's not ideal but AFAICT x86 doesn't have 64-bit
versions of in and out.

This patch also converts the single current user of 64-bit accesses -
virtio-pci.c to use the new mechanism, rather than a sequence of 8 byte
reads.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/libqos/pci-pc.c     | 13 +++++++++++++
 tests/libqos/pci-spapr.c  | 14 ++++++++++++++
 tests/libqos/pci.c        | 25 +++++++++++++++++++++++++
 tests/libqos/pci.h        |  4 ++++
 tests/libqos/virtio-pci.c | 16 ++++------------
 5 files changed, 60 insertions(+), 12 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 849ea56..ded1c54 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -57,6 +57,17 @@ static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     outl(addr, val);
 }
 
+static uint64_t qpci_pc_pio_readq(QPCIBus *bus, uint32_t addr)
+{
+    return (uint64_t)inl(addr) + ((uint64_t)inl(addr + 4) << 32);
+}
+
+static void qpci_pc_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
+{
+    outl(addr, val & 0xffffffff);
+    outl(addr + 4, val >> 32);
+}
+
 static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
 {
     memread(addr, buf, len);
@@ -113,10 +124,12 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.pio_readb = qpci_pc_pio_readb;
     ret->bus.pio_readw = qpci_pc_pio_readw;
     ret->bus.pio_readl = qpci_pc_pio_readl;
+    ret->bus.pio_readq = qpci_pc_pio_readq;
 
     ret->bus.pio_writeb = qpci_pc_pio_writeb;
     ret->bus.pio_writew = qpci_pc_pio_writew;
     ret->bus.pio_writel = qpci_pc_pio_writel;
+    ret->bus.pio_writeq = qpci_pc_pio_writeq;
 
     ret->bus.memread = qpci_pc_memread;
     ret->bus.memwrite = qpci_pc_memwrite;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index f26488a..1e5d015 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -78,6 +78,18 @@ static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(s->pio_cpu_base + addr, bswap32(val));
 }
 
+static uint64_t qpci_spapr_pio_readq(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap64(readq(s->pio_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writeq(s->pio_cpu_base + addr, bswap64(val));
+}
+
 static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
                                void *buf, size_t len)
 {
@@ -153,10 +165,12 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.pio_readb = qpci_spapr_pio_readb;
     ret->bus.pio_readw = qpci_spapr_pio_readw;
     ret->bus.pio_readl = qpci_spapr_pio_readl;
+    ret->bus.pio_readq = qpci_spapr_pio_readq;
 
     ret->bus.pio_writeb = qpci_spapr_pio_writeb;
     ret->bus.pio_writew = qpci_spapr_pio_writew;
     ret->bus.pio_writel = qpci_spapr_pio_writel;
+    ret->bus.pio_writeq = qpci_spapr_pio_writeq;
 
     ret->bus.memread = qpci_spapr_memread;
     ret->bus.memwrite = qpci_spapr_memwrite;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index bdffeb3..3021651 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -262,6 +262,19 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
     }
 }
 
+uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readq(dev->bus, addr);
+    } else {
+        uint64_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le64_to_cpu(val);
+    }
+}
+
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
 {
     uintptr_t addr = (uintptr_t)data;
@@ -297,6 +310,18 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     }
 }
 
+void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeq(dev->bus, addr, value);
+    } else {
+        value = cpu_to_le64(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+    }
+}
+
 void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
 {
     uintptr_t addr = (uintptr_t)data;
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index ce6ed08..531e3f7 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -26,10 +26,12 @@ struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
+    uint64_t (*pio_readq)(QPCIBus *bus, uint32_t addr);
 
     void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
     void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
+    void (*pio_writeq)(QPCIBus *bus, uint32_t addr, uint64_t value);
 
     void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
     void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
@@ -82,10 +84,12 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
 uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
 uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
 uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
+uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
 
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
+void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
 
 void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
 void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index fa82132..c69d09d 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -106,22 +106,14 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
 static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    int i;
-    uint64_t u64 = 0;
+    uint64_t val;
 
+    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
-        for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
-                                           + off + i) << (7 - i) * 8;
-        }
-    } else {
-        for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
-                                           + off + i) << i * 8;
-        }
+        val = bswap64(val);
     }
 
-    return u64;
+    return val;
 }
 
 static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 10/11] tests: Use qpci_mem{read, write} in ivshmem-test
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (8 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle David Gibson
  2016-10-21  9:38 ` [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling Laurent Vivier
  11 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

ivshmem implements a block of shared memory in a PCI BAR.  Currently our
test case accesses this using qtest_mem{read,write}.  However, deducing
the correct addresses for these requires making assumptions about the
internel format returned by qpci_iomap(), along with some ugly casts.

This patch changes the test to use the new qpci_mem{read,write} interfaces
which is neater.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/ivshmem-test.c | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
index f36bfe7..fbd8258 100644
--- a/tests/ivshmem-test.c
+++ b/tests/ivshmem-test.c
@@ -93,6 +93,25 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
     global_qtest = qtest;
 }
 
+static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
+{
+    QTestState *qtest = global_qtest;
+
+    global_qtest = s->qtest;
+    qpci_memread(s->dev, s->mem_base + off, buf, len);
+    global_qtest = qtest;
+}
+
+static inline void write_mem(IVState *s, uint64_t off,
+                             const void *buf, size_t len)
+{
+    QTestState *qtest = global_qtest;
+
+    global_qtest = s->qtest;
+    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
+    global_qtest = qtest;
+}
+
 static void cleanup_vm(IVState *s)
 {
     g_free(s->dev);
@@ -169,7 +188,7 @@ static void test_ivshmem_single(void)
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
         data[i] = i;
     }
-    qtest_memwrite(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
+    write_mem(s, 0, data, sizeof(data));
 
     /* verify write */
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
@@ -178,7 +197,7 @@ static void test_ivshmem_single(void)
 
     /* read it back and verify read */
     memset(data, 0, sizeof(data));
-    qtest_memread(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
+    read_mem(s, 0, data, sizeof(data));
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
         g_assert_cmpuint(data[i], ==, i);
     }
@@ -201,29 +220,29 @@ static void test_ivshmem_pair(void)
 
     /* host write, guest 1 & 2 read */
     memset(tmpshmem, 0x42, TMPSHMSIZE);
-    qtest_memread(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
+    read_mem(s1, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x42);
     }
-    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s2, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x42);
     }
 
     /* guest 1 write, guest 2 read */
     memset(data, 0x43, TMPSHMSIZE);
-    qtest_memwrite(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
+    write_mem(s1, 0, data, TMPSHMSIZE);
     memset(data, 0, TMPSHMSIZE);
-    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s2, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x43);
     }
 
     /* guest 2 write, guest 1 read */
     memset(data, 0x44, TMPSHMSIZE);
-    qtest_memwrite(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    write_mem(s2, 0, data, TMPSHMSIZE);
     memset(data, 0, TMPSHMSIZE);
-    qtest_memread(s1->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s1, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x44);
     }
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (9 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 10/11] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
@ 2016-10-21  1:19 ` David Gibson
  2016-10-21 13:23   ` Greg Kurz
  2016-10-21  9:38 ` [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling Laurent Vivier
  11 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  1:19 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The usual use model for the libqos PCI functions is to map a specific PCI
BAR using qpci_iomap() then pass the returned token into IO accessor
functions.  This, and the fact that iomap() returns a (void *) which
actually contains a PCI space address, kind of suggests that the return
value from iomap is supposed to be an opaque token.

..except that the callers expect to be able to add offsets to it.  Which
also assumes the compiler will support pointer arithmetic on a (void *),
and treat it as working with byte offsets.

To clarify this situation change iomap() and the IO accessors to take
a definitely opaque BAR handle (enforced with a wrapper struct) along with
an offset within the BAR.  This changes both the functions and all the
callers.

Asserts that iomap() returns non-NULL are removed in some places; iomap()
already asserts if it can't map the BAR

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/ahci-test.c         |   4 +-
 tests/e1000e-test.c       |   7 +-
 tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
 tests/ivshmem-test.c      |  16 ++---
 tests/libqos/ahci.c       |   3 +-
 tests/libqos/ahci.h       |   6 +-
 tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
 tests/libqos/pci.h        |  50 ++++++++-----
 tests/libqos/usb.c        |   6 +-
 tests/libqos/usb.h        |   2 +-
 tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
 tests/libqos/virtio-pci.h |   2 +-
 tests/rtl8139-test.c      |  10 ++-
 tests/tco-test.c          |  80 ++++++++++-----------
 tests/usb-hcd-ehci-test.c |   5 +-
 15 files changed, 309 insertions(+), 311 deletions(-)

diff --git a/tests/ahci-test.c b/tests/ahci-test.c
index 9c0adce..4358631 100644
--- a/tests/ahci-test.c
+++ b/tests/ahci-test.c
@@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
     g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
 
     /* If we haven't initialized, this is as much as can be validated. */
-    if (!ahci->hba_base) {
+    if (!ahci->hba_bar.addr) {
         return;
     }
 
     hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
-    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
+    hba_stored = ahci->hba_bar.addr;
     g_assert_cmphex(hba_base, ==, hba_stored);
 
     g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP), ==, ahci->cap);
diff --git a/tests/e1000e-test.c b/tests/e1000e-test.c
index 3979b20..8c42ca9 100644
--- a/tests/e1000e-test.c
+++ b/tests/e1000e-test.c
@@ -87,7 +87,7 @@
 
 typedef struct e1000e_device {
     QPCIDevice *pci_dev;
-    void *mac_regs;
+    QPCIBar mac_regs;
 
     uint64_t tx_ring;
     uint64_t rx_ring;
@@ -119,12 +119,12 @@ static QPCIDevice *e1000e_device_find(QPCIBus *bus)
 
 static void e1000e_macreg_write(e1000e_device *d, uint32_t reg, uint32_t val)
 {
-    qpci_io_writel(d->pci_dev, d->mac_regs + reg, val);
+    qpci_io_writel(d->pci_dev, d->mac_regs, reg, val);
 }
 
 static uint32_t e1000e_macreg_read(e1000e_device *d, uint32_t reg)
 {
-    return qpci_io_readl(d->pci_dev, d->mac_regs + reg);
+    return qpci_io_readl(d->pci_dev, d->mac_regs, reg);
 }
 
 static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
@@ -138,7 +138,6 @@ static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
 
     /* Map BAR0 (mac registers) */
     d->mac_regs = qpci_iomap(d->pci_dev, 0, NULL);
-    g_assert_nonnull(d->mac_regs);
 
     /* Reset the device */
     val = e1000e_macreg_read(d, E1000E_CTRL);
diff --git a/tests/ide-test.c b/tests/ide-test.c
index 86c4373..67c7df0 100644
--- a/tests/ide-test.c
+++ b/tests/ide-test.c
@@ -137,7 +137,7 @@ static void ide_test_quit(void)
     qtest_end();
 }
 
-static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
+static QPCIDevice *get_pci_device(QPCIBar *bmdma_bar, QPCIBar *ide_bar)
 {
     QPCIDevice *dev;
     uint16_t vendor_id, device_id;
@@ -156,9 +156,9 @@ static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
     g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
 
     /* Map bmdma BAR */
-    *bmdma_base = qpci_iomap(dev, 4, NULL);
+    *bmdma_bar = qpci_iomap(dev, 4, NULL);
 
-    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
+    *ide_bar = qpci_legacy_iomap(dev, IDE_BASE);
 
     qpci_device_enable(dev);
 
@@ -181,19 +181,18 @@ typedef struct PrdtEntry {
 
 static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
                             PrdtEntry *prdt, int prdt_entries,
-                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
+                            void(*post_exec)(QPCIDevice *dev, QPCIBar ide_bar,
                                              uint64_t sector, int nb_sectors))
 {
     QPCIDevice *dev;
-    void *bmdma_base;
-    void *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uintptr_t guest_prdt;
     size_t len;
     bool from_dev;
     uint8_t status;
     int flags;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     flags = cmd & ~0xff;
     cmd &= 0xff;
@@ -218,60 +217,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
     }
 
     /* Select device 0 */
-    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0 | LBA);
 
     /* Stop any running transfer, clear any pending interrupt */
-    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
-    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
+    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
+    qpci_io_writeb(dev, bmdma_bar, bmreg_status, BM_STS_INTR);
 
     /* Setup PRDT */
     len = sizeof(*prdt) * prdt_entries;
     guest_prdt = guest_alloc(guest_malloc, len);
     memwrite(guest_prdt, prdt, len);
-    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
+    qpci_io_writel(dev, bmdma_bar, bmreg_prdt, guest_prdt);
 
     /* ATA DMA command */
     if (cmd == CMD_PACKET) {
         /* Enables ATAPI DMA; otherwise PIO is attempted */
-        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
+        qpci_io_writeb(dev, ide_bar, reg_feature, 0x01);
     } else {
-        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
-        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
-        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
-        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_nsectors, nb_sectors);
+        qpci_io_writeb(dev, ide_bar, reg_lba_low,    sector & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_lba_middle, (sector >> 8) & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_lba_high,   (sector >> 16) & 0xff);
     }
 
-    qpci_io_writeb(dev, ide_base + reg_command, cmd);
+    qpci_io_writeb(dev, ide_bar, reg_command, cmd);
 
     if (post_exec) {
-        post_exec(dev, ide_base, sector, nb_sectors);
+        post_exec(dev, ide_bar, sector, nb_sectors);
     }
 
     /* Start DMA transfer */
-    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
+    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd,
                    BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
 
     if (flags & CMDF_ABORT) {
-        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
     }
 
     /* Wait for the DMA transfer to complete */
     do {
-        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
+        status = qpci_io_readb(dev, bmdma_bar, bmreg_status);
     } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
 
     g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
 
     /* Check IDE status code */
-    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
+    assert_bit_set(qpci_io_readb(dev, ide_bar, reg_status), DRDY);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), BSY | DRQ);
 
     /* Reading the status register clears the IRQ */
     g_assert(!get_irq(IDE_PRIMARY_IRQ));
 
     /* Stop DMA transfer if still active */
     if (status & BM_STS_ACTIVE) {
-        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
     }
 
     free_pci_device(dev);
@@ -282,7 +281,7 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
 static void test_bmdma_simple_rw(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
     uint8_t *buf;
     uint8_t *cmpbuf;
@@ -296,7 +295,7 @@ static void test_bmdma_simple_rw(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     buf = g_malloc(len);
     cmpbuf = g_malloc(len);
@@ -308,7 +307,7 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Write 0xaa pattern to sector 1 */
     memset(buf, 0xaa, len);
@@ -317,14 +316,14 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Read and verify 0x55 pattern in sector 0 */
     memset(cmpbuf, 0x55, len);
 
     status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -334,7 +333,7 @@ static void test_bmdma_simple_rw(void)
 
     status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -347,7 +346,7 @@ static void test_bmdma_simple_rw(void)
 static void test_bmdma_short_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -357,25 +356,25 @@ static void test_bmdma_short_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_one_sector_short_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     /* Read 2 sectors but only give 1 sector in PRDT */
@@ -386,25 +385,25 @@ static void test_bmdma_one_sector_short_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_long_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -414,28 +413,28 @@ static void test_bmdma_long_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_no_busmaster(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
      * able to access it anyway because the Bus Master bit in the PCI command
@@ -449,7 +448,7 @@ static void test_bmdma_no_busmaster(void)
     /* Not entirely clear what the expected result is, but this is what we get
      * in practice. At least we want to be aware of any changes. */
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_setup(void)
@@ -480,7 +479,7 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
 static void test_identify(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     uint16_t buf[256];
     int i;
@@ -491,25 +490,25 @@ static void test_identify(void)
         "-global ide-hd.ver=%s",
         tmp_path, "testdisk", "version");
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* IDENTIFY command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_IDENTIFY);
 
     /* Read in the IDENTIFY buffer and check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     for (i = 0; i < 256; i++) {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
         assert_bit_set(data, DRDY | DRQ);
         assert_bit_clear(data, BSY | DF | ERR);
 
-        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
+        buf[i] = qpci_io_readw(dev, ide_bar, reg_data);
     }
 
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, DRDY);
     assert_bit_clear(data, BSY | DF | ERR | DRQ);
 
@@ -535,13 +534,13 @@ static void test_identify(void)
 static void make_dirty(uint8_t device)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
     size_t len = 512;
     uintptr_t guest_buf;
     void* buf;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     guest_buf = guest_alloc(guest_malloc, len);
     buf = g_malloc(len);
@@ -560,7 +559,7 @@ static void make_dirty(uint8_t device)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     g_free(buf);
 }
@@ -568,14 +567,14 @@ static void make_dirty(uint8_t device)
 static void test_flush(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
 
     ide_test_start(
         "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
         tmp_path);
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
@@ -586,11 +585,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -598,11 +597,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
 
     /* Check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -614,7 +613,7 @@ static void test_flush(void)
 static void test_retry_flush(const char *machine)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     const char *s;
 
@@ -626,7 +625,7 @@ static void test_retry_flush(const char *machine)
         "rerror=stop,werror=stop",
         debug_path, tmp_path);
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
@@ -634,11 +633,11 @@ static void test_retry_flush(const char *machine)
     make_dirty(0);
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -649,11 +648,11 @@ static void test_retry_flush(const char *machine)
     qmp_discard_response(s);
 
     /* Check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -665,15 +664,15 @@ static void test_retry_flush(const char *machine)
 static void test_flush_nodev(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
 
     ide_test_start("");
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Just testing that qemu doesn't crash... */
 
@@ -700,7 +699,7 @@ typedef struct Read10CDB {
     uint16_t padding;
 } __attribute__((__packed__)) Read10CDB;
 
-static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
+static void send_scsi_cdb_read10(QPCIDevice *dev, QPCIBar ide_bar,
                                  uint64_t lba, int nblocks)
 {
     Read10CDB pkt = { .padding = 0 };
@@ -717,7 +716,7 @@ static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
 
     /* Send Packet */
     for (i = 0; i < sizeof(Read10CDB)/2; i++) {
-        qpci_io_writew(dev, ide_base + reg_data,
+        qpci_io_writew(dev, ide_bar, reg_data,
                        le16_to_cpu(((uint16_t *)&pkt)[i]));
     }
 }
@@ -732,16 +731,16 @@ static void nsleep(int64_t nsecs)
 static uint8_t ide_wait_clear(uint8_t flag)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     time_t st;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Wait with a 5 second timeout */
     time(&st);
     while (true) {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
         if (!(data & flag)) {
             return data;
         }
@@ -776,7 +775,7 @@ static void ide_wait_intr(int irq)
 static void cdrom_pio_impl(int nblocks)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     FILE *fh;
     int patt_blocks = MAX(16, nblocks);
     size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
@@ -795,15 +794,14 @@ static void cdrom_pio_impl(int nblocks)
 
     ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
                    "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* PACKET command on device 0 */
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
-    qpci_io_writeb(dev, ide_base + reg_lba_high,
-                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
+    qpci_io_writeb(dev, ide_bar, reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_PACKET);
     /* HP0: Check_Status_A State */
     nsleep(400);
     data = ide_wait_clear(BSY);
@@ -812,7 +810,7 @@ static void cdrom_pio_impl(int nblocks)
     assert_bit_clear(data, ERR | DF | BSY);
 
     /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
-    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
+    send_scsi_cdb_read10(dev, ide_bar, 0, nblocks);
 
     /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
      * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
@@ -836,8 +834,8 @@ static void cdrom_pio_impl(int nblocks)
 
         /* HP4: Transfer_Data */
         for (j = 0; j < MIN((limit / 2), rem); j++) {
-            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
-                                                       ide_base + reg_data));
+            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev, ide_bar,
+                                                       reg_data));
         }
     }
 
diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
index fbd8258..04a5c5d 100644
--- a/tests/ivshmem-test.c
+++ b/tests/ivshmem-test.c
@@ -41,7 +41,7 @@ static QPCIDevice *get_device(QPCIBus *pcibus)
 
 typedef struct _IVState {
     QTestState *qtest;
-    void *reg_base, *mem_base;
+    QPCIBar reg_bar, mem_bar;
     QPCIBus *pcibus;
     QPCIDevice *dev;
 } IVState;
@@ -75,7 +75,7 @@ static inline unsigned in_reg(IVState *s, enum Reg reg)
     unsigned res;
 
     global_qtest = s->qtest;
-    res = qpci_io_readl(s->dev, s->reg_base + reg);
+    res = qpci_io_readl(s->dev, s->reg_bar, reg);
     g_test_message("*%s -> %x\n", name, res);
     global_qtest = qtest;
 
@@ -89,7 +89,7 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
 
     global_qtest = s->qtest;
     g_test_message("%x -> *%s\n", v, name);
-    qpci_io_writel(s->dev, s->reg_base + reg, v);
+    qpci_io_writel(s->dev, s->reg_bar, reg, v);
     global_qtest = qtest;
 }
 
@@ -98,7 +98,7 @@ static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
     QTestState *qtest = global_qtest;
 
     global_qtest = s->qtest;
-    qpci_memread(s->dev, s->mem_base + off, buf, len);
+    qpci_memread(s->dev, s->mem_bar, off, buf, len);
     global_qtest = qtest;
 }
 
@@ -108,7 +108,7 @@ static inline void write_mem(IVState *s, uint64_t off,
     QTestState *qtest = global_qtest;
 
     global_qtest = s->qtest;
-    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
+    qpci_memwrite(s->dev, s->mem_bar, off, buf, len);
     global_qtest = qtest;
 }
 
@@ -127,16 +127,14 @@ static void setup_vm_cmd(IVState *s, const char *cmd, bool msix)
     s->pcibus = qpci_init_pc(NULL);
     s->dev = get_device(s->pcibus);
 
-    s->reg_base = qpci_iomap(s->dev, 0, &barsize);
-    g_assert_nonnull(s->reg_base);
+    s->reg_bar = qpci_iomap(s->dev, 0, &barsize);
     g_assert_cmpuint(barsize, ==, 256);
 
     if (msix) {
         qpci_msix_enable(s->dev);
     }
 
-    s->mem_base = qpci_iomap(s->dev, 2, &barsize);
-    g_assert_nonnull(s->mem_base);
+    s->mem_bar = qpci_iomap(s->dev, 2, &barsize);
     g_assert_cmpuint(barsize, ==, TMPSHMSIZE);
 
     qpci_device_enable(s->dev);
diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
index 716ab79..43b6695 100644
--- a/tests/libqos/ahci.c
+++ b/tests/libqos/ahci.c
@@ -210,8 +210,7 @@ void ahci_pci_enable(AHCIQState *ahci)
 void start_ahci_device(AHCIQState *ahci)
 {
     /* Map AHCI's ABAR (BAR5) */
-    ahci->hba_base = qpci_iomap(ahci->dev, 5, &ahci->barsize);
-    g_assert(ahci->hba_base);
+    ahci->hba_bar = qpci_iomap(ahci->dev, 5, &ahci->barsize);
 
     /* turns on pci.cmd.iose, pci.cmd.mse and pci.cmd.bme */
     qpci_device_enable(ahci->dev);
diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
index c69fb5a..6c4abf6 100644
--- a/tests/libqos/ahci.h
+++ b/tests/libqos/ahci.h
@@ -321,7 +321,7 @@ typedef struct AHCIPortQState {
 typedef struct AHCIQState {
     QOSState *parent;
     QPCIDevice *dev;
-    void *hba_base;
+    QPCIBar hba_bar;
     uint64_t barsize;
     uint32_t fingerprint;
     uint32_t cap;
@@ -488,12 +488,12 @@ typedef struct AHCIOpts {
 
 static inline uint32_t ahci_mread(AHCIQState *ahci, size_t offset)
 {
-    return qpci_io_readl(ahci->dev, ahci->hba_base + offset);
+    return qpci_io_readl(ahci->dev, ahci->hba_bar, offset);
 }
 
 static inline void ahci_mwrite(AHCIQState *ahci, size_t offset, uint32_t value)
 {
-    qpci_io_writel(ahci->dev, ahci->hba_base + offset, value);
+    qpci_io_writel(ahci->dev, ahci->hba_bar, offset, value);
 }
 
 static inline uint32_t ahci_rreg(AHCIQState *ahci, uint32_t reg_num)
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 3021651..f357324 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -104,7 +104,6 @@ void qpci_msix_enable(QPCIDevice *dev)
     uint32_t table;
     uint8_t bir_table;
     uint8_t bir_pba;
-    void *offset;
 
     addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
     g_assert_cmphex(addr, !=, 0);
@@ -114,18 +113,16 @@ void qpci_msix_enable(QPCIDevice *dev)
 
     table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE);
     bir_table = table & PCI_MSIX_FLAGS_BIRMASK;
-    offset = qpci_iomap(dev, bir_table, NULL);
-    dev->msix_table = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
+    dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL);
+    dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
 
     table = qpci_config_readl(dev, addr + PCI_MSIX_PBA);
     bir_pba = table & PCI_MSIX_FLAGS_BIRMASK;
     if (bir_pba != bir_table) {
-        offset = qpci_iomap(dev, bir_pba, NULL);
+        dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL);
     }
-    dev->msix_pba = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
+    dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
 
-    g_assert(dev->msix_table != NULL);
-    g_assert(dev->msix_pba != NULL);
     dev->msix_enabled = true;
 }
 
@@ -141,22 +138,25 @@ void qpci_msix_disable(QPCIDevice *dev)
     qpci_config_writew(dev, addr + PCI_MSIX_FLAGS,
                                                 val & ~PCI_MSIX_FLAGS_ENABLE);
 
-    qpci_iounmap(dev, dev->msix_table);
-    qpci_iounmap(dev, dev->msix_pba);
+    qpci_iounmap(dev, dev->msix_table_bar);
+    qpci_iounmap(dev, dev->msix_pba_bar);
     dev->msix_enabled = 0;
-    dev->msix_table = NULL;
-    dev->msix_pba = NULL;
+    dev->msix_table_bar = QPCI_BAR_INVALID;
+    dev->msix_pba_bar = QPCI_BAR_INVALID;
+    dev->msix_table_off = 0;
+    dev->msix_pba_off = 0;
 }
 
 bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry)
 {
     uint32_t pba_entry;
     uint8_t bit_n = entry % 32;
-    void *addr = dev->msix_pba + (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
+    uint64_t  off = (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
 
     g_assert(dev->msix_enabled);
-    pba_entry = qpci_io_readl(dev, addr);
-    qpci_io_writel(dev, addr, pba_entry & ~(1 << bit_n));
+    pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off);
+    qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off,
+                   pba_entry & ~(1 << bit_n));
     return (pba_entry & (1 << bit_n)) != 0;
 }
 
@@ -164,7 +164,7 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
 {
     uint8_t addr;
     uint16_t val;
-    void *vector_addr = dev->msix_table + (entry * PCI_MSIX_ENTRY_SIZE);
+    uint64_t vector_off = dev->msix_table_off + entry * PCI_MSIX_ENTRY_SIZE;
 
     g_assert(dev->msix_enabled);
     addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
@@ -174,8 +174,9 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
     if (val & PCI_MSIX_FLAGS_MASKALL) {
         return true;
     } else {
-        return (qpci_io_readl(dev, vector_addr + PCI_MSIX_ENTRY_VECTOR_CTRL)
-                                            & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
+        return (qpci_io_readl(dev, dev->msix_table_bar,
+                              vector_off + PCI_MSIX_ENTRY_VECTOR_CTRL)
+                & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
     }
 }
 
@@ -222,129 +223,115 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
     dev->bus->config_writel(dev->bus, dev->devfn, offset, value);
 }
 
-
-uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
+uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readb(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readb(dev->bus, token.addr + off);
     } else {
         uint8_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return val;
     }
 }
 
-uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
+uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readw(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readw(dev->bus, token.addr + off);
     } else {
         uint16_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le16_to_cpu(val);
     }
 }
 
-uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
+uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readl(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readl(dev->bus, token.addr + off);
     } else {
         uint32_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le32_to_cpu(val);
     }
 }
 
-uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
+uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readq(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readq(dev->bus, token.addr + off);
     } else {
         uint64_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le64_to_cpu(val);
     }
 }
 
-void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
+void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint8_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeb(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeb(dev->bus, token.addr + off, value);
     } else {
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
+void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint16_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writew(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writew(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le16(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
+void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint32_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writel(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writel(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le32(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
+void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint64_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeq(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeq(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le64(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
+void qpci_memread(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                  void *buf, size_t len)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    g_assert(addr >= QPCI_PIO_LIMIT);
-    dev->bus->memread(dev->bus, addr, buf, len);
+    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    dev->bus->memread(dev->bus, token.addr + off, buf, len);
 }
 
-void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
+void qpci_memwrite(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                   const void *buf, size_t len)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    g_assert(addr >= QPCI_PIO_LIMIT);
-    dev->bus->memwrite(dev->bus, addr, buf, len);
+    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    dev->bus->memwrite(dev->bus, token.addr + off, buf, len);
 }
 
-void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
+QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
     QPCIBus *bus = dev->bus;
     static const int bar_reg_map[] = {
         PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
         PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
     };
+    QPCIBar bar;
     int bar_reg;
     uint32_t addr, size;
     uint32_t io_type;
@@ -391,17 +378,19 @@ void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
         qpci_config_writel(dev, bar_reg, loc);
     }
 
-    return (void *)(uintptr_t)loc;
+    bar.addr = loc;
+    return bar;
 }
 
-void qpci_iounmap(QPCIDevice *dev, void *data)
+void qpci_iounmap(QPCIDevice *dev, QPCIBar bar)
 {
     /* FIXME */
 }
 
-void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
+QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
 {
-    return (void *)(uintptr_t)addr;
+    QPCIBar bar = { .addr = addr };
+    return bar;
 }
 
 void qpci_plug_device_test(const char *driver, const char *id,
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 531e3f7..279fa51 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -21,6 +21,7 @@
 
 typedef struct QPCIDevice QPCIDevice;
 typedef struct QPCIBus QPCIBus;
+typedef struct QPCIBar QPCIBar;
 
 struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
@@ -51,13 +52,21 @@ struct QPCIBus {
     uint64_t mmio_alloc_ptr, mmio_limit;
 };
 
+struct QPCIBar {
+    uint64_t addr;
+};
+
+static const QPCIBar QPCI_BAR_INVALID = {
+    .addr = (uint64_t)-1ULL,
+};
+
 struct QPCIDevice
 {
     QPCIBus *bus;
     int devfn;
     bool msix_enabled;
-    void *msix_table;
-    void *msix_pba;
+    QPCIBar msix_table_bar, msix_pba_bar;
+    uint64_t msix_table_off, msix_pba_off;
 };
 
 void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
@@ -81,22 +90,27 @@ void qpci_config_writeb(QPCIDevice *dev, uint8_t offset, uint8_t value);
 void qpci_config_writew(QPCIDevice *dev, uint8_t offset, uint16_t value);
 void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
 
-uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
-uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
-uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
-uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
-
-void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
-void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
-void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
-void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
-
-void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
-void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
-
-void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
-void qpci_iounmap(QPCIDevice *dev, void *data);
-void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
+uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off);
+
+void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint8_t value);
+void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint16_t value);
+void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint32_t value);
+void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint64_t value);
+
+void qpci_memread(QPCIDevice *bus, QPCIBar token, uint64_t off,
+                  void *buf, size_t len);
+void qpci_memwrite(QPCIDevice *bus, QPCIBar token, uint64_t off,
+                   const void *buf, size_t len);
+QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
+void qpci_iounmap(QPCIDevice *dev, QPCIBar addr);
+QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
 
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts);
diff --git a/tests/libqos/usb.c b/tests/libqos/usb.c
index f794d92..72d7a96 100644
--- a/tests/libqos/usb.c
+++ b/tests/libqos/usb.c
@@ -21,14 +21,12 @@ void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc, uint32_t devfn, int bar)
     hc->dev = qpci_device_find(pcibus, devfn);
     g_assert(hc->dev != NULL);
     qpci_device_enable(hc->dev);
-    hc->base = qpci_iomap(hc->dev, bar, NULL);
-    g_assert(hc->base != NULL);
+    hc->bar = qpci_iomap(hc->dev, bar, NULL);
 }
 
 void uhci_port_test(struct qhc *hc, int port, uint16_t expect)
 {
-    void *addr = hc->base + 0x10 + 2 * port;
-    uint16_t value = qpci_io_readw(hc->dev, addr);
+    uint16_t value = qpci_io_readw(hc->dev, hc->bar, 0x10 + 2 * port);
     uint16_t mask = ~(UHCI_PORT_WRITE_CLEAR | UHCI_PORT_RSVD1);
 
     g_assert((value & mask) == (expect & mask));
diff --git a/tests/libqos/usb.h b/tests/libqos/usb.h
index 8fe5687..423dcfd 100644
--- a/tests/libqos/usb.h
+++ b/tests/libqos/usb.h
@@ -5,7 +5,7 @@
 
 struct qhc {
     QPCIDevice *dev;
-    void *base;
+    QPCIBar bar;
 };
 
 void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc,
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index c69d09d..24ed779 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -62,13 +62,12 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
     *vpcidev = (QVirtioPCIDevice *)d;
 }
 
-#define CONFIG_BASE(dev) \
-    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
+#define CONFIG_BASE(dev) (VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
 
 static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
+    return qpci_io_readb(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
 }
 
 /* PCI is always read in little-endian order
@@ -84,7 +83,7 @@ static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint16_t value;
 
-    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
+    value = qpci_io_readw(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap16(value);
     }
@@ -96,7 +95,7 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint32_t value;
 
-    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
+    value = qpci_io_readl(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap32(value);
     }
@@ -108,7 +107,7 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint64_t val;
 
-    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
+    val = qpci_io_readq(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         val = bswap64(val);
     }
@@ -119,31 +118,31 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_HOST_FEATURES);
+    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_HOST_FEATURES);
 }
 
 static void qvirtio_pci_set_features(QVirtioDevice *d, uint32_t features)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES, features);
+    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES, features);
 }
 
 static uint32_t qvirtio_pci_get_guest_features(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES);
+    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES);
 }
 
 static uint8_t qvirtio_pci_get_status(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS);
+    return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS);
 }
 
 static void qvirtio_pci_set_status(QVirtioDevice *d, uint8_t status)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS, status);
+    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS, status);
 }
 
 static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
@@ -167,7 +166,7 @@ static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
             }
         }
     } else {
-        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 1;
+        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 1;
     }
 }
 
@@ -191,26 +190,26 @@ static bool qvirtio_pci_get_config_isr_status(QVirtioDevice *d)
             }
         }
     } else {
-        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 2;
+        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 2;
     }
 }
 
 static void qvirtio_pci_queue_select(QVirtioDevice *d, uint16_t index)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_SEL, index);
+    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_SEL, index);
 }
 
 static uint16_t qvirtio_pci_get_queue_size(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readw(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NUM);
+    return qpci_io_readw(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NUM);
 }
 
 static void qvirtio_pci_set_queue_address(QVirtioDevice *d, uint32_t pfn)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_PFN, pfn);
+    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_PFN, pfn);
 }
 
 static QVirtQueue *qvirtio_pci_virtqueue_setup(QVirtioDevice *d,
@@ -262,7 +261,7 @@ static void qvirtio_pci_virtqueue_cleanup(QVirtQueue *vq,
 static void qvirtio_pci_virtqueue_kick(QVirtioDevice *d, QVirtQueue *vq)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writew(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
+    qpci_io_writew(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
 }
 
 const QVirtioBus qvirtio_pci = {
@@ -309,14 +308,13 @@ QVirtioPCIDevice *qvirtio_pci_device_find(QPCIBus *bus, uint16_t device_type)
 void qvirtio_pci_device_enable(QVirtioPCIDevice *d)
 {
     qpci_device_enable(d->pdev);
-    d->addr = qpci_iomap(d->pdev, 0, NULL);
-    g_assert(d->addr != NULL);
+    d->bar = qpci_iomap(d->pdev, 0, NULL);
 }
 
 void qvirtio_pci_device_disable(QVirtioPCIDevice *d)
 {
-    qpci_iounmap(d->pdev, d->addr);
-    d->addr = NULL;
+    qpci_iounmap(d->pdev, d->bar);
+    d->bar = QPCI_BAR_INVALID;
 }
 
 void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
@@ -324,29 +322,33 @@ void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
 {
     uint16_t vector;
     uint32_t control;
-    void *addr;
+    uint64_t off;
 
     g_assert(d->pdev->msix_enabled);
-    addr = d->pdev->msix_table + (entry * 16);
+    off = d->pdev->msix_table_off + (entry * 16);
 
     g_assert_cmpint(entry, >=, 0);
     g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
     vqpci->msix_entry = entry;
 
     vqpci->msix_addr = guest_alloc(alloc, 4);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
-                                                    vqpci->msix_addr & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
-                                            (vqpci->msix_addr >> 32) & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
-
-    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
-                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_LOWER_ADDR, vqpci->msix_addr & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
+                   (vqpci->msix_addr >> 32) & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
+
+    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
+                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
+                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
 
     qvirtio_pci_queue_select(&d->vdev, vqpci->vq.index);
-    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR, entry);
-    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR);
+    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR, entry);
+    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR);
     g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
 }
 
@@ -355,10 +357,10 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
 {
     uint16_t vector;
     uint32_t control;
-    void *addr;
+    uint64_t off;
 
     g_assert(d->pdev->msix_enabled);
-    addr = d->pdev->msix_table + (entry * 16);
+    off = d->pdev->msix_table_off + (entry * 16);
 
     g_assert_cmpint(entry, >=, 0);
     g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
@@ -367,17 +369,21 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
     d->config_msix_data = 0x12345678;
     d->config_msix_addr = guest_alloc(alloc, 4);
 
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
-                                                    d->config_msix_addr & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
-                                            (d->config_msix_addr >> 32) & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
-
-    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
-                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
-
-    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR, entry);
-    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_LOWER_ADDR, d->config_msix_addr & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
+                   (d->config_msix_addr >> 32) & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
+
+    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
+                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
+                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
+
+    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR, entry);
+    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR);
     g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
 }
diff --git a/tests/libqos/virtio-pci.h b/tests/libqos/virtio-pci.h
index efcac2d..38c54c6 100644
--- a/tests/libqos/virtio-pci.h
+++ b/tests/libqos/virtio-pci.h
@@ -16,7 +16,7 @@
 typedef struct QVirtioPCIDevice {
     QVirtioDevice vdev;
     QPCIDevice *pdev;
-    void *addr;
+    QPCIBar bar;
     uint16_t config_msix_entry;
     uint64_t config_msix_addr;
     uint32_t config_msix_data;
diff --git a/tests/rtl8139-test.c b/tests/rtl8139-test.c
index c2f601a..7de7dc4 100644
--- a/tests/rtl8139-test.c
+++ b/tests/rtl8139-test.c
@@ -22,7 +22,7 @@ static void nop(void)
 
 static QPCIBus *pcibus;
 static QPCIDevice *dev;
-static void *dev_base;
+static QPCIBar dev_bar;
 
 static void save_fn(QPCIDevice *dev, int devfn, void *data)
 {
@@ -45,14 +45,14 @@ static QPCIDevice *get_device(void)
 #define PORT(name, len, val) \
 static unsigned __attribute__((unused)) in_##name(void) \
 { \
-    unsigned res = qpci_io_read##len(dev, dev_base+(val)); \
+    unsigned res = qpci_io_read##len(dev, dev_bar, (val));     \
     g_test_message("*%s -> %x\n", #name, res); \
     return res; \
 } \
 static void out_##name(unsigned v) \
 { \
     g_test_message("%x -> *%s\n", v, #name); \
-    qpci_io_write##len(dev, dev_base+(val), v); \
+    qpci_io_write##len(dev, dev_bar, (val), v);        \
 }
 
 PORT(Timer, l, 0x48)
@@ -186,9 +186,7 @@ static void test_init(void)
 
     dev = get_device();
 
-    dev_base = qpci_iomap(dev, 0, &barsize);
-
-    g_assert(dev_base != NULL);
+    dev_bar = qpci_iomap(dev, 0, &barsize);
 
     qpci_device_enable(dev);
 
diff --git a/tests/tco-test.c b/tests/tco-test.c
index 129577d..ef02ec5 100644
--- a/tests/tco-test.c
+++ b/tests/tco-test.c
@@ -41,7 +41,7 @@ typedef struct {
     const char *args;
     bool noreboot;
     QPCIDevice *dev;
-    void *tco_io_base;
+    QPCIBar tco_io_bar;
 } TestData;
 
 static void test_init(TestData *d)
@@ -70,42 +70,42 @@ static void test_init(TestData *d)
     /* set Root Complex BAR */
     qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
 
-    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
+    d->tco_io_bar = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
 }
 
 static void stop_tco(const TestData *d)
 {
     uint32_t val;
 
-    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
+    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
     val |= TCO_TMR_HLT;
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
 }
 
 static void start_tco(const TestData *d)
 {
     uint32_t val;
 
-    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
+    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
     val &= ~TCO_TMR_HLT;
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
 }
 
 static void load_tco(const TestData *d)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO_RLD, 4);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO_RLD, 4);
 }
 
 static void set_tco_timeout(const TestData *d, uint16_t ticks)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO_TMR, ticks);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO_TMR, ticks);
 }
 
 static void clear_tco_status(const TestData *d)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_STS, 0x0008);
-    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0002);
-    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0004);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_STS, 0x0008);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0002);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0004);
 }
 
 static void reset_on_second_timeout(bool enable)
@@ -128,25 +128,25 @@ static void test_tco_defaults(void)
     d.args = NULL;
     d.noreboot = true;
     test_init(&d);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD), ==,
                     TCO_RLD_DEFAULT);
     /* TCO_DAT_IN & TCO_DAT_OUT */
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_DAT_IN), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_DAT_IN), ==,
                     (TCO_DAT_OUT_DEFAULT << 8) | TCO_DAT_IN_DEFAULT);
     /* TCO1_STS & TCO2_STS */
-    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_STS), ==,
+    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_STS), ==,
                     (TCO2_STS_DEFAULT << 16) | TCO1_STS_DEFAULT);
     /* TCO1_CNT & TCO2_CNT */
-    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_CNT), ==,
+    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_CNT), ==,
                     (TCO2_CNT_DEFAULT << 16) | TCO1_CNT_DEFAULT);
     /* TCO_MESSAGE1 & TCO_MESSAGE2 */
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_MESSAGE1), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_MESSAGE1), ==,
                     (TCO_MESSAGE2_DEFAULT << 8) | TCO_MESSAGE1_DEFAULT);
-    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + TCO_WDCNT), ==,
+    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, TCO_WDCNT), ==,
                     TCO_WDCNT_DEFAULT);
-    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + SW_IRQ_GEN), ==,
+    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, SW_IRQ_GEN), ==,
                     SW_IRQ_GEN_DEFAULT);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_TMR), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_TMR), ==,
                     TCO_TMR_DEFAULT);
     qtest_end();
 }
@@ -171,23 +171,23 @@ static void test_tco_timeout(void)
     clock_step(ticks * TCO_TICK_NSEC);
 
     /* test first timeout */
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
 
     /* test clearing timeout bit */
     val |= TCO_TIMEOUT;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 0);
 
     /* test second timeout */
     clock_step(ticks * TCO_TICK_NSEC);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
     ret = val & TCO_SECOND_TO_STS ? 1 : 0;
     g_assert(ret == 1);
 
@@ -214,13 +214,13 @@ static void test_tco_max_timeout(void)
     start_tco(&d);
     clock_step(((ticks & TCO_TMR_MASK) - 1) * TCO_TICK_NSEC);
 
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD);
     g_assert_cmpint(val & TCO_RLD_MASK, ==, 1);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 0);
     clock_step(TCO_TICK_NSEC);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
 
@@ -358,11 +358,11 @@ static void test_tco_ticks_counter(void)
     start_tco(&d);
 
     do {
-        rld = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD) & TCO_RLD_MASK;
+        rld = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD) & TCO_RLD_MASK;
         g_assert_cmpint(rld, ==, ticks);
         clock_step(TCO_TICK_NSEC);
         ticks--;
-    } while (!(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS) & TCO_TIMEOUT));
+    } while (!(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS) & TCO_TIMEOUT));
 
     stop_tco(&d);
     qtest_end();
@@ -378,10 +378,10 @@ static void test_tco1_control_bits(void)
     test_init(&d);
 
     val = TCO_LOCK;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
     val &= ~TCO_LOCK;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_CNT), ==,
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_CNT), ==,
                     TCO_LOCK);
     qtest_end();
 }
@@ -405,13 +405,13 @@ static void test_tco1_status_bits(void)
     start_tco(&d);
     clock_step(ticks * TCO_TICK_NSEC);
 
-    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_IN, 0);
-    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_OUT, 0);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_IN, 0);
+    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_OUT, 0);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & (TCO_TIMEOUT | SW_TCO_SMI | TCO_INT_STS) ? 1 : 0;
     g_assert(ret == 1);
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS), ==, 0);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS), ==, 0);
     qtest_end();
 }
 
@@ -434,11 +434,11 @@ static void test_tco2_status_bits(void)
     start_tco(&d);
     clock_step(ticks * TCO_TICK_NSEC * 2);
 
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
     ret = val & (TCO_SECOND_TO_STS | TCO_BOOT_STS) ? 1 : 0;
     g_assert(ret == 1);
-    qpci_io_writew(d.dev, d.tco_io_base + TCO2_STS, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS), ==, 0);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO2_STS, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS), ==, 0);
     qtest_end();
 }
 
diff --git a/tests/usb-hcd-ehci-test.c b/tests/usb-hcd-ehci-test.c
index a4ceeaa..57af8a0 100644
--- a/tests/usb-hcd-ehci-test.c
+++ b/tests/usb-hcd-ehci-test.c
@@ -38,8 +38,7 @@ static void uhci_port_update(struct qhc *hc, int port,
 
 static void ehci_port_test(struct qhc *hc, int port, uint32_t expect)
 {
-    void *addr = hc->base + 0x64 + 4 * port;
-    uint32_t value = qpci_io_readl(hc->dev, addr);
+    uint32_t value = qpci_io_readl(hc->dev, hc->bar, 0x64 + 4 * port);
     uint16_t mask = ~(PORTSC_CSC | PORTSC_PEDC | PORTSC_OCC);
 
 #if 0
@@ -91,7 +90,7 @@ static void pci_ehci_port_1(void)
 static void pci_ehci_config(void)
 {
     /* hands over all ports from companion uhci to ehci */
-    qpci_io_writew(ehci1.dev, ehci1.base + 0x60, 1);
+    qpci_io_writew(ehci1.dev, ehci1.bar, 0x60, 1);
 }
 
 static void pci_uhci_port_2(void)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test David Gibson
@ 2016-10-21  8:31   ` Laurent Vivier
  2016-10-21  9:05     ` David Gibson
  0 siblings, 1 reply; 27+ messages in thread
From: Laurent Vivier @ 2016-10-21  8:31 UTC (permalink / raw)
  To: David Gibson, pbonzini, qemu-devel
  Cc: qemu-ppc, agraf, stefanha, mst, aik, mdroth, groug, thuth



On 21/10/2016 03:19, David Gibson wrote:
> ide-test uses many explicit inb() / outb() operations for its IO, which
> means it's not portable to non-x86 platforms.  This cleans it up to use
> the libqos PCI accessors instead.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
>  1 file changed, 118 insertions(+), 61 deletions(-)
> 
> diff --git a/tests/ide-test.c b/tests/ide-test.c
> index a8a4081..86c4373 100644
> --- a/tests/ide-test.c
> +++ b/tests/ide-test.c
...
> @@ -654,7 +700,8 @@ typedef struct Read10CDB {
>      uint16_t padding;
>  } __attribute__((__packed__)) Read10CDB;
>  
> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> +                                 uint64_t lba, int nblocks)
>  {
>      Read10CDB pkt = { .padding = 0 };
>      int i;
> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>  
>      /* Send Packet */
>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> +        qpci_io_writew(dev, ide_base + reg_data,
> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
>      }
>  }
>  
...
> @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
>  
>          /* HP4: Transfer_Data */
>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> +                                                       ide_base + reg_data));
>          }
>      }

Why do you swap le16_to_cpu() and cpu_to_le16()?

I think this is not semantically correct and the purpose of this patch
is only to replace inX()/outX() functions.

Laurent

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test
  2016-10-21  8:31   ` Laurent Vivier
@ 2016-10-21  9:05     ` David Gibson
  2016-10-21 10:07       ` Laurent Vivier
  0 siblings, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-21  9:05 UTC (permalink / raw)
  To: Laurent Vivier
  Cc: pbonzini, qemu-devel, qemu-ppc, agraf, stefanha, mst, aik,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 3159 bytes --]

On Fri, Oct 21, 2016 at 10:31:27AM +0200, Laurent Vivier wrote:
> 
> 
> On 21/10/2016 03:19, David Gibson wrote:
> > ide-test uses many explicit inb() / outb() operations for its IO, which
> > means it's not portable to non-x86 platforms.  This cleans it up to use
> > the libqos PCI accessors instead.
> > 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > ---
> >  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
> >  1 file changed, 118 insertions(+), 61 deletions(-)
> > 
> > diff --git a/tests/ide-test.c b/tests/ide-test.c
> > index a8a4081..86c4373 100644
> > --- a/tests/ide-test.c
> > +++ b/tests/ide-test.c
> ...
> > @@ -654,7 +700,8 @@ typedef struct Read10CDB {
> >      uint16_t padding;
> >  } __attribute__((__packed__)) Read10CDB;
> >  
> > -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> > +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> > +                                 uint64_t lba, int nblocks)
> >  {
> >      Read10CDB pkt = { .padding = 0 };
> >      int i;
> > @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >  
> >      /* Send Packet */
> >      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> > -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> > +        qpci_io_writew(dev, ide_base + reg_data,
> > +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
> >      }
> >  }
> >  
> ...
> > @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
> >  
> >          /* HP4: Transfer_Data */
> >          for (j = 0; j < MIN((limit / 2), rem); j++) {
> > -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> > +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> > +                                                       ide_base + reg_data));
> >          }
> >      }
> 
> Why do you swap le16_to_cpu() and cpu_to_le16()?

So, obviously le16_to_cpu() and cpu_to_le16() are functionally
identical.  But I think my version is conceptually more correct.

This is streaming data via PIO - we're essentially reading a byte
stream in 16-bit chunks.  So, overall we want to preserve byte address
order.  qpci_io_readw() (and inw()) is designed for reading registers
so it preserves byte significance, rather than address order.  So,
since the IDE registers are LE, that means if implicitly contains an
le16_to_cpu().  The cpu_to_le16() undoes that so that rx[] ends up
with the bytestream in the correct order.

> I think this is not semantically correct and the purpose of this patch
> is only to replace inX()/outX() functions.

No, I believe it is correct.  And I think the original was technically
wrong, because inw() reads the register in target endian rather than
LE.  Of course the only common platform with "real" in/out
instructions is x86, which is LE anyway, so it hardly matters.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling
  2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
                   ` (10 preceding siblings ...)
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle David Gibson
@ 2016-10-21  9:38 ` Laurent Vivier
  11 siblings, 0 replies; 27+ messages in thread
From: Laurent Vivier @ 2016-10-21  9:38 UTC (permalink / raw)
  To: David Gibson, pbonzini, qemu-devel
  Cc: qemu-ppc, agraf, stefanha, mst, aik, mdroth, groug, thuth



On 21/10/2016 03:19, David Gibson wrote:
> This series contains a number of cleanups to the libqos code for
> accessing PCI devices, and to tests which use it.
> 
> The general aim is to improve the consistency of semantics across
> functions, and reduce the amount of intimate knowledge of the libqos
> PCI layer needed by tests.
> 
> This should make it easier to write PCI tests which will be portable
> to different guest machines with different PCI host bridge
> arrangements.
> 
> This series is on top of my ppc-for-2.8 branch, since it contains
> patches enabling the virtio tests on ppc, which would otherwise
> conflict with the changes here.
> 
> Changes since v3:
>   * Fixed another endian bug introduced in ide-test

This works well now on LE and BE hosts and all targets.

Tested-by: Laurent Vivier <lvivier@redhat.com>

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO David Gibson
@ 2016-10-21 10:05   ` Greg Kurz
  2016-10-22  4:50     ` David Gibson
  0 siblings, 1 reply; 27+ messages in thread
From: Greg Kurz @ 2016-10-21 10:05 UTC (permalink / raw)
  To: David Gibson, aik
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, thuth

On Fri, 21 Oct 2016 12:19:45 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> The usual model for PCI IO with libqos is to use qpci_iomap() to map a
> specific BAR for a PCI device, then perform IOs within that BAR using
> qpci_io_{read,write}*().
> 
> However, certain devices also have legacy PCI IO.  In this case, instead of
> (or as well as) being accessed via PCI BARs, the device can be accessed
> via certain well-known, fixed addresses in PCI IO space.
> 
> Two existing tests use legacy PCI IO, and take different flawed approaches
> to it:
>     * tco-test manually constructs a tco_io_base value instead of calling
>       qpci_iomap(), which assumes internal knowledge of the structure of
>       the value it shouldn't have
>     * ide-test uses direct in*() and out*() calls instead of using
>       qpci_io_*() accessors, meaning it's not portable to non-x86 machine
>       types.
> 
> This patch implements a new qpci_iomap_legacy() interface which gets a
> handle in the same format as qpci_iomap() but refers to a region in
> the legacy PIO space.  For a device which has the same registers
> available both in a BAR and in legacy space (quite common), this
> allows the same test code to test both options with just a different
> iomap() at the beginning.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---
>  tests/libqos/pci.c | 5 +++++
>  tests/libqos/pci.h | 1 +
>  2 files changed, 6 insertions(+)
> 
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index bf1c532..98a2e56 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -350,6 +350,11 @@ void qpci_iounmap(QPCIDevice *dev, void *data)
>      /* FIXME */
>  }
>  
> +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
> +{
> +    return (void *)(uintptr_t)addr;
> +}
> +

Since both tco-test and ide-test are hardcoded for x86 machine types, is
it right for this implementation to sit in common PCI code instead of
the PC specific one ?

>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts)
>  {
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index f6f916d..b6f855e 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -94,6 +94,7 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
>  
>  void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
>  void qpci_iounmap(QPCIDevice *dev, void *data);
> +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts);

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test
  2016-10-21  9:05     ` David Gibson
@ 2016-10-21 10:07       ` Laurent Vivier
  2016-10-22  4:56         ` David Gibson
  0 siblings, 1 reply; 27+ messages in thread
From: Laurent Vivier @ 2016-10-21 10:07 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, agraf, stefanha, mst, aik,
	mdroth, groug, thuth



On 21/10/2016 11:05, David Gibson wrote:
> On Fri, Oct 21, 2016 at 10:31:27AM +0200, Laurent Vivier wrote:
>>
>>
>> On 21/10/2016 03:19, David Gibson wrote:
>>> ide-test uses many explicit inb() / outb() operations for its IO, which
>>> means it's not portable to non-x86 platforms.  This cleans it up to use
>>> the libqos PCI accessors instead.
>>>
>>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
>>> ---
>>>  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
>>>  1 file changed, 118 insertions(+), 61 deletions(-)
>>>
>>> diff --git a/tests/ide-test.c b/tests/ide-test.c
>>> index a8a4081..86c4373 100644
>>> --- a/tests/ide-test.c
>>> +++ b/tests/ide-test.c
>> ...
>>> @@ -654,7 +700,8 @@ typedef struct Read10CDB {
>>>      uint16_t padding;
>>>  } __attribute__((__packed__)) Read10CDB;
>>>  
>>> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>>> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
>>> +                                 uint64_t lba, int nblocks)
>>>  {
>>>      Read10CDB pkt = { .padding = 0 };
>>>      int i;
>>> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>>>  
>>>      /* Send Packet */
>>>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
>>> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
>>> +        qpci_io_writew(dev, ide_base + reg_data,
>>> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
>>>      }
>>>  }
>>>  
>> ...
>>> @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
>>>  
>>>          /* HP4: Transfer_Data */
>>>          for (j = 0; j < MIN((limit / 2), rem); j++) {
>>> -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
>>> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
>>> +                                                       ide_base + reg_data));
>>>          }
>>>      }
>>
>> Why do you swap le16_to_cpu() and cpu_to_le16()?
> 
> So, obviously le16_to_cpu() and cpu_to_le16() are functionally
> identical.  But I think my version is conceptually more correct.

I agree

> 
> This is streaming data via PIO - we're essentially reading a byte
> stream in 16-bit chunks.  So, overall we want to preserve byte address
> order.  qpci_io_readw() (and inw()) is designed for reading registers
> so it preserves byte significance, rather than address order.  So,
> since the IDE registers are LE, that means if implicitly contains an
> le16_to_cpu().  The cpu_to_le16() undoes that so that rx[] ends up
> with the bytestream in the correct order.

In fact, it has an implicit "le16 to target CPU", so you should not have
a "cpu_to_le16()" but a "qtest_big_endian() != host_big_endian
bswap16()" to swap it.

As this test works only with a little endian guest, the first "LE16 to
target CPU" is a no-op, and the second one does something only on big
endian host so when "qtest_big_endian() != host_big_endian".

>> I think this is not semantically correct and the purpose of this patch
>> is only to replace inX()/outX() functions.
> 
> No, I believe it is correct.  And I think the original was technically
> wrong, because inw() reads the register in target endian rather than
> LE.  Of course the only common platform with "real" in/out
> instructions is x86, which is LE anyway, so it hardly matters.

I agree, this is why I think a "qtest_big_endian() != host_big_endian
then bswap16()" is better than a "cpu_to_le16()".

But I also think it should be the purpose of another patch...

Thanks,
Laurent

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
@ 2016-10-21 10:08   ` Greg Kurz
  0 siblings, 0 replies; 27+ messages in thread
From: Greg Kurz @ 2016-10-21 10:08 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Fri, 21 Oct 2016 12:19:48 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> In the libqos PCI code we now have accessors both for registers (byte
> significance preserving) and for streaming data (byte address order
> preserving).  These exist in both the interface for qtest drivers and in
> the machine specific backends.
> 
> However, the register-style accessors aren't actually necessary in the
> backend.  They can be implemented in terms of the byte address order
> preserving accessors by the libqos wrappers.  This works because PCI is
> always little endian.
> 
> This does assume that the back end byte address order preserving accessors
> will perform the equivalent of a single bus transaction for short lengths.
> This is the case, and in fact they currently end up using the same
> cpu_physical_memory_rw() implementation within the qtest accelerator.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/libqos/pci-pc.c    | 38 --------------------------------------
>  tests/libqos/pci-spapr.c | 44 --------------------------------------------
>  tests/libqos/pci.c       | 20 ++++++++++++++------
>  tests/libqos/pci.h       |  8 --------
>  4 files changed, 14 insertions(+), 96 deletions(-)
> 
> diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
> index 84aee25..849ea56 100644
> --- a/tests/libqos/pci-pc.c
> +++ b/tests/libqos/pci-pc.c
> @@ -32,61 +32,31 @@ static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
>      return inb(addr);
>  }
>  
> -static uint8_t qpci_pc_mmio_readb(QPCIBus *bus, uint32_t addr)
> -{
> -    return readb(addr);
> -}
> -
>  static void qpci_pc_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
>  {
>      outb(addr, val);
>  }
>  
> -static void qpci_pc_mmio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
> -{
> -    writeb(addr, val);
> -}
> -
>  static uint16_t qpci_pc_pio_readw(QPCIBus *bus, uint32_t addr)
>  {
>      return inw(addr);
>  }
>  
> -static uint16_t qpci_pc_mmio_readw(QPCIBus *bus, uint32_t addr)
> -{
> -    return readw(addr);
> -}
> -
>  static void qpci_pc_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
>  {
>      outw(addr, val);
>  }
>  
> -static void qpci_pc_mmio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
> -{
> -    writew(addr, val);
> -}
> -
>  static uint32_t qpci_pc_pio_readl(QPCIBus *bus, uint32_t addr)
>  {
>      return inl(addr);
>  }
>  
> -static uint32_t qpci_pc_mmio_readl(QPCIBus *bus, uint32_t addr)
> -{
> -    return readl(addr);
> -}
> -
>  static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
>  {
>      outl(addr, val);
>  }
>  
> -static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
> -{
> -    writel(addr, val);
> -}
> -
>  static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
>  {
>      memread(addr, buf, len);
> @@ -148,14 +118,6 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
>      ret->bus.pio_writew = qpci_pc_pio_writew;
>      ret->bus.pio_writel = qpci_pc_pio_writel;
>  
> -    ret->bus.mmio_readb = qpci_pc_mmio_readb;
> -    ret->bus.mmio_readw = qpci_pc_mmio_readw;
> -    ret->bus.mmio_readl = qpci_pc_mmio_readl;
> -
> -    ret->bus.mmio_writeb = qpci_pc_mmio_writeb;
> -    ret->bus.mmio_writew = qpci_pc_mmio_writew;
> -    ret->bus.mmio_writel = qpci_pc_mmio_writel;
> -
>      ret->bus.memread = qpci_pc_memread;
>      ret->bus.memwrite = qpci_pc_memwrite;
>  
> diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
> index ad12c2e..f26488a 100644
> --- a/tests/libqos/pci-spapr.c
> +++ b/tests/libqos/pci-spapr.c
> @@ -48,72 +48,36 @@ static uint8_t qpci_spapr_pio_readb(QPCIBus *bus, uint32_t addr)
>      return readb(s->pio_cpu_base + addr);
>  }
>  
> -static uint8_t qpci_spapr_mmio32_readb(QPCIBus *bus, uint32_t addr)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    return readb(s->mmio32_cpu_base + addr);
> -}
> -
>  static void qpci_spapr_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
>  {
>      QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
>      writeb(s->pio_cpu_base + addr, val);
>  }
>  
> -static void qpci_spapr_mmio32_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    writeb(s->mmio32_cpu_base + addr, val);
> -}
> -
>  static uint16_t qpci_spapr_pio_readw(QPCIBus *bus, uint32_t addr)
>  {
>      QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
>      return bswap16(readw(s->pio_cpu_base + addr));
>  }
>  
> -static uint16_t qpci_spapr_mmio32_readw(QPCIBus *bus, uint32_t addr)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    return bswap16(readw(s->mmio32_cpu_base + addr));
> -}
> -
>  static void qpci_spapr_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
>  {
>      QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
>      writew(s->pio_cpu_base + addr, bswap16(val));
>  }
>  
> -static void qpci_spapr_mmio32_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    writew(s->mmio32_cpu_base + addr, bswap16(val));
> -}
> -
>  static uint32_t qpci_spapr_pio_readl(QPCIBus *bus, uint32_t addr)
>  {
>      QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
>      return bswap32(readl(s->pio_cpu_base + addr));
>  }
>  
> -static uint32_t qpci_spapr_mmio32_readl(QPCIBus *bus, uint32_t addr)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    return bswap32(readl(s->mmio32_cpu_base + addr));
> -}
> -
>  static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
>  {
>      QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
>      writel(s->pio_cpu_base + addr, bswap32(val));
>  }
>  
> -static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
> -{
> -    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> -    writel(s->mmio32_cpu_base + addr, bswap32(val));
> -}
> -
>  static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
>                                 void *buf, size_t len)
>  {
> @@ -194,14 +158,6 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
>      ret->bus.pio_writew = qpci_spapr_pio_writew;
>      ret->bus.pio_writel = qpci_spapr_pio_writel;
>  
> -    ret->bus.mmio_readb = qpci_spapr_mmio32_readb;
> -    ret->bus.mmio_readw = qpci_spapr_mmio32_readw;
> -    ret->bus.mmio_readl = qpci_spapr_mmio32_readl;
> -
> -    ret->bus.mmio_writeb = qpci_spapr_mmio32_writeb;
> -    ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
> -    ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
> -
>      ret->bus.memread = qpci_spapr_memread;
>      ret->bus.memwrite = qpci_spapr_memwrite;
>  
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index 146b063..bdffeb3 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -230,7 +230,9 @@ uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
>      if (addr < QPCI_PIO_LIMIT) {
>          return dev->bus->pio_readb(dev->bus, addr);
>      } else {
> -        return dev->bus->mmio_readb(dev->bus, addr);
> +        uint8_t val;
> +        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        return val;
>      }
>  }
>  
> @@ -241,7 +243,9 @@ uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
>      if (addr < QPCI_PIO_LIMIT) {
>          return dev->bus->pio_readw(dev->bus, addr);
>      } else {
> -        return dev->bus->mmio_readw(dev->bus, addr);
> +        uint16_t val;
> +        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        return le16_to_cpu(val);
>      }
>  }
>  
> @@ -252,7 +256,9 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
>      if (addr < QPCI_PIO_LIMIT) {
>          return dev->bus->pio_readl(dev->bus, addr);
>      } else {
> -        return dev->bus->mmio_readl(dev->bus, addr);
> +        uint32_t val;
> +        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        return le32_to_cpu(val);
>      }
>  }
>  
> @@ -263,7 +269,7 @@ void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
>      if (addr < QPCI_PIO_LIMIT) {
>          dev->bus->pio_writeb(dev->bus, addr, value);
>      } else {
> -        dev->bus->mmio_writeb(dev->bus, addr, value);
> +        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
>      }
>  }
>  
> @@ -274,7 +280,8 @@ void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
>      if (addr < QPCI_PIO_LIMIT) {
>          dev->bus->pio_writew(dev->bus, addr, value);
>      } else {
> -        dev->bus->mmio_writew(dev->bus, addr, value);
> +        value = cpu_to_le16(value);
> +        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
>      }
>  }
>  
> @@ -285,7 +292,8 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
>      if (addr < QPCI_PIO_LIMIT) {
>          dev->bus->pio_writel(dev->bus, addr, value);
>      } else {
> -        dev->bus->mmio_writel(dev->bus, addr, value);
> +        value = cpu_to_le32(value);
> +        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
>      }
>  }
>  
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index 2b08362..ce6ed08 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -27,18 +27,10 @@ struct QPCIBus {
>      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
>      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
>  
> -    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
> -    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
> -    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
> -
>      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
>      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
>      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
>  
> -    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> -    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> -    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> -
>      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
>      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
>  

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors David Gibson
@ 2016-10-21 10:31   ` Greg Kurz
  0 siblings, 0 replies; 27+ messages in thread
From: Greg Kurz @ 2016-10-21 10:31 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Fri, 21 Oct 2016 12:19:50 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> Currently the libqos PCI layer includes accessor helpers for 8, 16 and 32
> bit reads and writes.  It's likely that we'll want 64-bit accesses in the
> future (plenty of modern peripherals will have 64-bit reigsters).  This
> adds them.
> 
> For PIO (not MMIO) accesses on the PC backend, this is implemented as two
> 32-bit ins or outs.  That's not ideal but AFAICT x86 doesn't have 64-bit
> versions of in and out.
> 
> This patch also converts the single current user of 64-bit accesses -
> virtio-pci.c to use the new mechanism, rather than a sequence of 8 byte
> reads.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/libqos/pci-pc.c     | 13 +++++++++++++
>  tests/libqos/pci-spapr.c  | 14 ++++++++++++++
>  tests/libqos/pci.c        | 25 +++++++++++++++++++++++++
>  tests/libqos/pci.h        |  4 ++++
>  tests/libqos/virtio-pci.c | 16 ++++------------
>  5 files changed, 60 insertions(+), 12 deletions(-)
> 
> diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
> index 849ea56..ded1c54 100644
> --- a/tests/libqos/pci-pc.c
> +++ b/tests/libqos/pci-pc.c
> @@ -57,6 +57,17 @@ static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
>      outl(addr, val);
>  }
>  
> +static uint64_t qpci_pc_pio_readq(QPCIBus *bus, uint32_t addr)
> +{
> +    return (uint64_t)inl(addr) + ((uint64_t)inl(addr + 4) << 32);
> +}
> +
> +static void qpci_pc_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
> +{
> +    outl(addr, val & 0xffffffff);
> +    outl(addr + 4, val >> 32);
> +}
> +
>  static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
>  {
>      memread(addr, buf, len);
> @@ -113,10 +124,12 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
>      ret->bus.pio_readb = qpci_pc_pio_readb;
>      ret->bus.pio_readw = qpci_pc_pio_readw;
>      ret->bus.pio_readl = qpci_pc_pio_readl;
> +    ret->bus.pio_readq = qpci_pc_pio_readq;
>  
>      ret->bus.pio_writeb = qpci_pc_pio_writeb;
>      ret->bus.pio_writew = qpci_pc_pio_writew;
>      ret->bus.pio_writel = qpci_pc_pio_writel;
> +    ret->bus.pio_writeq = qpci_pc_pio_writeq;
>  
>      ret->bus.memread = qpci_pc_memread;
>      ret->bus.memwrite = qpci_pc_memwrite;
> diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
> index f26488a..1e5d015 100644
> --- a/tests/libqos/pci-spapr.c
> +++ b/tests/libqos/pci-spapr.c
> @@ -78,6 +78,18 @@ static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
>      writel(s->pio_cpu_base + addr, bswap32(val));
>  }
>  
> +static uint64_t qpci_spapr_pio_readq(QPCIBus *bus, uint32_t addr)
> +{
> +    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> +    return bswap64(readq(s->pio_cpu_base + addr));
> +}
> +
> +static void qpci_spapr_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
> +{
> +    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
> +    writeq(s->pio_cpu_base + addr, bswap64(val));
> +}
> +
>  static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
>                                 void *buf, size_t len)
>  {
> @@ -153,10 +165,12 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
>      ret->bus.pio_readb = qpci_spapr_pio_readb;
>      ret->bus.pio_readw = qpci_spapr_pio_readw;
>      ret->bus.pio_readl = qpci_spapr_pio_readl;
> +    ret->bus.pio_readq = qpci_spapr_pio_readq;
>  
>      ret->bus.pio_writeb = qpci_spapr_pio_writeb;
>      ret->bus.pio_writew = qpci_spapr_pio_writew;
>      ret->bus.pio_writel = qpci_spapr_pio_writel;
> +    ret->bus.pio_writeq = qpci_spapr_pio_writeq;
>  
>      ret->bus.memread = qpci_spapr_memread;
>      ret->bus.memwrite = qpci_spapr_memwrite;
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index bdffeb3..3021651 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -262,6 +262,19 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
>      }
>  }
>  
> +uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
> +{
> +    uintptr_t addr = (uintptr_t)data;
> +
> +    if (addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readq(dev->bus, addr);
> +    } else {
> +        uint64_t val;
> +        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        return le64_to_cpu(val);
> +    }
> +}
> +
>  void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
>  {
>      uintptr_t addr = (uintptr_t)data;
> @@ -297,6 +310,18 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
>      }
>  }
>  
> +void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
> +{
> +    uintptr_t addr = (uintptr_t)data;
> +
> +    if (addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writeq(dev->bus, addr, value);
> +    } else {
> +        value = cpu_to_le64(value);
> +        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +    }
> +}
> +
>  void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
>  {
>      uintptr_t addr = (uintptr_t)data;
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index ce6ed08..531e3f7 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -26,10 +26,12 @@ struct QPCIBus {
>      uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
>      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
>      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
> +    uint64_t (*pio_readq)(QPCIBus *bus, uint32_t addr);
>  
>      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
>      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
>      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> +    void (*pio_writeq)(QPCIBus *bus, uint32_t addr, uint64_t value);
>  
>      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
>      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
> @@ -82,10 +84,12 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
>  uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
>  uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
>  uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
> +uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
>  
>  void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
>  void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
>  void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
> +void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
>  
>  void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
>  void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
> diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
> index fa82132..c69d09d 100644
> --- a/tests/libqos/virtio-pci.c
> +++ b/tests/libqos/virtio-pci.c
> @@ -106,22 +106,14 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
>  static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    int i;
> -    uint64_t u64 = 0;
> +    uint64_t val;
>  
> +    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
> -        for (i = 0; i < 8; ++i) {
> -            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
> -                                           + off + i) << (7 - i) * 8;
> -        }
> -    } else {
> -        for (i = 0; i < 8; ++i) {
> -            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
> -                                           + off + i) << i * 8;
> -        }
> +        val = bswap64(val);
>      }
>  
> -    return u64;
> +    return val;
>  }
>  
>  static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle David Gibson
@ 2016-10-21 13:23   ` Greg Kurz
  2016-10-24  3:19     ` David Gibson
  0 siblings, 1 reply; 27+ messages in thread
From: Greg Kurz @ 2016-10-21 13:23 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Fri, 21 Oct 2016 12:19:52 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> The usual use model for the libqos PCI functions is to map a specific PCI
> BAR using qpci_iomap() then pass the returned token into IO accessor
> functions.  This, and the fact that iomap() returns a (void *) which
> actually contains a PCI space address, kind of suggests that the return
> value from iomap is supposed to be an opaque token.
> 
> ..except that the callers expect to be able to add offsets to it.  Which
> also assumes the compiler will support pointer arithmetic on a (void *),
> and treat it as working with byte offsets.
> 
> To clarify this situation change iomap() and the IO accessors to take
> a definitely opaque BAR handle (enforced with a wrapper struct) along with
> an offset within the BAR.  This changes both the functions and all the
> callers.
> 
> Asserts that iomap() returns non-NULL are removed in some places; iomap()
> already asserts if it can't map the BAR
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  tests/ahci-test.c         |   4 +-
>  tests/e1000e-test.c       |   7 +-
>  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
>  tests/ivshmem-test.c      |  16 ++---
>  tests/libqos/ahci.c       |   3 +-
>  tests/libqos/ahci.h       |   6 +-
>  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
>  tests/libqos/pci.h        |  50 ++++++++-----
>  tests/libqos/usb.c        |   6 +-
>  tests/libqos/usb.h        |   2 +-
>  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
>  tests/libqos/virtio-pci.h |   2 +-
>  tests/rtl8139-test.c      |  10 ++-
>  tests/tco-test.c          |  80 ++++++++++-----------
>  tests/usb-hcd-ehci-test.c |   5 +-
>  15 files changed, 309 insertions(+), 311 deletions(-)
> 
> diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> index 9c0adce..4358631 100644
> --- a/tests/ahci-test.c
> +++ b/tests/ahci-test.c
> @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
>      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
>  
>      /* If we haven't initialized, this is as much as can be validated. */
> -    if (!ahci->hba_base) {
> +    if (!ahci->hba_bar.addr) {

Isn't ahci->hba_bar supposed to be opaque ?

>          return;
>      }

Unrelated to this patch, does it make sense to call verify_state() if
ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?

>  
>      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> +    hba_stored = ahci->hba_bar.addr;
>      g_assert_cmphex(hba_base, ==, hba_stored);

Again since ahci->hba_bar is opaque, is it right to do that check here ?


I have another question about QPCI_BAR_INVALID far below (patch is long :)

>  
>      g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP), ==, ahci->cap);
> diff --git a/tests/e1000e-test.c b/tests/e1000e-test.c
> index 3979b20..8c42ca9 100644
> --- a/tests/e1000e-test.c
> +++ b/tests/e1000e-test.c
> @@ -87,7 +87,7 @@
>  
>  typedef struct e1000e_device {
>      QPCIDevice *pci_dev;
> -    void *mac_regs;
> +    QPCIBar mac_regs;
>  
>      uint64_t tx_ring;
>      uint64_t rx_ring;
> @@ -119,12 +119,12 @@ static QPCIDevice *e1000e_device_find(QPCIBus *bus)
>  
>  static void e1000e_macreg_write(e1000e_device *d, uint32_t reg, uint32_t val)
>  {
> -    qpci_io_writel(d->pci_dev, d->mac_regs + reg, val);
> +    qpci_io_writel(d->pci_dev, d->mac_regs, reg, val);
>  }
>  
>  static uint32_t e1000e_macreg_read(e1000e_device *d, uint32_t reg)
>  {
> -    return qpci_io_readl(d->pci_dev, d->mac_regs + reg);
> +    return qpci_io_readl(d->pci_dev, d->mac_regs, reg);
>  }
>  
>  static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
> @@ -138,7 +138,6 @@ static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
>  
>      /* Map BAR0 (mac registers) */
>      d->mac_regs = qpci_iomap(d->pci_dev, 0, NULL);
> -    g_assert_nonnull(d->mac_regs);
>  
>      /* Reset the device */
>      val = e1000e_macreg_read(d, E1000E_CTRL);
> diff --git a/tests/ide-test.c b/tests/ide-test.c
> index 86c4373..67c7df0 100644
> --- a/tests/ide-test.c
> +++ b/tests/ide-test.c
> @@ -137,7 +137,7 @@ static void ide_test_quit(void)
>      qtest_end();
>  }
>  
> -static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
> +static QPCIDevice *get_pci_device(QPCIBar *bmdma_bar, QPCIBar *ide_bar)
>  {
>      QPCIDevice *dev;
>      uint16_t vendor_id, device_id;
> @@ -156,9 +156,9 @@ static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
>      g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
>  
>      /* Map bmdma BAR */
> -    *bmdma_base = qpci_iomap(dev, 4, NULL);
> +    *bmdma_bar = qpci_iomap(dev, 4, NULL);
>  
> -    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
> +    *ide_bar = qpci_legacy_iomap(dev, IDE_BASE);
>  
>      qpci_device_enable(dev);
>  
> @@ -181,19 +181,18 @@ typedef struct PrdtEntry {
>  
>  static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>                              PrdtEntry *prdt, int prdt_entries,
> -                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
> +                            void(*post_exec)(QPCIDevice *dev, QPCIBar ide_bar,
>                                               uint64_t sector, int nb_sectors))
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base;
> -    void *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uintptr_t guest_prdt;
>      size_t len;
>      bool from_dev;
>      uint8_t status;
>      int flags;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      flags = cmd & ~0xff;
>      cmd &= 0xff;
> @@ -218,60 +217,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>      }
>  
>      /* Select device 0 */
> -    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0 | LBA);
>  
>      /* Stop any running transfer, clear any pending interrupt */
> -    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> -    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_status, BM_STS_INTR);
>  
>      /* Setup PRDT */
>      len = sizeof(*prdt) * prdt_entries;
>      guest_prdt = guest_alloc(guest_malloc, len);
>      memwrite(guest_prdt, prdt, len);
> -    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
> +    qpci_io_writel(dev, bmdma_bar, bmreg_prdt, guest_prdt);
>  
>      /* ATA DMA command */
>      if (cmd == CMD_PACKET) {
>          /* Enables ATAPI DMA; otherwise PIO is attempted */
> -        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
> +        qpci_io_writeb(dev, ide_bar, reg_feature, 0x01);
>      } else {
> -        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
> -        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
> -        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
> -        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_nsectors, nb_sectors);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_low,    sector & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_middle, (sector >> 8) & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_high,   (sector >> 16) & 0xff);
>      }
>  
> -    qpci_io_writeb(dev, ide_base + reg_command, cmd);
> +    qpci_io_writeb(dev, ide_bar, reg_command, cmd);
>  
>      if (post_exec) {
> -        post_exec(dev, ide_base, sector, nb_sectors);
> +        post_exec(dev, ide_bar, sector, nb_sectors);
>      }
>  
>      /* Start DMA transfer */
> -    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd,
>                     BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
>  
>      if (flags & CMDF_ABORT) {
> -        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
>      }
>  
>      /* Wait for the DMA transfer to complete */
>      do {
> -        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
> +        status = qpci_io_readb(dev, bmdma_bar, bmreg_status);
>      } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
>  
>      g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
>  
>      /* Check IDE status code */
> -    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
> +    assert_bit_set(qpci_io_readb(dev, ide_bar, reg_status), DRDY);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), BSY | DRQ);
>  
>      /* Reading the status register clears the IRQ */
>      g_assert(!get_irq(IDE_PRIMARY_IRQ));
>  
>      /* Stop DMA transfer if still active */
>      if (status & BM_STS_ACTIVE) {
> -        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
>      }
>  
>      free_pci_device(dev);
> @@ -282,7 +281,7 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>  static void test_bmdma_simple_rw(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>      uint8_t *buf;
>      uint8_t *cmpbuf;
> @@ -296,7 +295,7 @@ static void test_bmdma_simple_rw(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      buf = g_malloc(len);
>      cmpbuf = g_malloc(len);
> @@ -308,7 +307,7 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Write 0xaa pattern to sector 1 */
>      memset(buf, 0xaa, len);
> @@ -317,14 +316,14 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Read and verify 0x55 pattern in sector 0 */
>      memset(cmpbuf, 0x55, len);
>  
>      status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -334,7 +333,7 @@ static void test_bmdma_simple_rw(void)
>  
>      status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -347,7 +346,7 @@ static void test_bmdma_simple_rw(void)
>  static void test_bmdma_short_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -357,25 +356,25 @@ static void test_bmdma_short_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_one_sector_short_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      /* Read 2 sectors but only give 1 sector in PRDT */
> @@ -386,25 +385,25 @@ static void test_bmdma_one_sector_short_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_long_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -414,28 +413,28 @@ static void test_bmdma_long_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_no_busmaster(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
>       * able to access it anyway because the Bus Master bit in the PCI command
> @@ -449,7 +448,7 @@ static void test_bmdma_no_busmaster(void)
>      /* Not entirely clear what the expected result is, but this is what we get
>       * in practice. At least we want to be aware of any changes. */
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_setup(void)
> @@ -480,7 +479,7 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
>  static void test_identify(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      uint16_t buf[256];
>      int i;
> @@ -491,25 +490,25 @@ static void test_identify(void)
>          "-global ide-hd.ver=%s",
>          tmp_path, "testdisk", "version");
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* IDENTIFY command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_IDENTIFY);
>  
>      /* Read in the IDENTIFY buffer and check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      for (i = 0; i < 256; i++) {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>          assert_bit_set(data, DRDY | DRQ);
>          assert_bit_clear(data, BSY | DF | ERR);
>  
> -        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
> +        buf[i] = qpci_io_readw(dev, ide_bar, reg_data);
>      }
>  
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, DRDY);
>      assert_bit_clear(data, BSY | DF | ERR | DRQ);
>  
> @@ -535,13 +534,13 @@ static void test_identify(void)
>  static void make_dirty(uint8_t device)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>      size_t len = 512;
>      uintptr_t guest_buf;
>      void* buf;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      guest_buf = guest_alloc(guest_malloc, len);
>      buf = g_malloc(len);
> @@ -560,7 +559,7 @@ static void make_dirty(uint8_t device)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      g_free(buf);
>  }
> @@ -568,14 +567,14 @@ static void make_dirty(uint8_t device)
>  static void test_flush(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>  
>      ide_test_start(
>          "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
>          tmp_path);
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
> @@ -586,11 +585,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -598,11 +597,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
>  
>      /* Check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -614,7 +613,7 @@ static void test_flush(void)
>  static void test_retry_flush(const char *machine)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      const char *s;
>  
> @@ -626,7 +625,7 @@ static void test_retry_flush(const char *machine)
>          "rerror=stop,werror=stop",
>          debug_path, tmp_path);
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
> @@ -634,11 +633,11 @@ static void test_retry_flush(const char *machine)
>      make_dirty(0);
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -649,11 +648,11 @@ static void test_retry_flush(const char *machine)
>      qmp_discard_response(s);
>  
>      /* Check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -665,15 +664,15 @@ static void test_retry_flush(const char *machine)
>  static void test_flush_nodev(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>  
>      ide_test_start("");
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Just testing that qemu doesn't crash... */
>  
> @@ -700,7 +699,7 @@ typedef struct Read10CDB {
>      uint16_t padding;
>  } __attribute__((__packed__)) Read10CDB;
>  
> -static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> +static void send_scsi_cdb_read10(QPCIDevice *dev, QPCIBar ide_bar,
>                                   uint64_t lba, int nblocks)
>  {
>      Read10CDB pkt = { .padding = 0 };
> @@ -717,7 +716,7 @@ static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
>  
>      /* Send Packet */
>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> -        qpci_io_writew(dev, ide_base + reg_data,
> +        qpci_io_writew(dev, ide_bar, reg_data,
>                         le16_to_cpu(((uint16_t *)&pkt)[i]));
>      }
>  }
> @@ -732,16 +731,16 @@ static void nsleep(int64_t nsecs)
>  static uint8_t ide_wait_clear(uint8_t flag)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      time_t st;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Wait with a 5 second timeout */
>      time(&st);
>      while (true) {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>          if (!(data & flag)) {
>              return data;
>          }
> @@ -776,7 +775,7 @@ static void ide_wait_intr(int irq)
>  static void cdrom_pio_impl(int nblocks)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      FILE *fh;
>      int patt_blocks = MAX(16, nblocks);
>      size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
> @@ -795,15 +794,14 @@ static void cdrom_pio_impl(int nblocks)
>  
>      ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
>                     "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* PACKET command on device 0 */
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> -    qpci_io_writeb(dev, ide_base + reg_lba_high,
> -                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> +    qpci_io_writeb(dev, ide_bar, reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_PACKET);
>      /* HP0: Check_Status_A State */
>      nsleep(400);
>      data = ide_wait_clear(BSY);
> @@ -812,7 +810,7 @@ static void cdrom_pio_impl(int nblocks)
>      assert_bit_clear(data, ERR | DF | BSY);
>  
>      /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
> -    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
> +    send_scsi_cdb_read10(dev, ide_bar, 0, nblocks);
>  
>      /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
>       * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
> @@ -836,8 +834,8 @@ static void cdrom_pio_impl(int nblocks)
>  
>          /* HP4: Transfer_Data */
>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> -            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> -                                                       ide_base + reg_data));
> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev, ide_bar,
> +                                                       reg_data));
>          }
>      }
>  
> diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
> index fbd8258..04a5c5d 100644
> --- a/tests/ivshmem-test.c
> +++ b/tests/ivshmem-test.c
> @@ -41,7 +41,7 @@ static QPCIDevice *get_device(QPCIBus *pcibus)
>  
>  typedef struct _IVState {
>      QTestState *qtest;
> -    void *reg_base, *mem_base;
> +    QPCIBar reg_bar, mem_bar;
>      QPCIBus *pcibus;
>      QPCIDevice *dev;
>  } IVState;
> @@ -75,7 +75,7 @@ static inline unsigned in_reg(IVState *s, enum Reg reg)
>      unsigned res;
>  
>      global_qtest = s->qtest;
> -    res = qpci_io_readl(s->dev, s->reg_base + reg);
> +    res = qpci_io_readl(s->dev, s->reg_bar, reg);
>      g_test_message("*%s -> %x\n", name, res);
>      global_qtest = qtest;
>  
> @@ -89,7 +89,7 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
>  
>      global_qtest = s->qtest;
>      g_test_message("%x -> *%s\n", v, name);
> -    qpci_io_writel(s->dev, s->reg_base + reg, v);
> +    qpci_io_writel(s->dev, s->reg_bar, reg, v);
>      global_qtest = qtest;
>  }
>  
> @@ -98,7 +98,7 @@ static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
>      QTestState *qtest = global_qtest;
>  
>      global_qtest = s->qtest;
> -    qpci_memread(s->dev, s->mem_base + off, buf, len);
> +    qpci_memread(s->dev, s->mem_bar, off, buf, len);
>      global_qtest = qtest;
>  }
>  
> @@ -108,7 +108,7 @@ static inline void write_mem(IVState *s, uint64_t off,
>      QTestState *qtest = global_qtest;
>  
>      global_qtest = s->qtest;
> -    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
> +    qpci_memwrite(s->dev, s->mem_bar, off, buf, len);
>      global_qtest = qtest;
>  }
>  
> @@ -127,16 +127,14 @@ static void setup_vm_cmd(IVState *s, const char *cmd, bool msix)
>      s->pcibus = qpci_init_pc(NULL);
>      s->dev = get_device(s->pcibus);
>  
> -    s->reg_base = qpci_iomap(s->dev, 0, &barsize);
> -    g_assert_nonnull(s->reg_base);
> +    s->reg_bar = qpci_iomap(s->dev, 0, &barsize);
>      g_assert_cmpuint(barsize, ==, 256);
>  
>      if (msix) {
>          qpci_msix_enable(s->dev);
>      }
>  
> -    s->mem_base = qpci_iomap(s->dev, 2, &barsize);
> -    g_assert_nonnull(s->mem_base);
> +    s->mem_bar = qpci_iomap(s->dev, 2, &barsize);
>      g_assert_cmpuint(barsize, ==, TMPSHMSIZE);
>  
>      qpci_device_enable(s->dev);
> diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
> index 716ab79..43b6695 100644
> --- a/tests/libqos/ahci.c
> +++ b/tests/libqos/ahci.c
> @@ -210,8 +210,7 @@ void ahci_pci_enable(AHCIQState *ahci)
>  void start_ahci_device(AHCIQState *ahci)
>  {
>      /* Map AHCI's ABAR (BAR5) */
> -    ahci->hba_base = qpci_iomap(ahci->dev, 5, &ahci->barsize);
> -    g_assert(ahci->hba_base);
> +    ahci->hba_bar = qpci_iomap(ahci->dev, 5, &ahci->barsize);
>  
>      /* turns on pci.cmd.iose, pci.cmd.mse and pci.cmd.bme */
>      qpci_device_enable(ahci->dev);
> diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
> index c69fb5a..6c4abf6 100644
> --- a/tests/libqos/ahci.h
> +++ b/tests/libqos/ahci.h
> @@ -321,7 +321,7 @@ typedef struct AHCIPortQState {
>  typedef struct AHCIQState {
>      QOSState *parent;
>      QPCIDevice *dev;
> -    void *hba_base;
> +    QPCIBar hba_bar;
>      uint64_t barsize;
>      uint32_t fingerprint;
>      uint32_t cap;
> @@ -488,12 +488,12 @@ typedef struct AHCIOpts {
>  
>  static inline uint32_t ahci_mread(AHCIQState *ahci, size_t offset)
>  {
> -    return qpci_io_readl(ahci->dev, ahci->hba_base + offset);
> +    return qpci_io_readl(ahci->dev, ahci->hba_bar, offset);
>  }
>  
>  static inline void ahci_mwrite(AHCIQState *ahci, size_t offset, uint32_t value)
>  {
> -    qpci_io_writel(ahci->dev, ahci->hba_base + offset, value);
> +    qpci_io_writel(ahci->dev, ahci->hba_bar, offset, value);
>  }
>  
>  static inline uint32_t ahci_rreg(AHCIQState *ahci, uint32_t reg_num)
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index 3021651..f357324 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -104,7 +104,6 @@ void qpci_msix_enable(QPCIDevice *dev)
>      uint32_t table;
>      uint8_t bir_table;
>      uint8_t bir_pba;
> -    void *offset;
>  
>      addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
>      g_assert_cmphex(addr, !=, 0);
> @@ -114,18 +113,16 @@ void qpci_msix_enable(QPCIDevice *dev)
>  
>      table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE);
>      bir_table = table & PCI_MSIX_FLAGS_BIRMASK;
> -    offset = qpci_iomap(dev, bir_table, NULL);
> -    dev->msix_table = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
> +    dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL);
> +    dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
>  
>      table = qpci_config_readl(dev, addr + PCI_MSIX_PBA);
>      bir_pba = table & PCI_MSIX_FLAGS_BIRMASK;
>      if (bir_pba != bir_table) {
> -        offset = qpci_iomap(dev, bir_pba, NULL);
> +        dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL);
>      }
> -    dev->msix_pba = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
> +    dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
>  
> -    g_assert(dev->msix_table != NULL);
> -    g_assert(dev->msix_pba != NULL);
>      dev->msix_enabled = true;
>  }
>  
> @@ -141,22 +138,25 @@ void qpci_msix_disable(QPCIDevice *dev)
>      qpci_config_writew(dev, addr + PCI_MSIX_FLAGS,
>                                                  val & ~PCI_MSIX_FLAGS_ENABLE);
>  
> -    qpci_iounmap(dev, dev->msix_table);
> -    qpci_iounmap(dev, dev->msix_pba);
> +    qpci_iounmap(dev, dev->msix_table_bar);
> +    qpci_iounmap(dev, dev->msix_pba_bar);
>      dev->msix_enabled = 0;
> -    dev->msix_table = NULL;
> -    dev->msix_pba = NULL;
> +    dev->msix_table_bar = QPCI_BAR_INVALID;
> +    dev->msix_pba_bar = QPCI_BAR_INVALID;
> +    dev->msix_table_off = 0;
> +    dev->msix_pba_off = 0;
>  }
>  
>  bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry)
>  {
>      uint32_t pba_entry;
>      uint8_t bit_n = entry % 32;
> -    void *addr = dev->msix_pba + (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
> +    uint64_t  off = (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
>  
>      g_assert(dev->msix_enabled);
> -    pba_entry = qpci_io_readl(dev, addr);
> -    qpci_io_writel(dev, addr, pba_entry & ~(1 << bit_n));
> +    pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off);
> +    qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off,
> +                   pba_entry & ~(1 << bit_n));
>      return (pba_entry & (1 << bit_n)) != 0;
>  }
>  
> @@ -164,7 +164,7 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
>  {
>      uint8_t addr;
>      uint16_t val;
> -    void *vector_addr = dev->msix_table + (entry * PCI_MSIX_ENTRY_SIZE);
> +    uint64_t vector_off = dev->msix_table_off + entry * PCI_MSIX_ENTRY_SIZE;
>  
>      g_assert(dev->msix_enabled);
>      addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
> @@ -174,8 +174,9 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
>      if (val & PCI_MSIX_FLAGS_MASKALL) {
>          return true;
>      } else {
> -        return (qpci_io_readl(dev, vector_addr + PCI_MSIX_ENTRY_VECTOR_CTRL)
> -                                            & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
> +        return (qpci_io_readl(dev, dev->msix_table_bar,
> +                              vector_off + PCI_MSIX_ENTRY_VECTOR_CTRL)
> +                & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
>      }
>  }
>  
> @@ -222,129 +223,115 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
>      dev->bus->config_writel(dev->bus, dev->devfn, offset, value);
>  }
>  
> -
> -uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
> +uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readb(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readb(dev->bus, token.addr + off);
>      } else {
>          uint8_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return val;
>      }
>  }
>  
> -uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
> +uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readw(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readw(dev->bus, token.addr + off);
>      } else {
>          uint16_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le16_to_cpu(val);
>      }
>  }
>  
> -uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
> +uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readl(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readl(dev->bus, token.addr + off);
>      } else {
>          uint32_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le32_to_cpu(val);
>      }
>  }
>  
> -uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
> +uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readq(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readq(dev->bus, token.addr + off);
>      } else {
>          uint64_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le64_to_cpu(val);
>      }
>  }
>  
> -void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
> +void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint8_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writeb(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writeb(dev->bus, token.addr + off, value);
>      } else {
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
> +void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint16_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writew(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writew(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le16(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
> +void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint32_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writel(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writel(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le32(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
> +void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint64_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writeq(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writeq(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le64(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
> +void qpci_memread(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                  void *buf, size_t len)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    g_assert(addr >= QPCI_PIO_LIMIT);
> -    dev->bus->memread(dev->bus, addr, buf, len);
> +    g_assert(token.addr >= QPCI_PIO_LIMIT);
> +    dev->bus->memread(dev->bus, token.addr + off, buf, len);
>  }
>  
> -void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
> +void qpci_memwrite(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                   const void *buf, size_t len)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    g_assert(addr >= QPCI_PIO_LIMIT);
> -    dev->bus->memwrite(dev->bus, addr, buf, len);
> +    g_assert(token.addr >= QPCI_PIO_LIMIT);
> +    dev->bus->memwrite(dev->bus, token.addr + off, buf, len);
>  }
>  
> -void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
> +QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
>  {
>      QPCIBus *bus = dev->bus;
>      static const int bar_reg_map[] = {
>          PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
>          PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
>      };
> +    QPCIBar bar;
>      int bar_reg;
>      uint32_t addr, size;
>      uint32_t io_type;
> @@ -391,17 +378,19 @@ void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
>          qpci_config_writel(dev, bar_reg, loc);
>      }
>  
> -    return (void *)(uintptr_t)loc;
> +    bar.addr = loc;
> +    return bar;
>  }
>  
> -void qpci_iounmap(QPCIDevice *dev, void *data)
> +void qpci_iounmap(QPCIDevice *dev, QPCIBar bar)
>  {
>      /* FIXME */
>  }
>  
> -void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
> +QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
>  {
> -    return (void *)(uintptr_t)addr;
> +    QPCIBar bar = { .addr = addr };
> +    return bar;
>  }
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index 531e3f7..279fa51 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -21,6 +21,7 @@
>  
>  typedef struct QPCIDevice QPCIDevice;
>  typedef struct QPCIBus QPCIBus;
> +typedef struct QPCIBar QPCIBar;
>  
>  struct QPCIBus {
>      uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
> @@ -51,13 +52,21 @@ struct QPCIBus {
>      uint64_t mmio_alloc_ptr, mmio_limit;
>  };
>  
> +struct QPCIBar {
> +    uint64_t addr;
> +};
> +
> +static const QPCIBar QPCI_BAR_INVALID = {
> +    .addr = (uint64_t)-1ULL,
> +};
> +

In v2, you had:

void qpci_msix_disable(QPCIDevice *dev)
{
[...]
    memset(&dev->msix_table_bar, 0, sizeof(dev->msix_table_bar));
    memset(&dev->msix_pba_bar, 0, sizeof(dev->msix_pba_bar));  
[...]
}

and now they get filled with 0xff... is there a reason ?

>  struct QPCIDevice
>  {
>      QPCIBus *bus;
>      int devfn;
>      bool msix_enabled;
> -    void *msix_table;
> -    void *msix_pba;
> +    QPCIBar msix_table_bar, msix_pba_bar;
> +    uint64_t msix_table_off, msix_pba_off;
>  };
>  
>  void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
> @@ -81,22 +90,27 @@ void qpci_config_writeb(QPCIDevice *dev, uint8_t offset, uint8_t value);
>  void qpci_config_writew(QPCIDevice *dev, uint8_t offset, uint16_t value);
>  void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
>  
> -uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
> -uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
> -uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
> -uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
> -
> -void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
> -void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
> -void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
> -void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
> -
> -void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
> -void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
> -
> -void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
> -void qpci_iounmap(QPCIDevice *dev, void *data);
> -void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
> +uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +
> +void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint8_t value);
> +void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint16_t value);
> +void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint32_t value);
> +void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint64_t value);
> +
> +void qpci_memread(QPCIDevice *bus, QPCIBar token, uint64_t off,
> +                  void *buf, size_t len);
> +void qpci_memwrite(QPCIDevice *bus, QPCIBar token, uint64_t off,
> +                   const void *buf, size_t len);
> +QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
> +void qpci_iounmap(QPCIDevice *dev, QPCIBar addr);
> +QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts);
> diff --git a/tests/libqos/usb.c b/tests/libqos/usb.c
> index f794d92..72d7a96 100644
> --- a/tests/libqos/usb.c
> +++ b/tests/libqos/usb.c
> @@ -21,14 +21,12 @@ void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc, uint32_t devfn, int bar)
>      hc->dev = qpci_device_find(pcibus, devfn);
>      g_assert(hc->dev != NULL);
>      qpci_device_enable(hc->dev);
> -    hc->base = qpci_iomap(hc->dev, bar, NULL);
> -    g_assert(hc->base != NULL);
> +    hc->bar = qpci_iomap(hc->dev, bar, NULL);
>  }
>  
>  void uhci_port_test(struct qhc *hc, int port, uint16_t expect)
>  {
> -    void *addr = hc->base + 0x10 + 2 * port;
> -    uint16_t value = qpci_io_readw(hc->dev, addr);
> +    uint16_t value = qpci_io_readw(hc->dev, hc->bar, 0x10 + 2 * port);
>      uint16_t mask = ~(UHCI_PORT_WRITE_CLEAR | UHCI_PORT_RSVD1);
>  
>      g_assert((value & mask) == (expect & mask));
> diff --git a/tests/libqos/usb.h b/tests/libqos/usb.h
> index 8fe5687..423dcfd 100644
> --- a/tests/libqos/usb.h
> +++ b/tests/libqos/usb.h
> @@ -5,7 +5,7 @@
>  
>  struct qhc {
>      QPCIDevice *dev;
> -    void *base;
> +    QPCIBar bar;
>  };
>  
>  void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc,
> diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
> index c69d09d..24ed779 100644
> --- a/tests/libqos/virtio-pci.c
> +++ b/tests/libqos/virtio-pci.c
> @@ -62,13 +62,12 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
>      *vpcidev = (QVirtioPCIDevice *)d;
>  }
>  
> -#define CONFIG_BASE(dev) \
> -    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
> +#define CONFIG_BASE(dev) (VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
>  
>  static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
> +    return qpci_io_readb(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>  }
>  
>  /* PCI is always read in little-endian order
> @@ -84,7 +83,7 @@ static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint16_t value;
>  
> -    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
> +    value = qpci_io_readw(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          value = bswap16(value);
>      }
> @@ -96,7 +95,7 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint32_t value;
>  
> -    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
> +    value = qpci_io_readl(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          value = bswap32(value);
>      }
> @@ -108,7 +107,7 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint64_t val;
>  
> -    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
> +    val = qpci_io_readq(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          val = bswap64(val);
>      }
> @@ -119,31 +118,31 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
>  static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_HOST_FEATURES);
> +    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_HOST_FEATURES);
>  }
>  
>  static void qvirtio_pci_set_features(QVirtioDevice *d, uint32_t features)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES, features);
> +    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES, features);
>  }
>  
>  static uint32_t qvirtio_pci_get_guest_features(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES);
> +    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES);
>  }
>  
>  static uint8_t qvirtio_pci_get_status(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS);
> +    return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS);
>  }
>  
>  static void qvirtio_pci_set_status(QVirtioDevice *d, uint8_t status)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS, status);
> +    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS, status);
>  }
>  
>  static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
> @@ -167,7 +166,7 @@ static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
>              }
>          }
>      } else {
> -        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 1;
> +        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 1;
>      }
>  }
>  
> @@ -191,26 +190,26 @@ static bool qvirtio_pci_get_config_isr_status(QVirtioDevice *d)
>              }
>          }
>      } else {
> -        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 2;
> +        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 2;
>      }
>  }
>  
>  static void qvirtio_pci_queue_select(QVirtioDevice *d, uint16_t index)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_SEL, index);
> +    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_SEL, index);
>  }
>  
>  static uint16_t qvirtio_pci_get_queue_size(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readw(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NUM);
> +    return qpci_io_readw(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NUM);
>  }
>  
>  static void qvirtio_pci_set_queue_address(QVirtioDevice *d, uint32_t pfn)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_PFN, pfn);
> +    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_PFN, pfn);
>  }
>  
>  static QVirtQueue *qvirtio_pci_virtqueue_setup(QVirtioDevice *d,
> @@ -262,7 +261,7 @@ static void qvirtio_pci_virtqueue_cleanup(QVirtQueue *vq,
>  static void qvirtio_pci_virtqueue_kick(QVirtioDevice *d, QVirtQueue *vq)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writew(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
> +    qpci_io_writew(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
>  }
>  
>  const QVirtioBus qvirtio_pci = {
> @@ -309,14 +308,13 @@ QVirtioPCIDevice *qvirtio_pci_device_find(QPCIBus *bus, uint16_t device_type)
>  void qvirtio_pci_device_enable(QVirtioPCIDevice *d)
>  {
>      qpci_device_enable(d->pdev);
> -    d->addr = qpci_iomap(d->pdev, 0, NULL);
> -    g_assert(d->addr != NULL);
> +    d->bar = qpci_iomap(d->pdev, 0, NULL);
>  }
>  
>  void qvirtio_pci_device_disable(QVirtioPCIDevice *d)
>  {
> -    qpci_iounmap(d->pdev, d->addr);
> -    d->addr = NULL;
> +    qpci_iounmap(d->pdev, d->bar);
> +    d->bar = QPCI_BAR_INVALID;
>  }
>  
>  void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
> @@ -324,29 +322,33 @@ void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
>  {
>      uint16_t vector;
>      uint32_t control;
> -    void *addr;
> +    uint64_t off;
>  
>      g_assert(d->pdev->msix_enabled);
> -    addr = d->pdev->msix_table + (entry * 16);
> +    off = d->pdev->msix_table_off + (entry * 16);
>  
>      g_assert_cmpint(entry, >=, 0);
>      g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
>      vqpci->msix_entry = entry;
>  
>      vqpci->msix_addr = guest_alloc(alloc, 4);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
> -                                                    vqpci->msix_addr & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
> -                                            (vqpci->msix_addr >> 32) & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
> -
> -    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
> -                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_LOWER_ADDR, vqpci->msix_addr & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
> +                   (vqpci->msix_addr >> 32) & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
> +
> +    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
> +                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
> +                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
>  
>      qvirtio_pci_queue_select(&d->vdev, vqpci->vq.index);
> -    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR, entry);
> -    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR);
> +    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR, entry);
> +    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR);
>      g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
>  }
>  
> @@ -355,10 +357,10 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
>  {
>      uint16_t vector;
>      uint32_t control;
> -    void *addr;
> +    uint64_t off;
>  
>      g_assert(d->pdev->msix_enabled);
> -    addr = d->pdev->msix_table + (entry * 16);
> +    off = d->pdev->msix_table_off + (entry * 16);
>  
>      g_assert_cmpint(entry, >=, 0);
>      g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
> @@ -367,17 +369,21 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
>      d->config_msix_data = 0x12345678;
>      d->config_msix_addr = guest_alloc(alloc, 4);
>  
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
> -                                                    d->config_msix_addr & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
> -                                            (d->config_msix_addr >> 32) & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
> -
> -    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
> -                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> -
> -    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR, entry);
> -    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_LOWER_ADDR, d->config_msix_addr & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
> +                   (d->config_msix_addr >> 32) & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
> +
> +    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
> +                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
> +                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> +
> +    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR, entry);
> +    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR);
>      g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
>  }
> diff --git a/tests/libqos/virtio-pci.h b/tests/libqos/virtio-pci.h
> index efcac2d..38c54c6 100644
> --- a/tests/libqos/virtio-pci.h
> +++ b/tests/libqos/virtio-pci.h
> @@ -16,7 +16,7 @@
>  typedef struct QVirtioPCIDevice {
>      QVirtioDevice vdev;
>      QPCIDevice *pdev;
> -    void *addr;
> +    QPCIBar bar;
>      uint16_t config_msix_entry;
>      uint64_t config_msix_addr;
>      uint32_t config_msix_data;
> diff --git a/tests/rtl8139-test.c b/tests/rtl8139-test.c
> index c2f601a..7de7dc4 100644
> --- a/tests/rtl8139-test.c
> +++ b/tests/rtl8139-test.c
> @@ -22,7 +22,7 @@ static void nop(void)
>  
>  static QPCIBus *pcibus;
>  static QPCIDevice *dev;
> -static void *dev_base;
> +static QPCIBar dev_bar;
>  
>  static void save_fn(QPCIDevice *dev, int devfn, void *data)
>  {
> @@ -45,14 +45,14 @@ static QPCIDevice *get_device(void)
>  #define PORT(name, len, val) \
>  static unsigned __attribute__((unused)) in_##name(void) \
>  { \
> -    unsigned res = qpci_io_read##len(dev, dev_base+(val)); \
> +    unsigned res = qpci_io_read##len(dev, dev_bar, (val));     \
>      g_test_message("*%s -> %x\n", #name, res); \
>      return res; \
>  } \
>  static void out_##name(unsigned v) \
>  { \
>      g_test_message("%x -> *%s\n", v, #name); \
> -    qpci_io_write##len(dev, dev_base+(val), v); \
> +    qpci_io_write##len(dev, dev_bar, (val), v);        \
>  }
>  
>  PORT(Timer, l, 0x48)
> @@ -186,9 +186,7 @@ static void test_init(void)
>  
>      dev = get_device();
>  
> -    dev_base = qpci_iomap(dev, 0, &barsize);
> -
> -    g_assert(dev_base != NULL);
> +    dev_bar = qpci_iomap(dev, 0, &barsize);
>  
>      qpci_device_enable(dev);
>  
> diff --git a/tests/tco-test.c b/tests/tco-test.c
> index 129577d..ef02ec5 100644
> --- a/tests/tco-test.c
> +++ b/tests/tco-test.c
> @@ -41,7 +41,7 @@ typedef struct {
>      const char *args;
>      bool noreboot;
>      QPCIDevice *dev;
> -    void *tco_io_base;
> +    QPCIBar tco_io_bar;
>  } TestData;
>  
>  static void test_init(TestData *d)
> @@ -70,42 +70,42 @@ static void test_init(TestData *d)
>      /* set Root Complex BAR */
>      qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
>  
> -    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
> +    d->tco_io_bar = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
>  }
>  
>  static void stop_tco(const TestData *d)
>  {
>      uint32_t val;
>  
> -    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
> +    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
>      val |= TCO_TMR_HLT;
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
>  }
>  
>  static void start_tco(const TestData *d)
>  {
>      uint32_t val;
>  
> -    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
> +    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
>      val &= ~TCO_TMR_HLT;
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
>  }
>  
>  static void load_tco(const TestData *d)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO_RLD, 4);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO_RLD, 4);
>  }
>  
>  static void set_tco_timeout(const TestData *d, uint16_t ticks)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO_TMR, ticks);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO_TMR, ticks);
>  }
>  
>  static void clear_tco_status(const TestData *d)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_STS, 0x0008);
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0002);
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0004);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_STS, 0x0008);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0002);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0004);
>  }
>  
>  static void reset_on_second_timeout(bool enable)
> @@ -128,25 +128,25 @@ static void test_tco_defaults(void)
>      d.args = NULL;
>      d.noreboot = true;
>      test_init(&d);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD), ==,
>                      TCO_RLD_DEFAULT);
>      /* TCO_DAT_IN & TCO_DAT_OUT */
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_DAT_IN), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_DAT_IN), ==,
>                      (TCO_DAT_OUT_DEFAULT << 8) | TCO_DAT_IN_DEFAULT);
>      /* TCO1_STS & TCO2_STS */
> -    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_STS), ==,
> +    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_STS), ==,
>                      (TCO2_STS_DEFAULT << 16) | TCO1_STS_DEFAULT);
>      /* TCO1_CNT & TCO2_CNT */
> -    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_CNT), ==,
> +    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_CNT), ==,
>                      (TCO2_CNT_DEFAULT << 16) | TCO1_CNT_DEFAULT);
>      /* TCO_MESSAGE1 & TCO_MESSAGE2 */
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_MESSAGE1), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_MESSAGE1), ==,
>                      (TCO_MESSAGE2_DEFAULT << 8) | TCO_MESSAGE1_DEFAULT);
> -    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + TCO_WDCNT), ==,
> +    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, TCO_WDCNT), ==,
>                      TCO_WDCNT_DEFAULT);
> -    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + SW_IRQ_GEN), ==,
> +    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, SW_IRQ_GEN), ==,
>                      SW_IRQ_GEN_DEFAULT);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_TMR), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_TMR), ==,
>                      TCO_TMR_DEFAULT);
>      qtest_end();
>  }
> @@ -171,23 +171,23 @@ static void test_tco_timeout(void)
>      clock_step(ticks * TCO_TICK_NSEC);
>  
>      /* test first timeout */
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
>  
>      /* test clearing timeout bit */
>      val |= TCO_TIMEOUT;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 0);
>  
>      /* test second timeout */
>      clock_step(ticks * TCO_TICK_NSEC);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
>      ret = val & TCO_SECOND_TO_STS ? 1 : 0;
>      g_assert(ret == 1);
>  
> @@ -214,13 +214,13 @@ static void test_tco_max_timeout(void)
>      start_tco(&d);
>      clock_step(((ticks & TCO_TMR_MASK) - 1) * TCO_TICK_NSEC);
>  
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD);
>      g_assert_cmpint(val & TCO_RLD_MASK, ==, 1);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 0);
>      clock_step(TCO_TICK_NSEC);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
>  
> @@ -358,11 +358,11 @@ static void test_tco_ticks_counter(void)
>      start_tco(&d);
>  
>      do {
> -        rld = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD) & TCO_RLD_MASK;
> +        rld = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD) & TCO_RLD_MASK;
>          g_assert_cmpint(rld, ==, ticks);
>          clock_step(TCO_TICK_NSEC);
>          ticks--;
> -    } while (!(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS) & TCO_TIMEOUT));
> +    } while (!(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS) & TCO_TIMEOUT));
>  
>      stop_tco(&d);
>      qtest_end();
> @@ -378,10 +378,10 @@ static void test_tco1_control_bits(void)
>      test_init(&d);
>  
>      val = TCO_LOCK;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
>      val &= ~TCO_LOCK;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_CNT), ==,
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_CNT), ==,
>                      TCO_LOCK);
>      qtest_end();
>  }
> @@ -405,13 +405,13 @@ static void test_tco1_status_bits(void)
>      start_tco(&d);
>      clock_step(ticks * TCO_TICK_NSEC);
>  
> -    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_IN, 0);
> -    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_OUT, 0);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_IN, 0);
> +    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_OUT, 0);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & (TCO_TIMEOUT | SW_TCO_SMI | TCO_INT_STS) ? 1 : 0;
>      g_assert(ret == 1);
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS), ==, 0);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS), ==, 0);
>      qtest_end();
>  }
>  
> @@ -434,11 +434,11 @@ static void test_tco2_status_bits(void)
>      start_tco(&d);
>      clock_step(ticks * TCO_TICK_NSEC * 2);
>  
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
>      ret = val & (TCO_SECOND_TO_STS | TCO_BOOT_STS) ? 1 : 0;
>      g_assert(ret == 1);
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO2_STS, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS), ==, 0);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO2_STS, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS), ==, 0);
>      qtest_end();
>  }
>  
> diff --git a/tests/usb-hcd-ehci-test.c b/tests/usb-hcd-ehci-test.c
> index a4ceeaa..57af8a0 100644
> --- a/tests/usb-hcd-ehci-test.c
> +++ b/tests/usb-hcd-ehci-test.c
> @@ -38,8 +38,7 @@ static void uhci_port_update(struct qhc *hc, int port,
>  
>  static void ehci_port_test(struct qhc *hc, int port, uint32_t expect)
>  {
> -    void *addr = hc->base + 0x64 + 4 * port;
> -    uint32_t value = qpci_io_readl(hc->dev, addr);
> +    uint32_t value = qpci_io_readl(hc->dev, hc->bar, 0x64 + 4 * port);
>      uint16_t mask = ~(PORTSC_CSC | PORTSC_PEDC | PORTSC_OCC);
>  
>  #if 0
> @@ -91,7 +90,7 @@ static void pci_ehci_port_1(void)
>  static void pci_ehci_config(void)
>  {
>      /* hands over all ports from companion uhci to ehci */
> -    qpci_io_writew(ehci1.dev, ehci1.base + 0x60, 1);
> +    qpci_io_writew(ehci1.dev, ehci1.bar, 0x60, 1);
>  }
>  
>  static void pci_uhci_port_2(void)

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO
  2016-10-21 10:05   ` Greg Kurz
@ 2016-10-22  4:50     ` David Gibson
  0 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-22  4:50 UTC (permalink / raw)
  To: Greg Kurz
  Cc: aik, pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha,
	mst, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 3798 bytes --]

On Fri, Oct 21, 2016 at 12:05:06PM +0200, Greg Kurz wrote:
> On Fri, 21 Oct 2016 12:19:45 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > The usual model for PCI IO with libqos is to use qpci_iomap() to map a
> > specific BAR for a PCI device, then perform IOs within that BAR using
> > qpci_io_{read,write}*().
> > 
> > However, certain devices also have legacy PCI IO.  In this case, instead of
> > (or as well as) being accessed via PCI BARs, the device can be accessed
> > via certain well-known, fixed addresses in PCI IO space.
> > 
> > Two existing tests use legacy PCI IO, and take different flawed approaches
> > to it:
> >     * tco-test manually constructs a tco_io_base value instead of calling
> >       qpci_iomap(), which assumes internal knowledge of the structure of
> >       the value it shouldn't have
> >     * ide-test uses direct in*() and out*() calls instead of using
> >       qpci_io_*() accessors, meaning it's not portable to non-x86 machine
> >       types.
> > 
> > This patch implements a new qpci_iomap_legacy() interface which gets a
> > handle in the same format as qpci_iomap() but refers to a region in
> > the legacy PIO space.  For a device which has the same registers
> > available both in a BAR and in legacy space (quite common), this
> > allows the same test code to test both options with just a different
> > iomap() at the beginning.
> > 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> > ---
> >  tests/libqos/pci.c | 5 +++++
> >  tests/libqos/pci.h | 1 +
> >  2 files changed, 6 insertions(+)
> > 
> > diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> > index bf1c532..98a2e56 100644
> > --- a/tests/libqos/pci.c
> > +++ b/tests/libqos/pci.c
> > @@ -350,6 +350,11 @@ void qpci_iounmap(QPCIDevice *dev, void *data)
> >      /* FIXME */
> >  }
> >  
> > +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
> > +{
> > +    return (void *)(uintptr_t)addr;
> > +}
> > +
> 
> Since both tco-test and ide-test are hardcoded for x86 machine types, is
> it right for this implementation to sit in common PCI code instead of
> the PC specific one ?

Those tests are x86 specific for now.  tco-test is of some ICH9
internal device so it probably always will be.  But it's absolutely
possible to have a legacy IDE controller on another platform.

Even if those specific tests don't happen, it's likely someone will
want to test some sort of legacy device on another platform at some
point.

[Even though the "legacy" in question is old PCs, it's not uncommon to
see these devices on other platforms - a PC-style southbridge chip
including an ISA bridge and legacy devices is a cheap way of getting a
bunch of useful gadgets on your board, so it's pretty common on many
archs]

> >  void qpci_plug_device_test(const char *driver, const char *id,
> >                             uint8_t slot, const char *opts)
> >  {
> > diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> > index f6f916d..b6f855e 100644
> > --- a/tests/libqos/pci.h
> > +++ b/tests/libqos/pci.h
> > @@ -94,6 +94,7 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
> >  
> >  void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
> >  void qpci_iounmap(QPCIDevice *dev, void *data);
> > +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
> >  
> >  void qpci_plug_device_test(const char *driver, const char *id,
> >                             uint8_t slot, const char *opts);
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test
  2016-10-21 10:07       ` Laurent Vivier
@ 2016-10-22  4:56         ` David Gibson
  0 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-22  4:56 UTC (permalink / raw)
  To: Laurent Vivier
  Cc: pbonzini, qemu-devel, qemu-ppc, agraf, stefanha, mst, aik,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 4884 bytes --]

On Fri, Oct 21, 2016 at 12:07:28PM +0200, Laurent Vivier wrote:
> 
> 
> On 21/10/2016 11:05, David Gibson wrote:
> > On Fri, Oct 21, 2016 at 10:31:27AM +0200, Laurent Vivier wrote:
> >>
> >>
> >> On 21/10/2016 03:19, David Gibson wrote:
> >>> ide-test uses many explicit inb() / outb() operations for its IO, which
> >>> means it's not portable to non-x86 platforms.  This cleans it up to use
> >>> the libqos PCI accessors instead.
> >>>
> >>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> >>> ---
> >>>  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
> >>>  1 file changed, 118 insertions(+), 61 deletions(-)
> >>>
> >>> diff --git a/tests/ide-test.c b/tests/ide-test.c
> >>> index a8a4081..86c4373 100644
> >>> --- a/tests/ide-test.c
> >>> +++ b/tests/ide-test.c
> >> ...
> >>> @@ -654,7 +700,8 @@ typedef struct Read10CDB {
> >>>      uint16_t padding;
> >>>  } __attribute__((__packed__)) Read10CDB;
> >>>  
> >>> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >>> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> >>> +                                 uint64_t lba, int nblocks)
> >>>  {
> >>>      Read10CDB pkt = { .padding = 0 };
> >>>      int i;
> >>> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >>>  
> >>>      /* Send Packet */
> >>>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> >>> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> >>> +        qpci_io_writew(dev, ide_base + reg_data,
> >>> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
> >>>      }
> >>>  }
> >>>  
> >> ...
> >>> @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
> >>>  
> >>>          /* HP4: Transfer_Data */
> >>>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> >>> -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> >>> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> >>> +                                                       ide_base + reg_data));
> >>>          }
> >>>      }
> >>
> >> Why do you swap le16_to_cpu() and cpu_to_le16()?
> > 
> > So, obviously le16_to_cpu() and cpu_to_le16() are functionally
> > identical.  But I think my version is conceptually more correct.
> 
> I agree
> 
> > 
> > This is streaming data via PIO - we're essentially reading a byte
> > stream in 16-bit chunks.  So, overall we want to preserve byte address
> > order.  qpci_io_readw() (and inw()) is designed for reading registers
> > so it preserves byte significance, rather than address order.  So,
> > since the IDE registers are LE, that means if implicitly contains an
> > le16_to_cpu().  The cpu_to_le16() undoes that so that rx[] ends up
> > with the bytestream in the correct order.
> 
> In fact, it has an implicit "le16 to target CPU", so you should not have
> a "cpu_to_le16()" but a "qtest_big_endian() != host_big_endian
> bswap16()" to swap it.

No.. it really is le16_to_cpu().  For memory space accesses, there's
an explicit le16_to_cpu in qpci_io_readw() on top of the
order-preserving memread backend.  For IO space accesses, we call the
backend directly.  In all likely cases that will be turned into a (non
PCI) either inw() or readw().

Both inw() and readw() are defined to access in target endianness
(which is stupid, but that argument is in several other threads).  So
they have an implicit "target_to_host16()".  But the PCI versions are
defined to be always LE - on BE targets the backend will add an extra
swap, changing that target_to_host16() into an le16_to_cpu().

> As this test works only with a little endian guest, the first "LE16 to
> target CPU" is a no-op, and the second one does something only on big
> endian host so when "qtest_big_endian() != host_big_endian".
> 
> >> I think this is not semantically correct and the purpose of this patch
> >> is only to replace inX()/outX() functions.
> > 
> > No, I believe it is correct.  And I think the original was technically
> > wrong, because inw() reads the register in target endian rather than
> > LE.  Of course the only common platform with "real" in/out
> > instructions is x86, which is LE anyway, so it hardly matters.
> 
> I agree, this is why I think a "qtest_big_endian() != host_big_endian
> then bswap16()" is better than a "cpu_to_le16()".

No, cpu_to_le16() is correct.  Well.. actually using an
order-preserving read would be correct, but that's a bit awkward for
tne io port case.

> 
> But I also think it should be the purpose of another patch...
> 
> Thanks,
> Laurent
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-21 13:23   ` Greg Kurz
@ 2016-10-24  3:19     ` David Gibson
  2016-10-24  4:31       ` David Gibson
  2016-10-24 10:02       ` Greg Kurz
  0 siblings, 2 replies; 27+ messages in thread
From: David Gibson @ 2016-10-24  3:19 UTC (permalink / raw)
  To: Greg Kurz
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 4426 bytes --]

On Fri, Oct 21, 2016 at 03:23:51PM +0200, Greg Kurz wrote:
> On Fri, 21 Oct 2016 12:19:52 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > The usual use model for the libqos PCI functions is to map a specific PCI
> > BAR using qpci_iomap() then pass the returned token into IO accessor
> > functions.  This, and the fact that iomap() returns a (void *) which
> > actually contains a PCI space address, kind of suggests that the return
> > value from iomap is supposed to be an opaque token.
> > 
> > ..except that the callers expect to be able to add offsets to it.  Which
> > also assumes the compiler will support pointer arithmetic on a (void *),
> > and treat it as working with byte offsets.
> > 
> > To clarify this situation change iomap() and the IO accessors to take
> > a definitely opaque BAR handle (enforced with a wrapper struct) along with
> > an offset within the BAR.  This changes both the functions and all the
> > callers.
> > 
> > Asserts that iomap() returns non-NULL are removed in some places; iomap()
> > already asserts if it can't map the BAR
> > 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > ---
> >  tests/ahci-test.c         |   4 +-
> >  tests/e1000e-test.c       |   7 +-
> >  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
> >  tests/ivshmem-test.c      |  16 ++---
> >  tests/libqos/ahci.c       |   3 +-
> >  tests/libqos/ahci.h       |   6 +-
> >  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
> >  tests/libqos/pci.h        |  50 ++++++++-----
> >  tests/libqos/usb.c        |   6 +-
> >  tests/libqos/usb.h        |   2 +-
> >  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
> >  tests/libqos/virtio-pci.h |   2 +-
> >  tests/rtl8139-test.c      |  10 ++-
> >  tests/tco-test.c          |  80 ++++++++++-----------
> >  tests/usb-hcd-ehci-test.c |   5 +-
> >  15 files changed, 309 insertions(+), 311 deletions(-)
> > 
> > diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> > index 9c0adce..4358631 100644
> > --- a/tests/ahci-test.c
> > +++ b/tests/ahci-test.c
> > @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
> >      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
> >  
> >      /* If we haven't initialized, this is as much as can be validated. */
> > -    if (!ahci->hba_base) {
> > +    if (!ahci->hba_bar.addr) {
> 
> Isn't ahci->hba_bar supposed to be opaque ?

Ah, good point, missed that one.  And that test isn't even right, with
the INVALID_BAR stuff.

> >          return;
> >      }
> 
> Unrelated to this patch, does it make sense to call verify_state() if
> ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?

I'm pretty sure it is only called after PCI initialization, so I think
we should just remove the check.

> >      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> > -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> > +    hba_stored = ahci->hba_bar.addr;
> >      g_assert_cmphex(hba_base, ==, hba_stored);
> 
> Again since ahci->hba_bar is opaque, is it right to do that check here ?

Not, really no.  I was aware of that one, but decided to let it go
since it's just one pretty specific check.

But then again, if I'm fixing other things in AHCI, maybe I might as
well fix it to read the actual BAR register before the migration.

> I have another question about QPCI_BAR_INVALID far below (patch is
> long :)


[snip]
> > +struct QPCIBar {
> > +    uint64_t addr;
> > +};
> > +
> > +static const QPCIBar QPCI_BAR_INVALID = {
> > +    .addr = (uint64_t)-1ULL,
> > +};
> > +
> 
> In v2, you had:
> 
> void qpci_msix_disable(QPCIDevice *dev)
> {
> [...]
>     memset(&dev->msix_table_bar, 0, sizeof(dev->msix_table_bar));
>     memset(&dev->msix_pba_bar, 0, sizeof(dev->msix_pba_bar));  
> [...]
> }
> 
> and now they get filled with 0xff... is there a reason ?

Yes.  I realized an address of 0 is a bad way of marking an invalid
BAR, because it's actually a semi-plausible real BAR value.  For
example getting a legacy IO "BAR" at offset 0 would give you that.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24  3:19     ` David Gibson
@ 2016-10-24  4:31       ` David Gibson
  2016-10-24 10:14         ` Greg Kurz
  2016-10-24 10:02       ` Greg Kurz
  1 sibling, 1 reply; 27+ messages in thread
From: David Gibson @ 2016-10-24  4:31 UTC (permalink / raw)
  To: Greg Kurz
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 3640 bytes --]

On Mon, Oct 24, 2016 at 02:19:47PM +1100, David Gibson wrote:
> On Fri, Oct 21, 2016 at 03:23:51PM +0200, Greg Kurz wrote:
> > On Fri, 21 Oct 2016 12:19:52 +1100
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> > 
> > > The usual use model for the libqos PCI functions is to map a specific PCI
> > > BAR using qpci_iomap() then pass the returned token into IO accessor
> > > functions.  This, and the fact that iomap() returns a (void *) which
> > > actually contains a PCI space address, kind of suggests that the return
> > > value from iomap is supposed to be an opaque token.
> > > 
> > > ..except that the callers expect to be able to add offsets to it.  Which
> > > also assumes the compiler will support pointer arithmetic on a (void *),
> > > and treat it as working with byte offsets.
> > > 
> > > To clarify this situation change iomap() and the IO accessors to take
> > > a definitely opaque BAR handle (enforced with a wrapper struct) along with
> > > an offset within the BAR.  This changes both the functions and all the
> > > callers.
> > > 
> > > Asserts that iomap() returns non-NULL are removed in some places; iomap()
> > > already asserts if it can't map the BAR
> > > 
> > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > ---
> > >  tests/ahci-test.c         |   4 +-
> > >  tests/e1000e-test.c       |   7 +-
> > >  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
> > >  tests/ivshmem-test.c      |  16 ++---
> > >  tests/libqos/ahci.c       |   3 +-
> > >  tests/libqos/ahci.h       |   6 +-
> > >  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
> > >  tests/libqos/pci.h        |  50 ++++++++-----
> > >  tests/libqos/usb.c        |   6 +-
> > >  tests/libqos/usb.h        |   2 +-
> > >  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
> > >  tests/libqos/virtio-pci.h |   2 +-
> > >  tests/rtl8139-test.c      |  10 ++-
> > >  tests/tco-test.c          |  80 ++++++++++-----------
> > >  tests/usb-hcd-ehci-test.c |   5 +-
> > >  15 files changed, 309 insertions(+), 311 deletions(-)
> > > 
> > > diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> > > index 9c0adce..4358631 100644
> > > --- a/tests/ahci-test.c
> > > +++ b/tests/ahci-test.c
> > > @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
> > >      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
> > >  
> > >      /* If we haven't initialized, this is as much as can be validated. */
> > > -    if (!ahci->hba_base) {
> > > +    if (!ahci->hba_bar.addr) {
> > 
> > Isn't ahci->hba_bar supposed to be opaque ?
> 
> Ah, good point, missed that one.  And that test isn't even right, with
> the INVALID_BAR stuff.
> 
> > >          return;
> > >      }
> > 
> > Unrelated to this patch, does it make sense to call verify_state() if
> > ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?
> 
> I'm pretty sure it is only called after PCI initialization, so I think
> we should just remove the check.

Wait.. no.  There is one testcase which is called when the device has
been located, but not enabled/initialized.  That means the BAR pointer
isn't initialized, and the later checks in verify_state (which real IO
registers) can't be done.  So there is a real point to this test.  I
think I'll have to add something to allow checks for a valid BAR.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24  3:19     ` David Gibson
  2016-10-24  4:31       ` David Gibson
@ 2016-10-24 10:02       ` Greg Kurz
  2016-10-24 12:02         ` David Gibson
  1 sibling, 1 reply; 27+ messages in thread
From: Greg Kurz @ 2016-10-24 10:02 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 4645 bytes --]

On Mon, 24 Oct 2016 14:19:47 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Fri, Oct 21, 2016 at 03:23:51PM +0200, Greg Kurz wrote:
> > On Fri, 21 Oct 2016 12:19:52 +1100
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >   
> > > The usual use model for the libqos PCI functions is to map a specific PCI
> > > BAR using qpci_iomap() then pass the returned token into IO accessor
> > > functions.  This, and the fact that iomap() returns a (void *) which
> > > actually contains a PCI space address, kind of suggests that the return
> > > value from iomap is supposed to be an opaque token.
> > > 
> > > ..except that the callers expect to be able to add offsets to it.  Which
> > > also assumes the compiler will support pointer arithmetic on a (void *),
> > > and treat it as working with byte offsets.
> > > 
> > > To clarify this situation change iomap() and the IO accessors to take
> > > a definitely opaque BAR handle (enforced with a wrapper struct) along with
> > > an offset within the BAR.  This changes both the functions and all the
> > > callers.
> > > 
> > > Asserts that iomap() returns non-NULL are removed in some places; iomap()
> > > already asserts if it can't map the BAR
> > > 
> > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > ---
> > >  tests/ahci-test.c         |   4 +-
> > >  tests/e1000e-test.c       |   7 +-
> > >  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
> > >  tests/ivshmem-test.c      |  16 ++---
> > >  tests/libqos/ahci.c       |   3 +-
> > >  tests/libqos/ahci.h       |   6 +-
> > >  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
> > >  tests/libqos/pci.h        |  50 ++++++++-----
> > >  tests/libqos/usb.c        |   6 +-
> > >  tests/libqos/usb.h        |   2 +-
> > >  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
> > >  tests/libqos/virtio-pci.h |   2 +-
> > >  tests/rtl8139-test.c      |  10 ++-
> > >  tests/tco-test.c          |  80 ++++++++++-----------
> > >  tests/usb-hcd-ehci-test.c |   5 +-
> > >  15 files changed, 309 insertions(+), 311 deletions(-)
> > > 
> > > diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> > > index 9c0adce..4358631 100644
> > > --- a/tests/ahci-test.c
> > > +++ b/tests/ahci-test.c
> > > @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
> > >      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
> > >  
> > >      /* If we haven't initialized, this is as much as can be validated. */
> > > -    if (!ahci->hba_base) {
> > > +    if (!ahci->hba_bar.addr) {  
> > 
> > Isn't ahci->hba_bar supposed to be opaque ?  
> 
> Ah, good point, missed that one.  And that test isn't even right, with
> the INVALID_BAR stuff.
> 

Indeed.

> > >          return;
> > >      }  
> > 
> > Unrelated to this patch, does it make sense to call verify_state() if
> > ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?  
> 
> I'm pretty sure it is only called after PCI initialization, so I think
> we should just remove the check.
> 
> > >      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> > > -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> > > +    hba_stored = ahci->hba_bar.addr;
> > >      g_assert_cmphex(hba_base, ==, hba_stored);  
> > 
> > Again since ahci->hba_bar is opaque, is it right to do that check here ?  
> 
> Not, really no.  I was aware of that one, but decided to let it go
> since it's just one pretty specific check.
> 
> But then again, if I'm fixing other things in AHCI, maybe I might as
> well fix it to read the actual BAR register before the migration.
> 
> > I have another question about QPCI_BAR_INVALID far below (patch is
> > long :)  
> 
> 
> [snip]
> > > +struct QPCIBar {
> > > +    uint64_t addr;
> > > +};
> > > +
> > > +static const QPCIBar QPCI_BAR_INVALID = {
> > > +    .addr = (uint64_t)-1ULL,
> > > +};
> > > +  
> > 
> > In v2, you had:
> > 
> > void qpci_msix_disable(QPCIDevice *dev)
> > {
> > [...]
> >     memset(&dev->msix_table_bar, 0, sizeof(dev->msix_table_bar));
> >     memset(&dev->msix_pba_bar, 0, sizeof(dev->msix_pba_bar));  
> > [...]
> > }
> > 
> > and now they get filled with 0xff... is there a reason ?  
> 
> Yes.  I realized an address of 0 is a bad way of marking an invalid
> BAR, because it's actually a semi-plausible real BAR value.  For
> example getting a legacy IO "BAR" at offset 0 would give you that.
> 

Shouldn't all QPCIBar structures be initialized to BAR_INVALID then ?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24  4:31       ` David Gibson
@ 2016-10-24 10:14         ` Greg Kurz
  0 siblings, 0 replies; 27+ messages in thread
From: Greg Kurz @ 2016-10-24 10:14 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 3789 bytes --]

On Mon, 24 Oct 2016 15:31:12 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Mon, Oct 24, 2016 at 02:19:47PM +1100, David Gibson wrote:
> > On Fri, Oct 21, 2016 at 03:23:51PM +0200, Greg Kurz wrote:  
> > > On Fri, 21 Oct 2016 12:19:52 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > The usual use model for the libqos PCI functions is to map a specific PCI
> > > > BAR using qpci_iomap() then pass the returned token into IO accessor
> > > > functions.  This, and the fact that iomap() returns a (void *) which
> > > > actually contains a PCI space address, kind of suggests that the return
> > > > value from iomap is supposed to be an opaque token.
> > > > 
> > > > ..except that the callers expect to be able to add offsets to it.  Which
> > > > also assumes the compiler will support pointer arithmetic on a (void *),
> > > > and treat it as working with byte offsets.
> > > > 
> > > > To clarify this situation change iomap() and the IO accessors to take
> > > > a definitely opaque BAR handle (enforced with a wrapper struct) along with
> > > > an offset within the BAR.  This changes both the functions and all the
> > > > callers.
> > > > 
> > > > Asserts that iomap() returns non-NULL are removed in some places; iomap()
> > > > already asserts if it can't map the BAR
> > > > 
> > > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > > ---
> > > >  tests/ahci-test.c         |   4 +-
> > > >  tests/e1000e-test.c       |   7 +-
> > > >  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
> > > >  tests/ivshmem-test.c      |  16 ++---
> > > >  tests/libqos/ahci.c       |   3 +-
> > > >  tests/libqos/ahci.h       |   6 +-
> > > >  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
> > > >  tests/libqos/pci.h        |  50 ++++++++-----
> > > >  tests/libqos/usb.c        |   6 +-
> > > >  tests/libqos/usb.h        |   2 +-
> > > >  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
> > > >  tests/libqos/virtio-pci.h |   2 +-
> > > >  tests/rtl8139-test.c      |  10 ++-
> > > >  tests/tco-test.c          |  80 ++++++++++-----------
> > > >  tests/usb-hcd-ehci-test.c |   5 +-
> > > >  15 files changed, 309 insertions(+), 311 deletions(-)
> > > > 
> > > > diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> > > > index 9c0adce..4358631 100644
> > > > --- a/tests/ahci-test.c
> > > > +++ b/tests/ahci-test.c
> > > > @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
> > > >      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
> > > >  
> > > >      /* If we haven't initialized, this is as much as can be validated. */
> > > > -    if (!ahci->hba_base) {
> > > > +    if (!ahci->hba_bar.addr) {  
> > > 
> > > Isn't ahci->hba_bar supposed to be opaque ?  
> > 
> > Ah, good point, missed that one.  And that test isn't even right, with
> > the INVALID_BAR stuff.
> >   
> > > >          return;
> > > >      }  
> > > 
> > > Unrelated to this patch, does it make sense to call verify_state() if
> > > ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?  
> > 
> > I'm pretty sure it is only called after PCI initialization, so I think
> > we should just remove the check.  
> 
> Wait.. no.  There is one testcase which is called when the device has
> been located, but not enabled/initialized.  That means the BAR pointer
> isn't initialized, and the later checks in verify_state (which real IO
> registers) can't be done.  So there is a real point to this test.  I
> think I'll have to add something to allow checks for a valid BAR.
> 

Indeed, I now realize that test_migrate_sanity() does migrate a not yet
enabled device.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24 10:02       ` Greg Kurz
@ 2016-10-24 12:02         ` David Gibson
  0 siblings, 0 replies; 27+ messages in thread
From: David Gibson @ 2016-10-24 12:02 UTC (permalink / raw)
  To: Greg Kurz
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 5448 bytes --]

On Mon, Oct 24, 2016 at 12:02:31PM +0200, Greg Kurz wrote:
> On Mon, 24 Oct 2016 14:19:47 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Fri, Oct 21, 2016 at 03:23:51PM +0200, Greg Kurz wrote:
> > > On Fri, 21 Oct 2016 12:19:52 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > The usual use model for the libqos PCI functions is to map a specific PCI
> > > > BAR using qpci_iomap() then pass the returned token into IO accessor
> > > > functions.  This, and the fact that iomap() returns a (void *) which
> > > > actually contains a PCI space address, kind of suggests that the return
> > > > value from iomap is supposed to be an opaque token.
> > > > 
> > > > ..except that the callers expect to be able to add offsets to it.  Which
> > > > also assumes the compiler will support pointer arithmetic on a (void *),
> > > > and treat it as working with byte offsets.
> > > > 
> > > > To clarify this situation change iomap() and the IO accessors to take
> > > > a definitely opaque BAR handle (enforced with a wrapper struct) along with
> > > > an offset within the BAR.  This changes both the functions and all the
> > > > callers.
> > > > 
> > > > Asserts that iomap() returns non-NULL are removed in some places; iomap()
> > > > already asserts if it can't map the BAR
> > > > 
> > > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > > ---
> > > >  tests/ahci-test.c         |   4 +-
> > > >  tests/e1000e-test.c       |   7 +-
> > > >  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
> > > >  tests/ivshmem-test.c      |  16 ++---
> > > >  tests/libqos/ahci.c       |   3 +-
> > > >  tests/libqos/ahci.h       |   6 +-
> > > >  tests/libqos/pci.c        | 151 ++++++++++++++++++---------------------
> > > >  tests/libqos/pci.h        |  50 ++++++++-----
> > > >  tests/libqos/usb.c        |   6 +-
> > > >  tests/libqos/usb.h        |   2 +-
> > > >  tests/libqos/virtio-pci.c | 102 ++++++++++++++-------------
> > > >  tests/libqos/virtio-pci.h |   2 +-
> > > >  tests/rtl8139-test.c      |  10 ++-
> > > >  tests/tco-test.c          |  80 ++++++++++-----------
> > > >  tests/usb-hcd-ehci-test.c |   5 +-
> > > >  15 files changed, 309 insertions(+), 311 deletions(-)
> > > > 
> > > > diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> > > > index 9c0adce..4358631 100644
> > > > --- a/tests/ahci-test.c
> > > > +++ b/tests/ahci-test.c
> > > > @@ -90,12 +90,12 @@ static void verify_state(AHCIQState *ahci)
> > > >      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
> > > >  
> > > >      /* If we haven't initialized, this is as much as can be validated. */
> > > > -    if (!ahci->hba_base) {
> > > > +    if (!ahci->hba_bar.addr) {  
> > > 
> > > Isn't ahci->hba_bar supposed to be opaque ?  
> > 
> > Ah, good point, missed that one.  And that test isn't even right, with
> > the INVALID_BAR stuff.
> > 
> 
> Indeed.
> 
> > > >          return;
> > > >      }  
> > > 
> > > Unrelated to this patch, does it make sense to call verify_state() if
> > > ahci_pci_enable() hasn't been called before ? Shouldn't we assert instead ?  
> > 
> > I'm pretty sure it is only called after PCI initialization, so I think
> > we should just remove the check.
> > 
> > > >      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> > > > -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> > > > +    hba_stored = ahci->hba_bar.addr;
> > > >      g_assert_cmphex(hba_base, ==, hba_stored);  
> > > 
> > > Again since ahci->hba_bar is opaque, is it right to do that check here ?  
> > 
> > Not, really no.  I was aware of that one, but decided to let it go
> > since it's just one pretty specific check.
> > 
> > But then again, if I'm fixing other things in AHCI, maybe I might as
> > well fix it to read the actual BAR register before the migration.
> > 
> > > I have another question about QPCI_BAR_INVALID far below (patch is
> > > long :)  
> > 
> > 
> > [snip]
> > > > +struct QPCIBar {
> > > > +    uint64_t addr;
> > > > +};
> > > > +
> > > > +static const QPCIBar QPCI_BAR_INVALID = {
> > > > +    .addr = (uint64_t)-1ULL,
> > > > +};
> > > > +  
> > > 
> > > In v2, you had:
> > > 
> > > void qpci_msix_disable(QPCIDevice *dev)
> > > {
> > > [...]
> > >     memset(&dev->msix_table_bar, 0, sizeof(dev->msix_table_bar));
> > >     memset(&dev->msix_pba_bar, 0, sizeof(dev->msix_pba_bar));  
> > > [...]
> > > }
> > > 
> > > and now they get filled with 0xff... is there a reason ?  
> > 
> > Yes.  I realized an address of 0 is a bad way of marking an invalid
> > BAR, because it's actually a semi-plausible real BAR value.  For
> > example getting a legacy IO "BAR" at offset 0 would give you that.
> > 
> 
> Shouldn't all QPCIBar structures be initialized to BAR_INVALID then ?

Yes, which is easier said than done.  In my latest spin, I've just
given up on any concept of "invalid" BARs.  It wasn't used for
anything except some not very useful sanity checks.  The one exception
was this stuff in ahci-test, which I replaced with an explicit flag
indicating we've activated the AHCI.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2016-10-24 12:06 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-21  1:19 [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 01/11] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 02/11] libqos: Handle PCI IO de-multiplexing in common code David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 03/11] libqos: Move BAR assignment to " David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 04/11] libqos: Better handling of PCI legacy IO David Gibson
2016-10-21 10:05   ` Greg Kurz
2016-10-22  4:50     ` David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 05/11] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 06/11] libqos: Add streaming accessors for PCI MMIO David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 07/11] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
2016-10-21 10:08   ` Greg Kurz
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 08/11] tests: Clean up IO handling in ide-test David Gibson
2016-10-21  8:31   ` Laurent Vivier
2016-10-21  9:05     ` David Gibson
2016-10-21 10:07       ` Laurent Vivier
2016-10-22  4:56         ` David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 09/11] libqos: Add 64-bit PCI IO accessors David Gibson
2016-10-21 10:31   ` Greg Kurz
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 10/11] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
2016-10-21  1:19 ` [Qemu-devel] [PATCHv4 11/11] libqos: Change PCI accessors to take opaque BAR handle David Gibson
2016-10-21 13:23   ` Greg Kurz
2016-10-24  3:19     ` David Gibson
2016-10-24  4:31       ` David Gibson
2016-10-24 10:14         ` Greg Kurz
2016-10-24 10:02       ` Greg Kurz
2016-10-24 12:02         ` David Gibson
2016-10-21  9:38 ` [Qemu-devel] [PATCHv4 00/11] Cleanups to qtest PCI handling Laurent Vivier

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.